• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    GENERATION OF ASSEMBLY INSTRUCTIONS FOR CONSTRUCTION ELEMENT MODELS The present invention relates to a toy building system comprising a set of toy building elements with coupling means for releasably interconnecting the toy building elements; and a data processing system comprising image capture means, processing means and display means, in which the data processing system is adapted to capture an image of a partial toy construction model built from a subset of the toy building elements; processing the captured image to detect at least one position and orientation of the partial toy construction model; identify a user selection indicative of a user selection of at least one of a set of subsequent building elements, with each subsequent building element being connectable to the partial toy building model; in response to the detected position and orientation of the partial model of toy construction, to display in the said display medium a composite image that comprises the captured image that superimposed an image of at least (...).
    公开号:BR112013030077B1
    申请号:R112013030077-9
    申请日:2012-05-22
    公开日:2021-03-23
    发明作者:Siddharth Muthyala；Harm Jan Van Beek；Nicolas David Alain Guyon；Frantz Lasorne；Mikkel Holm Jensen
    申请人:Lego A/S；
    IPC主号:
    专利说明:

    [0001] [001] The present invention relates to the generation of assembly instructions for toy construction models. Background
    [0002] [002] There are several types of modeling concepts for physical construction toy sets. Especially, concepts that use modular or semi-modular concepts are very popular as they provide a challenging and interesting playing experience. Typically, these concepts provide a set of prefabricated building elements that can be interconnected with each other in some predetermined way according to modules of the prefabricated elements. Prefabricated elements can resemble well-known objects adapted for a specific modeling task. Thus, for example, in building a model of the house, the elements may resemble wall bricks, roof tiles, doors and windows. The purpose of selecting the elements in this way is that the work involved in building a model of a house is significantly reduced compared to a situation where all the details of a house must be defined each time a new model is made. However, complete freedom in building a house or other object is exchanged for the simplicity of building the model.
    [0003] [003] For example, toy building sets available under the name LEGO comprise a plurality of different types of interconnectable building elements that have coupling means in the form of corresponding protrusions and cavities. The coupling means are arranged according to regular grid patterns, thus allowing for a wide variety of interconnections between construction elements.
    [0004] [004] Typically, such toy building sets comprise a set of building elements suitable for creating one or more models of building element, for example, an animal, a robot or another creature, a car, an airplane, a spaceship, a building or the like. Typically, a construction set additionally includes printed assembly instructions or assembly instructions that illustrate how to build a certain model from the construction elements of the set.
    [0005] [005] Typically, the assembly instructions included in a toy construction set comprise a sequence of photos that illustrate step by step and in what order to add the building elements to the model. Such assembly instructions have the advantage that they are easy to follow, even by children without much experience with toy building sets and / or reading skills.
    [0006] [006] However, such assembly instructions have the disadvantage that they are expensive and require labor intensive to produce. Typically, the model for which the assembly instructions are to be created is deconstructed into reasonable construction steps and each construction step is subsequently drawn on a CAD system and finally printed.
    [0007] [007] More recently, the assembly instructions were generated in electronic form instead of in printed form. In particular, the instructions for animated assemblies in which the most complicated construction steps are animated. However, it remains a problem to provide toy building sets that inspire children to create their own models and rebuild the model in a different way, thereby increasing the value of playing the toy building set. summary
    [0008] - capturar uma imagem de um modelo parcial de construção de brinquedo construído a partir de um subconjunto dos elementos de construção de brinquedo; - processar uma imagem capturada para detectar pelo menos uma posição e uma orientação do modelo parcial de construção de brinquedo; - identificar uma seleção de usuário indicativa de uma seleção de usuário de pelo menos um dentre um conjunto de elementos de construção subsequentes, sendo que cada elemento de construção subsequente é conectável ao modelo parcial de construção de brinquedo; - em resposta à posição e orientação detectadas do modelo parcial de construção de brinquedo, exibir nos ditos meios de exibição uma imagem composta que compreende a imagem capturada que sobrepôs uma imagem do elemento de construção subsequente selecionado. [008] Revealed in the present document are modalities of a toy construction system that comprises a set of toy construction elements with coupling means to releasably interconnect the toy construction elements. Modalities of the toy building system comprise a data processing system comprising image capture means, processing means and display means, to which the data processing system is adapted - capture an image of a partial toy construction model built from a subset of the toy construction elements; - processing a captured image to detect at least one position and orientation of the partial toy construction model; - identifying a user selection indicative of a user selection of at least one of a set of subsequent building elements, each subsequent building element being connectable to the partial toy building model; - in response to the detected position and orientation of the partial model of toy construction, display in said means of display a composite image comprising the captured image that superimposed an image of the selected subsequent construction element.
    [0009] [009] Therefore, the user is presented with an image of the partial model built up to the present moment and with a choice of selecting one or more subsequent building elements from a set of possible alternative subsequent building elements, that is, the from a set of alternative continuations. Therefore, the subsequent building elements can be alternative, subsequent building elements.
    [0010] [0010] Consequently, the number of possible construction paths and possibly even the number of toy models available to the user is increased, thereby allowing for a variety of different construction experiences.
    [0011] [0011] As the system detects user selection and displays at least the subsequent selected building element superimposed on the captured image of the current partial building model and, in response to the detected position and orientation of the current partial model, the system can provide clear and easy-to-follow guidance to the user as to where and how the subsequent selected building element is to be connected.
    [0012] [0012] In some embodiments, the data processing system may be operable to display on said means of display a composite image comprising the captured image that superimposed an image of at least the subsequent building element selected in a relative position and orientation to the partial model of toy construction which corresponds to said subsequent construction element that is coupled to the partial model of toy construction.
    [0013] [0013] In some modalities, the data processing system may be operable to obtain a digital representation of the partial construction model; and, in response to the digital representation obtained, display the respective images of the set of subsequent building elements, for example, by displaying a composite image of the captured image overlaid with the images of the set of subsequent building elements. Examples of digital representations include an identifier or other suitable data structure that identifies a partial model of toy construction.
    [0014] [0014] The data processing system can be operable to generate a digital representation of an updated partial model of toy construction from the digital representation of the partial model of toy construction and from the identified user selection. Therefore, based on the detected user selections of respective subsequent building elements, the data processing system can determine which construction path the user has followed, thereby identifying which subsequent partial toy construction model the user arrived at when the The user couples the selected subsequent building element to the current partial building model according to the assembly instructions displayed by the data processing system. Consequently, obtaining a digital representation of the partial construction model can comprise determining or generating the digital representation of the partial toy construction model from a digital representation of a previous partial toy construction model and from a previous user selection .
    [0015] [0015] In some embodiments, the data processing system can be adapted to determine the set of subsequent building elements from said digital representation and from a data structure indicative of a plurality of construction step sequences, each sequence resulting in one of a set of alternative construction models that can be built from the toy construction set. In some embodiments, the digital representation can be understood in the data structure. For example, the data structure can be any tree data structure, for example, representing a directed acyclic graph comprising nodes and margins in which each node represents a toy building model and each margin connecting a first and a second node represents a connection of one or more subsequent building elements connectable to the partial toy building model represented by the first node in order to arrive at a toy building element (partial) represented by the second node.
    [0016] [0016] In some embodiments, processing the captured image to detect at least one position and orientation of the partial toy construction model comprises detecting at least one augmented reality (AR) marker and determining the position and orientation of the construction model toy from at least one detected augmented reality marker.
    [0017] [0017] The data processing system can be operable to identify user selection in several different ways. For example, the user can indicate the user selection through appropriate user input, for example, by pointing or clicking with a pointing device such as a computer mouse on the displayed image of a subsequent building element. Alternatively or additionally, in some modalities, processing the captured image comprises identifying at least one added construction element; and the data processing system can be adapted to determine user selection from said identified added construction element. Therefore, the user can simply couple the selected subsequent physical building element to the partial physical building model. When the data processing system detects which subsequent building element has been added to the model, the data processing system can determine that the selected subsequent building element is the one detected. In some embodiments, the data processing system can additionally detect the position and / or orientation of the added construction element in relation to the partial toy construction model.
    [0018] [0018] In some embodiments, identifying at least one added building element comprises detecting an augmented reality marker included in the added building element.
    [0019] [0019] The present invention can be implemented in different ways which include the toy building system described above and then a data processing system, additional product methods and means, each of which yields one or more of the benefits and advantages described in connection with the first mentioned toy construction system and each having one or more preferred modalities corresponding to the preferred modalities in connection with the first mentioned toy construction system and / or disclosed in the dependent claims.
    [0020] [0020] In particular, the features of the method described in this document can be deployed in software and executed in a data processing system or other processing means caused by the execution of executable instructions by computer. Instructions can be means of program code loaded into memory, such as RAM, from a storage medium or from another computer over a computer network. Alternatively, the features described can be deployed by the wired circuitry instead of software or in combination with software.
    [0021] [0021] Consequently, the invention additionally relates to a data processing system adapted to carry out the method described above and thereafter. The invention further relates to a computer program comprising program code means for carrying out all the steps of the method described above and thereafter when said program is executed on a computer. The invention further relates to a computer program product comprising program code means for carrying out the method described above and thereafter when said computer program product is executed on a computer. The program code means may be stored on a computer-readable medium and / or incorporated as a propagated data signal. Brief Description of Drawings
    [0022] [0022] Aspects of the invention will be explained more fully below in connection with a preferred modality and with reference to the drawings, in which:
    [0023] [0023] Figures 1a to d, each, show a toy construction brick of the prior art.
    [0024] [0024] Figure 2 shows an embodiment of a computer system as disclosed in the present document.
    [0025] [0025] Figure 3 shows a flowchart of a modality of a process for displaying construction instruction steps for a toy construction model.
    [0026] [0026] Figure 4 illustrates a modality of data structure to digitally represent a plurality of alternative sequences of construction stages of a construction process to build a toy construction model.
    [0027] [0027] Figure 5 illustrates an example of a display area for a computer system display as revealed in this document.
    [0028] [0028] Figure 6 shows a toy building element in the form of a toy building brick.
    [0029] [0029] Figure 7 illustrates how a marker construction element can be coupled with another toy construction element of a toy construction model.
    [0030] [0030] Figures 8a to g illustrate examples of composite marker construction elements.
    [0031] [0031] Figures 9a to b illustrate an example of a toy construction system that defines a regular grid. Detailed Description
    [0032] [0032] Various aspects and modalities of toy building systems disclosed in the present document will now be described with reference to the toy building elements in the form of bricks. However, the invention can be applied to other forms of construction elements used in toy construction sets.
    [0033] [0033] Figures 1a to d each show a prior art toy construction brick with coupling pins 105 on its top surface and a cavity 102 that extends into the brick from the bottom. Figures 1a to b show a top side of a toy building brick, while Figure 1 b shows the bottom side of the same toy building brick. Figures 1c to d show examples of similar toy building bricks of different sizes. The cavity has a central tube 103, and coupling pins from another brick can be received into the cavity in a frictional engagement as disclosed in US 3 005 282. The building bricks shown in the remaining Figures can have this known type of coupling means in the form of cooperation pins and cavities. However, other types of coupling means can also be used. The coupling pins are arranged in a flat square grid, that is, which defines orthogonal directions along which sequences of coupling pins are arranged. Generally, such an arrangement of coupling elements allows the toy bricks to be interconnected in a discrete number of orientations with respect to each other, in particular, at right angles to each other. It should be noted that other geometric arrangements of coupling elements may result in different orientation restrictions. For example, the coupling elements can be arranged on a regular triangular grid, allowing one building element to be placed on another building element in three different orientations.
    [0034] [0034] In general, the coupling means can include coupling elements that can be grouped into different classes of coupling elements, for example, connectors, receivers and various elements. Connectors are coupling elements that can be received by a receiver from another construction element, thereby providing a connection between the construction elements. For example, a connector can fit between parts of another element, in a hole or the like. Receivers are coupling elements that can receive a connector from another construction element. Miscellaneous elements are parts that can function as much as a receiver or a connector, typically depending on the type of cooperating connection element of the other building element.
    [0035] [0035] The construction elements of the type illustrated in Figure 1 are available under the name LEGO in a wide variety of shapes, sizes and colors. In addition, such construction elements are available with a variety of different coupling elements. It is understood that the building element above serves merely as examples of possible building elements.
    [0036] [0036] Figure 2 shows a schematic view of an example of a computer system. The computer system comprises a properly programmed computer 15 and a display 1. The computer can be a personal computer, a desktop computer, a laptop-type computer, a handheld computer, an electronic gaming device, an entertainment device. handheld or any other properly programmable computer. The display 1 can be integrated into or otherwise operatively coupled to the computer 15 and operable to display, under the control of the computer 15, a video image.
    [0037] [0037] It should be noted that the computer can understand or otherwise be coupled operatively to peripheral devices, such as a keyboard, 3, a mouse 2 or another pointing device, such as a touch-sensitive keyboard, a trackball , an optical pen, a touchscreen or the like.
    [0038] [0038] The computer system is adapted to facilitate the implantation of an augmented reality system as described in this document. For this purpose, the computer 15 comprises or is otherwise operatively coupled to a video camera 5. The video camera 5 is operable to capture video images of the environment in which the video camera is located, for example, of a field of view 7 including a visible surface area 8 of a surface 4, for example, a table, a floor or the like. Therefore, the visible surface area 8 is the projection of the video camera's field of view on the surface 4. The video camera is operable to route the captured video image to a processing unit of the computer 15, for example, by means of of a suitable computer 15 input interface. For example, the video camera can be a webcam connected to the one integrated in the computer 15. In the example in Figure 2, the video camera is positioned on a camera holder 6, such as a tripod, which holds the camera at a predetermined height above the surface 4.
    [0039] [0039] Here, the video camera 5 captures video images of the environment 8 in which the video camera is located and forwards the captured video images to the computer 15. For example, the environment in which the video camera is located it can comprise the toy building model 10. In addition to the toy building model 10, the environment can comprise additional objects such as household objects, toys or the like.
    [0040] [0040] The computer images are then rendered by computer 15 on the display 1. Therefore, a user can move the toy building model 10 around and / or otherwise manipulate the toy building model. in the field of view of the video camera 5 and view live video from the video camera 5 of the toy building model. Alternatively or additionally, the user can change the position and / or orientation of the video camera in order to capture images of a toy construction model (for example, stationary) from different positions. In addition, the computer may be operable to store the captured video images on a storage device such as a computer's hard drive, and / or forward the captured video to another computer, for example, over a computer network. For example, the computer may be operable to upload the captured video images to a website.
    [0041] [0041] Computer 15 is suitably programmed to operate in an augmented reality construction instruction mode in which the computer performs processing on the captured video image in order to detect one or more predetermined augmented reality markers or labels on the video image captured. In response to at least the detected AR marker (s) detected, the computer can be programmed to generate a modified video image, for example, a video image formed as the captured video image that has an image generated by superimposed computer or a video image where at least part of the captured video image is replaced with a computer generated image. Computer 15 is operable to display the modified video image on display 1. For the purpose of the present description, a computer operable to deploy operatively connected AR functionality to a video camera and a display will also be referred to as a system of AR.
    [0042] [0042] Image processing methods for detecting AR markers and for generating modified video images in response to detected AR markers are shown both in the technique (see, for example, Daniel Wagner and Dieter Schmalstieg, "ARToolKitPlus for Pose Tracking on Mobile Devices ", Computer Vision Winter Workshop 2007, Michael Grabner, Helmut Grabner (eds.), St. Lambrecht, Austria, February 6-8, University of Graz Technical)).
    [0043] [0043] In the example of Figure 2, the physical toy construction model 10 comprises a toy construction element 9 which has connected to it a marker construction element 11. The marker construction element 11 has an insignia in the form of a two-dimensional machine-readable code on its top surface.
    [0044] [0044] Computer 15 is operable to detect the presence of two-dimensional machine-readable code in the captured image. In addition, computer 15 can determine a position and relative orientation of the two-dimensional machine-readable code relative to the position of the video camera 5.
    [0045] [0045] Consequently, computer 15 can modify the captured image of the toy building model which results in a modified video image 12 displayed on the display 1. In this example, the modified image shows captured image 13 of the toy building model. and image generated by computer elements 14a and 14b overlaid on the captured image. The computer generates the image generated by computer elements in a position and orientation in the video image 12 as determined from the detected position and orientation of the AR marker of the marker building element 11.
    [0046] [0046] As the user manipulates the physical toy construction model 10 in the projection area 8 of the video camera, for example, when moving and / or rotating the physical model, computer 5 tracks the position and orientation of the marker construction element 11 of the physical toy. The computer 15 displays the live video transmission to the video camera (mirrored mode) on the display 1 and adds, in response to the insignia's detected position and orientation, special augmented reality effects to the live video transmission.
    [0047] [0047] As will be described in more detail below, computer 15 can be adapted to show images of alternative toy building elements 14a and 14b, respectively, which can be added to toy building model 10. User can select, then one of the alternative building elements and modifying the physical toy building model 10 by adding the selected building element.
    [0048] [0048] Figure 3 shows a flowchart of a modality of a process for displaying construction instruction steps for a toy construction model. The process can be carried out by a properly programmed AR system, for example, by the computer 15 of the system shown in Figure 2. In the initial step S1, the process receives an input indicative of the starting point of the construction process. For example, the starting point can be a partial model previously built, a base plate on which the model is to be built or even an empty building environment where no component of the construction set has been laid out yet. For example, the process may receive such input in the form of a suitable user input, for example, a user selection from several possible starting points. Alternatively or additionally, the process can receive input in the form of an image captured by a video camera from the AR system. In such an embodiment, the process can process the captured image in order to determine the starting point. For example, the user can position a base plate or other one or more toy construction elements in the camera's field of view 5. When the base plate or other toy construction element includes an AR marker (or other feature visible detectable), the AR system can detect the AR marker and extract information about the type of base plate or toy construction element from the information embedded in the AR marker. The input received can also comprise information about the model or models of toy construction to be built. For example, the information may include an identifier that identifies the toy building set from which the toy building model is to be built, or identifying one or more toy building models to be built.
    [0049] [0049] The process then continues in step S2 with the starting point detected as a model of current partial toy construction. It should be noted from the discussion above that the current toy building partial model may include none, one or more toy building elements from the toy building set.
    [0050] [0050] In step S2, the process determines, based on the current state of the construction process, that is, the current partial model of toy construction, several possible continuations of the construction process. In particular, the process determines a set of alternative toy building elements from the set of toy building elements that can be added to the current toy building partial model. For example, this determination can be based on a suitable data structure indicative, for a given partial toy construction model, of the set of possible subsequent toy construction elements that can be added to said partial toy construction model. A modality of such a data structure will be described below with reference to Figure 4. If the number of options determined is zero, that is, there is no further possible continuation, the process ends; otherwise, the process proceeds at step S3.
    [0051] [0051] In step S3, the process presents the options determined to the user, for example, by displaying images of alternative toy construction elements that can be added to the model in the next step, for example, as shown in Figure 5. In some modalities, the system additionally indicates to the user where the respective alternative toy construction elements should be added to the current toy construction partial model. For example, the AR system may display a combined image that shows the captured image of the current toy building partial model with images of the respective alternative toy building elements overlapping in respective positions and orientations in relation to the toy building partial model. toy indicating the respective positions where the alternative toy building elements should be coupled to the current toy building partial model. If two or more of the alternative toy building elements are added in the same position (or overlapping positions), the AR system can, for example, display a placeholder at the relevant position and images of the alternative toy building elements in proximity to the placeholder. Alternatively, the AR system can display the alternative toy building elements in the relevant position one at a time, for example, by circulating the alternative toy building elements and displaying each one for a predetermined period of time.
    [0052] [0052] In the subsequent step S4, the process determines a user selection among the presented options. For example, the AR system may receive a user input indicative of a selection from one of the options presented. For example, each of the displayed images of alternative building elements can correspond to an active element of a graphical user interface displayed by the AR system, so that the active element can be activated, for example, by clicking on it with a computer mouse, to select that building element. In one embodiment, a selected building element changes its appearance. For example, the selected building element can change color, texture, etc .; can be highlighted by showing a wraparound box around the selected building element or the like.
    [0053] [0053] Alternatively, the AR system can be operable to identify physical toy building elements added to the current toy building partial model by processing images of the toy building model captured by the AR system. If, in response to the display of alternative toy building elements in step S3, the AR system identifies an added toy building element that has been added to the current toy building partial model, and if the toy building model added identified corresponds to one of the alternatives presented in step S3, the AR system determines that the user selected the said alternative. If the construction element added by the toy does not correspond to any of the alternatives presented, the process may, for example, proceed with an appropriate error message. In some embodiments, the process may additionally determine a position and optionally an orientation of the added toy building element indicative of where the additional toy building element has been added to the model.
    [0054] [0054] In the subsequent step S5, the process may, in response to the detected selection, update a representation of the current toy building model to a subsequent toy building model. For example, the AR system can update a pointer or indicator that indicates the current toy building model in a toy building model identifier data structure to point to a subsequent toy building model. Optionally, the AR system can display image resources, for example, superimposed the video image of the captured image of the toy building model, in response to the detected user selection. For example, in a mode where user selection is detected by a user input before the user adds the selected toy building element to the current toy building partial model, the AR system can display, for example , in the form of an animation, where and how the selected building element should be connected to the current partial building model. Alternatively or additionally, the AR system can display an animated plot in which the continuation of the story depends on the selection of the user.
    [0055] [0055] Subsequently, the process returns to step S2 with an updated partial model of toy construction as the new current partial model of toy construction, in which the updated partial model of toy construction is derived from the current partial model building blocks from the previous iteration by adding the selected toy building element.
    [0056] [0056] Figure 4 illustrates a modality of data structure to digitally represent a plurality of alternative sequences of construction stages of a construction process to build a toy construction model.
    [0057] [0057] In one embodiment, data structure 401 is in the form of a tree structure illustrated in Figure 4a. The tree structure has a plurality of nodes and margins, where each node represents a model of toy construction and where each margin connects two nodes and represents a construction step in order to transform the construction model represented by one of the nodes (the predecessor node) in the building model represented by the second node (the successor node). The tree structure comprises at least one root node 411 which has no predecessor nodes and thus represents the starting point of a construction sequence. Similarly, the tree structure comprises one or more leaf nodes 412 without any successor nodes, each representing a respective toy construction model that can be built from the toy construction set. It should be noted that the tree structure may have one or more root nodes and one or more leaf nodes. Each non-leaf node thus represents a partial toy construction model that can be transformed into one or more subsequent (partial) alternative toy construction models as represented by the successor nodes of said non-leaf node. Each non-leaf node can therefore have one or more successor nodes. The tree structure can thus represent a directed acyclic graph. The tree structure represents a plurality of construction paths so that each construction path connects a root node with a leaf node. It should be noted that the construction paths can have different lengths.
    [0058] [0058] The data structure may comprise one or more data records that include global model parameters related to the entire set of construction models that can be built from construction sequences. Examples of such model parameters include a model name, a model creator name, a modeling application program version number, a creation date, and so on.
    [0059] The model 401 data structure additionally comprises a plurality of margin data structures, each of which is associated with one of the tree structure margins. For simplicity of illustration, only one of such 403 margin data structures is explicitly shown, associated with margin 413. It should be noted, however, that the other margins have corresponding data structures associated with them. Each margin data record represents one or more (e.g., a list of) toy building elements, each represented by a toy building element data record. Each toy building element data record may have the structure illustrated by data record 404 for "Building Element B" shown in Figure 4b.
    [0060] [0060] In particular, each building element data record 404 may comprise a building element ID 405 which indicates an identifier corresponding to the type of building element. Preferably, the building element ID uniquely identifies the properties of the building element or type of building element.
    [0061] [0061] The building element data record may additionally comprise several building element attributes 406 that indicate one or more attributes of the building element, such as color, texture, decorations, etc.
    [0062] [0062] Additionally, the building element data record 404 may comprise data items 407 and 408 that represent the position and orientation of an internal construction element coordinate system, respectively. The position and orientation of the building element can be defined by the coordinates of an origin of the building element's internal coordinate system in relation to a global "world" coordinate system, and by orienting the internal coordinate system in relation to to the global coordinate system.
    [0063] [0063] Additionally, the building element data record 404 may comprise data item 409 representing one or more enclosures of the building element.
    [0064] [0064] It is understood that the digital representation can be encoded in any suitable file or data format, for example, as a binary file, as a text file according to predetermined modeling description language or similar. An example of a data format for storing virtual building models that includes a coordinate system hierarchy is disclosed in US Patent 6,389,375.
    [0065] [0065] Based on such a data structure, a computerized process for presenting assembly instructions can, based on information about a given starting point, determine a plurality of subsequent alternative construction steps. In particular, the process can determine the node in the tree structure that corresponds to the starting point (for example, the root node 41) and the margins that connect the starting point with its successors. Each margin represents a subsequent alternative step in the construction process, for example, an alternative construction element (or set of construction elements). When the process receives information about a user selection from one of the alternatives, the process can determine the corresponding successor node in the tree structure. The process can proceed until the process reaches a leaf node. In the example in Figure 4, an example of a resulting construction path is illustrated by a bold line 414. In this example, the starting point is the root node 411 and the construction path ends at the leaf node 412. In the first iteration, the The process identifies the margins 415a, 415b, 415c that connect the root node 411 with its three successor nodes 416a, 416b, 416c, respectively. If the user selects margin 415a, the process proceeds to node 416a. It should be noted that some nodes may have only a single successor, that is, during some stages of the construction process there may be only one route ahead with no additional alternatives to choose from. Similarly, some nodes may have more than one predecessor, reflecting the fact that multiple construction paths can result in the same model of toy construction (partial).
    [0066] [0066] Figure 5 illustrates an example of a display area, generally referred to as 501, of a computer system display as revealed in this document, for example, the system in Figure 2. The computer system (not is shown explicitly in Figure 5) displays a captured image taken with a video camera (not explicitly shown in Figure 5) from the computer system. In this example, the image comprises an image of a physical toy construction model 510 located in the camera's field of view. In this example, the toy building model 510 is built on top of a base plate 521, and a part of a house built from one or more toy building elements, for example, elements similar to those shown in Figure 1 which has 525 coupling pins on its top surface, allowing additional construction elements to be coupled. It should be noted, however, that the toy construction model can be any other model structure with or without a base plate.
    [0067] [0067] In this example, the base plate comprises several AR 524 markers distributed around the margin of the base plate. Each AR marker comprises a unique insignia that allows an AR system to identify the position and orientation of the toy building model regardless of its position or relative orientation in relation to the camera. In this example, the AR 524 markers are in the form of a 2D barcode or similar matrix code and they include a wraparound box that provides a set of intersecting lines, allowing the AR system to detect the insignia and your position and orientation. However, as will be described below, different types of RA markers can be used. In some embodiments, one or more of the toy building elements from which the toy building model 510 is constructed may comprise AR markers in addition to or instead of AR markers on a base plate.
    [0068] [0068] In the example in Figure 5, the computer system determined three alternative sequences according to which the user can proceed by building the toy construction model 510. Consequently, the computer system displays images, for example, image generated by computers, of the corresponding alternative building elements 514a, 514b and 514c that overlaid the capture video image of building model 510. As alternative building elements would be added in the same position as the model, the computer system additionally displays a generated image by computer of a placeholder 515, for example, in the form of an enclosure, which indicates where in relation to the structure of the model 510 each of the alternative construction elements would be added. For example, the computer system can determine the size of the placeholder from the corresponding information included in a data structure representing the model, for example, the tree data structure in Figure 4. Similarly, the system computer can determine the correct position of the placeholder and / or alternative building elements in relation to the model image from the information included in a data structure representing the model and from a position and orientation of the structure of the model 510 detected by the computer system. For example, the computer system can process the captured image to detect the AR markers. From the detected AR markers, the computer system can determine the position of an appropriate model coordinate system. An example of such a coordinate system is illustrated as coordinate system 526 in Figure 5, even though the coordinate system determined as such may not necessarily need to be displayed by the system. The computer system can then determine the position and orientation of the placeholder and / or the position and orientation information of alternative building elements in the data structure model in relation to the model coordinate system.
    [0069] [0069] Figures 6 and 7 schematically illustrate examples of toy building elements that comprise AR markers that facilitate the detection of the toy building element by an AR system. For the purpose of the present description, such building elements that comprise an AR marker are also referred to as a marker building element
    [0070] [0070] Figure 6 shows a toy building element in the form of a toy building brick 611 similar to the building brick of Figure 1c, that is, a building brick comprising coupling pins 605 arranged in a square grid. flat on the top surface of the toy construction brick. The toy building brick also comprises one or more cavities in its bottom surface (not shown in Figure 6) for receiving and frictionally engaging the coupling pins of another similar toy building brick. The toy building brick comprises, on one of its side faces, an augmented reality marker 621 in the form of a 2D barcode.
    [0071] [0071] It should be noted that other forms of AR markers can be used instead of a 2D barcode. In general, the AR marker can be any object that is relatively easy to distinguish automatically in video images captured using known image recognition methods. When the AR marker is detected using image recognition methods, the AR marker can be, for example, a three-dimensional object such as a cube or cylinder or it can be a two-dimensional marker such as a square or circle. Typically, the AR marker comprises an easily distinguishable pattern such as a black and white square although other methods can be used for marker recognition such as the use of a particular color or pattern of colors and the like.
    [0072] [0072] The AR marker can include one or more elements that allow the computer to detect an AR marker position and / or orientation. For example, the AR marker can comprise two or more intersecting lines. Additionally or alternatively, the AR marker can comprise visual elements for encoding information, thus allowing the computer to identify and distinguish different types of markers and to generate computer-generated images selectively in response to specific types of markers.
    [0073] [0073] The AR marker can be arranged uniformly in relation to the coupling means, that is, the coupling pins on the top surface and / or the coupling cavity on the bottom. For example, the AR marker can define a parallel or normal direction for the flat grid defined by the coupling means. This makes marker bricks interchangeable and, in a toy structure built from bricks as in Figures 1 and 6, several marker bricks can be used interchangeably and a particular marker brick can be used in various constructions. A toy building system can comprise several of such marker bricks that have different insignia applied to them and causing the computer system to generate different computer-generated images. However, if all marker bricks include the badge in uniform positions, such marker bricks can easily be interchanged in a toy construction constructed from the building bricks described in this document. In addition, the AR system can use such uniformly positioned AR markers to accurately determine the position and orientation of construction elements in relation to a coordinate system defined by the coupling means. Therefore, the AR system can detect, based on a detected AR marker from a building element, that a building element has been added to an existing (partial) model, whose type of building element has been added and where the building element has been added. of construction was added in relation to the existing model.
    [0074] [0074] When the badge is located on a part of the surface of the marker building element so that the surface part comprising the badge has no coupling means, the badge is less likely to be obstructed by other building elements attached to the marker construction element. A marker construction element may comprise a plurality of insignia, for example, on different faces (or, otherwise, on different portions of the surface) of the toy construction element.
    [0075] [0075] Figure 7 illustrates how a marker construction element can be coupled with another toy construction element of a toy construction model. In this example, the toy construction model comprises toy construction elements 710 and 712 and a marker construction element 711. Figure 7a shows the individual construction elements, while Figure 7b shows the marker construction element 711 coupled. releasably to the construction element 710. For that purpose, the toy construction element 710 comprises coupling pins 705 on its top surface as described in connection with Figure 1. The marker construction element 704 comprises cavities in its bottom surface (not shown in Figure 7), allowing it to frictionally engage coupling pins 704.
    [0076] [0076] Therefore, the user can connect a marker construction element that comprises an AR marker to a toy model built in order to facilitate the detection and identification of the added marker construction element. Based on detection, the AR system can determine possible subsequent building elements that can be added to the model and display images of the determined subsequent elements superimposed on the captured image of the model and in appropriate positions in relation to the captured image of the model. As in the example of Figure 6, the insignia of the marker building element 711 is located on a surface of the marker building element that does not comprise coupling elements.
    [0077] [0077] Figures 8a ag illustrate examples of composite marker construction elements, that is, a plurality of marker construction elements directly or indirectly connected to each other through the coupling means of the toy construction system, for example, in a predetermined spatial relationship between them. Generally, composite marker construction elements allow the user to create a large set of distinct objects, each of which is identifiable by one of a large number of unique AR markers. In particular, the large number of unique AR markers can be created from a relatively limited set of individual marker construction elements, due to the fact that combining a smaller set of markers through a construction system dramatically increases the number total combinatorial possibilities. For example, the RA system can identify one (or a subset) of a large number of partial construction models from a composite marker construction element included in the partial construction model.
    [0078] [0078] Figure 8a shows an example of a human-like figure constructed from three marker construction elements, namely, an 811 element that resembles a figure head, an 812 element that resembles the torso of the figure. figure and an element 813 that resembles the legs of the figure. The construction elements are provided with coupling elements that allow a head and legs to be releasably coupled to the torso. The head, torso and legs may include insignia, for example, in the form of facial features of the head and clothing features of the torso and / or legs. When presented to the video camera of an AR system, the AR system can thus detect the characteristics of the head, torso and legs, for example, as illustrated schematically in Figure 8b by the detected characteristics 821, 822, 823 of the head, torso and legs, respectively. Therefore, by combining different heads, torsos and legs, a wide variety of combinations of RA markers can be created. Depending on which figure is detected by the AR system, the AR system can respond with different computer-created images, thus allowing for a wide variety of interactions.
    [0079] [0079] Figures 8c and d illustrate a composite element of similar marker construction, constructed from toy building bricks as shown in Figure 1c. The toy building bricks 831, 832 and 833 have different colors, for example, red, white and blue, respectively. When stacked on top of each other and interconnected by their respective coupling means, the building bricks define a sequence of colors that can then be detected by an AR system. By changing the order of colors, as shown in Figure 8d, different unique composite AR markers can be constructed from just three marker construction elements.
    [0080] [0080] Figure 8e shows a similar example of a composite marker element, including toy building bricks 841, 842, 843 similar to that shown in Figure 6 that has insignia-shaped AR markers on its side faces.
    [0081] [0081] Figures 8f to g show another example of a composite marker element. In this example, the composite marker element comprises a base building brick 850, for example, a brick similar to that of 1c, but with a greater number of coupling elements on its top surface. The composite marker element further comprises marker building elements 851, 852, 853 connected to the base building brick 850. Therefore, in this example, marker building elements 851, 852, and 853 are indirectly coupled together in a configuration rigid to form a composite marker construction element. To that end, the marker building elements have coupling elements, for example, cavities, on their bottom surface that can engage corresponding coupling elements, for example, pins, on the top surface of the base building brick 850. Each marker construction element has an insignia on its top surface, in this example, letters of the Roman alphabet. As the marker construction elements are arranged side by side, they define an insignia sequence detectable by the AR system.
    [0082] [0082] As shown in Figure 8g, the insignia of the marker construction elements can define an orientation in a plane, and individual marker elements can be rotated in relation to each other, thereby further increasing the degrees of freedom of defining multiple codes, as the AR system can detect the orientation of the respective AR markers in relation to each other and / or in relation to another feature of the toy building model.
    [0083] [0083] Therefore, in the examples in Figure 8, the individual marker building elements, each comprising a visually detectable feature distinct from the other marker building elements, are interconnected by means of their coupling elements, in order to define a sequence of visually detectable resources. Each distinct sequence of visually detectable resources can thus define a composite RA marker. For example, individual visually detectable resources can be detected by the AR system as symbols of a code and a detected symbol sequence can be decoded by the AR system in order to obtain information encoded by the symbol sequence. In some embodiments, the system may use known error detection and / or error correction techniques to make the code more robust against detection errors.
    [0084] [0084] Figures 9a to b illustrate an example of a toy construction system that defines a regular grid.
    [0085] [0085] Figure 9a shows a perspective view of a toy construction element and its corresponding connectivity grids. Toy building element 901 has a top surface 902 with eight buttons 903a ah, a bottom surface with matching holes (not shown) and side faces 904. In Figure 9a, connectivity grids 905 and 906 of the top surface and the bottom surface, respectively, are shown. The grid points are illustrated by circles as exemplified by circles 907a to k. Therefore, grid points 907a to h correspond to buttons 903a to h, respectively. As the side surfaces 904 have no connection elements, no connection grid needs to be defined by them.
    [0086] [0086] As can be seen from Figure 9a, a placement of the connection elements of the toy construction element defined by the grid points placed on a regular grid imposes certain restrictions on the physical placement of the connection elements. Grid 905 is located on the plane of the top surface of the toy building element from which buttons 903 extend. In the example in Figure 9a, the grid points are placed in a square grid where each square has a dimension of 5x5 units of an arbitrary extension unit (LU). Therefore, in this geometry, the connecting elements are also placed in a corresponding square grid and the distance between connecting elements in a plane of the building element is a multiple of 10 LU. In the example in Figure 9a, the top and bottom surfaces of the building element are rectangular and have a dimension of 20 LU x 40 LU and adjacent connecting elements are separated by 10 LU. In the vertical direction, on the other hand, the connection elements are separated by 12 LU. Therefore, the grid dimensions in different dimensions may vary. The location of the connection points is defined in relation to an internal coordinate system 908 of the toy building element. By defining a position and orientation of the building element (and thus its internal coordinate system) relevant to a global model coordinate system, the positions of the connecting elements relative to the model coordinate system can be defined.
    [0087] [0087] Figure 9b illustrates a regular 3D grid of a model coordinate system in which each grid point of the grid defines a valid position of a connecting element in a toy construction model. It should generally be noted that not all grid points on the grid really need to be filled with coupling elements; however, in some embodiments, all coupling elements are positioned at grid points on the regular grid. In particular, in the example of Figure 9b, the grid points are arranged on parallel planes 911a to d where the grid points on a plane define positions of connecting elements on a surface of one or more toys.
    [0088] [0088] The distance between the parallel planes defines surfaces of construction elements that have coupling elements. For example, in some embodiments, the building elements have a height that is equal to (or an integer multiple of) distance between two adjacent planes of the grid.
    [0089] [0089] When the toy building system defines a regular grid, the position and / or the orientation of the AR markers can be accurately determined by the AR system in relation to the grid. In addition, the tracking markers that are attached to the building elements in the building system grid provide more information about how the labeled elements are used in the model. In-app events can be selectively triggered if the labels are constructed correctly, that is, they fit into the grid.
    [0090] [0090] An AR marker can thus cause the AR system to initiate the creation of computer generated image elements from subsequent alternative building elements that can be added to the model in response to a predetermined position and / or orientation (and / or other positional restriction) of the AR marker (s) in relation to the grid. The application of AR can thus react to how the physical elements are used in the construction system.
    权利要求:
    Claims (12)
    [0001]
    Toy building system comprising a set of toy building elements with coupling means for releasably interconnecting the toy building elements; and a data processing system comprising processing means (15) and display means (1), characterized by the fact that the data processing system comprises image capture means (5) and the data processing system is adapted to - capturing an image (13) of a partial toy construction model (10, 510) constructed from a subset of the toy construction elements; - processing the captured image to detect at least one position and orientation of the partial toy construction model (10, 510); - identifying a user selection indicative of a user selection of at least one of a set of subsequent building elements, each subsequent building element being connectable to the partial toy building model (10, 510); - in response to the detected position and orientation of the partial model of toy construction, display on said display medium (1) a composite image comprising the captured image that superimposed an image (14a, 14b, 514a, 514b, 514c) at least of the selected subsequent building element.
    [0002]
    Toy building system, according to claim 1, characterized by the fact that the data processing system is additionally adapted to obtain a digital representation (411) of the partial toy building model (10, 510); and, in response to the digital representation obtained, display the respective images of the subsequent set of building elements.
    [0003]
    Toy construction system, according to claim 1 or 2, characterized by the fact that the data processing system is additionally adapted to generate a digital representation of an updated partial model of toy construction from a digital representation of the partial model of toy construction (10, 510) and the selection of identified user.
    [0004]
    Toy building system according to any one of the preceding claims, characterized in that the data processing system is additionally adapted to determine the subsequent set of building elements from a digital representation (411) of the partial model of toy construction (10, 510) and from a data structure indicative of a plurality of sequences of construction steps (413, 414, 415a, 415b, 415c), each sequence resulting in one of a set of alternative construction models (412) that can be built from the set of toy construction elements.
    [0005]
    Toy building system according to any one of the preceding claims, characterized in that the processing of the captured image to detect at least one position and orientation of the partial toy building model (10, 510) comprises detecting at least an augmented reality marker (11, 524) and determine the position and orientation of the toy construction model from at least one detected augmented reality marker (11, 524).
    [0006]
    Toy construction system according to any one of the preceding claims, characterized by the fact that the processing of the captured image comprises identifying at least one added construction element; and determining user selection from said identified added building element.
    [0007]
    Toy construction system, according to any of the preceding claims, characterized by the fact that the data processing system is additionally adapted to, in response to the selection of identified user and the detected position and orientation of the partial construction model of toy (10, 510), display on said display medium (1) a composite image comprising the captured image that superimposed at least one additional computer-generated image.
    [0008]
    Toy building system according to any one of the preceding claims, characterized in that the set of toy building elements comprises one or more marker building elements (611, 711) comprising such a coupling means and each one has a visual appearance recognizable by means of image processing.
    [0009]
    Toy construction system according to any one of the preceding claims, characterized by the fact that the coupling means defines a regular grid (905) that restricts a position and / or orientation of the toy construction elements in a construction model toy to different positions and / or different orientations related to said regular grid (905).
    [0010]
    Toy construction system according to any one of the preceding claims, characterized in that the coupling means comprise one or more protrusions and one or more cavities, each cavity being adapted to receive at least one of the protuberances in a coupling by friction.
    [0011]
    Computer-implemented method to generate assembly instructions for building a toy building model from the toy building elements of a toy building system, with the toy building elements comprising coupling means for releasably interconnecting the toy building elements; characterized by the fact that the method comprises: - capturing (S1) an image (13) of a partial toy construction model (10, 510) constructed from a subset of the toy construction elements; - processing (S1) the captured image to detect at least one position and orientation of the partial model of toy construction (10, 510); - identify (S4) a user selection indicative of a user selection of at least one among a set of subsequent building elements, each subsequent building element being connectable to the partial toy building model (10, 510); - in response to the detected position and orientation of the partial toy construction model (10, 510), display (S5) on said display medium (1) a composite image comprising the captured image that superimposed an image (14a, 14b, 514a, 514b, 514c) at least of the subsequent building element selected.
    [0012]
    Toy construction set characterized by the fact that it comprises toy construction elements, the construction elements comprising coupling means to releasably interconnect the toy construction elements; and user instructions for installing, in a data processing system comprising image capture means (5), processing means (15) and display means (1).
    类似技术:
    公开号 | 公开日 | 专利标题
    BR112013030077B1|2021-03-23|TOY CONSTRUCTION SYSTEM, METHOD IMPLEMENTED BY COMPUTER AND TOY CONSTRUCTION SET
    US11014014B2|2021-05-25|Toy construction system for augmented reality
    JP5977231B2|2016-08-24|Information processing system, information processing apparatus, and information processing method
    ES2351272T3|2011-02-02|AUTOMATIC GENERATION OF CONSTRUCTION INSTRUCTIONS FOR MODELS OF CONSTRUCTION ELEMENTS.
    BR112014000064B1|2020-09-15|METHOD IMPLEMENTED BY COMPUTER TO GENERATE A DIGITAL REPRESENTATION OF A CONSTRUCTION ELEMENT DEFINED BY THE USER, METHOD TO PRODUCE A CONSTRUCTION ELEMENT DEFINED BY THE USER AND DATA PROCESSING SYSTEM
    AU2019204462A1|2019-07-11|Mobile agents for manipulating, moving, and/or reorienting components
    JP2008502960A|2008-01-31|Automatic creation of assembly instructions for assembly block models
    BR122020022596B1|2021-11-03|TOY CONSTRUCTION SYSTEM, COMPUTER IMPLEMENTED METHOD, DATA PROCESSING SYSTEM, COMPUTER PROGRAM AND TOY CONSTRUCTIVE SET
    US20210375025A1|2021-12-02|Systems and methods performing object occlusion in augmented reality-based assembly instructions
    Bala2012|tCAD: a 3D modeling application on a depth enhanced tabletop computer
    同族专利:
    公开号 | 公开日
    US20140378022A1|2014-12-25|
    MX2013013544A|2014-05-27|
    CN103702726A|2014-04-02|
    DK2714223T3|2015-09-14|
    WO2012160057A1|2012-11-29|
    KR20140043903A|2014-04-11|
    CN103702726B|2016-01-13|
    BR112013030077A2|2017-06-20|
    CA2836505A1|2012-11-29|
    EP2714223B1|2015-07-01|
    JP6113718B2|2017-04-12|
    ES2544303T3|2015-08-28|
    KR101897311B1|2018-10-24|
    EP2714223A1|2014-04-09|
    HK1192187A1|2014-08-15|
    JP2014515962A|2014-07-07|
    PL2714223T3|2015-10-30|
    US9821242B2|2017-11-21|
    CA2836505C|2018-10-30|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US3005282A|1958-01-28|1961-10-24|Interlego Ag|Toy building brick|
    US3670941A|1970-03-02|1972-06-20|Fred I Grinnell|Retractable guard and guide for nailer|
    US3670942A|1970-10-15|1972-06-20|Omark Industries Inc|Automatic feeding of fasteners|
    DK154964C|1986-01-22|1989-05-29|Lego As|TOYS BUILDING ELEMENT WITH ELEMENTS FOR PROVIDING POSITION INFORMATION|
    US5724074A|1995-02-06|1998-03-03|Microsoft Corporation|Method and system for graphically programming mobile toys|
    US5867166A|1995-08-04|1999-02-02|Microsoft Corporation|Method and system for generating images using Gsprites|
    US6389375B1|1999-01-22|2002-05-14|Interlego Ag|Virtual reality modelling|
    US6741245B1|1999-04-23|2004-05-25|Mitsubishi Electric Research Laboratories, Inc.|Method for decorating a virtual model|
    US7502759B2|1999-08-30|2009-03-10|Digimarc Corporation|Digital watermarking methods and related toy and game applications|
    DE10006834A1|2000-02-16|2001-08-23|Joachim Schwarz|Construction kit has electronic data medium containing data for representation of arrangements of components of kit on screen|
    US20020127944A1|2000-07-20|2002-09-12|Donald Spector|Construction kit for custom toys or other personalized products|
    JP2002159019A|2000-11-16|2002-05-31|Canon Inc|Display control device, imaging position estimating device, display system, image pickup system, image positioning method, imaging position estimating method, and recording medium recorded with process program|
    US20020196250A1|2001-06-20|2002-12-26|Gateway, Inc.|Parts assembly for virtual representation and content creation|
    WO2004066200A2|2003-01-17|2004-08-05|Yeda Research And Development Co. Ltd.|Reactive animation|
    US7755620B2|2003-05-20|2010-07-13|Interlego Ag|Method and system for manipulating a digital representation of a three-dimensional object|
    US7596473B2|2003-05-20|2009-09-29|Interlego Ag|Method of constructing a virtual construction model|
    JP4085918B2|2003-07-18|2008-05-14|ソニー株式会社|3D model processing apparatus, 3D model processing method, and computer program|
    CN1993712A|2004-06-17|2007-07-04|英特莱格公司|Automatic generation of building instructions for building block models|
    JP2006040045A|2004-07-28|2006-02-09|Sony Corp|Information processor, cubic object, information processing system and information processing method|
    JP2007034981A|2005-07-29|2007-02-08|Canon Inc|Image processing system, and image processing device|
    WO2008107021A1|2007-03-08|2008-09-12|Siemens Aktiengesellschaft|Method and device for generating tracking configurations for augmented reality applications|
    US7979251B2|2007-03-16|2011-07-12|Lego A/S|Automatic generation of building instructions for building element models|
    US8374829B2|2007-03-16|2013-02-12|Lego A/S|Automatic generation of building instructions for building element models|
    WO2008146860A1|2007-05-28|2008-12-04|Stellararts Corporation|Assembled block and display system|
    US20090029771A1|2007-07-25|2009-01-29|Mega Brands International, S.A.R.L.|Interactive story builder|
    ES2531257T3|2007-10-11|2015-03-12|Lego As|Toy building system|
    US20090197678A1|2008-02-04|2009-08-06|Chung-Jen Huang|Pretend play toy with reality and virtual interaction|
    EP2157545A1|2008-08-19|2010-02-24|Sony Computer Entertainment Europe Limited|Entertainment device, system and method|
    US9824495B2|2008-09-11|2017-11-21|Apple Inc.|Method and system for compositing an augmented reality scene|
    US8894461B2|2008-10-20|2014-11-25|Eyecue Vision Technologies Ltd.|System and method for interactive toys based on recognition and tracking of pre-programmed accessories|
    US8754886B2|2008-12-29|2014-06-17|Intel Corporation|Systems and methods for transporting physical objects from real physical life into virtual worlds|
    GB2470073B|2009-05-08|2011-08-24|Sony Comp Entertainment Europe|Entertainment device, system and method|
    GB2470072B|2009-05-08|2014-01-01|Sony Comp Entertainment Europe|Entertainment device,system and method|
    US8418085B2|2009-05-29|2013-04-09|Microsoft Corporation|Gesture coach|
    WO2010150232A1|2009-06-25|2010-12-29|Zyx Play Aps|A game system comprising a number of building elements|
    CN102713931A|2009-08-04|2012-10-03|艾修视力科技有限公司|System and method for object extraction|
    US20110065496A1|2009-09-11|2011-03-17|Wms Gaming, Inc.|Augmented reality mechanism for wagering game systems|US8754886B2|2008-12-29|2014-06-17|Intel Corporation|Systems and methods for transporting physical objects from real physical life into virtual worlds|
    US20120304059A1|2011-05-24|2012-11-29|Microsoft Corporation|Interactive Build Instructions|
    KR101391582B1|2013-06-05|2014-05-07|캡보이트레이딩|Block and toy decoration cap|
    JP6193135B2|2014-01-21|2017-09-06|株式会社ソニー・インタラクティブエンタテインメント|Information processing apparatus, information processing system, and information processing method|
    JP6027554B2|2014-01-21|2016-11-16|株式会社ソニー・インタラクティブエンタテインメント|Information processing apparatus, information processing system, block system, and information processing method|
    WO2015111261A1|2014-01-21|2015-07-30|株式会社ソニー・コンピュータエンタテインメント|Information processing device and information processing method|
    EP3286718A4|2015-04-23|2018-12-05|Hasbro, Inc.|Context-aware digital play|
    US9947140B2|2015-09-15|2018-04-17|Sartorius Stedim Biotech Gmbh|Connection method, visualization system and computer program product|
    US10552550B2|2015-09-26|2020-02-04|Intel Corporation|Technologies for physical programming|
    CN105413198B|2015-11-06|2017-08-25|大连新锐天地传媒有限公司|AR children's early education modular systems|
    US10275222B2|2016-03-15|2019-04-30|Intel Corporation|Technologies for physical programming|
    US10353916B2|2016-03-25|2019-07-16|Bentley Systems, Incorporated|Techniques for conversion of CAD descriptions|
    DK3454956T3|2016-05-09|2021-10-25|Lego As|System and method of toy recognition|
    CN105915635B|2016-06-03|2019-01-08|杭州小丸子电子商务有限公司|Magnetic plate toy assembly builds skill developing and training system and method|
    US10074205B2|2016-08-30|2018-09-11|Intel Corporation|Machine creation of program with frame analysis method and apparatus|
    JP6980990B2|2016-08-31|2021-12-15|ソニーグループ株式会社|Information processing systems, information processing methods, and programs|
    US10890897B2|2017-09-26|2021-01-12|International Business Machines Corporation|Assembly of a modular structure|
    TWI639458B|2018-01-25|2018-11-01|智高實業股份有限公司|Multifunctional building block wheel set|
    USD844394S1|2018-03-29|2019-04-02|Kraft Foods Group Brands Llc|Mold|
    US10894342B2|2018-03-29|2021-01-19|Kraft Foods Group Brands Llc|System and method for molding comestible building blocks|
    KR102215118B1|2018-12-11|2021-02-09|최재영|Method for providing augmented reality contents using play item and server using the same|
    EP3914367A1|2019-01-23|2021-12-01|Lego A/S|A toy system for augmented reality|
    法律状态:
    2018-12-11| B06F| Objections, documents and/or translations needed after an examination request according [chapter 6.6 patent gazette]|
    2019-10-22| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|
    2020-09-15| B07A| Application suspended after technical examination (opinion) [chapter 7.1 patent gazette]|
    2021-02-09| B09A| Decision: intention to grant [chapter 9.1 patent gazette]|
    2021-03-23| B16A| Patent or certificate of addition of invention granted [chapter 16.1 patent gazette]|Free format text: PRAZO DE VALIDADE: 20 (VINTE) ANOS CONTADOS A PARTIR DE 22/05/2012, OBSERVADAS AS CONDICOES LEGAIS. |
    优先权:
    申请号 | 申请日 | 专利标题
    DKPA201170255|2011-05-23|
    DKPA201170255|2011-05-23|
    PCT/EP2012/059471|WO2012160057A1|2011-05-23|2012-05-22|Generation of building instructions for construction element models|
    [返回顶部]