• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Method to provide guided augmented reality, inspection and support in installation, or maintenance, of processes for complex assemblies compatible with s1000d and device that makes use of it. A method for providing guided augmented reality, inspection and support in installation, or maintenance, of processes for complex assemblies compatible with s1000d and a device for implementing said method is described. The object of the invention is the real-time implementation of visualizations superimposed on the environment that is being observed, generating an augmented reality that, given the nature of the data shown in superposition, allows to guide or provide information that allows the guidance and monitoring in processes assembly complexes. This document also details a device that, in different configurations, allows carrying out the method of the invention in such a way that the operator is facilitated to carry out the necessary tasks. (Machine-translation by Google Translate, not legally binding)
    公开号:ES2643863A1
    申请号:ES201630669
    申请日:2016-05-24
    公开日:2017-11-24
    发明作者:Rodrigo CASARRUBIOS
    申请人:Sonovision Ingenieros Espana S A U；Sonovision Ingenieros Espana Sau；
    IPC主号:
    专利说明:

    METHOD FOR PROVIDING BY GUIDED INCREASED REALITY. INSPECTION AND SUPPORT IN INSTALLATION OR MAINTENANCE OF PROCESSES FOR COMPLEX ASSEMBLIES COMPATIBLE WITH S1000D AND DEVICE
    WHAT DOES IT USE
    5
    DESCRIPTION
    OBJECT OF THE INVENTION
    10 The object of the invention falls within the field of industrial and aeronautical maintenance as well as other locomotion platforms.
    More specifically, this document describes a system that allows to save resources and provides a greater degree of accuracy when carrying out maintenance tasks, for this they are detailed in this document method to provide through augmented reality guided, inspection and support in installation or maintenance of processes for complex assemblies compatible with the S1000D standard and device that makes use of it.
    20 BACKGROUND OF THE INVENTION
    In production chains or in various industries that make use of complex mechanical or electromechanical systems, use is made of various machines or devices whose parts suffer wear or damage in their operation due to wear and tear or working conditions in which they perform their function. .
    Equipment maintenance is one of the class parts of any industry since it allows the production to be maintained at the prescribed or desired rate while allowing that rate to be maintained over time.
    30
    Industrial maintenance is mainly based on two components: parts or spare parts and qualified maintenance technicians. In this sense you have to know on the one hand what parts are those that require repair or replacement and on the other hand, related to the first, the technician must have the knowledge
    5
    10
    fifteen
    twenty
    25
    30
    necessary to carry out the identification of the problem the parts that relate to it and in turn to carry out the necessary tasks so that the machine, equipment or device on which it is being operated continues to function as expected avoiding stops, delays or slowdowns in productive chains.
    Normally all of the above is done by implementing the service of one or more qualified technicians who carry out the necessary tasks; This requires that a person have a certain qualification and experience, that their training is sufficient and updated to deal with possible changes in references of the different parts or their corresponding replacements, and also have adequate technical documentation. This implies that an operator may find himself in a situation in which an immediate response to a stop in an industry must be given, at that time a person, usually a qualified technician, must determine what happens Why does it happen and ^ How to solve problem In this sense, the qualified technician usually has the necessary knowledge to answer the three questions; But what if there is no qualified technician available Or ^ What happens if, due to the passage of time or the work environment, it is not possible to identify a particular piece
    On the other hand, regarding the technical documentation the S1000D standard is known which corresponds to an international specification for the acquisition and production of technical publications. This specification is an XML specification for the preparation, management and maintenance use of equipment and information on operations. The S1000D standard was initially developed by the European Aerospace and Defense Industries Association (ASD) for use with military aircraft and is maintained by the S1000D Steering Committee, which includes members of the board of TEA, Aerospace Industries Association of the United States (AIA), and the Air Transport Association (ATA), together with representatives of the national industry and defense of most of the countries that currently use the specification.
    The S1000D standard requires that the document be divided into individual data elements (called data modules) that can be marked with XML tags
    individuals and metadatas, and be part of a hierarchical XML structure. This allows the updating of the individual data elements without necessarily changing the path through the XML tree that points to them. Therefore, knowledge divided and classified in this way can be easily shared among many publications, and the updating of the elements in the underlying document S1000D (XML) implies the automatic updating of the dependent publications.
    It is necessary that in those cases in which a qualified technician cannot be immediately available or that he is not able to solve the problem due, for example, to the identification of a piece, to a change in the configurations or to a product update that the manufacturer knows but not the technician, the necessary maintenance tasks can be performed.
    DESCRIPTION OF THE INVENTION
    fifteen
    The object of the invention described herein allows to solve the problems set forth above in such a way that a user or a technician can carry out maintenance tasks in real time in the place where it is required using an augmented reality environment and using the S1000D standard to the generation of instructions 20 or aids related to the corresponding manuals necessary to carry out the procedure.
    For this purpose, a method of carrying out maintenance tasks and a system to carry out said method are proposed, both of which are oriented to offer the proposed solution that is based on the application of augmented reality techniques.
    The method described here is based on the real-time capture of the user's environment through the use of a system or device comprising a series of image capture means such as a camera, said image comprises several objects 30 being one of them the one corresponding to a piece on which it is going to act.
    The user selects said piece through interaction with the system by selecting an object in the image that is isolated from the rest of the objects in the image. An image of the selected object is used as a pattern to perform a
    comparison between said image and a series of images from a database of images of referenced pieces. When the part identification has been made, a call is made to a library of data related to said part and they are sent to the device so that it is displayed on a visualization means thereof that 5 can be a screen in which it is also shown the environment captured by the camera generating a visualization of augmented reality in which you have the image captured by the image capture means and superimposed on it the data received from the piece.
    10 The data received may contain different types of graphics, text or three-dimensional models and may represent instructions or steps to carry out the necessary maintenance tasks.
    Thus, with a modular device that has a process unit, a memory or data storage media, connection media to communication networks, display media and image capture media that have been configured in such a way. so that each of its elements can execute a series of actions generated by commands of the central unit following a series of instructions corresponding to the method described above, the maintenance task that is required can be carried out. In this regard, it should be noted that a smartphone or tablet can carry out this solution; Another possible solution is given by the use of portable devices such as ocular visualization devices that would allow you to have your hands free while doing the necessary work while the captured area always corresponds to the user's vision zone.
    The object of the invention, in any aspect thereof and as mentioned above, responds to the need to standardize and reproduce the installation procedures, assembly procedures or maintenance procedures that will be applied in the physical systems, structures, elements or sets, for orientation, testing and training purposes, mainly using a system composed of a process of composition of technical data, an administrator of technical data and augmented reality player in portable devices.
    5
    10
    fifteen
    twenty
    25
    PREFERRED EMBODIMENT OF THE INVENTION
    In one aspect of the invention there is a device for carrying out industrial maintenance tasks that allows, in real time, to obtain information related to the maintenance tasks to be carried out and the parts involved in said task.
    The device of the first aspect of the invention can be defined in several formats, but always being modular and comprising a series of basic elements interconnected through a central processing unit. In this way, the device of the invention has the following elements connected to the central processing unit:
    • Means of capturing images intended to capture images or videos of a vision area corresponding to a user,
    • Visualization media.
    • Media.
    • Data entry means.
    • Means of reproduction and data capture.
    • Digital data storage media.
    So that the device of the invention has the means of capturing images adapted to capture at least one image that allow to identify objects or environments within the vision zone, and the central processing unit is adapted to process the captured image by means of image capture, in order to identify at least one object, a set of objects or the environment in which they are located and obtain data related to said elements, so that the digital processing system is capable of displaying a visualization of said data, which in turn are related to said objects, which will appear superimposed on the images or videos of a vision area corresponding to a user.
    In possible preferred embodiment of the device of the invention, the device is defined in glasses such that the central unit of
    process, image capture media, and data storage media
    6
    5
    10
    fifteen
    twenty
    25
    30
    digitals are in a frame and the visualization means are defined in at least one of the lenses of the glasses.
    While in an alternative embodiment, the device has to be defined in a portable electronic device such as a tablet or smartphone, such that the image capture means and the digital data storage media are found in the inside of a housing of said portable electronic device and the display means are defined in said housing.
    In a second aspect of the invention there is an industrial maintenance procedure that allows, in real time, to obtain information related to the maintenance tasks to be carried out and the parts involved in said task. The method includes packaging, through a Process of Integration of Technical Data, mainly: 3-Dimensional data, animated data, verbal data and text data, all such data being in relation to one or more installation procedures, maintenance assembly to be applied in physical systems, structures, elements or assemblies.
    The data is grouped in Data Modules (DM) within a database (CSDB) and is managed by a Technical Data Manager (TDM). Each data module (DM) is loaded into an Augmented Reality Player (ARP) capable of:
    • Reproduce the procedure as a dynamic dynamic interactive Augmented Reality superimposed on the video image in real time.
    • Check in real time the visual position of the elements of at least one of each data module (DM) and compare them with the positions of the procedure, warning about the deviations or errors when they are detected.
    • Reproduce the procedure outside the model for training purposes.
    The ARP is composed of a tracking and recognition module (RTM), a playback module (REP), an interface module (INF), a text and voice module (TXS) and a module loader (LD) and is compatible with standard portable devices that run, for example, light operating systems for portable devices such as Android or iOS or others.
    The ARP is loaded in the portable device of the first aspect of the invention, in a tablet or in an ocular visualization device to allow the operator to see in the visualization means, screen or lenses, the augmented reality data having the 5 hands free to interact with the data module (DM) to perform the procedure.
    In order to carry out the method of the invention, the portable device must have a possible preferred configuration thereof: a camera, a storage memory, a processor, a screen and connectivity functions to connect to the remote server if necessary.
    The database (CSDB) can be implemented locally or accessible from the portable device in a remote location. The database data system (CSDB) is such that the data is organized in such a way that it is compatible with the S1000D standard or similar, which allows compatibility with interactive electronic technical manuals (IETM). English).
    The method is initiated by capturing at least one image by means of image capture means of a vision zone where said image comprising at least one object or environment, to subsequently identify at least one object or environment selected by doing use of the central unit process where said identification is carried out by at least one of the following methods: three-dimensional volume, recognition of marks on the object, recognition of identification codes or introduction of data that allow identifying the selected object 25 by means of means of data entry. Once this is done, data related to the object is obtained from the identification carried out, and display said data in the visualization means at the same time as the image corresponding to the user's viewing area is displayed, generating a visualization. of augmented reality that comprises a digital representation, which can be three-dimensional or not, according to the requirement since the data shown in the visualization means are digital images, which can be three-dimensional, static or moving.
    The identification is carried out by sending the captured data to a local or remote unit where said data is processed for identification. Such identification
    of the selected object comprises sending at least one image of the selected object or environment to that unit, where a correlation of said image is carried out with a database of images of parts or environments allowing the identification thereof.
    5
    In an alternative embodiment of the method of the invention it is necessary to determine a relative position of the selected object with respect to an axis, point or reference marker in such a way that the digital images to be displayed, which can be three-dimensional or not, they are displayed on the display media in the same position as that of the object selected in the image.
    权利要求:
    Claims (9)
    [1]
    5
    10
    fifteen
    twenty
    25
    30
    1. Device to provide through augmented reality guidance, inspection and support in installation or maintenance of processes for complex assemblies compatible with S1000D comprising a central processing unit and characterized, in turn comprising the following elements connected to it:
    • at least one image capture means intended to capture images or videos of a vision area corresponding to a user,
    • visualization media,
    • media,
    • means of data entry,
    • means of reproduction and data capture, and
    • digital data storage media;
    where the image capture means are adapted to capture at least one image that allow to identify objects or environments within the vision zone, and the central processing unit is adapted to process the image captured by the image capture means, in order to identify at least one object, a set of objects or the environment in which they are located and obtain data related to said elements, so that the digital processing system is capable of displaying a visualization of said data, which at its Once they are related to these objects, they will appear superimposed on the images or videos of a vision area corresponding to a user.
    [2]
    2. Device according to claim 1 characterized in that it is defined in glasses or ocular display device such that the central processing unit, the image capture means, and the digital data storage means are located in a frame or support and the visualization means are defined in at least one of the lenses of the glasses.
    [3]
    3. Device according to claim 1 characterized in that it is defined in a portable electronic device such as a tablet or smartphone, such that the image capture means and the digital data storage media are located inside of a housing of said device
    portable electronic and visualization means are defined in said
    10
    5
    10
    fifteen
    twenty
    25
    30
    Case.
    [4]
    4. Method to provide by augmented reality guidance, inspection and support in installation or maintenance of processes for complex assemblies compatible with S1000D that makes use of the system described in claim 1, 2 or 3, the method being characterized by comprising:
    • capture at least one image by means of image capture means of a vision area where said image comprising at least one object or environment,
    • identify at least one object or selected environment using the central unit process where said identification is carried out by at least one of the following methods: three-dimensional volume, recognition of marks on the object, recognition of identification codes or introduction of data that allow to identify the selected object by means of data entry,
    • obtain data related to the object from the identification carried out, and
    • display said data in the visualization means at the same time as the image corresponding to the user's viewing area is shown, generating an augmented reality visualization comprising a digital representation, which can be three-dimensional or not, according to the requirement.
    [5]
    5. Method according to claim 4 characterized in that the identification is carried out by sending the captured data to a local unit where said data is processed for identification.
    [6]
    6. Method according to claim 4 characterized in that the identification is carried out by an additional sending of the captured data to a remote unit where said data is processed for identification.
    [7]
    7. Method according to any one of claims 4 to 6, characterized in that the data shown on the display means are digital images, which can be three-dimensional, static or moving.
    [8]
    8. Method according to revindication 7 characterized in that it additionally comprises determining a relative position of the selected object with respect to an axis, point or reference mark such that the digital images to be displayed, which may be three-dimensional or not, are shown in the visualization media in the same position
    5 than that of the object selected in the image.
    [9]
    Method according to one of claims 4 to 8, characterized in that the identification of the selected object comprises sending at least one image of the selected object or environment to a remote unit where it is carried out.
    10 a correlation of said image with a database of images of parts or environments to carry out the identification thereof.
    类似技术:
    公开号 | 公开日 | 专利标题
    JP2019067383A|2019-04-25|3D mapping of process control environment
    US10504231B2|2019-12-10|Fiducial marker patterns, their automatic detection in images, and applications thereof
    ES2240837T3|2005-10-16|SYSTEM AND PROCEDURE FOR THE CREATION OF A DOCUMENTATION OF WORK PROCESSES FOR REPRESENTATION IN AN EXTENDED REALITY SYSTEM.
    US9746913B2|2017-08-29|Secured mobile maintenance and operator system including wearable augmented reality interface, voice command interface, and visual recognition systems and related methods
    US9448758B2|2016-09-20|Projecting airplane location specific maintenance history using optical reference points
    CN106569769A|2017-04-19|AR technology-based machine operation instruction information display method and apparatus
    US20150192774A1|2015-07-09|Support device and system for article picking work
    US20130275313A1|2013-10-17|Apparatus, methods and systems for knowledge based maintenance
    US20170018120A1|2017-01-19|System and method for superimposing spatially correlated data over live real-world images
    US11244509B2|2022-02-08|Drift correction for industrial augmented reality applications
    WO2015171887A1|2015-11-12|Hands-free asset identification, location and management system
    ES2551394T3|2015-11-18|System and method of installation and alarm configuration
    ES2643863B1|2018-10-26|METHOD FOR PROVIDING BY GUIDED INCREASED REALITY, INSPECTION AND SUPPORT IN INSTALLATION OR MAINTENANCE OF PROCESSES FOR COMPLEX ASSEMBLIES COMPATIBLE WITH S1000D AND DEVICE THAT MAKES SAME USE
    CN111512370B|2021-08-03|Voice tagging of video while recording
    KR101894471B1|2018-09-03|Method for providing virtual reality service of plant and device for the same
    CN107644457A|2018-01-30|A kind of monitoring method applied to industry spot
    Benecke et al.2016|Mobile solution for positioning, 3D-mapping and inspection in underground mining
    JP2021018710A|2021-02-15|Site cooperation system and management device
    ES2858549T3|2021-09-30|Using an image of a valve unit to identify the valve unit in a process line
    DE102018123635A1|2019-03-28|3D MAPPING OF A PROCESS CONTROL ENVIRONMENT
    KR20170089575A|2017-08-04|A distribution management system using augment reality
    US20220044016A1|2022-02-10|Quick Activation Techniques for Industrial Augmented Reality Applications
    KR20190134801A|2019-12-04|Management System, Method and Computer Program of Semiconductor Manufacturing Equipment
    CN111162840B|2020-09-29|Method and system for setting virtual objects around optical communication device
    ES2552400B1|2016-09-09|Procedure for the digitalization of enclosed spaces for the inclusion of information in digital images
    同族专利:
    公开号 | 公开日
    ES2643863B1|2018-10-26|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20110202468A1|2010-02-17|2011-08-18|Dan Crowell|Customizing an Extensible Markup Language Standard for Technical Documentation|
    US8775552B1|2010-04-23|2014-07-08|The Boeing Company|Methods and systems for distribution of technical manuals|
    US20140361988A1|2011-09-19|2014-12-11|Eyesight Mobile Technologies Ltd.|Touch Free Interface for Augmented Reality Systems|
    WO2015028978A1|2013-08-29|2015-03-05|Umpi Elettronica - Societa' A Responsabilita' Limitata|Method and system for the inspection and/or the maintenance of an electrical panel by means of augmented reality|
    US20160019212A1|2014-07-21|2016-01-21|Airbus Operations |Maintenance assistance for an aircraft by augmented reality|
    WO2016025585A1|2014-08-13|2016-02-18|Huntington Ingalls Incorporated|System and method for using augmented reality display in surface treatment procedures|US10891798B2|2017-06-05|2021-01-12|2689090 Canada Inc.|System and method for displaying an asset of an interactive electronic technical publication synchronously in a plurality of extended reality display devices|
    法律状态:
    2018-10-26| FG2A| Definitive protection|Ref document number: 2643863 Country of ref document: ES Kind code of ref document: B1 Effective date: 20181026 |
    优先权:
    申请号 | 申请日 | 专利标题
    ES201630669A|ES2643863B1|2016-05-24|2016-05-24|METHOD FOR PROVIDING BY GUIDED INCREASED REALITY, INSPECTION AND SUPPORT IN INSTALLATION OR MAINTENANCE OF PROCESSES FOR COMPLEX ASSEMBLIES COMPATIBLE WITH S1000D AND DEVICE THAT MAKES SAME USE|ES201630669A| ES2643863B1|2016-05-24|2016-05-24|METHOD FOR PROVIDING BY GUIDED INCREASED REALITY, INSPECTION AND SUPPORT IN INSTALLATION OR MAINTENANCE OF PROCESSES FOR COMPLEX ASSEMBLIES COMPATIBLE WITH S1000D AND DEVICE THAT MAKES SAME USE|
    [返回顶部]