• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Method and system of video and data synchronization. The method includes: - generating a marked video signal by extracting an image fragment (V) from a region (r) of an original frame (f), and having thereon a watermark (m); y - generating a data signal, including in it the extracted image fragment (V) and/or reconstruction information; - perform an additional stage of synchronization comprising: - obtaining, from the data signal, the image fragment (V) and/or the reconstruction information, - obtain, from the marked video signal, an incomplete frame (fr), and - try to reconstruct the original frame (f) from the information obtained in iiib1) and iiib2), and if the reconstruction is successful determine that the synchronization is correct. The system is adapted to implement the method of the invention. (Machine-translation by Google Translate, not legally binding)
    公开号:ES2629484A1
    申请号:ES201630148
    申请日:2016-02-09
    公开日:2017-08-10
    发明作者:Eloi MOLINAS LOMBART；Ángel ANDRÉS MORÁN
    申请人:Eloi MOLINAS LOMBART；Ángel ANDRÉS MORÁN；
    IPC主号:
    专利说明:

    METHOD AND SYNCHRONIZATION SYSTEM OF VIDEO AND DATA
    5 Technical sector
    The present invention generally concerns, in a first aspect, a method of video and data synchronization, which comprises using information from a watermark, and more particularly a method comprising performing an additional step of
    10 synchronization, with part of the information of said watermark ",
    A second aspect of the present invention concerns a video and data synchronization system adapted to implement the method of the first aspect.
    Prior art
    15 Video and data synchronization methods that meet the characteristics included in the preamble of claim 1 of the present invention are known in the state of the art, that is, they comprise:
    20 i) providing at least one marked video signal that includes a video signal comprising at least one marked frame that includes a watermark in a region of at least one original frame (meaning "original" by initial), where said watermark includes, encoded, synchronization information of said original frame, which is at least one, with data associated therewith;
    ii) sending said marked video signal to a receiver with access to a data signal containing said associated data; Y
    iii) perform a synchronization step comprising extracting, by said receiver,
    30 the synchronization information included in the watermark of said marked frame of the marked video signal, decoding it, and using it to synchronize said original frame with said data associated therewith.
    Such is the case of the methods described in US201502429S3A1,
    35 W02007072327A2 AND W020050S9476A2. In all of them, it is emphasized that the watermark must be imperceptible or not discernible. Even at US201502429S3A1, where it
    makes a discussion about the commitment to be achieved between the imperceptibility of the watermark and its robustness, it is considered that invasive techniques that provide greater robustness, as is the case of additive techniques, should be applied by dividing the value to add between a group of pixels to improve imperceptibility.
    Such imperceptibility in the inclusion of watermarks on the frames of the video signal means that, in the event that the communication channel through which the video signal is transmitted degrades its quality, it may become impossible retrieve the information encoded in the watermark (even using the "informed or" non-blind "methods described in US20150242983A1), as well as reconstruct the video signal if it reaches a very degraded destination.
    On the other hand, when very precise synchronization is needed, frame by frame, the proposed synchronization in such methods may not be sufficient for such a task.
    Likewise, the aforementioned methods also do not guarantee with certainty that neither the data that is determined to be associated with the frames of the video signal nor the video signal have undergone any type of manipulation before reaching the receiver, that is, both are the correct and associated with each other.
    It is therefore necessary to offer an alternative to the state of the art that covers the gaps found therein, providing a method and a system that allows to achieve the aforementioned precise synchronization between video and data, as well as the aforementioned guarantee that no there has been a manipulation or alteration of the data and / or the video signal.
    Explanation of the invention.
    To this end, the present invention concerns, in a first aspect, a method of video and data synchronization, comprising:
    i) providing at least one marked video signal that includes a video signal comprising at least one marked frame that includes a watermark in a region of at least one original frame, wherein said watermark includes, encoded, information of synchronization of said frame, which is at least one, with data associated therewith;
    ii) sending said marked video signal to a receiver with access to a data signal containing said associated data; Y
    iii) performing a synchronization step comprising extracting, by said receiver, the synchronization information included in the watermark of said marked frame of the marked video signal, decoding it, and using it to synchronize said frame with said associated data the same.
    Unlike the known methods, in the one proposed by the first aspect of the present invention, in a characteristic manner:
    the cited stage i) comprises:
    ia) generate said marked video signal by extracting an image fragment from a region of said original frame, and arranging the watermark in said region of said original frame, replacing the image fragment extracted from it, obtaining said frame. marked; Y
    ib) generate said data signal, including in it the extracted image fragment and / or reconstruction information that allows its reconstruction;
    the said stage iii) comprises:
    iiia) perform the mentioned synchronization stage; Y
    iiib) perform, by the receiver, an additional synchronization step comprising:
    iiib1) obtain, from the data signal, the image fragment and / or reconstruction information included therein,
    iiib2) obtain, from the marked video signal received, an incomplete frame that includes the original frame without the watermark and without the extracted image fragment, and
    iiib3) try to reconstruct the original frame from the information obtained in iiib1) and iiib2), and if the reconstruction is successful, determine that the frame of the video signal and the associated data of the data signal are properly synchronized.
    According to an exemplary embodiment, the sub-stage ib) comprises including in the data signal synchronization information of the data associated with the frame, and the sub-stage iia) comprises performing the synchronization stage by also extracting, on the part of the receiver, the synchronization information included in the data signal and also using it to synchronize the frame with the data associated therewith.
    Advantageously, according to the method proposed by the first aspect of the invention, step ii) comprises sending both the marked video signal and the data signal to the receiver, via independent communication channels.
    In general, the synchronization information of the video signal or that of the video signal and that of the data signal comprises at least one time code associated with the original frame.
    According to a preferred embodiment of the method proposed by the first aspect of the invention, said frame is a first frame, and the method comprises synchronizing a plurality of frames of the video signal with a corresponding plurality of associated data contained in the data signal, frame by frame, in the same way as with the first frame, that is to say in the same way as explained in the previous paragraphs.
    The method of the first aspect of the present invention comprises sending the video signal from a transmitter to said receiver, which is a remote receiver, through any type of communication path, even though it may cause signal loss or degradation. (as can happen in the case of satellite communications), since, on the one hand, the watermark is arranged in or on the frame (s) as invasive and non-reversible as necessary, so it is very unlikely that it cannot be decoded at destination, and the frame (s), even if they arrive very damaged to the receiver, can be reconstructed thanks to the reconstruction information contained in the data signal.
    According to an embodiment, the method of the first aspect of the present invention comprises, if the reconstruction of step iiib3) is successful, presenting at least one screen the video signal that includes the reconstructed frame or frames, and using the data associated to each reconstructed frame to, at the same moment in which the frame is presented, perform an action associated to it and determined by the associated data.
    According to some examples of realization, the aforementioned action is at least one of the following actions: present at least the aforementioned screen graphic and / or textual information (graphics, subtitles, legends, etc.) included in the associated data or accessible through them, reproduce audio from an audio signal included in the associated data
    or accessible through them and control a device connected to the receiver from control commands included in the associated data or accessible through them.
    Advantageously, the method of the present invention is applied to the television retransmission of the video signal to a plurality of televisions or the like, from the receiver, either together with the associated data duly synchronized frame by frame, if these have such purpose, or controlled based on them, if these are data that include control commands.
    A second aspect of the present invention concerns a video and data synchronization system, comprising:
    - a first computational entity, which is adapted to perform stages i) and ii) of the method of the first aspect; Y
    - a receiver that includes a second computational entity and is adapted to perform step iii) of the first aspect method.
    According to an exemplary embodiment, said first computational entity and said receiver have the capacity to communicate with each other through at least two independent communication channels, to implement the method of the first aspect.
    The meaning given to the terms "watermark" in the present invention is very broad, this being understood as any type of graphic mark to embed / embed / attach to one or more frames of the video signal in the spatial domain , frequency or a combination thereof, regardless of the data or "metadata" that is encoded (that is, the restrictive definition that is sometimes given to the terms "watermark" is not accepted, relative to that which includes identifying information of the author of the content on which it is associated).
    Brief description of the drawings
    The foregoing and other advantages and features will be more fully understood from the following detailed description of some embodiments with reference to the attached drawings, which should be taken by way of illustration and not limitation, in which:
    Figure 1 graphically shows how an original frame is manipulated to include a watermark according to the method proposed by the first aspect of the invention, for an exemplary embodiment.
    Figure 2 schematically illustrates, by means of a block diagram, the system proposed by the second aspect of the invention implementing the method of the first aspect.
    Detailed description of some embodiments
    The method proposed by the first aspect of the invention comprises the steps ia), ib), ii), iiia) and iiib) (iiib1, iiib2 and iiib3) already described in a previous section.
    As regards stage ia), that is to say that of the generation of the marked video signal, Figure 1 illustrates graphically how an original frame F (view a »is manipulated until it becomes a frame marked Fm a include in said cited video signal, in particular, as seen in view b), extracting a fragment of image V from a region R of said original frame F, co-inverting it into an incomplete frame Fr, and providing the mark of water M in the region R of the frame F, replacing the image fragment extracted V, thus obtaining the frame marked Fm (view e ».
    In step ib) the image fragment extracted V and / or reconstruction information is included in the data signal that is sent to the receiver R (see Figure 2)
    As regards the additional synchronization stage iiib), as described in a previous section, this includes the following sub-stages:
    iiib1) obtain, from the data signal, the image fragment V and / or the information of
    reconstruction included in it,
    iiib2) obtain, from the marked video signal received, an incomplete frame Fr that includes the original frame F without the watermark M and without the extracted image fragment V, and
    iiib3) try to reconstruct the original frame F from the information obtained in iiib1) and iiib2), and if the reconstruction is successful, determine that the frame of the video signal and the associated data of the data signal are properly synchronized.
    A possible implementation of the system proposed by the second aspect of the present invention is illustrated in Figure 2, for the transmission and synchronization of video and data according to the method of the first aspect
    The system is formed by a video analyzer 1 that extracts the video information necessary for the final reconstruction, that is the portion or portions of image V (one or more per frame) and / or reconstruction information, a video encoder 2 which creates a video signal that combines the original video signal V (manipulated, that is, with the frame or frames without the image portion or portions V) with a time code t in the form of a watermark M (one or more per frame), according to step ia) of the method, the combined video signal being sent via a communication channel P1 to the receiver R.
    The system also includes a data packer 3 that combines the timecode of the current frame with the information to be transmitted, that is to say with the data D, and the information necessary for subsequent reconstruction, that is with the image portion V and / or the reconstruction information, according to step ib) of the method, the combined data signal being sent to the receiver R by another communication path P2.
    At destination, the system comprises a receiver R consisting of a video decoder 4 that is capable of interpreting the time code contained in the watermark M, a synchronizer 5 that accumulates the combined data signal and synchronizes it with the code of time collected by the video decoder 4, and a reconstructor 6 that is capable of reconstructing the original video from the watermarked video M and the synchronized data from the same frame, that is, from Fm, Fr and V , so that at the output of the system you get the original video V synchronized with the original data D.
    In general, a large part or each of the frames of the video signal are synchronized with data associated univocally with them, frame by frame, according to the method of the first aspect of the invention.
    Advantages over known proposals:
    Reversible watermark coding techniques use the video signal itself to send "hidden" data so that it is imperceptible or the original image can be reconstructed only with the information contained in the frame itself. Through the present invention, the following advantages are obtained over the systems / methods that implement such techniques:
    - Greater security against signal degradation.
    - The usable bandwidth for each frame is not limited by the need not to "dirty" the image.
    - When traveling the data in a different way from the video signal, a higher level of security is provided than in the known proposals (both signals are necessary to reconstruct the original video and to access the data).
    Usage examples:
    The present invention can be used, among others, in the following scenarios:
    - To synchronize clocks between the broadcast point and the receiver (for example, to paint the clock of a soccer broadcast from the destination, synchronize audio and video, etc.)
    - To launch controlled graphics from the origin (at the end of a retransmission, launch the end of game graphics).
    - To control equipment at destination from the source (for example, a change of camera at source launches a video at destination).
    - To send associated metadata to an image (for example, if you see an image of a
    character, send the name of that character).
    A person skilled in the art could introduce changes and modifications in the described embodiments without departing from the scope of the invention as defined in the appended claims.
    权利要求:
    Claims (9)
    [1]
    1.-Method of synchronization of video and data, which includes:
    i) providing at least one marked video signal that includes a video signal comprising at least one marked frame (Fm) that includes a watermark (M) in a region (R) of at least one original frame (F) , wherein said watermark (M) includes, encoded, synchronization information of said original frame (F), which is at least one, with data associated therewith;
    ii) sending said marked video signal to a receiver (R) with access to a data signal containing said associated data; Y
    Ii) perform a synchronization step comprising extracting, by said receiver (R), the synchronization information included in the watermark (M) of said marked frame (Fm) of the marked video signal, decoding it, and use it to synchronize said original frame (F) with said data associated therewith;
    the method being characterized because:
    said step i) comprises:
    ia) generating said marked video signal by extracting an image fragment (V) from a region (R) of said original frame (F), and disposing said watermark (M) in said region (R) of said frame (F) ), replacing said extracted image fragment (V), obtaining said marked frame (Fm); Y
    ib) generate said data signal, including said extracted image fragment therein
    (V) and / or reconstruction information that allows its reconstruction;
    said stage iii) comprises:
    iiia) perform said synchronization stage; Y
    iiib) performing, by said receiver (R), an additional synchronization step comprising:
    iiib1) obtain, from the data signal, the image fragment (V) and / or the reconstruction information included therein,
    iiib2) obtain, from the marked video signal received, an incomplete frame (Fr) that includes said original frame (F) without the watermark (M) and without the extracted image fragment (V), and
    iiib3) try to reconstruct the original frame (F) from the information obtained in iiib1) and iiib2), and if the reconstruction is successful, determine that the frame of the video signal and the associated data of the data signal are properly synchronized .
    [2]
    2. Method according to claim 1, wherein the sub-stage ib) comprises including in the data signal synchronization information of said data associated with said original frame (F), and the sub-stage iia) comprises performing said synchronization stage also extracting, by the receiver (R), the synchronization information included in the data signal and also using it to synchronize said original frame (F) with the data associated therewith.
    [3]
    3. A method according to claim 1 or 2, wherein step ii) comprises sending to said receiver (R) both said marked video signal and said data signal, via independent communication channels.
    [4]
    4. A method according to any one of the preceding claims, wherein the synchronization information of the video signal or that of the video signal and that of the data signal comprises a time code associated with said original frame (F ).
    [5]
    5. Method according to any one of the preceding claims, wherein said original frame (F) is a first frame, and the method comprises synchronizing a plurality of frames of the video signal with a corresponding plurality of associated data contained in the signal of data, frame by frame, in the same way that is done with said first frame.
    [6]
    6. Method according to any one of the preceding claims, comprising, if the reconstruction of step iiib3) is successful, presenting at least one screen the video signal that includes the reconstructed frame or frames, and using the data associated with each reconstructed frame to, at the same moment in which the frame is presented, perform an action associated with it and determined by the associated data.
    [7]
    7. Method according to claim 6, wherein said action is at least one of the following
    5 actions: present on at least said screen graphic and / or textual information included in the associated data or accessible through them, reproduce audio from an audio signal included in the associated data or accessible through them and control a device connected to the receiver from control commands included in the associated data or accessible through them.
    [8]
    8.-Video and data synchronization system, which includes:
    - a first computational entity (E), which is adapted to perform steps i) and ii) of the method according to any one of the preceding claims; Y
    A receiver (R) that includes a second computational entity and is adapted to perform step iii) of the method according to any one of the preceding claims.
    [9]
    9. System according to claim 8, wherein said first computational entity (E) and
    Said receiver (R) has the capacity to communicate with each other through at least two independent communication channels, to implement the method according to claim 3.
    类似技术:
    公开号 | 公开日 | 专利标题
    CN102144237B|2014-10-22|Efficient watermarking approaches of compressed media
    US7623673B2|2009-11-24|Versatile watermark transport system
    EP1579686B1|2014-04-16|Personalized marking for protecting digital audiovisual streams
    WO2008013883A3|2008-03-13|Method and apparatus for fast channel change for digital video
    AR044118A1|2005-08-24|IMAGE CODING METHOD
    WO2001031838A1|2001-05-03|Active data hiding for secure electronic media distribution
    US20170034523A1|2017-02-02|Media content processing method
    ES2629484B1|2018-03-08|VIDEO AND DATA SYNCHRONIZATION METHOD AND SYSTEM
    JP5897570B2|2016-03-30|Method and system for watermark protection against unauthorized use of presented original audio or video data
    CN101897182B|2013-07-17|Method and system for the secure distribution of audiovisual data by transactional marking
    WO2008048356A3|2009-04-16|Device and method for digitally watermarking an image with gps data
    MY146452A|2012-08-15|Method and apparatus for signaling transmission code set in data broadcasting
    CN102946531A|2013-02-27|GOP | frame structure combined video watermarking method and system
    CN106792088A|2017-05-31|The encryption and decryption method and system of a kind of plug-in captions, plug-in caption system
    Saberian et al.2008|An invertible quantization based watermarking approach
    Pitropakis et al.2013|A Practical Steganographic Approach for Matroska Based High Quality Video Files
    CN101238731A|2008-08-06|Method and system for video copyright protection
    Freitas et al.2016|Secure self-recovery watermarking scheme for error concealment and tampering detection
    CN107680608A|2018-02-09|A kind of breakable watermark self- recoverage algorithm based on fountain codes
    US8745399B2|2014-06-03|Method for sharing and updating key using watermark
    KR20130138236A|2013-12-18|Method for guaranteing watermark embedding by using bit stream corruption
    JP6302194B2|2018-03-28|Inverse multiplexing apparatus and program
    WO2004032478A3|2004-06-17|Adaptive and progressive video stream scrambling
    US20210352381A1|2021-11-11|Methods and systems for reducing piracy of media content
    Zaid et al.2007|Wavelet domain watermark embedding strategy using TTCQ quantization
    同族专利:
    公开号 | 公开日
    EP3414912A1|2018-12-19|
    RU2018130772A|2020-03-10|
    JP2019508991A|2019-03-28|
    CN108605157A|2018-09-28|
    RU2018130772A3|2020-03-10|
    ES2629484B1|2018-03-08|
    US20190052777A1|2019-02-14|
    EP3414912B1|2020-04-08|
    US10587781B2|2020-03-10|
    WO2017137358A1|2017-08-17|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20090110231A1|2007-10-30|2009-04-30|General Instrument Corporation|Method, device and system for dynamically embedding watermark information into multimedia content|
    US20140013374A1|2012-07-05|2014-01-09|Lg Electronics Inc.|Method and apparatus for processing digital service signals|
    US20150264429A1|2014-03-13|2015-09-17|Verance Corporation|Interactive content acquisition using embedded codes|
    JP2565096B2|1992-07-31|1996-12-18|日本電気株式会社|Compressed video / audio signal expansion device|
    JP2000092461A|1998-09-10|2000-03-31|Sony Corp|Additional information superimposing method and video signal output device|
    EP1143698A3|2000-02-09|2005-01-05|Canon Kabushiki Kaisha|Data processing apparatus and method, and storage medium|
    US8407752B2|2004-03-18|2013-03-26|Digimarc Corporation|Synchronizing broadcast content with corresponding network content|
    GB2425906B|2005-05-05|2011-04-06|Sony Uk Ltd|Data processing apparatus and method|
    JP2009521170A|2005-12-22|2009-05-28|コーニンクレッカフィリップスエレクトロニクスエヌヴィ|Script synchronization method using watermark|
    CN101668170A|2009-09-23|2010-03-10|中山大学|Digital television program copyright protecting method for resisting time synchronization attacks|
    CN102523535B|2011-12-01|2014-04-23|南京师范大学|Video stream and global positioning system real-time registration and synchronous transmission method based on digital watermarking technology|
    WO2013103584A1|2012-01-06|2013-07-11|Thomson Licensing|Method and system for synchronising content on a second screen|
    US9299119B2|2014-02-24|2016-03-29|Disney Enterprises, Inc.|Overlay-based watermarking for video synchronization with contextual data|
    GB2527662B|2015-05-12|2016-05-25|Gamesys Ltd|Data synchronisation|GB2569547B|2017-12-19|2021-05-12|Samsung Electronics Co Ltd|Reconstruction of original images from modified images|
    法律状态:
    2018-03-08| FG2A| Definitive protection|Ref document number: 2629484 Country of ref document: ES Kind code of ref document: B1 Effective date: 20180308 |
    优先权:
    申请号 | 申请日 | 专利标题
    ES201630148A|ES2629484B1|2016-02-09|2016-02-09|VIDEO AND DATA SYNCHRONIZATION METHOD AND SYSTEM|ES201630148A| ES2629484B1|2016-02-09|2016-02-09|VIDEO AND DATA SYNCHRONIZATION METHOD AND SYSTEM|
    CN201780010542.9A| CN108605157A|2016-02-09|2017-02-06|Method and system for synchronization video and data|
    EP17705052.3A| EP3414912B1|2016-02-09|2017-02-06|Method and system for synchronising video and data|
    PCT/EP2017/052560| WO2017137358A1|2016-02-09|2017-02-06|Method and system for synchronising video and data|
    RU2018130772A| RU2018130772A3|2016-02-09|2017-02-06|
    US16/076,782| US10587781B2|2016-02-09|2017-02-06|Method and system for synchronising video and data|
    JP2018560250A| JP2019508991A|2016-02-09|2017-02-06|Method and system for synchronizing video and data|
    [返回顶部]