METHOD FOR PERFORMING MULTIPLE HYPOTHESIS PREDICTION LOCATED DURING VIDEO ENCODING OF AN ENCODING UN

专利摘要:
METHOD FOR CARRYING OUT MULTIPLE-HYPOTHESIS PREDICTION LOCALIZED DURING VIDEO ENCODING OF AN ENCODING UNIT AND ASSOCIATED APPARATUS. This is a method for performing localized multiple-hypothesis prediction during video encoding of an encoding unit which includes: dividing the encoding unit into a plurality of subcoding units; and performing motion vector prediction of each of the subcoding units. More particularly, step of performing motion vector prediction of each of the subcoding units further includes obtaining a plurality of motion vectors for motion compensation of multiple hypotheses of a subcoding unit specifies the subcoding units from a plurality of other subcoding/coding units. The method further includes performing multi-chance motion compensation in the subcoding unit particularly, includes using a linear combination of a plurality of pixel values from the plurality of other subcoding/coding units as a predicted pixel value of the specific subcoding unit. An associated device is also provided.
公开号:BR112012025275B1
申请号:R112012025275-5
申请日:2011-04-14
公开日:2021-08-03
发明作者:Jian-Liang Lin；Yu-Pao Tsai；Shaw-Min Lei
申请人:Hfi Innovation Inc；
IPC主号:

专利说明:

BACKGROUND OF THE INVENTION Cross Reference to Related Claims
This application claims the benefit of Interim Application No. 61/323,948, which was filed April 14, 2010 and entitled "MULTIHYPOTHESIS PREDICTION IN VIDEO CODING" and is incorporated herein by reference. This order further claims the benefit of Order No. US 13/080,668, which was filed on April 6, 2010 and entitled "METHOD FOR PERFORMING LOCALIZED MULTIHYPOTHESIS PREDICTION DURING VIDEO CODING OF A CODING UNIT, AND ASSOCIATED APPARATUS" and is incorporated herein invention by way of reference. Field of Invention
The present invention relates to video processing in relation to motion compensation, and more particularly to a method for performing localized multiple-hypothesis prediction during video encoding of an encoding unit and an associated apparatus. Description of Related Art
Motion compensation is a technique used in encoding and decoding video data for video compression and decompression. With the aid of motion compensation, a current picture can be represented in terms of some portions of one or more reference pictures, where the reference pictures may be earlier in time or even from the future. Typically, when motion compensation is used, images can be synthesized precisely from previously transmitted/stored images and compression efficiency can be improved. As coding techniques advance, designs against newer standards have been introduced. It seems unlikely that conventional motion compensation schemes can ever work properly. For example, in some situations, a problem of low coding efficiency might exist. As such, a new method is required to improve coding efficiency in video encoding. BRIEF SUMMARY OF THE INVENTION
It is therefore an object of the claimed invention to provide a method for performing localized multi-hypothesis prediction during video encoding of an encoding unit and to provide an associated apparatus in order to solve the above-mentioned problem.
An exemplary embodiment of a method for performing localized multiple-hypothesis prediction during video encoding of an encoding unit comprises: dividing the encoding unit into a plurality of subcoding units; and process each of the subcoding units. More particularly, the step of processing each of the subcoding units further comprises: obtaining motion information from a first set of coded units for multi-chance motion compensation of a specific subcoding unit among the subcoding units; using a linear combination of a plurality of pixel values derived from the motion information of the first set of coded units as a predicted pixel value of the specific subcoding unit; and deriving a predicted pixel value from another subcoding unit in the encoding unit using the motion information from a second set of coded units, wherein the second set of coded units comprises at least one coded unit that is not in the first. set of coded units.
An exemplary embodiment of an apparatus for performing multiple-hypothesis prediction located during video encoding of an encoding unit comprises a processing circuit arranged to perform video encoding in the encoding unit, wherein the processing circuit comprises a module of pre-processing and at least one coding module. The pre-processing module is arranged to divide the encoding unit into a plurality of subcoding units and to process each of the subcoding units. More particularly, the pre-processing module obtains the motion information of a first set of coded units for multi-chance motion compensation of a specific sub-coding unit of the sub-coding units and performs the multi-chance motion compensation in the unit. of specific subcoding according to the motion information, wherein the pre-processing module uses a linear combination of a plurality of pixel values derived from the motion information of the first set of the encoded units as a predicted pixel value of the unit of specific subcoding. Additionally, the at least one coding module is arranged to perform video coding in the coding unit based on the multi-chance motion compensation performed by the pre-processing module. Additionally, the pre-processing module derives a predicted pixel value from another subcoding unit in the encoding unit using motion information from a second set of coded units comprising at least one coded unit that is not in the first set of coded units.
An exemplary embodiment of a method for performing localized multiple-hypothesis prediction during video encoding of an encoding unit comprises: dividing the encoding unit into a plurality of subcoding units; and process each of the subcoding units. More particularly, the step of processing each of the subcoding units further comprises: obtaining motion information from a plurality of coded units for multi-chance motion compensation of a specific subcoding unit of the subcoding units; using a weighted sum of a plurality of pixel values derived from the motion information of the encoded units as a predicted pixel value of the specific subcoding unit; and deriving a predicted pixel value from another subcoding unit in the encoding unit using a different weighted sum of the pixel values derived from the motion information of the encoded units.
These and other objects of the present invention will no doubt become obvious to those skilled in the art upon reading the following detailed description of the preferred embodiment which is illustrated in the various figures and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1A is a diagram of an apparatus for performing localized multi-hypothesis prediction during video encoding of an encoding unit in accordance with a first embodiment of the present invention.
Figure 1B illustrates the Inter/Intra-Prediction module shown in Figure 1A according to an embodiment of the present invention.
Figure 1C illustrates the exemplary multi-hypothesis motion compensating operations of the multi-hypothesis Inter-Prediction circuit shown in Figure IB in accordance with an embodiment of the present invention.
Figure 2 is a flowchart of a method for performing localized multi-hypothesis prediction during video encoding of an encoding unit in accordance with an embodiment of the present invention.
Figure 3 illustrates exemplary coded blocks involved with the method shown in Figure 2 in accordance with an embodiment of the present invention.
Figure 4 illustrates deployment details involved with the method shown in Figure 2 in accordance with an embodiment of the present invention.
Figures 5A to 5D illustrate some implementation details of the prediction of multiple hypotheses involved with the method shown in Figure 2 according to different embodiments of the present invention.
Figures 6A to 6B illustrate exemplary sources for obtaining motion information that are involved with the method shown in Figure 2 in accordance with different embodiments of the present invention.
Figures 7A to 7B respectively illustrate temporal and spatial motion vectors (MVs) involved with the method shown in Figure 2 in accordance with different embodiments of the present invention.
Figures 8A to 8C illustrate exemplary sources for obtaining motion vectors that are involved with the method shown in Figure 2 in accordance with some embodiments of the present invention.
Figure 9 illustrates deployment details involved with the method shown in Figure 2 in accordance with an embodiment of the present invention.
Figure 10 illustrates some deployment details involved with the method shown in Figure 2 according to an embodiment of the present invention.
Figure 11 illustrates deployment details involved with the method shown in Figure 2 in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION
Certain terms used throughout the following description and claims that refer to particular components. As one skilled in the art will appreciate, electronic equipment manufacturers may refer to a component by different names. This document is not intended to distinguish between components that differ in name but not in function. In the following description and claims, the terms "include" and "comprise" are used in an unrestricted manner and thus are to be interpreted to mean "include, but not limited to". Also, the term "couple" is intended to mean an electrical or direct or indirect connection. Consequently, if a device is coupled to another device, that connection can be through a direct electrical connection or through an indirect electrical connection through other devices and connections.
Refer to Figure 1A, which illustrates a diagram of an apparatus 100 for performing localized multiple-hypothesis prediction during video encoding of an encoding unit in accordance with a first embodiment of the present invention. Apparatus 100 comprises an Inter/Intra-Prediction module 110 (labeled "Inter/Intra-Prediction" in Figure 1A), an arithmetic unit 120, a transform and quantization module 130 (labeled "Transform and Quantize" in Figure 1A), an entropy encoding circuit 140 (labeled “Entropy Encoding” in Figure 1A), an inverse transform and inverse quantization module 150 (labeled “Inverse Transformation and Inverse Quantization” in Figure 1A), a reconstruction 160 (labeled as "REC" in Figure 1A), an unblocking filter 170, and a frame buffer 180. Referring to Figure IB, the Inter/Intra-Prediction module 110 mentioned above may comprise an Inter circuit. -Multiple Hypothesis Prediction 112 (labeled “Inter-Prediction of Multiple Hypotheses” in Figure IB), an Inter-Prediction circuit 114 (labeled “Inter-Prediction” in Figure IB), an Intra-Prediction circuit 116 ( labeled "Intra-Prediction" in Figure IB) and a switching circuit 118.
In accordance with the embodiment shown in Figure 1A , apparatus 100 performs video encoding on an original signal 109 and generates an output signal that carries encoding results, such as the output of entropy encoding circuit 140. For example, the original signal 109 may represent the input video carrying data from the encoding unit and the output of the entropy encoding circuit 140 may be an output bitstream (or bitstream). Additionally, the Inter/Intra-Prediction module 110 is arranged to perform the Inter/Intra-Prediction and, more particularly, to perform the Inter-Prediction of multiple hypotheses, Inter-Prediction and Intra-Prediction using the Inter-Prediction circuit. -Multiple Hypothesis Prediction 112, the Inter-Prediction circuit 114 and the Intra-Prediction circuit 116 shown in Figure IB, respectively.
As shown in Figure 1A, the arithmetic unit 120 is arranged to perform an arithmetic operation such as a subtraction operation on the original signal 109 (which may represent the input video carrying data from the encoding unit, for example) and the signal of prediction 119 generated by the Inter/Intra-Prediction module 110. Additionally, the transform and quantize module 130, the entropy encoding circuit 140, the inverse transform and inverse quantize module 150 and the reconstruction circuit 160 are arranged to perform the transformation and quantization, entropy encoding, inverse transformation and inverse quantization, and reconstruction operations, respectively. As a result, reconstruction circuit 160 generates a temporarily reconstructed signal 169 which carries reconstructed results of the reconstruction operations. Additionally, the unlocking filter 170 is arranged to perform unlocking filtering on the reconstructed signal 169 in order to generate an unlocked signal 179 which carries unlocked data to be stored in the frame buffer 180 for the Inter-Prediction of the coding frames sequence and the Inter/Intra-Prediction module 110 has the ability to access the temporarily reconstructed signal 169 and the unlocked data from the earlier decoded frames loaded by a restored signal 189. This is for illustrative purposes only and should not be a limitation of the present invention. According to some variations of this modality, in a situation where the unlocking filter 170 and the unlocking filtering thereof can be omitted, the reconstructed results carried by the temporarily reconstructed signal 169 can be stored in the frame buffer 180 and the module of Inter/Intra-Prediction 110 has the ability to access the temporarily reconstructed signal 169 of the current encoding frame and the reconstructed results of the previous decoded frames via the restored signal 189.
Referring to Figure IB, the Multiple Hypothesis Inter-Prediction circuit 112 is arranged to perform the Multiple Hypothesis Inter-Prediction according to the original signal 109 and the reconstructed results of the previous decoded pictures through the restored signal 189 to generate a multi-hypothese Inter-Prediction output 113, the Inter-Prediction circuit 114 is arranged to perform the Inter-Prediction in accordance with the original signal 109 and the reconstructed results of the previous decoded pictures through the restored signal 189 to generate an output of Inter-Prediction 115 and the Intra-Prediction circuit 116 is arranged to perform the Intra-Prediction in accordance with the original signal 109 and the temporarily reconstructed signal 169 to generate an Intra-Prediction output 117, wherein the switching circuit 118 is arranged to dynamically select one among the multiple-hypothese Inter-Prediction output 113, the Inter-Prediction 115 output, and the Intra-Prediction output 117 as the prediction signal 119 mentioned above.
In practice, at least a portion (e.g., a portion or all) of the apparatus 100 can be deployed using the hardware circuitry. For example, apparatus 100 may be implemented with a processing circuit arranged to perform video encoding in the encoding unit, wherein the processing circuit may comprise a pre-processing module comprising the Inter/Intra-Prediction module 110 and may further comprise at least one encoding module comprising arithmetic unit 120, transform and quantize module 130, entropy encoding circuit 140, inverse transform and inverse quantize module 150, reconstruction circuit 160, and the unblocking filter 170. More particularly, one or more components of the apparatus 100 can be deployed with digital signal processing techniques. This is for illustrative purposes only and should not be a limitation of the present invention. Under some variations of this modality, at least a portion of the apparatus 100 can be deployed by software and/or firmware. For example, the processing circuit can be a processor that executes a plurality of program codes, wherein the processor that executes a first portion of the program codes can perform the same or similar operations as the pre-processing module mentioned above and the processor executing a second portion of the program codes can perform the same or similar operations as the coding module mentioned above.
It does not matter whether at least a portion (e.g., a portion or all) of apparatus 100 is deployed using hardware circuitry or using software realization, apparatus 100 is capable of performing video encoding in units of the encoding units. For example, encoding units can be macroblocks (MBs). In another example, the coding units may be coding units of a size between a predetermined larger coding unit (LCU) and a predetermined smaller coding unit (SCU). The gravure is first split into a plurality of LCUs and each of the LCUs is adaptively split into smaller encoding units until sheet encoding units are reached. According to this modality, the above-mentioned pre-processing module has the ability to divide the aforementioned encoding unit into a plurality of subcoding units (for example, divided by various types of partitions, such as square and non-square partitions. ) and has the ability to perform the prediction for each of the subcoding units. More particularly, the pre-processing module can perform Inter-Prediction, Intra-Prediction or multiple hypothesis motion compensation in each subcoding unit. When multiple hypothesis motion compensation is selected for a specific subcoding unit, the pre-processing module calculates a linear combination of a plurality of pixel values obtained according to a plurality of other coded units and uses the calculated result as a predicted pixel value of the specific subcoding unit. The plurality of other encoded units may be encoded subcoding units, encoded encoding units or a combination thereof. Additionally, the at least one coding module mentioned above is arranged to perform video coding in the coding unit based on the multi-case motion compensation performed by the pre-processing module.
Figure 1C illustrates exemplary multi-hypothesis motion compensation operations of the Inter/Intra-Prediction module 110 shown in Figure 1B in accordance with an embodiment of the present invention. In this modality, the notations F(to - 3), F(to - 2), F(to - 1) and F(to) are used to represent a portion of a plurality of subsequent frames {F(t)}, with frame F(t0) being the current frame. The current frame F(t0) may comprise a plurality of encoding units {CU(t0)} and a encoding unit CU(t0) may comprise a plurality of subcoding units {SubCU(t0)}. The pre-processing module mentioned above respectively obtains the motion information such as motion vectors {vk} from a plurality of other subcoding/coding units, which means that the pre-processing module uses the motion vectors of the plurality of other subcoding/coding units such as motion vectors {vk} for motion compensation of multiple assumptions of the specific subcoding unit. According to this embodiment, the pre-processing module can perform multi-case motion compensation in the specific subcoding unit such as the SubCU(t0) subcoding unit shown in Fig. 1C according to the plurality of motion vectors { vk}.
In general, the pre-processing module calculates a linear combination of a plurality of reference pixel values {y} obtained by the motion information of a plurality of other subcoding/coding units. Linear blend is a predicted pixel value yp of the specific subcoding unit. For example, the motion information of the first set of coded units comprises one or a combination of motion vector(s), frame of reference index(s) and prediction direction(s), and more particularly comprises some vectors of which are derived from a single subcoding unit/coding unit. In another example, the motion information can be motion vector(s), frame of reference index(s), prediction direction(s), or any combination of the above. Motion vectors are used as an example of motion information in the following modalities. Note that the linear combination can be a weighted sum of a plurality of reference pixel values, which means that the preprocessor module uses the weighted sum of a plurality of pixel values as the predicted pixel value of the subcoding unit specific. For example, in a situation where the specific predicted pixel having the predicted pixel value yp belongs to the i-th subcoding unit (for example, the subcoding unit SubCU(t0) shown in Figure 1C) and is located at the position x (for example, a vector indicating position, such as a two-dimensional vector in the image plane of the current frame F(t0)), the predicted pixel value yp can be rewritten as yp(i,x) and the value of predicted pixel yp (i, x) can be expressed as follows:
where the index k can vary within the set K, with the notification hk(i, x) representing a weighted parameter associated with the index k. For example, in a situation where the number of Possible Values of k is greater than one, the sum of the weighted parameters {hk(i, x)} might equal one for simplicity.
As shown in Figure 1C, the Motion Vectors {Vk} may comprise the Motion Vectors VA and VB of the subcoding units A and B of other encoding units within the current frame F(t0) and further comprise a Motion Vector VT of a subcoding unit T of a coding unit CU(to - 1) within another frame such as the previous frame F(to - 1). For example, in a situation where the encoding units are blocks, the encoding unit CU(to - 1) can be a block placed with respect to the encoding unit CU(t0). As a result, by applying the weighted parameters {hk(i, x)} to the reference pixel values {yr} obtained by the motion vectors {vk}, the pre-processing module can merge/mix the indicated partial images by motion vectors {vk} such as VA, VB and VT to generate a weighted sum image and use the weighted sum image as the predicted partial image of the specific subcoding unit such as the subcoding unit SubCU(t0) shown in Figure 1C. Refer to Figure 2 for additional details regarding the operations revealed above.
Figure 2 is a flowchart of a method 910 for performing localized multi-hypothesis prediction during video encoding of an encoding unit in accordance with an embodiment of the present invention. Method 910 can be applied to the apparatus 100 shown in Figure 1A and more particularly to the processing circuit mentioned above. The method is described as follows.
In Step 912, the above-mentioned pre-processing module divides the coding unit CU(t0) (e.g. the coding unit under consideration) into a plurality of subcoding units such as the subcoding units {SubCU(t0) } and performs the prediction on each of the subcoding units {SubCU(t0)}. In particular, the pre-processing module obtains the motion information, such as the motion vectors {vk} revealed above, for motion compensation of multiple assumptions from the specific subcoding unit SubCU(t0) from the subcoding units {SubCU (t0)} of a plurality of other coded units mentioned above (e.g. a plurality of other subcoding/coding units, which may comprise: other subcoding unit(s) of the coding unit CU(t0); subcoding unit(s) of at least one other coding unit; and/or other coding unit(s). According to an embodiment of the present invention, such as the one shown in Figure 3, the coding unit CU(t0) can be a block under processing (labeled as “Block processed” in Figure 3) and the specific subcoding unit SubCU( t0) may be a sub-block SB, wherein the shaded portions illustrated in Figure 3 may represent at least a portion of the coded units/coded blocks {CB}.
Referring to Figure 3, the coded blocks {CB} may comprise a left coded block CBL, an upper left coded block CBUL, an upper coded block CBU and an upper right coded block CBUR. For example, the motion vectors {vk} may comprise motion vectors of at least a portion (e.g. a portion or all) of the coded blocks {CB}, such as one or more of the coded blocks CBL, CBUL, CBU and CBUR.
In Step 914, the pre-processing module mentioned above performs the motion compensation of multiple hypotheses in the specific subcoding unit SubCU(t0) according to the plurality of motion vectors {vk}. In particular, the pre-processing module uses a linear combination of a plurality of pixel values from the plurality of other coded units mentioned in Step 912 (e.g., the plurality of other subcoding/coding units), such as the linear combination of the reference pixel values {yr} of the reference pixels within the plurality of other coded units mentioned above, as the predicted pixel value yp of the specific subcoding unit. For example, each encoding unit (for example, the encoding unit CU(t0)) may be a block, and more particularly a block comprising an array of pixels, such as an extended macroblock, a macroblock or a portion of a macroblock. In this way, a subcoding unit can be referred to as a subblock. According to an embodiment of the present invention, such as the one shown in Figure 3, the aforementioned pre-processing module can perform motion compensation of multiple assumptions in the block under processing (labeled "Block processed" in Figure 3) of according to the motion vectors {vk} revealed above, wherein the pre-processing module can merge/mix the reference pixel values {yr} of the reference pixels within at least a portion (eg a portion or all) of coded blocks {CB}, such as one or more of coded blocks CBL, CBUL, CBU and CBUR. In this mode, the coded blocks {CB} can be motion compensated blocks.
According to this embodiment, the pre-processing module is arranged to process each of the subcoding units mentioned in Step 912. For example, the pre-processing module obtains motion information from a first set of coded units for compensation It performs multi-case motion compensation of a first subcoding unit of the subcoding units mentioned in Step 912, such as the specific subcoding unit SubCU(t0), and performs multi-case motion compensation in the first subcoding unit such as the specific subcoding according to the motion information. More particularly, the pre-processing module uses a linear combination of a plurality of pixel values derived from the motion information of the first set of encoded units as a predicted pixel value of the first subcoding unit such as the specific subcoding unit. Additionally, the at least one coding module mentioned above is arranged to perform video coding in the coding unit based on the multi-case motion compensation performed by the pre-processing module. Additionally, the pre-processing module derives a predicted pixel value from another subcoding unit in the encoding unit, such as a second subcoding unit from the subcoding units mentioned in Step 912, using the movement information of a second set of coded units, wherein the second subcoding unit differs from the first subcoding unit.
With respect to the first subcoding unit, the first set of coded units may comprise: other subcoding unit(s) of the coding unit CU(t0) (i.e. subcoding unit(s) of the coding unit CU (t0) except the first subcoding unit); subcoding unit(s) of at least one other coding unit (i.e. subcoding unit(s) of at least one coding unit which differs from coding unit CU(t0)); and/or other encoding unit(s), such as one or more encoding units which differ from the encoding unit CU(t0). Additionally, in relation to the second subcoding unit, the second set of coded units may comprise: other subcoding unit(s) of the coding unit CU(t0) (i.e., subcoding unit(s) of the coding unit. CU(t0) encoding except the second subcoding unit); subcoding unit(s) of at least one other coding unit (i.e. subcoding unit(s) of at least one coding unit which differs from coding unit CU(t0)); and/or other encoding unit(s), such as one or more encoding units which differ from the encoding unit CU(t0). Note that the second set of coded units comprises at least one coded unit that is not in the first set of coded units. This is for illustrative purposes only and should not be a limitation of the present invention. According to some variations of this modality, the second set of coded units and the first set of coded units may be the same set of coded units. Meanwhile, the pre-processing module uses different linear combinations of a plurality of pixel values derived from the motion information of the same set of coded units as the respective predicted pixel values of the first subcoding unit and the second subcoding unit, respectively. . For example, the pre-processing module uses a first linear combination that matches a first set of parameters weighted to the plurality of pixel values as the predicted pixel value of the first subcoding unit and uses a second linear combination matches a second set of parameters weighted for the plurality of pixel values as the predicted pixel value of the second subcoding unit. In another example, the pre-processing module uses a first weighted sum of a plurality of pixel values derived from the motion information of the first set of encoded units as the predicted pixel value of the first subcoding unit such as the subcoding unit and derives the predicted pixel value of the second subcoding unit using a second weighted sum of the pixel values derived from the motion information of the same set of coded units (i.e., the first set of coded units), where the first weighted sum is different from the second weighted sum.
In some embodiments such as variations of the embodiment shown in Figure 3, each of the other encoded units mentioned in Step 912 is an encoded/sub-encoded unit. For example, in a portion of these variations, the plurality of other coded units mentioned in Step 912 comprise at least one spatially coded encoding/subcoding unit (e.g., one or more of the coded blocks {CB} in the current frame F(t0) or one or more coded subblocks within the block under processing or within the coded block) and/or at least one temporarily coded coding/subcoding unit (e.g. one or more coded subblocks/blocks in another frame that differs from the current frame F(t0)). In another example, in a portion of these variations, each of the other encoded units mentioned in Step 912 is a motion compensated encoding/subcoding unit. According to some of these variations, the motion vectors {vk} can be obtained from the motion estimate.
Figure 4 illustrates some deployment details involved with the method 910 shown in Figure 2 in accordance with an embodiment of the present invention. According to this embodiment, the pre-processing module may use a weighted sum of a plurality of pixel values mentioned in Step 914 (e.g., ∑keK(hk(i, x) Tr(x + vk))) in that mode, such as the predicted pixel value yp of the specific subcoding unit such as the ith subblock bi shown in Figure 4. For better understanding, the coded blocks CBL, CBUL, CBU and CBUR can be the same as those shown in Figure 3, respectively. For example, the reference pixel values {yr} of the reference pixels to calculate the predicted pixel value yp can be obtained from one or more of the coded blocks CBL, CBUL, CBU and CBUR and the plurality of motion vectors {vk} may comprise one or more of the respective motion vectors and coded blocks CBL, CBUL, CBU and CBUR
More particularly, for each sub-block such as sub-block bi shown in Figure 4, a predicted pixel value such as the predicted pixel value yp(i, x) can be derived from the blending/mixing of the pixel values of reference {yr} of some reference pixels within the coded blocks {CB} and can further be expressed as follows:
where the index k can vary within the set K and the notations vk and hk(i, x) can be used to represent the kth Reference Motion Vector and the associated weighted parameter, respectively. For example, bi and Bm and the notation Bm represent the set of subblocks of the block under processing (labeled “Block processed” in Figure 4). As shown in Figure 4, an exemplary motion vector Vk of the motion vectors {Vk} is illustrated within the coded block CBU which means that the motion vectors {Vk} comprise the motion vector VU of the coded block CBU in this situation.
In an embodiment, such as a Variation of the embodiment shown in Figure 4, the preprocessor module can perform optimal Wiener filtering on a plurality of pixel values such as reference pixel values {yr} with respect to a VREAL actual pixel value (i, x) of the current pixel under consideration, in order to adjust the respective weighted parameters {hk (i, x)} to the plurality of Pixel Values such as the reference pixel values {vr} and to generate the weighted sum (ie, ∑keK(hk(i, x) vIJr(x + vk))) on that variation. For example, the weighted parameters {hk(i, x)} can be obtained using an optimal Wiener filter within the pre-processing module as follows:
which means that the weighted parameters {hk(i, x)} can be obtained by searching for a set of (h0*, ..., hK*) that corresponds to a minimum value of |(¥REAL(Í, x) )) -∑keK(hk(i, x) 'Pr( + Vk))| during the optimal Wiener filtration performed by the pre-processing module. This is for illustrative purposes only and should not be a limitation of the present invention. According to a variation of the embodiment shown in Figure 4, the pre-processing module can determine respective weighted parameters {hk(i, x)} for the plurality of pixel values according to the contents of at least one unit of neighbor coding, such as the contents of one or more of the coded blocks CBL, CBUL, CBU and CBUR, in order to generate the weighted sum (i.e., ∑keK(hk(i, x) Tr(x + vk)) in that variation). According to another variation of the embodiment shown in Figure 4, the pre-processing module can determine respective weighted parameters {hk(i, x)} for the plurality of pixel values in accordance with the contents of the plurality of other units of coding mentioned the plurality of, (eg, the first set of coded units), in order to generate the weighted sum (ie, ∑keK(hk(i,x) Tr(x + vk)) in that variation).
In some other modalities, such as some variations of the modality shown in Figure 4, the preprocessor module can determine the respective weighted parameters {hk(i,x)} for the plurality of pixel values by off-line training or by online training in order to generate the weighted sum (ie, ∑keK(hk(i, x) vIJr(x + vk)) in these variations). According to another variation of the modality shown in Figure 4, the pre-processing module may use an average of the plurality of pixel values as the predicted pixel value of the specific subcoding unit, which means that any two of the weighted parameters { hk(i, x)} are equal to each other. More particularly, in this variation, each of the weighted parameters {hk(i, x)} is equal to 1/n(K), where n(K) represents the number of possible values of k within the set K.
Figures 5A to 5D illustrate some implementation details of the prediction of multiple hypotheses that are involved with the method 910 shown in Figure 2 according to different embodiments of the present invention, in which the linear combination mentioned in Step 914 can be considered as the sum (ie, ∑keK(hk(i, x) Tr(x + vk)) in these modalities). For better understanding, the coded blocks CBL, CBUL, CBU and CBUR can be the same as those shown in Figure 3, respectively. Additionally, the ith sub-block bi is illustrated as an example of the specific subcoding unit SubCU(t0).
Referring to Figure 5A, subcoding units A, B, C and D belong to the block under processing (labeled as “Block processed” in Figure 5A) or other coded units (such as CBUL, CBU, CBUR, CBL,) . More particularly, with respect to the ith sub-block bi shown in Figure 5A, subcoding unit A is the left adjacent subcoding unit, subcoding unit B is the upper adjacent subcoding unit, subcoding unit C is the upper right adjacent subcoding unit and the subcoding unit D is the upper left adjacent subcoding unit. In this modality, the motion vectors {vk} can be obtained from the subcoding units A, B, C, and/or D shown in Figure 5A and therefore can include the respective motion vectors VA, VB, VC, and/or DV thereof. For example, motion vectors {vk} may include all motion vectors VA, VB, VC, and VD of subcoding units A, B, C, and D. In another example, motion vectors {vk} may include the motion vectors VA, VB and VC if the subcoding unit C to the specific subcoding unit such as the ith subblock bi exists. In another example, the motion vectors {vk} may include the motion vectors VA, VB and VD if the subcoding unit C for the specific subcoding unit such as the ith subblock bi does not exist.
Thus, the linear combination mentioned in Step 914 can be the weighted sum revealed above ∑keK(hk(i, x) Tr(x + vk)) with the motion vectors {vk} including the motion vectors VA, VB, VC , and/or VD. For brevity, the predicted pixel value yp can be expressed as follows:
where the Weighted_Sum notation represents the weighted sum in the situation revealed above.
According to a variation of the modality shown in Figure 5, the pre-processing module uses an average of the plurality of pixel values as the predicted pixel value of the specific subcoding unit, which means that the linear combination mentioned in Step 914 can be considered as the average. In this situation, any two weighted parameters {hk(i, x)} are equal to each other. For brevity, the predicted pixel value yp can be expressed as follows:
where the Average notation represents the average in this situation. Similar descriptions are not repeated in detail for this variation.
Referring to Figure 5B, the subcoding units α1, α2, α3, α4, α5, α6, α7 and α8 belong to the left coded block CBL, the subcoding units β1, β2, β3, β4, β5, α6, β7 and β8 belong to the upper coded block CBU and the subcoding units β9 and δ belong to the upper right coded block CBUR and the upper left coded block CBUL, respectively. More particularly, with respect to the i-th subblock bi shown in Figure 5B (for example, a subblock within the upper right quadrant of the block under processing in this mode), motion vectors {vk} can be obtained from of the A, B, C, and/or D subcoding units of that modality, wherein the subcoding unit A can be defined as the nearest subcoding unit within the left coded block CBL, the subcoding unit B can be defined as the nearest subcoding unit within the upper coded block CBU, subcoding unit C can be defined as the right adjacent subcoding unit of subcoding unit B, and subcoding unit D can be defined as the left adjacent subcoding unit of subcoding unit B. In the situation shown in Figure 5B, the subcoding units A, B, C and D are the subcoding units α1, α5, α6, α7 and α5, respectively.
In this modality, the motion vectors {vk} can be obtained from the subcoding units A, B, C, and/or D shown in Figure 5A and therefore can include the respective motion vectors VA, VB, VC, and/or DV thereof. For example, motion vectors {vk} may include all motion vectors VA, VB, VC, and VD of subcoding units A, B, C, and D. In another example, motion vectors {vk} may include those motion vectors VA, VB and VC if the subcoding unit C to the specific subcoding unit such as the ith subblock bi exists. In another example, the motion vectors {vk} may include the motion vectors VA, VB and VD if the subcoding unit C for the specific subcoding unit such as the ith subblock bi does not exist.
Thus, the linear combination mentioned in Step 914 can be the weighted sum revealed above ∑keK(hk(i, x) Tr(x + vk)) with the motion vectors {vk} including the motion vectors VA, VB, VC , and/or VD. For brevity, the predicted pixel value yp can be expressed as follows:
where the Weighted_Sum notation represents the weighted sum in the situation revealed above. For example, the pre-processing module can determine the size of the specific subcoding unit to be a transform size in relation to the video encoding of the encoding unit and the size of the subcoding units such as the subblocks shown in Figure 5B can be the same as the transform size (for example, 4 by 4, 8 by 8, 16 by 16, and so on).
According to a variation of the modality shown in Figure 5B, the pre-processing module uses an average of the plurality of pixel values as the predicted pixel value of the specific subcoding unit, which means that the operation to obtain said linear combination in Step 914 it can be considered as an average operation. In this situation, any two weighted parameters {hk(i, x)} are equal to each other. For brevity, the predicted pixel value yp can be expressed as follows:
where the Average notation represents the average in this situation. Similar descriptions are not repeated in detail for this variation.
Referring to Figure 5C, the definitions of subcoding units {α1, α2, α3, α4, α5, α6, α7, α8, β1, β2, β3, β4, β5, β6, β7, β8, β9 and δ} are the same as those of the modality shown in Figure 5B. Additionally, in relation to the i-th subblock bi shown in Figure 5B (for example, a subblock within the upper right quadrant of the block under processing in this mode), the motion vectors {vk} can be obtained from the subcoding units A, B, C, D, E, F and/or Z of that modality, where the definitions of subcoding units A, B, C and D are the same as those of the modality shown in Figure 5B, the unit subcoding unit E can be defined as the upper adjacent subcoding unit of subcoding unit A, subcoding unit F can be defined as the lower adjacent subcoding unit of subcoding unit A, and subcoding unit Z can be defined as a nearest subcoding unit within the upper left coded block CBUL. In the situation shown in Figure 5C, the subcoding units A, B, C, D, E, F, and Z are the subcoding units α4, β6, β7, β5, α3, α5 and δ, respectively.
In this modality, the motion vectors {vk} can be obtained from the subcoding units, B, C, D, E, F and/or Z of that modality and, therefore, can include the respective motion vectors VA, VB, VC, VD, VE, VF and/or Vz thereof. For example, motion vectors {vk} can include all motion vectors VA, VB, VC, VD, VE, VF, and Vz of subcoding units A, B, C, D, E, F, and Z. For example, motion vectors {vk} may include motion vectors VA, VB, VC, VD, VE and VF if subcoding unit E for specific subcoding unit such as ith subblock bi exists. In another example, the motion vectors {vk} may include the motion vectors VA, VB, VC, VD, VE, VF and Vz if the E subcoding unit for the specific subcoding unit such as the ith sub- bi block does not exist. In another example, the motion vectors {vk} can include the motion vectors VA, VB, VD, VE, VF and Vz if the subcoding units C and E for the specific subcoding unit such as the ith sub- bi block do not exist.
Thus, the linear combination mentioned in Step 914 can be the weighted sum revealed above ∑keK(hk(i, x) Tr(x + vk)) with the motion vectors {vk} including the motion vectors VA, VB, VC , VD, VE, VF and/or Vz. For brevity, the predicted pixel value yp can be expressed as follows:
where the Weighted_Sum notation represents the weighted sum in the situation described above. For example, the pre-processing module can determine the size of the specific subcoding unit to be a transform size with respect to the video coding of the coding unit and the size of the subcoding units such as the subblocks shown in the figure. 5e can be the same as the transform size (for example, 4 by 4, 8 by 8, 16 by 16, and so on).
According to a variation of the modality shown in figure 5e, the pre-processing module uses an average of the plurality of pixel values as the predicted pixel value of the specific subcoding unit, which means that the operation of obtaining the linear combination mentioned in Step 914 can be related as an averaging operation. In this situation, any two of the weighted parameters {hk(i, x)} are equal to each other. For brevity, the predicted pixel value 0p can be expressed as follows:
where the Average notation represents the average in this situation. Similar descriptions are not repeated in detail for this variation.
Referring to Figure 5D, the definitions of subcoding units {α1, α2, α3, α4, α5, α6, α7, α8, β1, β2, β3, β4, β5, β6, β7, β8, β9, δ} are the same as those for the modality shown in figure 5B. Additionally, with respect to the i-th sub-block bi shown in figure 5D (for example, a sub-block within the right fourth and bottom of the block under processing in this mode), the motion vectors {vk} can be obtained from of the subcoding units A, B, and and/or D of that modality, wherein the definitions of the subcoding units A, B, e and D are the same as those of the modality shown in Figure 5B. In the situation shown in Figure 5e, the subcoding units A, B, C and D are the subcoding units α5, β6, β7 and β5, respectively.
In this modality, the motion vectors {vk} can be obtained from the subcoding units A, B and, and/or D of that modality and, therefore, can include the respective motion vectors VA, VB, Ve, and/ or DV of them. For example, motion vectors {vk} may include all motion vectors VA, VB, Ve, and/or VD of subcoding units A, B, e and D. In another example, motion vectors {vk} may include the motion vectors VA, VB and Ve if the subcoding unit e for the specific subcoding unit such as the ith subblock bi exists. In another example, the motion vectors {vk} may include the motion vectors VA, VB and VD of the subcoding unit and for the specific subcoding unit such that the ith subblock bi does not exist.
Additionally, a weighted parameter hk(i, x) of this modality may be inversely proportional to a distance between the specific subcoding unit and that associated with the other coded units mentioned in Step 912. For example, the weighted parameter hk(i, x) can be rewritten as wk(i) which does not depend on x position and the weighted parameter wk(i) is inversely proportional to (dk(i))m (ie wk(i) 1/(dk(i))m), where the notation dk(i) represents the distance between the specific subcoding unit (eg the ith subblock bi) and the other associated subcoding/coding unit having kth reference motion vector vk (for example, the kth subcoding unit within subcoding units A, B, C and D) and the notation m represents a positive constant. Thus, the predicted pixel value Tp can be expressed as follows:
wherein the Motion Vectors {Vk} may include the VA, VB, VC and/or VD Motion Vectors. In the situation shown in Figure 5D, the motion vectors {Vk} include the movement vectors VA, VB and VC and the notations dist_w, dist_h and dist_h' represent the distances dA(i), dB(i) and dC(i) , respectively. As a result, the predicted pixel value Tp can be expressed as follows:
where WA°<= 1(dist_w)m, WB"-1(dist_h)m and w^c 1(dist_h')m
In some Variations of the modalities shown in 5A to 5D, the preprocessor modules can triage a set of Motion Vectors by discard from at least one Extreme Motion Vector in the motion information of multiple coded units and more particularly by discard of at least one extreme motion vector from the set of motion vectors in order to select a remaining portion of the set of motion vectors as the motion information to derive the predicted pixel value (for example, the motion information of the first set of coded units). For example, the aforementioned at least one extreme motion vector may comprise one or more motion vectors that are very different from the others in the set of motion vectors, with their lengths and/or directions being considered.
In accordance with some variations of the modalities shown in Figures 5A to 5D, the pre-processing module may selectively operate in any mode of a plurality of modes comprising Modes 1, 2, 3 and 4, wherein the respective Modes operations 1, 2, 3 and 4 may be the same or similar to those disclosed in the embodiments shown in Figures 5A to 5D, respectively. For example, in Mode 1, the preprocessing module may operate according to the same method as the modalities shown in Figure 5A, while in another mode such as any of Modes 2, 3 and 4, the preprocessing module can operate in accordance with the same method of the associated modality as the corresponding modality within that shown in Figures 5B to 5D, respectively. Additionally, Modes 1, 2, and 3 mentioned above allow apparatus 100 to process with ease compared to some other mode(s) and are useful in increasing the overall processing speed of apparatus 100 and, therefore, they can be referred to as simplified modes.
According to a variation of any of the embodiments shown in Figures 5A to 5D, in Step 912, the pre-processing module may divide the encoding unit into the plurality of subcoding units based on at least one mode of at least one close coding unit, such as Modes 1, 2, 3 and/or 4 of one or more of the coded blocks CBL, CBu, CBUR and CBUL. According to another variation of any of the embodiments shown in Figures 5A to 5D, at Step 912, the pre-processing module may divide the encoding unit into the plurality of subcoding units based on the contents of at least one encoding unit such as one or more of the coded blocks CBL, CBu, CBUR and CBUL.
Figures 6A to 6B illustrate exemplary sources for obtaining motion information that are involved with the method 910 shown in Figure 2 in accordance with different embodiments of the present invention. For better understanding, the coded blocks CBL, CBUL, CBu and CBUR can be the same as those shown in Figure 3, respectively.
In this modality, in a situation where the size of the specific subcoding unit SubCU(t0) is extended up to the size of the encoding unit CU(t0), the division/partition of the encoding unit CU(t0) is not needed in Step 912. Thus, in Step 912, the pre-processing module mentioned above can perform motion vector prediction of the coding unit CU(t0), and more particularly, obtain a plurality of motion vectors, such as the motion vectors. move {vk} disclosed above, for compensating for multi-chance moves of the coding unit CU(t0) from a plurality of other coded units such as at least a portion of the coded blocks {CB} (e.g. one or more of the CBL, CBUL, CBu and CBUR coded blocks). Additionally, in Step 914, the pre-processing module can perform multi-case motion compensation in the coding unit CU(t0) according to motion information such as the plurality of motion vectors {vk}, and more particularly , using a linear combination of a plurality of pixel values from the plurality of other encoding units as a predicted pixel value of the encoding unit, such as the predicted pixel value Tp(x) having no need to use index i . Similarly, the predicted pixel value Tp(x) can be expressed as follows:

In practice, the plurality of modes may further comprise some jump modes corresponding respectively to different motion block sizes (e.g. 16 by 16, 32 by 32, etc.). According to this embodiment, A notation can be used to represent a subcoding/coding unit comprising at least a portion (e.g., a portion or all) of the CBL coded block, D notation can be used to represent a unit of subcoding/coding which comprises at least a portion (example, a portion or all) of the CBUL encoded block, the notation B can be used to represent a subcoding/coding unit that comprises at least a portion (example, a portion or all ) of the CBU coded block and the SIZE notation such as C16, C32, etc. can be used to represent a subcoding/coding unit comprising at least a portion (example, a portion or all) of the coded block CBUR, where the suffix SIZE of the notation CSIZE represents the size of the coding unit CU(t0) such as the block under processing (identified as "block processed" in Figures 6A to 6B). For example, in a situation where the block under processing includes (32 * 32) pixels, the SIZE subcoding/coding unit can be the C32 subcoding/coding unit shown in Figure 6A. In another example, in a situation where the block under processing includes (16 * 16) pixels, the SIZE subcoding/coding unit can be the C16 subcoding/coding unit shown in Fig. 6B. Thus, the location of the SIZE subcoding/coding unit is independent of the motion block size.
According to this modality, the motion vectors {vk} can include all the motion vectors VA, VB, VC and VD of the subcoding unit/coding units A, B, SIZE and D. Thus, the linear combination of this modality can be the weighted sum ∑keK(hk(x) Tr(x + vk)) with the motion vectors {vk} including the motion vectors VA, VB, VC, and VD. For brevity, the predicted pixel value Tp can be expressed as follows:
where the Weighted_Sum notation represents the weighted sum in the situation revealed above.
According to a variation of the embodiment shown in Figures 6A-6B, the pre-processing module uses an average of the plurality of pixel values from the plurality of other encoding units as the predicted pixel value of the encoding unit, which means that the linear combination mentioned above can be related to the mean. In this situation, any two of the weighted parameters {hk(x)} are equal to each other. For brevity, the predicted pixel value Tp can be expressed as follows:
where the Average notation represents the average in this situation. Similar descriptions are not repeated in detail for this variation.
Figures 7A to 7B respectively illustrate temporal and spatial motion vectors (MVs) involved in method 910 shown in Figure 2 in accordance with different embodiments of the present invention. For better understanding, the coded blocks CBL, CBUL, CBU and CBUR can be the same as those shown in Figure 3, respectively. Additionally, the ith sub-block bi is illustrated as an example of the specific subcoding unit SubCU(t0).
In the embodiment shown in Figure 7A, the motion vectors {vk} mentioned in Step 912 may comprise one or more temporal motion vectors {vT,k} with respect to a window that closes the specific subcoding unit SubCU(t0) such as the ith sub-block bi shown in Figure 7A, where the window can have a predetermined size. In the embodiment shown in Figure 7B, the motion vectors {vk} mentioned in Step 912 may comprise one or more spatial motion vectors {vS,k}, such as that of the coded block CBU shown in Figure 7B. According to some variations of the modalities shown in Figures 7A to 7B, the motion vectors {vk} mentioned in Step 912 may comprise the temporal motion vectors {vT, k} and the spatial motion vectors {vS, k}. Similar descriptions are not repeated in detail for these modalities/variations.
Figures 8A to 8C illustrate exemplary sources for obtaining motion vectors that are involved with the method 910 shown in Figure 2 in accordance with different embodiments of the present invention. For better understanding, the coded blocks CBL, CBUL, CBU and CBUR can be the same as those shown in Figure 3, respectively. Additionally, the ith sub-block bi is illustrated as an example of the specific subcoding unit SubCU(t0).
Referring to Figure 8A, the temporal motion vectors {vT, k} disclosed above may comprise a temporal motion vector vT0 which is typically obtained from a colocalized subcoding unit of another frame, such as a T subblock colocalized with respect to the i-th sub-block bi shown in Figure 8A. Thus, the temporal motion vector vT0 can be related as a colocalized motion vector. Referring to Figure 8B, the spatial motion vectors {vS, k} disclosed above may comprise at least a portion (e.g., a portion or all) of the motion vectors vA, vB, vC, vD, vE, vF and vZ of subcoding units A, B, C, D, E, F, and Z.
Thus, the linear combination mentioned in Step 914 can be the weighted sum revealed above ∑keK(hk(i, x) Tr(x + vk)) with the motion vectors {vk} that comprise both the temporal motion vector vT0 and the motion vectors vA, vB, vC, vD, vE, vF and/or vZ. For example, in a situation where the motion vectors {vk} comprise the temporal motion vector vT0 and the motion vectors VA, VB, vc, VD, VE, VF and vz, the predicted pixel value Tp can be expressed as follows:
where the Weighted_Sum notation represents the weighted sum in the situation revealed above. Similar descriptions are not repeated in detail for this modality.
According to a variation of the modality shown in Figures 8A to 8B, the pre-processing module uses an average of the plurality of pixel values as the predicted pixel value of the specific encoding unit, which means that the linear combination mentioned in Step 914 can be related as the average. In this situation, any two of the weighted parameters {hk(i, x)} are equal to each other. For brevity, the predicted pixel value Tp can be expressed as follows:
where the Average notation represents the average in this situation. Similar descriptions are not repeated in detail for this variation.
Referring to Figure 8c, the temporal motion vectors {vT, k} revealed above may comprise the temporal motion vector vT0 revealed above and some other temporal motion vectors vTUL, vTU, vTUR, vTL, vTR, vTDL and vTDR that are obtained from closely colocated subcoding units of another frame, such as those adjacent to the T subblock that has the temporal motion vector vT0, that is, the TUL, TU, TUR, TL,TR, TDL subblocks , TD and TDR shown in Figure 8c, respectively. Additionally, the spatial motion vectors {vS, k} disclosed above may comprise at least a portion (e.g., a portion or all) of the motion vectors vA, vB, vC, vD, vE, vF and vZ of the subcoding units A, B, C, D, E, F and Z, shown in Figure 8B.
Thus, the linear combination mentioned in Step 914 can be the weighted sum revealed above ∑keK(hk(i, x) Tr(x + vk)) with the motion vectors {vk} that comprise both the temporal motion vectors vr0, vTUL, vTU, vTUR, vTL, vTR, vTDL, vTD and vTDR as the motion vectors vA, vB, vC, vD, vE, vF and/or vZ. For example, in a situation where the motion vectors {vk} comprise the temporal motion vectors vr0, vTUL, vTU, vTUR, vTL, vTR, vTDL, vTD and vTDR and the motion vectors VA, VB, vc, VD, VE, VF and vz, the predicted pixel value Tp can be expressed as follows:
where the Weighted_Sum notation represents the weighted sum in the situation revealed above. Similar descriptions are not repeated in detail for this modality.
According to a variation of the modality shown in Figure 8c, the pre-processing module uses an average of the plurality of pixel values as the predicted pixel value of the specific encoding unit, which means that the linear combination mentioned in Step 914 can be related as the mean. In this situation, any two of the weighted parameters {hk(i, x)} are equal to each other. For brevity, the predicted pixel value Tp can be expressed as follows:
where the Average notation represents the average in this situation. Similar descriptions are not repeated in detail for this variation.
According to an embodiment, such as a variation of any of the modalities/variations disclosed above, the pre-processing module can dynamically select a set of motion vectors from a plurality of sets of motion vectors (which may be listed as candidate motion vectors) and use the selected set of motion vectors as the plurality of motion vectors {vk}. More particularly, the pre-processing module dynamically selects the set of motion vectors mentioned above from the plurality of sets of motion vectors and uses the selected set of motion vectors as the motion information from the plurality of other coded units mentioned. in Step 912 (for example, the first set of coded units) based on an explicit signaling and uses the selected set of motion vectors to derive the predicted pixel value of the specific subcoding unit. For example, the plurality of motion vector sets may comprise a first set of motion vectors such as the temporal motion vectors {vr, k} disclosed above and further comprise a second set of motion vectors such as the motion vectors {vs, k} described above. In practice, the pre-processing module can perform multiple-hypothesis prediction based on a cue (eg, explicit cue) that dynamically indicates the set that is selected. Similar descriptions are not repeated in detail for this modality.
According to another embodiment, such as a variation that any of the embodiments/variations disclosed above, the pre-processing module can obtain at least one motion vector from the coding unit CU(t0) by performing motion estimation, to use of performance rate distortion optimization in order to obtain the predicted pixel value of the specific subcoding unit. For example, in Step 912, the pre-processing module obtains at least a portion of the plurality of motion vectors {vk} for compensating for multiple hypothesis motions of the specific subcoding unit SubCU(t0) by performing motion estimation. Additionally, in Step 914, the pre-processing module may further use a linear combination of a plurality of pixel values {Tr} of subcoding/coding units associated with the plurality of motion vectors {vk} as the predicted pixel value Tp of the SubCU(t0) specific subcoding unit. For example, in a situation where the specific predicted pixel that has the predicted pixel value Tp is located at the x position (for example, a vector that indicates the position, such as a two-dimensional vector in the image plane of the current frame F( t0)), the predicted pixel value Tp can be rewritten as Tp(x) and the predicted pixel value Tp(x) can be expressed as follows:
where the index k can vary within the set K, with the notation hk(x) representing the weighted parameter associated with the index k and the estimated motion vector vo is the motion vector to be estimated during motion estimation, with the notation ho (x) representing the weighted parameter associated with the suffix o of the estimated motion vector vo.
During the rate skew optimization mentioned above, the pre-processing module can calculate an IDIST skew indicator, where the skew indicator typically represents the difference between the original partial image of the SubCU(to) specific subcoding unit and the reconstructed partial image of the specific subcoding unit SubCU(to) after prediction of multiple hypotheses with respect to a possible state (eg length and angle) of the estimated motion vector vo. For example, the IDIST distortion indicator and the associated motion vector difference MVD can be expressed as follows:
where TREAL(x) notation represents the actual pixel value of the current pixel under consideration and MVP notation represents the motion vector predictor. More particularly, within the above equations, the upper one for the IDIST distortion indicator can be rearranged as follows:
where rREAL(x) = TREAL(X) - ∑keK(hk(x) Tr(x + Vk)))/ho(x). In the above equation, the term rREAL(x) and the term (TREAL(X)) - ∑keK(hk(x) Tr(x + Vk))) are independent of the estimated motion vector vo and, therefore, at least one of these terms, such as the term rREAL(x) and/or the term (TREAL(X)) - ∑keK(hk(x) Tr(x + vk))), can be precalculated and temporarily stored to be accessed in order to increase the processing speed of the apparatus 100 in accordance with this modality.
According to this modality, the pre-processing module of that modality can optimize the estimated motion vector v0 by finding the best state that corresponds to the minimum Lagrange function of the IDIST distortion indicator and the bits used to encode the vector difference of MVD motion, among other possible states of the estimated motion vector v0. Thus, the linear combination of this modality can be the weighted sum ∑keK (hk(x) Tr( + vk)) + ho(x) Tr( + vo) with the estimated motion vector v0 was obtained and the modulus of preprocessing can use the weighted sum ∑keK (hk(x) Tr(x + vk)) + ho(x) Tr(x + vo) as the predicted pixel value Tp of the specific subcoding unit SubCU(t0). Similar descriptions are not repeated in detail for this modality.
Figure 9 illustrates some deployment details with the method 91o shown in Figure 2 according to an embodiment of the present invention, in which the pre-processing module can perform prediction of multiple variable partition hypotheses, and more particularly, it can adaptively determine the partition of the encoding unit CU(to) in response to the partition of at least one nearby encoding unit. For better understanding, the coded blocks CBL, CBUL, CBU and CBUR can be the same as those shown in Figure 3, respectively.
According to this embodiment, in Step 912, the pre-processing module may divide the coding unit CU(to) such as the block under processing (indicated by "processed block" in Fig. 9) into the plurality of subcoding units { SubCU(to)} based on the partition of at least one nearby encoding unit (for example, one or more of the encoded blocks CBL, CBUL, CBU and CBUR). For example, the plurality of subcoding units {SubCU(to)} may comprise subblocks b11, b12, b13, b21, b22, b23, b31, b32, b33, b41, b42 and b43, whose sizes correspond to the partition of the close coded blocks such as the CBL and CBU coded blocks, and more particularly, the associated sizes of some sub-blocks of these close coded blocks. In the situation shown in Figure 9, the sub-blocks of these closely encoded blocks are those adjacent to the block under processing, in which the sub-blocks are shared and illustrated with different patterns, respectively. Similar descriptions are not repeated in detail for this modality.
Figure 10 illustrates some deployment details involved with the method 910 shown in Figure 2 in accordance with another embodiment of the present invention, wherein the pre-processing module can adaptively determine the blend weights (e.g., weighted parameters {hk(i,x)} revealed above) according to the contents of at least one nearby encoding unit. For better understanding, the coded blocks CBL, CBUL, CBU and CBUR can be the same as those shown in Figure 3, respectively. Additionally, subblock SB is illustrated as an example of specific subcoding unit SubCU(t0).
For example, the motion vectors {vk} can comprise the motion vectors vA, vB and vC of the subcoding units A, B and C, where the subcoding unit A belongs to the left coded block CBL and the subcoding units B and C belong to the upper coded block CBU. During the generation processing of the weighted sum ∑keK (hk(i, x) Tr(x + vk) (for example, the weighted sum Weighted_Sum(Tr(vA), ^r(vβ), ^r(vc)) in situation shown in Figure 10), where subcoding units B and C are texture subcoding units (for example texture subblocks) and subcoding unit A is a non-texture subcoding unit (for example, a non-texture sub-block), the pre-processing module can adaptively determine that the weighted parameters hB(i, x) and hC(i, x) are associated with the motion vectors vB and vC of the subcoding units B and C are greater than the weighted parameter hA(i, x) associated with the motion vector vA of the subcoding unit A, respectively. Similar descriptions are not repeated in detail for this modality.
Figure 11 illustrates some deployment details involved with the method 910 shown in Figure 2 in accordance with an embodiment of the present invention. According to this embodiment, the pre-processing module further divides the coding unit CU(t0) (e.g. the coding unit under consideration) into the plurality of subcoding units such as the subcoding units {SubCU(t0) } and performs the prediction on each of the subcoding units {SubCU(t0)} in Step 912. However, at the time/before getting the motion information such as the motion vectors {vk} revealed above, the module of pre-processing can designate a motion vector (e.g. a reference motion vector) for each subcoding unit so this can be referred to later by other subcoding unit(s)/coding unit(s). The rule for motion vector designation can be arbitrary as it needs to be adjusted and adhered to by both encoders and decoders. Some default rules for motion vector designation include motion vector predictor defined in H.264, motion vector prediction defined in H.264 but with motion vector scaling, left block fill, top block fill, and filling the left or top block. According to a left or upper block filling modality, the preprocessing module designates a motion vector which uses the motion vector of a subcoding unit of another encoding unit (except for the encoding unit CU(t0) ) under consideration), such as an encoded subcoding unit of an adjacent encoding unit CUADJ(t0) which is adjacent to the encoding unit CU(t0). More particularly, with respect to the specific subcoding unit SubCU(t0), the subcoding unit of the other encoding unit (for example, the encoding unit CUADJ(t0)) is the closest subcoding unit within the other encoding unit and the pre-processing module can use the motion vector of the closest subcoding unit within the other encoding unit (for example, the encoding unit CUADJ(t0)) as the motion vector assigned to the specific subcoding unit SubCU(t0). The motion vector assigned to an encoding unit or a subcoding unit is mainly used as a reference motion vector for other encoding units or subcoding units, where it need not be used for motion prediction.
As shown in Figure 11, with respect to the coding unit CU(t0) such as the block under processing (indicated by "processed block" in Figure 11), the adjacent encoding units (CUADJ(t0)) that are involved in this modality may comprise at least a portion of the left coded block CBL, the upper coded block CBU, the upper right coded block CBUR and the upper left coded block CBUL, wherein the subcoding units in these coded blocks CBL, CBU, CBUR and CBUL may be related as sub-blocks. For better understanding, the motion vectors y1, y2, y3, y4, y5, y6, y7 and y8 of the rightmost column of subcoding units within the left coded block CBL are indicated in the rightmost column of subcoding units, respectively. Similarly, the motion vectors x1, x2, x3, x4, X5, X6, X7 and x8 of the lowermost row of subcoding units within the upper coded block CBU are indicated in the lowermost row of subcoding units, respectively. Additionally, the motion vector d of the lower right subcoding unit within the upper left encoded block CBUL and the motion vector x9 of the lower left subcoding unit within the upper right encoded block CBUR are indicated thereon, respectively.
According to this embodiment, the pre-processing module receives the coding unit CU(t0) which comprises a plurality of subcoding units {SubCU(t0)} and analyzes the data from the coding unit CU(t0) and designates a reference motion vector of the specific subcoding unit SubCU(t0) of the subcoding units {SubCU(t0)} according to a predetermined rule, wherein the reference motion vector is used for reference by at least one other unit of subcoding and is not used for motion vector prediction of specific subcoding unit SubCU(t0). With the reference motion vectors being used for reference, the pre-processing module can operate quickly, having no need to wait for completion of any complicated calculations with respect to the actual motion vector of the specific subcoding unit SubCU(t0). In practice, the pre-processing module can designate this motion vector (i.e. the reference motion vector mentioned above) to be a motion vector of at least a portion of an adjacent encoding unit. More particularly, the at least one portion of the adjacent encoding unit may be a portion that is closest to the specific subcoding unit within the proximal encoding unit.
For example, suppose the specific subcoding unit SubCU(t0) represents the jth subblock of a certain row of subblocks in the block under processing (for example, the processed block shown in Figure 11), in that j can range from 1 to 8. In a first designation rule such as "vertical designation", the preprocessor module can use the motion vector xj of the nearest subcoding unit within the upper coded block CBU as the vector of reference movement of the specific subcoding unit SubCU(t0).
In another example, suppose the specific subcoding unit SubCU(t0) represents the jth subblock of a certain column of subblocks in the block under processing (for example, the processed block shown in Figure 11), where j can range from 1 to 8. In a second designation rule such as "horizontal designation", the preprocessor module can use the motion vector yj of the nearest subcoding unit within the left coded block CBL as the Reference motion vector of specific subcoding unit SubCU(t0).
In another example, a third assignment rule such as "hybrid assignment" can be selected to determine the reference motion vector. Suppose the specific subcoding unit SubCU(t0) represents the jth subblock of the fourth/other row of subblocks in the block under processing (indicated by "Block processed" in Figure 11), where j can range from 1 to 8. Referring to Figure 11, as illustrated with the arrow pointing to the jth subblock of the fourth/other row of subblocks, the preprocessing module uses the motion vector xj of the nearest subcoding unit within the upper coded block CBU as the reference motion vector of the specific subcoding unit SubCU(t0). Additionally, suppose the specific subcoding unit SubCU(t0) represents the jth subblock of the fourth/eighth column of subblocks in the block under processing (indicated by "Block processed" in Figure 11), where j can range from 1 to 7 except for 4. Referring to Figure 11, as illustrated with the arrow pointing to the jth subblock of the fourth/eighth column of subblocks, the preprocessing module uses the motion vector yj of the nearest subcoding unit within the left coded block CBL as the reference motion vector of the specific subcoding unit SubCU(t0). In this embodiment, the reference motion vectors of the remaining subcoding units within the coding unit CU(t0) can be obtained by using the same or similar method in any of the embodiments/variations disclosed above. Similar descriptions are not repeated in detail for this modality.
It should be noted that, in some modes such as some variations of the above mode, the pre-processing module can control the operations revealed by using a signal, and more particularly, by explicitly sending a signal. For example, an encoder deployed according to one of these modalities may send a signal indicating whether the assignment rule to be applied is one of "vertical assignment" and "horizontal assignment" (and even a "hybrid assignment"), then by receiving of a bit stream carrying the signaling, an associated decoder can be notified of the assignment rule to be applied.
Additionally, in some embodiments, various motion vector designation methods disclosed above (e.g., "vertical designation" and "horizontal designation" and even "hybrid designation") can be used to perform local motion vector derivation. For example, in a situation where "vertical designation" is applied, the motion vectors of each row of subcoding units are derived from the corresponding locations within the upper coded block CBU (for example, the closest subcoding unit within the block top coded CBU), respectively. It is an advantage of the present invention that the methods and apparatus of the present invention can appropriately perform localized multiple-hypothesis prediction, and more particularly, can appropriately perform multiple-hypothesis motion compensations in the unit's subcoding units code under consideration with ease. As a result of using the method and apparatus of the present invention, both motion vector prediction operations and multiple hypothesis motion compensation operations can be performed without introducing any related art problem such as low coding efficiency.
Those skilled in the art will readily appreciate that numerous modifications and changes to the device and method can be made while maintaining the teachings of the invention. Accordingly, the above disclosure is to be construed as limited only by the distributions and links in the appended claims.

权利要求:
Claims (20)
[0001]
1. Method to perform localized multiple hypotheses prediction during video encoding of a coding unit, the method being CHARACTERIZED by the fact that it comprises: dividing the coding unit into a plurality of subcoding units; and processing each of the subcoding units, wherein the processing step of each of the subcoding units further comprises: obtaining motion information from a first set of encoded units, wherein the encoded units are motion compensated and the motion information comprises a first motion vector and a second motion vector corresponding to subcoding units of other encoding units within a current frame where the encoding unit is located; obtaining a plurality of pixel values from the motion information of the first set of coded units; using a linear combination of the plurality of pixel values as a prediction pixel value of a specific subcoding unit of the subcoding units; and to derive a prediction pixel value from another subcoding unit in the encoding unit by using the motion information of a second set of coded units, wherein the second set of coded units comprises at least one coded unit that is not in the first set of coded units.
[0002]
2. Method according to claim 1, CHARACTERIZED by the fact that the step of using the linear combination of the plurality of pixel values as the prediction pixel value of the specific subcoding unit further comprises: using a weighted sum of the plurality of pixel values as the prediction pixel value of the specific subcoding unit.
[0003]
3. Method according to claim 2, CHARACTERIZED by the fact that the step of using the linear combination of the plurality of pixel values as the prediction pixel value of the specific subcoding unit further comprises: performing ideal Wiener filtering in the plurality of pixel values with respect to an actual pixel value of a current pixel, in order to adjust the respective weighted parameters for the plurality of pixel values and to generate the weighted sum.
[0004]
4. Method according to claim 2, CHARACTERIZED by the fact that the step of using the linear combination of the plurality of pixel values as the prediction pixel value of the specific subcoding unit further comprises: determining the respective weighted parameters for the plurality of pixel values according to the contents of at least one proximate coding unit, in order to generate the weighted sum.
[0005]
5. Method according to claim 2, CHARACTERIZED by the fact that the step of using the linear combination of the plurality of pixel value as the prediction pixel value of the specific subcoding unit further comprises: determining the respective weighted parameters for the plurality of pixel values according to the contents of the first set of coded units, in order to generate the weighted sum.
[0006]
6. Method according to claim 2, CHARACTERIZED by the fact that the step of using the linear combination of the plurality of pixel value as the prediction pixel value of the specific subcoding unit further comprises: determining the respective weighted parameters for the plurality of pixel values by offline training in order to generate the weighted sum.
[0007]
7. Method according to claim 2, CHARACTERIZED by the fact that the step of using the linear combination of the plurality of pixel value as the prediction pixel value of the specific subcoding unit further comprises: determining the respective weighted parameters for the plurality of pixel values by online training in order to generate the weighted sum.
[0008]
8. Method according to claim 1, CHARACTERIZED by the fact that the motion information of the first set of coded units comprises at least one or a combination of motion vector, reference frame index and prediction direction.
[0009]
9. Method according to claim 1, CHARACTERIZED by the fact that the processing step of each of the subcoding units further comprises: dynamically selecting a set of motion vectors from a plurality of sets of motion vectors and use the selected set of motion vectors as the motion information of the first set of coded units based on an explicit signal.
[0010]
10. Method according to claim 1, CHARACTERIZED by the fact that the step of dividing the encoding unit into the plurality of subcoding units further comprises: determining a specific subcoding unit size to be a transformation size with respect to encoding unit video encoding.
[0011]
11. Method according to claim 1, CHARACTERIZED by the fact that the step of dividing the encoding unit into the plurality of subcoding units further comprises: dividing the encoding unit into the plurality of subcoding units based on at least one mode of at least one coding unit close.
[0012]
12. Method according to claim 1, CHARACTERIZED by the fact that the step of dividing the encoding unit into the plurality of subcoding units further comprises: dividing the encoding unit into the plurality of subcoding units based on contents/partition of at least one nearby encoding unit.
[0013]
13. Method according to claim 1, CHARACTERIZED by the fact that the encoded units comprise at least one spatially encoded subcoding/coding unit and/or at least one temporally encoded subcoding/coding unit.
[0014]
14. Method according to claim 1, CHARACTERIZED by the fact that the processing step of each of the subcoding units further comprises: obtaining at least one motion vector of the encoding unit by performing motion estimation, for use to perform distortion rate optimization in order to obtain the prediction pixel value of the specific subcoding unit.
[0015]
15. Method according to claim 1, CHARACTERIZED by the fact that the processing step of each of the subcoding units further comprises: when the movement information is obtained, designating a reference movement vector for each subcoding unit.
[0016]
16. Method according to claim 1, CHARACTERIZED by the fact that the motion information of the encoding unit further comprises a third motion vector corresponding to a subcoding unit of another encoding unit located within another frame that is not the current frame where the encoding unit is located.
[0017]
17. Method according to claim 15, CHARACTERIZED by the fact that the step of designating the reference motion vector further comprises: designating the reference motion vector using a motion vector of a subcoding unit of another encoding unit.
[0018]
18. Apparatus to perform prediction of multiple hypotheses located during video coding of a coding unit, and the apparatus is CHARACTERIZED by the fact that it further comprises: a processing circuit arranged to perform video coding in the coding unit, in which the processing circuit comprises: a pre-processing module arranged to divide the encoding unit into a plurality of subcoding units and to process the subcoding units, wherein the pre-processing module obtains the motion information of a first set of coded units, where the coded units are motion compensated and the motion information comprises a first motion vector and a second motion vector corresponding to subcoding units of other encoding units within a current frame where the encoding unit is located and performs the compensation of multiple movements. s hypotheses in the subcoding unit specific subcoding unit according to the motion information and the pre-processing module uses a linear combination of a plurality of pixel values obtained from the motion information of the first set of units encoded as a value of prediction pixel of the specific subcoding unit; and at least one coding module arranged to perform video coding in the coding unit on the basis of multi-chance motion compensation performed by the pre-processing module; wherein the pre-processing module derives a prediction pixel value from another subcoding unit in the encoding unit using the motion information of a second set of encoded units comprising at least one encoded unit which is not in the first set of encoded units, the second set of encoded units are motion compensated encoding units .
[0019]
19. Apparatus according to claim 18, CHARACTERIZED by the fact that the pre-processing module uses a weighted sum of the plurality of pixel values as the prediction pixel value of the specific subcoding unit.
[0020]
20. Apparatus according to claim 18, CHARACTERIZED by the fact that the pre-processing module divides the encoding unit into the plurality of subcoding units based on at least one mode, content or partition of at least one unit of close encoding.

类似技术:

---

公开号 | 公开日 | 专利标题

---

BR112012025275B1|2021-08-03|METHOD FOR PERFORMING MULTIPLE HYPOTHESIS PREDICTION LOCATED DURING VIDEO ENCODING OF AN ENCODING UNIT AND ASSOCIATED APPARATUS

---

US8929460B1|2015-01-06|Method and encoding/decoding of video using common merging candidate set of asymmetric partitions

---

BR112019025566A2|2020-06-23|MOTION VECTOR PREDICTION

---

TW201739252A|2017-11-01|Method and apparatus of video coding with affine motion compensation

---

KR20170125086A|2017-11-13|Image prediction method and related apparatus

---

BR112019017252A2|2020-04-14|deriving motion vector information in a video decoder

---

BR112012019560B1|2021-08-24|METHOD FOR MANAGING CANDIDATES FOR PREDICTED MOVEMENT VECTOR, AND, VIDEO ENCODING AND DECODING APPARATUS

---

BR122020002123A2|2020-12-01|VIDEO DECODING

---

JP5277257B2|2013-08-28|Video decoding method and video encoding method

---

BR112020014654A2|2020-12-01|Affine motion compensation in video encoding

---

BR112019027261A2|2020-07-14|motion vector refinement for multi-reference prediction

---

BR112013017208B1|2019-01-29|PREDICTIVE CODING METHOD AND MOVEMENT PREDICTIVE CODING DEVICE, AND PREDICTIVE DECODING METHOD AND PREDICTIVE MOVEMENT DECODING DEVICE

---

BR122015013871A2|2020-10-27|method for decoding an image

---

TWI729402B|2021-06-01|Weighted interweaved prediction

---

BR112020006969A2|2020-10-06|limited memory access window for motion vector refinement

---

BR112019028012A2|2020-07-07|device and method for determining a motion vector, video encoder and decoder to encode and decode a plurality of photos

---

BR112020026988A2|2021-03-30|EXACT REFINING METHOD AND APPARATUS OF ERROR SURFACE-BASED SUBPIXEL FOR MOVING VECTOR REFINEMENT ON THE DECODER SIDE

---

TW202021356A|2020-06-01|Efficient affine merge motion vector derivation

---

CN111886870A|2020-11-03|Motion vector correction search using integer pixel resolution

---

CN111567047A|2020-08-21|Memory access window for sub-prediction block motion vector derivation

---

TW202015405A|2020-04-16|Method and apparatus of inter prediction for video coding

---

JP2021022936A|2021-02-18|Image prediction method and related device

---

JP2022044612A|2022-03-17|Image prediction method and related equipment

---

JP2022044613A|2022-03-17|Image prediction method and related equipment

---

TW202032989A|2020-09-01|Mv planar mode with block level

同族专利:

---

公开号 | 公开日

---

BR112012025275A2|2016-06-21|

---

TWI469645B|2015-01-11|

---

CN102223542A|2011-10-19|

---

US8879620B2|2014-11-04|

---

TWI520586B|2016-02-01|

---

AU2011240486B2|2014-02-13|

---

CN105791858A|2016-07-20|

---

KR20120126115A|2012-11-20|

---

TWI514855B|2015-12-21|

---

RU2012141233A|2014-06-10|

---

IL222337A|2016-05-31|

---

IL245480D0|2016-06-30|

---

CN105791858B|2019-11-15|

---

US20110255600A1|2011-10-20|

---

KR101452921B1|2014-10-23|

---

US20110255598A1|2011-10-20|

---

IL245480A|2018-01-31|

---

EP2559250A4|2016-01-06|

---

JP5211263B2|2013-06-12|

---

CN102907095B|2016-10-12|

---

RU2530318C2|2014-10-10|

---

CN102907095A|2013-01-30|

---

US20120027097A1|2012-02-02|

---

US8837592B2|2014-09-16|

---

BR112012025275B8|2021-08-24|

---

EP2559250A1|2013-02-20|

---

US8891608B2|2014-11-18|

---

TW201511525A|2015-03-16|

---

CN107071472A|2017-08-18|

---

TW201511524A|2015-03-16|

---

TWI445411B|2014-07-11|

---

WO2011127828A1|2011-10-20|

---

TW201218776A|2012-05-01|

---

EP3220646A1|2017-09-20|

---

AU2011240486A1|2012-10-11|

---

IL222337D0|2012-12-31|

---

EP2559250B1|2020-07-08|

---

CN105872558A|2016-08-17|

---

CN105872558B|2019-06-07|

---

TW201143462A|2011-12-01|

---

JP2012513179A|2012-06-07|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

GB2231751B|1989-04-27|1993-09-22|Sony Corp|Motion dependent video signal processing|

---

DE69027966T2|1989-10-06|1997-02-06|Philips Electronics Nv|Adaptive circuitry for ghost cancellation|

---

US5134475A|1990-12-11|1992-07-28|At&T Bell Laboratories|Adaptive leak hdtv encoder|

---

KR970002964B1|1993-12-24|1997-03-13|대우전자 주식회사|A filter of time interpolation for image signal|

---

EP0765087B1|1995-08-29|2003-11-12|Sharp Kabushiki Kaisha|Video coding device|

---

WO1997046020A2|1996-05-24|1997-12-04|Philips Electronics N.V.|Motion vector processing|

---

JP3633159B2|1996-12-18|2005-03-30|ソニー株式会社|Moving picture signal encoding method and apparatus, and moving picture signal transmission method|

---

US6807231B1|1997-09-12|2004-10-19|8×8, Inc.|Multi-hypothesis motion-compensated video image predictor|

---

US6532265B1|1999-03-25|2003-03-11|Imec Vzw|Method and system for video compression|

---

JP2000299864A|1999-04-12|2000-10-24|Canon Inc|Method for processing dynamic image|

---

US6633677B1|1999-12-30|2003-10-14|Stmicroelectronics, Inc.|Method and apparatus for processing an image in an image compression/decompression system that uses hierachical coding|

---

US6438168B2|2000-06-27|2002-08-20|Bamboo Media Casting, Inc.|Bandwidth scaling of a compressed video stream|

---

US20040001546A1|2002-06-03|2004-01-01|Alexandros Tourapis|Spatiotemporal prediction for bidirectionally predictive pictures and motion vector prediction for multi-picture reference motion compensation|

---

US20040141555A1|2003-01-16|2004-07-22|Rault Patrick M.|Method of motion vector prediction and system thereof|

---

US7184602B2|2003-05-02|2007-02-27|Microsoft Corp.|System and method for low bandwidth video streaming for face-to-face teleconferencing|

---

HU0301368A3|2003-05-20|2005-09-28|Amt Advanced Multimedia Techno|Method and equipment for compressing motion picture data|

---

MXPA05014211A|2003-06-25|2006-05-31|Thomson Licensing|Fast mode-decision encoding for interframes.|

---

CN101917621A|2003-08-26|2010-12-15|汤姆森特许公司|Method and apparatus for decoding hybrid intra-inter coded blocks|

---

US7599438B2|2003-09-07|2009-10-06|Microsoft Corporation|Motion vector block pattern coding and decoding|

---

JP3880985B2|2004-08-05|2007-02-14|松下電器産業株式会社|Motion vector detection apparatus and motion vector detection method|

---

US7447337B2|2004-10-25|2008-11-04|Hewlett-Packard Development Company, L.P.|Video content understanding through real time video motion analysis|

---

RU2368095C1|2005-07-22|2009-09-20|Мицубиси Электрик Корпорейшн|Image coder and image decoder, method of image coding and method of image decoding, program of image coding and program of image decoding and computer-readable record medium, on which image coding program is recorded, and computer-readable record medium, on which image decoding program is recorded|

---

JP2007142521A|2005-11-15|2007-06-07|Sharp Corp|Apparatus and method for calculating motion vector|

---

JP4763549B2|2006-08-18|2011-08-31|富士通セミコンダクター株式会社|Inter-frame prediction processing apparatus, image encoding apparatus, and image decoding apparatus|

---

CN101222627A|2007-01-09|2008-07-16|华为技术有限公司|Multi-viewpoint video coding and decoding system, method and device for estimating vector|

---

KR101366241B1|2007-03-28|2014-02-21|삼성전자주식회사|Method and apparatus for video encoding and decoding|

---

JP4325708B2|2007-07-05|2009-09-02|ソニー株式会社|Data processing device, data processing method and data processing program, encoding device, encoding method and encoding program, and decoding device, decoding method and decoding program|

---

JP4786612B2|2007-08-14|2011-10-05|Ｋｄｄｉ株式会社|Predicted motion vector generation apparatus for moving picture encoding apparatus|

---

US8000411B2|2008-01-04|2011-08-16|Qualcomm Incorporated|Decoding scheme using multiple hypotheses about transmitted messages|

---

JP4990927B2|2008-03-28|2012-08-01|三星電子株式会社|Method and apparatus for encoding / decoding motion vector information|

---

US8195001B2|2008-04-09|2012-06-05|Intel Corporation|In-loop adaptive wiener filter for video coding and decoding|

---

KR101691199B1|2008-04-11|2016-12-30|톰슨 라이센싱|Method and apparatus for template matching prediction in video encoding and decoding|

---

US8908763B2|2008-06-25|2014-12-09|Qualcomm Incorporated|Fragmented reference in temporal compression for video coding|

---

CN102160384A|2008-09-24|2011-08-17|索尼公司|Image processing device and method|

---

US8724697B2|2008-09-26|2014-05-13|Qualcomm Incorporated|Locating motion vectors for video data units|

---

US8660176B2|2008-09-26|2014-02-25|Qualcomm Incorporated|Resolving geometric relationships among video data units|

---

US8634457B2|2008-09-26|2014-01-21|Qualcomm Incorporated|Determining availability of video data units|

---

KR101279573B1|2008-10-31|2013-06-27|에스케이텔레콤 주식회사|Motion Vector Encoding/Decoding Method and Apparatus and Video Encoding/Decoding Method and Apparatus|

---

JP4957780B2|2009-11-20|2012-06-20|カシオ計算機株式会社|Motion compensated predictive coding apparatus, motion compensated predictive coding method, and program|US9118929B2|2010-04-14|2015-08-25|Mediatek Inc.|Method for performing hybrid multihypothesis prediction during video coding of a coding unit, and associated apparatus|

---

US8971400B2|2010-04-14|2015-03-03|Mediatek Inc.|Method for performing hybrid multihypothesis prediction during video coding of a coding unit, and associated apparatus|

---

KR102086145B1|2010-12-13|2020-03-09|한국전자통신연구원|Method for intra prediction and apparatus thereof|

---

EP2661080A4|2010-12-31|2016-06-29|Korea Electronics Telecomm|Method for encoding video information and method for decoding video information, and apparatus using same|

---

RU2610294C1|2011-01-12|2017-02-08|Мицубиси Электрик Корпорейшн|Image encoding device, image decoding device, image encoding method and image decoding method|

---

WO2013023518A1|2011-08-17|2013-02-21|Mediatek Singapore Pte. Ltd.|Method and apparatus for intra prediction using non-square blocks|

---

US9736489B2|2011-09-17|2017-08-15|Qualcomm Incorporated|Motion vector determination for video coding|

---

US9386305B2|2011-11-28|2016-07-05|Qualcomm Incorporated|Largest coding unitor partition-based syntax for adaptive loop filter and sample adaptive offset in video coding|

---

US20130332701A1|2011-12-23|2013-12-12|Jayashankar Bharadwaj|Apparatus and method for selecting elements of a vector computation|

---

WO2013107028A1|2012-01-19|2013-07-25|Mediatek Singapore Pte. Ltd.|Methods and apparatuses of amvp simplification|

---

KR102030205B1|2012-01-20|2019-10-08|선 페이턴트 트러스트|Methods and apparatuses for encoding and decoding video using temporal motion vector prediction|

---

EP2811743B1|2012-02-03|2021-03-03|Sun Patent Trust|Image encoding method, image decoding method, image encoding device, image decoding device, and image encoding/decoding device|

---

WO2013132792A1|2012-03-06|2013-09-12|パナソニック株式会社|Method for coding video, method for decoding video, device for coding video, device for decoding video, and device for coding/decoding video|

---

US9544612B2|2012-10-04|2017-01-10|Intel Corporation|Prediction parameter inheritance for 3D video coding|

---

KR101789954B1|2013-12-27|2017-10-25|인텔 코포레이션|Content adaptive gain compensated prediction for next generation video coding|

---

CN111193929A|2013-12-16|2020-05-22|浙江大学|Method and device for coding and decoding forward double-hypothesis coding image block|

---

JP6482191B2|2014-06-12|2019-03-13|キヤノン株式会社|Image encoding device, image encoding method and program, image decoding device, image decoding method and program|

---

GB2531001B|2014-10-06|2019-06-05|Canon Kk|Method and apparatus for vector encoding in video coding and decoding|

---

US10958927B2|2015-03-27|2021-03-23|Qualcomm Incorporated|Motion information derivation mode determination in video coding|

---

TWI610558B|2016-05-26|2018-01-01|晨星半導體股份有限公司|Bit Allocation Method and Video Encoding Device|

---

WO2018097692A2|2016-11-28|2018-05-31|한국전자통신연구원|Method and apparatus for encoding/decoding image, and recording medium in which bit stream is stored|

---

WO2018128345A2|2017-01-04|2018-07-12|삼성전자 주식회사|Decoding method and apparatus therefor|

---

WO2019192491A1|2018-04-02|2019-10-10|Mediatek Inc.|Video processing methods and apparatuses for sub-block motion compensation in video coding systems|

---

WO2019229683A1|2018-05-31|2019-12-05|Beijing Bytedance Network Technology Co., Ltd.|Concept of interweaved prediction|

---

CN110620929A|2018-06-19|2019-12-27|北京字节跳动网络技术有限公司|Selected motion vector difference accuracy without motion vector prediction truncation|

---

TWI735902B|2018-07-02|2021-08-11|大陸商北京字節跳動網絡技術有限公司|Lookup table with intra frame prediction and intra frame predication from non adjacent blocks|

---

JP2022500909A|2018-09-19|2022-01-04|北京字節跳動網絡技術有限公司Beijing Bytedance Network Technology Co., Ltd.|Use of syntax for affine mode with adaptive motion vector resolution|

---

CN110602493A|2018-09-19|2019-12-20|北京达佳互联信息技术有限公司|Method and equipment for interlaced prediction of affine motion compensation|

---

CN111050164A|2018-10-15|2020-04-21|华为技术有限公司|Method and device for encoding and decoding|

---

US11240497B2|2018-11-15|2022-02-01|Telefonaktiebolaget Lm Ericsson |Deblocking in a video encoder and/or video decoder|

法律状态:
2017-07-18| B25A| Requested transfer of rights approved|Owner name: HFI INNOVATION, INC. (CN) |

---

2018-03-27| B15K| Others concerning applications: alteration of classification|Ipc: H04N 19/139 (2014.01), H04N 19/513 (2014.01), H04N |

---

2018-12-26| B06F| Objections, documents and/or translations needed after an examination request according [chapter 6.6 patent gazette]|

---

2020-03-17| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|

---

2020-11-10| B25K| Entry of change of name and/or headquarter and transfer of application, patent and certificate of addition of invention: republication|Owner name: HFI INNOVATION INC. (CN) Free format text: RETIFICACAO DO DESPACHO 25.1 ? TRANSFERENCIA DE TITULAR PUBLICADO NA RPI NO 2428, DE 18/07/2017, QUANTO AO ITEM (71) - DEPOSITANTE.ONDE SE LE: HFI INNOVATION, INC.LEIA-SE: HFI INNOVATION INC. |

---

2021-07-06| B09A| Decision: intention to grant [chapter 9.1 patent gazette]|

---

2021-08-03| B16A| Patent or certificate of addition of invention granted [chapter 16.1 patent gazette]|Free format text: PRAZO DE VALIDADE: 20 (VINTE) ANOS CONTADOS A PARTIR DE 14/04/2011, OBSERVADAS AS CONDICOES LEGAIS. PATENTE CONCEDIDA CONFORME ADI 5.529/DF, QUE DETERMINA A ALTERACAO DO PRAZO DE CONCESSAO. |

---

2021-08-24| B16C| Correction of notification of the grant [chapter 16.3 patent gazette]|Free format text: REF. RPI 2639 DE 03/08/2021 QUANTO AO ENDERECO. |

优先权:

---

申请号 | 申请日 | 专利标题

---

US32394810P| true| 2010-04-14|2010-04-14|

---

US61/323,948|2010-04-14|

---

US13/080,668|US8879620B2|2010-04-14|2011-04-06|Method for performing localized multihypothesis prediction during video coding of a coding unit, and associated apparatus|

---

US13/080,668|2011-04-06|

---

PCT/CN2011/072783|WO2011127828A1|2010-04-14|2011-04-14|Method for performing localized multihypothesis prediction during video coding of a coding unit, and associated apparatus|

---