image processing device and method, and, computer readable storage media

专利摘要:

公开号:BR112014020801B1
申请号:R112014020801
申请日:2013-02-20
公开日:2018-09-25
发明作者:Tanaka Junichi；Morigami Yoshitaka
申请人:Sony Corp；
IPC主号:

专利说明:

(54) Title: IMAGE PROCESSING DEVICE AND METHOD, AND, LEGIBLE STORAGE MEDIA BY COMPUTER (51) Int.CI .: H04N 19/126 (30) Unionist Priority: 29/02/2012 JP 2012-044009 (73 ) Holder (s): SONY CORPORATION (72) Inventor (s): JUNICHI TANAKA; YOSHITAKA MORIGAMI (85) National Phase Start Date: 08/22/2014 / 137 “IMAGE PROCESSING DEVICE AND METHOD, AND, LEGIBLE COMPUTER STORAGE MEDIA”
Technical Field [001] The present exhibition relates to an image processing device and method.
Background to the technique [002] In H.264 / AVC (Advanced Video Encoding), which is one of the model specifications for video encoding schemes, High Profile or higher profiles allow quantization of image data with sizes of quantization step that differ from one component of orthogonal transform coefficient to another. The quantization step size for each component of the orthogonal transform coefficient can be established based on a reference step value and a quantization matrix (also called a scale list) defined by a size equivalent to the unit of an orthogonal transform.
[003] A specified value of a quantization matrix is prepared for each prediction mode (intraprediction mode, interpretation mode) and for each transform unit size (4x4, 8x8). In addition, users are allowed to specify a single quantization matrix other than the values specified in a sequence parameter set or frame parameter set. In a case where no quantization matrix is used, quantization step sizes used for quantization have an equal value for all components.
[004] In HEVC (High Efficiency Video Encoding), which is being standardized as a next generation video encoding scheme and which is a successor to H.264 / AVC, the concept of encoding units (CUs) corresponding traditional macroblocks were introduced (see, for example, NPL 1). The size range of coding units is specified by a set of values that are powers of 2, called the largest coding unit (LCU) and the smallest coding unit
Petition 870170066495, of September 6, 2017, p. 10/294 / 137 (SCU), in a sequence parameter set. In addition, the size of the specific coding unit in the range specified by the LCU and SCU is specified using split_flag.
[005] In HEVC, a coding unit can be divided into one or more orthogonal transform units, or one or more transform units (TUs). An available transformed unit size is any of 4x4, 8x8, 16x16 and 32x32.
[006] Meanwhile, the DC component (also called the direct current component) of a quantization matrix (scale list) is transmitted as different data from the AC components (also called the alternating current components) of this for purposes such as reducing the amount of encoding during transmission. Specifically, the DC component of a scale list is transmitted as a DC coefficient (also called a direct current coefficient) other than AC coefficients (also called alternating current coefficients), which are the AC components of the scale.
[007] In order to reduce the amount of coding of the coefficient of
CC during transmission, it has been suggested that a constant (for example, 8) be subtracted from the CC coefficient value and the resulting value (scaling_list_CC_coef_minus8) is encoded using signed exponential Golomb encoding (see, for example, NPL 1).
Citation List
Non-Patent Literature [008] NPL 1: Benjamim Bross, Fraunhofer HHI, Galantear-Jin Han,
Gachon University, Jens-Rainer Ohm, RWTH Aachen, Gary J. Sullivan, Microsoft, Thomas Wiegand, Fraunhofer HHI / TU Berlin, JCTVC-H1003, High Efficiency Video Coding (HEVC) text specification draft 6, Joint Collaborating Team on Coding video (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 ^to 7 Meeting: Geneva, CH, 21-30 November 2011.
Petition 870170066495, of September 6, 2017, p. 11/294 / 137
Summary of the Invention
Technical Problem [009] However, there is a concern that the method described above will not provide sufficient compression efficiency although it facilitates processes.
[0010] The present exposure was made due to the situation described above, and it is an objective of the present exposure to enable the suppression of an increase in the amount of coding in a scale list.
Solution to the Problem [0011] One aspect of the present exhibit provides an image processing device including an establishment unit configured to establish a coefficient located at the beginning of a quantization matrix whose size is limited to no more than one transmission size which is a maximum size allowed in transmission, adding a substitution difference coefficient which is a difference between a substitution coefficient and the coefficient located at the beginning of the quantization matrix for the coefficient located at the beginning of the quantization matrix, the substitution coefficient being used to replace a coefficient located at the beginning of a quantization matrix converted by upward conversion which is obtained by converting upward the quantization matrix to the same size as a block size which is a processing unit on which decanting is performed; an upward conversion unit configured to upward convert the quantization matrix established by the establishment unit to establish the upward conversion converted quantization matrix; and a decanting unit configured to decantify quantized data obtained by decoding encoded data, use a quantization matrix converted by upward conversion in which a coefficient located at the beginning of the quantization matrix converted by upward conversion established by the upward conversion unit has been replaced with the coefficient of substitution.
Petition 870170066495, of September 6, 2017, p. 12/294 / 137 [0012] The establishment unit can establish the substitution coefficient by adding a difference between the substitution coefficient and an initial value established for the quantization matrix to the initial value. [0013] The establishment unit can establish coefficients of the quantization matrix using the substitution difference coefficient and difference coefficients that are differences between the coefficients of the quantization matrix.
[0014] The substitution difference coefficient and the difference coefficients that are the differences between the coefficients of the quantization matrix can be transmitted collectively. The establishment unit can establish the coefficients of the quantization matrix using the substitution difference coefficient and difference coefficients transmitted collectively.
[0015] The substitution difference coefficient and the difference coefficients that are the differences between the coefficients of the quantization matrix may have been coded. The establishment unit can decode the encoded substitution difference coefficient and the encoded difference coefficients.
[0016] The upward converting unit can convert upwardly the quantization matrix whose size is limited to no larger than the transmission size, performing one more neighboring interpolation process in matrix elements of the quantization matrix.
[0017] The transmission size can be an 8x8 size. The reverse conversion unit can convert by reverse conversion a quantization matrix having an 8x8 size to a quantization matrix having a 16x16 size, performing the nearest neighbor interpolation process in matrix elements of the quantization matrix having the size 8x8.
[0018] The upward conversion unit can convert by
Petition 870170066495, of September 6, 2017, p. 13/294 / 137 upward conversion a quantization matrix having an 8x8 size to a quantization matrix having a 32x32 size, performing the nearest neighbor interpolation process on matrix elements of the quantization matrix having an 8x8 size.
[0019] A coding unit which is a processing unit on which a decoding process is performed and a transform unit which is a processing unit on which a transform process is performed may have a layered structure. The image processing device may additionally include a decoding unit configured to perform a decoding process on the encoded data using a unit having a layered structure to generate the quantized data. The upward conversion unit can upwardly convert the transmission size quantization matrix to a size of a transform unit which is a processing unit on which decanting is performed. [0020] One aspect of the present exposure provides an image processing method including establishing a coefficient located at the beginning of a quantization matrix whose size is limited to no greater than a transmission size which is a maximum size allowed in transmission, adding a substitution difference coefficient which is a difference between a substitution coefficient and the coefficient located at the beginning of the quantization matrix for the coefficient located at the beginning of the quantization matrix, the substitution coefficient being used to replace a coefficient located at the beginning of an upward converting quantization matrix that is obtained by converting upward quantization matrix to the same size as a block size which is a processing unit on which decanting is performed; convert by quantitative conversion the established quantization matrix to establish the quantization matrix converted by ascending conversion; and decanting quantized data obtained
Petition 870170066495, of September 6, 2017, p. 14/294 / 137 decoding encoded data, using an upward converting quantization matrix in which a coefficient located at the beginning of the upward converting quantization matrix has been replaced with the substitution coefficient.
[0021] Another aspect of the present exhibition provides an image processing device including an establishment unit configured to establish a substitution difference coefficient which is a difference between a substitution coefficient and a coefficient located at the beginning of a quantization matrix whose size is limited to no larger than a transmission size which is a maximum allowed transmission size, the substitution coefficient being used to replace a coefficient located at the beginning of a quantization matrix converted by upward conversion which is obtained by converting by upward conversion the quantization matrix at the same size as a block size which is a processing unit on which decanting is performed; a quantization unit configured to quantize an image to generate quantized data; and a transmission unit configured to transmit encoded data obtained encoding the quantized data generated by the quantization unit, substitution coefficient data obtained encoding the substitution coefficient, and substitution difference coefficient data obtained encoding the established substitution difference coefficient by the establishment unit.
[0022] The establishment unit can establish a difference between the substitution coefficient and an initial value established for the quantization matrix.
[0023] The establishment unit can establish difference coefficients that are differences between coefficients of the quantization matrix. The transmission unit can transmit difference coefficient data obtained by encoding the difference coefficients established by the
Petition 870170066495, of September 6, 2017, p. 15/294 / 137 establishment.
[0024] The transmission unit can collectively transmit the replacement coefficient data and the replacement difference coefficient data.
[0025] The transmission unit can transmit the replacement coefficient data and the replacement difference coefficient data in order of the replacement coefficient data and the replacement difference coefficient data.
[0026] The quantization unit can quantize the image using the quantization matrix or the quantization matrix converted by upward conversion.
[0027] A coding unit which is a processing unit in which a coding process is performed and a transform unit which is a processing unit in which a transforming process is performed can have a layered structure. The image processing device may additionally include an encoding unit configured to encode the quantized data generated by the quantization unit.
[0028] Another aspect of the present exhibition provides an image processing method including establishing a substitution difference coefficient which is a difference between a substitution coefficient and a coefficient located at the beginning of a quantization matrix whose size is limited to no larger than a transmission size that is a maximum size allowed in transmission, the substitution coefficient being used to replace a coefficient located at the beginning of a quantization matrix converted by upward conversion which is obtained by converting upward the quantization matrix at the same size as a block size which is a processing unit on which decanting is performed; quantize an image to generate quantized data; and transmit encoded data obtained by encoding the data
Petition 870170066495, of September 6, 2017, p. 16/294 / 137 quantized generated, substitution coefficient data obtained encoding the substitution coefficient, and substitution difference coefficient data obtained encoding the established substitution difference coefficient.
[0029] Yet another aspect of the present exhibition provides an image processing device including a decoding unit configured to decode encoded data to generate quantized data; and a decanting unit configured to decant the quantized data generated by the decoding unit, using a standard quantization matrix having the same size as a block size which is a processing unit on which decanting is performed, when in a copy mode in which a quantization matrix is copied, quantization matrix reference data identifying a quantization matrix reference destination matches quantization matrix identification data identifying the quantization matrix.
[0030] The decanting unit can decantify the quantized data by analyzing syntax whose semantics are established so that the standard quantization matrix is referred to when the quantization matrix reference data matches the quantization matrix identification data.
[0031] The decanting unit can decantify the quantized data by analyzing syntax whose semantics are established so that the standard quantization matrix is referred to when a difference between the quantization matrix reference data and the quantization matrix identification data is equal to 0.
[0032] Yet another aspect of the present exhibit provides an image processing method including decoding encoded data to generate quantized data; and decanting the quantized data generated in the decoding, using a standard quantization matrix having the same
Petition 870170066495, of September 6, 2017, p. 17/294 / 137 size as a block size which is a processing unit in which decanting is performed, when in a copy mode in which a quantization matrix is copied, quantization matrix reference data identifying a reference destination of the quantization matrix matches the quantization matrix identification data identifying the quantization matrix.
[0033] Yet another aspect of the present exhibit provides an image processing device including an encoding unit configured to encode an image to generate encoded data; and an establishment unit configured to establish, as a syntax of the encoded data generated by the encoding unit, a syntax whose semantics are established such that a standard quantization matrix having the same size as a block size which is a processing unit in which quantization is performed whether referred to when in a copy mode in which a quantization matrix is copied, quantization matrix reference data identifying a quantization matrix reference destination matches quantization matrix identification data identifying the quantization matrix .
[0034] Yet another aspect of exposure present provides an image processing method including encoding an image to generate encoded data; and establish, as the syntax of the generated coded data, syntax whose semantics are established so that a standard quantization matrix having the same size as a block size which is a processing unit on which quantization is performed is referred to when in a copy in which a quantization matrix is copied, quantization matrix reference data identifying a quantization matrix reference destination matches quantization matrix identification data identifying the quantization matrix.
[0035] In one aspect of the present exposure, a coefficient located at the beginning of a quantization matrix whose size is
Petition 870170066495, of September 6, 2017, p. 18/294 / 137 limited to no greater than a transmission size that is a maximum allowed transmission size is established by adding a substitution difference coefficient which is a difference between a substitution coefficient and the coefficient located at the beginning of the matrix of quantization for the coefficient located at the beginning of the quantization matrix, the substitution coefficient being used to replace a coefficient located at the beginning of a quantization matrix converted by upward conversion which is obtained by converting upward quantization matrix to the same size as a block size which is a processing unit on which decanting is performed; the established quantization matrix is converted by upward conversion to establish the quantization matrix converted by upward conversion; and quantized data obtained by decoding encoded data are de-quantized using an upward converting quantization matrix in which a coefficient located at the beginning of the upward converting quantization matrix has been replaced with the substitution coefficient.
[0036] In another aspect of the present exposure, a substitution difference coefficient that is a difference between a substitution coefficient and a coefficient located at the beginning of a quantization matrix whose size is limited to no greater than a transmission size that is a maximum size allowed in transmission is established, the substitution coefficient being used to replace a coefficient located at the beginning of a quantization matrix converted by upward conversion that is obtained by converting upward the quantization matrix to the same size as a size of block which is a processing unit on which decanting is performed; an image is quantized to generate quantized data; and encoded data obtained by encoding the generated quantized data, substitution coefficient data obtained by encoding the substitution coefficient, and
Petition 870170066495, of September 6, 2017, p. 19/294 / 137 substitution difference coefficient obtained by encoding the established substitution difference coefficient are transmitted.
[0037] In yet another aspect of the present exhibition, encoded data is decoded to generate quantized data; and the quantized data generated in the decoding is de-quantized using a standard quantization matrix having the same size as a block size which is a processing unit on which de-quantization is performed, when in a copy mode in which a quantization matrix is copied , quantization matrix reference data identifying a quantization matrix reference destination matches quantization matrix identification data identifying the quantization matrix.
[0038] In yet another aspect of the present exhibition, an image is encoded to generate encoded data; and syntax whose semantics are established such that a standard quantization matrix having the same size as a block size which is a processing unit in which quantization is performed is referred to when in a copy mode in which a quantization matrix is copied , quantization matrix reference data identifying a quantization matrix reference destination match quantization matrix identification data identifying the quantization matrix are established as syntax of the generated coded data.
Advantageous Effects of the Invention [0039] According to the present exposure, it is possible to process an image. In particular, it is possible to suppress an increase in the amount of coding in a quantization matrix.
Brief Description of the Drawings [0040] Figure 1 is a diagram illustrating an example of a scale list.
[0041] Figure 2 is a diagram illustrating an example of upward conversion.
Petition 870170066495, of September 6, 2017, p. 20/294 / 137 [0042] Figure 3 is a diagram illustrating an example of how a scale list is used in a decoder.
[0043] Figure 4 is a diagram illustrating an example of coding a scale list.
[0044] Figure 5 is a diagram illustrating an example of coding a scale list using the present technology.
[0045] Figure 6 is a diagram illustrating an example of exponential Golomb codes.
[0046] Figure 7 includes diagrams illustrating an example of the syntax for a scale list.
[0047] Figure 8 is a diagram illustrating an example of the syntax for a standard matrix.
[0048] Figure 9 includes diagrams illustrating examples of the semantics of a standard matrix.
[0049] Figure 10 is a diagram illustrating an example of the syntax for a scale list.
[0050] Figure 11 is a diagram illustrating an example of the syntax for a scale list using the present technology.
[0051] Figure 12 includes diagrams illustrating an example of the syntax of a scale list in the related technique.
[0052] Figure 13 is a diagram illustrating an example of the syntax of a scale list.
[0053] Figure 14 is a block diagram illustrating an example of a main configuration of an image encoding device. [0054] Figure 15 is a block diagram illustrating an example of a main configuration of an orthogonal transform / quantization unit.
[0055] Figure 16 is a block diagram illustrating an example of a main configuration of a matrix processing unit. [0056] Figure 17 is a diagram illustrating an example of
Petition 870170066495, of September 6, 2017, p. 21/294 / 137 subsampling.
[0057] Figure 18 is a diagram illustrating an example of removing an overlapping portion.
[0058] Figure 19 is a block diagram illustrating an example of a main configuration of a DPCM unit.
[0059] Figure 20 is a flow chart illustrating an example of the flow of a quantization matrix coding process.
[0060] Figure 21 is a flow chart illustrating an example of the flow of a DPCM process.
[0061] Figure 22 is a block diagram illustrating an example of a main configuration of an image decoding device.
[0062] Figure 23 is a block diagram illustrating an example of a main configuration of a reverse orthogonal decanting / transforming unit.
[0063] Figure 24 is a block diagram illustrating an example of a main configuration of a matrix generation unit.
[0064] Figure 25 is a diagram illustrating an example of a closer neighbor interpolation process.
[0065] Figure 26 is a block diagram illustrating an example of a main configuration of a reverse DPCM unit.
[0066] Figure 27 is a flow chart illustrating an example of the flow of a matrix generation process.
[0067] Figure 28 is a flow chart illustrating an example of the flow of a residual signal decoding process.
[0068] Figure 29 is a flow chart illustrating an example of the flow of an inverse DPCM process.
[0069] Figure 30 is a diagram illustrating another example of the syntax of a scale list.
[0070] Figure 31 is a block diagram illustrating another
Petition 870170066495, of September 6, 2017, p. 22/294 / 137 example configuration of the DPCM unit.
[0071] Figure 32 is a flow chart illustrating another example of the DPCM process flow.
[0072] Figure 33 is a block diagram illustrating another example configuration of the reverse DPCM unit.
[0073] Figure 34 is a flowchart illustrating another example of the reverse DPCM process flow.
[0074] Figure 35 is a diagram illustrating yet another example of the syntax of a scale list.
[0075] Figure 36 is a flowchart illustrating yet another example of the reverse DPCM process flow.
[0076] Figure 37 is a diagram illustrating yet another example of the syntax of a scale list.
[0077] Figure 38 is a block diagram illustrating yet another example configuration of the DPCM unit.
[0078] Figure 39 is a flow chart illustrating yet another example of the DPCM process.
[0079] Figure 40 is a block diagram illustrating yet another example configuration of the reverse DPCM unit.
[0080] Figure 41 is a flowchart illustrating yet another example of the reverse DPCM process flow.
[0081] Figure 42 is a continued flowchart of Figure 41, illustrating yet another example of the reverse CMCM process flow.
[0082] Figure 43 includes diagrams illustrating yet another example of the syntax of a scale list.
[0083] Figure 44 includes diagrams illustrating yet another example of the syntax of a scale list.
[0084] Figure 45 includes diagrams illustrating yet another example of the syntax of a scale list.
[0085] Figure 46 is a diagram illustrating an example of a
Petition 870170066495, of September 6, 2017, p. 23/294 / 137 multivision image encoding scheme.
[0086] Figure 47 is a diagram illustrating an example of a main configuration of a multivision image encoding device to which the present technology is applied.
[0087] Figure 48 is a diagram illustrating an example of a main configuration of a multivision image decoding device to which the present technology is applied.
[0088] Figure 49 is a diagram illustrating an example of a layered image encoding scheme.
[0089] Figure 50 is a diagram illustrating an example of a main configuration of a layered image encoding device to which the present technology is applied.
[0090] Figure 51 is a diagram illustrating an example of a main configuration of a layered image decoding device to which the present technology is applied.
[0091] Figure 52 is a block diagram illustrating an example of a main configuration of a computer.
[0092] Figure 53 is a block diagram illustrating an example of a main configuration of a television set.
[0093] Figure 54 is a block diagram illustrating an example of a main configuration of a mobile terminal device.
[0094] Figure 55 is a block diagram illustrating an example of a main configuration of a recording / playback device.
[0095] Figure 56 is a block diagram illustrating an example of a main configuration of an imaging device.
[0096] Figure 57 is a block diagram illustrating an example of using scalable coding.
[0097] Figure 58 is a block diagram illustrating another example of the use of scalable coding.
[0098] Figure 59 is a block diagram illustrating yet another
Petition 870170066495, of September 6, 2017, p. 24/294 / 137 example of using gradable encoding.
Description of the Embodiments [0099] Ways to carry out the present exposure (hereinafter referred to as the Embodiments) will now be described. In this regard, the description will be made in the following order.
1. First embodiment (exemplary application of the present technology)
2. Second embodiment (image encoding device, image decoding device: first method)
3. Third embodiment (image encoding device, image decoding device: second method)
4. Fourth embodiment (image encoding device, image decoding device: third method)
5. Fifth embodiment (image encoding device, image decoding device: fourth method)
6. Sixth embodiment (image encoding device, image decoding device: other methods)
7. Seventh embodiment (multivision image encoding device, multivision image decoding device)
8. Eighth embodiment (layered image encoding device, layered image decoding device)
9. Ninth embodiment (computer)
10. Sample applications
11. Sample applications for scalable coding
1. First embodiment [00100] In this embodiment, a description will be given of an exemplary application of the present technology, which will be described in detail in the second and subsequent embodiments thereof.
1-1. Exemplary application of the present technology
Petition 870170066495, of September 6, 2017, p. 25/294 / 137 [00101] First, an exemplary example in which the present technology is applicable will be described. The present technology is a technology related to the encoding and decoding of a scale list used in quantization and decanting processes performed when image data is encoded and decoded.
[00102] The encoding and decoding of image data may involve quantizing and decanting coefficient data. Such quantization and decanting are performed in units of a block having a predetermined size, and a scale list (or quantization matrix) having a size corresponding to the block size is used. For example, in HEVC (High Efficiency Video Coding), quantization (or decanting) is performed with sizes such as 4x4, 8x8, 16x16 and 32x32. In HEVC, quantization matrices having sizes 4x4 and 8x8 can be prepared.
[00103] Figure 1 illustrates an example of an 8x8 scale list. As illustrated in Figure 1, a scale list includes a DC coefficient and AC coefficients. The CC coefficient composed of a value is the coefficient (0, 0) of a quantization matrix, and corresponds to the CC coefficient of a discrete cosine transform (DCT). The AC coefficients are coefficients of the quantization matrix different from the coefficient (0, 0), and correspond to DCT coefficients different from the CC coefficient. Note that, as shown in Figure 1, the AC coefficients are represented by a matrix. That is, the AC coefficients also include the coefficient (0, 0) (hereinafter also called the AC coefficient (0, 0)), and the coefficient (0, 0), which is located at the beginning of the quantization matrix, is replaced with the CC coefficient when used for quantization / decanting. Consequently, the CC coefficient is also called a substitution coefficient. In the example illustrated in Figure 1, AC coefficients form an 8x8 matrix.
[00104] In HEVC, moreover, a version converted by conversion
Petition 870170066495, of September 6, 2017, p. 26/294 / 137 upward (upward conversion) of an 8x8 quantization matrix is used for 16x16 or 32x32 quantization (or decanting).
[00105] Figure 2 illustrates an example of the upward conversion of an 8x8 scale list to a 16x16 scale list. As illustrated in Figure 2, a scale list is converted by upward conversion using, for example, a closer neighbor interpolation process. The details of the nearest neighbor interpolation process will be described below with reference, for example, to Figure 25. As illustrated in Figure 2, upward conversion is performed on the AC coefficients of the scale list. Then, the coefficient (0, 0) between the AC coefficients converted by upward conversion is replaced with the CC coefficient.
[00106] Two types of 8x8 scale lists are prepared, that is, the one used for upward conversion to 16x16 (8x8 to 16x16) and the one used for upward conversion to 32x32 (8x8 to 32x32).
[00107] The scale list used for quantization during encoding (using an encoder) is also used for decanting during decoding (using a decoder). That is, the scale list is transmitted from the encoding side (encoder) to the decoding side (decoder). Figure 3 illustrates an example of transmitting scale lists.
[00108] As in the example illustrated in Figure 3, the two types of 8x8 scale lists, that is, the one used for upward conversion to a 16x16 size and the one used for upward conversion to a 32x32 size, as described above, are transmitted. Although not shown in the drawings, a 4x4 scale list is also transmitted.
[00109] The AC coefficients of the 8x8 scale list used for upward conversion to a 16x16 size, which was transmitted in the manner described above, are converted by upward conversion to 16x16 size on the decoding side (the decoder) using the interpolation process nearest neighbor described above, and are used for decanting
Petition 870170066495, of September 6, 2017, p. 27/294 / 137 of a block having a size 16x16 after the coefficient (0, 0) is replaced with the CC coefficient.
[00110] Similarly, the CA coefficients of the 8x8 scale list used for upward conversion to a 32x32 size, which were transmitted as described above, are also converted by upward conversion to 32x32 size on the decoding side (the decoder) using the nearest neighbor interpolation process described above, and are used for the decanting of a block having a size 32x32 after the coefficient (0, 0) is replaced with the CC coefficient.
1-2. Scale list encoding [00111] Transmitting scale lists in the manner described above will increase the amount of encoding, therefore. Thus, in order to suppress a reduction in coding efficiency, the scale lists are coded using a certain method to reduce the amount of coding of the scale lists. Figure 4 illustrates an example of coding a scale list. Specifically, an 8x8 scale list is transmitted as follows. [00112] In the case of upward conversion from an 8x8 matrix to a 16x16 matrix:
(1) A difference between the coefficient (0, 0) (that is, the coefficient of CA (0, 0)) of the matrix 8x8 and an initial predetermined value 8 is taken.
(2) Differences between coefficients (that is, AC coefficients) (adjacent coefficients in a sequence of coefficients arranged unidimensionally in order of scanning) of the 8x8 matrix are taken.
(3) A difference between the coefficient (0, 0) (that is, the DC coefficient) of the 16x16 matrix and an initial predetermined value 8 is taken.
(4) The differences obtained in (1) and (2) and the difference obtained in (3) are transmitted separately.
Petition 870170066495, of September 6, 2017, p. 28/294 / 137 [00113] In the case of upward conversion from an 8x8 matrix to a 32x32 matrix:
(1) A difference between the coefficient (0, 0) (that is, the AC coefficient (0, 0)) of the matrix 8x8 and a predetermined initial value 8 is taken.
(2) Differences between coefficients (that is, AC coefficients) (adjacent coefficients in a sequence of coefficients arranged unidimensionally in order of scanning) of the 8x8 matrix are taken.
(3) A difference between the coefficient (0, 0) (that is, the DC coefficient) of the 32x32 matrix and an initial predetermined value 8 is taken.
(4) The differences obtained in (1) and (2) and the difference obtained in (3) are transmitted separately.
[00114] In the method described above, however, the differences are encoded using signed exponential Golomb encoding and are transmitted in (4). As described above, the difference obtained in (1) is the difference between the AC coefficient (0, 0) and the initial value 8. Thus, there is a concern that the amount of coding can be increased if the value of the AC coefficient (0, 0) is not a value close to the initial value 8.
[00115] For example, in Figure 4, the value of the AC coefficient (0, 0) is 12, and the value 4 is encoded using signed exponential Golomb encoding and is transmitted as the difference obtained in (1). That is, 7 bits are required for the transmission of the difference obtained in (1) and coding efficiency can be reduced accordingly. If the value of the difference obtained in (1) increases, coding efficiency can be further reduced. The same is true of an 8x8 scale list used for upward conversion to a 16x16 size and an 8x8 scale list used for upward conversion to a 32x32 size.
[00116] Meanwhile, the energy of DCT coefficients is usually concentrated in the DC coefficient and low order coefficients
Petition 870170066495, of September 6, 2017, p. 29/294 / 137 neighbors. Therefore, in general, a quantization matrix also has small values for the CC coefficient and neighboring coefficients. In addition, if values that are significantly different are used for individual frequencies, a quantization error can be perceived subjectively. In order to suppress such visual deterioration in image quality, consecutive values are used for the CC coefficient and neighboring coefficients. [00117] The coefficient (0, 1), coefficient (1.0) coefficient and (1.1) obtained after upward conversion correspond to the coefficient of CA (0, 0) before upward conversion. In addition, the coefficient (0, 0) obtained after upward conversion corresponds to the CC coefficient.
[00118] Thus, in scale lists, the AC coefficient value (0, 0) and the CC coefficient value are generally close to each other. For example, standard MPEG2, AVC, and HEVC arrays take values having such a relationship. Also in the example illustrated in Figure 4, the CC coefficient value is equal to the AC coefficient value (0, 0), that is, 12. Thus, the difference value obtained in (3), that is, the difference between the CC coefficient and the initial value 8, it is also 4.
That is, taking a difference between each of the DC coefficient and the AC coefficient (0, 0), whose values are close to each other, and the initial value can increase the difference value between them, and can also cause redundancy. . It can be said that there will be a risk of further reducing coding efficiency.
[00119] To address this, a scale list is transmitted using the following method instead of using the method illustrated in Figure 4. Figure 5 illustrates an example of this method.
[00120] In the case of upward conversion from an 8x8 matrix to a 16x16 matrix:
(1) A difference between the coefficient (0, 0) (that is, the AC coefficient (0, 0)) of the 8x8 matrix and the coefficient (0, 0) (that is, the CC coefficient) of the 16x16 matrix is taken.
Petition 870170066495, of September 6, 2017, p. 30/294 / 137 (2) Differences between coefficients (that is, AC coefficients) (adjacent coefficients in a sequence of coefficients arranged unidimensionally in order of scanning) of the 8x8 matrix are taken.
(3) A difference between the coefficient (0, 0) (that is, the DC coefficient) of the 16x16 matrix and a predetermined initial value 8 is taken.
(4) The differences obtained in (1) to (3) are transmitted collectively.
[00121] In the case of upward conversion from an 8x8 matrix to a 32x32 matrix:
(1) A difference between the coefficient (0, 0) (that is, the AC coefficient (0, 0)) of the 8x8 matrix and the coefficient (0, 0) (that is, the CC coefficient) of the 32x32 matrix is taken.
(2) Differences between coefficients (that is, AC coefficients) (adjacent coefficients in a sequence of coefficients arranged unidimensionally in order of scanning) of the 8x8 matrix are taken.
(3) A difference between the coefficient (0, 0) (that is, the DC coefficient) of the 32x32 matrix and an initial predetermined value 8 is taken.
(4) The differences obtained in (1) to (3) are transmitted collectively.
[00122] Similar to the method illustrated in Figure 4, in (4), the differences are coded using exponential Golomb coding and are transmitted as exponential Golomb codes.
[00123] At the destination to which the differences are transmitted as exponential Golomb codes, when the exponential Golomb codes are received, the exponential Golomb codes received are decoded to obtain the individual differences, and the reverse processes to those in (1) a (3) described above are performed on the differences obtained to determine the individual coefficients (the CC coefficient and the
Petition 870170066495, of September 6, 2017, p. 31/294 / 137 AC coefficients).
1-3. Exemplary characteristics of the present technology [00124] Exemplary characteristics of the present technology related to the transmission method described above will now be described.
1-3-1. DPCM between AC coefficient (0, 0) and DC coefficient [00125] Scale lists are coded using differential pulse code modulation (DPCM) and are transmitted. In the example shown in Figure 4, the AC coefficients and the CC coefficient are encoded in DPCM individually, while, according to one of the characteristics of the present technology, as in the example illustrated in Figure 5, a difference (also called a coefficient of substitution difference) between the AC coefficient (0, 0) and the DC coefficient is determined and transmitted. [00126] As described above, the AC coefficient (0, 0) and the DC coefficient generally take values that are close to each other. Thus, a difference between the AC coefficient (0, 0) and the DC coefficient may possibly be less than a difference between the AC coefficient (0, 0) and the initial value 8. That is, the transmission of a substitution difference coefficient which is a difference between the AC coefficient (0, 0) and the DC coefficient using the present technology may be more likely to reduce the amount of encoding.
[00127] For example, in the example shown in Figure 5, the value of the difference obtained in (1) is 0.
[00128] Figure 6 is a table illustrating an example of signed exponential Golomb coding. As indicated in the table illustrated in Figure 6, the exponential Golomb code for the value 4 has a code length of 7 bits while the Golomb code exponential for the value 0 has a code length of 1 bit. That is, the method illustrated in Figure 5 can reduce the amount of encoding by 6 bits compared to the method illustrated in Figure 4.
[00129] In general, an encoding amount of approximately
Petition 870170066495, of September 6, 2017, p. 32/294 / 137
100 bits to 200 bits is required for the transmission of a quantization matrix having an 8x8 size. Consequently, 6 bits occupy approximately 6% of the total amount. A 6% reduction in the amount of coding in High Level Syntax can be said to be a very large effect.
1-3-2. Collective transmission of DC coefficient and AC coefficients [00130] Figure 7 illustrates an example of the syntax of a scale list. The syntax for the example illustrated in Figure 4 is illustrated in an example illustrated in part A of Figure 7. Specifically, after the difference between the AC coefficient (0, 0) and the initial value 8 and the differences between the coefficients of CA (scaling_list_delta_coef) are transmitted, the difference between the CC coefficient and the initial value 8 (scaling_list_CC_coef_minus8) is transmitted separately.
[00131] In contrast, one of the characteristics of the present technology is that the difference between the DC coefficient and the AC coefficient (0, 0) and the differences between the AC coefficients are arranged in this order and are transmitted collectively. Specifically, as illustrated in Figure 5, after the DC coefficient and the AC coefficients arranged in a predetermined scan order are arranged one-dimensionally and the difference between the CC coefficient and the initial value 8 is determined, the differences between adjacent coefficients following coefficients are determined. Additionally, the resulting differences (differences between coefficients) are arranged unidimensionally in the order of being obtained and are transmitted collectively.
[00132] The syntax in this case is illustrated in an example in part B of Figure 7. Specifically, initially, the difference between the CC coefficient and the initial value 8 (scaling_list_CC_coef_minus8) is transmitted, and then the difference between the CC coefficient and the AC coefficient (0, 0) and the differences between the AC coefficients (scaling_list_delta_coef) are transmitted. I mean, the DC coefficient and the AC coefficients are
Petition 870170066495, of September 6, 2017, p. 33/294 / 137 collectively encoded and transmitted.
[00133] In this way, the collective transmission of the differences arranged in the order of being obtained allows the decoding side (the decoder) to which the differences are transmitted to decode the differences in the order of being transmitted and to obtain the individual coefficients. That is, a scale list encoded in DPCM can be decoded easily. More specifically, the processing load can be reduced. In addition, rearrangement of differences is no longer necessary, resulting in a reduction in temporary memory capacity. In addition, the respective differences can be decoded in the order they are provided, resulting in the suppression of an increase in processing time.
1-3-3. Standard matrix transmission [00134] Figure 8 is a diagram illustrating an example of the syntax for the transmission of a standard matrix. In the related technique, as illustrated in Figure 8, the initial coefficient (that is, the CC coefficient) is transmitted as 0 to transmit information indicating the use of a standard matrix. That is, the value of the difference between the CC coefficient and the initial value 8 (scaling_list_CC_coef_minus8) is -8. However, as illustrated in Figure 6, the exponential Golomb code for the value -8 has a code length of 9 bits. That is, there is a concern that coding efficiency can be significantly reduced. In general, it is desirable that the number of High Level Syntax bits be as small as possible. In addition, as illustrated in Figure 8, due to the increased complexity of the syntax, the processing load can be increased.
[00135] To address these issues, the initial coefficient is not set at 0, but the semantics of scaling_list_pred_matrix_id_delta are modified. More specifically, the semantics of scaling_list_pred_matrix_id_delta is modified from that illustrated in part A
Petition 870170066495, of September 6, 2017, p. 34/294 / 137 of Figure 9 to that illustrated in part B of Figure 9. That is to say, in the related technique, as illustrated in part A of Figure 9, the value equal to 0 indicates that the matrix immediately preceding (MatrixID - 1) is referred to. Instead of this description, as illustrated in part B of Figure 9, the scaling_list_pred_matrix_id_delta value of 0 means that a standard matrix is referred to.
[00136] Therefore, the code length of an exponential Golomb code for the transmission of information indicating the use of a standard matrix can be equal to 1 bit, and a reduction in coding efficiency can be suppressed. In addition, in the related technique, syntax as illustrated in parts A and B of Figure 10 is required for a scale list. This syntax can be simplified as in an example illustrated in Figure 11. That is, the processing load involved in encoding and decoding a scale list can be reduced.
-4. Syntax characteristics using the present technology
Syntax will be described more specifically.
[00137] In the example of the related technique illustrated in parts A and B of Figure 10, the prefix determination needs to be performed twice, that is, scaling_list_CC_coef_minus8 and scaling_list_delta_coef. In addition, for scaling_list_delta_coef, determination is made in the middle of the loop for, and the loop comes out when useDefaultScalingMatrixFlag = 1. In addition, an intermediate flag called stopNow is needed, and, because of this condition, an extension such as replacing nextCoef in value scalingList additionally exists. In this way, the syntax of the related technique involves complicated processing.
[00138] In the present technology, therefore, as in the example illustrated in Figure 11, the calculated CC coefficient of scaling_list_CC_coef_minus8 is replaced in nextCoef to establish the initial value of scaling_list_delta_coef to the CC coefficient.
[00139] Furthermore, in semantics, the value of
Petition 870170066495, of September 6, 2017, p. 35/294 / 137 scaling_list_pred_matrix_id_delta, which is represented by +1 in the related technique, remains unchanged, and the value 0 is used as a special value.
[00140] I mean, in the related technique, when ScalingList [0] [2] is to be decoded (matrixId = 2), if scaling_list_pred_matrix_id_delta = 0, then matrixId = 2 is obtained from refMatrixId = matrixId - (1+ scaling_list_pred_matrix_id_del. Thus, refMatrixId = 1 is obtained, and the value of ScalingList [0] [1] is copied.
[00141] In contrast, in the present technology, refMatrixId = matrixId scaling_list_pred_matrix_id_delta is established. When ScalingList [0] [2] is to be decoded (matrixId = 2), scaling_list_pred_matrix_id_delta = 1 can be established if ScalingList [0] [1] is to be copied (or if refMatrixId = 1 is to be obtained).
[00142] Therefore, as illustrated in Figure 11, the number of syntax queues for a scale list can be reduced significantly. In addition, two variables to be included as intermediate data, that is, UseDefaultScalingMatrix and stopNow, can be omitted. In addition, branch made to the mesh for as illustrated in Figure 10 may no longer be required. Therefore, the processing load involved in encoding and decoding a scale list can be reduced.
1-5. Processing units implementing the present technology [00143] In a case where the present technology is applied to the transmission of a scale list, a scale list is encoded and decoded in the manner described above. Specifically, an image encoding device 10 described below with reference to Figure 14 encodes a scale list and transmits the encoded scale list, and an image decoding device 300 described below with reference to Figure 22 receives and decodes the image list. coded scale.
[00144] A scale list is encoded by a matrix processing unit 150 (Figure 15) in a transform unit
Petition 870170066495, of September 6, 2017, p. 36/294 / 137 orthogonal / quantization 14 (Figure 14) of the image encoding device 10. More specifically, a scale list is encoded by a DPCM 192 unit and an exp-G 193 unit (both are illustrated in Figure 16) in an entropy coding unit 164 (Figure 16) in the matrix processing unit 150. That is, the DPCM unit 192 determines differences between coefficients (the CC coefficient and the AC coefficients) of the scale list, and the exp-G 193 unit encodes individual differences using exponential Golomb encoding.
[00145] In order to encode a scale list using the present technology as described above, the DPCM 192 unit can have an example configuration as illustrated, for example, in Figure 19, and can perform a DPCM process as in a example illustrated in Figure 21. In addition, semantics as in an illustrated example part C of Figure 44 or part C of Figure 45 can be used.
[00146] In other words, only the DPCM 192 unit and the exp-G 193 unit can be required to achieve encoding a scale list using the present technology, and other components having any configuration can be used as desired. A necessary configuration, such as a processing unit for converting a scale list upwardly and a processing unit for performing quantization using a scale list, can be provided according to embodiments.
[00147] In addition, a scale list is decoded by a matrix generation unit 410 (Figure 23) into a reverse orthogonal decanting / transforming unit 313 (Figure 22) of the image decoding device 300. More specifically, a scale list is decoded by an exp-G 551 unit and an inverse DPCM unit 552 (Figure 24) into an entropy decoding unit 533 (Figure 24) in the matrix generation unit 410. That is, the exp-G 551 decodes the Golomb code for differences, and the DPCM unit
Petition 870170066495, of September 6, 2017, p. 37/294 / 137 inverse 552 determines individual coefficients (the DC coefficient and the AC coefficients) from the scale list of the respective differences.
[00148] In order to decode a coded scale list using the present technology as described above, the reverse DPCM unit 552 can have an example configuration as illustrated, for example, in Figure 26, and can perform a reverse DPCM process as in an example illustrated in Figure 29. In addition, semantics as in an example illustrated in part C of Figure 44 or part C of Figure 45 can be used. [00149] In other words, only the exp-G 551 unit and reverse DPCM unit 552 can be required to achieve decoding a scale list using the present technology, and other components having any configuration can be used as desired. A necessary configuration, such as a processing unit for converting a scale list upwardly and a processing unit for performing decanting using a scale list, can be provided according to embodiments.
[00150] Individual embodiments to which the present technology is applied will now be described for a more detailed description of the present technology.
2. Second embodiment
2-1. Syntax: First method (1) Syntax of the related technique [00151] First, Figure 12 illustrates an example of the syntax of a quantization matrix (or scale list) in the related technique. In actual use, a difference matrix between a scale list and a prediction matrix for that, instead of the scale list, is usually transmitted. Thus, in the following syntax description and so on, it is assumed that the description of a scale list can also apply to a difference matrix.
[00152] Part A of Figure 12 illustrates the syntax for list data
Petition 870170066495, of September 6, 2017, p. 38/294 / 137 scale (scale list data syntax), and part B of Figure 12 illustrates the scale list syntax (scale list syntax).
(1-1) Scale list data syntax [00153] As illustrated in Part A of Figure 12, the syntax for scale list data specifies that a flag (scaling_list_present_flag) indicating whether or not a scale list is provided, a flag (scaling_list_pred_mode_flag) indicating whether or not the current mode is a copy mode, information (scaling_list_pred_matrix_id_delta) indicating which scale list to refer to in copy mode, and so on. (1-2) Scaling list syntax [00154] As illustrated in part B of Figure 12, the scaling list syntax specifies that the CC coefficient from which a constant (for example, 8) is subtracted (scaling_list_CC_coef_minus8) , a difference value (scaling_list_delta_coef) between AC coefficients, and so on are read and that the DC coefficient and the AC coefficients are restored.
[00155] However, there is a concern that the pieces of syntax described above will not provide sufficient compression efficiency of the CC coefficient although it facilitates processes.
[00156] Therefore, in order to obtain sufficient compression efficiency of a DC coefficient (also called a direct current coefficient), which is the coefficient of the DC component (direct current component), a difference between the coefficient of CC and another coefficient is determined, and the difference value is transmitted instead of the CC coefficient. That is, the difference value is information to calculate the CC coefficient, and, in other words, it is substantially equivalent to the CC coefficient. However, the difference value is generally less than the CC coefficient. Therefore, transmitting the difference value instead of the CC coefficient can result in a reduction in the amount of encoding.
Petition 870170066495, of September 6, 2017, p. 39/294 / 137 [00157] In the following description, for convenience of description, a scale list (quantization matrix) is 8x8 in size. A specific example of the method for transmitting a difference between the CC coefficient and another coefficient, instead of the CC coefficient, described above will be described hereinafter.
(2) Syntax for first method [00158] For example, 65 coefficients can be transmitted using DPCM (Differential Pulse Code Modulation), where the CC coefficient is considered as the element located at the beginning of an 8x8 matrix (AC coefficients ) (first method).
[00159] I mean, first, a difference between a predetermined constant and the CC coefficient is calculated, and is used as the initial DPCM data coefficient. Then, a difference between the CC coefficient and the initial AC coefficient is calculated, and is used as the second coefficient of the DPCM data. Then, a difference between the initial AC coefficient and the second AC coefficient is calculated, and is used as the third coefficient of the DPCM data. Subsequently, a difference from the immediately preceding AC coefficient is calculated, and is used as the fourth coefficient of the DPCM data, and the following coefficients of the DPCM data are determined in a similar manner to that described above. The DPCM data coefficients generated in the manner described above are transmitted sequentially, starting from the initial coefficient.
[00160] Therefore, the compression ratio can be improved when the coefficient values (0, 0) (AC coefficient) of an 8x8 matrix and the CC coefficient are close to each other. By implementing the first method described above, an image encoding device can process the DC coefficient in a manner similar to that of AC coefficients (alternating current coefficients), which are the coefficients of the AC components (also called the current components
Petition 870170066495, of September 6, 2017, p. 40/294 / 137 alternate). Note that, in order to implement the first method described above, an image decoding device to which the coefficients described above are transmitted needs to operate especially only the initial coefficient. Specifically, the image decoding device needs to extract the DC coefficient from among the AC coefficients.
[00161] Figure 13 illustrates the syntax of a scale list in the case described above. In the example illustrated in Figure 13, 65 difference values (scaling_list_delta_coef) between coefficients are read, and, between coefficients (nextcoef) determined from the difference values, the coefficient (nextcoef) located at the beginning is used as the CC coefficient (scaling_list_CC_coef) while the other coefficients are used as the AC coefficients (ScalingList [i]).
[00162] An image encoding device that implements the syntax for the first method described above will be described below.
2-2. Image encoding device [00163] Figure 14 is a block diagram illustrating an example configuration of an image encoding device 10 according to an embodiment of the present exposure. The image encoding device 10 illustrated in Figure 14 is an image processing device to which the present technology is applied and which is configured to encode input image data and produce the encoded image data. Referring to Figure 14, the image encoding device 10 includes an A / D (Analog to Digital) conversion unit 11 (A / D), a rearrangement buffer 12, a subtraction unit 13, a transform unit orthogonal / quantization 14, a lossless coding unit 16, an accumulation buffer 17, a rate control unit 18, a decanting unit 21, an inverse orthogonal transform unit 22, a summation unit 23, a filter 'deblocking' 24, a frame memory 25, a selector 26, an intraprediction unit 30, a search unit
Petition 870170066495, of September 6, 2017, p. 41/294 / 137 of movement 40 and a mode selection unit 50.
[00164] The A / D conversion unit 11 converts an image signal input in analog form to image data in digital form, and produces a sequence of digital image data for the rearrangement buffer 12.
[00165] The rearrangement buffer 12 rearranges images included in the image data sequence input from the A / D conversion unit 11. After rearranging the images according to a GOP (Group of Frames) structure for use in an encoding process, the rearrangement buffer 12 produces the image data in which the images have been rearranged to the subtraction unit 13, the intraprediction unit 30 and the motion search unit 40.
[00166] The subtraction unit 13 is provided with the image data entered from the rearrangement buffer 12 and the prediction image data selected by the mode selection unit 50, which will be described below. The subtraction unit 13 calculates prediction error data that represents the difference between the image data input from the rearrangement buffer 12 and the prediction image data input from the mode selection unit 50, and produces the error data from prediction calculated for the orthogonal transform / quantization unit 14.
The orthogonal transform / quantization unit 14 performs an orthogonal transform and quantization on the prediction error data entered from the subtraction unit 13, and produces quantized transform coefficient data (hereinafter called quantized data) for the lossless coding unit 16 and the decanting unit 21. The bit rate of the quantized data produced from the orthogonal transform / quantization unit 14 is controlled according to a rate control signal provided by the rate control unit 18. A detailed configuration of the orthogonal transform / quantization unit 14 will be further described below.
Petition 870170066495, of September 6, 2017, p. 42/294 / 137 [00167] The lossless coding unit 16 is provided with the quantized data entered from the orthogonal transform / quantization unit 14, information for generating a scale list (or quantization matrix) on the decoding side, and information regarding intraprediction or interpretation which is selected by the mode selection unit 50. Information regarding intraprediction may include, for example, prediction mode information indicating an optimal intraprediction mode for each block. In addition, interpreting information can include, for example, prediction mode information for block-by-block prediction of motion vectors, differential motion vector information, reference image information, and so on. In addition, the information for generating a scale list on the decoding side can include identifying information indicating a maximum size of a scale list to be transmitted (or a difference matrix between a scale list (quantization matrix) and a prediction matrix).
[00168] The lossless coding unit 16 performs a lossless coding process on the quantized data to generate an encoded stream. The lossless coding performed by the lossless coding unit 16 can be, for example, variable length coding, arithmetic coding, or the like. In addition, the lossless coding unit 16 multiplexes the information to generate a header scale list (e.g., a sequence parameter set and a frame parameter set) of the encoded stream. The lossless coding unit 16 additionally multiplexes the information regarding the intraprediction or interpretation described above in the header of the coded stream. Thereafter, the lossless coding unit 16 produces the generated coded stream to the accumulation buffer 17.
[00169] The accumulation buffer 17 temporarily accumulates the coded stream entered from the coding unit without
Petition 870170066495, of September 6, 2017, p. 43/294 / 137 loss 16, using a storage medium such as a semiconductor memory. Thereafter, the accumulation buffer 17 produces the encoded stream accumulated at a rate corresponding to the bandwidth of a transmission path (or an output line of the image encoding device 10).
[00170] Rate control unit 18 monitors accumulation buffer 17 to check capacity availability. The rate control unit 18 generates a rate control signal according to the available capacity of the accumulation buffer 17, and produces the rate control signal generated for the orthogonal transform / quantization unit 14. For example, when the capacity available from accumulation buffer 17 is low, the rate control unit 18 generates a rate control signal to reduce the bit rate of the quantized data. Alternatively, for example, when the available capacity of the accumulation buffer 17 is high enough, the rate control unit 18 generates a rate control signal to increase the bit rate of the quantized data.
[00171] The decanting unit 21 performs a decanting process on the quantized data entered from the orthogonal transform / quantization unit 14. Thereafter, the decanting unit 21 produces transform coefficient data acquired by the decanting process from the orthogonal transform unit. reverse 22.
[00172] The inverse orthogonal transform unit 22 performs an inverse orthogonal transform process on the transform coefficient data entered from the decanting unit 21 to restore prediction error data. Thereafter, the inverse orthogonal transform unit 22 produces the prediction error data restored to the summation unit 23.
[00173] Summing unit 23 adds together the restored prediction error data input from the inverse orthogonal transform unit
Petition 870170066495, of September 6, 2017, p. 44/294 / 137 and the prediction image data input from the mode selection unit 50 to generate decoded image data. Thereafter, the summation unit 23 produces the decoded image data generated for the deblocking filter 24 and frame memory 25.
[00174] The 'deblocking' filter 24 performs a filtering process to reduce blocking artifacts caused by the encoding of an image. The deblocking filter 24 filters the decoded image data input from the adding unit 23 to remove (or at least reduce) block forming artifacts, and produces the filtered decoded image data to the frame memory 25.
[00175] Frame memory 25 stores the decoded image data input from the adding unit 23 and the filtered decoded image data input from the deblocking filter 24, using a storage medium.
[00176] Selector 26 reads decoded image data to be filtered, which is used for intraprediction, from frame memory 25, and provides the decoded image data read to intraprediction unit 30 as reference image data. Selector 26 additionally reads filtered decoded image data, which is used in interpretation, from frame memory 25, and provides the decoded image data read to motion search unit 40 as reference image data.
[00177] The intraprediction unit 30 performs an intraprediction process in each intraprediction mode on the basis of the image data to be encoded which are entered from the rearrangement buffer 12 and the decoded image data provided by the selector 26. For example, the intraprediction unit 30 evaluates a prediction result obtained in each intraprediction mode using a predetermined cost function. Then, the intraprediction unit 30 selects an intraprediction mode that minimizes the cost function value, that is, an intraprediction mode that provides the highest compression ratio, such as an intraprediction mode
Petition 870170066495, of September 6, 2017, p. 45/294 / 137 great. In addition, the intraprediction unit 30 produces prediction mode information indicating the optimal intraprediction mode, prediction image data, and intraprediction information, such as the cost function value, for the mode selection unit 50 .
[00178] The motion search unit 40 performs an interpretation process (or an interframe prediction process) on the basis of the image data to be encoded, which are input from the rearrangement buffer 12 and the decoded image data provided by the selector 26. For example, motion search unit 40 evaluates a prediction result obtained in each prediction mode using a predetermined cost function. Then, the motion search unit 40 selects a prediction mode that minimizes the cost function value, that is, a prediction mode that provides the highest compression ratio, as an optimal prediction mode. In addition, the motion search unit 40 generates prediction image data according to the optimal prediction mode. The motion search unit 40 produces information on interpretation which includes prediction mode information indicating the selected optimal prediction mode, the prediction image data, and information on interpretation, such as the cost function value, for the mode selection unit 50.
[00179] The mode selection unit 50 compares the cost function value for intraprediction, which is entered from intraprediction unit 30, with the cost function value for interpretition, which is entered from motion search unit 40. Then, the mode selection unit 50 selects a prediction technique having the lowest of the cost function values for intraprediction and interpredition. If intraprediction is selected, the mode selection unit 50 produces the intraprediction information for the lossless coding unit 16, and also produces the prediction image data for the subtraction unit 13 and the summation unit 23. Alternatively, if interpretition is selected, the unit of selection of
Petition 870170066495, of September 6, 2017, p. 46/294 / 137 mode 50 produces the information relating to the interpretation described above for the lossless coding unit 16, and also produces the prediction image data for the subtraction unit 13 and the summation unit 23.
2-3. Example configuration of orthogonal transform / quantization unit [00180] Figure 15 is a block diagram illustrating an example of a detailed configuration of the orthogonal transform / quantization unit 14 of image encoding device 10 illustrated in Figure 14. Referring to In Figure 15, the orthogonal transform / quantization unit 14 includes a selection unit 110, an orthogonal transform unit 120, a quantization unit 130, a scale list buffer 140 and a matrix processing unit 150.
(1) Selection unit [00181] Selection unit 110 selects a transform unit (TU) to be used for the orthogonal transformation of image data to be encoded from a plurality of transform units having different sizes. Examples of possible sizes of transform units selectable by selection unit 110 include 4x4 and 8x8 for H.264 / AVC (Advanced Video Encoding), and include 4x4, 8x8, 16x16 and 32x32 for HEVC (High Efficiency Video Encoding) . The selection unit 110 can select a transform unit according to, for example, the size or quality of an image to be encoded, the performance of the image encoding device 10, or the like. The selection of a transformed unit by the selection unit 110 can be fine-tuned by hand by a user who develops the image encoding device 10. After that, the selection unit 110 produces information that specifies the size of the selected transformation unit to the orthogonal transform unit 120, quantization unit 130, lossless coding unit 16 and
Petition 870170066495, of September 6, 2017, p. 47/294 / 137 decanting unit 21.
(2) Orthogonal transform unit [00182] The orthogonal transform unit 120 performs an orthogonal transform on the image data (that is, prediction error data) provided by the subtraction unit 13, in units of the transform unit selected by the unit selection 110. The orthogonal transform performed by the orthogonal transform unit 120 can, for example, be a discrete cosine transform (DCT), a Karhunen-Loève transform, or the like. Thereafter, the orthogonal transform unit 120 produces transform coefficient data acquired by the orthogonal transform process to the quantization unit 130.
(3) Quantization unit [00183] Quantization unit 130 quantizes the transform coefficient data generated by the orthogonal transform unit 120, using a scale list corresponding to the transform unit selected by the selection unit 110. In addition, the quantization unit 130 changes the quantization step size according to the rate control signal provided by the rate control unit 18 to change the bit rate of the quantized data to be produced.
[00184] Furthermore, the quantization unit 130 makes sets of scale lists corresponding respectively to a plurality of transform units selectable by the selection unit 110 to be stored in the scale list buffer 140. For example, as in HEVC, if there are four possible sizes of transform units, that is, 4x4, 8x8, 16x16 and 32x32, four sets of scale lists corresponding to the four sizes respectively can be stored in the scale list temporary memory 140. Note that if a list of specified scale is used for a given size, only a flag indicating that the specified scale list is used (a user-defined scale list is not used) can be stored in memory
Petition 870170066495, of September 6, 2017, p. 48/294 / 137 scale list 140 in association with the given size.
[00185] A set of scale lists that can be used by the quantization unit 130 can be established typically for each sequence of the encoded flow. In addition, the quantization unit 130 can update a set of scale lists that is established for each sequence on a frame-by-frame basis. Information to control the establishment and updating of a set of scale lists can be inserted, for example, in a sequence parameter set and a frame parameter set.
(4) Scale list buffer [00186] Scale list buffer 140 temporarily stores a set of scale lists corresponding respectively to a plurality of transform units selectable by selection unit 110, using a storage medium such as like a semiconductor memory. The set of scale lists stored in the scale list buffer 140 is referred to when the matrix processing unit 150 performs a process described below.
(5) Matrix processing unit [00187] The matrix processing unit 150 encodes a scale list to be used for encoding (quantization). Thereafter, the encoded scale list data (hereinafter called encoded scale list data) generated by the matrix processing unit 150 is produced to the lossless encoding unit 16, and can be inserted into the header of the encoded stream.
2-4. Detailed example configuration of matrix processing unit [00188] Figure 16 is a block diagram illustrating an example of a more detailed configuration of matrix processing unit 150. Referring to Figure 16, matrix processing unit 150 includes a prediction unit 161, a generation matrix generation unit
Petition 870170066495, of September 6, 2017, p. 49/294 / 137 difference 162, a difference matrix size transformation unit 163, an entropy coding unit 164, a decoding unit 165 and an output unit 166.
(1) Prediction unit [00189] Prediction unit 161 generates a prediction matrix. As illustrated in Figure 16, the prediction unit 161 includes a copy unit 171 and a matrix generation prediction unit 172.
[00190] In a copy mode, copy unit 171 copies a previously transmitted scale list, and uses the copied quantization matrix as a prediction matrix (or predicts a scale list of an orthogonal transform unit to be processed ). More specifically, copy unit 171 acquires the size and list ID (ListID) of a scale list previously transmitted from a storage unit 202 in decoding unit 165. The size is information indicating the size of the scale list (varying from, for example, 4x4 to 32x32). The list ID is information indicating the type of prediction error data to be quantized.
[00191] For example, the list ID includes identification information indicating that the prediction error data to be quantized is prediction error data (Intra-Luma) of the luminance component that is generated using a prediction image subject to intraprediction, prediction error data (Intra-Cr) of the color difference component (Cr) that is generated using a prediction image subject to intraprediction, prediction error data (Intra-Cb) of the color difference component ( Cb) which is generated using a prediction image subject to intraprediction, or prediction error data (Inter-Luma) of the luminance component that is generated using a prediction image subject to interpretation.
[00192] Copy unit 171 selects, as a scale list to be copied, a previously transmitted scale list of the same size as the scale list (scale list of a transformed unit
Petition 870170066495, of September 6, 2017, p. 50/294 / 137 orthogonal to be processed) input to matrix processing unit 150, and provides the list ID of the scale list to be copied to output unit 166 to produce the list ID for devices outside the processing unit matrix 150 (lossless coding unit 16 and decanting unit 21). That is, in this case, only the list ID is transmitted next to the decoding (or is included in encoded data) as information indicating a prediction matrix generated by copying the previously transmitted scale list. Thus, the image encoding device 10 can suppress an increase in the amount of encoding in a scale list.
[00193] In addition, in a normal mode, matrix generation prediction unit 172 acquires a scale list previously transmitted from storage unit 202 in decoding unit 165, and generates a prediction matrix using the scale list ( or predicts a scale list of an orthogonal transform unit to be processed). The matrix generation prediction unit 172 provides the prediction matrix generated for the difference matrix generation unit 162.
(2) Difference matrix generation unit [00194] The difference matrix generation unit 162 generates a difference matrix (residual matrix) that is a difference between the prediction matrix provided by the prediction unit 161 (the matrix generation prediction 172) and the scale list entered to matrix processing unit 150. As illustrated in Figure 16, the difference matrix generation unit 162 includes a prediction matrix size transformation unit 181, a computing unit 182 and quantizing unit 183.
[00195] The prediction matrix size transformation unit 181 transforms (hereinafter also called convert) the prediction matrix size provided by the matrix generation prediction unit 172 so that the prediction matrix size matches the size from the list
Petition 870170066495, of September 6, 2017, p. 51/294 / 137 scale input to matrix processing unit 150.
[00196] For example, if the size of the prediction matrix is larger than the size of the scale list, the prediction matrix size transformation unit 181 converts the one below (hereinafter also called convert by downward conversion) to the prediction. More specifically, for example, when the prediction matrix is 16x16 in size and the scale list is 8x8 in size, the prediction matrix size transformation unit 181 converts the prediction matrix downwardly to an 8x8 prediction matrix. . Note that any method for downward conversion can be used. For example, the prediction matrix size transformation unit 181 can reduce the number of elements in the prediction matrix (hereinafter also called subsampling) using a filter (by computation). Alternatively, the prediction matrix size transformation unit 181 can also reduce the number of elements in the prediction matrix, for example, as shown in Figure 17, by removing some of the elements (for example, only the even numbered elements (in Figure 17, the elements in solid black) between the two-dimensional elements) without using a filter (hereinafter also called subsampling).
[00197] Furthermore, for example, if the size of the prediction matrix is smaller than the scale list size, the prediction matrix size transformation unit 181 converts by upward conversion (hereinafter also called converting by upward conversion ) the prediction matrix. More specifically, for example, when the prediction matrix has an 8x8 size and the scale list has a 16x16 size, the prediction matrix size transformation unit 181 converts the prediction matrix upward to a 16x16 prediction matrix. . Note that any method for upward conversion can be used. For example, the prediction matrix size transformation unit 181 can increase the number of elements in the prediction matrix (hereinafter also
Petition 870170066495, of September 6, 2017, p. 52/294 / 137 called oversampling) using a filter (by computation). Alternatively, the prediction matrix size transformation unit 181 can also increase the number of elements in the prediction matrix, for example, by copying the individual elements in the prediction matrix without using a filter (hereinafter also called reverse subsampling).
[00198] The prediction matrix size transformation unit 181 provides the prediction matrix whose size was matched to that of the scale list for computing unit 182.
[00199] Computing unit 182 subtracts the scale list entered to matrix processing unit 150 from the prediction matrix provided by the prediction matrix size transformation unit 181, and generates a difference matrix (residual matrix). Computing unit 182 provides the difference matrix calculated for quantization unit 183.
[00200] The quantization unit 183 quantizes the difference matrix provided by the computing unit 182. The quantization unit 183 provides the quantized difference matrix for the transformation unit of difference matrix size 163. The quantization unit 183 provides additional information used for quantization, such as quantization parameters, for output unit 166 to produce information for devices outside matrix processing unit 150 (lossless coding unit 16 and decanting unit 21). Note that the quantization unit 183 can be omitted (that is, the quantization of the difference matrix may not necessarily be performed).
(3) Difference matrix size transformation unit [00201] The difference matrix size transformation unit 163 converts the difference matrix size (quantized data) provided by the difference matrix generation unit 162 (the unit 183) to a size less than or equal to a maximum allowed transmission size (hereinafter also called a
Petition 870170066495, of September 6, 2017, p. 53/294 / 137 transmission), if necessary. The maximum size may have some optional value, and is, for example, 8x8.
[00202] The encoded data produced from the image encoding device 10 is transmitted to an image decoding device corresponding to the image encoding device 10, for example, by a transmission path or a storage medium, and is decoded by the image decoding device. The upper limit of the size (maximum size) of the difference matrix (quantized data) during such transmission, or in the encoded data produced from the image coding device 10, is established in the image coding device 10.
[00203] If the difference matrix size is greater than the maximum size, the difference matrix size transformation unit 163 converts the difference matrix downwardly so that the difference matrix size becomes less than or equal to the maximum size.
[00204] Note that, similar to the downward conversion of the prediction matrix described above, the difference matrix can be converted by downward conversion using any method. For example, subsampling can be performed using a filter or the like, or subsampling that involves removing elements can be performed.
[00205] In addition, the difference matrix converted by downward conversion can be any size smaller than the maximum size. However, in general, the greater the difference in size between before and after conversion, the greater the error becomes. It is therefore desirable that the difference matrix is converted by downward conversion to the maximum size. [00206] The difference matrix size transformation unit 163 provides the difference matrix converted by downward conversion to the entropy coding unit 164. Note that if the size of the difference matrix is less than the maximum size, the
Petition 870170066495, of September 6, 2017, p. 54/294 / 137 downward conversion described above is not necessary, and therefore the difference matrix size transformation unit 163 provides the difference matrix input to it for the entropy coding unit 164 as is (that is, the downward conversion of the difference matrix is omitted).
(4) Entropy coding unit [00207] Entropy coding unit 164 encodes the difference matrix (quantized data) provided with the difference matrix size transformation unit 163 using a predetermined method. As illustrated in Figure 16, the entropy coding unit 164 includes an overlap determination unit (135 degree unit) 191, a DPCM unit (Differential Pulse Code Modulation) 192 and an exp-G unit 193.
[00208] The overlay determination unit 191 determines symmetry of the difference matrix provided by the difference matrix size transformation unit 163. If the residue (difference matrix) represents a symmetric 135 degree matrix, for example, as illustrated in Figure 18, the overlay determination unit 191 removes the data (matrix elements) from the symmetrical part that is overlay data. If the residue does not represent a symmetric 135-degree matrix, the overlay determination unit 191 omits the removal of the data (matrix elements). The overlay determination unit 191 provides the data from the difference matrix from which the symmetric part has been removed, if necessary, for the DPCM unit 192.
[00209] The DPCM unit 192 performs DPCM encoding of the difference matrix data from which the symmetric part has been removed, if necessary, which the overlay determination unit 191 is provided, and generates DPCM data. The DPCM unit 192 provides the DPCM data generated for the exp-G unit 193.
[00210] The exp-G 193 unit encodes the DPCM data provided from the DPCM 192 unit using signed exponential Golomb codes
Petition 870170066495, of September 6, 2017, p. 55/294 / 137 or unsigned (hereinafter also called exponential Golomb codes). The exp-G 193 unit provides the encoding result for decoding unit 165 and output unit 166.
(5) Decoding unit [00211] Decoding unit 165 restores a scaling list of data provided by the exp-G 193 unit. Decoding unit 165 provides information regarding the restored scaling list for prediction unit 161 as a scale list previously transmitted.
[00212] As shown in Figure 16, the decoding unit 165 includes a scale list restoration unit 201 and storage unit 202.
[00213] The scale list restoration unit 201 decodes the exponential Golomb codes provided by the entropy coding unit 164 (the exp-G unit 193) to restore a scale list to be input to the matrix processing unit 150. For example, the scale list restoration unit 201 decodes the exponential Golomb codes using the method corresponding to the coding method for the entropy coding unit 164, and obtains a difference matrix by performing transformation opposite to the size transformation performed by the difference matrix size transformation unit 163 and performing decanting corresponding to the quantization performed by the quantization unit 183. The scale list restoration unit 201 additionally subtracts the difference matrix obtained from the prediction matrix to restore a list of scale.
[00214] The scale list restoration unit 201 provides the scale list restored to storage unit 202 for storage in association with the size and list ID of the scale list.
[00215] The storage unit 202 stores information relating to the scale list provided by the scale list restoration unit 201. The information relating to the scale list stored in the storage unit
Petition 870170066495, of September 6, 2017, p. 56/294 / 137 storage 202 is used to generate prediction matrices from other orthogonal transform units that are processed later in time. That is, storage unit 202 provides stored information regarding the scale list to prediction unit 161 as information relating to a previously transmitted scale list.
[00216] Note that, instead of storing the information relating to the restored scale list as described above, the storage unit 202 can store the scale list entered to the matrix processing unit 150 in association with the size and ID list entry list list. In this case, the scale list restoration unit 201 can be omitted.
(6) Output unit [00217] Output unit 166 produces the various types of information provided for devices outside of matrix processing unit 150. For example, in copy mode, output unit 166 provides the list ID from the prediction matrix provided by copy unit 171 to lossless coding unit 16 and decanting unit 21. In addition, for example, in normal mode, output unit 166 provides the exponential Golomb codes provided by the exp-G 193 and the quantization parameters provided by the quantization unit 183 for the lossless coding unit 16 and the decanting unit 21.
[00218] Output unit 166 additionally provides identification information indicating a maximum size (transmission size) allowed in the transmission of a scale list (or a difference matrix between a scale list and a prediction matrix thereof) for the lossless coding unit 16 as information to generate a scale list on the decoding side. As described above, the lossless encoding unit 16 creates an encoded stream including the information to generate a scale list, and provides the encoded stream for the decoding side. The identification information indicating the transmission size can be
Petition 870170066495, of September 6, 2017, p. 57/294 / 137 specified in advance by level, profile, and similar. In this case, information regarding the transmission size is shared in advance between the device on the encoding side and the device on the decoding side. Thus, the transmission of the identification information described above can be omitted.
2-5. Detailed example configuration of the DPCM unit [00219] Figure 19 is a block diagram illustrating an example of a more detailed configuration of the DPCM unit 192. Referring to Figure 19, the DPCM unit 192 includes a unit coding unit DC 211 coefficient and a DPCM unit of AC 212 coefficient. [00220] The DC coefficient coding unit 211 acquires the DC coefficient from the coefficients provided by the overlay determination unit 191, subtracts the value from the coefficient of CC of a predetermined initial value (for example, 8) to determine a difference value, and uses the difference value as the initial difference value (i = 0) (scaling_list_delta_coef). The CC coefficient coding unit 211 provides the calculated difference value (scaling_list_delta_coef (i = 0)) for the exp-G 193 unit as the initial coefficient of the scale list corresponding to the region of interest being processed.
[00221] The DPCM unit of AC coefficient 212 acquires an AC coefficient among the coefficients provided by the overlay determination unit 191, and subtracts the value of the AC coefficient from the coefficient immediately processed to determine a difference value (scaling_list_delta_coef (i> 0)). The DPCM unit of CA 212 coefficient provides the determined difference value (scaling_list_delta_coef (i> 0)) for the exp-G 193 unit as a coefficient of the scale list corresponding to the region of interest being processed. Note that when i = 1, the immediately preceding coefficient is represented by i = 0. Thus, the CC coefficient is the coefficient processed immediately beforehand. [00222] In this way, the DPCM 192 unit can transmit the
Petition 870170066495, of September 6, 2017, p. 58/294 / 137 DC coefficient as the element located at the beginning of the scale list (AC coefficients). Therefore, the coding efficiency of the scale list can be improved.
2-6. Quantization matrix encoding process flow [00223] The following is an example of the flow of a quantization matrix encoding process performed by the matrix processing unit 150 illustrated in Figure 16 and will be described with reference to a flow chart illustrated in Figure 20.
[00224] When the quantization matrix coding process is started, in step S101, the prediction unit 161 acquires a scale list (or quantization matrix) for a current region (also called a region of interest), which is an orthogonal transform unit to be processed.
[00225] In step S102, the prediction unit 161 determines whether or not the current mode is the copy mode. If it is determined that the current mode is not the copy mode, the prediction unit 161 advances the process to step S103. [00226] In step S103, the prediction matrix generation unit 172 acquires a scale list previously transmitted from storage unit 202, and generates a prediction matrix using the scale list. [00227] In step S104, the prediction matrix size transformation unit 181 determines whether or not the prediction matrix size generated in step S103 is different from that of the scale list for the current region (region of interest) acquired in step S101. If it is determined that both sizes are different, the prediction matrix size transformation unit 181 proceeds the process to step S105.
[00228] In step S105, the prediction matrix size transformation unit 181 converts the size of the prediction matrix generated in step S103 to the size of the scale list for the current region acquired in step S101.
[00229] When the processing of the step S105 is completed, the
Petition 870170066495, of September 6, 2017, p. 59/294 / 137 prediction matrix size transformation unit 181 proceeds the process to step S106. If it is determined in step S104 that the size of the prediction matrix is equal to the size of the scale list, the prediction matrix size transformation unit 181 advances the process to step S106 while skipping the process from step S105 (or without executing processing of step S105).
[00230] In step S106, the computing unit 182 subtracts the scale list from the prediction matrix to calculate a difference matrix between the prediction matrix and the scale list.
[00231] In step S107, the quantization unit 183 quantizes the difference matrix generated in step S106. Note that this processing can be omitted.
[00232] In step S108, the difference matrix size transformation unit 163 determines whether or not the quantized difference matrix size is larger than the transmission size (the maximum size allowed in transmission). If it is determined that the size of the quantized difference matrix is greater than the transmission size, the difference matrix size transformation unit 163 advances the process to step S109, and converts the difference matrix to the size of the difference matrix. transmission or less.
[00233] When the processing of step S109 is completed, the difference matrix size transformation unit 163 advances the process to step S110. In addition, if it is determined in step S108 that the size of the quantized difference matrix is less than or equal to the transmission size, the difference matrix size transformation unit 163 advances the process to step S110 while skipping the processing of step S109 (or without performing the processing of step S109).
[00234] In step S110, the overlay determination unit 191 determines whether or not the quantized difference matrix has a symmetry of 135
Petition 870170066495, of September 6, 2017, p. 60/294 / 137 degrees. If it is determined that the quantized difference matrix has a symmetry of 135 degrees, the overlay determination unit 191 advances the process to step S111.
[00235] In step S111, the overlay determination unit 191 removes the overlap portion (overlap data) in the quantized difference matrix. After the overlay data is removed, the overlay determination unit 191 advances the process to step S112.
[00236] Furthermore, if it is determined in step S110 that the quantized difference matrix has no symmetry of 135 degrees, the overlay determination unit 191 advances the process to step S112 while skipping the processing of step S111 (or without executing the processing of step S111).
[00237] In step S112, the DPCM unit 192 performs DPCM encoding of the difference matrix from which the overlapping portion has been removed, if necessary.
[00238] In step S113, the exp-G 193 unit determines whether or not DPCM data generated in step S112 has a positive or negative sign. If it is determined that a signal is included, the exp-G 193 unit advances the process to step S114.
[00239] In step S114, the exp-G 193 unit encodes the DPCM data using signed exponential Golomb encoding. Output unit 166 produces exponential Golomb codes generated to lossless coding unit 16 and decanting unit 21. When processing of step S114 is completed, exp-G unit 193 advances the process to step S116.
[00240] In addition, if it is determined in step S113 that no signal is included, the exp-G 193 unit advances the process to step S115. [00241] In step S115, the exp-G 193 unit encodes the DPCM data using unsigned exponential Golomb encoding. The unity
Petition 870170066495, of September 6, 2017, p. 61/294 / 137 output 166 produces exponential Golomb codes generated to the lossless coding unit 16 and to the decanting unit 21. When processing of step S115 is completed, the exp-G unit 193 advances the process to step S116 .
[00242] Furthermore, if it is determined in step S102 that the current mode is the copy mode, the copy unit 171 copies a previously transmitted scale list, and uses the copied scale list as a prediction matrix. Output unit 166 produces the list ID corresponding to the prediction matrix, the lossless coding unit 16 and the decanting unit 21 as information indicating the prediction matrix. Then, copy unit 171 proceeds to step S116.
[00243] In step S116, the scale list restoration unit 201 restores a scale list. In step S117, storage unit 202 stores the scale list restored in step S116.
[00244] When the processing of step S117 is completed, the matrix processing unit 150 terminates the quantization matrix coding process.
2-7. DPCM process flow [00245] Below, an example of a DPCM process flow performed in step S112 in Figure 20 will be described with reference to a flow chart illustrated in Figure 21.
[00246] When the DPCM process is started, in step S131, the CC coefficient coding unit 211 determines a difference between the CC coefficient and a constant. In step S132, the DPCM unit of AC coefficient 212 determines a difference between the DC coefficient and the initial AC coefficient.
[00247] In step S133, the DPCM unit of AC coefficient 212 determines whether or not all AC coefficients have been processed. If it is determined that there is an unprocessed AC coefficient, the DPCM unit of AC coefficient 212 advances the process to step S134.
Petition 870170066495, of September 6, 2017, p. 62/294 / 137 [00248] In step S134, the DPCM unit of AC coefficient 212 shifts the processing target to the subsequent AC coefficient. In step S135, the DPCM unit of AC coefficient 212 determines a difference between the AC coefficient previously processed and the current AC coefficient being processed. When processing of step S135 is completed, the DPCM unit of AC coefficient 212 returns the process to step S133.
[00249] Thus, as long as it is determined in step S133 that there is an unprocessed AC coefficient, the DPCM unit of AC coefficient 212 repeatedly performs the processing of steps S133 to S135. If it is determined in step S133 that there is no unprocessed AC coefficient, the DP coefficient unit of AC 212 ends the DPCM process, and returns the process to Figure 20.
[00250] As described above, a difference between the DC coefficient and the AC coefficient located at the beginning between the AC coefficients is determined, and the difference instead of the DC coefficient is transmitted to an image decoding device. Thus, the image encoding device 10 can suppress an increase in the amount of encoding in a scale list.
[00251] In the following, an example configuration of an image decoding device according to an embodiment of the present exposure will be described.
2-8. Image decoding device [00252] Figure 22 is a block diagram illustrating an example configuration of an image decoding device 300 according to an embodiment of the present exposure. The image decoding device 300 illustrated in Figure 22 is an image processing device to which the present technology is applied and which is configured to decode encoded data generated by the image encoding device 10. Referring to Figure 22, the device in
Petition 870170066495, of September 6, 2017, p. 63/294 / 137 image decoding 300 includes an accumulation buffer 311, a lossless decoding unit 312, a reverse orthogonal decanting / transforming unit 313, a summing unit 315, a deblocking filter 316, a memory temporary rearrangement 317, a D / A conversion unit (Digital to Analog) 318, a frame memory 319, selectors 320 and 321, an intraprediction unit 330 and a motion compensation unit 340.
[00253] The accumulation buffer 311 temporarily accumulates an encoded stream entered through a transmission path, using a storage medium.
[00254] The lossless decoding unit 312 decodes the encoded stream entered from the accumulation buffer 311 according to the encoding scheme used for encoding. The lossless decoding unit 312 additionally decodes the multiplexed information in the header region of the encoded stream. The multiplexed information in the header region of the encoded stream can include, for example, the information to generate a scale list described above, and information about the intraprediction and information about the interpretation that are contained in the block header. The lossless decoding unit 312 produces the decoded quantized data and the information to generate a scale list for the reverse orthogonal decanting / transforming unit 313. The lossless decoding unit 312 additionally produces the information regarding the intraprediction for the intraprediction unit 330. The lossless decoding unit 312 additionally produces information regarding the interpretation to the motion compensation unit 340.
[00255] The inverse orthogonal decanting / transforming unit 313 performs decanting and an inverse orthogonal transform on the quantized data entered from the lossless decoding unit 312 to generate prediction error data. Thereafter, the inverse orthogonal decanting / transforming unit 313 produces the error data
Petition 870170066495, of September 6, 2017, p. 64/294 / 137 of prediction generated for the adding unit 315.
[00256] The summing unit 315 adds together the prediction error data input from the inverse orthogonal decanting / transform unit 313 and prediction image data input from selector 321 to generate decoded image data. Thereafter, the adding unit 315 produces the decoded image data generated for the deblocking filter 316 and the frame memory 319.
[00257] The deblocking filter 316 filters the decoded image data input from the summing unit 315 to remove block forming artifacts, and produces the filtered decoded image data for the rearrangement buffer 317 and the frame memory 319 .
[00258] The 317 rearrangement buffer rearranges images introduced from the 316 deblocking filter to generate a time series image data sequence. Thereafter, the rearrangement buffer 317 produces the image data generated for the D / A conversion unit 318.
[00259] The D / A conversion unit 318 converts the image data in digital form that is entered from the temporary rearrangement memory 317 to an image signal in analog form. Thereafter, the D / A conversion unit 318 produces the analog image signal for, for example, a display (not shown) connected to the image decoding device 300 to display an image.
[00260] The frame memory 319 stores the decoded image data to be filtered, which is input from the summing unit 315, and the filtered decoded image data input from the deblocking filter 316, using a storage medium.
[00261] Selector 320 changes the destination to which the image data provided in the frame memory 319 are to be produced between the intraprediction unit 330 and the movement compensation unit 340, for each block in the image, according to mode information acquired by
Petition 870170066495, of September 6, 2017, p. 65/294 / 137 lossless decoding unit 312. For example, if an intrapredictive mode is specified, selector 320 produces the decoded image data to be filtered, which is provided with frame memory 319, for the intrapredicting unit 330 as reference image data. In addition, if an interpretation mode is specified, selector 320 produces the filtered decoded image data provided from frame memory 319 to the motion compensation unit 340 as reference image data.
[00262] Selector 321 changes the source from which prediction image data to be provided to the adding unit 315 are to be produced between the intraprediction unit 330 and the movement compensation unit 340, for each block in the image, according to information acquired by the lossless decoding unit 312. For example, if the intraprediction mode is specified, selector 321 provides the prediction image data produced from the intraprediction unit 330 to the summing unit 315. If the interpretation mode is specified, selector 321 provides the prediction image data produced from the motion compensation unit 340 to the adding unit 315.
[00263] The intraprediction unit 330 performs intrathecal prediction of a pixel value based on the intraprediction information, which is input from the lossless decoding unit 312 and the reference image data provided from the frame memory 319, and generates data of prediction image. Thereafter, the intraprediction unit 330 produces the prediction image data generated for selector 321.
[00264] The motion compensation unit 340 performs a motion compensation process based on the interpretation information, which is input from the lossless decoding unit 312 and the reference image data provided from the frame memory 319, and generates prediction image data. After that, the motion compensation unit 340 produces the prediction image data generated for the
Petition 870170066495, of September 6, 2017, p. 66/294 / 137 selector 321.
2-9. Example configuration of reverse orthogonal decanting / transformed unit [00265] Figure 23 is a block diagram illustrating an example of a main configuration of the reverse orthogonal decanting / transformed unit 313 of the image decoding device 300 shown in Figure 22. Referring to Figure 23, the inverse orthogonal decanting / transform unit 313 includes a matrix generating unit 410, a selection unit 430, a decanting unit 440 and an inverse orthogonal transform unit 450.
(1) Matrix generation unit [00266] Matrix generation unit 410 decodes encoded scale list data that is extracted from a bit stream and provided by lossless decoding unit 312, and generates a scale list. The matrix generation unit 410 provides the scale list generated for the decanting unit 440.
(2) Selection unit [00267] Selection unit 430 selects a transform unit (TU) to be used for the inverse orthogonal transform of image data to be decoded from a plurality of transform units having different sizes. Examples of possible sizes of transform units selectable by the selection unit 430 include 4x4 and 8x8 for H.264 / AVC, and include 4x4, 8x8, 16x16 and 32x32 for HEVC. The selection unit 430 can select a transform unit according to, for example, the LCU, SCU, and split_flag contained in the coded flow header. Thereafter, the selection unit 430 produces information specifying the size of the selected transform unit to the decanting unit 440 and the inverse orthogonal transform unit 450.
(3) Decanting unit [00268] Decanting unit 440 decanting data from
Petition 870170066495, of September 6, 2017, p. 67/294 / 137 transform coefficient quantized when the images are encoded, using a scale list of the transform unit selected by the selection unit 430. After that, the decanting unit 440 produces the transform coefficient data decommissioned to the unit of inverse orthogonal transform 450.
(4) Inverse orthogonal transform unit [00269] The inverse orthogonal transform unit 450 performs an inverse orthogonal transform on the transform coefficient data decanted by the decanting unit 440 in units of the selected transform unit according to the orthogonal transform scheme used for coding to generate prediction error data. Thereafter, the inverse orthogonal transform unit 450 produces the prediction error data generated for the summing unit 315.
2-10. Detailed example configuration of matrix generation unit [00270] Figure 24 is a block diagram illustrating an example of a detailed configuration of matrix generation unit 410 illustrated in Figure 23. Referring to Figure 24, the generation unit matrix 410 includes a parameter analysis unit 531, a prediction unit 532, an entropy decoding unit 533, a scale list restoration unit 534, an output unit 535 and a storage unit 536.
(1) Parameter analysis unit [00271] The parameter analysis unit 531 analyzes the various flags and parameters related to the scale list, which are provided with the lossless decoding unit 312. In addition, according to the analysis results, the parameter analysis unit 531 provides various types of information provided from the lossless decoding unit 312, such as encoded data from the difference matrix, to the prediction unit 532 or the entropy decoding unit 533.
[00272] For example, if pred_mode is equal to 0, the unit of analysis
Petition 870170066495, of September 6, 2017, p. 68/294 / 137 of parameter 531 determines that the current mode is copy mode, and provides pred_matrix_id_delta to a copy unit 541. In addition, for example, if pred_mode is equal to 1, the parameter analysis unit 531 determines that the current mode is a full scan mode (normal mode), and provides pred_matrix_id_delta and pred_size_id_delta to a matrix generation prediction unit 542.
[00273] Furthermore, for example, if residual_flag is true, the parameter analysis unit 531 provides the encoded data (exponential Golomb codes) of the scale list provided by the lossless decoding unit 312 to an exp-G unit 551 of the entropy decoding unit 533. The parameter analysis unit 531 additionally provides residual_symmetry_flag to the exp-G unit 551.
[00274] In addition, parameter analysis unit 531 provides residual_down_sampling_flag to a difference matrix size transformation unit 562 of the scale list restoration unit 534.
(2) Prediction unit [00275] Prediction unit 532 generates a prediction matrix as controlled by parameter analysis unit 531. As shown in Figure 24, prediction unit 532 includes copy unit 541 and the unit of matrix generation prediction 542.
[00276] In copy mode, copy unit 541 copies a previously transmitted scale list, and uses the copied scale list as a prediction matrix. More specifically, copy unit 541 reads a previously transmitted scale list corresponding to pred_matrix_id_delta and having the same size as the scale list for the current region of storage unit 536, uses the scale list read as a prediction image, and provides the prediction image for output unit 535.
[00277] In normal mode, matrix generation prediction unit 542 generates (or predicts) a prediction matrix using a previously transmitted scale list. More specifically, the power generation unit
Petition 870170066495, of September 6, 2017, p. 69/294 / 137 prediction matrix 542 reads a previously transmitted scale list corresponding to pred_matrix_id_delta and pred_size_id_delta from storage unit 536, and generates a prediction matrix using the read scale list. In other words, the matrix generation prediction unit 542 generates a prediction matrix similar to the prediction matrix generated by the matrix generation prediction unit 172 (Figure 16) of the image encoding device 10. The image generation unit prediction matrix 542 provides the prediction matrix generated for a prediction matrix size transformation unit 561 of the scale list restoration unit 534.
(3) Entropy decoding unit [00278] Entropy decoding unit 533 restores a difference matrix of the exponential Golomb codes provided by the parameter analysis unit 531. As illustrated in Figure 24, the entropy decoding unit 533 includes the exp-G 551 unit, a reverse DPCM unit 552 and a reverse overlay determination unit 553.
[00279] The exp-G 551 unit decodes the signed or unsigned exponential Golomb codes (hereinafter also called exponential Golomb decoding) to restore DPCM data. The exp-G 551 unit provides the DPCM data restored together with residual_symmetry_flag for the reverse DPCM unit 552.
[00280] Reverse DPCM unit 552 performs DPCM decoding of data from which the overlap portion has been removed to generate residual data from DPCM data. The reverse DPCM unit 552 provides the residual data generated together with residual_symmetry_flag for the reverse overlay determination unit 553.
[00281] If residual_symmetry_flag is true, that is, if the residual data is a remaining portion of a symmetric 135-degree matrix from which the data (matrix elements) of the symmetric overlap was
Petition 870170066495, of September 6, 2017, p. 70/294 / 137 removed, the reverse overlay determination unit 553 restores the symmetric part data. In other words, a difference matrix from a symmetric 135 degree matrix is restored. Note that if residual_symmetry_flag is not true, that is, if the residual data represents a matrix that is not a symmetric 135 degree matrix, the reverse overlay determination unit 553 uses the residual data as a difference matrix without restoring data from a symmetrical part. The inverse overlap determination unit 553 provides the difference matrix restored in the manner described above for the scale list restoration unit 534 (the difference matrix size transformation unit 562).
(4) Scale list restore unit [00282] The scale list restore unit 534 restores a scale list. As shown in Figure 24, the scale list restoration unit 534 includes the prediction matrix size transformation unit 561, the difference matrix size transformation unit 562, a decanting unit 563 and a computing unit 564. [00283] If the size of the prediction matrix provided by the prediction unit 532 (the matrix generation prediction unit 542) is different from the size of the scale list for the current region to be restored, the transformation unit of prediction matrix size 561 converts the prediction matrix size.
[00284] For example, if the prediction matrix size is larger than the scale list size, the prediction matrix size transformation unit 561 converts the prediction matrix by downconversion. In addition, for example, if the prediction matrix size is smaller than the scale list size, the prediction matrix size transformation unit 561 converts the prediction matrix upward. The same method as that for the prediction matrix size transformation unit 181 (Figure 16) of
Petition 870170066495, of September 6, 2017, p. 71/294 / 137 image encoding device 10 is selected as a conversion method.
[00285] The prediction matrix size transformation unit 561 provides the prediction matrix whose size has been matched to that of the scale list for computing unit 564.
[00286] If residual_down_sampling_flag is true, that is, if the size of the difference matrix transmitted is smaller than the size of the current region to be unquantified, the difference matrix size transformation unit 562 converts by conversion upward to the matrix of difference to increase the size of the difference matrix to a size corresponding to the current region to be unquantified. Any method for upward conversion can be used. For example, a method corresponding to the downward conversion method performed by the difference matrix size transformation unit 163 (Figure 16) of the image encoding device 10 can be used.
[00287] For example, if the difference matrix size transformation unit 163 has the difference matrix subsampled, the difference matrix size transformation unit 562 may over-show the difference matrix. Alternatively, if the difference matrix size transformation unit 163 has subsampled the difference matrix, the difference matrix size transformation unit 562 can perform reverse subsampling of the difference matrix.
[00288] For example, the difference matrix size transformation unit 562 can perform a closer neighbor (closer neighbor) interpolation process as illustrated in Figure 25 in place of general linear interpolation. The nearest neighbor interpolation process can reduce memory capacity.
[00289] Therefore, even if a scale list having a large size is not transmitted, data obtained after over-sampling does not need to be stored for over-sampling a scale list.
Petition 870170066495, of September 6, 2017, p. 72/294 / 137 having a small size. In addition, a buffer or similar buffer is not required when data involved in computation during oversampling is stored.
[00290] Note that if residual_down_sampling_flag is not true, that is, if the difference matrix is transmitted with the same size as that used for the quantization process, the difference matrix size transformation unit 562 omits upward conversion difference matrix (or you can convert the difference matrix upwards by a factor of 1).
[00291] The difference matrix size transformation unit 562 provides the difference matrix converted by upward conversion in the manner described above, as necessary, for the decanting unit 563.
[00292] The decanting unit 563 decantifies the difference matrix provided (quantized data) using a method corresponding to that for quantization performed by the quantization unit 183 (Figure 16) of the image coding device 10, and provides the decantized difference matrix computation unit 564. Note that if quantization unit 183 is omitted, that is, if the difference matrix provided by the difference matrix size transformation unit 562 is not quantized data, the decanting unit 563 can be omitted .
[00293] Computing unit 564 adds together the prediction matrix provided by the prediction matrix size transformation unit 561 and the difference matrix provided by the decanting unit 563, and restores a scale list for the current region. Computing unit 564 provides the restored scale list for output unit 535 and storage unit 536.
(5) Output unit [00294] Output unit 535 produces the information provided for a device outside the matrix generation unit 410. For example, in
Petition 870170066495, of September 6, 2017, p. 73/294 / 137 copy, output unit 535 provides the prediction matrix provided by copy unit 541 for decanting unit 440 as a scale list for the current region. In addition, for example, in normal mode, output unit 535 provides the scale list for the current region provided with scale list restoration unit 534 (computing unit 564) for decanting unit 440.
(6) Storage unit [00295] Storage unit 536 stores the scale list provided with the scale list restoration unit 534 (computing unit 564) along with the size and list ID of the scale list. The information regarding the scale list stored in the storage unit 536 is used to generate prediction matrices from other orthogonal transform units that are processed later in time. In other words, the storage unit 536 provides the stored information relating to the scale list for the prediction unit 532 as information relating to a previously transmitted scale list. 2-11. Detailed example configuration of the reverse DPCM unit [00296] Figure 26 is a block diagram illustrating an example of a detailed configuration of the reverse DPCM unit 552 illustrated in Figure 24. Referring to Figure 26, the reverse DPCM unit 552 includes an initial settling unit 571, a DPCM decoding unit 572 and a DC coefficient extraction unit 573.
[00297] The initial establishment unit 571 acquires sizeID and MatrixID, and sets several variables to initial values. The initial establishment unit 571 provides the information acquired and established for the DPCM decoding unit 572.
[00298] The DPCM 572 decoding unit determines individual coefficients (the CC coefficient and the AC coefficients) of the difference values (scaling_list_delta_coef) of the CC coefficient and the AC coefficients using the initial and similar settings provided by the
Petition 870170066495, of September 6, 2017, p. 74/294 / 137 initial establishment unit 571. The DPCM decoding unit 572 provides the coefficients determined for the CC coefficient extraction unit 573 (ScalingList [i]).
[00299] The DC coefficient extraction unit 573 extracts the DC coefficient from among the coefficients (ScalingList [i]) provided by the DPCM 572 decoding unit. The DC coefficient is located at the beginning of the AC coefficients. That is, the initial coefficient (ScalingList [0]) among the coefficients provided by the DPCM 572 decoding unit is the CC coefficient. The DC 573 coefficient extraction unit extracts the coefficient located at the beginning as the DC coefficient, and produces the extracted coefficient for the reverse overlap determination unit 553 (CC_coef). The DC coefficient extraction unit 573 produces the other coefficients (ScalingList [i] (i> 0)) for the reverse overlap determination unit 553 such as the AC coefficients. [00300] Therefore, the reverse DPCM unit 552 can perform correct DPCM decoding, and can obtain the DC coefficient and the AC coefficients. That is, the image decoding device 300 can suppress an increase in the amount of encoding in a scale list.
2-12. Quantization matrix decoding process flow [00301] An example of the quantization matrix flow decoding process performed by the matrix generation unit 410 having the configuration described above will be described with reference to a flow chart illustrated in Figure 27.
[00302] When the quantization matrix decoding process is started, in step S301, the parameter analysis unit 531 reads the quantized values (Qscale0 to Qscale3) from regions 0 to 3.
[00303] In step S302, the parameter analysis unit 531 reads pred_mode. In step S303, the parameter analysis unit 531 determines whether or not pred_mode is equal to 0. If pred_mode is determined to be equal to
Petition 870170066495, of September 6, 2017, p. 75/294 / 137
0, parameter analysis unit 531 determines that the current mode is copy mode, and proceeds to step S304.
[00304] In step S304, the parameter analysis unit 531 reads pred_matrix_id_delta. In step S305, copy unit 541 copies a scaled list that has been transmitted, and uses the copied scale list as a prediction matrix. In copy mode, the prediction matrix is produced as the scale list for the current region. When the process of step S305 is completed, the copy unit 541 finishes the quantization matrix decoding process.
[00305] Furthermore, if it is determined in step S303 that pred_mode is not equal to 0, the parameter analysis unit 531 determines that the current mode is the full scan mode (normal mode), and proceeds the process to step S306 .
[00306] In step S306, the parameter analysis unit 531 reads pred_matrix_id_delta, pred_size_id_delta, and residual_flag. In step S307, matrix generation prediction unit 542 generates a prediction matrix from a scale list that has been transmitted.
[00307] In step S308, the parameter analysis unit 531 determines whether or not residual_flag is true. If it is determined that residual_flag is not true, no residual matrix exists, and the prediction matrix generated in the step that S307 is produced as the scale list for the current region. In this case, therefore, the parameter analysis unit 531 ends the quantization matrix decoding process.
[00308] In addition, if it is determined in step S308 which residual_flag is true, the parameter analysis unit 531 advances the process to step S309.
[00309] In step S309, the parameter analysis unit 531 reads residual_down_sampling_flag and residual_symmetry_flag.
[00310] In step S310, the exp-G 551 unit and reverse DPCM unit 552 decode the exponential Golomb codes of the matrix
Petition 870170066495, of September 6, 2017, p. 76/294 / 137 residual, and generate residual data.
[00311] In step S311, the reverse overlay determination unit 553 determines whether or not residual_symmetry_flag is true. If it is determined that residual_symmetry_flag is true, the reverse overlay determination unit 553 advances the process to step S312, and restores the overlap portion removed from the residual data (or performs an inverse symmetry process). When a difference matrix that is a symmetric 135 degree matrix is generated in the manner described above, the reverse overlap determination unit 553 advances the process to step S313. [00312] Furthermore, if it is determined in step S311 that residual_symmetry_flag is not true (or if the residual data is a difference matrix that is not a symmetric 135 degree matrix), the reverse overlap determination unit 553 proceeds to step S313 while skipping the processing of step S312 (or without performing an inverse symmetry process).
[00313] In step S313, the difference matrix size transformation unit 562 determines whether or not residual_down_sampling_flag is true. If it is determined that residual_down_sampling_flag is true, the difference matrix size transformation unit 562 advances the process to step S314, and converts the difference matrix upward to a size corresponding to the current region to be unquantified. After the difference matrix is converted by upward conversion, the difference matrix size transformation unit 562 proceeds the process to step S315.
[00314] Furthermore, if it is determined in step S313 that residual_down_sampling_flag is not true, the difference matrix size transformation unit 562 advances the process to step S315 while skipping the processing of step S314 (or without converting by upward conversion to difference matrix).
[00315] In step S315, computing unit 564 adds the
Petition 870170066495, of September 6, 2017, p. 77/294 / 137 difference matrix to the prediction matrix to generate a scale list for the current region. When the processing of step S315 is completed, the quantization matrix decoding process ends.
2-13. Residual signal decoding process flow [00316] Below, an example of the residual signal decoding process flow performed in step S310 in Figure 27 will be described with reference to a flow chart illustrated in Figure 28.
[00317] When the residual signal decoding process is started, in step S331, the exp-G 551 unit decodes the exponential Golomb codes provided.
[00318] In step S332, the reverse DPCM unit 552 performs a reverse DPCM process on DPCM data obtained by the expG 551 unit by decoding.
[00319] When the reverse DPCM process is completed, the reverse DPCM unit 552 ends the residual signal decoding process, and returns the process to Figure 27.
2-14. Reverse DPCM process flow [00320] Below, an example of the reverse DPCM process flow performed in step S332 in Figure 28 will be described with reference to a flow chart illustrated in Figure 29.
[00321] When the reverse DPCM process is started, in step S351, the initial establishment unit 571 acquires sizeID and MatrixID. [00322] In step S352, the initial establishment unit 571 establishes coefNum as follows.
coefNum = min ((1 << (4 + (sizeID << 1))), 65) [00323] In step S353, the initial establishment unit 571 establishes a variable i and a nextcoef variable as follows.
i = 0 nextcoef = 8 [00324] In step S354, the DPCM 572 decoding unit
Petition 870170066495, of September 6, 2017, p. 78/294 / 137 determines whether or not variable i <coefNum. If variable i is less than coefNum, initial establishment unit 571 advances the process to step S355.
[00325] In step S355, the DPCM 572 decoding unit reads DPCM data from the coefficient (scaling_list_delta_coef).
[00326] In step S356, the DPCM 572 decoding unit determines nextcoef as below using the read DPCM data, and additionally determines scalingList [i].
nextcoef = (nextcoef + scaling_list_delta_coef + 256)% 256 scalingList [i] = nextcoef [00327] In step S357, the CC coefficient extraction unit 573 determines whether or not sizeID is greater than 1 and whether or not variable i is equal to 0 (that is, the coefficient located at the beginning). If it is determined that sizeID is greater than 1 and the variable i represents the coefficient located at the beginning, the CC coefficient extraction unit 573 advances the process to step S358, and uses the coefficient as the CC coefficient (CC_coef = nextcoef ). When the processing of step S358 is completed, the DC coefficient extraction unit 573 advances the process to step S360.
[00328] Furthermore, if it is determined in step S357 that sizeID is less than or equal to 1 or that variable i does not represent the coefficient located at the beginning, the DC coefficient extraction unit 573 advances the process to step S359, and shifts variable i for each coefficient by one because the CC coefficient has been extracted. (ScalingList [(i- (sizeID)> 1) 1; 0] = nextcoef). If the processing of step S359 is completed, the DC coefficient extraction unit 573 advances the process to step S360.
[00329] In step S360, the DPCM 572 decoding unit increments variable i to change the processing objective to the subsequent coefficient, and then returns the process to step S354.
[00330] In step S354, the processing of steps S354 to S360 is performed repeatedly until it is determined that variable i is greater than
Petition 870170066495, of September 6, 2017, p. 79/294 / 137 that is equal to coefNum. If it is determined in step S354 that the variable i is greater than or equal to coefNum, the DPCM 572 decoding unit ends the reverse PCM process, and returns the process to Figure 28.
[00331] Therefore, the difference between the DC coefficient and the AC coefficient located at the beginning of the AC coefficients can be decoded correctly. Therefore, the image decoding device 300 can suppress an increase in the amount of encoding in a scale list.
3. Third embodiment
3-1. Syntax: Second method [00332] Another method for transmitting a difference between the CC coefficient and another coefficient, instead of the CC coefficient, can be, for example, transmitting a difference between the CC coefficient and the component (0, 0 ) of an 8x8 matrix as DPCM data different from the DPCM data of the 8x8 matrix (second method). For example, after DPCM transmission from an 8x8 matrix, the difference between the CC coefficient and the component (0, 0) of the 8x8 matrix can be transmitted.
[00333] Therefore, similarly to the first method, the compression ratio can be further improved when the coefficient value (0, 0) (AC coefficient) of an 8x8 matrix and the value of the CC coefficient are close to each other .
[00334] Figure 30 illustrates the syntax of a scale list in the second method. In the example illustrated in Figure 30, 64 difference values (scaling_list_delta_coef) between coefficients are read. Finally, the difference (scaling_list_CC_coef_delta) between the CC coefficient and the coefficient (0, 0) (AC coefficient) is read, and the CC coefficient is determined from the difference. [00335] In the second method, therefore, syntax for decoding AC coefficients can be similar to that of the related technique illustrated in Figure 12. That is, the syntax for the second method can be obtained by modifying the example of the related technique by an amount
Petition 870170066495, of September 6, 2017, p. 80/294 / 137 small, and may be more possible than for the first method.
[00336] However, while the second method does not allow an image decoding device to obtain the CC coefficient until the image decoding device has received all the coefficients and decompressed all DPCM data, the first method allows for a device image decoding restore the DC coefficient on time when the image decoding device receives the initial coefficient. [00337] An image encoding device that implements the syntax for the second method described above will be described hereinafter.
3-2. Detailed example configuration of DPCM unit [00338] In the second method, the image encoding device 10 has a configuration basically similar to that in the first method described above. Specifically, the image encoding device 10 has a configuration as in the example illustrated in Figure 14. In addition, the orthogonal transform / quantization unit 14 has a configuration as in the example illustrated in Figure 15. In addition, the matrix processing unit 150 has a configuration as in the example illustrated in Figure 16.
[00339] An example configuration of the DPCM unit 192 in the second example is illustrated in Figure 31. As illustrated in Figure 31, in the second example, the DPCM unit 192 includes an AC 611 coefficient buffer, an encoding unit of coefficient of AC 612, a unit of DPCM of coefficient of AC 613 and a unit of DPCM of coefficient of DC 614.
[00340] The AC 611 buffer memory stores the initial AC coefficient (that is, the coefficient (0, 0)) provided by the overlay determination unit 191. The AC 611 buffer memory provides the AC coefficient Stored initial AC (AC coefficient (0, 0)) for the DC coefficient DPCM unit 614 at a predetermined time after all AC coefficients have been
Petition 870170066495, of September 6, 2017, p. 81/294 / 137 subject to a DPCM process, or in response to a request.
[00341] The AC coefficient encoding unit 612 acquires the initial AC coefficient (AC coefficient (0, 0)) provided by the overlay determination unit 191, and subtracts the value of the initial AC coefficient from a constant ( for example, 8). The CA 612 coefficient coding unit provides a subtraction result (difference) for the exp-G 193 unit as the initial coefficient (scaling_list_delta_coef (i = 0)) of the DPCM data of the CA coefficients.
[00342] The DPCM unit of AC coefficient 613 acquires the AC coefficients provided by the overlay determination unit 191, determines, for each of the second and subsequent AC coefficients, the difference (DPCM) of the immediately preceding AC coefficient , and provides the differences determined for the exp-G 193 unit as DPCM data (scaling_list_delta_coef (i = 1 to 63)).
[00343] The DC coefficient DPCM unit 614 acquires the DC coefficient provided by the overlay determination unit 191. The DC coefficient DPCM unit 614 additionally acquires the initial AC coefficient (AC coefficient (0, 0 )) contained in the CA 611 coefficient buffer. The DPCM unit of DC coefficient 614 subtracts the initial AC coefficient (AC coefficient (0, 0)) from the DC coefficient to determine the difference between them, and provides the difference determined for the exp-G 193 unit as DPCM data from the CC coefficient (scaling_list_CC_coef_delta).
[00344] As described above, in the second method, a difference between the CC coefficient and another coefficient (the initial AC coefficient) is determined. Then the difference is transmitted, as DPCM data from the CC coefficient (scaling_list_CC_coef_delta) different from DPCM data from the AC coefficients, after the transmission of DPCM data from the AC coefficients (scaling_list_delta_coef) which is a difference between the coefficients of HERE. Therefore, similarly to the first method, the
Petition 870170066495, of September 6, 2017, p. 82/294 / 137 image encoding device 10 can improve the encoding efficiency of a scale list.
3-3. DPCM process flow [00345] Also in the second method, the image encoding device 10 performs a quantization matrix encoding process in a similar manner to that in the first method described with reference to the flow chart illustrated in Figure 20.
[00346] An example of the flow of a DPCM process in the second method, which is performed in step S112 in Figure 20, will be described with reference to a flow chart illustrated in Figure 32.
[00347] When the DPCM process is started, in step S401, the AC coefficient buffer 611 contains the initial AC coefficient.
[00348] In step S402, the AC coefficient encoding unit 612 subtracts the initial AC coefficient from a predetermined constant (for example, 8) to determine the difference between them (initial DPCM data). [00349] The processing of steps S403 to S405 is performed by the DPCM unit of AC coefficient 613 in a similar way to the processing of steps S133 to S135 in Figure 21. That is, the processing of steps S403 to S405 is performed repeatedly for generate DPCM data for all AC coefficients (the differences from the immediately preceding AC coefficients).
[00350] If it is determined in step S403 that all AC coefficients have been processed (that is, if there is no unprocessed AC coefficient), the DPCM unit of AC coefficient 613 advances the process to step S406.
[00351] In step S406, the DP coefficient of DC coefficient 614 subtracts the initial AC coefficient contained in step S401 from the CC coefficient to determine a difference between them (DPCM data for the CC coefficient).
Petition 870170066495, of September 6, 2017, p. 83/294 / 137 [00352] When the processing of step S406 is completed, the DPCM unit of CC coefficient 614 ends the DPCM process, and returns the process to Figure 20.
[00353] Therefore, a difference between the CC coefficient and another coefficient is also determined and transmitted to an image decoding device as DPCM data. Thus, the image encoding device 10 can suppress an increase in the amount of encoding in a scale list.
3-4. Detailed example configuration of reverse DPCM unit [00354] In the second method, the image decoding device 300 has a configuration basically similar to that in the first method. Specifically, also in the second method, the image decoding device 300 has a configuration as in the example illustrated in Figure 22. In addition, the inverse orthogonal decanting / transforming unit 313 has a configuration as in the example illustrated in Figure 23. In addition , the matrix generating unit 410 has a configuration as in the example illustrated in Figure 24.
[00355] Figure 33 is a block diagram illustrating an example of a detailed configuration of the reverse DPCM unit 552 illustrated in Figure 24 in the second method. Referring to Figure 33, the reverse DPCM unit 552 includes an initial establishment unit 621, an AC coefficient DPCM decoding unit 622, an AC coefficient buffer 623 and a DP coefficient DPCM decoding unit. CC 624.
[00356] The initial establishment unit 621 acquires sizeID and MatrixID, and sets several variables to initial values. The initial settlement unit 621 provides the acquired and established information to the DPCM decoding unit of CA 622 coefficient.
[00357] DPCM decoding unit of AC coefficient 622 acquires DPCM data of AC coefficients
Petition 870170066495, of September 6, 2017, p. 84/294 / 137 (scaling_list_delta_coef) provided with the exp-G 551 unit. The DPCM decoding unit of AC coefficient 622 decodes the DPCM data acquired from the AC coefficients using the initial and similar settings provided from the initial establishment unit 621 to determine AC coefficients. The DPC decoding unit of AC coefficient 622 provides the determined AC coefficients (ScalingList [i]) for the reverse overlap determination unit 553. The DPCM decoding unit of AC coefficient 622 additionally provides the AC coefficient initial (ScalingList [0], that is, the AC coefficient (0, 0)) among the AC coefficients determined for the CA 623 buffer memory for containment.
[00358] The CA 623 buffer memory stores the initial CA coefficient (ScalingList [0], that is, the CA coefficient (0, 0)) provided by the DPCM CA 622 decoding unit. CA coefficient buffer 623 provides the initial AC coefficient (ScalingList [0], that is, the AC coefficient (0, 0)) for the DC coefficient DPCM decoding unit 624 at a predetermined timing or at response to a request.
[00359] The DC coefficient DPCM decoding unit 624 acquires the DC coefficient DPCM data (scaling_list_CC_coef_delta) provided by the exp-G 551 unit. The DC coefficient DPCM decoding unit 624 additionally acquires the coefficient initial CA (ScalingList [0], that is, the CA coefficient (0, 0)) stored in the CA coefficient buffer 623. The DC coefficient DPCM decoding unit 624 decodes the DPCM data from the coefficient of DC using the initial AC coefficient to determine the DC coefficient. The DPCM decoding unit with CC coefficient 624 provides the determined CC coefficient (CC_coef) for the reverse overlap determination unit 553.
[00360] Therefore, the reverse DPCM unit 552 can
Petition 870170066495, of September 6, 2017, p. 85/294 / 137 perform correct DPCM decoding, and can obtain the DC coefficient and the AC coefficients. That is, the image decoding device 300 can suppress an increase in the amount of encoding in a scale list.
3-5. Reverse DPCM process flow [00361] Also in the second method, the image decoding device 300 performs a quantization matrix decoding process in a manner similar to that in the first method described above with reference to the flow chart illustrated in Figure 27. Similarly , the image decoding device 300 performs a residual signal decoding process in a manner similar to that in the first method described above with reference to the flowchart illustrated in Figure 28.
[00362] An example of the reverse DPCM process flow performed by the reverse DPCM unit 552 will be described with reference to a flow chart illustrated in Figure 34.
[00363] When the reverse DPCM process is started, in step S421, the initial establishment unit 621 acquires sizeID and MatrixID. [00364] In step S422, the initial establishment unit 621 establishes coefNum as follows.
coefNum = min ((1 << (4 + (sizeID << 1))), 64) [00365] In step S423, the initial establishment unit 621 establishes a variable i and a nextcoef variable as follows.
i = 0 nextcoef = 8 [00366] In step S424, the DPCM 572 decoding unit determines whether or not variable i <coefNum. If variable i is less than coefNum, initial establishment unit 621 advances the process to step S425.
[00367] In step S425, the DPCM decoding unit of
Petition 870170066495, of September 6, 2017, p. 86/294 / 137 AC coefficient 622 reads DPCM data from AC coefficients (scaling_list_delta_coef).
[00368] In step S426, the DPCM decoding unit of CA 622 coefficient determines nextcoef as below using the DPCM data read, and additionally determines scalingList [i].
nextcoef = (nextcoef + scaling_list_delta_coef + 256)% 256 scalingList [i] = nextcoef [00369] Note that the calculated initial AC coefficient (ScalingList [0], that is, the AC coefficient (0, 0)) is contained in 623 CA coefficient buffer.
[00370] In step S427, the DPCM decoding unit with CA 622 coefficient increments variable i to change the objective to be processed to the subsequent coefficient, and then returns the process to step S424.
[00371] In step S424, the processing of steps S424 to S427 is performed repeatedly until it is determined that variable i is greater than or equals coefNum. If it is determined in step S424 that the variable i is greater than or equal to coefNum, the DPCM decoding unit of CA 622 advances the process to step S428.
[00372] In step S428, the DC coefficient DPCM decoding unit 624 determines whether or not sizeID is greater than 1. If sizeID is determined to be greater than 1, the DC coefficient DPCM decoding unit 624 advances the process to step S429, and reads the DPCM data from the CC coefficient (scaling_list_CC_coef_delta).
[00373] In step S430, the DPCM decoding unit of CC coefficient 624 acquires the initial AC coefficient (ScalingList [0], that is, the AC coefficient (0, 0)) contained in the temporary coefficient memory CA 623, and decodes the DPCM data from the CC coefficient (CC_coef) using the initial AC coefficient as follows.
DC_coef = scaling_list_dc_coef_delta + ScalingList [0]
Petition 870170066495, of September 6, 2017, p. 87/294 / 137 [00374] When the CC coefficient (DC_coef) is obtained, the DPCM decoder unit of CC coefficient 624 ends the reverse DPCM process, and returns the process to Figure 28.
[00375] Furthermore, if it is determined in step S428 that sizeID is less than or equal to 1, the DPCM decoder unit of CC coefficient 624 ends the reverse DPCM process, and returns the process to Figure 28.
[00376] Therefore, the difference between the DC coefficient and the AC coefficient located at the beginning of the AC coefficients can be decoded correctly. Therefore, the image decoding device 300 can suppress an increase in the amount of encoding in a scale list.
4. Fourth embodiment
4-1. Syntax: Third method [00377] In the second method described above, the DC coefficient can also be limited to a value lower than the initial AC coefficient (AC coefficient (0, 0)) (third method).
[00378] This ensures that the DPCM data of the CC coefficient, that is, a difference value obtained by subtracting the initial AC coefficient from the CC coefficient, can be a positive value. This DPCM data can thus be encoded using unsigned exponential Golomb codes. Therefore, the third method can prevent the CC coefficient from being greater than the initial AC coefficient, but it can reduce the amount of coding compared to the first method and the second method.
[00379] Figure 35 illustrates the syntax of a scale list in the third method. As shown in Figure 35, in this case, the DPCM data for the CC coefficient (scaling_list_CC_coef_delta) is limited to a positive value.
[00380] The syntax for the third method described above can be
Petition 870170066495, of September 6, 2017, p. 88/294 / 137 implemented by an image encoding device 10 similar to that in the second method. In the third method, however, the exp-G 193 unit can encode the DPCM data of the CC coefficient using unsigned exponential Golomb codes. Note that the image encoding device 10 can perform processes such as a quantization matrix encoding process and a DPCM process in a similar manner to that in the second method.
[00381] In addition, the syntax for the third method can be implemented by the image decoding device 300 in a similar way to that in the second method. In addition, the image decoding device 300 can perform a quantization matrix decoding process in a manner similar to that in the second method.
4-2. Reverse DPCM process flow [00382] An example of the reverse DPCM process flow performed by the reverse DPCM unit 552 will be described with reference to a flow chart illustrated in Figure 36.
[00383] The processing of steps S451 to S459 is performed in a similar way to the process of steps S421 to S429 in Figure 34.
[00384] In step S460, the DPCM decoding unit of CC coefficient 624 acquires the initial AC coefficient (ScalingList [0], that is, the AC coefficient (0, 0)) contained in the coefficient buffer memory CA 623, and decodes the DPCM data of the CC coefficient (CC_coef) as below using the initial AC coefficient.
DC_coef = ScalingList [0] - scaling_list_dc_coef_delta [00385] When the CC coefficient (DC_coef) is obtained, the DPCM decoding unit of CC coefficient 624 ends the reverse DPCM process, and returns the process to Figure 28.
[00386] Furthermore, if it is determined in step S458 that sizeID is less than or equal to 1, the DPCM decoding unit of CC coefficient 624 ends the reverse DPCM process, and returns the process to the
Petition 870170066495, of September 6, 2017, p. 89/294 / 137
Figure 28.
[00387] Therefore, the difference between the DC coefficient and the AC coefficient located at the beginning of the AC coefficients can be decoded correctly. Therefore, the image decoding device 300 can suppress an increase in the amount of encoding in a scale list.
5. Fifth embodiment
5-1. Syntax: Fourth method [00388] Another method for transmitting a difference between the CC coefficient and another coefficient, instead of the CC coefficient, can be, for example, collecting only the CC coefficients from a plurality of scale lists and executing DPCM taking differences between the CC coefficients separately from the AC coefficients of the individual scale lists (fourth method). In this case, DPCM data from the CC coefficients is a collection of pieces of data for the plurality of scale lists, and are transmitted as data other than DPCM data from the CA coefficients of the individual scale lists.
[00389] Therefore, the compression ratio can be further improved when, for example, there are correlations between the CC coefficients of the scale lists (MatrixID).
[00390] Figure 37 illustrates the syntax for the CC coefficient of a scale list in the fourth method. In this case, since the CC coefficients are processed in different cycles than those for the AC coefficients of the individual scale lists, as illustrated in the example shown in Figure 37, processes for the AC coefficients and processes for the CC coefficients need to be independent of each other.
[00391] This ensures that several more methods for scale list encoding and decoding processes can be achieved although the complexity of the DPCM process and the reverse DPCM process can be increased. For example, a process to copy only the coefficients of
Petition 870170066495, of September 6, 2017, p. 90/294 / 137
AC and making the values of the DC coefficients different in the copy mode can be implemented easily.
[00392] The number of scale lists in which the CC coefficients are processed collectively is arbitrary.
5-2. Detailed example configuration of DPCM unit [00393] In the fourth method, the image encoding device 10 has a configuration basically similar to that in the first method described above. Specifically, the image encoding device 10 has a configuration as in the example illustrated in Figure 14. In addition, the orthogonal transform / quantization unit 14 has a configuration as in the example illustrated in Figure 15. In addition, the image processing unit matrix 150 has a configuration as in the example illustrated in Figure 16.
[00394] An example configuration of the DPCM unit 192 in the fourth method is illustrated in Figure 38. As illustrated in Figure 38, in this case, the DPCM unit 192 includes a DPCM unit with AC coefficient 631, a buffer of coefficient of CC 632 and a DPCM unit of coefficient of CC 633.
[00395] The DPCM unit of AC coefficient 631 performs a DPCM process of the individual AC coefficients of each scale list which are provided the overlay determination unit 191. Specifically, the DPCM unit of AC coefficient 631 subtracts , for each scale list, the initial AC coefficient of a predetermined constant (for example, 8), and subtracts the AC coefficient being processed (current AC coefficient) from the immediately preceding AC coefficient. The DPCM unit of CA 631 coefficient provides DPCM data (scaling_list_delta_coef) generated for each scale list to the exp-G 193 unit.
[00396] The DC coefficient buffer 632 stores the DC coefficients of the individual scale lists provided by the control unit.
Petition 870170066495, of September 6, 2017, p. 91/294 / 137 overlap determination 191. DC coefficient buffer 632 provides the DC coefficients stored for the DC coefficient DPCM unit 633 at a predetermined timeout or in response to a request.
[00397] The DC coefficient DPCM unit 633 acquires the DC coefficients accumulated in the DC coefficient buffer 632. The DC coefficient DPCM unit 633 determines DPCM data of the acquired DC coefficients. Specifically, the DPCM unit of CC coefficient 633 subtracts the initial CC coefficient from a predetermined constant (for example, 8), and subtracts the CC coefficient being processed (current CC coefficient) from the immediately preceding CC coefficient. The DC 633 DPCM unit provides the generated DPCM data (scaling_list_delta_coef) for the 193 exp-G unit.
[00398] Therefore, the image encoding device 10 can improve the encoding efficiency of a scale list.
5-3. DPCM process flow [00399] Also in the fourth method, the image encoding device 10 performs a quantization matrix encoding process in a similar manner to that in the first method described above with reference to the flow chart illustrated in Figure 20.
[00400] An example of the flow of a DPCM process in the fourth method that is performed in step S112 in Figure 20 will be described with reference to a flow chart illustrated in Figure 39.
[00401] The processing of steps S481 to S485 is performed in a way by the DPCM unit of CA 631 coefficient similar to the processing of steps S401 to S405 (processing in the second method) in Figure 32.
[00402] If it is determined in step S483 that all AC coefficients have been processed, the DP coefficient unit of AC 631 advances
Petition 870170066495, of September 6, 2017, p. 92/294 / 137 the process to step S486.
[00403] In step S486, the DPCM unit of CA 631 coefficient determines whether or not all scale lists (or difference matrices) in which the CC coefficients are collectively encoded in DPCM have been processed. If it is determined that there is an unprocessed scale list (or difference matrix), the DP coefficient DPCM unit of CA 631 returns the process to step S481.
[00404] If it is determined in step S486 that all scale lists (or difference matrices) have been processed, the DPCM unit of AC coefficient 631 advances the process to step S487.
[00405] The DPCM unit of DC coefficient 633 performs the processing of steps S487 to S491 in the DC coefficients stored in a way in the buffer memory of DC 632 similar to the process of steps S481 to S485.
[00406] If it is determined in step S489 that all DC coefficients stored in the DC coefficient buffer 632 have been processed, the DC coefficient DPCM unit 633 ends the DPCM process, and returns the process to Figure 20.
[00407] By performing a DPCM process in the manner described above, the image encoding device 10 can improve the encoding efficiency of a scale list.
5-4. Detailed example configuration of reverse DPCM unit [00408] The image decoding device 300 in the fourth method has a configuration basically similar to that in the first method. Specifically, also in the fourth method, the image decoding device 300 has a configuration as in the example shown in Figure 22. Additionally, the inverse orthogonal decanting / transforming unit 313 has a configuration as in the example shown in Figure 23. In addition, the matrix generating unit 410 has a configuration as in the example illustrated in Figure 24.
Petition 870170066495, of September 6, 2017, p. 93/294 / 137 [00409] Figure 40 is a block diagram illustrating an example of a detailed configuration of the reverse DPCM unit 552 illustrated in Figure 24 in the fourth method. Referring to Figure 40, the reverse DPCM unit 552 includes an initial establishment unit 641, an AC coefficient DPCM decoding unit 642 and a DC coefficient DPCM decoding unit 643.
[00410] The initial establishment unit 641 acquires sizeID and MatrixID, and sets several variables to initial values. The initial establishment unit 641 provides the information acquired and established to the DPCM decoding unit of AC 642 coefficient and to the DPCM decoding unit of DC 643 coefficient.
[00411] The DPCM decoding unit of AC coefficient
642 acquires DPCM data from the AC coefficients (scaling_list_delta_coef (ac)) provided by the exp-G 551 unit. The DP coefficient DPCM decoding unit 642 decodes the DPCM data acquired from the AC coefficients using the initial settings and similar provided with the initial establishment unit 641, and determines AC coefficients. The CA coefficient DPCM decoding unit 642 provides the determined CA coefficients (ScalingList [i]) for the reverse overlap determination unit 553. The CA coefficient DPCM decoding unit 642 performs the process described above in a plurality of scale lists.
[00412] The DC coefficient DPCM decoding unit
643 acquires the DPCM data of the CC coefficient (scaling_list_delta_coef (dc)) provided by the exp-G 551 unit. The DPCM decoding unit of the CC coefficient 643 decodes the DPCM data acquired from the CC coefficient using the initial settings and similar provided from initial establishment unit 641, and determines CC coefficients from the individual scale lists. The DPCM decoding unit with CC 643 coefficient provides the coefficients of
Petition 870170066495, of September 6, 2017, p. 94/294 / 137
Determined CC (scaling_list_CC_coef) for the reverse overlay determination unit 553.
[00413] Therefore, the reverse DPCM unit 552 can perform correct DPCM decoding, and can obtain the DC coefficients and the AC coefficients. That is, the image decoding device 300 can suppress an increase in the amount of scalar list encoding.
5-5. Reverse DPCM process flow [00414] Also in the fourth method, the image decoding device 300 performs a quantization matrix decoding process in a manner similar to that in the first method described above with reference to the flow chart illustrated in Figure 27. Similarly , the image decoding device 300 performs a residual signal decoding process in a manner similar to that in the first method described above with reference to the flowchart illustrated in Figure 28. [00415] An example of the flow of a reverse DPCM process performed by the unit of reverse DPCM 552 will be described with reference to a flow chart illustrated in Figures 41 and 42.
[00416] When the reverse DPCM process is started, the initial settlement unit 641 and the DP coefficient DPCM decoding unit 642 perform the processing of steps S511 to S517 in a similar manner to that in the process of steps S421 to S427 in Figure 34.
[00417] If it is determined in step S514 that the variable i is greater than or equal to coefNum, the DPCM decoding unit of CA 642 coefficient advances the process to step S518.
[00418] In step S518, the DPCM decoding unit of CA 642 coefficient determines whether or not all scale lists (difference matrices) in which the CC coefficients are collectively subjected to a DPCM process have been processed. If it is determined that there is a list
Petition 870170066495, of September 6, 2017, p. 95/294 / 137 unprocessed scale (difference matrix), the CA 642 DPCM decoding unit returns the process to step S511, and repeatedly performs the subsequent process.
[00419] Also, if it is determined that there is no unprocessed scale list (difference matrix), the DPCM decoding unit of CA 642 coefficient advances the process to Figure 42.
[00420] In step S521 in Figure 42, the initial establishment unit 641 establishes sizeID and a nextcoef variable as follows.
sizeID = 2 nextcoef = 8 [00421] In addition, in step S522, the initial establishment unit 641 establishes MatrixID as follows.
MatrixID = 0 [00422] In step S523, the DC coefficient DPCM decoding unit 643 determines whether or not sizeID <4. If sizeID is determined to be less than 4, the DC coefficient DPCM decoding unit 643 proceeds the process to step S524.
[00423] In step S524, the DC coefficient DPCM decoding unit 643 determines whether or not MatrixID <(sizeID == 3) 2: 6 is satisfied. If it is determined that MatrixID <(sizeID == 3) 2: 6 is satisfied, the DC coefficient DPCM decoding unit 643 proceeds to step S525.
[00424] In step S525, the DC coefficient DPCM decoding unit 643 reads the DC coefficient DPCM data (scaling_list_delta_coef).
[00425] In step S526, the DPCM decoding unit with CC coefficient 643 determines nextcoef as below using the DPCM data read, and additionally determines scaling_CC_coef.
nextcoef = (nextcoef + scaling_list_delta_coef + 256)% 256 scaling_dc_coef [sizeID - 2] [MatrixID] = nextcoef
Petition 870170066495, of September 6, 2017, p. 96/294 / 137 [00426] In step S527, the DC coefficient DPCM decoding unit 643 increments MatrixID to change the processing target to the subsequent CC coefficient (the subsequent scale list or residual matrix), and then returns the process to step S524. [00427] If it is determined in step S524 that MatrixID <(sizeID ==
3) 2: 6 is not satisfied, the DPCM decoding unit with DC coefficient 643 advances the process to step S528.
[00428] In step S528, the DPCM DC coefficient decoding unit 643 increments sizeID to change the processing target to the subsequent CC coefficient (the subsequent scale list or residual matrix), and then returns the process to step S523 . [00429] If it is determined in step S523 that sizeID is greater than or equal to 4, the DC coefficient DPCM decoding unit 643 ends the reverse DPCM process, and returns the process to Figure 28. [00430] By therefore, the differences between CC coefficients can be decoded correctly. Therefore, the image decoding device 300 can suppress an increase in the amount of encoding of scale lists.
6. Sixth embodiment
6-1. Another syntax: First example [00431] Figure 43 illustrates another example of the syntax for a scale list. This drawing corresponds to Figure 12. In the example illustrated in Figure 12, the initial value of nextcoef is established at a predetermined constant (for example, 8). Alternatively, as shown in Figure 43, the initial value of nextcoef can be written with the DPCM data of the CC coefficient (scaling_list_CC_coef_minus8).
[00432] Therefore, the amount of encoding of the initial AC coefficients (AC coefficients (0, 0)) in a 16x16 scale list and a 32x32 scale list can be reduced.
6-2. Another syntax: Second example
Petition 870170066495, of September 6, 2017, p. 97/294 / 137 [00433] Figure 44 illustrates another example of the syntax for a scale list. This drawing corresponds to Figure 12.
[00434] In the example illustrated in Figure 12, when the value of scaling_list_pred_matrix_id_delta, which is information that specifies the reference destination in copy mode, is 0, the scale list that precedes the current scale list being processed by a scale list is referenced, and when the scaling_list_pred_matrix_id_delta value is 1, the scale list that precedes the current scale list being processed by two scale lists is referred to. [00435] In contrast, in the example illustrated in Figure 44, as illustrated in part C of Figure 44, when the value of scaling_list_pred_matrix_id_delta, which is information that specifies the reference destination in copy mode is 0, the default scale list is referred, and when the value of scaling_list_pred_matrix_id_delta is 1, the immediately preceding scale list is referred.
[00436] In this way, modifying the semantics of scaling_list_pred_matrix_id_delta can simplify the syntax in a way illustrated in part B of Figure 44 and can reduce the burden of the DPCM process and the reverse DPCM process.
6-3. Another syntax: Third example [00437] Figure 45 illustrates another example of the syntax for a scale list. This drawing corresponds to Figure 12.
[00438] In the example shown in Figure 45, both of the example shown in Figure 43 and the example shown in Figure 44 described above are used.
[00439] In the example illustrated in Figure 45, therefore, the amount of encoding of the initial AC coefficients (AC coefficients (0, 0)) in a 16x16 scale list and a 32x32 scale list can be reduced. In addition, syntax can be simplified and the load of the DPCM process and the reverse DPCM process can be reduced.
[00440] In the preceding embodiments, the values of the
Petition 870170066495, of September 6, 2017, p. 98/294 / 137 predetermined constants are arbitrary. In addition, the scale list sizes are also arbitrary.
[00441] Furthermore, while the preceding description was given of a size transformation process for a scale list, a prediction matrix, or a difference matrix between them, the size transformation process can be a process for generating in fact a matrix whose size has been transformed, or it can be a process to establish how to read each element in a matrix from a memory (matrix data read control) without actually generating data from the matrix.
[00442] In the size transformation process described above, each element in a matrix whose size has been transformed consists of any of the elements in the matrix whose size has not yet been transformed. That is, a matrix whose size has been transformed can be generated by reading elements in a matrix whose size has not yet been transformed, which is stored in memory, using a certain method such as reading some of the elements in the matrix or reading an element a plurality of times. In other words, a method for reading each element is defined (or matrix data read control is performed) to substantially implement the size transformation described above. This method can remove a process such as writing matrix data whose size has been transformed into memory. In addition, the reading of matrix data whose size has been transformed depends basically on how to perform closer and similar neighbor interpolation, and therefore size transformation can be implemented by a comparatively low load process such as selecting an appropriate one from a plurality of options prepared in advance. Therefore, the method described above can reduce the size transformation load. [00443] That is, the size transformation process described above includes a process to actually generate matrix data whose size has been transformed and also includes control of reading the matrix data.
Petition 870170066495, of September 6, 2017, p. 99/294 / 137 [00444] Note that while the preceding description was made in the context of a difference matrix being encoded and transmitted, this is purely illustrative and a scale list can be encoded and transmitted. In other words, the AC coefficients and CC coefficient of a scale list that was described above as coefficients to be processed can be the AC coefficients and CC coefficient of a difference matrix between a scale list and a matrix of prediction.
[00445] In addition, the amount of coding for information about parameters, flags, and so on from a scale list, such as the size and list ID of the scale list, can be reduced, for example, by making a difference between the information and the information previously transmitted and transmitting the difference.
[00446] Furthermore, while the preceding description was made in the context of a quantization matrix or a large size difference matrix being converted by downward conversion and transmitted, this is merely illustrative and a quantization matrix or a difference matrix it can be transmitted without being converted by downward conversion, while the size of the quantization matrix used for quantization is kept unchanged.
[00447] The present technology can be applied to any type of image encoding and decoding that involves quantization and decanting.
[00448] In addition, the present technology can also be applied to, for example, an image encoding device and an image decoding device used to receive compressed image information (bit stream) using an orthogonal transform such as an discrete cosine transform and motion compensation, such as MPEG or H.26x, by a network medium such as satellite broadcasting, cable television, the Internet, or a mobile phone. The present technology can also be applied to an image encoding device and an
Petition 870170066495, of September 6, 2017, p. 100/294 / 137 image decoding device used for processing on a storage medium such as an optical disc, a magnetic disc, and a flash memory. In addition, the present technology can also be applied to a quantizing device and a decanting device included in the image encoding device and the image decoding device described above, and the like.
7. Seventh embodiment
Application to multivision image encoding and multivision image decoding [00449] The series of processes described above can be applied to multivision image encoding and multivision image decoding. Figure 46 illustrates an example of a multivision image encoding scheme.
[00450] As illustrated in Figure 46, multivision images include images from a plurality of points of view (or views). The plurality of views in multivision images include base views, each of which is encoded and decoded using an image of it without using an image from another view, and non-base views, each of which is encoded and decoded using an image of another eyesight. Each of the non-base views can be encoded and decoded using an image from a base view or using an image from any other non-base view.
[00451] When the multivision images illustrated in Figure 46 are to be encoded and decoded, an image of each view is encoded and decoded. The method described above in the preceding embodiments can be applied to the encoding and decoding of each view. This can suppress a reduction in the image quality of the individual views.
[00452] In addition, flags and parameters used in the method described above can be shared in the previous embodiments in the encoding and decoding of each view. This can suppress a reduction
Petition 870170066495, of September 6, 2017, p. 101/294 / 137 in coding efficiency.
[00453] More specifically, for example, information relating to a scale list (for example, parameters, flags, and so on) can be shared in the encoding and decoding of each view.
[00454] Needless to say, any other necessary information can be shared in the encoding and decoding of each view.
[00455] For example, when a scale list or information relating to the scale list that is included in a sequence parameter set (SPS) or a frame parameter set (PPS) is to be transmitted, if these (SPS and PPS) are shared between views, the scale list or information related to the scale list is also shared accordingly. This can suppress a reduction in coding efficiency.
[00456] In addition, matrix elements in a scale list (or quantization matrix) for a base view can be changed according to disparity values between views. In addition, an offset value for fitting a non-base view matrix element with respect to a matrix element in a scale list (quantization matrix) for a base view can be transmitted. Therefore, an increase in the amount of coding can be suppressed.
[00457] For example, a scale list for each view can be transmitted separately in advance. When a scale list is to be changed for each view, only information indicating the difference from the corresponding scalar lists transmitted in advance can be transmitted. The information indicating the difference is arbitrary, and can be, for example, information in units of 4x4 or 8x8 or a difference between matrices.
[00458] Note that if a scale list or information relating to the scale list is shared between views although an SPS or PPS is not shared, the SPSs or PPSs for other views may be able to be
Petition 870170066495, of September 6, 2017, p. 102/294 / 137 referrals (that is, scale lists or information regarding scalar lists for other views can be used).
[00459] Furthermore, if such multivision images are represented as images having, as components, YUV images and images of depth (depth) corresponding to the amount of disparity between views, an independent scale list or information relating to the scale list for the image of each component (Y, U, V, and Depth) can be used.
[00460] For example, since an image of depth (Depth) is an image of a border, scale lists are not necessary. Thus, although an SPS or PPS specifies the use of a scale list, a scale list cannot be applied (or a scale list to which all matrix elements are the same (or plane) can be applied) for a depth image (Depth).
Multivision image encoding device [00461] Figure 47 is a diagram illustrating a multivision image encoding device for performing the multivision image encoding operation described above. As illustrated in Figure 47, a multivision image encoding device 700 includes an encoding unit 701, an encoding unit 702 and a multiplexing unit 703.
[00462] The coding unit 701 encodes an image of a base view, and generates an image stream of the base coded view. The coding unit 702 encodes an image from a non-base view, and generates a non-base coded view image stream. The multiplexing unit 703 multiplexes the base coded view image stream generated by the coding unit 701 and the non base coded view image stream generated by the coding unit 702, and generates a coded multivision image stream.
[00463] The image coding device 10 (Figure 14) can
Petition 870170066495, of September 6, 2017, p. 103/294 / 137 be used for each of the coding unit 701 and the coding unit 702 of the multivision image coding device 700. That is, an increase in the amount of coding of a scale list in the coding of each view can be suppressed, and a reduction in image quality for each view can be suppressed. In addition, the coding unit 701 and the coding unit 702 can perform processes such as quantizing and decanting using the same flags or parameters (that is, flags and parameters can be shared). Therefore, a reduction in coding efficiency can be suppressed.
Multivision image decoding device [00464] Figure 48 is a diagram illustrating a multivision image decoding device to perform the multivision image decoding operation described above. As illustrated in Figure 48, a multivision image decoding device 710 includes a demultiplexing unit 711, a decoding unit 712 and a decoding unit 713.
[00465] The demultiplexing unit 711 demultiplexes an encoded multivision image stream in which an encoded base view image stream and an encoded non-base view image stream have been multiplexed, and extracts the encoded base view image stream and the image stream non-coded base view. The decoding unit 712 decodes the coded base view image stream extracted by the demultiplexing unit 711, and obtains an image of a base view. The decoding unit 713 decodes the encoded non-base view image stream extracted by the demultiplexing unit 711, and obtains an image from a non-base view.
[00466] The image decoding device 300 (Figure 22) can be used for each of the decoding unit 712 and the decoding unit 713 of the image decoding device of
Petition 870170066495, of September 6, 2017, p. 104/294 / 137 multivision 710. That is, an increase in the amount of coding of a scale list in the decoding of each view can be suppressed, and a reduction in the image quality of each view can be suppressed. In addition, decoding unit 712 and decoding unit 713 can perform processes such as quantizing and decanting using the same flags and parameters (that is, flags and parameters can be shared). Therefore, a reduction in coding efficiency can be suppressed.
8. Eighth embodiment
Application to layered image encoding and layered image decoding [00467] The series of processes described above can be applied to layered image encoding and layered image decoding (scalable encoding and scalable decoding). Figure 49 illustrates an example of a layered image encoding scheme.
[00468] Layered image encoding (scalable encoding) is a process for dividing an image into a plurality of layers (layered formation) to provide image data with the scalability function for a predetermined parameter and for encoding the layers individual. Layered image decoding (scalable decoding) is a decoding process corresponding to the layered image encoding.
[00469] As illustrated in Figure 49, in formation in image layers, an image is divided into a plurality of sub-images (or layers) using as a reference a predetermined parameter with a grading capability function. That is, images decomposed into layers (or images in layers) include multiple images in layers (or layer) having values different from the predetermined parameter. The plurality of layers in the layered images include base layers, each of which is encoded and decoded using an image thereof.
Petition 870170066495, of September 6, 2017, p. 105/294 / 137 without using an image from another layer, and non-base layers (also called enhancement layers), each of which is encoded and decoded using an image from another layer. Each of the non-base layers can be encoded and decoded using an image of a base layer or using an image of any other non-base layer. [00470] In general, each of the non-base layers is composed of data from a difference image (difference data) between an image of that and an image from another layer in order to reduce redundancy. For example, in a case where an image is decomposed into two layers, that is, a base layer and a non-base layer (also called an enhancement layer), an image with a lower quality than the original image can be obtained using only the data from the base layer, and the original image (that is, an image with a high quality) can be obtained by combining the data from the base layer and the data from the non-base layer. [00471] Layering an image in the manner described above can facilitate obtaining images with a wide variety of qualities depending on situations. This ensures that image compression information can be transmitted from a server according to the capabilities of terminals and networks without implementing transcoding such that, for example, image compression information in only base layers is transmitted to terminals having low processing capacities, such as such as mobile phones, to reproduce moving images having a low spatial-temporal resolution or low quality, and image compression information on enhancement layers in addition to base layers is transmitted to terminals having high processing capabilities, such as television sets and personal computers, to reproduce images in motion having a high spatio-temporal resolution or a high quality.
[00472] When layered images as in the example illustrated in Figure 49 are to be encoded and decoded, one image from each
Petition 870170066495, of September 6, 2017, p. 106/294 / 137 layer is encoded and decoded. The method described above in each of the preceding embodiments can be applied to the encoding and decoding of each layer. This can suppress a reduction in the image quality of the individual layers.
[00473] In addition, flags and parameters used in the method described above can be shared in each of the preceding embodiments in the encoding and decoding of each layer. This can suppress a reduction in coding efficiency.
[00474] More specifically, for example, information related to a scale list (for example, parameters, flags, and so on) can be shared in the encoding and decoding of each layer. [00475] Needless to say, any other necessary information can be shared in the encoding and decoding of each layer. [00476] Examples of layered images include layered images in spatial resolution (also called scaling capability for spatial resolution) (spatial scaling capability). In layered images capable of scaling spatial resolution, image resolutions differ from layer to layer. For example, a layer of an image having the spatially lower resolution is designated as a base layer, and a layer of an image having a higher resolution than the base layer is designated as a non-base layer (an enhancement layer).
[00477] Image data from a non-base layer (an enhancement layer) can be data independent of the other layers, and, similarly to the base layers, an image having a resolution equivalent to the resolution of that layer can be obtained only using the data of image. Generally, however, image data from a non-base layer (an enhancement layer) is data corresponding to an image of difference between the image on that layer and an image on another layer (for example, a layer a layer under that layer). In this
Petition 870170066495, of September 6, 2017, p. 107/294 / 137 case, an image having a resolution equivalent to that of a base layer is obtained only using the image data of the base layer while an image having a resolution equivalent to that of a non-base layer (an enhancement layer) is obtained by combining the image data from that layer and the image data from another layer (for example, a layer one layer under that layer). This can suppress redundancy of image data between layers.
[00478] In layered images having the ability to scale the spatial resolution described above, the resolutions of the images differ from layer to layer. Thus, the resolutions of the processing units by which the individual layers are encoded and decoded also differ. Therefore, if a scale list (quantization matrix) is shared in the coding and decoding of the individual layers, the scale list (quantization matrix) can be converted by upward conversion according to the resolution ratios of the individual layers. [00479] For example, it is assumed that an image of a base layer has a resolution of 2K (for example, 1920x1080), and an image of a non-base layer (an enhancement layer) has a resolution of 4K (for example, 3840x2160). In this case, for example, the 16x16 size of the basic layer image (2K image) corresponds to the 32x32 size of the non-base layer image (4K image). The scale list (quantization matrix) is converted by upward conversion as appropriate according to the resolution ratio.
[00480] For example, a 4x4 quantization matrix used for the quantization and decanting of a base layer is converted by upward conversion to 8x8 in the quantization and decanting of a non-base layer and is used. Similarly, an 8x8 scale list of a base layer is converted by upward conversion to 16x16 into a non-base layer. Similarly, a quantization matrix converted by upward conversion to 16x16 into a base layer and used is converted by conversion
Petition 870170066495, of September 6, 2017, p. 108/294
100/137 ascending to 32x32 in a non-base layer.
[00481] Note that the parameter for which the grading capacity is provided is not limited to spatial resolution, and examples of the parameter may include temporal resolution (temporal grading capacity). In layered images having the ability to scale temporal resolution, frame rates differ from layer to layer. Other examples include bit depth scaling capability in which the bit depth of image data differs from layer to layer, and chroma grading capability in which component shape differs from layer to layer.
[00482] Still other examples include SNR scaling capability in which signal-to-noise ratios (SNRs) in images differ from layer to layer.
[00483] Due to the improvement in image quality, desirably, the lower the signal to noise ratio an image has, the smaller the quantization error is made. For this purpose, in SNR grading capacity, desirably, different scale lists (unusual scale lists) are used for the quantization and decanting of the individual layers according to the signal-to-noise ratio. For this reason, as described above, if a scale list is shared between layers, an offset value for adjusting matrix elements for an enhancement layer with respect to matrix elements in a scale list for a base layer can be transmitted. . More specifically, information indicating the difference between a common scale list and a scale list actually used can be transmitted on a layer-by-layer basis. For example, information indicating the difference can be transmitted in a sequence parameter set (SPS) or frame parameter set (PPS) for each layer. The information indicating the difference is arbitrary. For example, the information can be a matrix having elements representing values of difference between corresponding elements in both scale lists, or
Petition 870170066495, of September 6, 2017, p. 109/294
101/137 can be a function indicating the difference.
Layered image encoding device [00484] Figure 50 is a diagram illustrating a layered image encoding device for performing the layered image encoding operation described above. As illustrated in Figure 50, a layered image encoding device 720 includes an encoding unit 721, an encoding unit 722 and a multiplexing unit 723.
[00485] The encoding unit 721 encodes a base layer image, and generates an encoded base layer image stream. The encoding unit 722 encodes a non-base layer image, and generates an encoded non-base layer image stream. The multiplexing unit 723 multiplexes the encoded base layer image stream generated by the encoding unit 721 and the non-encoded layer image stream generated by the encoding unit 722, and generates an encoded layered image stream.
[00486] The image encoding device 10 (Figure 14) can be used for each of the encoding unit 721 and the encoding unit 722 of the layered image encoding device 720. That is, an increase in the amount of encoding of a scale list in the coding of each layer can be suppressed, and a reduction in the image quality of each layer can be suppressed. In addition, coding unit 721 and coding unit 722 can perform processes such as quantizing and decanting using the same flags or parameters (that is, flags and parameters can be shared). Therefore, a reduction in coding efficiency can be suppressed.
Layered image encoding device [00487] Figure 51 is a diagram illustrating a layered image encoding device for performing the scanning operation
Petition 870170066495, of September 6, 2017, p. 110/294
102/137 layered image decoding described above. As illustrated in Figure 51, a layered image encoding device 730 includes a demultiplexing unit 731, a decoding unit 732 and a decoding unit 733.
[00488] Demultiplexing unit 731 demultiplexes an encoded layered image stream in which an encoded base layer image stream and an encoded non-base layer image stream have been multiplexed, and extracts the encoded layer base image stream and the encoded non-base layer image stream. The decoding unit 732 decodes the encoded base layer image stream extracted by the demultiplexing unit 731, and obtains an image of a base layer. The decoding unit 733 decodes the encoded non-base layer image stream extracted by the demultiplexing unit 731, and obtains an image of a non-base layer.
[00489] The image decoding device 300 (Figure 22) can be used for each of the decoding unit 732 and the decoding unit 733 of the layered image encoding device 730. That is, an increase in the amount of encoding of a scale list in the decoding of each layer can be suppressed, and a reduction in the image quality of each layer can be suppressed. In addition, decoding unit 712 and decoding unit 713 can perform processes such as quantizing and decanting using the same flags or parameters (that is, flags and parameters can be shared). Thus, a reduction in coding efficiency can be suppressed.
9. Ninth Embodiment
Computer [00490] The series of processes described above can be performed through hardware or can also be performed through software. In this case, the series of processes can be implemented as, for example, a
Petition 870170066495, of September 6, 2017, p. 111/294
103/137 computer illustrated in Figure 52.
[00491] In Figure 52, a CPU (Central Processing Unit)
801 on a computer 800 performs various processing operations according to a program stored in a ROM (Read-only memory)
802 or a program loaded in RAM (Random Access Memory) 803 from a storage unit 813. RAM 803 also stores, as desired, data and the like necessary for CPU 801 to perform various processing operations.
[00492] CPU 801, ROM 802 and RAM 803 are connected to each other by a bus 804. An input / output interface 810 is also connected to bus 804.
[00493] The input / output interface 810 is connected to an input unit 811, an output unit 812, the storage unit 813 and a communication unit 814. The input unit 811 includes a keyboard, a mouse, a touch panel, an input terminal, and so on. The 812 output unit includes desired output devices, such as a speaker and a display including a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), and an OELD (Organic Electroluminescence Display), a output terminal, and so on. The storage unit 813 includes a desired storage medium such as a hard disk or flash memory, and a control unit that controls the entry and exit of the storage medium. The communication unit 814 includes desired wired or wireless communication devices such as a modem, a LAN interface, a USB (Universal Serial Bus) device, and a Bluetooth device (trademark). The communication unit 814 performs communication processing with other communication devices over networks including, for example, the Internet.
[00494] A disk unit 815 is additionally connected to the input / output interface 810, if necessary. A removable medium 821 such
Petition 870170066495, of September 6, 2017, p. 112/294
104/137 as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is placed in the disk unit 815, as desired. The disk drive 815 reads a computer program, data, and the like from the removable medium 821 placed on it according to control, for example, of CPU 801. The data read and computer program are provided, for example, to RAM 803. The computer program read from removable medium 821 is additionally installed on storage unit 813, if necessary.
[00495] When the series of processes described above is executed through software, a program constituting the software is installed from a network or a recording medium.
[00496] Examples of the recording medium include, as illustrated in Figure 52, the removable medium 821, which is distributed separately from the device body to deliver the program to a user, such as a magnetic disk (including a floppy disk), a optical disc (including a CDROM (Compact Disc Read Only Memory) and a DVD (Digital Versatile Disc)), a magneto-optical disc (including an MD (Mini Disc)), or a semiconductor memory in which the program is recorded. Other examples of the recording medium include devices distributed to a user in a way to be incorporated in advance in the device body, such as the ROM 802 and the hard disk included in the storage unit 813 in which the program is recorded.
[00497] Note that the program that computer 800 executes may be a program in which processing operations are performed in a time series manner in the order stated here, or it may be a program in which processing operations are performed in parallel or necessary timings such as when called.
[00498] Furthermore, steps describing a program stored on a recording medium, as used here, certainly include processing operations performed in a time series in order
Petition 870170066495, of September 6, 2017, p. 113/294
105/137 declared, and processing operations performed in parallel or individually, but not necessarily executed in a time series manner.
[00499] In addition, the term system, as used here, refers to a set of constituent elements (devices, modules (components), etc.) regardless of whether all the constituent elements are accommodated in the same housing or not. Thus, a plurality of devices accommodated in separate housings and connected by a network, and a single device including a plurality of modules accommodated in a single housing is defined as a system.
[00500] In addition, a configuration described above as a single device (or processing units) can be divided into a plurality of devices (or processing units). Conversely, configurations described above as a plurality of devices (or processing units) can be combined into a single device (or processing unit). In addition, of course, a different configuration from the one described above can be added to the configuration of each device (or each processing unit). In addition, part of the configuration of a certain device (or processing unit) can be included in the configuration of another device (or another processing unit) if the devices (or processing units) have substantially the same configuration and / or operation in terms of an entire system. In other words, embodiments of the present technology are not limited to the preceding embodiments, and a variety of modifications can be made without departing from the extent of the present technology.
[00501] While preferred embodiments of the present exhibition have been described in detail with reference to the accompanying drawings, the technical extent of the present exhibition is not limited to the examples shown here. It is apparent that a person having knowledge
Petition 870170066495, of September 6, 2017, p. 114/294
106/137 ordinary in the technical field of the present exhibition could achieve several changes or modifications without departing from the extension of the technical concept as defined in the claims, and it is to be understood that such changes or modifications also fall within the technical extension of the present exhibition as usual .
[00502] For example, the present technology can be implemented with a cloud computing configuration in which a plurality of devices share and cooperate to process a single function over a network.
[00503] In addition, each of the steps illustrated in the flowcharts described above can be performed by a single device or by a plurality of devices in a shared manner.
[00504] Furthermore, if a single step includes a plurality of processes, the plurality of processes included in the single step can be performed by a single device or by a plurality of devices in a shared manner.
[00505] The image encoding device 10 (Figure 14) and the image decoding device 300 (Figure 22) according to the preceding embodiments can be applied to various pieces of electronic equipment such as a transmitter or receiver used to deliver data via satellite broadcasting, wired broadcasting such as cable TV, or the Internet or used to deliver data to or from terminals over cellular communication, a recording device that records images on media such as an optical disc, a magnetic disk, and a flash memory, and a playback device that reproduces images from such storage media. Four example applications will now be described.
10. Sample applications
First example application: Television receiver [00506] Figure 53 illustrates an example of a schematic configuration of a television set to which the embodiments
Petition 870170066495, of September 6, 2017, p. 115/294
Preceding 107/137 are applied. A television set 900 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, a display unit 906, an audio signal processing unit 907, a speaker 908, an external interface 909, a control unit 910, a user interface 911 and a bus 912.
[00507] Tuner 902 extracts a signal on a desired channel from a broadcast signal received by antenna 901, and demodulates the extracted signal. Then, tuner 902 produces an encoded bit stream obtained by demodulation to demultiplexer 903. In other words, tuner 902 functions as a transmission unit on television 900 to receive an encoded stream including encoded images.
[00508] Demultiplexer 903 demultiplexes the encoded bit stream into a video stream and an audio stream from a program to be viewed, and produces demultiplexed streams to decoder 904. Demultiplexer 903 additionally extracts auxiliary data such as EPG ( Electronic Program Guide) of the encoded bit stream, and provides the extracted data to the 910 control unit. Note that the demultiplexer 903 can also unscramble the encoded bit stream if the encoded bit stream has been scrambled.
[00509] The decoder 904 decodes the video stream and audio stream input from the demultiplexer 903. Then, the decoder 904 produces the video data generated by the decoding process to the video signal processing unit 905. The decoder 904 additionally produces data generated by the decoding process to the audio signal processing unit 907.
[00510] The video signal processing unit 905 reproduces the video data input from the decoder 904, and causes video to be displayed on the display unit 906. The video signal processing unit 905 can also make an application screen provided network is displayed on the
Petition 870170066495, of September 6, 2017, p. 116/294
108/137 display unit 906. The video signal processing unit 905 can additionally perform additional processing, such as noise removal, on the video data as per settings. In addition, the 905 video signal processing unit can also generate a GUI (Graphical User Interface) image such as a menu, a button, or a cursor, and overlay the generated image on an output image.
[00511] The display unit 906 is excited by an excitation signal provided by the video signal processing unit 905, and displays video or an image on a video surface of a display device (such as a liquid crystal display) , a plasma display, or an OELD (Organic Electroluminescence Display) (EL organic display).
[00512] The audio signal processing unit 907 performs reproduction processing, such as D / A conversion and amplification, on the audio data input from decoder 904, and causes audio to be produced from speaker 908. The processing unit of audio signal 907 can additionally perform processing, such as noise removal, on audio data.
[00513] External interface 909 is an interface for connecting the television set 900 to an external device or a network. For example, a video stream or audio stream received through the external interface 909 can be decoded by the decoder 904. In other words, the external interface 909 also functions as a transmission unit on the television set 900 to receive an encoded stream including images. coded.
[00514] The control unit 910 includes a processor such as a CPU, and memories such as a RAM and a ROM. The memories store a program to be executed by the CPU, program data, EPG data, data acquired by a network, and so on. The program stored in the memories is read and executed by the CPU when, for
Petition 870170066495, of September 6, 2017, p. 117/294
109/137 example, the television set 900 is started. The CPU executes the program to control the operation of the television set 900 according to, for example, an operation signal input from the user interface 911.
[00515] The 911 user interface is connected to the 910 control unit. For example, the 911 user interface includes buttons and keys to allow the user to operate the 900 television set, a receiver unit for a remote control signal, and so on. The 911 user interface detects user operation by the components described above to generate an operation signal, and produces the generated operation signal for the 910 control unit.
[00516] Bus 912 serves to connect tuner 902, demultiplexer 903, decoder 904, video signal processing unit 905, audio signal processing unit 907, external interface 909 and control unit 910 each other.
[00517] In the television set 900 having the configuration described above, the decoder 904 has the function of the image decoding device 300 (Figure 22) according to the preceding embodiments. Therefore, the television set 900 can suppress an increase in the amount of coding in a scale list.
Second example application: Mobile phone [00518] Figure 54 illustrates an example of a schematic configuration of a mobile phone to which the preceding embodiments are applied. A mobile phone 920 includes an antenna 921, a communication unit 922, an audio codec 923, a speaker 924, a microphone 925, a camera unit 926, an image processing unit 927, a multiplexing unit / demultiplexing 928, a recording / output unit 929, a display unit 930, a control unit 931, an operating unit 932 and a bus 933.
[00519] The antenna 921 is connected to the communication unit 922. The
Petition 870170066495, of September 6, 2017, p. 118/294
110/137 loudspeaker 924 and microphone 925 are connected to audio codec 923. Operator unit 932 is connected to control unit 931. Bus 933 serves to connect communication unit 922, audio codec 923, the camera unit 926, the image processing unit 927, the multiplexing / demultiplexing unit 928, the recording / output unit 929, the display unit 930 and the control unit 931 with each other.
[00520] The 920 mobile phone performs operations such as transmitting and receiving an audio signal, transmitting and receiving electronic mail or image data, capturing an image, and recording data, in various modes of operation including a call mode. voice, a data communication mode, an image capture mode and a videophone mode.
[00521] In voice call mode, an analog audio signal generated by microphone 925 is provided to audio codec 923. Audio codec 923 converts the analog audio signal into audio data, and performs A / D conversion and compression in the converted audio data. The audio codec 923 then produces the compressed audio data for the communication unit 922. The communication unit 922 encodes and modulates the audio data, and generates a transmission signal. The communication unit 922 then transmits the generated transmission signal to a base station (not shown) by antenna 921. Additionally, communication unit 922 amplifies a radio signal received by antenna 921, and performs frequency conversion on the amplified signal to acquire a receive signal. The communication unit 922 then demodulates and decodes the receive signal to generate audio data, and produces the generated audio data for the 923 audio codec. The 923 audio codec expands the audio data, and performs D / A to generate an analog audio signal. The 923 audio codec then provides the generated audio signal to the 924 speaker to make audio produced.
[00522] Furthermore, in data communication mode, for example,
Petition 870170066495, of September 6, 2017, p. 119/294
111/137 the control unit 931 generates text data that forms an electronic mail according to a user operation by the operator unit 932. In addition, the control unit 931 causes text to be displayed on the display unit 930. The control unit 931 additionally generates e-mail data according to a transmission instruction given by the user by operator unit 932, and produces the e-mail data generated for communication unit 922. Communication unit 922 encodes and modulates e-mail data for generate a transmission signal. The communication unit 922 then transmits the generated transmission signal to the base station (not shown) by antenna 921. Additionally, communication unit 922 amplifies a radio signal received by antenna 921, and performs frequency conversion on the amplified signal to acquire a receive signal. The communication unit 922 then demodulates and decodes the reception signal to restore e-mail data, and produces the restored e-mail data for the 931 control unit. The 931 control unit causes the e-mail content to be displayed on the display unit 930, and also causes email data to be stored on a storage medium of the 929 recording / playback unit.
[00523] The 929 recording / playback unit includes a desired readable / recordable storage medium. The storage medium can be, for example, an embedded storage medium such as a RAM or flash memory, or an external storage medium such as a hard disk, a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card.
[00524] Furthermore, in the image capture mode, for example, the camera unit 926 captures an image of an object to generate image data, and produces the image data generated for the image processing unit 927. A image processing unit 927 encodes the image data input from the camera unit 926, and makes a
Petition 870170066495, of September 6, 2017, p. 120/294
112/137 encoded stream be stored on the storage medium of the 929 recording / playback unit.
[00525] Furthermore, in videophone mode, for example, the multiplexing / demultiplexing unit 928 multiplexes the video stream encoded by the image processing unit 927 and the incoming audio stream from the audio codec 923, and produces a stream multiplexed for communication unit 922. Communication unit 922 encodes and modulates the flow to generate a transmission signal. Then, communication unit 922 transmits the generated transmission signal to the base station (not shown) by antenna 921. Communication unit 922 additionally amplifies a radio signal received by antenna 921, and performs frequency conversion on the amplified signal to acquire a reception signal. The transmit signal and the receive signal may include an encoded bit stream. The communication unit 922 demodulates and decodes the reception signal to restore a stream, and produces the restored stream to the 928 multiplex / demultiplex unit. Then the 928 multiplex / demultiplex unit demultiplexes the input stream into a video stream. and an audio stream, and produces the video stream and the audio stream for the image processing unit 927 and the audio codec 923, respectively. The image processing unit 927 decodes the video stream to generate video data. Video data is provided to the display unit 930, and a series of images is displayed by the display unit 930. The audio codec 923 expands the audio stream, and performs D / A conversion to generate an analog audio signal. The 923 audio codec then provides the generated audio signal to the 924 speaker to make audio produced.
[00526] In the mobile phone 920 having the configuration described above, the image processing unit 927 has the function of the image encoding device 10 (Figure 14) and the function of the image decoding device 300 (Figure 22) accordingly with ways of
Petition 870170066495, of September 6, 2017, p. 121/294
113/137 preceding achievements. Therefore, the mobile phone 920 can suppress an increase in the amount of coding in a scale list.
[00527] In addition, while a description has been given of the mobile phone 920, for example, an image encoding device and an image decoding device to which the present technology is applied can be used, similarly to the mobile phone 920, in any device having an image generation function and a communication function similar to that of the 920 mobile phone, such as a PDA (Personal Digital Assistant), a smartphone, a UMPC (Ultra-Mobile Personal Computer), a netbook, or a computer notebook staff. Third example application: Recording / reproduction apparatus [00528] Figure 55 illustrates an example of a schematic configuration of a recording / reproduction apparatus to which the preceding embodiments are applied. A 940 recording / playback device encodes, for example, audio data and video data from a received broadcast program, and records the encoded audio data and video data on a recording medium. In addition, the 940 recording / playback device can also encode audio data and video data acquired from, for example, another device, and record the encoded audio data and video data on a recording medium. In addition, the 940 recording / playback device reproduces, for example, data recorded on a recording medium using a monitor and a loudspeaker as instructed by a user. In this case, the recording / playback device 940 decodes audio data and video data.
[00529] The recording / playback apparatus 940 includes a tuner 941, an external interface 942, an encoder 943, an HDD (Hard Disk Drive) 944, a disk drive 945, a selector 946, a decoder 947, an OSD (Screen Display) 948, a 949 control unit and a 950 user interface.
[00530] Tuner 941 extracts a signal on a desired channel from
Petition 870170066495, of September 6, 2017, p. 122/294
114/137 a broadcast signal received by an antenna (not shown), and demodulates the extracted signal. Tuner 941 then produces an encoded bit stream obtained by demodulation to selector 946. In other words, tuner 941 functions as a transmission unit in the recording / playback device 940.
[00531] The external interface 942 is an interface for connecting the recording / playback device 940 to an external device or a network. The external interface 942 can be, for example, an IEEE 1394 interface, a network interface, a USB interface, a flash memory interface, or the like. For example, video data and audio data received by external interface 942 are input to encoder 943. In other words, external interface 942 functions as a transmission unit in the recording / playback device 940.
[00532] Encoder 943 encodes video data and audio data input from external interface 942 if the video data and audio data have not been encoded. Encoder 943 then produces an encoded bit stream for selector 946.
[00533] The HDD 944 records an encoded bit stream including compressed content data such as video and audio, various programs, and other data to an internal hard drive. In addition, the HDD 944 reads the data described above from the hard disk when playing video and audio.
[00534] Disk unit 945 writes and reads data to and from a recording medium placed on it. The recording medium placed in the 945 disc drive can be, for example, a DVD disc (such as DVD-Video, DVDRAM, DVD-R, DVD-RW, DVD + R, or DVD + RW) or a disc of Blu-ray (registered trademark).
[00535] Selector 946 selects an encoded bit stream entered from tuner 941 or encoder 943 when recording video and audio, and produces the selected encoded bit stream for HDD 944 or disk unit 945. When playing video and audio , selector 946 produces a bit stream
Petition 870170066495, of September 6, 2017, p. 123/294
115/137 encoded input from HDD 944 or disk unit 945 to decoder 947.
[00536] Decoder 947 decodes the encoded bit stream to generate video data and audio data. The decoder 947 then produces the generated video data for the OSD 948. The decoder 904 additionally produces the generated audio data for an external speaker. [00537] OSD 948 reproduces the video data input from decoder 947, and displays video. In addition, OSD 948 can also overlay a GUI image such as a menu, button, or cursor on the video to be displayed.
[00538] The control unit 949 includes a processor such as a CPU, and memories such as a RAM and a ROM. The memories store a program to be executed by the CPU, program data, and so on. The program stored in the memories is read and executed by the CPU when, for example, the recording / playback device 940 is started. The CPU executes the program to control the operation of the 940 recording / playback device according to, for example, an operation signal input from the 950 user interface.
[00539] User interface 950 is connected to control unit 949. User interface 950 includes, for example, buttons and keys to allow the user to operate the recording / playback device 940, a receiver unit for a control signal remote, and so on. The user interface 950 detects a user operation by the components described above to generate an operation signal, and produces the generated operation signal for the 949 control unit.
[00540] In the recording / reproduction apparatus 940 having the configuration described above, the encoder 943 has the function of the image encoding device 10 (Figure 14) according to the preceding embodiments. In addition, the decoder 947 has the function of the image decoding device 300 (Figure 22) according to the embodiments
Petition 870170066495, of September 6, 2017, p. 124/294
Previous 116/137. Therefore, the recording / playback apparatus 940 can suppress an increase in the amount of coding in a scale list. Fourth example application: Imaging apparatus [00541] Figure 56 illustrates an example of a schematic configuration of an imaging apparatus to which the preceding embodiments are applied. A 960 imaging device captures an image of an object to generate image data, encodes the image data, and records the encoded image data on a recording medium.
[00542] The imaging apparatus 960 includes an optical block 961, an imaging unit 962, a signal processing unit 963, an image processing unit 964, a display unit 965, an external interface 966 , a memory 967, a disk drive 968, an OSD 969, a control unit 970, a user interface 971 and a bus 972.
[00543] The optical block 961 is connected to the imaging unit 962. The imaging unit 962 is connected to the signal processing unit 963. The display unit 965 is connected to the image processing unit 964. The interface User 971 is connected to the control unit 970. The bus 972 serves to connect the image processing unit 964, the external interface 966, the memory 967, the disk unit 968, the OSD 969 and the control unit 970 between itself. [00544] Optical block 961 includes a focus lens, an opening mechanism, and so on. Optical block 961 forms an optical image of the object on an imaging surface of the imaging unit 962. The imaging unit 962 includes an image sensor such as a CCD or CMOS image sensor, and converts the optical image formed on the imaging surface in an image signal serving as an electrical signal performing photoelectric conversion. The 962 imaging unit then produces the
Petition 870170066495, of September 6, 2017, p. 125/294
117/137 image for signal processing unit 963.
[00545] The 963 signal processing unit performs various camera signal processing operations, such as knee correction, gamma correction, and color correction, on the image signal input from the 962 imaging unit. The unit signal processing 963 produces the image data subject to camera signal processing operations for the image processing unit 964.
[00546] The image processing unit 964 encodes the image data input from the signal processing unit 963 to generate encoded data. The image processing unit 964 then produces the encoded data generated for the external interface 966 or the media unit 968. In addition, the image processing unit 964 decodes the encoded data input from the external interface 966 or the media unit 968 to generate image data. The image processing unit 964 then produces the image data generated for the display unit 965. In addition, the image processing unit 964 can also produce the image data input from the signal processing unit 963 to the image processing unit. display 965 to make an image appear. In addition, the image processing unit 964 can also superimpose display data acquired from the OSD 969 on the image to be produced on the display unit 965.
[00547] OSD 969 generates a GUI image such as a menu, button, or cursor, and produces the image generated for the 964 image processing unit.
[00548] The external interface 966 is formed, for example, as a USB input / output terminal. For example, external interface 966 connects the imaging device 960 to a printer when printing an image. A disk unit is additionally connected to the external interface 966, if necessary. A removable medium such as a magnetic disk or an optical disk is placed in the disk drive, and a program
Petition 870170066495, of September 6, 2017, p. 126/294
118/137 read from the removable medium can be installed in the imaging device 960. In addition, the external interface 966 can also be formed as a network interface to be connected to a network such as a LAN or the Internet. In other words, the external interface 966 functions as a transmission unit in the imaging device 960.
[00549] The recording medium to be placed in the media unit 968 can be, for example, any removable readable / recordable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory. Alternatively, a recording medium can be fixedly attached to the media unit 968, and can form a built-in hard drive or a non-portable storage unit such as an SSD (Solid State Unit).
[00550] The control unit 970 includes a processor such as a CPU, and memories such as a RAM and a ROM. The memories store a program to be executed by the CPU, program data, and so on. The program stored in the memories is read and executed by the CPU when, for example, the imaging device 960 is started. The CPU executes the program to control the operation of the 960 imaging device according to, for example, an operation signal entered from the 971 user interface.
[00551] The 971 user interface is connected to the 970 control unit. For example, the 971 user interface includes buttons, switches, and so on to allow the user to operate the 960 imaging device. The 971 user interface detects a user operation by the components described above to generate an operation signal, and produces the generated operation signal for the 970 control unit.
[00552] In the imaging device 960 having the configuration described above, the image processing unit 964 has the function of the image encoding device 10 (Figure 14) and the function of the image decoding device 300 (Figure 22 ) according to the forms of
Petition 870170066495, of September 6, 2017, p. 127/294
119/137 preceding achievements. Thus, the imaging device 960 can suppress an increase in the amount of coding in a scale list.
7. Sample Applications for Gradable Coding
First system [00553] Below, a specific example of using scalable encoded data that was encoded using scalable encoding (layered encoding (image)) will be described. Gradable coding can be used, for example, for the selection of data to be transmitted, as in an example illustrated in Figure 57.
[00554] In a data transmission system 1000 illustrated in Figure 57, a distribution server 1002 reads scalable encoded data stored in a coded scalable data storage unit 1001, and distributes scalable encoded data to terminal devices, such as a personal computer 1004, an AV device 1005, a tablet device 1006 and a mobile phone 1007, over a network 1003.
[00555] In this case, the distribution server 1002 selects encrypted data having the desired quality according to the performance of the terminal device, the communication environment, and the like, and transmits the selected encrypted data. Even if the distribution server 1002 transmits data that is of higher quality than necessary, the terminal device may not always obtain a high quality image, and delay or overflow may be caused. In addition, such data may take up more than necessary communication bandwidth, or it may increase the load on the terminal device more than necessary. Conversely, even if the distribution server 1002 transmits data having a lower quality than necessary, the terminal device may not necessarily obtain an image of sufficient quality. Thus, the distribution server 1002 reads the scalable encoded data stored in the encoded scalable data storage unit 1001, if
Petition 870170066495, of September 6, 2017, p. 128/294
120/137 necessary, as encoded data having appropriate quality for the performance of the terminal device, communication environment, and the like, and transmits the read encoded data.
[00556] For example, it is assumed that the encoded scalable data storage unit 1001 stores scalable encoded data (BL + EL) 1011 that have been scaled encoded. Gradable encoded data (BL + EL) 1011 is encoded data including a base layer and an enhancement layer, and is data that is decoded to obtain both an image of the base layer and an image of the enhancement layer.
[00557] The distribution server 1002 selects an appropriate layer according to the performance of a terminal device that transmits data, the communication environment, and the like, and reads the data from the layer. For example, the distribution server 1002 reads high-quality scalable encoded data (BL + EL) 1011 from the encoded scalable data storage unit 1001, and transmits the readable scalable encoded data (BL + EL) 1011 to personal computer 1004 or the tablet device 1006, which has high processing capabilities, as they are. In contrast, for example, distribution server 1002 extracts data from the base layer from scalable encoded data (BL + EL) 1011, and transmits the data from the base layer to the AV device 1005 and the mobile phone 1007, which has low capacities processing, such as scalable encoded data (BL) 1012 having the same content as scalable encoded data (BL + EL) 1011, but having lower quality than encoded scalable data (BL + EL) 1011.
[00558] The use of encoded data gradable in this way facilitates the adjustment of the amount of data, thereby suppressing the occurrence of delay or overflow and suppressing an unnecessary increase in load on a terminal device or a means of communication. In addition, scalable encoded data (BL + EL) 1011 reduces redundancy between
Petition 870170066495, of September 6, 2017, p. 129/294
121/137 layers, and therefore have a smaller amount of data than data having individually encoded data from the respective layers. Therefore, the storage area of the encodable scalable data storage unit 1001 can be used more effectively.
[00559] Note that since various devices such as personal computer 1004, AV device 1005, tablet device 1006, and mobile phone 1007 can be used as terminal devices, the hardware performance of terminal devices differs from device for device. In addition, since multiple applications can be run by terminal devices, the software capabilities of the applications may vary. In addition, network 1003 serving as a means of communication can be implemented like any communication line network that can be wired, wireless, or both, such as the Internet and a LAN (Local Area Network), and have various capabilities data transmission. Such performance and capabilities may vary depending on other and similar communications.
[00560] Therefore, before the start of data transmission, the distribution server 1002 can communicate with a terminal device to which the data is to be transmitted, and can obtain information regarding the capabilities of the terminal device, such as performance hardware of the terminal device or the application performance (software) executed by the terminal device, as well as information related to the communication environment, such as the available bandwidth of the 1003 network. In addition, the distribution server 1002 can select a layer appropriate on the basis of the information obtained.
[00561] Note that a layer can be extracted by a terminal device. For example, personal computer 1004 can decode transmitted scalable encoded data (BL + EL) 1011, and display an image of a base layer or an image of an enhancement layer. Alternatively, for example, the personal computer 1004 can extract
Petition 870170066495, of September 6, 2017, p. 130/294
122/137 scalable encoded data (BL) 1012 of the base layer of transmitted scalable encoded data (BL + EL) 1011, store the extracted scalable encoded data (BL) 1012, transfer the extracted scalable encoded data (BL) 1012 to another device, or decode the extracted scalable encoded data (BL) 1012 to display an image of the base layer.
[00562] Needless to say, the number of encodable scalable data storage units 1001, the number of distribution servers 1002, the number of networks 1003, and the number of terminal devices are arbitrary. In addition, while a description has been given of an example in which the distribution server 1002 transmits data to a terminal device, usage examples are not limited to this example. The data transmission system 1000 can be used in any system that selects an appropriate layer, when transmitting encoded data that has been encoded using scalable encoding to a terminal device, depending on the capabilities of the terminal device, the communication environment, and the like.
[00563] In addition, the present technology can also be applied to the data transmission system 1000 as illustrated in Figure 57 described above in a manner similar to an application for hierarchical encoding and hierarchical decoding described above with reference to Figures 49 to 51 , thereby achieving advantages similar to those described above with reference to Figures 49 to 51.
According to the [00564] scalable encoding system, it can also be used, for example, as in an example illustrated in Figure 58, transmission through a plurality of communication media.
[00565] In a data transmission system 1100 illustrated in Figure 58, a broadcasting station 1101 transmits scalable encoded (BL) data 1121 from a base layer by terrestrial broadcasting 1111. A
Petition 870170066495, of September 6, 2017, p. 131/294
123/137 broadcasting station 1101 additionally transmits (for example, packages and transmits) scalable encoded data (EL) 1122 from an enhancement layer over a desired network 1112 formed from a communication network that can be wired, wireless, or both.
[00566] A terminal device 1102 has a function to receive terrestrial broadcasting 1111 from broadcasting station 1101, and receives scalable encoded data (BL) 1121 from the base layer transmitted by terrestrial broadcasting 1111. Terminal device 1102 additionally has a function of communication to perform communication over the 1112 network, and receives scalable encoded (EL) data 1122 from the enhancement layer transmitted over the 1112 network.
[00567] Terminal device 1102 decodes the scalable encoded data (BL) 1121 of the base layer acquired by terrestrial broadcasting 1111 according to, for example, a user instruction or similar to obtain an image of the base layer, stores the encodable data scalable (BL ) 1121, or transfers the scalable encoded data (BL) 1121 to another device.
[00568] Furthermore, the terminal device 1102 combines the scalable encoded data (BL) 1121 of the base layer acquired by terrestrial broadcasting 1111 with the scalable encoded data (EL) 1122 of the enhancement layer acquired by the network 1112 according to, for example, a user instruction or similar to obtain scalable encoded data (BL + EL), and decode the scalable encoded data (BL + EL) to obtain an image of the enhancement layer, store the scalable encoded data (BL + EL), or transfer the scalable encoded data (BL + EL) to another device.
[00569] As described above, scalable encoded data can be transmitted, for example, by different means of communication from one layer to another. Thus, the load can be distributed, and delay or overflow can be suppressed from occurring.
Petition 870170066495, of September 6, 2017, p. 132/294
124/137 [00570] In addition, a communication medium to be used for transmission can be selectable for each layer depending on the situation. For example, scalable encoded (BL) data 1121 of the base layer having a relatively large amount of data can be transmitted over a communication medium having a large bandwidth, and scalable encoded (EL) data 1122 of the enhancement layer having a relatively small amount of data can be transmitted over a communication medium having a narrow bandwidth. Alternatively, for example, the communication medium by which the scalable encoded data (EL) 1122 of the enhancement layer is to be transmitted can be exchanged between the 1112 network and the terrestrial broadcast 1111 according to the available bandwidth of the 1112 network. as usual, the former applies similarly to data from an arbitrary layer.
[00571] Control in the manner described above can additionally suppress an increase in the data transmission load.
[00572] Needless to say, the number of layers is arbitrary, and the number of media to be used for transmission is also arbitrary. In addition, the number of terminal devices 1102 to which data is to be distributed is also arbitrary. In addition, while a description has been given in the broadcasting context of broadcasting station 1101 by way of example, usage examples are not limited to this example. The data transmission system 1100 can be used in any system that divides encoded data using scalable encoding into a plurality of segments in units of layers and transmits the data segments over a plurality of lines.
[00573] In addition, the present technology can also be applied to the 1100 data transmission system as illustrated in Figure 58 described above in a manner similar to an application for hierarchical encoding and hierarchical decoding described above with reference to Figures 49 to 51 , thereby achieving advantages similar to those described above
Petition 870170066495, of September 6, 2017, p. 133/294
125/137 with reference to Figures 49 to 51.
Third system [00574] Scalable encoding can also be used, for example, as in an example illustrated in Figure 59, for the storage of encoded data.
[00575] In an imaging system 1200 illustrated in Figure 59, an imaging device 1201 performs scalable encoding on image data obtained by capturing an image of an object 1211, and provides the resulting data to a storage device of scalable coded data 1202 as scalable coded data (BL + EL) 1221.
[00576] The scalable encoded data storage device 1202 stores the scalable encoded data (BL + EL) 1221 provided with the image generation device 1201 to the quality corresponding to the situation. For example, at normal time, the encoded scalable data storage device 1202 extracts data from a base layer from the encoded scalable data (BL + EL) 1221, and stores the data extracted from the base layer as scalable encoded data (BL) 1222 from base layer having a low quality and a small amount of data. In contrast, for example, the contrast encoded scalable data storage device 1202 stores the scalable encoded data (BL + EL) 1221 having a high quality and a large amount of data, as is.
[00577] Therefore, the encodable scalable data storage device 1202 can save an image at high quality only when necessary. This can suppress an increase in the amount of data while suppressing a reduction in the value of the image due to a decrease in quality, and can improve the efficiency of use of the storage area. [00578] For example, it is assumed that the 1201 imaging device is a security camera. If an object to be monitored (for example,
Petition 870170066495, of September 6, 2017, p. 134/294
126/137 example, an intruder) does not appear in a captured image (normal time), it may be likely that the captured image has no important content. Thus, a reduction in the amount of data is prioritized, and the image data (scalable encoded data) of the image is stored at low quality. In contrast, if an object to be monitored appears as the 1211 object in a captured image (attention span), the captured image may be likely to have important content. Thus, image quality is prioritized, and the image data (scalable encoded data) of the image is stored at high quality.
[00579] Note that either the normal time or the attention time can be determined, for example, by the coded scalable data storage device 1202 analyzing an image. Alternatively, the imaging device 1201 can determine the normal time or attention span, and can transmit the determination result to the encoded scalable data storage device 1202.
[00580] Note that the determination of either normal time or attention span can be based on an arbitrary model, and an image on which the determination is based can have any content. Needless to say, conditions other than the content of an image can be used as the model of determination. The state can be changed according to, for example, the magnitude, waveform, or similar recorded audio, or it can be changed at intervals of a predetermined period of time. Alternatively, the state can be changed according to an external instruction such as a user instruction.
[00581] Furthermore, while a description has been given of an example of changing between two states, that is, normal time and attention span, the number of states is arbitrary, and the change of state can be made between more than two states , such as normal time, attention time, more attention time, and much more attention time. Note that the number of upper limit states to be changed depends on the number of data layers
Petition 870170066495, of September 6, 2017, p. 135/294
127/137 codable gradable.
[00582] In addition, the imaging device 1201 can be configured to determine the number of layers of scalable coding according to the state. For example, at normal time, the imaging device 1201 can generate scalable encoded data (BL) 1222 of the base layer having a low quality and small amount of data, and provide the generated scalable encoded data (BL) 1222 for the encodable scalable data storage device 1202. In addition, for example, at attention time, the imaging device 1201 can generate scalable encoded data (BL + EL) 1221 of the base layer having a high quality and a large amount of data, and provide the generated scalable encoded data (BL + EL) 1221 to the encoded scalable data storage device 1202.
[00583] While a security camera has been described as an example, the 1200 imaging system can be used in any application, and can be used in applications other than a security camera.
[00584] In addition, the present technology can also be applied to the image generation system 1200 illustrated in Figure 59 described above in a manner similar to an application for hierarchical encoding and hierarchical decoding described above with reference to Figures 49 to 51, hereby achieving advantages similar to those described above with reference to Figures 49 to 51.
[00585] Note that the present technology can also be applied to HTTP streaming, such as MPEG DASH, in which an appropriate piece of encoded data is selected and used in units of a segment among a plurality of pieces of encoded data prepared in advance and having different resolutions. In other words, information related to encoding and decoding can also be shared between a plurality of pieces of encoded data.
Petition 870170066495, of September 6, 2017, p. 136/294
128/137 [00586] Needless to say, an image encoding device and an image decoding device to which the present technology is applied can also be applied to devices other than the devices described above or to systems.
[00587] Note that an example has been described here in which a quantization matrix (or a coefficient used to form a quantization matrix) is transmitted from the encoding side to the decoding side. One technique for transmitting a quantization matrix may be to transmit or record the quantization matrix as separate data associated with an encoded bit stream without multiplexing the quantization parameter into the encoded bit stream. The term associate, as used here, means to allow an image (which can be part of an image, such as a slice or block) included in a bit stream to be linked to the information corresponding to the image when the image is decoded. That is, the information can be transmitted in a different transmission path than the image (or bit stream). In addition, information can be recorded on a recording medium other than that for the image (or bit stream) (or recorded on a recording area other than the same recording medium). In addition, the information and the image (or bit stream) can be associated with each other in arbitrary units such as a plurality of frames, a frame, or a portion in a frame.
[00588] Note that the present technology can also provide the following configurations.
(1) An image processing device including:
an establishment unit configured to establish a coefficient located at the beginning of a quantization matrix whose size is not limited to greater than a transmission size which is a maximum size allowed in transmission, adding a substitution difference coefficient which is a difference between a substitution coefficient and the coefficient located at the beginning of the quantization matrix
Petition 870170066495, of September 6, 2017, p. 137/294
129/137 for the coefficient located at the beginning of the quantization matrix, the substitution coefficient being used to replace a coefficient located at the beginning of a quantization matrix converted by upward conversion which is obtained by converting upwards the quantization matrix to the same size as a block size that is a processing unit on which decanting is performed;
an upward conversion unit configured to upward convert the quantization matrix established by the establishment unit to establish the upward conversion converted quantization matrix; and a decanting unit configured to decantify quantized data obtained by decoding encoded data, use a quantization matrix converted by upward conversion in which a coefficient located at the beginning of the quantization matrix converted by upward conversion established by the upward conversion unit has been replaced with the coefficient of substitution.
(2) The image processing device according to any of (1) and (3) to (9), in which:
the establishment unit establishes the substitution coefficient by adding a difference between the substitution coefficient and an initial value established for the quantization matrix to the initial value.
(3) The image processing device according to any of (1), (2), and (4) to (9), in which:
the establishment unit establishes coefficients of the quantization matrix using the substitution difference coefficient and difference coefficients that are differences between the coefficients of the quantization matrix.
(4) The image processing device according to any of (1) to (3) and (5) to (9), where:
the substitution difference coefficient and the coefficients of
Petition 870170066495, of September 6, 2017, p. 138/294
130/137 difference which are the differences between the coefficients of the quantization matrix are transmitted collectively, and the establishment unit establishes the coefficients of the quantization matrix using the substitution difference coefficient and difference coefficients transmitted collectively.
(5) The image processing device according to any of (1) to (4) and (6) to (9), where:
the substitution difference coefficient and the difference coefficients which are the differences between the coefficients of the quantization matrix have been coded, and the establishment unit decodes the coded substitution difference coefficient and the coded difference coefficients.
(6) The image processing device according to any of (1) to (5) and (7) to (9), in which:
the reverse conversion unit converts the quantization matrix whose size is limited to no larger than the transmission size by converting the closest neighbor interpolation process into matrix elements of the quantization matrix.
(7) The image processing device according to any of (1) to (6), (8), and (9), in which:
the transmission size is 8x8, and the reverse conversion unit converts a quantization matrix having a size 8x8 to a quantization matrix having a size 16x16, by converting the nearest neighbor interpolation process into matrix elements of the matrix of quantization having the size 8x8.
(8) The image processing device according to any of (1) to (7) and (9), in which:
upward conversion unit converts by conversion
Petition 870170066495, of September 6, 2017, p. 139/294
131/137 ascending a quantization matrix having an 8x8 size to a quantization matrix having a 32x32 size, performing the nearest neighbor interpolation process on matrix elements of the quantization matrix having the size 8x8.
(9) The image processing device according to any of (1) to (8), where:
a coding unit which is a processing unit on which a decoding process is performed and a transform unit which is a processing unit on which a transform process is performed have a layered structure, the image processing device additionally includes a decoding unit configured to perform a decoding process on the encoded data using a unit having a layered structure to generate the quantized data, and the reverse conversion unit converts the transmission size quantization matrix to a size by conversion. of a transform unit that is a processing unit on which decanting is performed.
(10) An image processing method including:
establish a coefficient located at the beginning of a quantization matrix whose size is limited to no greater than a transmission size which is a maximum size allowed in transmission, adding a substitution difference coefficient which is a difference between a substitution coefficient and the coefficient located at the beginning of the quantization matrix for the coefficient located at the beginning of the quantization matrix, the substitution coefficient being used to replace a coefficient located at the beginning of a quantization matrix for upward conversion that is obtained by converting by upward conversion to the matrix quantization to the same size as a block size which is a processing unit in which
Petition 870170066495, of September 6, 2017, p. 140/294
132/137 executed;
convert by quantitative conversion the established quantization matrix to establish the quantization matrix converted by ascending conversion; and decanting quantized data obtained by decoding encoded data, using an upward converting quantization matrix in which a coefficient located at the beginning of the established upward converting quantization matrix has been replaced with the substitution coefficient.
(11) An image processing device including:
an establishment unit configured to establish a substitution difference coefficient which is a difference between a substitution coefficient and a coefficient located at the beginning of a quantization matrix whose size is limited to no greater than a transmission size which is a size maximum allowed in transmission, the substitution coefficient being used to replace a coefficient located at the beginning of a quantization matrix converted by upward conversion which is obtained by converting upward quantization matrix to the same size as a block size that is a unit processing on which decanting is performed;
a quantization unit configured to quantize an image to generate quantized data; and a transmission unit configured to transmit encoded data obtained by encoding the quantized data generated by the quantization unit, substitution coefficient data obtained encoding the substitution coefficient, and substitution difference coefficient data obtained encoding the established substitution difference coefficient by the establishment unit.
(12) The image processing device according to any of (11) and (13) to (17), in which:
Petition 870170066495, of September 6, 2017, p. 141/294
133/137 the establishment unit establishes a difference between the substitution coefficient and an initial value established for the quantization matrix.
(13) The image processing device according to any of (11), (12), and (14) to (17), in which:
the unit of establishment establishes difference coefficients that are differences between coefficients of the quantization matrix, and the transmission unit transmits data of difference coefficient obtained by encoding the difference coefficients established by the unit of establishment.
(14) The image processing device according to any of (11) to (13) and (15) to (17), in which:
the transmission unit collectively transmits the replacement coefficient data and the replacement difference coefficient data.
(15) The image processing device according to any of (11) to (14), (16), and (17), in which:
the transmission unit transmits the replacement coefficient data and the replacement difference coefficient data in order of the replacement coefficient data and the replacement difference coefficient data.
(16) The image processing device according to any of (11) to (15) and (17), in which:
the quantization unit quantizes the image using the quantization matrix or the quantization matrix converted by upward conversion.
(17) The image processing device according to any of (11) to (16), where:
a coding unit which is a processing unit on which a coding process is performed and a
Petition 870170066495, of September 6, 2017, p. 142/294
134/137 transform unit which is a processing unit in which a transform process is performed has a layered structure, and the image processing device additionally includes an encoding unit configured to encode the quantized data generated by the quantization unit .
(18) An image processing method including:
establish a substitution difference coefficient that is a difference between a substitution coefficient and a coefficient located at the beginning of a quantization matrix whose size is limited to no greater than a transmission size which is a maximum size allowed in transmission, the substitution coefficient being used to replace a coefficient located at the beginning of a quantization matrix converted by upward conversion which is obtained by converting upward quantization matrix to the same size as a block size which is a processing unit in which decanting is performed;
quantize an image to generate quantized data; and transmitting encoded data obtained by encoding the generated quantized data, substitution coefficient data obtained by encoding the substitution coefficient, and substitution difference coefficient data obtained by encoding the established substitution difference coefficient.
(19) An image processing device including:
a decoding unit configured to decode encoded data to generate quantized data; and a decanting unit configured to decant the quantized data generated by the decoding unit, using a standard quantization matrix having the same size as a block size which is a processing unit on which decanting is performed, when in a copy mode in which a quantization matrix is copied,
Petition 870170066495, of September 6, 2017, p. 143/294
135/137 quantization matrix reference data identifying a quantization matrix reference destination matches quantization matrix identification data identifying the quantization matrix.
(20) The image processing device according to any of (19) and (21), in which:
the decanting unit disquantifies the quantized data by analyzing the syntax whose semantics are established so that the standard quantization matrix is referred to when the quantization matrix reference data matches the quantization matrix identification data.
(21) The image processing device according to any of (19) and (20), in which:
the decanting unit disquantifies the quantized data by analyzing the syntax whose semantics are established so that the standard quantization matrix is referred to when a difference between the quantization matrix reference data and the quantization matrix identification data is equal to 0 .
(22) An image processing method including:
decode encoded data to generate quantized data; and decanting the quantized data generated in decoding, using a standard quantization matrix having the same size as a block size which is a processing unit on which decanting is performed, when in a copy mode in which a quantization matrix is copied , quantization matrix reference data identifying a quantization matrix reference destination match quantization matrix identification data identifying the quantization matrix.
(23) An image processing device including:
an encoding unit configured to encode an image to generate encoded data; and an establishment unit configured to establish,
Petition 870170066495, of September 6, 2017, p. 144/294
136/137 as the syntax of the coded data generated by the coding unit, a syntax whose semantics are established so that a standard quantization matrix having the same size as a block size which is a processing unit in which quantization is performed is referred to when in a copy mode in which a quantization matrix is copied, quantization matrix reference data identifying a quantization matrix reference destination matches quantization matrix identification data identifying the quantization matrix.
(24) An image processing method including:
encode an image to generate encoded data; and establish, as a syntax of the generated coded data, a syntax whose semantics are established so that a standard quantization matrix having the same size as a block size which is a processing unit on which quantization is performed is referred to when in a copy in which a quantization matrix is copied, quantization matrix reference data identifying a quantization matrix reference destination match quantization matrix identification data identifying the quantization matrix.
List of Reference Signs
- image encoding device, 14 - orthogonal transform / quantization unit, 16 - lossless encoding unit, 150 - matrix processing unit, 192 - DPCM unit, 211 - DC coefficient encoding unit, 212 - AC coefficient DPCM unit, 300 - image decoding device, 312 - lossless decoding unit, 313 - reverse orthogonal decanting / transforming unit, 410 - matrix generating unit, 552 - reverse DPCM unit, 571 - initial establishment unit, 572 - DPCM decoding unit, 573 - DC coefficient extraction unit, 611 - AC coefficient buffer, 612 - AC coefficient encoding unit, 613 - DPCM coefficient unit in
Petition 870170066495, of September 6, 2017, p. 145/294
137/137
CA, 614 - DC coefficient DPCM unit, 621 - initial settlement unit, 622 - AC coefficient DPCM decoding unit, 623 - AC coefficient buffer, 624 DC coefficient DPCM decoding unit , 631 - AC coefficient DPCM unit, 632 - DC coefficient buffer, 633 - DC coefficient DPCM unit, 641 - initial establishment unit, 642 - AC coefficient DPCM decoding unit, 643 - DPCM decoding unit with CC coefficient.
Petition 870170066495, of September 6, 2017, p. 146/294 / 7

权利要求:
Claims (19)
[1]
1. Image processing device, characterized by the fact that it comprises:
a decoding unit (312) configured to decode encoded data including a substitution difference coefficient which is a difference between a substitution coefficient being used to replace a coefficient located at the beginning of a 16x16 quantization matrix that is obtained by upward conversion of an 8x8 quantization matrix according to a nearest neighbor interpolation process in the 8x8 quantization matrix and a coefficient located at the beginning of the 8x8 quantization matrix to generate quantized data;
an establishment unit (410) configured to establish a 16x16 quantization matrix by performing a nearest neighbor interpolation process in the 8x8 quantization matrix in which a coefficient obtained by adding the substitution coefficient to the substitution difference coefficient is established as the coefficient of substitution located at the beginning of an 8x8 quantization matrix;
a replacement unit (553) configured to replace the coefficient located at the beginning of the 16x16 quantization matrix with the replacement coefficient; and a decanting unit (313, 440) configured to decantize quantized data generated by the decoding unit, using the 16x16 quantization matrix in which the coefficient located at the beginning of the 16x16 quantization matrix has been replaced with the substitution coefficient by the replacement unit , and generates transform coefficient data.
[2]
2. Image processing device, according to claim 1, characterized by the fact that:
the decoding unit (312) is configured to
Petition 870180056525, dated 06/29/2018, p. 18/24
2/7 decode the encoded data including difference coefficients that are differences between coefficients of the 8x8 quantization matrix; and the establishment unit (410) is configured to establish the 8x8 quantization matrix by performing an addition process on the difference coefficients.
[3]
3. Image processing device, according to claim 2, characterized by the fact that:
the decoding unit (312) is configured to decode the encoded data including as a syntax in which the substitution difference coefficient and the difference coefficient are correctly included as a difference coefficient group.
[4]
4. Image processing device, according to claim 3, characterized by the fact that:
the substitution difference coefficient and difference coefficients are included as the syntax of the data encoded in the order of the substitution difference coefficient and the difference coefficients; and the decoding unit (312, 533) is configured to decode the substitution difference coefficient and the difference coefficients in the order of the substitution difference coefficient and the difference coefficients.
[5]
5. Image processing device, according to claim 4, characterized by the fact that:
the decoding unit (312) is configured to decode the encoded data including an initial difference value between the substitution coefficient and an initial value defined for a quantization matrix; and the establishment unit (410) is configured to establish the substitution coefficient by adding the initial value to the initial difference value.
Petition 870180056525, dated 06/29/2018, p. 19/24
3/7
[6]
6. Image processing device, according to claim 5, characterized by the fact that:
the initial difference value and the difference coefficient group are included as the syntax of the data encoded in the order of the initial difference value and the difference coefficient group; and the decoding unit (312, 533) is configured to decode the initial difference value and the difference coefficient group in the order of the initial difference value and the difference coefficient group.
[7]
7. Image processing device, according to claim 6, characterized by the fact that:
the initial difference value and the difference coefficient group are, through the execution of an exponential Golomb encoding process, included in the encoded data; and the decoding unit (533) is configured to perform an exponential Golomb decoding process at the initial difference value and the difference coefficient group obtained by executing the exponential Golomb encoding process in the order of the initial difference value and the difference coefficient group.
[8]
8. Image processing device, according to claim 7, characterized by the fact that it also comprises:
an inverse transform unit (313, 450) configured to inversely transform the transform coefficient data generated by the decanting unit.
[9]
9. Image processing device, according to claim 5, characterized by the fact that:
the reverse transform unit (313, 450) is configured to transform inversely according to the 16x16 transform unit.
[10]
10. Image processing method, characterized by the
Petition 870180056525, dated 06/29/2018, p. 20/24
4/7 fact to understand:
decode encoded data including a substitution difference coefficient which is a difference between a substitution coefficient being used to replace a coefficient located at the beginning of a 16x16 quantization matrix that is obtained by upward conversion of an 8x8 quantization matrix according to a interpolation process of nearest neighbor in the 8x8 quantization matrix and a coefficient located at the beginning of the 8x8 quantization matrix to generate quantized data;
establish a 16x16 quantization matrix by executing a nearest neighbor interpolation process in the 8x8 quantization matrix in which a coefficient obtained by adding the substitution coefficient to the substitution difference coefficient is established as the coefficient located at the beginning of an 8x8 quantization matrix ;
replace the coefficient located at the beginning of the 16x16 quantization matrix with the substitution coefficient; and decanting quantized data generated by decoding, using the 16x16 quantization matrix in which the coefficient located at the beginning of the 16x16 quantization matrix was replaced with the substitution substitution coefficient, and generating transform coefficient data.
[11]
11. Image processing method, according to claim 10, characterized by the fact that it comprises:
decode the encoded data including difference coefficients which are differences between coefficients of the 8x8 quantization matrix; and establish an 8x8 quantization matrix by performing an addition process on the difference coefficients.
[12]
12. Image processing method, according to claim 11, characterized by the fact that it comprises:
Petition 870180056525, dated 06/29/2018, p. 21/24
5/7 decode the encoded data including as a syntax where the substitution difference coefficient and difference coefficients are correctly included as a difference coefficient group.
[13]
13. Image processing method, according to claim 12, characterized by the fact that it comprises:
decode the substitution difference coefficient and difference coefficients in the order of the substitution difference coefficient and difference coefficients; and where the substitution difference coefficient and difference coefficients are included as the syntax of the data encoded in the order of the substitution difference coefficient and the difference coefficients.
[14]
14. Image processing method, according to claim 13, characterized by the fact that it comprises:
decode the encoded data including an initial difference value between the substitution coefficient and an initial value defined for a quantization matrix; and establish the substitution coefficient by adding the initial value to the initial difference value.
[15]
15. Image processing method, according to claim 14, characterized by the fact that it comprises:
decode the initial difference value and the difference coefficient group in the order of the initial difference value and the difference coefficient group; and where the initial difference value and the difference coefficient group are included as the syntax of the data encoded in the order of the initial difference value and the difference coefficient group.
[16]
16. Image processing method, according to claim 15, characterized by the fact that it comprises:
perform a Golomb decoding process
Petition 870180056525, of 06/29/2018, p. 22/24
6/7 exponential in the initial difference value and the difference coefficient group obtained by executing the exponential Golomb coding process in the order of the initial difference value and the difference coefficient group; and where the initial difference value and the difference coefficient group are, through the execution of an exponential Golomb encoding process, included in the encoded data.
[17]
17. Image processing method, according to claim 16, characterized by the fact that it comprises:
inversely transform the transform coefficient data generated by the decanting unit.
[18]
18. Image processing method, according to claim 17, characterized by the fact that it comprises:
transform inversely according to the 16x16 transform unit.
[19]
19. Computer-readable storage media for implementing the method as defined in claim 10 characterized by understanding data that, when executed on a computer, cause the computer to implement the steps of:
decode encoded data including a substitution difference coefficient which is a difference between a substitution coefficient being used to replace a coefficient located at the beginning of a 16x16 quantization matrix that is obtained by upward conversion of an 8x8 quantization matrix according to a interpolation process of nearest neighbor in the 8x8 quantization matrix and a coefficient located at the beginning of the 8x8 quantization matrix to generate quantized data;
establish a 16x16 quantization matrix by executing a nearest neighbor interpolation process in the quantization matrix
Petition 870180056525, dated 06/29/2018, p. 23/24
7/7
8x8 in which a coefficient obtained by adding the substitution coefficient to the substitution difference coefficient is established as the coefficient located at the beginning of an 8x8 quantization matrix;
replace the coefficient located at the beginning of the 16x16 quantization matrix with the substitution coefficient; and decanting quantized data generated by the decoding unit, using the 16x16 quantization matrix in which the coefficient located at the beginning of the 16x16 quantization matrix has been replaced with the substitution coefficient by the replacement unit, and generates transform coefficient data.
Petition 870180056525, of 06/29/2018, p. 24/24
1/58

类似技术:

---

公开号 | 公开日 | 专利标题

---

JP6635184B2|2020-01-22|Image processing apparatus and method

---

AU2020205233B2|2021-05-27|Image processing device and method

---

BR122015004025B1|2018-07-17|image processing device and method

同族专利:

---

公开号 | 公开日

---

WO2013129617A1|2013-09-06|

---

DK3282708T3|2019-05-20|

---

KR102153271B1|2020-09-07|

---

DK3432584T3|2020-06-15|

---

CN104126306B|2017-09-19|

---

US10349053B2|2019-07-09|

---

KR20140143138A|2014-12-15|

---

US20140369407A1|2014-12-18|

---

KR102154967B1|2020-09-10|

---

RU2597509C2|2016-09-10|

---

CA3041408A1|2013-09-06|

---

CA2862282C|2021-03-30|

---

AR112588A2|2019-11-13|

---

JP6465226B2|2019-02-06|

---

KR102090344B1|2020-03-17|

---

HK1243572A1|2018-07-13|

---

RU2016120989A3|2019-10-23|

---

JP6075485B2|2017-02-08|

---

US10404985B2|2019-09-03|

---

US20200045314A1|2020-02-06|

---

CN104126304B|2017-11-21|

---

US9955164B2|2018-04-24|

---

KR102074601B1|2020-02-06|

---

PL3300370T3|2019-07-31|

---

KR102074600B1|2020-02-06|

---

KR20200008018A|2020-01-22|

---

JP2018110430A|2018-07-12|

---

MX352223B|2017-11-15|

---

MX2014010113A|2014-10-14|

---

CN110913224A|2020-03-24|

---

US10574990B2|2020-02-25|

---

CA2862282A1|2013-09-06|

---

EP3432584A1|2019-01-23|

---

CN107592536A|2018-01-16|

---

TW201519638A|2015-05-16|

---

JP6465227B2|2019-02-06|

---

US20150010075A1|2015-01-08|

---

WO2013129618A1|2013-09-06|

---

EP3282708B1|2019-04-03|

---

JP2016105658A|2016-06-09|

---

KR20180017209A|2018-02-20|

---

KR102074602B1|2020-02-06|

---

CN107257491B|2019-10-18|

---

US9648316B2|2017-05-09|

---

CN104137552A|2014-11-05|

---

JPWO2013129616A1|2015-07-30|

---

US20180176566A1|2018-06-21|

---

EP3282708A1|2018-02-14|

---

AR090124A1|2014-10-22|

---

US20180103254A1|2018-04-12|

---

ES2725674T3|2019-09-26|

---

KR20200010571A|2020-01-30|

---

EP3432584B1|2020-05-13|

---

CN107257491A|2017-10-17|

---

EP3930334A4|2021-12-29|

---

KR102248057B1|2021-05-04|

---

US10225553B2|2019-03-05|

---

RU2014134533A|2016-03-20|

---

US20170142413A1|2017-05-18|

---

EP3709652B1|2021-09-22|

---

RU2582057C2|2016-04-20|

---

DK3300370T3|2019-05-20|

---

EP3432583A1|2019-01-23|

---

EP3300370B1|2019-04-03|

---

CN104126306A|2014-10-29|

---

CN109803149B|2021-10-01|

---

TWI612798B|2018-01-21|

---

JP2020036372A|2020-03-05|

---

US9813714B2|2017-11-07|

---

CN109803149A|2019-05-24|

---

HUE049148T2|2020-09-28|

---

EP3930334A1|2021-12-29|

---

JP6299890B2|2018-03-28|

---

WO2013129203A1|2013-09-06|

---

US20150023412A1|2015-01-22|

---

US20180098071A1|2018-04-05|

---

JP5904268B2|2016-04-13|

---

HUE043260T2|2019-08-28|

---

EP3432582B1|2020-04-01|

---

CN104126304A|2014-10-29|

---

US9838682B2|2017-12-05|

---

CN109379589B|2021-06-15|

---

CN109379589A|2019-02-22|

---

KR20200103875A|2020-09-02|

---

CN107592536B|2020-02-11|

---

AU2013227608B2|2016-07-21|

---

JP6075487B2|2017-02-08|

---

JP2016131383A|2016-07-21|

---

KR102154968B1|2020-09-10|

---

CN107277516B|2020-02-18|

---

CN104137552B|2019-01-11|

---

KR102154965B1|2020-09-10|

---

JP5904267B2|2016-04-13|

---

TW201715885A|2017-05-01|

---

TWI612797B|2018-01-21|

---

JP6075486B2|2017-02-08|

---

US10264258B2|2019-04-16|

---

JP2017085640A|2017-05-18|

---

US9686541B2|2017-06-20|

---

RU2016120989A|2017-11-28|

---

EP3432583B1|2020-04-01|

---

EP2822275A1|2015-01-07|

---

KR102154964B1|2020-09-10|

---

KR102153269B1|2020-09-07|

---

AR112641A2|2019-11-20|

---

JP2016131382A|2016-07-21|

---

ZA201405569B|2014-11-26|

---

JPWO2013129203A1|2015-07-30|

---

CN107277516A|2017-10-20|

---

RU2020107177A|2021-08-17|

---

JP2017085639A|2017-05-18|

---

TW201715884A|2017-05-01|

---

MY176387A|2020-08-05|

---

CN113286155A|2021-08-20|

---

MY172126A|2019-11-14|

---

KR20200008662A|2020-01-28|

---

CN107743240B|2020-03-17|

---

RU2718415C2|2020-04-02|

---

EP3694214B1|2021-08-25|

---

EP3432582A1|2019-01-23|

---

US10277898B2|2019-04-30|

---

JP2016131384A|2016-07-21|

---

RU2015103531A|2015-06-20|

---

CN104170391B|2018-06-29|

---

KR20200013789A|2020-02-07|

---

JP5904269B2|2016-04-13|

---

JP6299889B2|2018-03-28|

---

KR102153270B1|2020-09-07|

---

DK3694214T3|2021-09-27|

---

KR20200014435A|2020-02-10|

---

CN107743240A|2018-02-27|

---

JP6075484B2|2017-02-08|

---

US20180020219A1|2018-01-18|

---

MX339576B|2016-05-31|

---

EP3300370A1|2018-03-28|

---

KR102154966B1|2020-09-10|

---

KR20200008013A|2020-01-22|

---

TWI519133B|2016-01-21|

---

KR20200008015A|2020-01-22|

---

CA3041408C|2021-06-08|

---

US20170214919A1|2017-07-27|

---

AR112716A2|2019-12-04|

---

EP2822275A4|2015-12-02|

---

EP2822275B1|2018-09-12|

---

KR20180017210A|2018-02-20|

---

PL3432584T3|2020-09-07|

---

KR20200009125A|2020-01-29|

---

JPWO2013129617A1|2015-07-30|

---

US20140369408A1|2014-12-18|

---

TW201336315A|2013-09-01|

---

US9967566B2|2018-05-08|

---

JP2018110431A|2018-07-12|

---

JP5904266B2|2016-04-13|

---

JP6635184B2|2020-01-22|

---

TWI568246B|2017-01-21|

---

WO2013129616A1|2013-09-06|

---

IN2015DN00926A|2015-07-10|

---

US20190260993A1|2019-08-22|

---

US20190174129A1|2019-06-06|

---

JPWO2013129618A1|2015-07-30|

---

EP3694214A1|2020-08-12|

---

EP3709652A1|2020-09-16|

---

JP2019071653A|2019-05-09|

---

AU2013227608A1|2014-08-21|

---

CN104170391A|2014-11-26|

---

KR20180015756A|2018-02-13|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

JP2753091B2|1990-01-09|1998-05-18|株式会社リコー|Image data expansion circuit|

---

KR100287624B1|1992-09-28|2001-04-16|야스카와 히데아키|Image data compression, decompression method and device|

---

US6865291B1|1996-06-24|2005-03-08|Andrew Michael Zador|Method apparatus and system for compressing data that wavelet decomposes by color plane and then divides by magnitude range non-dc terms between a scalar quantizer and a vector quantizer|

---

US6125201A|1997-06-25|2000-09-26|Andrew Michael Zador|Method, apparatus and system for compressing data|

---

US8625665B2|1996-09-20|2014-01-07|At&T Intellectual Property Ii, L.P.|Video coder providing implicit coefficient prediction and scan adaptation for image coding and intra coding of video|

---

JP2900927B2|1997-10-30|1999-06-02|日本ビクター株式会社|Encoding method and encoding device|

---

FI974244A|1997-11-14|1999-05-15|Nokia Mobile Phones Ltd|Procedure for compressing an image|

---

US6993076B1|1999-05-11|2006-01-31|Thomson Licensing S.A.|Apparatus and method for deriving an enhanced decoded reduced-resolution video signal from a coded high-definition video signal|

---

JP2004147095A|2002-10-24|2004-05-20|Canon Inc|Method of decoding|

---

US7206355B2|2002-12-02|2007-04-17|Nortel Networks Limited|Digitally convertible radio|

---

KR100584552B1|2003-01-14|2006-05-30|삼성전자주식회사|Method for encoding and decoding video and apparatus thereof|

---

CN101247524B|2003-02-21|2011-05-04|松下电器产业株式会社|Picture coding method|

---

US7233703B2|2003-03-25|2007-06-19|Sharp Laboratories Of America, Inc.|Computation-reduced IDCT method for video coding|

---

US20050002337A1|2003-07-01|2005-01-06|Nokia Corporation|Reducing effects caused by transmission channel errors during a streaming session|

---

US7426308B2|2003-07-18|2008-09-16|Microsoft Corporation|Intraframe and interframe interlace coding and decoding|

---

WO2005076614A1|2004-01-30|2005-08-18|Matsushita Electric Industrial Co., Ltd.|Moving picture coding method and moving picture decoding method|

---

US7630435B2|2004-01-30|2009-12-08|Panasonic Corporation|Picture coding method, picture decoding method, picture coding apparatus, picture decoding apparatus, and program thereof|

---

US7460596B2|2004-04-29|2008-12-02|Mediatek Incorporation|Adaptive de-blocking filtering apparatus and method for MPEG video decoder|

---

CN100571389C|2004-06-29|2009-12-16|奥林巴斯株式会社|Be used for the compressed and decompressed method and apparatus of encoding/decoding image and expanded images|

---

DE102004059993B4|2004-10-15|2006-08-31|Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.|Apparatus and method for generating a coded video sequence using interlayer motion data prediction, and computer program and computer readable medium|

---

EP1675402A1|2004-12-22|2006-06-28|Thomson Licensing|Optimisation of a quantisation matrix for image and video coding|

---

JP2007140200A|2005-11-18|2007-06-07|Yamaha Corp|Language learning device and program|

---

WO2007063808A1|2005-11-30|2007-06-07|Kabushiki Kaisha Toshiba|Image encoding/image decoding method and image encoding/image decoding apparatus|

---

KR100763205B1|2006-01-12|2007-10-04|삼성전자주식회사|Method and apparatus for motion prediction using motion reverse|

---

US8711925B2|2006-05-05|2014-04-29|Microsoft Corporation|Flexible quantization|

---

RU2419855C2|2006-06-26|2011-05-27|Квэлкомм Инкорпорейтед|Reducing errors when calculating inverse discrete cosine transform|

---

JP4274234B2|2006-12-04|2009-06-03|ソニー株式会社|Compressed data amount control method and image data compression apparatus|

---

CN100520732C|2006-12-26|2009-07-29|中国电信股份有限公司|Performance test script generation method|

---

WO2008120577A1|2007-03-29|2008-10-09|Kabushiki Kaisha Toshiba|Image coding and decoding method, and apparatus|

---

CA2680140A1|2007-04-16|2008-11-06|Kabushiki Kaisha Toshiba|Image encoding and decoding method and apparatus|

---

US8213498B2|2007-05-31|2012-07-03|Qualcomm Incorporated|Bitrate reduction techniques for image transcoding|

---

KR101365441B1|2007-10-11|2014-02-19|삼성전자주식회사|Video encoding apparatus and method and video decoding apparatus and method|

---

DE102007053038A1|2007-11-07|2009-05-14|Robert Bosch Gmbh|Electronic control circuit e.g. booster circuit, for controlling injector in internal-combustion engine of tricycle, has regulator with comparator with hysteresis for regulating current to be supplied, where regulator exhibits switch|

---

KR101375663B1|2007-12-06|2014-04-03|삼성전자주식회사|Method and apparatus for encoding/decoding image hierarchically|

---

KR101426272B1|2008-03-10|2014-08-07|삼성전자주식회사|Apparatus of encoding image and apparatus of decoding image|

---

US8238675B2|2008-03-24|2012-08-07|Microsoft Corporation|Spectral information recovery for compressed image restoration with nonlinear partial differential equation regularization|

---

JP5472105B2|2008-06-27|2014-04-16|ソニー株式会社|Image processing apparatus and image processing method|

---

CN101785318B|2008-06-27|2012-12-12|索尼公司|Image processing device and image processing method|

---

US8325796B2|2008-09-11|2012-12-04|Google Inc.|System and method for video coding using adaptive segmentation|

---

CN101771868B|2008-12-31|2016-03-02|华为技术有限公司|To quantizing method and the device of image|

---

KR101480412B1|2009-01-22|2015-01-09|삼성전자주식회사|Method and apparatus for transforming image, and method and apparatus for inverse-transforming image|

---

FR2944936A1|2009-04-23|2010-10-29|Thomson Licensing|METHODS OF ENCODING AND DECODING AN IMAGE DATA BLOCK, CODING AND DECODING DEVICES IMPLEMENTING SAID METHODS|

---

JP2010288166A|2009-06-15|2010-12-24|Panasonic Corp|Moving picture encoder, broadcast wave recorder, and program|

---

JP2011029956A|2009-07-27|2011-02-10|Sony Corp|Image encoding device and image encoding method|

---

JP5233897B2|2009-07-31|2013-07-10|ソニー株式会社|Image processing apparatus and method|

---

KR101675116B1|2009-08-06|2016-11-10|삼성전자 주식회사|Method and apparatus for encoding video, and method and apparatus for decoding video|

---

KR20110069740A|2009-12-17|2011-06-23|에스케이 텔레콤주식회사|Video coding method and apparatus|

---

US8588536B2|2010-02-22|2013-11-19|Texas Instruments Incorporated|Guaranteed-rate tiled image data compression|

---

JP5213896B2|2010-03-03|2013-06-19|日本電信電話株式会社|Image processing method, image processing apparatus, and program|

---

EP2375745A1|2010-04-12|2011-10-12|Panasonic Corporation|Efficient storing of picture representation using adaptive non-uniform quantizer|

---

JP2011259205A|2010-06-09|2011-12-22|Sony Corp|Image decoding device, image encoding device, and method and program thereof|

---

AU2011308204A1|2010-09-28|2013-04-11|Samsung Electronics Co., Ltd.|Video encoding method and device and decoding method and device|

---

US9167252B2|2010-12-01|2015-10-20|Texas Instruments Incorporated|Quantization matrix compression in video coding|

---

CN101984669A|2010-12-10|2011-03-09|河海大学|Iteration method of frame-hierarchy adaptive Wiener interpolation filter|

---

ES2708940T3|2011-02-10|2019-04-12|Velos Media Int Ltd|Image processing device and image processing procedure|

---

US20140010284A1|2011-02-25|2014-01-09|Samsung Electronics Co., Ltd.|Image transform and inverse transform method, and image encoding and decoding device using same|

---

CN102281075B|2011-03-21|2013-03-06|中国人民解放军信息工程大学|Hierarchical encoding, operation and indexing method of hexagonal grid with aperture of 4|

---

CN107018424B|2011-05-20|2020-06-09|索尼公司|Image processing apparatus, image processing method, and program|

---

US9143802B2|2011-10-31|2015-09-22|Qualcomm Incorporated|Fragmented parameter set for video coding|

---

US10277915B2|2011-11-07|2019-04-30|Qualcomm Incorporated|Signaling quantization matrices for video coding|

---

JP2013129616A|2011-12-21|2013-07-04|Shikoku Chem Corp|Brominating agent and application of the same|

---

CN109194958B|2012-01-20|2021-08-20|韩国电子通信研究院|Video encoding and decoding device using quantization matrix|

---

EP3694214B1|2012-02-29|2021-08-25|Sony Group Corporation|Image processing device and method|

---

JPWO2013154028A1|2012-04-13|2015-12-17|ソニー株式会社|Image processing apparatus and method|

---

JP5559372B2|2013-02-01|2014-07-23|キヤノン株式会社|Image forming apparatus|DE69515262T2|1994-11-17|2000-08-03|Canon Kk|Imaging device|

---

LU90924B1|2002-05-24|2003-11-25|Wurth Paul Sa|Metallurgical treatment process on a metal bath|

---

ES2708940T3|2011-02-10|2019-04-12|Velos Media Int Ltd|Image processing device and image processing procedure|

---

CN109194958B|2012-01-20|2021-08-20|韩国电子通信研究院|Video encoding and decoding device using quantization matrix|

---

EP3694214B1|2012-02-29|2021-08-25|Sony Group Corporation|Image processing device and method|

---

JP6287035B2|2013-10-11|2018-03-07|ソニー株式会社|Decoding device and decoding method|

---

GB2520002B|2013-11-04|2018-04-25|British Broadcasting Corp|An improved compression algorithm for video compression codecs|

---

EP3183877B1|2014-09-19|2021-11-03|Huawei Technologies Co., Ltd.|Method and apparatus for non-uniform mapping for quantization matrix coefficients between different sizes of matrices|

---

US10302463B2|2015-11-19|2019-05-28|Corning Incorporated|Distributed fiber sensors and systems employing multicore optical fibers|

---

FR3047379A1|2016-01-29|2017-08-04|Orange|METHOD FOR ENCODING AND DECODING DATA, DEVICE FOR ENCODING AND DECODING DATA AND CORRESPONDING COMPUTER PROGRAMS|

---

EP3481063A4|2016-07-04|2019-05-08|Sony Corporation|Image processing device and method|

---

WO2018049549A1|2016-09-13|2018-03-22|Mediatek Inc.|Method of multiple quantization matrix sets for video coding|

---

US10567768B2|2017-04-14|2020-02-18|Apple Inc.|Techniques for calculation of quantization matrices in video coding|

---

CN109031440B|2018-06-04|2020-02-21|南京航空航天大学|Gamma radioactive imaging method based on deep learning|

---

CN109714132B|2019-01-23|2021-04-20|中国电子科技集团公司第二十八研究所|Multi-link anti-interference real-time transmission method used in complex electromagnetic environment|

---

EP3918797A1|2019-03-10|2021-12-08|MediaTek Inc|Mehtod and apparatus of the quantization matrix computation and representation for video coding|

---

CN113994689A|2019-03-20|2022-01-28|威诺瓦国际有限公司|Residual filtering in signal enhancement coding|

---

US20220007023A1|2019-05-15|2022-01-06|Hyundai Motor Company|Inverse quantization device and method used in video decoding device|

---

WO2020251275A1|2019-06-11|2020-12-17|엘지전자 주식회사|Scaling list-based video or image coding|

---

CN113940068A|2019-06-11|2022-01-14|Lg 电子株式会社|Video or image compilation based on zoom list parameters|

---

WO2020251279A1|2019-06-11|2020-12-17|엘지전자 주식회사|Video or image coding based on signaling of scaling list data|

---

WO2021060846A1|2019-09-23|2021-04-01|엘지전자 주식회사|Image encoding/decoding method and apparatus using quantization matrix, and method for transmitting bitstream|

---

US11206395B2|2019-09-24|2021-12-21|Mediatek Inc.|Signaling quantization matrix|

法律状态:
2018-02-14| B15K| Others concerning applications: alteration of classification|Ipc: H04N 19/126 (2014.01) |

---

2018-04-10| B06A| Patent application procedure suspended [chapter 6.1 patent gazette]|

---

2018-08-14| B09A| Decision: intention to grant [chapter 9.1 patent gazette]|

---

2018-09-25| B16A| Patent or certificate of addition of invention granted [chapter 16.1 patent gazette]|Free format text: PRAZO DE VALIDADE: 20 (VINTE) ANOS CONTADOS A PARTIR DE 20/02/2013, OBSERVADAS AS CONDICOES LEGAIS. |

优先权:

---

申请号 | 申请日 | 专利标题

---

JP2012044009|2012-02-29|

---

PCT/JP2013/054126|WO2013129203A1|2012-02-29|2013-02-20|Image processing device and method|BR122015004025A| BR122015004025B1|2012-02-29|2013-02-20|image processing device and method|