multimetric filtering

专利摘要:
MULTIMETRIC FILTRATIONA filter unit for a video encoderor video decoder can determine a firstmetric for a group of pixels within a block ofpixels based on a comparison of a subset ofpixels in the block with other pixels in the block; determine afilter based on the first metric and generate an imagefiltered by applying the filter to the pixel group. Osubset of pixels can be selected so as not toinclude pixels at the edge of the pixel block.
公开号:BR112013021617A2
申请号:R112013021617-4
申请日:2012-02-22
公开日:2020-09-29
发明作者:In Suk Chong；Marta Karczewicz
申请人:Qualcomm Incorporated；
IPC主号:

专利说明:

“MULTIMETRIC FILTRATION” This order claims priority for: U.S. Provisional Order
No. 61 / 445,967, filed on February 23, 2011; U.S. Provisional Order
No. 61 / 448,771, filed on March 3, 2011; U.S. Provisional Order
No. 61 / 473,713, filed on April 8, 2011; U.S. Provisional Order
No. 61 / 476,260, filed on April 16, 2011; U.S. Provisional Order
No. 61 / 478,287, filed on April 22, 2011; U.S. Provisional Order
No. 61 / 503,426, deposited on June 30, 2011; U.S. Provisional Order
No. 61 / 503,440, deposited on June 30, 2011; U.S. Provisional Order
No. 61 / 527,463, filed on August 25, 2011; U.S. Provisional Application
No. 61 / 531,571, deposited on September 6, 2011; the entire content of each of which is incorporated herein by reference in its entirety.
Field of the Invention This description refers to the block-based digital video encoding used to compress video data and, more particularly, to techniques for filtering video blocks.
Description of the Prior Art Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless communication devices such as radio telephony devices, wireless broadcast systems, personal digital assistants (PDAs), laptop computers, desktop computers, tablet computers, digital cameras, digital recording devices, video game devices, video game consoles, and the like.
Digital video devices implement video compression techniques, such as MPEG-2, MPEG-4, or ITU-T H.264 / MPEG-4, Part 10, Advanced Video Encoding (AVC), to transmit and receive video more efficiently.
Video compression techniques perform spatial and temporal prediction to reduce or remove redundancy inherent in video sequences.
New video standards, such as the High Efficiency Video Coding (HEVC) standard being developed by the "Joint Collaboration Team - Video Coding" (JCTVC), which is a collaboration between MPEG and ITU-T, continue to emerge and evolve.
This new HEVC standard is also sometimes referred to as H.265. Block-based video compression techniques can perform spatial and / or temporal prediction.
Intracoding is based on spatial prediction to reduce or remove spatial redundancy between video blocks within a given encoded video unit, which may comprise a video frame, a slice of a video frame, or the like.
In contrast, intercoding is based on temporal prediction to reduce or remove temporal redundancy between video blocks of successive encoding units in a video sequence.
For intracoding, the video encoder performs spatial prediction to compress data based on other data within the same encoded video unit.
For intercoding, the video encoder performs motion estimation and rod motion compensation to track the movement of corresponding video blocks from two or more adjacent encoded video units.
An encoded video block can be represented by the prediction information that can be used to create or identify a prediction block, and a residual data block indicating the differences between the block being encoded and the prediction block. In the case of intercoding, one or more motion vectors are used to identify the data prediction block from an anterior or posterior coding unit, while in the case of intrododification, the prediction mode can be used to generate the chunk. prediction based on data within the CU associated with the video block being encoded. Both intracoding and intercoding can define several prediction modes: different, which can define different block sizes and / or prediction techniques used in coding. Additional “types of syntax elements can also be | included as part of the encoded video data in order to control or define the encoding techniques or parameters used in the encoding process.
After block-based prediction encoding, the video encoder can apply entropy and quantization and transformed encoding processes to further reduce the bit rate associated with the communication of a residual block. Transform techniques can comprise discrete cosine transforms (DCTs) or conceptually similar processes, such as short wave (wavelet), integer transform, or other types of transform. In a discrete cosine transform process, as an example, The transform process converts a set of values from Alf aranAms Ada nivaAit Am AAA fiAijiAntam Ada (locks farmaeds AA can represent the energy of pixel values in the frequency domain.
Quantization is applied to the transform coefficients and usually involves a process that limits the number of bits associated with any given transform coefficient.
Entropy coding comprises one or more processes that collectively compress a sequence of quantized transform coefficients.
Video block filtering can be applied as part of the encoding and decoding loops, or as part of a post-filtering process on reconstructed video blocks.
Filtering is commonly used, for example, to reduce blocking or other common artifacts of block-based video encoding.
Filter coefficients (sometimes called filter outputs) can be defined or selected in order to promote. desired levels of video block filtering that can reduce blocking and / or improve video quality in 4 other ways.
A set of filter coefficients, for example, can define how filtering is applied along the edges of the video blocks or other locations within the video blocks.
Different filter coefficients can cause different levels of filtration with respect to different pixels of the video blocks.
Filtering, for example, can smooth out or accentuate differences in intensity of adjacent pixel values to help eliminate unwanted artifacts.
Summary of the Invention This description describes techniques associated with filtering video data in a video encoding and / or video decoding process.
According to this description, filtering is applied to an encoder, and the filter information is encoded in the bit stream for it was applied to the encoder.
The decoder receives encoded video data that includes the filter information, decodes the video data, and applies filtering based on the filtering information.
In this way, the decoder applies the same filtering that was applied to the encoder.
According to the techniques of that description, frame-by-frame, slice-by-slice, LCU-by-LCU, an encoder can select one or more sets of filters and encoded unit-encoded entity, the encoder can determine whether or not to apply filtering.
For the coded units (CUs) that are to be filtered, the encoder can perform filtering based on pixel-by-pixel or group-by-group, where a group can, for example, be a block of 2x2 pixels, or a 4x4 pixel block. In one example, a BR video encoding method includes determining a first metric for a block of pixels, where the first metric is determined based on 'a comparison of a subset of pixels in the block to other pixels in the block based in the first metric, determining a filter for the pixel block, and generating an image filtered by applying the filter to the pixel block.
In another example, a video encoding device includes a filter unit configured to determine a first metric for a block of pixels, where the first metric is determined based on a comparison of a subset of pixels in the block with other pixels in the block , determining a filter for the pixel block based on the first metric, and generating an image filtered by applying the filter to the pixel block, and a memory configured to store a filtered result from the filter unit.
In another example, the video encoding apparatus includes mechanisms for determining a first metric for a block of pixels, where the first metric is determined based on a comparison of a subset of pixels in the block to other pixels in the block; mechanisms for determining a filter for the pixel block based on the first metric; and mechanisms for generating a filtered image by applying the filter to the pixel block.
In another example, the computer-readable storage medium stores instructions that, when executed, cause one or more processors to determine a first metric for a block of pixels, where the first metric is determined based on a comparison of a subset of pixels in the block to other pixels in the block; determining a filter for the pixel block based on. first metric; and generating a filtered image by applying the filter to the pixel block.
* Details of one or more examples are shown in the following figures and in the description below.
Other characteristics, objectives and advantages will be apparent from the description and figures and from the claims.
Brief Description of the Drawings Figure 1 is a block diagram illustrating an illustrative video encoding and decoding system.
Figure 2A and figure 2B are conceptual diagrams illustrating an example of quaternary tree partitioning (quadtree) applied to a larger coding unit (LCU).
Figures 2C and 2D are conceptual diagrams illustrating an example of a filter map for a series of video blocks corresponding to the quaternary tree partitioning example in Figures 2A and 2B.
Figure 3 is a block diagram illustrating an example video encoder consistent with this description.
Figure 4A is a conceptual diagram illustrating a range mapping for two metrics for the filters.
Figure 4B is a conceptual diagram illustrating a range mapping for an activity metric and a direction metric for filters.
Figure 5 is a block diagram illustrating an example video decoder consistent with this description.
Figures 6A, 6B and 6C illustrate conceptual diagrams of a 4x4 pixel block. . Figure 7 is a flow chart illustrating the coding techniques consistent with this description. and Figures 8A and 8B are flowcharts illustrating coding techniques consistent with that description. Figures 9A and 9B are flowcharts illustrating coding techniques consistent with this description.
Figure 10 is a flow chart illustrating coding techniques consistent with this description.
Figure 11 is a flowchart illustrating coding techniques consistent with this description.
Detailed Description of the Invention This description describes the techniques associated with filtering video data in a video encoding and / or video decoding process. According to that description, filtering is applied to an encoder, and the filter information is encoded in the bit stream to allow a decoder to identify and receive encoded video data that includes filtering information, decodes the video data, and applies filtering based on the filtering information.
In this way, the decoder applies the same filtering based on the filtering information.
In this way, the decoder applies the same filtering that was applied to the encoder.
According to the techniques of that description, frame-by-frame, slice-by-slice or LCU-by-LCU, an encoder can select one or more sets of filters, and unit-coded-unit-coded, an encoder can determine whether or not to apply filtering.
For coded units (CUs) to be filtered, the encoder can perform pixel-by-pixel or group-by-group filtering, where a group can, for example, be a 2x2 pixel block or a 4x4 pixel block. . According to the techniques of that description, the video data can be encoded in units referred to as LU encoded units (CUs). CUs can be partitioned into smaller CUs, or subunits, using a quaternary tree partitioning scheme.
The syntax identifying the quaternary tree partitioning scheme for a particular CU can be passed from an encoder to a decoder.
Multiple entries associated with each subunit of a given CU can be filtered during the process - from decoding and reconstructing the encoded video data.
According to the techniques of that description, the filter description syntax can describe a set of filters, such as how many filters are in the set or what format the filters take.
The additional syntax in the bit stream received by the decoder can identify the filters (that is, OS filter coefficients) used in the encoder for a nartienlar ananuity. The filter used for the particular entry can be selected based on two or more metrics, where certain combinations of values for the two or more metrics are indexed for specific filters within a set of filters. In other cases, two or more metrics can be combined to form a single metric. Mapping filters to metrics can also be signaled in the bit stream. Different types of filtering can be applied to pixels or pixel blocks based on two or more metrics determined for the video data. The filter used for a particular pixel can be selected based on two or more metrics, such as some combination of an activity metric and a direction metric. An activity metric, for example, can quantify the activity associated with one or more blocks of. pixels within the video data. The activity metric can comprise an indicative variation metric of. pixel variation within a set of pixels. An activity metric can be direction-specific or non-direction-specific. For example, a non-specific activity metric can include a modified sum Laplacian value, as explained in more detail below.
Examples of direction-specific activity metrics include a horizontal activity metric, a vertical activity metric, a 45 degree activity metric, and a 135 degree activity metric. A direction metric can for a block of pixels quantify any horizontal activity, vertical activity, or diagonal activity of a pixel or group of pixels, or a direction metric can include a comparison of horizontal activity, vertical activity, and / or diagonal activity , and in pixel values in a horizontal direction, vertical activity generally refers to changes in pixel values in a vertical direction, and diagonal activity generally refers to changes in pixel values in a diagonal direction.
According to the techniques of this description, when determining a filter for a block of pixels, a subset of pixels within the block can be used to reduce the encoding and decoding complexity.
For example, when determining a filter for a 4x4 pixel block, it may not be necessary to use all sixteen 4x4 block pixels. Additionally, according to the techniques of this description, the subset of pixels within a current block being encoded can be selected so that the metrics are calculated only. using pixel values from the current block and not pixel values from neighboring blocks. For example, the metric for one. pixel being evaluated can be calculated based on the comparison of the pixel with neighboring pixels. At times, one or more of the pixels neighboring the pixel being evaluated may be in a different block from the pixel being evaluated. In other cases, however, one or more of the pixels neighboring the pixel may be in the same block as the pixel. According to the techniques of this description, theThe subset of pixels can be selected to include pixels that do not have neighboring pixels in neighboring blocks. In addition or alternatively, the subset of pixels may include pixels that have neighboring pixels in neighboring blocks, but those neighboring pixels in neighboring blocks may not be used when determining the metric. Based on the determination of a particular metric in pixels within a current block and not in the pixels of the blocks and / or decoder, it can, in some cases, be reduced or even eliminated.
In some cases, according to the technique of this description, the subset of pixels from within a current block being encoded can be selected so that the metrics are calculated only using pixel values from the current block and the neighboring blocks left and right, but not the pixel values of the upper neighboring blocks or lower neighboring blocks.
As a result of the scan order used when encoding video blocks, the line stores for upper and lower neighboring blocks tend to need to store many more pixel values than the line stores for storing pixel values of the left neighboring blocks. and right. and According to the techniques of that description, a filter unit, such as an inlet loop filter. adaptive, can be configured to use multiple filters based on the multimetric filter mapping.
Multiple filters can be used in conjunction with a single entry or multiple entries.
As will be described in more detail below, the multiple inputs described in this description generally refer to the intermediate video block data or image data that is produced during the encoding and decoding processes.
Multiple entries associated with a given video block can include, for example, a reconstructed block or image (RI), a pre-unlocked reconstructed block or image (pRI), a prediction block or image (PI) and / or an image of quantized prediction error (EI). In a single input scheme, a filter can be applied to only one of the above inputs, such as IR.
In addition, as is shown in more detail abaivwo., The filtering techniques of this description can be applied to CUs of various sizes using a quaternary tree partitioning scheme. By using multiple filters with multiple metric filter mapping for split CUs using the quaternary tree partitioning scheme, video encoding performance, as measured by one or both compression rates and reconstructed video quality, can be perfected. To implement the multi-metric filtering techniques described above, an encoder maintains, through generation, update, storage - or other mechanisms, a mapping of range combinations to filters. As an example, the combination of a first range for a first metric and a first range for a second metric can map to a first filter. THE . Combining the first range for the first metric and a second range for the second metric can also map the * first filter or can map a second filter. If a first metric has eight ranges and the second metric has four ranges, for example, then the first and second metrics can have thirty-two combinations of ranges, and each of the thirty-two combinations can be mapped to a filter. Each combination, however, is not necessarily mapped to a single filter. In this way, the thirty-two combinations can map four filters, eight filters, ten filters, or some other number of filters. In order to apply the same filters as an encoder, the decoder can also maintain the same range combinations mappings for filters.
This description describes the techniques for signaling from an encoder to a decoder, in an encoded bit stream, a mapping of combinations of silicon and filtering OD manually in it (nAr avemelae and associate each band combination with a filter identification (ID). A simple way to signal this mapping is to use a code word for each filter ID and then, for each combination of ranges, send the code word of the corresponding filter ID, but this technique is typically inefficient. techniques of the present description can explore correlations within the mapping by using differential encoding methods. Combinations of ranges that share a common range sometimes use the same filter. As an example, combining a first range for a first metric and from a first range to a second metric and the combination of the first range to the first metric and a second range to the second metric share a common range (the first range of the first 2 metric). This way, these two combinations can, in some cases, map the same filter ID. For exploration from this correlation, the techniques of that description can reduce the number of bits needed to signal the mapping of the track combinations to the filter IDs from an encoder to a decoder.
In addition to signaling the mapping of range combinations to filter IDs, this description also describes signaling techniques, in an encoded bit stream, filter coefficients for filters. The techniques of this description include the use of differential encoding methods to signal the filter coefficients from an encoder to a decoder. In this way, the filter coefficients for a second filter must be communicated to a decoder as difference information, where the difference information describes how to modify the filter coefficients of a first IR filter.
Differential encoding techniques can be more efficient (that is, can result in greater bit savings) when the filter coefficients of the first and second filters are more similar than compared to when the filter coefficients of the first and second filters sound. more similar compared to when the filter coefficients of the first and second filters are less similar.
The techniques of this description include determining a sequential order in which to signal the filter coefficients for the filters.
The sortings determined using the techniques described in this description can result in coding of improved differential filter coefficients, and thus can in some cases result in a bit saving when signaling the filter coefficients. z Although the techniques in this description can sometimes be described with reference to 7-input loop filtering, the techniques can be applied to input loop filtering, post-loop filtering, and other filtering schemes such as switched filtering .
Input loop filtering generally refers to filtering in which the filtered data is part of the encoding and decoding loops so that the filtered data is used for prediction intra or intercoding.
Post-loop filtering refers to the filtering that is applied to the reconstructed video data after the encoding loop.
With post-loop filtration, unfiltered data, as opposed to filtered data, is used for prediction intra or intercoding.
In some implementations, the type of filtering can switch between post-loop filtering and incoming loop filtering, for example, frame-by-frame, slice-by-slice, or another base, and at
.
input loop can be signaled from the encoder to the decoder for each slice, frame, etc. The techniques in this description are not limited to input loop filtering or post-filtering, and can apply to a wide range of filtering applied during video encoding. In this description, the term "encoding" refers to encoding or decoding. Similarly, the term "encoder" generally refers to any video encoder, video decoder, or video encoder and decoder (codec). Accordingly, the term "encoder" is used here to refer to a specialized computer device or device that performs video encoding or decoding. Additionally, in this description, the term "filter" generally refers to a set of coefficients of: filter. For example, a 3x3 filter can be defined by a set of 9 filter coefficients, a 5x5 filter can * be defined by a set of 25 filter coefficients, a 9x5 filter can be defined by a set of 45 filter coefficients, and so on. The term "filter set" generally refers to a group of more than one filter. For example, a set of two 3x3 filters, can include a first set of 9 filter coefficients and a second set of 9 filter coefficients. According to the techniques described in that description, for a series of video blocks, such as a frame, slice or larger encoding unit (LCU), filter sets of information identification are signaled from the encoder to the decoder in a header for the video block series. The term "format" sometimes called "filter holder", generally refers to the number of rows and filter coefficients and the particular number. For example, 9x9 is an example of a first format, 9x5 is an example of a second format, and 5x9 is an example of a third format. In some cases, filters can take on non-rectangular shapes including diamond shapes, diamond shapes, circular shapes, circular shapes, hex shapes, octagon shapes, cross shapes, X shapes, T shapes, other geometric shapes or countless others formats or settings.
Figure 1 is a block diagram illustrating an illustrative video encoding and decoding system 110 that can implement the techniques of that description. As shown in Figure 1, system 110 includes a source device 112 that transmits encoded video data to a destination device 116 over a communication channel 115. The source device 112 and the destination device 116 can comprise any an . from a wide range of devices. In some cases, the source device 112 and the destination device 116 may comprise wireless communication device devices, such as satellite or cellular radiotelephones. The techniques of this description, however, which apply more generally to filtering video data, are not necessarily limited to wireless applications or configurations, and can be applied to wired devices including video encoding and / or decoding capabilities. In the example in Figure 1, the source device 112 includes a video source 120, a video encoder 122, a modulator / demodulator (modem) 123 and a transmitter
124. The target device 116 includes a receiver 126, a modem 127, a video decoder 128 and a video encoder 122 of the source device 112 can be configured to select one or more sets of filter coefficients for multiple inputs in a video block filtering process and then encode the one or more sets of selected filter coefficients. Filters specific to one or more sets of filter coefficients can be selected based on one or more metrics for one or more entries, and the filter coefficients can be used to filter one or more entries. The filtering techniques in this description are generally compatible with any techniques for coding or signaling filter coefficients in an encoded bit stream.
According to the techniques of that description, a device including a video encoder 122 can - signal to a device including video decoder 128 one or more sets of filter coefficients. for a series of video blocks, such as a frame or a slice. For the video block series, video encoder 122 can, for example, signal a set of filters to be used with all inputs, or it can signal multiple sets of filters to be used with multiple inputs (one set per input, for example). Each video block or CU within the series of video blocks can then contain additional syntax to identify which filter or filters in the filter set should be used for each entry in that video block, or according to the techniques of that description, which filter or filters from the filter set to be used can be determined based on two or more metrics associated with one or more of the inputs. More specifically, the video encoder 122
"filter sets for a series of video blocks, apply filters from the set to pixels or pixel groups of inputs associated with the CUs of the video block series during the encoding process, and then encode the filter sets ( that is, sets of coefficients - of filter) for communication to the OO video decoder 128 of the target device 116. The video encoder 122 can determine one or more metrics associated with the inputs of the encoded CUs in order to select which filters from filter sets to use with the pixels or groups of pixels for that particular CU.
The video encoder 122 can also signal to the video decoder 128, as part of the encoded bit stream, a mapping of track combinations to filters within a set of. filters.
On the decoder side, the decoder. video 128 can determine the filter coefficients based on the filter information received in the bitstream syntax.
The video decoder 128 can decode the filter coefficients based on direct decoding or prediction decoding depending on how the filter coefficients were decoded, which can be flagged as part of the bitstream syntax.
In addition, the bit stream can include filter description syntax information to describe the filters for a set of filters.
Based on the filter description syntax, the decoder 128 can reconstruct the filter coefficients based on additional information received from the encoder 122. The illustrated system 110 of figure 1 is merely illustrative.
The filtering techniques of that description can be performed by any source 112 and the target device 116 are merely examples of coding devices that can support such techniques. The video decoder 128 can also determine the mapping of combinations of bands to filters based on the filter information received in the bitstream syntax.
The video encoder 122 of the source device 112 can encode the video data received from the video source 120 using the techniques of that description. The video source 120 may comprise a video capture device, such as a video camera, a video file containing previously captured video, or a video feed from a video content provider. As an added alternative, the video source 120 can generate data based on. computer graphics as the source video, or a combination of live video, archived video, and computer generated video. In some cases, if the video source 120 is a video camera, the source device 112 and the target device 115 can form so-called camera phones or video phones. In each case, the captured, pre-captured or computer generated video can be encoded by the video encoder. Since the video data is encoded by the video encoder 122, the encoded video information can then be modulated by the modem 123 according to a communication standard, for example, such as code division multiple access (CDMA), frequency division multiple access (FDMA), orthogonal frequency division multiplexing (OFDM), or any other standard or communication technique, and transmitted to the destination device 116 through transmitter 124. Modem 123 can
. other components designed for signal modulation. Transmitter 124 may include circuits designed to transmit data, including amplifiers, filters and one or more antennas.
Receiver 126 of destination device 116 receives information over channel 115, and modem 127 demodulates the information. The video decoding process performed by video decoder 128 may include filtering, for example, as part of the input loop decoding or as a post-filtering step following the decoding loop. In any case, the filter set applied by the video decoder 128 to a particular slice or frame can be decoded using the techniques of that description. The decoded filter information may include identification of the syntax. filter description in the encoded bit stream. If, for example, prediction coding is used to filter coefficients, similarities between different filter coefficients can be explored to reduce the amount of information carried through the channel.
115. In particular, a filter (that is, a set of filter coefficients) can be predictably encoded as difference values relative to another set of filter coefficients associated with a different filter. The different filter can, for example, be associated with a different slice or frame. In such a case, the video decoder 128 may receive an encoded bit stream comprising video blocks and filter information that identifies the different frame or slice with which the different filter is the associated filter. The filter information also includes difference values that define the current filter with respect to the different CU filter. In ma ie ia mo tra IA VAS = As Alpha vANAHS ear AAm AAméÉvvAAHAAV
. filter coefficient difference values that define filter coefficients for the current filter with respect to the filter coefficients of a different filter used for a different CU.
Video encoder 128 decodes the video blocks, generates the filter coefficients, and filters the decoded video blocks based on the generated filter coefficients. The video decoder 128 can generate the filter coefficients based on the filter description syntax retrieved from the bit stream. The decoded and filtered video blocks can be mounted on video frames to form the decoded video data. The display device 128 displays the decoded video data for a user and can comprise any of a variety of video devices. display such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma monitor, organic light emitting diode (OLED) monitor, or other type of display device.
Communication channel 115 can comprise any means of wired or wireless communication, such as a radio frequency (RF) spectrum or one or more physical transmission lines, or any combination of wired or wireless media. Communication channel 115 can form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. Communication channel 115 generally represents any suitable communication medium, or collection of different communication mechanisms, for transmitting video data from source device 112 to destination device 116. Again, figure 1 is merely illustrative and the techniques of this
.
(eg video encoding or decoding) that do not necessarily include any data communication between the encoding and decoding devices. In other examples, data must be retrieved from local memory, sequenced over a network, or the like.
Alternatively, encoded data can be sent from video encoder 122 to storage device 132. Similarly, encoded data can be accessed from storage device 132 by video decoder 128. Storage device 132 can include any one of a variety of data storage media accessed locally or distributed such as a hard drive, Blu-ray discs, DVDs, CD-ROMs, flash memory,. volatile or non-volatile memory, or any other digital storage medium suitable for the storage of 7 encoded video data. In an additional example, storage device 132 can correspond to a file server or other intermediate storage device that can hold the encoded video generated by the source device 112. The destination device 116 can access the stored video data from the device 132 through streaming and download. The file server can be any type of server capable of storing encoded video data and transmitting that encoded video data to the target device 116. Illustrative file servers include a network server (for example, for a network site) , an FTP server, network attached storage devices (NAS), or a local hard drive. The target device 14 can access the pattern data, including an Internet connection. This can include a wireless channel (for example, a Wi-Fi connection), a wired connection (for example, DSL, cable modem, etc.), or a combination of both that is suitable for accessing video data encoded files stored on a file server. The transmission of the encoded video data from the storage device 132 can be a stream, a download stream, or a combination of both.
The techniques in this description are not necessarily limited to wireless applications or settings. The techniques can be applied to video encoding in support of any multimedia application, such as broadcast television over the air, cable television broadcasts, satellite television broadcasts, etc. streaming video streams, for example, over the Internet, digital video encoding for. storage on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some instances, system 110 may be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting and / or video telephony.
Video encoder 122 and video decoder 128 can operate according to a video compression standard such as ITU-T H.264, alternatively referred to as MPEG-4, part 10, Advanced Video Encoding (AVC), which will be used in parts of that description for explanation purposes. However, many of the techniques in this description can be readily applied to any one of a variety of newly emerging ones. Generally, any standard that allows filtering on the encoder and decoder can benefit from the various aspects of teaching that description.
Although not shown in Figure 1, in some respects, video encoder 122 and video decoder 28 can each be integrated with an audio encoder and decoder, and may include suitable MUX-DEMUX units, or other hardware and software, to handle the encoding of both audio and video in a common data stream or separate data stream. If applicable, MUX-DEMUX units can conform to the ITU H.223 multiplexer protocol or other protocols such as the User Datagram Protocol (UDP).
Video encoder 122 and video decoder 128 can each be implemented as one or more. microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICS), "field programmable port arrangements (FPGAs), discrete 'logic, software, hardware, firmware, or any combination thereof. Each video encoder 122 and video decoder 128 can be included in one or more encoders or decoders, any of which can be integrated as part of a combined encoder / decoder (CODEC) in a respective mobile device, subscriber device, broadcast device, server or similar.
In some cases, devices 112, 116 may operate in a substantially symmetrical manner. For example, each of the devices 112, 116 can include video encoding and decoding components. In this way, system 110 can support one-way or two-way video transmission between video devices 112,
. 116, for example, for video streaming, video playback, video broadcasting or video telephony.
During the encoding process, Video Encoder 122 can perform a number of encoding techniques or steps.
In general, video encoder 122 operates on video blocks within individual video frames in order to encode video data.
In one example, a video block can correspond to a macro block or a partition of a macro block.
Macro blocks are a type of video block defined by the ITU H.264 standard and other standards.
Macro blocks typically refer to 16x16 data blocks, although the term is sometimes used generically to refer to any NxXN or NxM size video block.
The ITU-T H.264 standard supports intraprediction in various block sizes, such as 16x16,. 8x8 or 4x4 for luminance components, and 8x8 for chrominance components, in addition to interpretation in several "block sizes, such as l16x16, 16x8, 8x16, 8x8, 8x4, 1x8 and 4x4 for luminance components and corresponding scaled sizes for chrominance components.
In this description, "NxN" refers to the pixel dimensions of the block in terms of vertical and horizontal dimensions, for example, 16x16 pixels.
In general, a 16x1l6 tile will have 16 pixels in a vertical direction and 16 pixels in a horizontal direction.
Likewise, an NxN block usually has N pixels in a vertical direction and N pixels in a horizontal direction, where N represents a positive integer value.
The pixels in a block can be arranged in rows and columns.
The emerging HEVC standard defines new terms for video blocks.
In particular, video blocks (or partitions thereof) can be referred to as "units of
“Larger coded (LCUS) can be divided into smaller CUs according to a quaternary tree partitioning scheme, and the different CUs that are defined in the scheme can be further divided into prediction units (PUs). LCUs, CUSs and PUs are all video blocks within the meaning of this description.
Other types of video blocks can also be added, consistent with the HEVC standard or other video encoding standards.
Thus, the phrase "video blocks" refers to any video block size.
Separate CUs can be included for luminance components and scaled sizes for chrominance components for a given pixel, although other colored spaces can be used as well.
Video blocks can be fixed or “variable sizes, and may differ in size according to a specified encoding standard.
Each video frame "can include a plurality of slices.
Each slice can include a plurality of video blocks, which can be partitioned, also referred to as sub-blocks.
According to the quaternary tree partitioning scheme mentioned above and described in more detail below, a first CU N / 2xN / 2x can comprise a sub-block of an NxN LCU, a second CU N / 4xN / 4x can also comprise a sub-block of the first CU.
An N / 8xN / 8x PU may comprise a sub-block of the second CU.
Similarly, as an additional example, block sizes that are less than 11-16x16 can be referred to as partitions of a 16x16 video block or as 16x16 video block sub-blocks. Likewise, for an NxN block, block sizes smaller than NXN can be referred to as partitions or NxXN block sub-blocks.
The video blocks a AAA a and a.
+ pixel, or blocks of transform coefficients in the transform domain, for example, following the application of a transform such as a discrete cosine transform (DCT), an integer transform, a short wave transform, or a conceptually similar transform residual video block data representing pixel differences between encoded video blocks and prediction video blocks. In some cases, a video block may comprise blocks of transform coefficients quantized in the transform domain.
Syntax data within a bit stream can define an LCU for a frame or slice, which is a larger coding unit in terms of the number of pixels for that frame or slice. In general, an LCU or CU serves a purpose similar to a coded macro block. according to H.264, except that LCUS and CUs do not have a specific size distinction. Instead, an LCU size can be defined frame-by-frame, slice-by-slice, and an LCU is divided into CUs. In general, references in this description to a CU can refer to an LCU of an image or a sub-CU of an LCU. An LCU can be divided into sub-CUs, and each sub-CU can be divided into sub-CUs. The syntax data for a bit stream can define a maximum number of times that an LCU can be divided, with reference to a CU depth. Accordingly, a bit stream can also define a smaller encoding unit (SCU). This description also uses the terms "block" and "video block" to refer to any one of LCU, CU, PU, SCU or TU. As introduced above, an LCU can be associated with a quaternary tree data structure. In general, a quaternary tree data structure
* 1 28/104 «a CU is divided into four sub-CUs, oThe node corresponding to the CU includes four leaf nodes, each of which corresponds to one of the sub-CUs. Each node in the quaternary tree data structure can provide syntax data for the corresponding CU. For example, a quaternary tree node can include a split indicator, indicating whether the CU corresponding to the node is divided into sub-CUs. The syntax elements for a CU can be defined recursively, and may depend on whether the CU is divided into sub-CUs.
A CU that is not split can include one or more prediction units (PUs). In general, a PU represents all or a part of the corresponding CU, and includes data for retrieving a reference sample for the PU. For example, when the PU is encoded by intramode, the PU may include data describing a mode. intraprediction for PU. As another example, when the PU is encoded in between, the PU can include data defining a "motion vector for the PU. The data defining the motion vector can describe, for example, a horizontal component of the motion vector, a vertical component of the motion vector, a resolution for the motion vector (for example, precision of a quarter pixel or precision of an eighth of a pixel), a frame of reference to which the motion vector points and / or a reference list (for example, list O or list 1) for the motion vector. The data for the CU defining the PUs can also describe, for example, the partition of the CU into one or more PUs. The partition modes may differ between whether the CU was not coded, coded intrapredition mode, or coded interpretition mode.
A CU having one or more PUs can also include one or more transform units (TUS). The TUs
. Í 29/104 “residual transforms, which are typically quantized.
In particular, the following prediction using a PU, a video encoder can calculate the residual values for the part of the CU corresponding to the PU.
Residual values can be transformed, quantized, scanned and stored in a TU, which can have variable sizes corresponding to the size of the transform that was performed.
Accordingly, a TU is not necessarily limited to the size of a PU.
Thus, the TUs can be larger or smaller than the corresponding PUs for the same CU.
In some examples, the maximum size of a TU may be the size of the corresponding CU.
Again, TUs can understand data structures that include the residual transform coefficients associated with a given CU. . Figures 2A and 2B are conceptual diagrams illustrating an illustrative quaternary tree 250 and a corresponding LCU '272. Figure 2A shows an illustrative quaternary tree 250, which includes nodes arranged hierarchically.
Each node in a quaternary tree, such as quaternary tree 250, can be a leaf node without children, or it can have four child nodes.
In the example in figure 2A, quaternary tree 250 includes in the example in figure 2A, quaternary tree 250 includes root node 252. Root node 252 has four child nodes, including leaf nodes 256A to 256C (leaf nodes 256) and node 254. Seen whereas node 254 is not a leaf node, node 254 includes four child nodes, which, in this example, are leaf nodes 258A to 258D (leaf nodes 258). Quaternary tree 250 can include data describing the characteristics of a corresponding LCU, such as LCU 272 in this example.
For example, quaternary tree 250, by its structure, can describe a and A o A DN AAA ro SSS Ah z 30/104. a size of 2Nx2N.
LCU 272, in this example, has four sub-CUs 276A to 276C (sub-CUs 276) and 274, each of size NxN.
Sub-CU 274 is further divided into four sub-CUs 278A-278D (sub-CUs 278), each of size N / 2xN / 2. The quaternary tree structure 250 corresponds to the division of LCU 272, in this example.
That is, root node 252 corresponds to LCU 272, leaf nodes 256 correspond to sub-CUs 276, node 254 corresponds to sub-CU 274, and leaf nodes 258 correspond to sub-CUs 278. The data for the quaternary tree 250 can describe whether the CU correspond to the node is split.
If the CU is split, four additional nodes may be present in the quaternary tree 250. In some examples, a quaternary tree node may be implemented similarly to the following pseudocode:: quadtree node (boolean split flag (1); "/ / if (split flag) signaling data (quadtree node child1; quadtree node child2; quadtree node child3; quadtree node child4;)) The split flag value can be a one-bit value representative of whether the CU corresponding to the current node is divided .
If the CU is not split, the split value can be "O", while if the CU is split, the split value can be "1". With respect to the quaternary tree example 250, a set of split indicator values can be 101000000.
.
THE . 31/104 “In some examples, each of sub-CUs 276 and sub-CUs 278 can be encoded by intraprediction using the same intraprediction mode. Accordingly, the video encoder 122 can provide an indication of the intraprediction mode at root node 252. In addition, certain sizes of sub-CUs may have multiple possible transforms for a particular intraprediction mode. Video encoder 122 can provide an indication of the transform for use for such sub-CUs at root node 252. For example, sub-CUs of size N / 2xN / 2x may have multiple possible transforms available. Video encoder 122 can signal the transform for use at root node 252. Accordingly, video decoder 128 can determine the transform for application of sub-CUs 278 based on the intraprediction mode signaled at root node 252 and the transform. signaled at the root node 252. As such, the video encoder 122 does not need to signal the transforms for application to sub-CUs 276 and sub-CUs 278 on leaf nodes 256 and leaf nodes 258, but may instead simply signal an intrapredictive mode and, in some examples, a transform for application of certain sizes of sub-CUs, in the root node 252, according to the techniques of that description. In this way, these techniques can reduce the overhead cost of signaling transform functions for each sub-CU of an LCU, such as the LCU 272.
In some examples, the intraprediction modes for sub-CUs 276 and / or sub-CUs 278 may be different from the intraprediction modes for LCU 272. Video encoder 122 and video decoder 130 can be configured with the functions that map a signaled intraprediction mode at the root node 252 to a non-level intranredition mode nara enb-Clcoc 276 e / O0O1 enb-Clla 2978.
* P 32/104 The function can provide many to one mapping of intraprediction modes available for LCU 272 to intraprediction modes for sub-CUs 276 and / or sub-CUs 278. A slice can be divided into video blocks (or LCUS) and each video block can be divided according to the quaternary tree structure described in relation to figures 2A and B. Additionally, as illustrated in figure 2C, the quaternary tree sub-blocks indicated by 'ON' can be filtered by the loop filters described here, while the quaternary tree sub-blocks indicated by 'OFF' may not be filtered. The decision of whether or not to filter a particular block or sub-block can be determined in the encoder by comparing the filtered result and unfiltered result with respect to the original block being encoded. Figure 2D: is a decision tree representing the decisions of Ú partition that result in the partitioning in a quaternary tree illustrated in figure 2C. Actual filtering applied to any pixels for 'ON' blocks can be determined based on the metrics discussed here.
In particular, Figure 2C can represent a relatively large video block that is divided according to a quaternary tree partitioning scheme into smaller video blocks of various sizes.
Each video block is labeled (on or off) in figure 2C to illustrate whether filtering should be applied or avoided for that video block. The video encoder can define this filter map by comparing filtered and unfiltered versions of each video block with the original video block being encoded.
Again, the 2D figure is a decision tree corresponding to the partition decisions that result in the hand AiAnNAmMAnNdhA Am freruoArPA Ana tarnrária dJignotrada at fimqmora
. 33/104 7; 2C, In figure 2C, each circle can correspond to a CU.
If the circle includes a "l" indicator, then that CU is further divided into four more CUs, but if the circle includes a "0" indicator, then that CU is no longer divided.
Each circle (for example, corresponding to the CUs) also includes an associated diamond.
If the indicator on the diamond for a given CU is set to 1, then filtration is "ON" for that CU, but if the indicator on the diamond for a given CU is set to 0, then filtration is turned off.
In this way, figures 2C and 2D can be viewed individually or collectively since the filter map that can be generated in an encoder and communicated to a decoder at least once per encoded video data slice in order to communicate the level of tree partitioning quaternary for a given video block (for example, an LCU) whether or not filtering is applied to each partitioned video block (for example, each CU within the LCU). Smaller video blocks can provide better resolution and can be used for locations in a video frame that include high levels of detail.
Larger video blocks can provide greater encoding efficiency and can be used for locations in a video frame that include a lower level of detail.
A slice can be considered a plurality of video blocks and / or sub-blocks.
Each slice can be a decodable series independently of video blocks in a video frame.
Alternatively, the frames themselves can be a decodable series of video blocks, or other parts of a frame can be defined as decodable series of video blocks.
The term "video block series" can refer to any part
'34/104. , as a whole frame, a slice of a frame, a group of images (GOP) also referred to as a sequence, or another decodable unit independently defined according to the applicable coding techniques. Aspects of this description can be described with reference to the tables or slices, but such references are merely illustrative. It should be understood that generally any series of video blocks can be used instead of a frame or a slice.
Syntax data can be defined based on the encoded unit so that each CU includes associated syntax data. The filter information described here may be part of such a syntax for a CU, but it may more likely be part of the syntax for a series of video blocks such as a frame, a slice, a GOP, LCU or a: frame sequence. video, rather than a CU. Syntax data can indicate the filter set or sets to be used with slice or frame CUs. In addition, not all filter information necessarily needs to be included in the header of a common series of video blocks. For example, the filter description syntax can be passed in a frame header, while other filter information is signaled in a header to an LCU.
Video encoder 122 can perform prediction encoding in which a video block being encoded is compared with a prediction frame (or other CU) in order to identify a prediction block. The differences between the current video block being encoded and the prediction block are encoded as a residual block, and the prediction syntax is used to identify the prediction block. The residual block can be transformed and
'35/104 eo, a DCT process or conceptually similar process, transformed entirely, transformed with short waves, or other types of transformed. In a DCT process, as an example, the transform process converts a set of pixel values into transform coefficients that can represent the energy of the pixel values in the frequency domain. Quantization is typically applied to transform coefficients, and generally involves a process that limits the number of bits associated with any given transform coefficient.
After the transformation and quantization, entropy coding can be performed on the quantized and transformed residual video blocks. Syntax elements, such as filter information and prediction vectors defined during encoding, can also be: included in the entropy-encoded bit stream for each CU. In general, entropy coding comprises one or more processes that collectively compress a sequence of quantized transform coefficients and / or other syntax information. Scanning techniques, such as zigzag scanning techniques, are performed on quantized transform coefficients, for example, as part of the entropy coding process, in order to define one or more one-dimensional serialized coefficient vectors from the blocks. two-dimensional video. Other scanning techniques, including other scan orders or adaptive scans, can also be used, and possibly signaled in the encoded bit stream. In any case, the scanned coefficients are then encoded by entropy together with any syntax information, for example, through adaptive variable length encoding of content (CAVLC),
36/104 Adaptive Binary Context Arithmetic Coding (CABAC), or other entropy coding process.
As part of the encoding process, the encoded video blocks can be decoded in order to generate the video data used for encoding based on subsequent prediction of subsequent video blocks.
At this stage, filtration can be performed in order to improve the video quality, and, for example, remove the blocking artifacts from the decoded video.
The filtered data must be used to predict other video blocks, in which case the filtering is referred to as "input loop" filtration. Alternatively, the prediction of other video blocks can be based on unfiltered data, in which case filtering is referred to as "post filtering". "Based on frame-by-frame, slice-by-slice and LCU-by-LCU, Video encoder 122 can select one or more filter sets, and based on encoded-by-encoded unit, the encoder can determine whether or not to apply filtering.
For the CUs that are to be filtered, the encoder can perform filtering based on pixel-by-pixel or group-by-group, where a group can, for example, be a 2x2 pixel block or a 4x4 pixel block. These selections can be made to promote video quality.
Such filter sets can be selected from predefined filter sets, or they can be adaptively defined to promote video quality.
As an example, video encoder 122 can select or define multiple sets of filters for a given frame or slice so that different filters are used for different pixels or groups of pixels from
* 37/104 associated with a CU, several sets of filter coefficients can be defined, and two or more metrics associated with the CU pixels can be used to determine which filter in the filter set to use with such pixels or groups of pixels.
In some cases, the video encoder 122 can apply several sets of filter coefficients and select one or more sets that produce the best quality video in terms of the amount of encoded block and original block distortion, and / or higher levels of compression.
In any case, once selected, the set of filter coefficients applied by the video encoder 122 for each CU can be encoded and communicated to the video decoder 128 of the target device 15) so that the video decoder 128. can apply the same filtering that was applied during the coding process for each determined CU. : When two or more metrics are used to determine which filter to use with a particular input for a CU, the selection of the filter for that particular CU does not necessarily need to be communicated to the 128 video decoder. video 128 can also calculate the two or more metrics, if based on the filter information previously provided by the video encoder 122, it combines the combination of two or more metrics with a particular filter.
Figure 3 is a block diagram illustrating a video encoder 350 consistent with that description.
The video encoder 350 may correspond to the video encoder 122 of the device 120, or a video encoder of a different device.
As illustrated in figure 3, THE MILL AA of JA soda s ICE A Jan Ii love methods of this AIATA 999
'38/104 ES' adders 348 and 351, and a memory 334. The video encoder 350 also includes a transform unit 338 and a quantization unit 340, in addition to a reverse quantization unit 342 and a reverse transform unit 344 The video encoder 350 also includes an unlock filter 347 and an adaptive filter unit 349. The video encoder 350 also includes an entropy encoding unit 346. The filter unit 349 of the video encoder 350 can perform operations filtering and may also include a filter selection unit (FSU) 353 for identifying a desirable or preferred filter or set of filters to be used for decoding.
The filter unit 349 can also generate filter information by identifying the selected filters so that the selected filters 'can be effectively communicated as filter information to another device to be used during a' decoding operation.
During the encoding process, the video encoder 350 receives a video block, such as an LCU, to be encoded, and prediction module 332 performs prediction encoding techniques on the video block.
Using the quaternary tree partitioning scheme discussed above, prediction module 332 can split the video block and performs prediction encoding techniques on CUs of different sizes.
For intercoding, prediction module 332 compares the video block to be encoded, including the video block sub-blocks, with multiple blocks without one or more frames or video reference slices in order to define a prediction block For intracoding, prediction module 332 generates a prediction block based on neighboring data within
'39/104 prediction and adder 348 subtracts the prediction block from the video block being encoded in order to generate a residual block.
For intercoding, prediction module 332 can comprise motion estimation and motion compensation units that identify a motion vector that points to a prediction block and generates the prediction block based on the motion vector. Typically, motion estimation is considered the process of generating a motion vector, which estimates motion. For example, the motion vector can indicate the displacement of a prediction block within a prediction frame in relation to the current block being encoded within the current frame. Movement compensation is typically considered the process of collecting or generating a block. prediction based on the motion vector determined by the motion estimate. For intracoding, prediction module 332 generates a prediction block based on neighboring data within the same CU. One or more intraprediction modes can define how an intraprediction block can be defined.
After the prediction module 332 sends the prediction block and adder 348 subtracts the prediction block from the video block being encoded in order to generate a residual block, transform unit 338 applies a residual block transform. The transform can comprise a discrete cosine transform (DCT) or a conceptually similar transform such as that defined by a coding standard such as the HEVC standard for short wave transform, integer transform, subband transform or other types of can be used as well. In any
40/104. ; residual block, producing a block of residual transform coefficients. The transform can convert residual information from a pixel domain to a frequency domain.
The quantization unit 340 then quantizes the residual transform coefficients to further reduce the bit rate. The quantization unit 340, for example, can limit the number of bits used to encode each of the coefficients. After quantization, the entropy coding unit 346 scans the quantized coefficient block from a two-dimensional representation for one or more one-dimensional serialized vectors. The scan order can be pre-programmed to occur in a defined order (such as zigzag scan, horizontal scan, vertical scan, combinations and other pre-defined order), or possibly adaptive “defined based on 'coding statistics previous ones. Following this scanning process, the entropy coding unit 346 encodes the quantized transform coefficients (along with any syntax elements) according to an entropy coding methodology, such as CAVLC or CABAC, to further compress the data. The syntax elements included in the entropy-encoded bit stream can include prediction syntax from prediction module 332, such as motion vectors for intercoding or prediction modes for intracoding. The syntax elements included in the entropy-encoded bit stream can also include filter formation of the filter unit 349, which can be encoded as described herein.
'41/104
CAVLC is a type of entropy coding technique supported by ITU H.264 / MPEG4, AVC standard, which can be applied on a vectorized basis by the entropy coding unit 346. CAVLC uses variable length coding tables (VLC) of a form that effectively compresses the serialized "runs" of the transform coefficients and / or syntax elements.
CABAC is another type of entropy coding technique supported by the AVC ITU H. 264 / MPEG4 standard that can be applied on a vectorized basis by the entropy coding unit 346. CABAC involves several stages, including binarization, context model selection, and binary arithmetic coding.
In that case, the entropy coding unit 346 encodes the transform coefficients and syntax elements according to CABAC.
Like the AVC * ITU H.264 / MPEGA standard, the emerging HEVC standard can also support both CAVLC and CABAC entropy coding. 'In addition, many other types of entropy coding techniques also exist, and new entropy coding techniques are likely to emerge in the future.
This description is not limited to any specific entropy coding technique.
Following entropy encoding by entropy encoding unit 346, the encoded video can be transmitted to another device or archived for later transmission or retrieval.
Again, the encoded video can comprise vectors encoded by entropy and various syntaxes, which can be used by the decoder to properly configure the decoding process.
The inverse quantization unit 342 and the inverse transform unit 344 apply the inverse quantization and the inverse transform, respectively, for Amada do A LIANA [vas iAdes in a mioA damínia The r layer
42/104 351 adds the reconstructed residual block to the prediction block produced by the prediction module 332 to produce a pre-unlocked reconstructed video block sometimes referred to as the pre-unlocked reconstructed image. The 347 unlock filter can apply filtering to the pre-unlocked reconstructed video block to improve the video quality by removing the lock or other artifacts. The output of the unlock filter 347 can be referred to as a post-unlocked video block, reconstructed video block, or reconstructed image.
The filter unit 349 can be configured to receive a single input or multiple inputs. In the example in figure 3, the filter unit 349 receives as input the reconstructed post-unlocked image (RI), image reconstructed pre-unlocked (pRI), prediction image (PI) and reconstructed residual block (EI). The 349 filter unit can use any of these inputs individually or in combination to produce a reconstructed image to store in memory 334. Additionally, as will be discussed in more detail below, based on two or more metrics, the output of the filter unit filter 348 can be an additional filter applied to IR. In another example, the output of filter unit 349 may be an additional filter applied to pRI. In other examples, however, the output of filter unit 349 may be based on multiple inputs. For example, filter unit 349 can first apply a filter to pRI and then use the filtered version of pRI in conjunction with the filtered versions of EI and PI to create a reconstructed image. In cases where the output of filter unit 349 is the product of an additional filter being applied to a single inlet, the
'43/104 other entries, but these filters can all have coefficients equal to zero. Similarly, if the output of filter unit 349 is the product of applying three filters to three inputs, filter unit 349 may in fact apply a filter to the fourth input, but that filter can have all the same coefficients to zero.
The filter unit 349 can also be configured to receive a single input. For example, although figure 3 illustrates PI, EI, pRI and RI being entered into filter unit 349, in some implementations RI may be the only input received by filter unit 349. In such an implementation, filter unit 349 may apply a filter to IR so that the filtered version of IR is more similar to the original image than the unfiltered version of IR. In other implementations, the filter unit 349 and the release filter 347 can be combined into a single filter unit that applies' filtragen and pRI. The techniques in this description, which generally refer to filter mapping based on multiple metrics, are compatible with both single-entry and multiple-entry filtering schemes that use multiple filters.
Filtration by filter unit 349 can improve compression by generating prediction video blocks that more closely match the video blocks being encoded than unfiltered prediction video blocks. After filtering, the reconstructed video block can be used by the prediction module 332 as a reference block to intercode a block into a subsequent video frame or another CU. Although the 349 filter unit is illustrated "in a loop", the techniques of that description should also be used with powders
44/104 filtered data) would be used for data prediction purposes in subsequent CUs.
For a series of video blocks, such as a slice or frame, filter unit 349 can select sets of filters for each input to promote video quality.
For example, filter unit 349 can select filter sets from predefined sets of coefficients, or it can adaptively define filters to promote improved video quality or compression.
Filter unit 349 may select or define one or more filter sets for a given CU so that the same filter sets are used for the pixels of different blocks and video of that CU.
For a particular frame, slice or LCU, filter unit 349 can apply several. filter sets with multiple inputs, and FSU 353 can select the set that produces the best quality video or the highest levels of compression.
Alternatively, FSU 353 can sequence a new filter by analyzing the auto-correlations and cross-correlations between multiple entries and an original image.
A new set of filters can, for example, be determined by solving the Wienter-Hopt equations based on auto-correlations and cross-correlations.
Regardless of whether a new set of filters is sequenced or an existing set of filters is selected, filter unit 349 generates syntax for inclusion in the bit stream that allows a decoder to also identify the set or sets of filters to be used for the particular frame or slice.
According to this description, for each pixel of a CU within the series of video blocks, the unit of
45/104 of filters should be used based on two or more metrics that quantify the properties associated with one or more sets of pixels within the CU. In this way, FSU 353 can determine the filter sets for a more level-coded unit such as a frame or slice, while filter unit 349 determines which filters of the sets should be used for a particular pixel of a coded unit of lowest level based on two or more metrics associated with the pixels of that lowest level encoded unit.
A set of M filters can be used for each entry. Depending on the design preferences, M can, for example, be as little as 2 or as large as 16, or even larger. A large number of filters per entry can improve the video quality, but also. can increase the overhead associated with signaling filter sets from the encoder to the decoder. The set of M filters can be determined by FSU 353 as described above and signaled to the decoder for each frame or slice. A segmentation map can be used to indicate how a CU is segmented and whether or not a particular subunit of the CU should be filtered. The segmentation map can, for example, include for a CU a set of indicators divided as described above in addition to additional bit signaling in case each sub-CU is filtered. For each entry associated with a CU pixel to be filtered, a specific filter from the filter set can be chosen based on two or more metrics. Combinations of values for two or more metrics can be indexed for particular filters from the set of M filters. Figure 4A is a conceptual diagram illustrating
'46/104 filters from a set of filters.
The particular example in figure 4A illustrates eight filters (i.e., filter 1, filter 2, .., filter 8), but more or less filters can be used in a similar way.
Figure 4A illustrates two metrics that can be used to select a filter according to the techniques of this description.
The two metrics can, for example, quantify the properties of pixel data related to the non-specific activity of direction (for example, modified sum Laplacian value) and direction, the specific activity of direction and edge detection, a direction metric and an edge metric, a horizontal activity metric, and a vertical activity metric, or two other similar metrics.
In some cases, three or more metrics can be used, in which case the conceptual diagram in figure 4A. would include a third dimension for mapping metrics ranges to filters from the filter set. 'In the example in figure 4A, a first metric (Metric 1) has four bands (Tracks 1-1, 1-2, 1-3 and 1-4) and a second metric (Metric 2) also has four bands (Tracks 2 -1, 2-2, 2-3, 2-4). Therefore, the example in Figure 4A has sixteen combinations of ranges for Metric 1 and Metric 2. As can be seen from Figure 4A, however, each combination is not necessarily associated with a single filter.
The combination of Range 1-1 and Range 2-1 in addition to combinations 1-1 and 2-2, and 1-1 and 2-3, for example, are all mapped to Filter 1, in the example in figure 4A.
Filter 4, in contrast, is only mapped to a combination (1-1 and 2-4). Although the bands in Figure 4A are illustrated as being relatively equal, the sizes of the bands may vary.
For example,
in some implementations, Track 1-1 may encompass a
. 47/104 despite figure 4A. illustrate Metric 1 and Metric 2 as having the same number of ranges, The number of ranges for a first metric and The number of ranges for a second metric do not necessarily have to be the same. If, for example, Metric 1 is a variation metric and Metric 2 is a direction metric, Metric 1 can use eight ranges while Metric 2 uses three ranges.
In some examples, the Metric 1 and Metric 2 ranges may represent a continuous spectrum of values. For example, if Metric 1 is a modified sum Laplacian value, Range 1-2 may correspond to more activity than Range 1-3. Within a beech, the amount of activity determined for a particular pixel or group of pixels can increase by. similarly along the axis of Metric l. In other examples, the Metric 1 and Metric 2 ranges may not 'represent the actual ranges, but may instead represent discrete determinations. For example, if Metric 2 is a direction metric, Range 1-1 may correspond to a non-direction determination, Range 2-2 may correspond to a horizontal direction determination, Range 2-3 may correspond to a determination of the vertical direction, and Range 2-4 can represent a determination of the diagonal direction. As will be described in greater detail below, no direction, horizontal direction, vertical direction and diagonal direction can be discrete determinations, and thus the ranges for Metric 2 may not represent a continuous spectrum of values in the same way as the ranges of Metric 1. Figure 4B is a conceptual diagram illustrating the ranges of values for an activity metric and an A RN AA EA a.
'48/104 direction includes three discrete determinations (No direction, Horizontal and Vertical). The techniques for determining no direction, horizontal and vertical in addition to the techniques for determining activity will be explained in greater detail below.
The particular example in figure 4B illustrates six filters (ie, Filter 1, Filter 2, ..., Filter 6), but more or less filters can be used in a similar way.
As can be seen from figure 4B, the two metrics (activity and direction) create 15 combinations, identified as combinations 421 to 435. In some cases, however, additional combinations not explicitly illustrated in figure 4B can also be used.
For example, a combination corresponding to no activity can be 16a. combination that also has a corresponding filter. 'Filter unit 349 can store a filter mapping for combinations of ranges of two or more metrics, such as the illustrative mappings in figures 4A and 4B, and use the mapping to determine which filter from a set of filters to apply to a particular pixel or group of pixels in a CU.
The filter mapping for combinations of ranges of two or more metrics can, for example, be determined by filter unit 349 as part of the filter selection process described above.
Regardless of how the mapping is determined, filter unit 349 can generate information by allowing a decoder to reconstruct the mapping.
This information can be included in the encoded bit stream to signal the mapping of band combinations to filters.
The mapping of combinations to ranges, signaled by mapping range combinations to IDs of filter IDs.
The real coefficients for a filter that doesn’t filter out any specific cinalivadeoes.
'49/104 In order to generate this information, the filter unit 349 first determines a transmission order for the combinations. The transmission order generally refers to the order in which the filters will be signaled for combinations of ranges. Using figure 4A as an example, filter unit 349 can use a transmission order from left to right and top to bottom where the filter for combination 401 is signaled first, the filter for combination 402 is signaled second place, and the remaining combinations are flagged in the order of 403 => 404 => 405 => 406 => 407 => 408 => 409 => 410 => 411 => 412 => 413 => 414 => 415 => 416. The filter unit 349 can also use a top-down, zigzag transmission order where the filters for combinations are signaled in the order of 401 => 402 => 403. => 404 => 405 => 406 => 407 => 408 => 409 => 410 => 411 => 412 => 416 => 415 => 414 => 413. The filter unit 349] can also use a transmission order from top to bottom from left to right where the filters for
20. combinations are flagged in the order of 401 => 405 => 409: => 413 => 402 => 406 => 410 => 414 => 403 => 407 => 411 => 415 => 404 => 408 = > 412 => 416. The filter unit 349 can also use a left-to-right zigzag transmission order where the filters for combinations are signaled in the order of 401 => 405 => 409 => 413 => 414 => 410 => 406 => 402 => 403 => 407 => 411 => 415 => 416 => 412 => 408 => 404. With reference to figure 4B, filter unit 349 can use a transmission order from left to right and bottom to top so that the order of transmission is 421 => 422 => 423 => 424 => 425 => 426 => 427 => 428 => 429 => 430 => 431 => 432 => 433 => 434 => 435. As you can imagine, there are
'50/104 only a few of the many transmission orders that are possible.
According to a technique of that description, filter unit 349 may use a series of code words to signal the mapping to a decoder. For example, filter unit 349 can generate a first code word to indicate whether a current combination being decoded maps the same filter as the most recently decoded combination that shares the same range for the first metric. If a current combination being decoded maps the same filter as the most recently decoded combination that shares the same range for the second metric, then filter unit 349 can generate a second code word instead of the first code word. If a current combination being. decoded does not map the same filter as any of the most recently decoded combinations, so the '349 filter unit can generate a third code word instead of the first code word or the second code word, which indicates the filter corresponding to the current combination being decoded. The first and second code words of the current example can be relatively short compared to the third code word. For example, the first code word and the second code word can each be two bits (for example, 00 and 01, respectively), while the third code word is more bits (a first bit of 1, plus additional bits) . In this particular context, a current combination being decoded or a previous combination being decoded refers to the part of the encoding and decoding processes where the mapping of the filters for track combinations is being signaled by an ADA sd AD and ANNA AIAIiAAAo NEL
"51/104 necessarily for a transmission or decoding of the combination itself.
Examples of the techniques described above will now be provided with reference to figure 4A and transmission order from top to bottom and from left to right.
If, for example, combination 407 in the combination is currently being decoded, then combination 406 is the most recently decoded combination that shares the same range for Metric 1, and combination 403 is the most recently decoded combination that shares the same range for Metric 2. If combination 407 maps to the same filter (filter 7 in figure 4A) as the most recently decoded combination that shares the same range for a second metric (that is, Range 2-3 for Metric 2), then the unit filter 349 can transmit * a second code word (for example, 01) to indicate that the current combination being decoded (combination 407) Í maps the same filter as the most recently decoded combination that shares the same range as a second metric (combination 403). If, for example, combination 410 is the current combination being decoded, then combination 409 is the most recently decoded combination that shares the same range for Metric 1, and combination 406 is the most recently decoded combination that shares the same range for Metric 2. If combination 410 maps the same filter (Filter 2 in figure 4A) as the most recently decoded combination that shares the same range for a first metric (that is, Range 1-2 for Metric 1), then the filter unit 349 can transmit a first code word (for example, 00) to indicate that the current combination being decoded fLmmmm bin AHEA monthly EIAs (mà masmA child f / MDÍITLEA 33% ANA ss
'52/104 most recently decoded combination that shares the same range for a first metric (combination 409). If, for example, combination 411 is the current combination being decoded, then combination 410 is the most recently decoded combination that shares the same range for Metric 1, and combination 407 the most recently decoded combination that shares the same range for Metric 2. If combination 411 does not map the same filter as any of combination 410 or combination 407, then filter unit 349 can transmit a third code word (for example, 1 + additional bits) to indicate that the combination current being decoded (combination 411) maps to a different filter (Filter 3) than both the most recently decoded combination that shares the same range for 'Metric 1 and the most recently decoded combination that shares the same range for Metric 2.' To these current combinations where a combination that shares the same range for Metric 1 or a combination that shares the same range for Metric 2 has not yet has been decoded, then these options can be considered unavailable or can be replaced with a different combination.
If, for example, combination 409 is the current combination to be decoded, then combination 405 is the most recently decoded combination that shares the same track for Metric 2, but no combination that shares a track for Metric 1 has yet to be decoded. .
In such chaos, the most recently decoded combination that shares a range for Metric 1 can be considered as not mapping the same filter as the current combination being decoded.
Thus, in this case, the first word NO O O AA EA AMA Zz ad di TI ins IA MA = AA EIN AHÃA Year
] 53/104
Alternatively, the combination that shares a track for Metric 1 can be replaced with another combination, such as a most recently decoded combination or a previously different decoded combination.
In such a case, the most recently decoded combination before combination 409 will be combination 408. Thus, if combination 408 maps the same filter as combination 409, then filter unit 349 can generate the first code word.
Similar techniques can be used for these combinations where a previous combination sharing a common range for Metric 1 has not yet been decoded.
For the first combination in a transmission order (for example, combination 401 in the example in figure 4A), where no combination that shares the same range for Metric 1 nor a combination that shares the same * beech for Metric 2 has been decoded, the filter unit 349 can generate a code word indicating the 'filter that maps the first combination.
The filter can, for example, be signaled using the third code word or it can be signaled using a different technique, in which case the techniques described in that description can start with the second combination in a transmission order or a later combination.
According to another technique of that description, filter unit 349 may use a series of code words to signal the mapping to a decoder.
In some implementations, filter unit 349 may generate a first code word to indicate whether a current combination being decoded maps the same filter as the most recently decoded combination that shares the same range for the first metric.
If a current combination being decoded does not map the same
'54/104 shares this range for the first metric, so filter unit 349 can generate a second code word, instead of the first code word, which indicates the filter that maps the current combination being decoded. In this example, the first code word can be relatively short compared to the second code word. for example, the first codeword can be one bit (for example, 0), while the second codeword is more bits (for example, a bit equal to 1, plus additional bits). Unlike the previous technique where a short code word can be generated if a current combination maps the same filter as a previously decoded combination that shares the same range for either Metric 1 or Metric 2, this technique only includes the generation of a short code word if the current combination. map the same filter as the previously decoded combination that shares the same range as Metric '1. Thus, even if the current combination maps the same filter as a previously decoded combination that shares the same range to Metric 2, the unit of filter 349 still generates a second code word (for example, 1 + additional bits). Although this description is using Metric 1 for explanation purposes, the same techniques can also be applied using Metric 2 only.
According to another technique of that description, filter unit 349 may use a different series of code words to signal the mapping to a decoder. For example, filter unit 340 can generate a first code word to indicate whether a current combination being decoded maps the same filter as the most recently decoded combination,
'55/104 common with the previously decoded combination. If the current combination being decoded does not map the same filter as the most recently decoded combination, then filter unit 349 can generate a second code word identifying the filter that maps the current combination. In that particular implementation, the first code word can be relatively short compared to the second code word. For example, the first codeword can be one bit (for example, 0) while the second codeword is more bits (for example, a first bit equal to 1, plus additional bits).
Again, using the example in figure 4A and a transmission order from top to bottom and from left to right, combination 401 would be the most recently decoded combination if combination 402 is. currently being decoded, the 402 combination would be the most recently decoded combination if the '403 combination is the current combination, and so on. 404 would be the most recently decoded combination if combination 405 is the current combination being decoded. Thus, filter unit 349 can generate the first code word if combination 402 maps the same filter as combination 401, if combination 403 maps the same filter as combination 402, etc. Otherwise, filter unit 349 can generate the second code word identifying the filter that maps the current combination.
According to another technique of that description, filter unit 349 may use two code words to signal the mapping of filters to the combinations. A first code word, such as "0" can be used to signal that a current combination uses the same filter as a previous combination. An
'56/104 to signal that a current combination has a different filter than the previous combination. The second code word, however, does not need to identify a new filter. Instead, the new filter can be determined based on the order of transmission for the classes and the order in which the filter coefficients are transmitted. Using the left-to-right and bottom-up transmission order described above for figure 4B as an example, the code words can be transmitted according to: 421 (0) => 422 (0) => 423 ( 1) => 424 (0) => 425 (0) => 426 (0) => 427 (0) => 428 (1) => 429 (0) => 430 (0) => 431 (0) => 432 (1) => 433 (0) => 434 (0) => 435 (0), with the number in parentheses representing the code word for this combination. In this example, combinations 421-422 would be mapped to a first filter, combinations 423-427 for a second and filter, combinations 428-43l1l for a third filter and combinations 432-435 for a fourth filter. The coefficients' for the first filter, second filter, third filter, and fourth filter can correspond to the order in which the sets of filter coefficients are signaled, where the first set of signaled filter coefficients corresponds to the first filter, the second set of signaled filter coefficients corresponds to the second filter and so on. The determination of an order for the transmission of sets of filter coefficients is discussed in more detail below.
The various techniques described in this description for signaling a filter mapping for range combinations are not mutually exclusive alternatives, but can instead be used in conjunction with each other. For example, in some implementations, certain combinations “can be signaled using a
: 57/104 signaled using a second technique.
As an example, where one of a combination that shares the same range for Metric 1 or a combination that shares the same range for Metric 2 has not yet been decoded (for example, 402, 404, 505, 405, 409 and 413 combinations) , then the filter unit 349 can use a first technique.
Where both a combination that shares the same range for Metric 1 and a combination that shares the same range for Metric 2 have been decoded (for example, combinations 406, 407, 408, 410, 411, 412, 414, 415 and 416 ), then a second technique can be used.
In addition, the code words used for any of the first, second and third code words described above can be any one of fixed length code words, variable length code words, or context adaptive variable length code words. : In addition to generating information by allowing a decoder to reconstruct the filter mapping for range combinations, filter unit 349 also generates information by allowing a decoder to reconstruct the filters themselves.
The reconstruction of filters includes the reconstruction of filter coefficients of the filters.
As will be described in more detail below, the filter unit 349 can use differential encoding techniques to signal the filter coefficients.
To use the differential encoding technique, filter unit 349 determines an order in which to signal sets of filter coefficients.
As part of order determination, filter unit 349 determines a combination ID (ID) that represents a sequential value for each combination of
'58/104 combinations can be assigned combination IDs that represent sequential values in an order from left to right and from top to bottom, in which case combination 401 would be assigned to the first sequential value, combination 402 would be assigned to the second sequential value, and the remaining combinations would be assigned to the sequence values in the order of 403 => 404 => 405 => 406 => 407 => 408 => 409 => 410 => 411 => 412 => 412 => 414 = > 415 => 416. Filter unit 349 can also assign combination IDs using the top-down and zigzag order where combinations would be assigned combination IDs with sequential values that are in an order of 401 => 402 => 403 => 404 => 408 => 407 => 406 => 405 => 409 => 410 => 411 => 412 => 416 => 415 => 414 => 413. The filter unit 349 can also to assign . combination IDs using a top-down and left-to-right order where combinations' receive combination IDs with sequential values that are in an order of 401 => 405 => 409 => 413 => 402 => 406 => 410 => 414 => 403 => 407 => 411 => 415 => 404 => 408 => 412 => 416. The 349 filter unit can also use a left-to-right and zigzag order where combinations - “receive combination IDs with sequential values in an order of 401 => 405 => 409 => 413 => 414 => 410 => 406 => 402 => 403 => 407 => 411 => 415 = > 416 => 412 => 408 => 404. As you can imagine, these are just some of the many orders that can be used. In addition, any of the orders described can be lower or higher or higher to lower.
After the filter unit 349 has determined the mapping of the filters for the range combinations, the filter unit 349 can identify the groups of md AE AA dh Sides mA REA meessdas mava The maAsmA filters
'59/104 Using figure 4A as an example, the groupings would be as follows: Filter Group 1: combinations 413, 414 and 415 Filter Group 2: combinations 409, 410 Filter Group 3: combinations 411, 412 Filter Group 4: 416 combination Filter Group 5: 401 and 405 combinations Filter Group 6: 402 and 406 combinations Filter Group 7: 403 and 407 combinations Filter Group 8: 404 and 408 combinations. The 349 filter unit can then assign a group ID to each group, and the group ID can represent a sequential value.
Group IDs can be assigned to groups based on the sequential values associated with the combinations that comprise the group.
Per . For example, the group that has the combination with the lowest associated sequential value based on the combination IDs can 'assign the group ID with the lowest sequential value.
Of the remaining groups, the remaining group that has the combination with the lowest associated sequential value can receive the group ID with the next lowest sequential value.
This process can be repeated until all groups have received a group ID.
In some implementations group IDs can be assigned based on combinations with the highest associated sequential values instead of the lowest.
In some implementations, the group that has the combination with the lowest associated sequential value based on the combination IDs, can receive the group ID with the highest sequential value, or vice versa.
Again, using figure 4A as an example, and assuming that combinations 401-416 receive combination IDs with sequential values in an order from AI MAE = alive A ALA ms Im ai MAS Lea ldoa A m
. 60/104 filter unit 349 can assign group IDs to the filter groups, as illustrated below in Table 1. Group name Combinations | Combination with | group ID in the lowest sequence value group Filter Group 1 413, 414, 415 / aã3ã - q “Filter Group 2 109, 410 oo WB Filter Group 3 | 411, 412 jar Filero Group 4 [416 ane fe Filter Group 5 401, 405 Filter Group 6 402, 406 Filter Group 7 (403, 407 Filter Group 8 | 404, 408 oa In the example in Figure 4A, shown in Table 1, filter unit 349 is assigned Filter Group 5 has the group ID with the lowest sequential value since Filter Group 5 includes the range combination with the lowest sequential value (ie, the 401 combination). Filter unit 349 assigns the Filter Group 6 the group ID with O 'second lowest sequential value since, of the remaining 10' filter groups (that is, all groups excluding Filter Group 5), Filter Group 6 includes the range combination with the second lowest sequential value (ie combination 402). Filter unit 349 assigns Filter Group 7 the group ID with the third sequence value l lower since, of the remaining filter groups (that is, all filter groups excluding Filter Group 5 and Filter Group 6), Filter Group 7 includes the range combination with the lowest sequential value ( that's 403). Filter unit 349 assigns Filter Group 8 the group ID with the fourth lowest sequential value since, of the remaining filter groups (that is, all filter groups excluding Filter Groups 1 5, 6 and 7) , Filter Group 8 includes the range combination with the fourth lowest sequential value (combination 404). Filter unit 349 assigns Filter Group 2 the ID
'61/104 group with the lowest fifth sequential value since, of the remaining filter groups (that is, excluding Filter Groups 5, 6, 7, and 8), Filter Group 2 includes the range combination with the lowest sequential value (combination 409). Filter unit 349 assigns Filter Group 3 the group ID with the lowest sixth sequential value since, of the remaining filter groups (that is, excluding Filter Groups 5, 6, 7, 8 and 2), Filter Group 3 includes the range combination with the lowest sequential value (combination 411). Filter unit 349 assigns Filter Group 1 the group ID with the lowest seventh sequential value since, of the remaining filter groups (that is, excluding Filter Groups 5, 6, 7, 8, 2 and 3 ), Filter Group 1 includes the range combination with the lowest sequential value (combination 413). Finally, - filter unit 349 assigns Filter group 4, the final remaining filter group, the group ID with the highest
'sequential value (8, in this particular example). Based on the filter group IDs, filter unit 349 determines an order in which to signal a filter's filter coefficients.
Again, using the example in figure 4A and Table 1, filter unit 349 first signals The coefficient for Filter 5, then the coefficient for Filter 6, then the coefficient for Filter 7, then the coefficient for Filter 8, then the coefficient for Filter 2, then the coefficient for Filter 3, then the coefficient for Filter 1, and finally the coefficient for Filter 4. Using differential coding techniques, as described in this description, the unit of filter 349 can encode the coefficients for Filter 6 as difference information in relation to the filter coefficients of Filter 5, encode the AS finiantaco n to RWilbtro 7 when there is no difference
'62/104 with respect to the filter coefficients for Filter 6, and so on, based on the sequential ordering of group IDs.
The mapping of two or more metrics to entries in the filters can be implemented in multiple ways.
For example, in some implementations each entry may have a single set of filters, while in some implementations the entries share a common set of filters.
Additionally, in some implementations, two or more metrics for each entry can be used to identify a particular filter for each entry.
In other implementations, however, two or more metrics for a single entry can be used to identify the filters for all entries.
In other implementations, two or more metrics. for a first entry can be used to identify a filter for a different second entry. According to that description, filter unit 349 can perform encoding techniques with respect to filter information which can reduce the amount of data needed to encode and carry filter information from encoder 350 to another device.
Again, for each frame or slice, filter unit 340 can define or select one or more sets of filter coefficients to be applied to the CPU pixels for that frame or slice.
The filter unit 349 applies the filter coefficients in order to filter the video blocks of reconstructed video frames stored in memory 334, which can be used for prediction encoding consistent with loop filtering.
The filter unit 349 can encode the filter coefficients as filter information is sent to the filter unit
'63/104 entropy encoding 346 for inclusion in the encoded bit stream.
In addition, the techniques of this description can exploit the fact that some of the filter coefficients defined or selected by FSU 353 may be very similar to other filter coefficients applied with respect to the CPU pixels of another frame or slice. The same type of filter can be applied to different frames or slices (for example, the same filter support), but the filters can be different in terms of the first coefficient values associated with different filter support indices. Accordingly, in order to reduce the amount of data needed to carry such filter coefficients, filter unit 349 can by prediction code one or more filter coefficients to be used for. filtering based on the filter coefficients of another CU, potentially exploring 'similarities between the' filter coefficients. In some cases, however, it may be more desirable to encode the filter coefficients directly, for example, without using any prediction Various techniques, such as techniques that exploit the use of an activity metric to define when to code filter coefficients using prediction coding techniques and when to code filter coefficients directly without any prediction coding, can be used to communicate filter coefficients for a decoder efficiently. In addition, symmetry can also be imposed so that a subset of coefficients (for example, 5, -2, 10) known from the decoder can be used to define the total set of coefficients ( for example, 5, -2, 10, 10, -2, 5). Symmetry can be imposed on both
'64/104
As described above, video encoder 350 represents an example of a video encoder configured to determine a first metric for a group of pixels within a pixel block, determine a second metric for a group of pixels, determine a filter based on on the first metric and the second metric, and generate an image filtered by applying the filter to the pixel group.
Video encoder 350 also represents an example of a video encoder configured to determine a first metric for a block of pixels, where the first metric is determined based on a comparison of a subset of pixels in the block to other pixels in the block; determine a second metric for the The pixel block; determine a filter based on the first metric and the second metric; and generate a filtered image
. by applying the filter to the pixel block.
As described above, the 350 "video encoder also represents an example of a video encoder configured to determine a range mapping for filters, where a range combination comprises a range for a first metric and a range for a second metric, where each range combination has a unique range combination ID (ID), where each unique range combination ID corresponds to a sequential value for a range combination; assigning unique group IDs to groups of range combinations based on the sequential values for the range combinations, where each unique group ID corresponds to a sequential value for a group, and to encode the sets of filter coefficients corresponding to the filters based on the unique group IDs.
The video encoder 350 can encode the filter coefficient sets by
'65/104 bit stream encoded in an order that is selected based on the sequential values of the unique group IDs. The video encoder 350 can signal sets of filter coefficients using differential encoding techniques.
As described above, video encoder 350 also represents an example of a video encoder configured to determine a range mapping for filters, where a range combination comprises a range of values for a first metric and a range of values for a second metric; generate a first code word if a current range combination is mapped to the same filter as a previous range combination that comprises the same range of values for the first metric; generate a second code word if a combination. current range is mapped to the same filter as a previous range combination that comprises the same range of values for the second metric and generate a third code word if the current range combination is mapped to a different filter than the previous range comprising the same range of values for the first metric and the previous combination range comprising the same range of values for the second metric. The video encoder 350 also represents an example of a video encoder configured to determine a range mapping combination for filters, where a range combination comprises a range for a first metric and a range for a second metric; generate a first code word if a current range combination is mapped to the same filter as a previous range combination, and generate a second code word if the current beech combination is mapped to a filter
'66/104 second code word identifies a filter mapped to the current range combination.
Figure 5 is a block diagram illustrating an example of a video decoder 560, which decodes a video sequence that is encoded in the manner described here.
The received video sequence may comprise a coded set of image frames, a set of frame slices, a common form-encoded group of images (GOPs), or a wide variety of types of video block series including the blocks encoded video and syntax to define how to decode such video blocks.
The video decoder 560 includes an entropy decoding unit 552, which performs the alternate decoding function of the encoding performed by. Entropy encoding unit 346 of figure 3. In particular, the entropy decoding unit 552 "can perform CAVLC or CABAC decoding or any other type of entropy decoding used by video encoder 350. Entropy decoded video blocks in a one-dimensional serialized format they can be scanned in reverse to convert one or more one-dimensional vectors of the coefficients back into a two-dimensional block format.
The number and size of vectors, in addition to the scan order defined for the video blocks, can define how the two-dimensional block is reconstructed.
Entropy decoded prediction syntax can be sent from entropy decoding unit 552 to prediction module 554, and entropy decoded filter information can be sent from entropy decoding unit 552 to the entropy unit. filter 559.
: 67/104 The 560 video decoder also includes a 554 prediction module, 556 reverse quantization unit, a 558 reverse transform unit, a memory and a 564 adder. In addition, the 560 video decoder also includes an unlock filter 557 which filters the output of the adder 564. Consistent with this description, the filter unit 559 can receive entropy-decoded filter information that includes one or more filters to be applied to one or more inputs. Although not shown in figure 5, the unlocking filter 557 can also receive entropy-decoded filter information that includes one or more filters to be applied.
The filters applied by the 559 filter unit can be defined by the 15) filter coefficient sets. The 559 filter unit can be configured for. generate the sets of filter coefficients based on the filter information received from the entropy unit 'by entropy 552. The filter information may include the filter description syntax that identifies a maximum number of filters in a filter set and / or a filter format in a filter set, for example, The filter description syntax can be included in a header in a series of video blocks, for example, an LCU header, a frame header, a slice header, a GOP header, a sequence header, or similar. In other examples, the filter description syntax can be included in a footer or other data structure. Based on the filter description syntax, the filter unit 559 can reconstruct the set of filters used in the encoder. The filter information can also include additional signaling syntax that signals O
"68/104, any given set of coefficients. In some implementations, the filter information can, for example, also include ranges for two or more metrics for which any given set of coefficients should be used. After filter decoding, the filter unit 559 can filter the pixel values of the decoded video blocks based on one or more sets of filter coefficients and the signaling syntax that includes the ranges for which the different sets of filter coefficients are to be used.
The filter unit 559 can receive in the bit stream one or more elements of syntax indicating a set of filters for each frame or slice in addition to a filter mapping for the two or more metrics. For example, if an encoder uses Oo track mapping for. metrics for the filters illustrated in Figure 4A, then the encoder will signal this mapping or transmit the data to allow filter unit 559 to reconstruct that mapping. Regardless of whether or not this mapping is explicitly flagged, the filter unit 559 can maintain the same filter mapping for range combinations as used by the encoder.
As mentioned above, the filter unit 559 generates a mapping based on the filter information | 25 signaled in the bit stream. Based on this mapping, filter unit 559 can determine groups and assign group IDs to groups in the same way as described above with respect to filter unit 349. Using these group IDs, filter unit 559 can associate the filter coefficients received with ...
For each CU within the frame or slice, the filter unit 559 can calculate one or more associated metrics AND AA NH to Aa sounds.
'69/104 entries (ie, PI, EI, pRI, and RI) in order to determine which filters in the sets apply to each entry.
Alternatively, filter unit 559 can calculate one or more metrics for a single entry, such as pRI or RI.
The filter unit 559 determines which filter to apply based on the metrics determined for a particular pixel or group of pixels.
Using a modified sum and direction Laplacian value as examples for Metric 1 and Metric 2 and using the mappings illustrated in figure 4A as an example, if filter unit 559 determines that a pixel or group of pixels has a Laplacian value of modified sum in Range 1-2 and a direction corresponding to Range 2-3, then filter unit 559 can apply Filter 2 to the pixel or group of pixels.
If the filter unit 559 determines that a pixel. or group of pixels has a modified sum Laplacian value in Range 1-4 and a direction corresponding to: Range 2-2, then filter unit 559 can apply Filter 6 to that pixel or group of pixels, and so on .
The filter can generally assume any type of format or arrangement of filter holder.
The filter holder refers to the shape of the filter with respect to a particular pixel being filtered, and the filter coefficients can define weights applied to neighboring pixel values according to the filter holder.
According to the techniques of this description, syntax data can be included in the bit stream to signal to the decoder how filters were encoded (for example, how filter coefficients were encoded), in addition to activity metric ranges for which the different filters should be used.
For each CU within the frame or slice, the unit is 70/104 with the pixels decoded from a CU for multiple inputs (ie, PI, EI, pRI, and RI) in order to determine which filters in the sets apply to which input.
Alternatively, filter unit 559 can calculate one or more metrics for a single entry, such as pRI or RI.
The filter unit 559 determines which filter to apply based on the metrics determined for a particular pixel or group of pixels.
Using a modified sum and direction Laplacian value as examples of Metric 1 and Metric 2 and using the mappings illustrated in figure 4A as an example, if filter unit 559 determines that a pixel or group of pixels has a value Sum-modified Laplacian in Range 1-2 and a direction corresponding to Range 2-3, then filter unit 559 can apply Filter 2 to that pixel or group. of pixels.
If filter unit 559 determines that a pixel or group of pixels has a Laplacian sum: modified value in Range 1-4 and a direction corresponding to Range 2-2, then filter unit 559 can apply Filter 6 to that pixel or group of pixels, and so on.
The filter can generally assume any type of format or arrangement of filter holder.
The filter holder refers to the shape of the filter with respect to a given pixel being filtered, and the filter coefficients can define the weight applied to neighboring pixel values according to the filter holder.
According to the techniques of the present description, syntax data can be included in the bit stream to signal to the decoder how the filters were encoded (for example, how the filter coefficients were encoded), in addition to the activity metric ranges for which different filters should be used.
'71/104 The precision module 554 receives the prediction syntax (such as motion vectors) from the entropy decoding unit 552. Using the prediction syntax, the prediction module 554 generates the prediction blocks that were used to encode the video blocks.
The reverse quantization unit 556 performs the reverse quantization, the reverse transform unit 558 performs the reverse transforms to change the coefficients of the residual video blocks back to the pixel domain.
The adder 564 combines each prediction block with the output of the corresponding residual block by the reverse transform unit 558 in order to reconstruct the video block.
The filter unit 559 generates the filter coefficients to be applied to each input of a CU, and then. applies such filter coefficients in order to filter the reconstructed video blocks of that CU.
Filtering, for example, may comprise additional unlock filtering that smooths the edges and / or eliminates artifacts associated with the video blocks, noise removal filtering to reduce quantization noise, or any other type of filtering that can improve encoding quality.
The filtered video blocks are accumulated in memory 562 in order to reconstruct the decoded frames (or other decodable units) of the video information.
Decoded units can be sent from the 560 video decoder for presentation to a user, but can also be stored for use in subsequent prediction decoding.
In the video coding field, it is common to apply filtering in the encoder and decoder in order to
'72/104 Filtration can be applied through a post-filter, in which case the filtered frame is not used to predict future frames. Alternatively, filtering can be applied "in a loop", in which case the filtered frame can be used to predict future frames. A desirable filter can be designed by minimizing the error between the original signal and the decoded filtered signal. Typically, such filtering has been based on applying one or more filters to a reconstructed image. For example, the unlock filter can be applied to a reconstructed image before the image is stored in memory, or an unlock filter and an additional filter can be applied to a reconstructed image before the image is stored in memory.
In a similar way to the quantization of - transform coefficients, the filter coefficients h (k, 1) where k = K, ..., K, and 1 = L, ... L can be quantized 'as well. K and L can represent integer values. The filter coefficients (k, 1) can be quantized as: f (k, 1) = round) normFact - h (k, 1)) where normFact is a normalization factor and round is the rounding operation performed to achieve the quantization to a desired bit depth. The quantization of the filter coefficients can be performed by the filter unit 349 of figure 3 during coding and the decanting or inverse quantization can be performed on the filter coefficients decoded by the filter unit 559 of figure 5. The filter h (k, 1 ) should generally represent any filter. For example, the filter h (k, l1) can be applied to any of the multiple inputs. In some cases the multiple inputs associated with a video block will use different
"73/104 filters, in which case multiple filters similar to h (k, 1l) | can be quantized and de-quantized as described above.
The quantized filter coefficients are encoded and sent from the source device associated with encoder 350 to a destination device associated with decoder 560 as part of an encoded bit stream. In the example above, the value of normFact is usually equal to 2n although other values can be used. Larger normFact values result in more accurate quantization so that the quantized filter coefficients f (K, 1) provide better performance. However, higher normFact values can produce coefficients f (k, l1) that require more bits to signal to the decoder.
In the decoder 560 the filter coefficients. decoded f (Kk, / 1) can be applied to the appropriate input. For example, if the decoded filter coefficients' are applied to RI, the filter coefficients can be applied to the reconstructed post-unlocked image RI (i, j) where i = 0, ... M ej = 0, ... , N as follows:
KLKL RIGD => LSADRIGHA ADS D7 / (kD k = -Kil = -L ka Kia The variables M, N, K and L can represent integers. K and L can define a block of pixels that covers two dimensions from -K to K and from -L to L. Filters applied to other inputs can be applied in a similar way.
The techniques in this description can improve the performance of a post-filter or filter in loop, and can also reduce the number of bits required to signal filter coefficients f (k, 1). In some cases, a number DA TO AAA O AAA TO NAZI. IICA AL
- 74/104 for the decoder for each video block series, for example, for each frame, slice, part of a frame, group of frames (GOP), or similar.
For each filter, additional information is included in the bit stream to identify the CUs, macroblocks and / or pixels to which a particular filter must be applied.
Frames can be identified by frame number and / or frame type (for example, frames I, frames P or frames B). Tables I refer to intra-frames that suffer intrapredition.
P frames refer to prediction frames that have predicted video blocks based on a data list (for example, a previous frame). Tables B refer to bidirectional prediction tables that are predicted based on two data lists (for example, a previous or subsequent table). The macroblocks. can be identified by listing the types of macroblocks and / or range of parameter values of. quantization (QP) used to reconstruct the macroblock.
The filter coefficients f (k, l1) for any input can be coded using the precision of coefficients signaled for the previous CUs.
For each entry of a CU m (for example, each frame, slice or GOP), The encoder can encode and transmit a set of filters M: gi, where i = 0, ..., M-1 For each filter, the bitstream should also be encoded to identify the combination of ranges for two or more metrics for which the filter is to be used.
The filter coefficients can be predicted at 30 using the reconstructed filter coefficients; used in a previous CU.
The filter coefficients at AA
'75/104 f "; where i = O, ..., N-1l In this case, the CU number n can be used to identify one or more filters used to predict the current filters, and the number n can be sent to the decoder as part of the encoded bit stream In addition, the information - can be encoded and transmitted to the decoder to identify the range combinations for the two or more metrics for which the prediction encoding is used.
The amplitude of the filter coefficients g (k, 1) depends on values k and l. Normally, the coefficient with the greatest amplitude is oO coefficient g (0.0). The other coefficients that must have large amplitudes are the coefficients for which the value of k or 1 is equal to O.
. 15 This phenomenon can be used to further reduce the number of bits needed to signal the 'coefficients. The index values k and 1 can define locations within a known filter holder. The coefficients: gi (Kk, 1), i = 0, ..., M-1 for each frame m can be coded using parameterized variable length codes such as Golomb and exp-Golomb codes defined according to a parameter P. By changing the value of the parameter p that defines the parameterized variable length codes, these codes can be used to efficiently represent the wide range of font distributions. The distribution of coefficients g (k, 1) (that is, your probability of having large or small values) depends on the values of k and 1l. Thus, to increase coding efficiency, to
] 76/104 each pair (K, l). The parameter p can be used to encode parameterized variable length when coding coefficients: JA (Kk, 1) where k = -K, ..., K, 1 = -L, ..., L As described above, the video decoder 560 represents an example of a video decoder configured to determine a first metric for a group of pixels within a pixel block, determine a second metric for the group of pixels, determine a filter based on the first metric, and second metric, and generate a filtered image by applying the filter to the pixel group. The video decoder 560 also represents an example of a video encoder configured to determine a first metric for a block of pixels, where the first metric is determined with 15 based on a comparison of a subset of pixels in the block to other pixels in the block; determine a second metric 'for the pixel block; determine a filter based on the first metric and second metric; and generate a filtered image by applying the filter to the pixel block.
As described above, video decoder 560 also represents an example of a video decoder configured to determine a mapping of beech combinations to filters, where a range combination comprises a range for a first metric and a range for a second metric , where each range combination has a unique range combination identification (TD), where each unique range combination ID corresponds to a sequential value for a range combination; assign unique group IDs to groups of range combinations based on sequential values for range combinations, where each unique group ID corresponds to a value
"77/104 filter coefficients corresponding to filters based on unique group IDs. The 560 video decoder can encode the filter and coefficient sets by generating and filter coefficient sets based on the information received in an encoded bit stream The 560 video decoder can generate sets of filter coefficients using differential techniques and encoding.
The video decoder 560 also represents an example of a video decoder configured to map a first range combination to a first filter, where the first range combination comprises a first range of values for a first metric and a first range of values for a second metric; map a second range combination to a second filter, where the second range combination comprises a second range of values for the first metric and a second range of values for the second metric; map a current range combination to a filter, where the current range combination comprises the first range of values for the first metric and a second range of values for the second metric. Mapping the current range combination to The filter may include mapping the current range combination to the first filter in response to receiving a first code word, where the first code word indicates that the current range combination has been mapped to The same filter than the first range combination; The mapping of the current range combination to the second filter in response to receiving a second code word, where the second code word indicates that the current range combination is mapped to the same filter as the second combination; and, the mapping of the combination of atnhal nara nm tarceiro filter reclining to the
. 78/104 receiving a third code word, where the third code word identifies that third filter.
The video decoder 560 also represents an example of a video decoder configured to generate a range mapping for filters, where a range combination comprises a range for a first metric and a range for a second metric; maps a current range combination to the same filter as a previous range combination in response to receiving a first code word signaling that the current range combination is mapped to the same filter as the previous range combination; and, maps the current range combination to a filter identified by a second code word in response to receiving the second code word signaling that the current range combination is mapped to. a different filter than the previous track combination.
As was introduced above, several types: different metrics can be used in conjunction with the multiple metric filtering techniques described in that description.
Some of these metrics are activity metrics that quantify the activity associated with one or more blocks of pixels within the video data.
Activity metrics can comprise variation metrics indicative of pixel variation within a set of pixels.
As will be described, some of these activity metrics are direction specific.
For example, the horizontal activity metric quantifies the activity along a horizontal axis, a vertical activity metric quantifies the activity along a vertical axis, a diagonal activity metric quantifies the activity along a diagonal axis, and so onwards.
Í 79/104 Some activity metrics are not direction specific.
For example, a modified sum Laplacian value is an activity metric based on a window of two-dimensional pixels surrounding a current pixel or group of current pixels.
For a current pixel (i, j), a modified sum Laplacian value can be calculated as follows: var (i, => SPRGEJ + D = RGE j + D = RGE jello k = -K1l = -L | 2RG + k, j + D— = RG + k, j + l-1) - RGE + k, j + l + D) O where k represents a value of a sum of pixel values from -K to K and 1 represents a value of a sum of -L to L for a two-dimensional window spanning from - KakKe-LaL, where i and 5j represent pixel coordinates of pixel data, RI (i, j) represents a pixel value determined in iej coordinates, and var (i, j) is the metric of. activity (that is, the modified sum Laplacian value). The techniques of the present description can also be implemented using specific direction metrics for horizontal activity, vertical activity, and diagonal activity.
Equations 2 and 3 illustrate examples of how horizontal activity and vertical activity can be computed for a current pixel (x, y) by comparing a pixel value (Rec), such as intensity, from the current pixel to a value of pixel from neighboring pixels.
Hor act (x, y) = R (2 * RecIx] [y] -Rec [x + 1] [y] -Rec [x-1] [y]) O See act (x, y) = R (2 "Rec [x] [y] -Rec [x] [y + 1] -Rec [x] [y + 1]) 6) As illustrated by equation 2, when determining horizontal activity, the current pixel (x, y ) can be compared with a left neighbor (xl, y) and a right neighbor (x + l1, y). As illustrated by equation 3, when determining vertical activity, the current pixel can be
'80/104 compared to an upper neighbor (x, y + 1) and a lower neighbor (x, y-1l).
Equations 4 and 5 illustrate examples of how diagonal activity can be computed for a current pixel (X *, y) by comparing a pixel value (Rec) of the current pixel with the pixel values of neighboring pixels. 45deg act (x, y) = RO * Rec [x] [y] -Rec [x + 1] [y + 1] -Rec [x-1] [y-1])) "(M) 135deg act ( x, y) = R (2 * Rec [x] [y] -Rec [x-1] [y + 1] -Rec [x + 11 [y-1)) (5) As illustrated by equation 4, the diagonal activity can be computed, for example, in the 45 degree direction by comparing a current pixel (x, y) with an upper right neighbor (x + 1, y + 1) and a lower left neighbor (xl, yl). As illustrated by equation 5, diagonal activity can also be in the 135 degree direction by comparing a current pixel (x + y) with an upper left neighbor (xl, y + 1) and a lower right neighbor (x + 1 , y-1).
Equations 2 to 5 above illustrate how horizontal activity, vertical activity and diagonal activity can be determined pixel-by-pixel, but in some implementations, horizontal activity, vertical activity and diagonal activity can be determined group-a -group, where a group of pixels is a block of pixels 2x2, 4x4, or NXxN. In such an implementation, horizontal activity, for example, can be determined by comparing pixel values of a current group with pixel values of a left group and a right group, in a manner similar to equation 2; and, the vertical activity can be determined by comparing a current group with an upper group and a lower group, similar to equation 3. Likewise, the 45 degree diagonal activity can be determined by comparing a current group of
'81/104 lower left neighbor in a similar way to equation 4, and diagonal activity of 135 degrees can be determined by comparing a current group of pixels with an upper left neighbor group and a lower right neighbor group in a similar way to equation 5. In some implementations, horizontal activity, vertical activity, 45 degree diagonal activity and 135 degree diagonal activity can be determined by comparing a current pixel or group of pixels with neighboring pixels or groups of pixels in only one direction .
For example, instead of determining horizontal activity based on a comparison of a current pixel with a left neighbor and a right neighbor, horizontal activity can be determined based on just one left neighbor or only one right neighbor. . Additionally, in some implementations, horizontal activity, vertical activity, diagonal activity of: 45 degrees, and diagonal activity of 135 degrees can be determined using weighted averages or averages of neighboring pixel areas instead of single neighboring pixels or groups of unique pixels.
The values resulting from equations 2 to 5 can be divided into a finite number of ranges, such as 2, 4, 8 or any other finite number, and each range can be assigned a range identification.
Referring again to Figure 4A, for example, Range 1-1, Range 1-2, Range 2-1, etc. they are all examples of range identification.
As an example, the horizontal activity values can be divided into four ranges, and the ranges can be assigned Range 1-1, Range 1-2, Range 1-3, and Range 1-4 IDs, horizontal limit values (this is, ThH1, ..., ThHpy 1) can determine where the tracks start and
Ú 82/104 finish. Table 2 below illustrates the generic case of how horizontal IDs can be assigned to ranges P. Table 2: Activity metric index to A |
EEE Using the example in Table 2, if a current pixel has a horizontal activity value greater than ThHi, but less than ThH ,, then the current pixel is in the 2-2 range for metric 2. The current pixels can be assigned for vertical bands with Vertical IDs, 45 degree diagonal bands with 45 degree diagonal IDs, and 135 degree diagonal bands with 135 degree diagonal IDs, in a similar way as described above in Table 2 for horizontal bands and IDs horizontal. 'Any horizontal activity, activity.: Vertical, 45 degree diagonal activity, and 135 degree diagonal activity can be used as a metric according to the multi-metric filter filtering techniques described in this description. For example, with reference again to figure 4A, metric 1 can be a measure of vertical activity, and metric 2 can be a measure of horizontal activity. In such an example, a filter unit, such as filter unit 349 in figure 4A or filter 559 in figure 5, can determine a filter for a pixel or group of pixels based on the horizontal activity of the pixel or group of pixels and the vertical activity of the pixel or group of pixels. If, for example, a current pixel has a horizontal activity metric in Range 2-3 and a vertical activity metric in Range 1-3, then the filter unit filters the pixel using Filter 4. In
'83/104 a similar shape, the combinations of diagonal activity of 45 degrees and diagonal activity of 135 degrees, diagonal activity of 45 degrees and horizontal activity, diagonal activity of 45 degrees and vertical activity, diagonal activity of 135 degrees and horizontal activity , or 135 degree diagonal activity and vertical activity can also be used by a filter unit to select a filter for a pixel or group of pixels. In some implementations, three or all four horizontal, vertical, 45 degree diagonal and 135 degree diagonal activities can be used by a filter unit to select a one-pixel filter or group of pixels.
In the implementations described above, horizontal activity, vertical activity, 45 degree diagonal activity, and 135 degree diagonal activity can all. be used as metrics, such as metric 1 and / or metric 2 in figure 4A, for example. In some modalities, however, horizontal activity, vertical activity, 45 degree diagonal activity, and 135 degree diagonal activity may not be metrics per se, but can instead be used as intermediate determinations for determining a general direction metric. The direction metric generally describes in which direction (for example, no direction, horizontal, vertical, 45 degree diagonal, or 135 degree diagonal) the pixels are changing most.
In one example, using only horizontal activity and vertical activity as described in equations 2 and 3, a direction for a pixel can be determined based on the following equations: Direction 1 = horizontal, if Hor activity> kl * See activity Direction 2 = vertical, if See activity> k2 * Hor activity Nniranãio NO = none
"84/104 The constants kl and k2 can be selected so that the direction is considered only direction 1 or direction 2 if the horizontal activity is substantially greater than the vertical activity or the vertical activity is substantially greater than the activity If the horizontal activity and the vertical activity are the same or approximately the same, then the direction is direction 0. Direction 1 generally indicates that the pixel values are changing more in the horizontal direction than in the vertical direction, and direction 2 indicates that pixel values are changing more in the vertical direction than in the horizontal direction. Direction O indicates that the change in pixel values in the horizontal direction is approximately equal to the change in pixel values in the vertical direction.
The given direction metric (for example, direction 0, direction 1, direction 2) can be used as a metric in the multiple metric filtering techniques' described in this description. Using the example in figure 4, again, metric 1 can be a variation metric, such as the modified sound Laplacian value, while metric 2 can be a direction of determination as described above. As described with reference to figure 4A, each of the directions 1, 2 and O can be associated with a metric range 2 although the direction 1, 2 and O represent finite determinations instead of a spectrum of values.
In addition to using only horizontal activity and vertical activity as described above, the techniques in that description also include the use of 45 degree diagonal activity and 135 degree diagonal activity, as described in equations 4 and 5, to determine the directions, based on the following conditions: THieaAaSãXA = 1 who find smA: Piúioidto Ss V1 4 196LADAS saAqbPricido
'85/104 Direction = 2, if 135deg activity> k2 * 45deg activity Direction = O otherwise.
Direction determinations based on 45 degree diagonal activity and 135 degree diagonal activity can be used as a metric with another metric, such as a modified sum Laplacian value, as described above.
In addition, a direction metric can also be determined, based on the following conditions: Direction = 1, if 45deg activity> kl * 135deg activity, k2 * Hor activity, E k3 * See activity Direction = 2, if 135deg activity >> k4 * d5deg activity, k5 * Hor activity, E k6 * See activity Direction = 3 if Hor activity> K7 * See activity, k8 * 135deg activity, E k9 * 45deg activity. Direction = 4 if See activity> kl10 * Hor activity, kl1 * 135deg activity, and Kl2 * 45deg activity] Direction = O, otherwise.
As described above, kl to kl2 are constants selected to determine how much greater than one of horizontal activity, vertical activity, 45 degree activity and 135 degree activity needs to be compared with others in order for a given direction to be selected. Direction determination based on horizontal, vertical, 45 degree diagonal and 135 degree diagonal activity can be used as a metric with another metric, such as a modified sum Laplacian value, as described above.
Another metric that can be used with the techniques in this description includes an edge metric. An edge metric generally quantifies the activity that can be indicative of the presence of an edge in an Aa block.
'86/104 pixels if that pixel block contains the boundary of an object within an image.
An example of edge detection includes using four neighboring pixels of the current pixel (for example, left, right, top and bottom) or using the eight neighboring pixels of the current pixel (left, right, top, bottom, top right, top left, bottom right, bottom left). In addition, border type detection can include the use of two neighboring pixels, such as upper and lower, left and right, upper left and lower right, or upper right and lower left.
The pseudocode below illustrates examples of how edge information can be computed for a current pixel (x, y) by comparing a pixel value (Rec), such as. intensity, of the current pixel with the pixel values of those neighboring pixels (that is, 4/8 pixels). : An EdgeType variable is set to 0. Each time a statement is true, the EdgeType variable is incremented by 1 (as shown in the pseudocode by EdgeType ++) or decremented by 1 (as shown in the pseudocode by EdgeType -). Rec [x] [y] refers to a pixel value, such as the pixel intensity, of the pixel located at (x, y). The first grouping of "if" statements are for comparing the current pixel with the top, bottom, left and right neighbors.
The second grouping of "if" statements is to compare the current pixel with the upper left, upper right, lower left and lower right neighbors.
The techniques of this description can be implemented using any group or both groups.
And generates sm if (ReclvliIvl1l> Recíx = 11f1v1) FdageTvone +++
'87/104 if (Rec [x] [y] <Rec [x-1] [y]) EdgeType -; if (Rec [x] [y]> Rec [x + t1] [y]) EdgeType ++; if (Rec [x] [y] <Rec [x + 1] [y]) EdgeType -; if (Rec [x] [y]> Rec [x] [y-1]) EdgeType ++; if (Rec [x] [y] <Rec [x] [y-1]) EdgeType -; if (Rec [x] [y]> Rec [x] [y + 1]) EdgeType ++; if (Rec [x] [y] <Rec [x] [y + 1]) EdgeType -; = if (Rec [x] [y] <Rec [x-1] [y-1]) EdgeType -; if (Rec [x] [y]> Rec [x + 1] [y-1]) EdgeType ++; if (Rec [x] [y] <Rec [x + t1] [y-1]) EdgeType -; if (Rec [x] [y]> Rec [x-1] [y + 1]) EdgeType ++; if (Rec [x] [y] <Rec [x-1] [y + 1]) EdgeType -; if (Rec [x] [y]> Rec [x + 1] [y + 1]) EdgeType ++; if (Rec [x] [y] <Rec [x + 1] [y + 1]) EdgeType -; If a current pixel is a maximum location, then the pixel's pixel value will be greater than all of its. neighbors and will have a border type of 4 if using four neighbors or a border type of 8 is using. 5 eight neighbors.
If a current pixel is a local minimum, then the pixel value of the pixel will be less than all its neighbors and will have a border type of -4 if using four neighbors or a border type of -8 if using eight neighbors.
Thus, using the illustrative techniques described above to determine an edge type between -4 and 4 or -8 and 8, it can be used in determining a filter.
The values determined for the border type (that is, values from -4 to 4 or values from -8 to 8) can be mapped to ranges of a metric, such as Metric 1 or Metric 2 in figure 4A.
In some implementations, the absolute values of the edge type determination can be mapped to ranges, so that an edge type from -3 to 3, for example, map the same filter.
S 88/104 The calculations for the various metrics described in this description are intended as examples only and are not exhaustive.
For example, metrics can be determined using windows or lines of pixels that include more neighboring pixels than described in this description.
Additionally, in some implementations, the metrics described in this description can be calculated using the subsampling of pixels in a particular line or window.
For example, to calculate a block activity metric for a 4x4 pixel block, the metrics for activity and direction can be calculated as follows: * Direction metric eVer act (i, j) = abs (X (i, j) << l - X (i, j-l1) - X (i, j + 1)) eHor act (i, j) = abs (X (i, j) << l - X (il, j3) -. X (itl1,3)) * Hp = Xi = 0,2 Ej = oil, 2 Hor act (i, j). * Vg = Zi-o, 2 2j = 0.2 Vert act (i, j) eDirection = O, 1 (Hg> k1 * Vz), 2 (Vr> K2 * Hr) and Activity Metric eL; = Hg + Vz and 5 classes (0, 1, 2, 3, 4) and Metric and Combination of Activity and Direction (for example, 15 or 16 combinations as explained above in the example in Figure 4B). Hor act (1i, j) generally refers to the horizontal activity of the current pixel (i, j), and Vert act (i, j) generally refers to the vertical activity of the current pixel (i, j). X (i, j) generally refers to a pixel value of the pixel (i, j). Hs refers to the horizontal activity of the 4x4 block, which, in this example, is determined based on a sum of the horizontal activity for pixels (0.0), (0.2), (2.0) and (2.2) . V; It refers to the vertical activity of the 4x4 block, which, in this example, is determined based on a sum of the activity
89/104. . vertical of the pixels (0,0), (0,2), (2,0) and (2,2). "<< l" represents a multiplication operation by two.
As explained above, based on the values of H; and Vx, A direction can be determined.
Using the example above, if the value of H; is greater than k times the value of Vs, then the direction can be determined to be direction 1 (that is, horizontal), which may correspond to more horizontal activity than vertical activity.
If the value of V5 is greater than k times the value of Hi, then the direction can be determined to be direction 2 (that is, vertical) which can correspond to more vertical activity than horizontal activity.
Otherwise, the direction can be determined to be the O direction (that is, without direction), meaning that neither the vertical nor the horizontal direction is dominant.
The labels for the various. directions and the reasons used to determine the directions are merely an example, and other labels] and reasons can also be used.
Activity (Lg) for the 4x4 block can be determined as a sum of horizontal and vertical activity.
The value of Ly can be classified in a range, as described above.
This particular example illustrates five ranges although more or less ranges can be used in a similar way.
Based on the combination of activity and direction, a filter for the 4x4 pixel block can be selected.
As described above, a filter can be selected based on a two-dimensional mapping of filter activity and direction, as described with reference to figures 4A and 4B, or activity and direction can be combined into a single metric, and this single metric can be used to select a filter.
Figure 6A represents block of 4x4 pixels. AAA Ni
. 290/104 only four of the sixteen pixels are used.
The four pixels are pixels (0,0) which are labeled as pixel 601, pixel (2,0) which are labeled as pixel 602, pixel (0,2) which are labeled as pixel 603, and pixel (2,2) which is labeled as pixel 604. The horizontal activity of pixel 601 (that is, hor act (0.0)), for example, is determined based on a left neighbor pixel and a right neighbor pixel.
The right neighbor pixel is labeled with pixel 605. The left neighbor pixel is located in a different block than the 4x4 block and is not shown in figure 6A.
The vertical activity of pixel 602 (that is, see act (2.0)), for example, is determined based on an upper neighbor pixel and a lower neighbor pixel.
The lower neighbor pixel is labeled as pixel i 606, and the upper neighbor pixel is located in a different block than the 4x4 block and is not shown in figure 6A. . Generally using the same techniques described above, a block activity metric can also. be calculated using a different subset of pixels as follows: * Direction Metric eVer act (i, j) = abs (X (i, j) << l - X (i, j-1) - X (i, 3 + l)) * Hor act (i, j) = abs (X (i, j) << l - X (i-1, j3) - X (i + l, 3)) * Hz = Xic1,2 Xje1, 2 H (i, j) * VB = Xi-1,2 2je1,2 V (i,)) eDirection = O, 1 (H> kl1 * V), 2 (V> k2 * H) * Activity Metric * Lg = Hp; + Vg * e5 classes (0, 1, 2, 3, 4) and Metric eCombination of Activity and Direction (for example, 15 or 16 combinations as explained above in the example in Figure 4B). This different subset of pixels to calculate H; and V5 includes pixels (1.1), (2.1), (1.2) and (2.2)
91/104 shown in Figure 6B as pixels 611, 612, 613 and 614, respectively. As can be seen from figure 6B, all upper neighboring pixels, lower neighboring pixels, right neighboring pixels and left neighboring pixels for pixels 611, 612, 613, 614 are all located within the 4x4 block. In the example of figure 6B, pixels 611, 612, 613 and 614 are all located inside the block as opposed to the location at the block boundary. Pixels 601, 602, 603, and 605 in figure 6A and pixels 621, 624, 625 and 628 in figure 6C are examples of pixels located at the block boundary. In other implementations, additional different subsets of pixels can be chosen. For example, subsets can be selected so that the upper and lower neighboring pixels for the subset pixels are within. 4x4 block, but some left and right neighboring pixels are in neighboring blocks. The subsets can also be selected so that the left and right neighboring pixels for the subset pixels are within the 4x4 block, but some upper and lower neighboring pixels are in the neighboring blocks.
Generally using the same techniques as described above, a block activity metric can also be calculated using a subset of eight pixels as follows: * Direction Metric eVer act (i, j) = abs (X (i, j) << l1 - X (i, j-1) - X (i, j + l1)) eHor act (i, j) = abs (X (i, j) << 1 - X (i-1,3) - X (itl1,3)) Hp = Xico, 1,2,3 Ljei, 2 H (i, j) eVB = Xiso, 1, 2,3 Zje1,2 V (i, Í) eDirection = O, 1 ( H> k1 * Vv), 2 (V> k2 * H) and Activity Metric a and Aro LN O AO e5 classes (0, 1, 2, 3, 4) * Metric and Combination of Activity and Direction (for example, 15 or 16 combinations as explained above in the example in Figure 4B). This subset is different from eight pixels for calculating H; and V5 includes pixels (0.1), (1.1), (2.1), (3.1), (0.2), (1.2), (2.2) and (3.2 ) shown in Figure 6C as pixels 621, 622, 623, and 624, 625, 626, 627 and 628, respectively.
As can be seen from figure 6C, all upper neighboring pixels and lower neighboring pixels for pixels 621, 622, 623, and 624, 625, 626, 627 and 628 are located within the 4x4 block, despite the 621 pixels and 625 each have left neighboring pixels in a left neighboring block, and pixels 624 and 628 each have right neighboring pixels in a block. right neighbor.
This particular selection of pixels can reduce the complexity of the encoder and / or decoder. avoiding the need for a line store for the storage of pixel values of the upper and / or lower neighboring blocks.
Due to the scan order from left to right and from top to bottom, line stores for pixel values of upper and lower neighboring blocks often need to store pixel values for the entire top or bottom line, which in the case of 1080P video , for example, can be 1920 pixels.
Line stores for the left and right neighboring blocks, however, often only need to store the pixel values for an LCU or a pair of LCUs, which can be only 64 or 128 pixels, for example.
Thus, line stores for pixel values of upper and lower neighbor blocks may need to be significantly larger than Arma sAanadarão of the nilioadeoe line and nara ne level valares
. 93/104 of the left and right neighbor blocks. The pixel selection illustrated in figure 6C may also be able to avoid the use of line stores for upper and lower neighbor pixel values, thus reducing the encoding complexity.
The examples in figures 6A to 6C are merely techniques introduced from that description. It is contemplated that these techniques can be extended to blocks beyond just 4x4 and that different subsets of pixels can be selected.
When computing a block activity metric, instead of original pixels, quantized pixels (ie, X (i, j) >> N) can be used to reduce the complexity of operations, such as addition operations.
Additionally, the calculations can be the absolute difference. based on Laplacian. For example, when computing Hor act (i, j) or Ver act (i, j), absolute differences can be used instead of Laplacian values, as follows: * Directional Metric eVer act (i, j) = abs (X (i, j) - X (i, j-1)) eHor act (i, j) = abs (X (i, j) - X (i-1,) 3)) * Hp = Xi = 0.1.2 Zj = 0.1.2 H (i, j)) * VB = Xi-o, 1.2 2j-0.1.2 V (i, j) eDirection = 0, 1 (H> 2V), 2 (V> 2H) * eL Activity Metric; = Hg; + Vg +5 classes (0, 1, 2, 3, 4) and Metric eativity + Direction (for example, 15 or 16 combinations as explained above in the example in Figure 4B).
This description described subsampling techniques with reference to a limited set of specific metrics. It is contemplated, however, that these
. 94/104 other metrics, such as the other metrics discussed in that description, that can be used for the purposes of determining a filter. Additionally, although the subsampling techniques of that description have been described with reference to 4x4 pixel blocks, the techniques can also be applicable to blocks of other sizes.
Figure 7 is a flow chart illustrating a video encoding technique consistent with this description. The techniques described in figure 7 can be performed by the filter unit of a video encoder or a video decoder, such as filter unit 349 of video encoder 350 or filter unit 559 of video decoder 560. The unit filter determines a first metric for a group of pixels within one. block of pixels 710. The first metric can, for example, be an activity metric such as a modified sum Laplacian value 7, or the first metric can be a direction metric. The first metric can be determined, for example, based on a comparison of the set of pixels in the block, or based on a subset of pixels in the block, with other pixels in the block. The filter unit additionally determines a second metric for block 720. The second metric can, for example, be a direction metric that is determined based on the comparison of a horizontal activity measure with a vertical activity measure. Based on the first metric and the second metric, the filter unit determines a filter
730. The filter unit generates a filtered image by applying the filter to the 740 block. As discussed above, in some implementations, the block can be a 2x2, 4x4 or MXN pixel block, used to determine the first MADIAKS A 4 qaAMe SAS mÁáÁSViIiAS Dm ms Amas imeéelamaeantamãããà £ o -
'95/104 first metric can be a horizontal activity metric while the second metric is a vertical activity metric, or the first metric can be an edge metric while the second metric is a direction metric.
Figure 8A is a flow chart illustrating the video encoding techniques consistent with this description. The techniques described in figure 8A can be performed by the filter unit of a video decoder, such as the filter unit 559 of the video decoder 560. The filter unit 559 maps a first band combination with a first filter 810A. The first range combination is a combination of a first range of values for a first metric and a first range of values for a second metric. The first metric can,. «For example, be a modified sum Laplacian value and the second metric can be a direction metric, although 7 other metrics can also be used. Filter unit 559 maps a second range combination to a second 820A filter. The second range combination is a combination of a second range of values for the first metric and a second range of values for the second metric. The 559 filter unit then maps a current range combination to a filter based on a received code word. The current range combination includes the first range of values for the first metric and the second range of values for the second metric. If the code word is a first code word 830A, (yes), then the filter unit 559 maps the current range combination with the first filter 840A. The first code word indicates that the current range combination is mapped to the same filter as the first range combination. If the REAR A
'96/104 the 559 filter unit maps the current range combination to the second 860A filter.
The second code word indicates that the current range combination is mapped to the same filter as the second combination.
If the code word is neither a first code word nor a second code word 850A, (no), then filter unit 559 maps the current range combination to a third 870A filter.
If in response to receiving a third code word, where the third code word identifies that third filter.
In the example of figure 8A, the first code word and the second code word can each include fewer bits than the third code word.
Figure 8B is a flow chart illustrating video encoding techniques consistent with this description.
The techniques described in figure 8B can be. performed by the filter unit of a video decoder, such as the filter unit 559 of the 1 video decoder 560. The filter unit 559 generates a mapping of 810B filter range combinations.
Each range combination, for example, can include a range for a first metric and a range for a second metric.
In response to receiving the first code word that signals a current strip combination for the same filter as a previous strip combination 820B, (yes), filter unit 559 maps the current strip combination to the same filter as the combination previous track 830B.
In response to receiving a second code word that signals the current range combination is mapped to a different filter than the previous range combination 820B, (no), the filter unit 559 maps the current range combination to a new 840B filter.
As described above, the current range combination can be determined on the basis of NM E AND ED ATA Amo ASSES AS ADVANCES The new filter can be determined based on the order in which the filter coefficients are signaled.
Figure 9 is a flow chart illustrating video coding techniques consistent with this description.
The techniques described in figure 9A can be performed by the filter unit of a video encoder, such as the filter unit 349 of the video encoder 350. The filter unit 349 determines a mapping of track combinations to the 910A filters.
Each range combination includes a range of values for a first metric and a range of values for a second metric.
For a current range combination, if a current range combination is mapped to the same filter as a previous range combination that comprises the same range of values for the first 920A metric, (yes), then - the 349 filter unit generates a first code word 930A.
If the current range combination is mapped to the same filter as a previous range combination that comprises the same range of values for the second 940A metric, (yes), then filter unit 349 generates a second code word 950A.
If the current range combination is not mapped to the previous range combination that comprises the same range of values for the first metric Or the previous range combination that comprises the same range of values for the second 950A metric, (no), then filter unit 349 generates a third code word 960A.
The third code word can identify a filter mapped to the current range combination.
Figure 98 is a flowchart illustrating video encoding techniques consistent with this description.
The techniques described in figure 9BA can be performed by the filter unit of an Ns encoder
98/104 for video 350. Filter unit 349 determines a range mapping for 910B filters. Each range combination can, for example, include a range for a first metric and a range for a second metric. When a current range combination being coded has the same filter as a previously coded range combination 920B, (yes), filter unit 349 can generate a first code word to signal that the current range combination is mapped to The same filter that a combination of previous 930B range. When a current range combination being encoded does not have the same filter as a previously coded range combination 920B, (no), filter unit 349 can generate a second code word 940B. The second code word can identify the filter mapped to the combination. current track. As described above, the current track combination can be determined based on a known transmission order. In the example of figure 9B, the first code word may include fewer bits than a second code word.
In the examples in figures 8A and 8B and figures 9A and 9B, the terms "first code word", "second code word" and "third code word" are used to differentiate between different code words and do not mean in a sequential order of words code.
Figure 10 is a flow chart illustrating the video encoding techniques consistent with this description. The techniques described in figure 10 can be performed by the filter unit of a video encoder, such as a filter unit 349 of the video encoder 350, or the filter unit of a video decoder, such as the filter unit 559 The unit of
. 99/104 for 1010 filters. Range combinations include a range for a first metric and a range for a second metric. The filter unit determines a unique range combination ID (ID) for each 1020 range combination. Unique range combination IDs correspond to sequential values. The filter unit assigns a first unique group ID to a first group of range combinations based on the sequential value of a range combination ID of at least one range combination in the first group of 1030 range combinations. range combinations include range combinations mapped to the same filter, unique group IDs corresponding to a set of sequential values. The filter unit encodes a first set of filter coefficients corresponding to the same 'filter based on the sequential value of the first unique filter ID 1040. In the case of the video encoder, the' encoding of the first set of filter coefficients may include, for example, signaling filter coefficients in an encoded bit stream using differential encoding techniques. In the case of a video decoder, the encoding of the first set of filter coefficients may include the reconstruction of the filter coefficients based on the information received in an encoded bit stream.
Figure 11 is a flow chart illustrating video encoding techniques consistent with this description. The techniques described in figure 11 can be performed by the filter unit of a video encoder, such as the filter unit 349 of the video encoder 350, or the filter unit of a video decoder, such as the filter unit 559 The filter unit determines a maintenance of the nara filterne 111N
. 100/104 range combinations can include a range for a first metric and a range for a second metric. Each range combination can have a unique range combination ID (ID), and each unique range combination ID can correspond to a sequential value for the range combination. The filter unit can assign a unique group ID to each group of range combinations
1120. The filter unit can assign unique group IDs, for example, based on the sequential values of the range combinations. A group of range combinations can include range combinations mapped to a common filter, and unique group IDs can correspond to a set of sequential values. The filter unit can encode sets of filter coefficients for the filters based on the unique group IDs 1140.. In the example of figure 11, the filter unit may assign unique group IDs, for example, by assigning 'a unique group ID corresponding to a lower sequential value of the unique group IDs for a group of range combinations comprising a range combination with a range combination ID corresponding to a lower sequence value than the range combination IDs. In another example, the filter unit may assign the unique group ID corresponding to a higher sequential value of the unique group IDs for a range combination group that comprises a range combination with a range combination ID corresponding to a highest sequential value of the range combination IDs.
In cases where the filter unit is part of a video decoder, the filter unit can encode the filter coefficient sets by generating filter coefficient sets based on the info received in an encoded bit stream, at the
. 101/104 filter unit can, for example, generate sets of filter coefficients using differential coding techniques. In cases where the filter unit is part of a video encoder, the filter unit can encode the sets of filter coefficients by signaling filter sets and coefficients in a bit stream encoded in an order selected based on the values sequential numbers of unique group IDs. The filter unit can, for example, signal sets of filter coefficients using differential coding techniques.
The description above has been simplified to some extent in order to provide details. For example, the description generally describes sets of filters being flagged frame by frame, but filter sets can also be flagged. signaled sequence by sequence, image group by image group, band group by slice group, CU-a-: CU, LCU-a-LCU, and based on other things. In general, filters can be flagged for any grouping of one or more CUs. Additionally, in the implementation, there may be numerous filters per CU input, numerous coefficients per filter, and numerous different levels of variation with each of the filters being defined for a different range of variation. For example, in some cases there may be sixteen or more filters defined for each CU input and sixteen different ranges corresponding to each filter. In addition, when this description describes the transmission of filter information, it must be assumed that all filter information is transmitted at the same coding level. For example, in some implementations, some filter information such as filter description syntax can be flagged as A Reid Beside LfLadidsoo AmRNMHIANdhA Amd ryrsa din farmasãa
102/104 of filter as well as filter coefficients are signaled LCU-to-LCU. Syntax at other levels of the coding hierarchy, such as sequence level, GOP level, or other levels can also be defined to carry some or all of the filter information.
Each of the filters for each entry can include many coefficients. In one example, the filters comprise two-dimensional filters with 81 different coefficients defined by a filter holder that extends in two dimensions. However, the number of filter coefficients that are signaled for each filter may be less than 81 in some cases. Coefficient symmetry, for example, can be imposed so that the filter coefficients in a dimension or quadrant can correspond to inverted or symmetrical values with respect. coefficients in other dimensions or quadrants. Coefficient symmetry can allow 81 different 7 coefficients to be represented by fewer coefficients, in which case the encoder and decoder can assume that the inverted or mirrored values of coefficients define other coefficients. For example, the coefficients (5, —-2, 10, 10, -2, 5) can be coded and flagged as the subset of coefficients (5, -2, 10). In this case, the decoder can know that these three coefficients define the largest symmetric set of coefficients (5, -2, 10, 10, -2, 5).
The techniques of this description can be implemented on a wide variety of devices or devices, including a wireless device, and integrated circuit (IC) or a set of ICs (that is, a chip set). Any components, modules or units have been described provided to emphasize the functional aspects and do not necessarily require realization by different hardware units. Accordingly, the techniques described here can be implemented in hardware, software, firmware or any combination thereof. If implemented in hardware, any features described as modules, units or components can be implemented together in an integrated logic device or separately as discrete yet interoperable logic devices. If implemented in software, the techniques can be performed at least in part by a computer-readable medium comprising instructions that, when executed on a processor, perform one or more of the methods described above. The computer-readable medium can comprise a computer-readable storage medium and can form. part of a computer program product, which may include packaging materials. Computer-readable storage media 7 may comprise random access memory (RAM), such as synchronized dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), memory programmable and electrically erasable read-only (EEPROM), FLASH memory, magnetic or optical data storage media, and the like. The techniques may additionally or alternatively be performed at least in part by a computer-readable communication medium that carries or communicates the code in the form of instructions or data structures and that can be accessed, read and / or executed by a computer.
The code can be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, an Md Ae idAa dJSadavvaAdasA by am TVTiAaASASIA nqeomaAmÍ £ fias fias / NCTOYV aVrVraniacao
. 104/104 field programmable logic (FPGA), or other discrete or integrated integrated circuit set.
Accordingly, the term "processor", as used herein, may refer to any of the above structures or any other structure suitable for implementing the techniques described here.
In addition, in some respects, the functionality described here can be provided within dedicated software modules or dedicated hardware modules configured for encoding and decoding, or incorporated into a combined video codec.
In addition, the techniques can be fully implemented in one or more circuits or logic elements.
Various aspects of the description have been described.
These and other aspects are within the scope of the following claims.

权利要求:
Claims (15)
[1]
1. Video encoding method, the method comprising: determining a direction of at least one pixel from a pixel block; determine a metric for at least one pixel in the pixel block, where the metric is determined based on a comparison of pixel values for a subset of pixels in the block with at least one pixel, and where the pixel block is a block of pixels MxN, and where at least one of M or N is 2 or greater, and where the metric comprises an activity metric; determine a filter for at least one pixel of the pixel block based on the metric and direction, and generate a filtered image by applying the filter to at least one pixel of the pixel block.
[2]
2. Method according to claim 1, in which the direction comprises one of a horizontal direction, a vertical direction, a direction at 45 degrees, and a direction at 135 degrees.
[3]
3. Method according to claim 1, in which determining the direction comprises comparing a measure of horizontal activity to a measure of vertical activity, and in which the horizontal activity is determined based on the comparison of a pixel value of at least a pixel in the subset with a pixel value of a left neighboring pixel and a pixel value of a right neighboring pixel, where the left neighboring pixel and the right neighboring pixel are both in the pixel block.
[4]
A method according to claim 3, in which vertical activity is determined based on a comparison of a pixel value of at least one pixel in the subset with a pixel value of a higher neighbor pixel and a pixel value of a lower neighbor pixel, where the upper neighbor pixel and the lower neighbor pixel are both in the pixel block.
[5]
5. Method according to claim 1, in which the subset of pixels comprises pixels not at the limit of the MxN block.
[6]
6. Method according to claim 1, in which the pixel block is a 4x4 pixel block including sixteen pixels, and in which the subset of pixels includes four pixels.
[7]
A method according to claim 1, wherein determining the metric comprises: comparing at least one pixel with a first neighboring pixel of the subset of pixels; incrementing an end metric variable in response to a pixel value of at least one pixel being greater than a pixel value of the first neighboring pixel; decrement the end metric variable in response to the pixel value of at least one pixel being less than the pixel value of the first neighboring pixel;
[8]
A method according to claim 7, in which determining the metric further comprises: comparing the at least one pixel with a second neighboring pixel of the subset of pixels; incrementing the end metric variable in response to a pixel value of at least one pixel being greater than a pixel value of the second neighboring pixel; decrement the end metric variable in response to the pixel value of at least one pixel being less than the pixel value of the second neighboring pixel;
[9]
A method according to claim 1, in which the first neighboring pixel and the second neighboring pixel comprise one of: a left neighboring pixel and a right neighboring pixel; an upper neighbor pixel and a lower neighbor pixel; a lower left neighbor pixel and an upper right neighbor pixel; and an upper left neighboring pixel and a lower right neighboring pixel.
[10]
A method according to claim 1, in which the method is carried out by a video encoding device comprising a video encoder.
[11]
11. Method according to claim 1, in which the method is carried out by a video encoding device comprising a video decoder.
[12]
12. Apparatus, comprising: mechanisms for determining a direction of at least one pixel from a pixel block; mechanisms for determining a metric for oO at least one pixel in the pixel block, where the metric is determined based on a comparison of pixel values for a subset of the pixels in the block with at least one pixel, and in which the block of pixels is a block of pixels MxN, where at least one of M or N is 2 or greater, and where the metric comprises an activity metric; mechanisms for determining a filter for at least one pixel of the pixel block based on the metric and the direction; and mechanisms for generating a filtered image by applying the filter to at least one pixel of the pixel block.
[13]
Apparatus according to claim 12, wherein the direction comprises one of a horizontal direction, a vertical direction, a direction at 45 degrees, and a direction at 135 degrees.
[14]
Apparatus according to claim 13, in which the mechanisms for determining the metric comprise mechanisms for comparing a measure of horizontal activity to a measure of vertical activity, and in which the mechanisms for comparing the measure of horizontal activity for the measure vertical activity comprises mechanisms for comparing a pixel value of at least one pixel in the subset with a pixel value of a left neighbor pixel and a pixel value of a right neighbor pixel, where the left neighbor pixel and the right neighbor pixel they are both in the pixel block.
[15]
15. Computer-readable storage medium having instructions stored in the same as when executed cause one or more processors to perform the method according to any of the claims of 1 all.
i 1/15
THE
DEVICE DEVICE
ORIGIN DESTINATION 1u2 116. DEVICE OF
VIDEO SOURCE DISPLAY 120 130
THE MON CODIFIER | VIDEO DECODER] VIDEO STORAGE 12 Pao 128 lo ——— o [À
MODEM MODEM 123 127
TRANSMITTER RECEIVER o

类似技术:

---

公开号 | 公开日 | 专利标题

---

BR112013021617A2|2020-09-29|multimetric filtering

---

KR20130070636A|2013-06-27|Filter description signaling for multi-filter adaptive filtering

---

IL227994A|2017-05-29|Multi-metric filtering

同族专利:

---

公开号 | 公开日

---

WO2012116088A1|2012-08-30|

---

ES2816423T3|2021-04-05|

---

US9258563B2|2016-02-09|

---

RU2013143011A|2015-03-27|

---

EP3796653A1|2021-03-24|

---

US8982960B2|2015-03-17|

---

US20150117554A1|2015-04-30|

---

HUE051435T2|2021-03-01|

---

EP3700203A1|2020-08-26|

---

US20120213291A1|2012-08-23|

---

KR20130129442A|2013-11-28|

---

CN107277525B|2020-05-01|

---

AU2012220639B2|2015-11-19|

---

US20150117555A1|2015-04-30|

---

EP2679008B1|2020-07-08|

---

JP2016026449A|2016-02-12|

---

AU2012220632A1|2013-09-05|

---

CN103380621B|2017-05-10|

---

TWI499267B|2015-09-01|

---

DK2679008T3|2020-08-03|

---

KR20150058528A|2015-05-28|

---

DK2679009T3|2020-08-03|

---

MX2013009722A|2013-09-16|

---

MY167114A|2018-08-10|

---

JP2014511615A|2014-05-15|

---

SI2679009T1|2020-11-30|

---

CN103477639B|2017-09-22|

---

US9819936B2|2017-11-14|

---

WO2012116095A1|2012-08-30|

---

TW201242374A|2012-10-16|

---

CN107396114A|2017-11-24|

---

JP5815756B2|2015-11-17|

---

CN107396114B|2020-09-22|

---

EP3687171A1|2020-07-29|

---

MY166573A|2018-07-17|

---

JP2014508473A|2014-04-03|

---

JP6370960B2|2018-08-08|

---

JP2016006985A|2016-01-14|

---

CA2830381C|2016-05-17|

---

US8964852B2|2015-02-24|

---

CA2828406A1|2012-08-30|

---

US20160156906A1|2016-06-02|

---

KR101581098B1|2015-12-29|

---

EP2679007A1|2014-01-01|

---

KR101788948B1|2017-10-20|

---

HUE051433T2|2021-03-01|

---

JP2014511613A|2014-05-15|

---

RU2013142925A|2015-03-27|

---

KR101578986B1|2015-12-18|

---

IL227636A|2017-05-29|

---

EP3687170A1|2020-07-29|

---

ES2824831T3|2021-05-13|

---

JP2014511614A|2014-05-15|

---

UA110637C2|2016-01-25|

---

CN107277525A|2017-10-20|

---

SG192123A1|2013-08-30|

---

KR20130129441A|2013-11-28|

---

JP5897609B2|2016-03-30|

---

ZA201307111B|2018-12-19|

---

CN103392339B|2016-08-24|

---

WO2012116094A1|2012-08-30|

---

WO2012116090A1|2012-08-30|

---

CN103404142A|2013-11-20|

---

KR20130129440A|2013-11-28|

---

PL2679009T3|2020-12-28|

---

RU2579688C2|2016-04-10|

---

JP5815755B2|2015-11-17|

---

KR20150056663A|2015-05-26|

---

EP2679008A1|2014-01-01|

---

US20120213292A1|2012-08-23|

---

CN103404142B|2018-06-01|

---

CN103392339A|2013-11-13|

---

JP5752812B2|2015-07-22|

---

AU2012220639A1|2013-10-03|

---

JP2017175640A|2017-09-28|

---

JP6141917B2|2017-06-07|

---

EP2679009A1|2014-01-01|

---

JP6105011B2|2017-03-29|

---

EP2679010A1|2014-01-01|

---

EP2679009B1|2020-07-08|

---

CN103380621A|2013-10-30|

---

CN103477639A|2013-12-25|

---

KR101552032B1|2015-09-09|

---

US9877023B2|2018-01-23|

---

KR20130129439A|2013-11-28|

---

SG192743A1|2013-09-30|

---

US8989261B2|2015-03-24|

---

US8964853B2|2015-02-24|

---

KR101552031B1|2015-09-09|

---

AU2012220632B2|2015-09-10|

---

ZA201307110B|2018-12-19|

---

BR112013021476A2|2016-10-11|

---

IL227636D0|2013-09-30|

---

RU2584961C2|2016-05-20|

---

CA2830381A1|2012-08-30|

---

US20120213271A1|2012-08-23|

---

PT2679009T|2020-08-31|

---

US20120213293A1|2012-08-23|

---

CA2828406C|2016-07-19|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

CA1270322A|1983-12-22|1990-06-12|Kotaro Asai|Adaptive comb filter|

---

JPH02912B2|1984-06-15|1990-01-09|Mitsubishi Electric Corp|

---

JP2673778B2|1994-02-22|1997-11-05|国際電信電話株式会社|Noise reduction device for video decoding|

---

JPH0970044A|1995-08-31|1997-03-11|Sony Corp|Image signal processor and method therefor|

---

US5798795A|1996-03-01|1998-08-25|Florida Atlantic University|Method and apparatus for encoding and decoding video signals|

---

US5844613A|1997-03-17|1998-12-01|Microsoft Corporation|Global motion estimator for motion video signal encoding|

---

KR100265722B1|1997-04-10|2000-09-15|백준기|Image processing method and apparatus based on block|

---

EP1033883A1|1998-09-22|2000-09-06|Matsushita Electric Industrial Co., Ltd.|Video signal encoding method, video signal encoder, and program recorded medium|

---

US6421720B2|1998-10-28|2002-07-16|Cisco Technology, Inc.|Codec-independent technique for modulating bandwidth in packet network|

---

US6529638B1|1999-02-01|2003-03-04|Sharp Laboratories Of America, Inc.|Block boundary artifact reduction for block-based image compression|

---

US7003038B2|1999-09-27|2006-02-21|Mitsubishi Electric Research Labs., Inc.|Activity descriptor for video sequences|

---

FI117533B|2000-01-20|2006-11-15|Nokia Corp|Procedure for filtering digital video images|

---

US7203234B1|2000-03-31|2007-04-10|Sharp Laboratories Of America, Inc.|Method of directional filtering for post-processing compressed video|

---

US7289154B2|2000-05-10|2007-10-30|Eastman Kodak Company|Digital image processing method and apparatus for brightness adjustment of digital images|

---

US6504872B1|2000-07-28|2003-01-07|Zenith Electronics Corporation|Down-conversion decoder for interlaced video|

---

US20030026495A1|2001-03-07|2003-02-06|Gondek Jay Stephen|Parameterized sharpening and smoothing method and apparatus|

---

DE10120395A1|2001-04-25|2002-10-31|Bosch Gmbh Robert|Device for the interpolation of samples as well as image encoder and image decoder|

---

US7266150B2|2001-07-11|2007-09-04|Dolby Laboratories, Inc.|Interpolation of video compression frames|

---

CN1298171C|2001-09-18|2007-01-31|松下电器产业株式会社|Image encoding method and image decoding method|

---

US6983079B2|2001-09-20|2006-01-03|Seiko Epson Corporation|Reducing blocking and ringing artifacts in low-bit-rate coding|

---

MXPA03006715A|2001-11-29|2003-10-24|Matsushita Electric Ind Co Ltd|Coding distortion removal method, video encoding method, video decoding method, and apparatus and program for the same.|

---

KR100418437B1|2001-12-24|2004-02-14|씨앤에스 테크놀로지|A moving picture decoding processor for multimedia signal processing|

---

US7379501B2|2002-01-14|2008-05-27|Nokia Corporation|Differential coding of interpolation filters|

---

EP2894856A1|2002-01-31|2015-07-15|Samsung Electronics Co., Ltd|Filtering method and apparatus for reducing block artifacts or ringing noise|

---

JP4102973B2|2002-04-24|2008-06-18|日本電気株式会社|Encoding method and decoding method of moving image, apparatus and program using the same|

---

AT343302T|2002-05-02|2006-11-15|Fraunhofer Ges Forschung|CODING AND DECODING TRANSFORMATION COEFFICIENTS IN PICTURE OR VIDEO CODERS|

---

AU2003246185A1|2002-06-25|2004-01-06|Matsushita Electric Industrial Co., Ltd.|Motion detection device and noise reduction device using that|

---

AU2003244072B2|2002-07-11|2007-08-16|Godo Kaisha Ip Bridge 1|Filtering Strength Determination Method, Moving Picture Coding Method and Moving Picture Decoding Method|

---

US7391812B2|2002-07-14|2008-06-24|Apple Inc.|Adaptively post filtering encoded video|

---

WO2004082290A1|2003-03-10|2004-09-23|Mitsubishi Denki Kabushiki Kaisha|Video signal encoding device and video signal encoding method|

---

US7430335B2|2003-08-13|2008-09-30|Apple Inc|Pre-processing method and system for data reduction of video sequences and bit rate reduction of compressed video sequences using spatial filtering|

---

US7599438B2|2003-09-07|2009-10-06|Microsoft Corporation|Motion vector block pattern coding and decoding|

---

US8625680B2|2003-09-07|2014-01-07|Microsoft Corporation|Bitstream-controlled post-processing filtering|

---

EP1665804A1|2003-09-17|2006-06-07|Thomson Licensing S.A.|Adaptive reference picture generation|

---

US7822286B2|2003-11-07|2010-10-26|Mitsubishi Electric Research Laboratories, Inc.|Filtering artifacts in images with 3D spatio-temporal fuzzy filters|

---

US7437013B2|2003-12-23|2008-10-14|General Instrument Corporation|Directional spatial video noise reduction|

---

US7453938B2|2004-02-06|2008-11-18|Apple Inc.|Target bitrate estimator, picture activity and buffer management in rate control for video coder|

---

US8165389B2|2004-03-15|2012-04-24|Microsoft Corp.|Adaptive interpolation with artifact reduction of images|

---

JP4468734B2|2004-04-27|2010-05-26|オリンパス株式会社|Video signal processing apparatus and video signal processing program|

---

US7460596B2|2004-04-29|2008-12-02|Mediatek Incorporation|Adaptive de-blocking filtering apparatus and method for MPEG video decoder|

---

US20070230565A1|2004-06-18|2007-10-04|Tourapis Alexandros M|Method and Apparatus for Video Encoding Optimization|

---

KR20070044455A|2004-07-20|2007-04-27|콸콤 인코포레이티드|Method and apparatus for encoder assisted-frame rate up conversion for video compression|

---

US20060028562A1|2004-08-09|2006-02-09|Martin Schmitz|Fast area-selected filtering for pixel-noise and analog artifacts reduction|

---

US7370126B2|2004-11-03|2008-05-06|Cisco Technology, Inc.|System and method for implementing a demand paging jitter buffer algorithm|

---

US7634148B2|2005-01-07|2009-12-15|Ntt Docomo, Inc.|Image signal transforming and inverse-transforming method and computer program product with pre-encoding filtering features|

---

US20090022220A1|2005-04-13|2009-01-22|Universitaet Hannover|Method and apparatus for enhanced video coding|

---

US7680355B2|2005-05-02|2010-03-16|Intel Corporation|Detection of artifacts resulting from image signal decompression|

---

US8422546B2|2005-05-25|2013-04-16|Microsoft Corporation|Adaptive video encoding using a perceptual model|

---

US20060285597A1|2005-06-20|2006-12-21|Flextronics International Usa, Inc.|Reusing interpolated values in advanced video encoders|

---

US8208564B2|2005-06-24|2012-06-26|Ntt Docomo, Inc.|Method and apparatus for video encoding and decoding using adaptive interpolation|

---

US7778169B2|2005-09-02|2010-08-17|Cisco Technology, Inc.|Packetizing media for a time slotted communication system|

---

US7894522B2|2005-09-16|2011-02-22|Sony Corporation|Classified filtering for temporal prediction|

---

JP4455487B2|2005-12-16|2010-04-21|株式会社東芝|Decoding device, decoding method, and program|

---

WO2007111292A1|2006-03-27|2007-10-04|Matsushita Electric Industrial Co., Ltd.|Picture coding apparatus and picture decoding apparatus|

---

WO2008010929A2|2006-07-18|2008-01-24|Thomson Licensing|Methods and apparatus for adaptive reference filtering|

---

US8253752B2|2006-07-20|2012-08-28|Qualcomm Incorporated|Method and apparatus for encoder assisted pre-processing|

---

US8731064B2|2006-09-11|2014-05-20|Apple Inc.|Post-processing for decoder complexity scalability|

---

US20080075165A1|2006-09-26|2008-03-27|Nokia Corporation|Adaptive interpolation filters for video coding|

---

TWI368443B|2006-11-09|2012-07-11|Lg Electronics Inc|Method and apparatus for decoding/encoding a video signal|

---

EP2123051B1|2006-12-18|2010-11-10|Koninklijke Philips Electronics N.V.|Image compression and decompression|

---

EP2127391A2|2007-01-09|2009-12-02|Nokia Corporation|Adaptive interpolation filters for video coding|

---

KR100856551B1|2007-05-31|2008-09-04|한국과학기술원|Deblock filter and deblock filtering method in h.264/avc|

---

WO2008148272A1|2007-06-04|2008-12-11|France Telecom Research & Development Beijing Company Limited|Method and apparatus for sub-pixel motion-compensated video coding|

---

US7965900B2|2007-09-26|2011-06-21|Hewlett-Packard Development Company, L.P.|Processing an input image to reduce compression-related artifacts|

---

EP2191651A1|2007-09-28|2010-06-02|Dolby Laboratories Licensing Corporation|Video compression and tranmission techniques|

---

EP2048886A1|2007-10-11|2009-04-15|Panasonic Corporation|Coding of adaptive interpolation filter coefficients|

---

CN101184221A|2007-12-06|2008-05-21|上海大学|Vision attention based video encoding method|

---

US8576906B2|2008-01-08|2013-11-05|Telefonaktiebolaget L M Ericsson |Adaptive filtering|

---

CN101911704A|2008-01-14|2010-12-08|汤姆森许可贸易公司|Methods and apparatus for de-artifact filtering using multi-lattice sparsity-based filtering|

---

US8831086B2|2008-04-10|2014-09-09|Qualcomm Incorporated|Prediction techniques for interpolation in video coding|

---

WO2009131508A2|2008-04-23|2009-10-29|Telefonaktiebolaget Lm Ericsson |Template-based pixel block processing|

---

US10123050B2|2008-07-11|2018-11-06|Qualcomm Incorporated|Filtering video data using a plurality of filters|

---

US8290782B2|2008-07-24|2012-10-16|Dts, Inc.|Compression of audio scale-factors by two-dimensional transformation|

---

US8736751B2|2008-08-26|2014-05-27|Empire Technology Development Llc|Digital presenter for displaying image captured by camera with illumination system|

---

US8150191B2|2008-10-14|2012-04-03|Interra Systems Inc.|Method and system for calculating blur artifacts in videos using user perception threshold|

---

US8792564B2|2008-10-28|2014-07-29|Sony Corporation|Adaptive preprocessing method using feature-extracted video maps|

---

US8761538B2|2008-12-10|2014-06-24|Nvidia Corporation|Measurement-based and scalable deblock filtering of image data|

---

US9143803B2|2009-01-15|2015-09-22|Qualcomm Incorporated|Filter prediction based on activity metrics in video coding|

---

WO2010102935A1|2009-03-09|2010-09-16|Thomson Licensing|Estimation of the prediction mode for the intra coding mode|

---

CN101854540B|2009-04-01|2014-07-02|辉达公司|Intra prediction method and device for employing H.264 video coding standard|

---

EP2262267A1|2009-06-10|2010-12-15|Panasonic Corporation|Filter coefficient coding scheme for video coding|

---

WO2011126759A1|2010-04-09|2011-10-13|Sony Corporation|Optimal separable adaptive loop filter|

---

US9094658B2|2010-05-10|2015-07-28|Mediatek Inc.|Method and apparatus of adaptive loop filtering|

---

CN101945281B|2010-09-10|2014-09-10|中兴通讯股份有限公司|Method and device for filtering video codes|

---

US8982960B2|2011-02-23|2015-03-17|Qualcomm Incorporated|Multi-metric filtering|

---

JP5818755B2|2012-08-23|2015-11-18|有限会社イザキ|Incineration ash storage method and incineration ash storage container used therefor|EP2100385A4|2007-01-05|2012-07-25|Lg Electronics Inc|Layer mapping method and data transmission metho for mimo system|

---

US10123050B2|2008-07-11|2018-11-06|Qualcomm Incorporated|Filtering video data using a plurality of filters|

---

US9143803B2|2009-01-15|2015-09-22|Qualcomm Incorporated|Filter prediction based on activity metrics in video coding|

---

US8982960B2|2011-02-23|2015-03-17|Qualcomm Incorporated|Multi-metric filtering|

---

WO2012144876A2|2011-04-21|2012-10-26|한양대학교 산학협력단|Method and apparatus for encoding/decoding images using a prediction method adopting in-loop filtering|

---

US9344742B2|2012-08-10|2016-05-17|Google Inc.|Transform-domain intra prediction|

---

US20140056363A1|2012-08-23|2014-02-27|Yedong He|Method and system for deblock filtering coded macroblocks|

---

US9819965B2|2012-11-13|2017-11-14|Intel Corporation|Content adaptive transform coding for next generation video|

---

FR3011429A1|2013-09-27|2015-04-03|Orange|VIDEO CODING AND DECODING BY HERITAGE OF A FIELD OF MOTION VECTORS|

---

KR20150037371A|2013-09-30|2015-04-08|삼성전자주식회사|Method and apparatus for image processing using DMA|

---

US9628822B2|2014-01-30|2017-04-18|Qualcomm Incorporated|Low complexity sample adaptive offset encoding|

---

CN106105047B|2014-03-14|2018-10-26|夏普株式会社|Video compress with color space scalability|

---

CN104023241B|2014-05-29|2017-08-04|华为技术有限公司|The method for video coding and video coding apparatus of intraframe predictive coding|

---

US10057574B2|2015-02-11|2018-08-21|Qualcomm Incorporated|Coding tree unitlevel adaptive loop filter |

---

CN104918057B|2015-05-08|2018-07-13|上海交通大学|A kind of motion vector after-treatment system using neighborhood motion information|

---

MX2017014914A|2015-05-21|2018-06-13|Huawei Tech Co Ltd|Apparatus and method for video motion compensation.|

---

EP3313078B1|2015-06-18|2020-12-23|LG Electronics Inc.|Image properties-based adaptive filtering method and device in image coding system|

---

EP3313079B1|2015-06-18|2021-09-01|LG Electronics Inc.|Image filtering method in image coding system|

---

CN105049846B|2015-08-14|2019-05-21|广东中星微电子有限公司|The method and apparatus of image and coding and decoding video|

---

CN106470341B|2015-08-17|2020-10-02|恩智浦美国有限公司|Media display system|

---

US9883183B2|2015-11-23|2018-01-30|Qualcomm Incorporated|Determining neighborhood video attribute values for video data|

---

KR101788183B1|2015-12-28|2017-10-20|현대자동차주식회사|Vehicle and controlling method for the vehicle|

---

US20170238020A1|2016-02-15|2017-08-17|Qualcomm Incorporated|Geometric transforms for filters for video coding|

---

US10382766B2|2016-05-09|2019-08-13|Qualcomm Incorporated|Signalling of filtering information|

---

US10419755B2|2016-05-16|2019-09-17|Qualcomm Incorporated|Confusion of multiple filters in adaptive loop filtering in video coding|

---

US10572978B2|2016-12-05|2020-02-25|Kennesaw State University Research And Service Foundation, Inc.|Moran's / for impulse noise detection and removal in color images|

---

JP2018182444A|2017-04-07|2018-11-15|株式会社Ｊｖｃケンウッド|Image encoding device, image encoding method and image encoding program, as well as image decoding device, image decoding method and image decoding program|

---

CN108737841B|2017-04-21|2020-11-24|腾讯科技（深圳）有限公司|Coding unit depth determination method and device|

---

US10225578B2|2017-05-09|2019-03-05|Google Llc|Intra-prediction edge filtering|

---

CN111133757A|2017-09-27|2020-05-08|索尼公司|Encoding device, encoding method, decoding device, and decoding method|

---

US20190116359A1|2017-10-12|2019-04-18|Qualcomm Incorporated|Guided filter for video coding and processing|

---

US10992939B2|2017-10-23|2021-04-27|Google Llc|Directional intra-prediction coding|

---

CN108122268B|2017-12-19|2021-07-23|网易（杭州）网络有限公司|Mapping processing method and device|

---

US20190297603A1|2018-03-23|2019-09-26|Samsung Electronics Co., Ltd.|Method and apparatus for beam management for multi-stream transmission|

---

US20210021820A1|2018-03-23|2021-01-21|Sharp Kabushiki Kaisha|Image filtering apparatus, image decoding apparatus, and image coding apparatus|

---

KR20200054720A|2018-11-12|2020-05-20|삼성전자주식회사|Display apparatus, method for controlling thereof and recording media thereof|

---

US11051017B2|2018-12-20|2021-06-29|Qualcomm Incorporated|Adaptive loop filterindex signaling|

---

RU2737343C2|2019-01-10|2020-11-27|Федеральное государственное казенное военное образовательное учреждение высшего образования "Военный учебно-научный центр Военно-воздушных сил "Военно-воздушная академия имени профессора Н.Е. Жуковского и Ю.А. Гагарина"Министерства обороны Российской Федерации|Method of determining object movement pattern on frames of video sequence|

法律状态:
2020-10-20| B06F| Objections, documents and/or translations needed after an examination request according [chapter 6.6 patent gazette]|

---

2020-11-10| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|

---

2020-11-10| B15K| Others concerning applications: alteration of classification|Free format text: AS CLASSIFICACOES ANTERIORES ERAM: H04N 7/26 , H04N 7/50 Ipc: H04N 19/117 (2014.01), H04N 19/14 (2014.01), H04N |

---

2021-11-23| B350| Update of information on the portal [chapter 15.35 patent gazette]|

优先权:

---

申请号 | 申请日 | 专利标题

---

US201161445967P| true| 2011-02-23|2011-02-23|

---

US61/445,967|2011-02-23|

---

US201161448771P| true| 2011-03-03|2011-03-03|

---

US61/448,771|2011-03-03|

---

US201161473713P| true| 2011-04-08|2011-04-08|

---

US61/473,713|2011-04-08|

---

US201161476260P| true| 2011-04-16|2011-04-16|

---

US61/476,260|2011-04-16|

---

US201161478287P| true| 2011-04-22|2011-04-22|

---

US61/478,287|2011-04-22|

---

US201161503440P| true| 2011-06-30|2011-06-30|

---

US201161503434P| true| 2011-06-30|2011-06-30|

---

US201161503426P| true| 2011-06-30|2011-06-30|

---

US61/503,426|2011-06-30|

---

US61/503,434|2011-06-30|

---

US61/503,440|2011-06-30|

---

US201161527463P| true| 2011-08-25|2011-08-25|

---

US61/527,463|2011-08-25|

---

US201161531571P| true| 2011-09-06|2011-09-06|

---

US61/531,571|2011-09-06|

---

US13/401,685|US8964853B2|2011-02-23|2012-02-21|Multi-metric filtering|

---

US13/401,685|2012-02-21|

---

PCT/US2012/026166|WO2012116095A1|2011-02-23|2012-02-22|Multi-metric filtering|

---