Re: [FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock

2024-02-24 Thread J. Dekker

Martin Storsjö  writes:
> [...]
>
> Are you sure the input is within valid range? It's always possible that
> checkasm produces inputs that the real decoder wouldn't - but it's also
> possible that this is a real decoder bug that just hasn't been triggered by 
> any
> other test yet.
>
> // Martin

The checkasm was just written to just to trigger all the theoretical
edgecases. I know there is a decent range of values which pass the d0 +
d3 < beta check and overflow in (9 * (q0 - p0) - 3 * (q1 - p1) + 8) for
int16_t. I'm not 100% sure that these values can be output by the
decoder, and even if so they're rare.

-- 
jd
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock

2024-02-24 Thread Martin Storsjö

On Sat, 24 Feb 2024, J. Dekker wrote:



Nuo Mi  writes:


On Wed, Feb 21, 2024 at 7:10 PM J. Dekker  wrote:


Over/underflow in some cases.

Signed-off-by: J. Dekker 
---
 libavcodec/x86/hevcdsp_init.c | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/libavcodec/x86/hevcdsp_init.c b/libavcodec/x86/hevcdsp_init.c
index 31e81eb11f..11cb1b3bfd 100644
--- a/libavcodec/x86/hevcdsp_init.c
+++ b/libavcodec/x86/hevcdsp_init.c
@@ -1205,10 +1205,11 @@ void ff_hevc_dsp_init_x86(HEVCDSPContext *c, const
int bit_depth)
 if (EXTERNAL_SSE2(cpu_flags)) {
 c->hevc_v_loop_filter_chroma =
ff_hevc_v_loop_filter_chroma_12_sse2;
 c->hevc_h_loop_filter_chroma =
ff_hevc_h_loop_filter_chroma_12_sse2;
-if (ARCH_X86_64) {
-c->hevc_v_loop_filter_luma =
ff_hevc_v_loop_filter_luma_12_sse2;
-c->hevc_h_loop_filter_luma =
ff_hevc_h_loop_filter_luma_12_sse2;
-}
+// FIXME: 12-bit luma deblock over/underflows in some cases
+// if (ARCH_X86_64) {
+// c->hevc_v_loop_filter_luma =
ff_hevc_v_loop_filter_luma_12_sse2;
+// c->hevc_h_loop_filter_luma =
ff_hevc_h_loop_filter_luma_12_sse2;
+// }
 SAO_BAND_INIT(12, sse2);
 SAO_EDGE_INIT(12, sse2);


Hi Dekker,
VVC will utilize this function as well.
Could you please share the HEVC clip or data that caused the overflow?
We'll make efforts to address it during the VVC porting



You can just run ./tests/checkasm/checkasm --test=hevc_deblock to
find a failing case.


To clarify, this is with the new checkasm test added in this patchset, not 
currently in git master - otherwise fate would be failing for everybody on 
x86.



My guess is that delta0 overflows before the right
shift, see the ARM64 asm which specfically widens this calculation on 12
bit variant but I'm not 100%, I don't know x86 asm.


Are you sure the input is within valid range? It's always possible that 
checkasm produces inputs that the real decoder wouldn't - but it's also 
possible that this is a real decoder bug that just hasn't been triggered 
by any other test yet.


// Martin
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock

2024-02-24 Thread J. Dekker


Andreas Rheinhardt  writes:
> J. Dekker:
>>  SAO_BAND_INIT(12, sse2);
>>  SAO_EDGE_INIT(12, sse2);
>>  
>
> If you disable them here, you should also ensure that they are not
> assembled at all.
>
> - Andreas

Sure, will do on push if no other things to resolve in the latest set.

-- 
jd
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock

2024-02-24 Thread J. Dekker

Nuo Mi  writes:

> On Wed, Feb 21, 2024 at 7:10 PM J. Dekker  wrote:
>
>> Over/underflow in some cases.
>>
>> Signed-off-by: J. Dekker 
>> ---
>>  libavcodec/x86/hevcdsp_init.c | 9 +
>>  1 file changed, 5 insertions(+), 4 deletions(-)
>>
>> diff --git a/libavcodec/x86/hevcdsp_init.c b/libavcodec/x86/hevcdsp_init.c
>> index 31e81eb11f..11cb1b3bfd 100644
>> --- a/libavcodec/x86/hevcdsp_init.c
>> +++ b/libavcodec/x86/hevcdsp_init.c
>> @@ -1205,10 +1205,11 @@ void ff_hevc_dsp_init_x86(HEVCDSPContext *c, const
>> int bit_depth)
>>  if (EXTERNAL_SSE2(cpu_flags)) {
>>  c->hevc_v_loop_filter_chroma =
>> ff_hevc_v_loop_filter_chroma_12_sse2;
>>  c->hevc_h_loop_filter_chroma =
>> ff_hevc_h_loop_filter_chroma_12_sse2;
>> -if (ARCH_X86_64) {
>> -c->hevc_v_loop_filter_luma =
>> ff_hevc_v_loop_filter_luma_12_sse2;
>> -c->hevc_h_loop_filter_luma =
>> ff_hevc_h_loop_filter_luma_12_sse2;
>> -}
>> +// FIXME: 12-bit luma deblock over/underflows in some cases
>> +// if (ARCH_X86_64) {
>> +// c->hevc_v_loop_filter_luma =
>> ff_hevc_v_loop_filter_luma_12_sse2;
>> +// c->hevc_h_loop_filter_luma =
>> ff_hevc_h_loop_filter_luma_12_sse2;
>> +// }
>>  SAO_BAND_INIT(12, sse2);
>>  SAO_EDGE_INIT(12, sse2);
>>
> Hi Dekker,
> VVC will utilize this function as well.
> Could you please share the HEVC clip or data that caused the overflow?
> We'll make efforts to address it during the VVC porting
>

You can just run ./tests/checkasm/checkasm --test=hevc_deblock to
find a failing case. My guess is that delta0 overflows before the right
shift, see the ARM64 asm which specfically widens this calculation on 12
bit variant but I'm not 100%, I don't know x86 asm.

-- 
jd
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock

2024-02-23 Thread Nuo Mi
On Wed, Feb 21, 2024 at 7:10 PM J. Dekker  wrote:

> Over/underflow in some cases.
>
> Signed-off-by: J. Dekker 
> ---
>  libavcodec/x86/hevcdsp_init.c | 9 +
>  1 file changed, 5 insertions(+), 4 deletions(-)
>
> diff --git a/libavcodec/x86/hevcdsp_init.c b/libavcodec/x86/hevcdsp_init.c
> index 31e81eb11f..11cb1b3bfd 100644
> --- a/libavcodec/x86/hevcdsp_init.c
> +++ b/libavcodec/x86/hevcdsp_init.c
> @@ -1205,10 +1205,11 @@ void ff_hevc_dsp_init_x86(HEVCDSPContext *c, const
> int bit_depth)
>  if (EXTERNAL_SSE2(cpu_flags)) {
>  c->hevc_v_loop_filter_chroma =
> ff_hevc_v_loop_filter_chroma_12_sse2;
>  c->hevc_h_loop_filter_chroma =
> ff_hevc_h_loop_filter_chroma_12_sse2;
> -if (ARCH_X86_64) {
> -c->hevc_v_loop_filter_luma =
> ff_hevc_v_loop_filter_luma_12_sse2;
> -c->hevc_h_loop_filter_luma =
> ff_hevc_h_loop_filter_luma_12_sse2;
> -}
> +// FIXME: 12-bit luma deblock over/underflows in some cases
> +// if (ARCH_X86_64) {
> +// c->hevc_v_loop_filter_luma =
> ff_hevc_v_loop_filter_luma_12_sse2;
> +// c->hevc_h_loop_filter_luma =
> ff_hevc_h_loop_filter_luma_12_sse2;
> +// }
>  SAO_BAND_INIT(12, sse2);
>  SAO_EDGE_INIT(12, sse2);
>
Hi Dekker,
VVC will utilize this function as well.
Could you please share the HEVC clip or data that caused the overflow?
We'll make efforts to address it during the VVC porting

Thank you.

>
> --
> 2.43.2
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
>
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


Re: [FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock

2024-02-22 Thread Andreas Rheinhardt
J. Dekker:
> Over/underflow in some cases.
> 
> Signed-off-by: J. Dekker 
> ---
>  libavcodec/x86/hevcdsp_init.c | 9 +
>  1 file changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/libavcodec/x86/hevcdsp_init.c b/libavcodec/x86/hevcdsp_init.c
> index 31e81eb11f..11cb1b3bfd 100644
> --- a/libavcodec/x86/hevcdsp_init.c
> +++ b/libavcodec/x86/hevcdsp_init.c
> @@ -1205,10 +1205,11 @@ void ff_hevc_dsp_init_x86(HEVCDSPContext *c, const 
> int bit_depth)
>  if (EXTERNAL_SSE2(cpu_flags)) {
>  c->hevc_v_loop_filter_chroma = 
> ff_hevc_v_loop_filter_chroma_12_sse2;
>  c->hevc_h_loop_filter_chroma = 
> ff_hevc_h_loop_filter_chroma_12_sse2;
> -if (ARCH_X86_64) {
> -c->hevc_v_loop_filter_luma = 
> ff_hevc_v_loop_filter_luma_12_sse2;
> -c->hevc_h_loop_filter_luma = 
> ff_hevc_h_loop_filter_luma_12_sse2;
> -}
> +// FIXME: 12-bit luma deblock over/underflows in some cases
> +// if (ARCH_X86_64) {
> +// c->hevc_v_loop_filter_luma = 
> ff_hevc_v_loop_filter_luma_12_sse2;
> +// c->hevc_h_loop_filter_luma = 
> ff_hevc_h_loop_filter_luma_12_sse2;
> +// }
>  SAO_BAND_INIT(12, sse2);
>  SAO_EDGE_INIT(12, sse2);
>  

If you disable them here, you should also ensure that they are not
assembled at all.

- Andreas

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".


[FFmpeg-devel] [PATCH 2/3] avcodec/x86: disable hevc 12b luma deblock

2024-02-21 Thread J. Dekker
Over/underflow in some cases.

Signed-off-by: J. Dekker 
---
 libavcodec/x86/hevcdsp_init.c | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/libavcodec/x86/hevcdsp_init.c b/libavcodec/x86/hevcdsp_init.c
index 31e81eb11f..11cb1b3bfd 100644
--- a/libavcodec/x86/hevcdsp_init.c
+++ b/libavcodec/x86/hevcdsp_init.c
@@ -1205,10 +1205,11 @@ void ff_hevc_dsp_init_x86(HEVCDSPContext *c, const int 
bit_depth)
 if (EXTERNAL_SSE2(cpu_flags)) {
 c->hevc_v_loop_filter_chroma = 
ff_hevc_v_loop_filter_chroma_12_sse2;
 c->hevc_h_loop_filter_chroma = 
ff_hevc_h_loop_filter_chroma_12_sse2;
-if (ARCH_X86_64) {
-c->hevc_v_loop_filter_luma = 
ff_hevc_v_loop_filter_luma_12_sse2;
-c->hevc_h_loop_filter_luma = 
ff_hevc_h_loop_filter_luma_12_sse2;
-}
+// FIXME: 12-bit luma deblock over/underflows in some cases
+// if (ARCH_X86_64) {
+// c->hevc_v_loop_filter_luma = 
ff_hevc_v_loop_filter_luma_12_sse2;
+// c->hevc_h_loop_filter_luma = 
ff_hevc_h_loop_filter_luma_12_sse2;
+// }
 SAO_BAND_INIT(12, sse2);
 SAO_EDGE_INIT(12, sse2);
 
-- 
2.43.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".