Re: [FFmpeg-devel] [PATCH] videodsp: fix 1-byte overread in top/bottom READ_NUM_BYTES iterations.

2016-01-18 Thread Michael Niedermayer
On Sat, Jan 16, 2016 at 02:44:47PM -0500, Ronald S. Bultje wrote:
> This can overread (either before start or beyond end) of the buffer in
> Nx1 (i.e. height=1) images.
> 
> Fixes mozilla bug 1240080.
> ---
>  libavcodec/x86/videodsp.asm | 21 ++---
>  1 file changed, 6 insertions(+), 15 deletions(-)

probably ok but iam not confident that i would spot a bug in this
code if there was one

[...]
-- 
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No human being will ever know the Truth, for even if they happen to say it
by chance, they would not even known they had done so. -- Xenophanes


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


Re: [FFmpeg-devel] [PATCH] videodsp: fix 1-byte overread in top/bottom READ_NUM_BYTES iterations.

2016-01-18 Thread Ronald S. Bultje
Hi,

On Mon, Jan 18, 2016 at 10:44 AM, Michael Niedermayer <
mich...@niedermayer.cc> wrote:

> On Sat, Jan 16, 2016 at 02:44:47PM -0500, Ronald S. Bultje wrote:
> > This can overread (either before start or beyond end) of the buffer in
> > Nx1 (i.e. height=1) images.
> >
> > Fixes mozilla bug 1240080.
> > ---
> >  libavcodec/x86/videodsp.asm | 21 ++---
> >  1 file changed, 6 insertions(+), 15 deletions(-)
>
> probably ok but iam not confident that i would spot a bug in this
> code if there was one


TY, pushed.

The second patch (videodsp checkasm) is meant to assist in bug-finding by
doing full-coverage runs - i.e. height=1 and alternate edge conditions
(which fate-$codec doesn't really test) - under asan or valgrind.

Ronald
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel


[FFmpeg-devel] [PATCH] videodsp: fix 1-byte overread in top/bottom READ_NUM_BYTES iterations.

2016-01-16 Thread Ronald S. Bultje
This can overread (either before start or beyond end) of the buffer in
Nx1 (i.e. height=1) images.

Fixes mozilla bug 1240080.
---
 libavcodec/x86/videodsp.asm | 21 ++---
 1 file changed, 6 insertions(+), 15 deletions(-)

diff --git a/libavcodec/x86/videodsp.asm b/libavcodec/x86/videodsp.asm
index 48f5ac0..a807d3b 100644
--- a/libavcodec/x86/videodsp.asm
+++ b/libavcodec/x86/videodsp.asm
@@ -193,14 +193,10 @@ hvar_fn
 movvalb, [srcq+%2-1]
 %elif (%2-%%off) == 2
 movvalw, [srcq+%2-2]
-%elifidn %1, body
+%else
 movvalb, [srcq+%2-1]
-salvald, 16
+rorvald, 16
 movvalw, [srcq+%2-3]
-%elifidn %1, bottom
-movd mm %+ %%mmx_idx, [srcq+%2-4]
-%else ; top
-movd mm %+ %%mmx_idx, [srcq+%2-3]
 %endif
 %endif ; (%2-%%off) >= 1
 %endmacro ; READ_NUM_BYTES
@@ -253,18 +249,13 @@ hvar_fn
 mov [dstq+%2-1], valb
 %elif (%2-%%off) == 2
 mov [dstq+%2-2], valw
-%elifidn %1, body
-mov [dstq+%2-3], valw
-sarvald, 16
-mov [dstq+%2-1], valb
 %else
-movd   vald, mm %+ %%mmx_idx
-%ifidn %1, bottom
-sarvald, 8
-%endif
 mov [dstq+%2-3], valw
-sarvald, 16
+rorvald, 16
 mov [dstq+%2-1], valb
+%ifnidn %1, body
+rorvald, 16
+%endif
 %endif
 %endif ; (%2-%%off) >= 1
 %endmacro ; WRITE_NUM_BYTES
-- 
2.1.2

___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel