Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On 11/25/2017 12:07 AM, Michael Niedermayer wrote: > I do not know that but i would be surprised if null dereferences tests > where unwelcome > > oss-fuzz will already report null derferences and OOM conditions, as > well as undefined behavior. So in some sense various points on the map > surrounding this here are already tested for https://github.com/dwbuiten/nullfuzzer Works without patching FFmpeg now, and is thread-safe. Haven't put it into oss-fuzz yet. - Derek ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On 11/25/2017 12:07 AM, Michael Niedermayer wrote: > I do not know that but i would be surprised if null dereferences tests > where unwelcome > > oss-fuzz will already report null derferences and OOM conditions, as > well as undefined behavior. So in some sense various points on the map > surrounding this here are already tested for Locally, I've made this work with something like: configure --malloc-prefix=fuzzer_ --extra-libs=-lallocfuzz I'll push that library up to a git repo some time today. Should be pretty easy to integrate into oss-fuzz like this, I think? - Derek ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
2017-11-26 2:05 GMT+01:00 Derek Buitenhuis : > On 11/26/2017 12:14 AM, Carl Eugen Hoyos wrote: >> I am of course in favour of such checks but is there an allocator we support >> that actually returns NULL on oom? > > Anything that doesn't use overcommit. Windows is the big obvious one here. > Also > various UNIX-like things, and even Linux is not guaranteed to return non-NULL, > depending on how the kernel is set up (e.g. on some RHELs I think, or on > plenty of embedded setups.) > > Some libcs will fail if the requested size is outside of the allowed range. Thank you for the explanation! Carl Eugen ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On 11/26/2017 12:14 AM, Carl Eugen Hoyos wrote: > I am of course in favour of such checks but is there an allocator we support > that actually returns NULL on oom? Anything that doesn't use overcommit. Windows is the big obvious one here. Also various UNIX-like things, and even Linux is not guaranteed to return non-NULL, depending on how the kernel is set up (e.g. on some RHELs I think, or on plenty of embedded setups.) Some libcs will fail if the requested size is outside of the allowed range. In any case, the checks should be done. - Derek ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On Sun, Nov 26, 2017 at 01:14:31AM +0100, Carl Eugen Hoyos wrote: > 2017-11-24 20:45 GMT+01:00 Derek Buitenhuis : > > I've had this kicking around for like 4 years, maybe it can be of use to > > some people. > > I haven't done full scale fuzzing with this because laziness. I just > > sometimes run it > > when I'm bored. It's not thread-safe, but it would be trivial to make it so. > > > > It's based off my old LD_PRELOAD hack from here: > > > > https://gist.github.com/dwbuiten/7101755 > > > > Optionally takes two env vars, MALLOC_SEED (the seed), and MALLOC_FAILPROB > > for the > > probability of failing. > > > > I've been running it directly integrated inside FFmpeg's allocator because > > it makes > > it easier to run under gdb to find where it actually crashes, if the stack > > trace of > > the failure is not enough info/context. > > > > Currently FFmpeg has a lot of unchecked allocations - just one single FATE > > run with > > this found: > > I am of course in favour of such checks but is there an allocator we support > that actually returns NULL on oom? try ulimit -S -v 1 and then try to malloc() more [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB No human being will ever know the Truth, for even if they happen to say it by chance, they would not even known they had done so. -- Xenophanes signature.asc Description: Digital signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
2017-11-24 20:45 GMT+01:00 Derek Buitenhuis : > I've had this kicking around for like 4 years, maybe it can be of use to some > people. > I haven't done full scale fuzzing with this because laziness. I just > sometimes run it > when I'm bored. It's not thread-safe, but it would be trivial to make it so. > > It's based off my old LD_PRELOAD hack from here: > > https://gist.github.com/dwbuiten/7101755 > > Optionally takes two env vars, MALLOC_SEED (the seed), and MALLOC_FAILPROB > for the > probability of failing. > > I've been running it directly integrated inside FFmpeg's allocator because it > makes > it easier to run under gdb to find where it actually crashes, if the stack > trace of > the failure is not enough info/context. > > Currently FFmpeg has a lot of unchecked allocations - just one single FATE > run with > this found: I am of course in favour of such checks but is there an allocator we support that actually returns NULL on oom? Carl Eugen ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On 11/24/2017 11:35 PM, Derek *Buitenhuis* wrote: > It would probably make an absolute ton of reports, since there are quite > a few unchecked allocs in FFmpeg... might be kinda spammy. That might not be the case any more, most of the checks have been added in the Coverity effort of 2015 and continued later on, so the report might actually have interesting findings. -- Vittorio ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On Fri, Nov 24, 2017 at 11:41:06PM +, Derek Buitenhuis wrote: > On 11/24/2017 11:35 PM, Michael Niedermayer wrote: > > Maybe integrating this in: > > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > > > would make sense > > > > That would run it automatically on ffmpeg master HEAD on powerfull hw > > Could make sense, yeah - wouldn't be that hard. > > It would probably make an absolute ton of reports, since there are quite > a few unchecked allocs in FFmpeg... might be kinda spammy. > > Also not sure how Google feels about using oss-fuzz to look for missing > NULL checks? Is there some set of guidelines? I do not know that but i would be surprised if null dereferences tests where unwelcome oss-fuzz will already report null derferences and OOM conditions, as well as undefined behavior. So in some sense various points on the map surrounding this here are already tested for [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In a rich man's house there is no place to spit but his face. -- Diogenes of Sinope signature.asc Description: Digital signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On 11/24/2017 11:35 PM, Michael Niedermayer wrote: > Maybe integrating this in: > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg > > would make sense > > That would run it automatically on ffmpeg master HEAD on powerfull hw Could make sense, yeah - wouldn't be that hard. It would probably make an absolute ton of reports, since there are quite a few unchecked allocs in FFmpeg... might be kinda spammy. Also not sure how Google feels about using oss-fuzz to look for missing NULL checks? Is there some set of guidelines? - Derek ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On Fri, Nov 24, 2017 at 07:45:46PM +, Derek Buitenhuis wrote: > I've had this kicking around for like 4 years, maybe it can be of use to some > people. > I haven't done full scale fuzzing with this because laziness. I just > sometimes run it > when I'm bored. It's not thread-safe, but it would be trivial to make it so. Maybe integrating this in: https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg would make sense That would run it automatically on ffmpeg master HEAD on powerfull hw [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Asymptotically faster algorithms should always be preferred if you have asymptotical amounts of data signature.asc Description: Digital signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On 11/24/2017 8:09 PM, Paul B Mahol wrote: > Do you have backtrace of this one? Yes, but the alloc failure is not in lavfi: my_posix_memalign:77 in libavutil/posixmemalign.c av_malloc:89 in libavutil/mem.c av_mallocz:240 in libavutil/mem.c av_packet_alloc:53 in libavcodec/avpacket.c av_bsf_alloc:106 in libavcodec/bsf.c bsfs_init:224 in libavcodec/decode.c avcodec_send_packet:655 in libavcodec/decode.c decode:2241 in fftools/ffmpeg.c decode_video:2385 in fftools/ffmpeg.c process_input_packet:2626 in fftools/ffmpeg.c process_input:4432 in fftools/ffmpeg.c transcode_step:4552 in fftools/ffmpeg.c transcode:4606 in fftools/ffmpeg.c main:4812 in fftools/ffmpeg.c - Derek ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
Re: [FFmpeg-devel] [PATCH 0/1][TOOL][HACK] Allocation NULL check fuzzer
On 11/24/17, Derek Buitenhuis wrote: > I've had this kicking around for like 4 years, maybe it can be of use to > some people. > I haven't done full scale fuzzing with this because laziness. I just > sometimes run it > when I'm bored. It's not thread-safe, but it would be trivial to make it so. > > It's based off my old LD_PRELOAD hack from here: > > https://gist.github.com/dwbuiten/7101755 > > Optionally takes two env vars, MALLOC_SEED (the seed), and MALLOC_FAILPROB > for the > probability of failing. > > I've been running it directly integrated inside FFmpeg's allocator because > it makes > it easier to run under gdb to find where it actually crashes, if the stack > trace of > the failure is not enough info/context. > > Currently FFmpeg has a lot of unchecked allocations - just one single FATE > run with > this found: > > daemon404@bbvm:~/dev/f/ffmpeg/tests/data/fate$ grep Seg *.err > adpcm-ima-amv.err:Segmentation fault > adpcm-ima-apc.err:Segmentation fault > caf.err:Segmentation fault > filter-mergeplanes.err:Segmentation fault Do you have backtrace of this one? ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
