On Sun, Aug 25, 2019 at 11:39 AM Michael Niedermayer <mich...@niedermayer.cc>
wrote:
> On Fri, Aug 23, 2019 at 04:03:10PM -0700, Niki Bowe wrote:
> > On Thu, Aug 22, 2019 at 2:30 AM Paul B Mahol <one...@gmail.com> wrote:
> >
> > > On Thu, Aug 22, 2019 at 11:19 AM Carl Eugen Hoyos <ceffm...@gmail.com>
> > > wrote:
> > >
> > > > Am Mi., 21. Aug. 2019 um 23:05 Uhr schrieb Niki Bowe
> > > > <nbowe-at-google....@ffmpeg.org>:
> > > > >
> > > > > On Mon, Aug 19, 2019 at 7:22 PM Carl Eugen Hoyos <
> ceffm...@gmail.com>
> > > > wrote:
> > > > >
> > > > > >
> > > > > > This score would mean that mjpeg can never be detected.
> > > > > > I suspect you have to reduce one of the demuxers to "- 1".
> > > > > >
> > > > > >
> > > > > Thanks Carl.
> > > > > Attached patch to reduce mpeg probe by -1, which also fixes the
> issue.
> > > >
> > > > Sorry, I misread the original report, it looked to me as if mJpeg was
> > > > the culprit.
> > > >
> > > > Imo, the mpeg probing should be fixed (return a smaller value) for
> your
> > > > sample
> > > > by detecting that it is not mpeg, not by returning a smaller value
> for
> > > > all samples.
> > > >
> > >
> > > 1000% agree.
> > >
> > >
> > It didn't return a smaller value for all samples, only the "invalid VDR
> > files and short PES streams" case.
> > Most mpegps files still return 26 immediately, because they have pack
> > headers.
> >
> > However, here is another patch where I try to limit it to only changing
> > score for these jpegs.
> > I noticed that these jpegs had a lot of 0x00000100 sequences, which
> matches
> > mpeg picture header start code. I added another heuristic which matches
> > these jpegs, but not any mpegps files I could find.
> >
> > Alternatively I could make reduce score if it doesn't start with a start
> > code? At the moment its happy to search until it finds start codes.
> >
> >
> > Is everyone really sure the best approach is to modify mpegps_probe for
> > this?
> > The mpegps_probe function returns 25 in many instances where it may not
> be
> > mpegps. It does only minimal structural checking, and allows invalid data
> > to still classify as mpegps.
> > jpeg probing returns 25 in some cases where it is almost certainly a jpeg
> > (Has to go through multiple tags to get to SOS, many of which early out
> if
> > they find invalid data).
> > Note that 25 is still treated as "low confidence" for jpeg. It logs
> "Format
> > jpeg_pipe detected only with low score of 25, misdetection possible!" for
> > these jpegs.
> > So I still think a score of 25 is too low for these jpegs, and that a
> > better fix would be to return 26 for jpeg_pipe and mjpeg if it makes it
> > past multiple tags to SOS.
>
> jpegs can be in other container formats its not jpeg in that case but the
> other container format
>
about this patch
> it breaks this:
>
> ./ffplay tickets//3327/issue3327-libc-2.17.so
>
> https://trac.ffmpeg.org/raw-attachment/ticket/3327/issue3327-libc-2.17.so
>
> which is detected as mpeg after the patch.
> really nothing should be detected as mpeg after this that was not before
> the idea IIUC is make something that was detected as mpeg to be not anymore
>
>
Thanks Michael. Good find.
Attached patch which only applies the extra sanity checks to the "Invalid
VDR files and short PES streams" path.
> Thanks
>
> [...]
>
>
> --
> Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Awnsering whenever a program halts or runs forever is
> On a turing machine, in general impossible (turings halting problem).
> On any real computer, always possible as a real computer has a finite
> number
> of states N, and will either halt in less than N cycles or never halt.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
--
Nikolas Bowe | SWE | nb...@google.com | 408-565-5137
From 7b054280926cdf837b7e8b361779cdf7ffcbdc56 Mon Sep 17 00:00:00 2001
From: Nikolas Bowe <nbowe@google.com>
Date: Thu, 8 Aug 2019 15:32:51 -0700
Subject: [PATCH] avformat: Fix probing on some JPEGs
Fixes "Invalid data found when processing input" on some JPEGs.
Some large extensionless JPEGs can get probe score collisions with mpeg
eg
$ ffprobe -loglevel trace /tmp/foo
[NULL @ 0x55c130ab04c0] Opening '/tmp/foo' for reading
[file @ 0x55c130ab0f40] Setting default whitelist 'file,crypto'
Probing jpeg_pipe score:6 size:2048
Probing jpeg_pipe score:6 size:4096
Probing jpeg_pipe score:6 size:8192
Probing jpeg_pipe score:6 size:16384
Probing jpeg_pipe score:25 size:32768
Probing jpeg_pipe score:25 size:65536
Probing jpeg_pipe score:25 size:131072
Probing jpeg_pipe score:25 size:262144
Probing jpeg_pipe score:25 size:524288
Probing mpeg score:25 size:1048576
Probing jpeg_pipe score:25 size:1048576
[AVIOContext @ 0x55c130ab9300] Statistics: 1048576 bytes read, 0 seeks
/tmp/foo: Invalid data found when processing input
This patch fixes this by reducing probe score for mpeg in some rare situations
---
libavformat/mpeg.c | 26 +++++++++++++++++---------
1 file changed, 17 insertions(+), 9 deletions(-)
diff --git a/libavformat/mpeg.c b/libavformat/mpeg.c
index 3205f209e6..8a97826551 100644
--- a/libavformat/mpeg.c
+++ b/libavformat/mpeg.c
@@ -68,8 +68,8 @@ static int mpegps_probe(const AVProbeData *p)
{
uint32_t code = -1;
int i;
- int sys = 0, pspack = 0, priv1 = 0, vid = 0;
- int audio = 0, invalid = 0, score = 0;
+ int sys = 0, pspack = 0, priv1 = 0, vid = 0, pic = 0;
+ int audio = 0, invalid = 0;
int endpes = 0;
for (i = 0; i < p->buf_size; i++) {
@@ -92,18 +92,15 @@ static int mpegps_probe(const AVProbeData *p)
else if ((code & 0xe0) == AUDIO_ID && pes) {audio++; i+=len;}
else if (code == PRIVATE_STREAM_1 && pes) {priv1++; i+=len;}
else if (code == 0x1fd && pes) vid++; //VC1
-
else if ((code & 0xf0) == VIDEO_ID && !pes) invalid++;
else if ((code & 0xe0) == AUDIO_ID && !pes) invalid++;
else if (code == PRIVATE_STREAM_1 && !pes) invalid++;
+ else if (code == 0x100) pic++;
}
}
- if (vid + audio > invalid + 1) /* invalid VDR files nd short PES streams */
- score = AVPROBE_SCORE_EXTENSION / 2;
-
-// av_log(NULL, AV_LOG_ERROR, "vid:%d aud:%d sys:%d pspack:%d invalid:%d size:%d \n",
-// vid, audio, sys, pspack, invalid, p->buf_size);
+// av_log(NULL, AV_LOG_ERROR, "vid:%d aud:%d sys:%d pspack:%d pic:%d invalid:%d size:%d \n",
+// vid, audio, sys, pspack, pic, invalid, p->buf_size);
if (sys > invalid && sys * 9 <= pspack * 10)
return (audio > 12 || vid > 3 || pspack > 2) ? AVPROBE_SCORE_EXTENSION + 2
@@ -116,10 +113,21 @@ static int mpegps_probe(const AVProbeData *p)
return (audio > 12 || vid > 6 + 2 * invalid) ? AVPROBE_SCORE_EXTENSION + 2
: AVPROBE_SCORE_EXTENSION / 2;
+ if (vid + audio > invalid + 1) {
+ if (pspack == 0 && vid > 0 && vid * 10 < pic) {
+ // If we don't find a pack header, and we have many more picture headers codes than
+ // mpeg video stream codes, then its probably not mpeg.
+ return AVPROBE_SCORE_EXTENSION / 2 - 1;
+ } else {
+ // Invalid VDR files and short PES streams
+ return AVPROBE_SCORE_EXTENSION / 2;
+ }
+ }
+
// 02-Penguin.flac has sys:0 priv1:0 pspack:0 vid:0 audio:1
// mp3_misidentified_2.mp3 has sys:0 priv1:0 pspack:0 vid:0 audio:6
// Have\ Yourself\ a\ Merry\ Little\ Christmas.mp3 0 0 0 5 0 1 len:21618
- return score;
+ return 0;
}
typedef struct MpegDemuxContext {
--
2.23.0.187.g17f5b7556c-goog
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
