Did you start from the MVC decoder from a long time ago? On Tue, 25 Aug 2026, at 18:49, Dom Cobley via ffmpeg-devel wrote: > This series adds decoding of H.264 MVC streams - Blu-ray 3D and the > consumer camcorder recordings that use the same coding - to the existing > h264 decoder, alongside the MV-HEVC support hevcdec already has, and using > the same view_ids/view_ids_available/view_pos_available option interface so > that the ffmpeg CLI view specifiers work for both. > > Only the base view is decoded by default, so nothing changes for a caller > that does not ask for more. > > This work was significantly helped by Claude Opus 5. I have tried to keep the > commits as clean and self contained as possible. I will take ownership of the > changes and help with resolving any future issues. > > The series is bisectable - every commit builds and passes fate-h264 - and it > has > had a good deal of testing beyond that, including a public LibreELEC test > build: > > https://forum.libreelec.tv/thread/30548-3d-support-builds-for-raspberry-pi > > FATE SAMPLE > =========== > > Patch 18 adds three tests that need a new sample, h264/mvc-2view.264 > (18628 bytes, md5 99901e05413ff29aecc6f191c664d8b2): > > > https://drive.google.com/uc?export=download&id=1y6xxbM8nwFCgtCvZizujChhcQoz8_S4w > > I have sent a separate mail to samples-request. I generated the file with > the JMVC 8.5 reference encoder from synthetic input, so it is freely > redistributable: nothing in the real world MVC material I tested against > can be, being either commercial films or camcorder captures of unclear > origin. Both views decode bit-identically to JMVC's own reconstruction, so > the tests check conformance rather than only locking in current behaviour. > > APPROACH > ======== > > Rather than address every view explicitly, as hevcdec does with > HEVCLayerContext, the decoder keeps the per-view state that actually > differs - POC and the reference lists - in a small H264ViewContext and > swaps it in and out around a view change. That avoids touching the ~250 > sites that would otherwise have needed a layer index threaded through them, > and keeps the diff to the parts of the decoder that genuinely differ per > view. > > Patch 8 converts the decoder to receive_frame(). An MVC access unit > produces two frames from one packet, which the decode() callback cannot > express. > > Patches 1 and 2 are not multiview at all. Both fix frame_num gap > concealment on damaged input, and both reproduce on an unmodified tree > without any of the rest of this series. They come first because the > multiview paths make the second one about five times easier to reach: on > one corpus file it deadlocks 1 run in 14 as things stand, and 5 in 14 once > a view is selected with -map 0:v:0:view:1. > > WHAT WORKS > ========== > > Both views, bit-identical to a single threaded decode, under frame > threading, slice threading and no threading, at every thread count I > tested, including on a Matroska file whose blocks do not start on an access > unit boundary - the dependent view slices of one access unit sit at the > head of the packet that goes on to carry the base view slices of the next. > That case drove several of the fixes folded into patches 11 and 14. > > Field coded multiview works (patch 17). > > Hardware acceleration does not, and is refused with a warning: every h264 > hwaccel builds its reference list from short_ref/long_ref and skips > pictures with reference == 0, which is exactly what an inter-view reference > looks like. Patch 15 has the detail. > > TESTING > ======= > > - make fate: 5649/5649 passing. > - Each of the 18 commits builds and passes fate-h264 standalone, so the > series bisects cleanly. > - A corpus of 43 real MVC and 3D files, 35 GB, decoded end to end with both > views mapped: Blu-ray rips, camcorder .264/.MTS/.ssif, and Matroska. Every > file that carries two views decodes both of them with no errors and with > matching frame counts, the one exception being a Matroska file that is > itself missing the dependent view slices of its last access unit (899 > against 900, which is what the file contains). Frame threading and slice > threading give byte-identical framecrc output over the whole corpus. Where > I compared against an unpatched build - the files exercising the awkward > paths, not all 43 - the base view output was bit-exact. > - Corrupted input, as doc/developer.texi asks for: 12000 decodes of damaged > streams across 16 files, under 10 threading configurations and 13 > mapping/decoder-option combinations, with the noise bitstream filter, > truncation and byte flipping. Clean under ASan and UBSan, and no hangs. > Getting there found four deadlocks and a batch of undefined behaviour, > all fixed in the patches that introduced them, except the concealment one > described above. > > Three faults that turned up during that and are NOT this series' - each > reproduces on an unmodified tree, and I will report them separately: > > - fftools/ffmpeg_sched.c:2168 asserts on some damaged input. > - -err_detect +explode with frame threading deadlocks on plain > single-view H.264. > - Mapping both views in one ffmpeg command collapses the dependent view's > timestamps in fftools when the two views share a PTS, which they do by > definition. Mapping either view alone is correct, and the decoder emits > correct timestamps in both cases. This looks like it would affect > MV-HEVC equally. > > KNOWN GAPS > ========== > > - mpegts carries the two views on separate PIDs (0x1011 and 0x1012) and > presents them as separate streams, so a Blu-ray .ssif or .MTS still > decodes only the view that was mapped. Associating the PIDs needs demuxer > work that is not in this series. > - cbs_h264 still refuses NAL 15 and NAL 20. > > Dom Cobley (18): > avcodec/h264dec: always initialise a concealed frame_num gap picture > avcodec/h264dec: do not await a field that was never decoded > avcodec/h264_ps: fix swapped MVC profile comments > avcodec/h264: parse subset SPS (NAL 15) and its MVC extension > avcodec/h2645_parse: parse the H.264 MVC NAL unit header extension > avcodec/h264dec: add multiview view_ids options > avcodec/h264dec: add a per-view context > avcodec/h264dec: switch to receive_frame() > avcodec/h264: decode dependent views and inter-view references > avcodec/h264_slice: do not reclaim the picture awaiting output > avcodec/h264dec: output both views of an access unit together > avcodec/h264: implement inter-view reference list modification > avcodec/h264dec: pick up the view list wherever the subset SPS appears > avcodec/h264dec: support frame and slice threading for multiview > avcodec/h264dec: refuse multiview with hardware acceleration > avformat/mov: read the mvcC box > avcodec/h264dec: support field coded multiview > doc/decoders, Changelog, fate: document and test H.264 MVC > > Changelog | 1 + > doc/decoders.texi | 45 +++ > libavcodec/bsf/extract_extradata.c | 2 +- > libavcodec/h2645_parse.c | 38 ++ > libavcodec/h2645_parse.h | 13 + > libavcodec/h264_direct.c | 15 +- > libavcodec/h264_parse.c | 4 + > libavcodec/h264_parser.c | 3 + > libavcodec/h264_picture.c | 2 + > libavcodec/h264_ps.c | 161 +++++++- > libavcodec/h264_ps.h | 57 +++ > libavcodec/h264_refs.c | 285 +++++++++++++- > libavcodec/h264_slice.c | 573 +++++++++++++++++++++++++++-- > libavcodec/h264dec.c | 307 ++++++++++++++-- > libavcodec/h264dec.h | 182 +++++++++ > libavformat/mov.c | 122 ++++++ > tests/fate/h264.mak | 9 + > tests/ref/fate/flv-demux | 2 +- > tests/ref/fate/h264-mvc-base | 21 ++ > tests/ref/fate/h264-mvc-both | 42 +++ > tests/ref/fate/h264-mvc-dependent | 21 ++ > tests/ref/fate/mov-zombie | 2 +- > tests/ref/fate/ts-small-demux | 2 +- > 23 files changed, 1823 insertions(+), 86 deletions(-) > create mode 100644 tests/ref/fate/h264-mvc-base > create mode 100644 tests/ref/fate/h264-mvc-both > create mode 100644 tests/ref/fate/h264-mvc-dependent > > -- > 2.53.0 > > _______________________________________________ > ffmpeg-devel mailing list -- [email protected] > To unsubscribe send an email to [email protected]
-- Jean-Baptiste Kempf - President +33 672 704 734 https://jbkempf.com/ _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
