So, I have a few more questions before I start working on a "production-ready"
patch for HEVC in MP4 and Matroska.
First one is fairly trivial: movenc.c and matroskaenc.c check whether the
extradata is in avcC/hvcC format to decide whether packets should be
reformatted (Annex B -> MP4 4-byte prefixes). But they both use a different
check:
movenc
if (enc->codec_id == AV_CODEC_ID_H264 && trk->vos_len > 0 && *(uint8_t
*)trk->vos_data != 1)
check whether the first byte of extradata is 1 (configurationVersion field of
avcC, always 1)
-> if not avcC, do reformatting
matroskaenc
if (codec->codec_id == AV_CODEC_ID_H264 && codec->extradata_size > 0 &&
(AV_RB24(codec->extradata) == 1 || AV_RB32(codec->extradata) == 1))
check whether the first 3 or 4 bytes of extradata are 1 (Annex B start code)
-> if Annex B, do reformatting
If, while updating these checks to enable reformatting for HEVC, I wanted to
make the check consistent for both muxers (the intent is the same, unless I'm
really mistaken), which one should I use (movenc, matroskaenc, matroskaenc but
check for extradata_size >= 4)?
Then, HEVC muxing is already enabled for matroskaenc and movenc with MODE_MOV,
both seemingly unintended side effects of decoding support added in 959bea1 and
ea29f96. Since we don't write the extradata properly nor reformat the input
packets when they're in Annex B format, should:
- fixing these issues (extradata, Annex B -> NAL prefixes) warrant a version
bump?
OR
- fixing these issues (extradata, Annex B -> NAL prefixes) warrant backporting
to libav10?
AND/OR
- HEVC muxing in libav10 be disabled so as to not spread invalid files?
- adding support for HEVC muxing in MODE_MP4 warrant a version bump?
OR
- adding support for HEVC muxing in MODE_MP4 warrant backporting to libav10?
Finally, ISO/IEC 14496-15 provides two different codec "tags" (sample entry
names) for HEVC in MP4, 'hvc1' and 'hev1'.
The difference being, if I understand correctly, that:
- if 'hvc1'
parameter sets (VPS/SPS/PPS) are *mandatory* in the sample entry (in the
HEVCDecoderConfigurationRecord found in the HEVCConfigurationBox, aka 'hvcC')
parameter sets (VPS/SPS/PPS) are *forbidden* in the samples themselves
HEVCDecoderConfigurationRecord is formatted with array_completeness set to
1 for all parameter set arrays
- if 'hev1'
parameter sets (VPS/SPS/PPS) may be found in the sample entry (there has to
be an HEVCDecoderConfigurationRecord but - AFAICT - it's probably OK if it does
not contain any parameter sets, as long as the other fields are set correctly)
parameter sets (VPS/SPS/PPS) may also be found in the samples themselves
HEVCDecoderConfigurationRecord is formatted with array_completeness set to
0 for all arrays
AFAIK, there are two main scenarios for bitstreams reaching the muxers:
- content encoded/re-encoded from another format (whatever -> HEVC encoder ->
avformat)
- content passed through from another container (MP4/Matroska -> avformat, or
MPEG-TS -> avformat)
In the former case, it's easy to configure the encoder to encode a continuous
stream, get the parameter sets and place them in extradata, and ensuring there
are no further parameter sets in the bitstream, so 'hvc1' could be used.
In the latter case, however, when the source is an 'hev1' MP4 file or a
transport stream, there is no way to know whether parameter sets will or will
not be repeated throughout the bitstream, so 'hev1' should probably be used.
It would seems that the easiest and/or more correct solution would be to use
'hev1' and format the hvcC accordingly, though there are further considerations:
- movenc was set up to use 'hvc1' for MODE_MOV - see above - but since we do
not write the hvcC at present, the produced files are invalid regardless of
what name is used for sample entries, so we should probably just fix it to
match MODE_MP4 once we've decided what to do
- according to Yusuke Nakamura, some players (don't know which ones) only
support files with 'hev1' sample entries, whereas others only support files
with 'hvc1' sample entries (and some support both), though someone did suggest
"screw broken players, do the right thing"
- some software (e.g. VLC) already produce files with 'hvc1', though TBH I
don't know whether they'd be very widespread or if it's terribly relevant to
what we should do
- we could theoretically use 'hvc1' and filter the bitstream of any further
parameter sets, possibly adding additional sample entries and storing them
there instead
- I may have misread ISO/IEC 14496-15, (obviously) feel free to point out any
mistakes on my part
There could always be an option to switch between the two, I suppose (though
this would still require a default value).
Tim
_______________________________________________
libav-devel mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-devel
