On 29/09/18 08:43, Linjie Fu wrote:
> Add "new_nal_slice" to indicate the first coming nal_slice (including
> H264_NAL_IDR_SLICE and H264_NAL_SLICE)
>
> Extract extradata of streams when the first nal_slice comes, not only
> H264_NAL_IDR_SLICE but also H264_NAL_SLICE.
>
> This patch aims at the following issues:
> 1. the IDR frame is missing in the first GOP of a stream
> (common in live stream, IDR.No.Inband.SPPS.mkv in ticket #6418)
> 2. there is no IDR frame in the input stream.
> (No.Inband.SPPS.No.IDR.mkv in ticket #6418)
>
> Both clips could be decoded successfully using software decoding
> but had problems in hwaccel using qsv before applying the patch.
>
> V3: modified the commit message and the code duplication.
>
> Signed-off-by: Linjie Fu
> ---
> libavcodec/h264_mp4toannexb_bsf.c | 9 +++--
> 1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/h264_mp4toannexb_bsf.c
> b/libavcodec/h264_mp4toannexb_bsf.c
> index fb3f24ea40..8dd9159b22 100644
> --- a/libavcodec/h264_mp4toannexb_bsf.c
> +++ b/libavcodec/h264_mp4toannexb_bsf.c
> @@ -33,6 +33,7 @@ typedef struct H264BSFContext {
> int32_t pps_offset;
> uint8_t length_size;
> uint8_t new_idr;
> +uint8_t new_nal_slice;
> uint8_t idr_sps_seen;
> uint8_t idr_pps_seen;
> int extradata_parsed;
> @@ -236,13 +237,17 @@ static int h264_mp4toannexb_filter(AVBSFContext *ctx,
> AVPacket *out)
> if (!s->new_idr && unit_type == H264_NAL_IDR_SLICE && (buf[1] &
> 0x80))
> s->new_idr = 1;
>
> -/* prepend only to the first type 5 NAL unit of an IDR picture, if
> no sps/pps are already present */
> -if (s->new_idr && unit_type == H264_NAL_IDR_SLICE &&
> !s->idr_sps_seen && !s->idr_pps_seen) {
> +/* prepend to the first type 5 NAL unit of an IDR picture,
> + * or the first type 1 NAL unit of an IDR missing picture,
> + * if no sps/pps are already present */
> +if (s->new_idr && !s->idr_sps_seen && !s->idr_pps_seen
> +&& (unit_type == H264_NAL_IDR_SLICE ||
> (!s->new_nal_slice && H264_NAL_SLICE == unit_type))) {
> if ((ret=alloc_and_copy(out,
> ctx->par_out->extradata,
> ctx->par_out->extradata_size,
> buf, nal_size, 1)) < 0)
> goto fail;
> s->new_idr = 0;
> +s->new_nal_slice = 1;
> /* if only SPS has been seen, also insert PPS */
> } else if (s->new_idr && unit_type == H264_NAL_IDR_SLICE &&
> s->idr_sps_seen && !s->idr_pps_seen) {
> if (s->pps_offset == -1) {
>
I'm not really against this change, but it's hard to be sure it doesn't perturb
something else because the h264_mp4toannexb BSF gets used in a lot of places.
It would seem safer to only do this on real recovery points, rather than
blindly splicing it in front of any frame? The frame you actually put it with
here is still unlikely to be usefully decoable if you cut a live stream at a
random point.
As an alternative for your libmfx case, most of the other decoders using
h264_mp4toannexb (CrystalHD, CUVID, Mediacodec, RKMPP; not V4L2M2M) consume the
extradata separately rather than requiring it to be in the stream - if libmfx
could do that then it would already be available and there wouldn't be a need
to splice it into the stream in a questionable place like this.
(Or, since the normal decoder is fine, just use the appropriate hwaccel (+
hwmap for libmfx output if wanted) rather than the legacy libmfx decoders.)
Thanks,
- Mark
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
