On Fri, Oct 25, 2013 at 4:41 PM, Anton Khirnov <[email protected]> wrote:
> diff --git a/libavcodec/hevc.c b/libavcodec/hevc.c
> new file mode 100644
> index 0000000..91ebaac
> --- /dev/null
> +++ b/libavcodec/hevc.
> +static int verify_md5(HEVCContext *s, AVFrame *frame)
> +{
> + const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(frame->format);
> + int pixel_shift = desc->comp[0].depth_minus1 > 7;
> + int i, j;
> +
> + if (!desc)
> + return AVERROR(EINVAL);
I think this popped out on IRC some time ago, but there was no clear
decision on whether something should be done on it.
The ideas were:
- if it should never happen, then let's not check it?
- or let's add a proper averror?
- why av_pix_fmt_desc_get returns a pointer?
In anycase, while we fix this more thoroughly we could just skip this
check or move the pixel_shift initialization after the check.
> +
> + av_log(s->avctx, AV_LOG_DEBUG, "Verifying checksum for frame with POC
> %d: ",
> + s->poc);
> +
> + /* the checksums are LE, so we have to byteswap for >8bpp formats
> + * on BE arches */
> +#if HAVE_BIGENDIAN
> + if (pixel_shift && !s->checksum_buf) {
> + av_fast_malloc(&s->checksum_buf, &s->checksum_buf_size,
> + FFMAX3(frame->linesize[0], frame->linesize[1],
> + frame->linesize[2]));
> + if (!s->checksum_buf)
> + return AVERROR(ENOMEM);
> + }
> +#endif
> +
[..]
> +
> +int ff_hevc_output_frame(HEVCContext *s, AVFrame *out, int flush)
I did a small simplification for this function, please check it out from my repo
https://github.com/kodabb/libav/commit/50cd4982acbfb4697d48ad6ebcc7f0b5874fdccf
Vittorio
_______________________________________________
libav-devel mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-devel
