Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[John P Poet]

On Wed, Jul 5, 2017 at 3:30 PM Marton Balint  wrote:

>
> On Fri, 9 Jun 2017, Devin Heitmueller wrote:
>
> > Hello Marton,
> >
> > On Tue, Jun 6, 2017 at 5:45 PM, Marton Balint  wrote:
> >
> >> As far as I remember multiple side data of the same type is not
> something we
> >> wanted to support. Why do you need it? Can't a single
> AV_FRAME_DATA_A53_CC
> >> side data packet contain many CC entries?
> >
> > Could you please expand on where this was discussed?  Is there any
> > design documentation for side data infrastructure that's been
> > introduced into ffmpeg?  Is there some list of other known design
> > constraints/limitations?
> >
> > While I agree it would be great to simply say that you should never
> > have multiple side data items of the same type, I'm not sure how
> > realistic that is.  It would be helpful if I could better understand
> > the rationale in that thinking.
>
> I guess there are two reasons for prohibiting same type side data:
>
> - existing API implicitly gave the impression that is is not supported
>
> Av_frame_get_side_data gives you no possibility to retrieve multiple side
> data of the same type. API users might assumed that the reason for that is
> because it is not possible/permitted.
>
> - consistency with packet/stream side data types
>
> For stream side data is is already explicitly disallowed, API would be
> inconsistent if multiple side data entries of the same types were possible
> for one kind of side data (frame), but not for the other (stream).
>
> > I'm starting a rather large project which will likely stretch the
> > design for side data in order to support a variety of other ancillary
> > data protocols (e.g. SCTE-104, SMPTE 2038, etc), and it would be great
> > to better understand where the constraints are (so I can either plan
> > to address those issues, or if too significant then choose a different
> > multimedia framework to work with before making a significant
> > investment building out a bunch of features in ffmpeg).
>
> Yeah, you definitely have to think about this, depending on your actual
> goals libavcodec/format might not be the best choice for complex ancillary
> data handling.
>
> It seems multiple side data of the same type is not going to pick up
> support from other developers, so you have to come up with something else
> to move forward.
>

Hi Marton,

If I understand what you and others are saying, AVFrameSideDataType should
not be used to hold "MPEG user data", such as CEA-708, as described here
https://www.atsc.org/wp-content/uploads/2015/03/a_53-Part-4-2009.pdf in
section 6.2.

What I am unclear about, is the proper way to handle MPEG user data.  Are
you saying FFmpeg has no desire to handle this type of data, and that I
need to look to another project?  Or, are you just saying I need to define
a new structure (AVFrameUserDataType or AVFrameMPEGUserDataType) for this
type of data?

Thank you,

John
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[Marton Balint]

On Fri, 9 Jun 2017, Devin Heitmueller wrote:


Hello Marton,

On Tue, Jun 6, 2017 at 5:45 PM, Marton Balint  wrote:


As far as I remember multiple side data of the same type is not something we
wanted to support. Why do you need it? Can't a single AV_FRAME_DATA_A53_CC
side data packet contain many CC entries?


Could you please expand on where this was discussed?  Is there any
design documentation for side data infrastructure that's been
introduced into ffmpeg?  Is there some list of other known design
constraints/limitations?

While I agree it would be great to simply say that you should never
have multiple side data items of the same type, I'm not sure how
realistic that is.  It would be helpful if I could better understand
the rationale in that thinking.


I guess there are two reasons for prohibiting same type side data:

- existing API implicitly gave the impression that is is not supported

Av_frame_get_side_data gives you no possibility to retrieve multiple side 
data of the same type. API users might assumed that the reason for that is 
because it is not possible/permitted.


- consistency with packet/stream side data types

For stream side data is is already explicitly disallowed, API would be 
inconsistent if multiple side data entries of the same types were possible 
for one kind of side data (frame), but not for the other (stream).



I'm starting a rather large project which will likely stretch the
design for side data in order to support a variety of other ancillary
data protocols (e.g. SCTE-104, SMPTE 2038, etc), and it would be great
to better understand where the constraints are (so I can either plan
to address those issues, or if too significant then choose a different
multimedia framework to work with before making a significant
investment building out a bunch of features in ffmpeg).


Yeah, you definitely have to think about this, depending on your actual 
goals libavcodec/format might not be the best choice for complex ancillary 
data handling.


It seems multiple side data of the same type is not going to pick up 
support from other developers, so you have to come up with something else 
to move forward.


Regards,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[John P Poet]

On Fri, Jun 9, 2017 at 11:41 AM Devin Heitmueller <
dheitmuel...@kernellabs.com> wrote:

> Hello Marton,
>
> On Tue, Jun 6, 2017 at 5:45 PM, Marton Balint  wrote:
>
> > As far as I remember multiple side data of the same type is not
> something we
> > wanted to support. Why do you need it? Can't a single
> AV_FRAME_DATA_A53_CC
> > side data packet contain many CC entries?
>
> Could you please expand on where this was discussed?  Is there any
> design documentation for side data infrastructure that's been
> introduced into ffmpeg?  Is there some list of other known design
> constraints/limitations?
>
> While I agree it would be great to simply say that you should never
> have multiple side data items of the same type, I'm not sure how
> realistic that is.  It would be helpful if I could better understand
> the rationale in that thinking.
>
> I'm starting a rather large project which will likely stretch the
> design for side data in order to support a variety of other ancillary
> data protocols (e.g. SCTE-104, SMPTE 2038, etc), and it would be great
> to better understand where the constraints are (so I can either plan
> to address those issues, or if too significant then choose a different
> multimedia framework to work with before making a significant
> investment building out a bunch of features in ffmpeg).
>
> Thanks,
>
> Devin
>

Is there anything I need to be doing, to get this committed?

Thanks,

John
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[Brian Matherly]

Indeed, multiple entries of the same type are really a bad idea and we
basically made them impossible with stream sidedata, although maybe
not with frame side data yet. We should not add API for them or
encourage their use.
If there is a real need for multiple of the same type, maybe the type
should be expanded to hold more information.



The cc_count is only 5 bits, which mean that only 31 3-byte "closed caption
constructs" will fit in a "block".Testing this with 1080i60 material, I
found that 2 or 3 blocks was often necessary to hold all of the CC data.

I tried ignoring that limit of 31 "constructs" per block, and ended up with
corrupt captions.   By preserving the 2 or 3 separate blocks I observed
from the original source, the captions are perfect.

According to CEA-708, in the case of 1080i60, the correct number for 
cc_count is 10. The highest number that cc_count is allowed to be is 30 
in the case of repeating a frame three times for film mode. Exceeding 
the correct cc_count for the frame rate would cause the CC channel data 
rate to exceed 9,600bps.


~Brian
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[Devin Heitmueller]

Hello Marton,

On Tue, Jun 6, 2017 at 5:45 PM, Marton Balint  wrote:

> As far as I remember multiple side data of the same type is not something we
> wanted to support. Why do you need it? Can't a single AV_FRAME_DATA_A53_CC
> side data packet contain many CC entries?

Could you please expand on where this was discussed?  Is there any
design documentation for side data infrastructure that's been
introduced into ffmpeg?  Is there some list of other known design
constraints/limitations?

While I agree it would be great to simply say that you should never
have multiple side data items of the same type, I'm not sure how
realistic that is.  It would be helpful if I could better understand
the rationale in that thinking.

I'm starting a rather large project which will likely stretch the
design for side data in order to support a variety of other ancillary
data protocols (e.g. SCTE-104, SMPTE 2038, etc), and it would be great
to better understand where the constraints are (so I can either plan
to address those issues, or if too significant then choose a different
multimedia framework to work with before making a significant
investment building out a bunch of features in ffmpeg).

Thanks,

Devin

-- 
Devin J. Heitmueller - Kernel Labs
http://www.kernellabs.com
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[John P Poet]

On Tue, Jun 6, 2017 at 4:48 PM John P Poet  wrote:

> On Tue, Jun 6, 2017 at 4:40 PM Kieran Kunhya  wrote:
>
>> >
>> > The cc_count is only 5 bits, which mean that only 31 3-byte "closed
>> caption
>> > constructs" will fit in a "block".Testing this with 1080i60
>> material, I
>> > found that 2 or 3 blocks was often necessary to hold all of the CC data.
>> >
>> > I tried ignoring that limit of 31 "constructs" per block, and ended up
>> with
>> > corrupt captions.   By preserving the 2 or 3 separate blocks I observed
>> > from the original source, the captions are perfect.
>> >
>>
>> Odd, ATSC specifies specific bitrate requirements in this area. Are you
>> sure your insertion process isn't  bursting?
>>
>> Kieran
>>
>
> The source is SDI with embedded 708 captions.  I supposed there may be an
> issue there.  I have not tried this with any other source.
>
> John
>

From my SDI source, most frames have cc_count=20.  Then there can be two or
three frames without any CC packets, which are often followed by a single
frame with with two or three VBI lines of CC packets:  e.g. two or three
groups of cc_count=20 in a single frame.

When you asked if it is "bursting", I assume this is what you mean?  If so,
what would you suggest I do about it?

Thanks,

John
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[John P Poet]

On Tue, Jun 6, 2017 at 4:40 PM Kieran Kunhya  wrote:

> >
> > The cc_count is only 5 bits, which mean that only 31 3-byte "closed
> caption
> > constructs" will fit in a "block".Testing this with 1080i60
> material, I
> > found that 2 or 3 blocks was often necessary to hold all of the CC data.
> >
> > I tried ignoring that limit of 31 "constructs" per block, and ended up
> with
> > corrupt captions.   By preserving the 2 or 3 separate blocks I observed
> > from the original source, the captions are perfect.
> >
>
> Odd, ATSC specifies specific bitrate requirements in this area. Are you
> sure your insertion process isn't  bursting?
>
> Kieran
>

The source is SDI with embedded 708 captions.  I supposed there may be an
issue there.  I have not tried this with any other source.

John
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[Brian Matherly]

Indeed, multiple entries of the same type are really a bad idea and we
basically made them impossible with stream sidedata, although maybe
not with frame side data yet. We should not add API for them or
encourage their use.
If there is a real need for multiple of the same type, maybe the type
should be expanded to hold more information.



The cc_count is only 5 bits, which mean that only 31 3-byte "closed caption
constructs" will fit in a "block".Testing this with 1080i60 material, I
found that 2 or 3 blocks was often necessary to hold all of the CC data.

I tried ignoring that limit of 31 "constructs" per block, and ended up with
corrupt captions.   By preserving the 2 or 3 separate blocks I observed
from the original source, the captions are perfect.

According to CEA-708, in the case of 1080i60, the correct number for 
cc_count is 10. The highest number that cc_count is allowed to be is 30 
in the case of repeating a frame three times for film mode. Exceeding 
the correct cc_count for the frame rate would cause the CC channel data 
rate to exceed 9,600bps.


~Brian
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[Kieran Kunhya]

>
> The cc_count is only 5 bits, which mean that only 31 3-byte "closed caption
> constructs" will fit in a "block".Testing this with 1080i60 material, I
> found that 2 or 3 blocks was often necessary to hold all of the CC data.
>
> I tried ignoring that limit of 31 "constructs" per block, and ended up with
> corrupt captions.   By preserving the 2 or 3 separate blocks I observed
> from the original source, the captions are perfect.
>

Odd, ATSC specifies specific bitrate requirements in this area. Are you
sure your insertion process isn't  bursting?

Kieran
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[John P Poet]

On Tue, Jun 6, 2017 at 3:59 PM Hendrik Leppkes  wrote:

> On Tue, Jun 6, 2017 at 11:45 PM, Marton Balint  wrote:
> >
> > On Tue, 6 Jun 2017, John Poet wrote:
> >
> >> ff_mpeg1_encode_picture_header: Add support for AV_FRAME_DATA_A53_CC
> >> frame: Add av_frame_get_side_data_at() to allow retrival of multiple
> side
> >>data of the same type.
> >
> >
> > As far as I remember multiple side data of the same type is not
> something we
> > wanted to support. Why do you need it? Can't a single
> AV_FRAME_DATA_A53_CC
> > side data packet contain many CC entries?
> >
>
> Indeed, multiple entries of the same type are really a bad idea and we
> basically made them impossible with stream sidedata, although maybe
> not with frame side data yet. We should not add API for them or
> encourage their use.
> If there is a real need for multiple of the same type, maybe the type
> should be expanded to hold more information.
>
>
The cc_count is only 5 bits, which mean that only 31 3-byte "closed caption
constructs" will fit in a "block".Testing this with 1080i60 material, I
found that 2 or 3 blocks was often necessary to hold all of the CC data.

I tried ignoring that limit of 31 "constructs" per block, and ended up with
corrupt captions.   By preserving the 2 or 3 separate blocks I observed
from the original source, the captions are perfect.

If you would like me to go about this a different way, please give me some
direction.

Thank you,

John
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[Hendrik Leppkes]

On Tue, Jun 6, 2017 at 11:45 PM, Marton Balint  wrote:
>
> On Tue, 6 Jun 2017, John Poet wrote:
>
>> ff_mpeg1_encode_picture_header: Add support for AV_FRAME_DATA_A53_CC
>> frame: Add av_frame_get_side_data_at() to allow retrival of multiple side
>>data of the same type.
>
>
> As far as I remember multiple side data of the same type is not something we
> wanted to support. Why do you need it? Can't a single AV_FRAME_DATA_A53_CC
> side data packet contain many CC entries?
>

Indeed, multiple entries of the same type are really a bad idea and we
basically made them impossible with stream sidedata, although maybe
not with frame side data yet. We should not add API for them or
encourage their use.
If there is a real need for multiple of the same type, maybe the type
should be expanded to hold more information.

- Hendrik
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] Add A53 Closed Captions to MPEG header if they are available.

[Marton Balint]

On Tue, 6 Jun 2017, John Poet wrote:


ff_mpeg1_encode_picture_header: Add support for AV_FRAME_DATA_A53_CC
frame: Add av_frame_get_side_data_at() to allow retrival of multiple side
   data of the same type.


As far as I remember multiple side data of the same type is not something 
we wanted to support. Why do you need it? Can't a single 
AV_FRAME_DATA_A53_CC side data packet contain many CC entries?


In general, split API additions and ordinary features into separate 
patches.


Regards,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel