Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Wednesday, August 31, 2022 6:02 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of
> > Anton Khirnov
> > Sent: Wednesday, August 31, 2022 3:40 AM
> > To: FFmpeg development discussions and patches > de...@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering
> 2022
> >
> > Quoting Soft Works (2022-08-27 00:48:01)
> > > 2. "There's no reason why this cannot be handled using the buffer
> > > and data fields"
> > >
> > > I had explained the reasons and in conversation on IRC, Lynne was
> > > no longer insisting on this AFAIR.
> >
> > I did not see this explanation, can you restate it here?
>
You had asked me to restate the explanation.
I did that (below) but you never responded.
Thanks,
softworkz
> Sure. Let's look at the AVSubtitleArea struct first:
>
>
> typedef struct AVSubtitleArea {
>
> enum AVSubtitleType type;
> int flags;
>
> int x; ///< top left corner of area.
> int y; ///< top left corner of area.
> int w; ///< widthof area.
> int h; ///< height of area.
> int nb_colors; ///< number of colors in bitmap palette (@ref
> pal).
>
> /**
> * Buffers and line sizes for the bitmap of this subtitle.
> *
> * @{
> */
> AVBufferRef *buf[AV_NUM_BUFFER_POINTERS];
> int linesize[AV_NUM_BUFFER_POINTERS];
> /**
> * @}
> */
>
> uint32_t pal[256]; ///< RGBA palette for the bitmap.
>
> char *text;///< 0-terminated plain UTF-8 text
> char *ass; ///< 0-terminated ASS/SSA compatible event
> line.
>
> } AVSubtitleArea;
>
>
>
> These structs are stored in AVFrame like this:
>
>
> /**
> * Number of items in the @ref subtitle_areas array.
> */
> unsigned num_subtitle_areas;
>
> /**
> * Array of subtitle areas, may be empty.
> */
> AVSubtitleArea **subtitle_areas;
>
>
>
> The question was "why this cannot be handled using the buffer
> and data fields?" - there are multiple reasons:
>
> 1. Area Count
>
> In ASS subtitles it's not uncommon that animations are defined
> through subtitle events (regular ass events). This can go as
> far as that dust particles are flying around on the screen and
> each particle has its own subtitle event/line which defines
> how it flies from x to y and how fast, etc.
> Not yet, but in a future update, the ass decoder should be
> improved in a way that it doesn't emit an individual AVFrame
> for each event line but bundles subsequent events together
> when these would have the same start time and duration.
> As a result, we can have AVFrames with dozens or even a hundred
> AVSubtitleArea items in extreme cases.
>
> The buffer and data fields are an array of fixed size (currently
> 8). Increasing to 16 or 32 would not help: there will still be
> cases where this does not suffice.
>
> 2. What exactly should be stored in frame->buf[]?
>
> Should we store the AVSubtitleArea structs as AVBuffer there?
>
> Or should we only store data in those buffers? If yes, which
> data? The subtitle bitmap probably. How about the subtitle
> text - maybe and area has even both. And should we also
> store the palette in some frame->buf[n]?
> If yes, how to keep track of which data is stored in which
> buffer?
> Further, there's a documented convention that requires that
> non-zero buffers are contiguous, i.e. there must not be
> an empty buffer pointer between two other a non-empty buffer
> pointers. This would require to re-arrange the buffers to
> close any gap when some subtitle area data is removed, zeroed
> out or has been converted to another format.
> Closing such gap also require to update any mapping information
> ("which buffer is related to which area?")
>
> That's a lot of tedious work and every API user (or codec/filter
> developer) would need to do it in the right way.
>
>
>
> 3. AVFrame Code Logic Mistmatch
>
> A subtitle frame can have 0 to N areas, which means that a
> frame without any area is still valid. Opposed to that, existing
> (code all over the place in ffmpeg) is treating an AVFrame
> as invalid when the first data buffer is empty,
> i.e. frame->buf[0] != NULL
>
&g
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Wednesday, August 31, 2022 3:40 AM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Quoting Soft Works (2022-08-27 00:48:01) > > 2. "There's no reason why this cannot be handled using the buffer > > and data fields" > > > > I had explained the reasons and in conversation on IRC, Lynne was > > no longer insisting on this AFAIR. > > I did not see this explanation, can you restate it here? > > If you claim the other points were addressed I will look at the > patches > again. > > -- > Anton Khirnov > ___ Friendly Ping :-) ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Soft Works
> Sent: Wednesday, August 31, 2022 6:02 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
>
>
>
> > -Original Message-
> > From: ffmpeg-devel On Behalf Of
> > Anton Khirnov
> > Sent: Wednesday, August 31, 2022 3:40 AM
> > To: FFmpeg development discussions and patches > de...@ffmpeg.org>
> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering
> 2022
> >
> > Quoting Soft Works (2022-08-27 00:48:01)
> > > 2. "There's no reason why this cannot be handled using the buffer
> > > and data fields"
> > >
> > > I had explained the reasons and in conversation on IRC, Lynne was
> > > no longer insisting on this AFAIR.
> >
> > I did not see this explanation, can you restate it here?
>
> Sure. Let's look at the AVSubtitleArea struct first:
>
>
> typedef struct AVSubtitleArea {
>
> enum AVSubtitleType type;
> int flags;
>
> int x; ///< top left corner of area.
> int y; ///< top left corner of area.
> int w; ///< widthof area.
> int h; ///< height of area.
> int nb_colors; ///< number of colors in bitmap palette (@ref
> pal).
>
> /**
> * Buffers and line sizes for the bitmap of this subtitle.
> *
> * @{
> */
> AVBufferRef *buf[AV_NUM_BUFFER_POINTERS];
> int linesize[AV_NUM_BUFFER_POINTERS];
> /**
> * @}
> */
>
> uint32_t pal[256]; ///< RGBA palette for the bitmap.
>
> char *text;///< 0-terminated plain UTF-8 text
> char *ass; ///< 0-terminated ASS/SSA compatible event
> line.
>
> } AVSubtitleArea;
>
>
>
> These structs are stored in AVFrame like this:
>
>
> /**
> * Number of items in the @ref subtitle_areas array.
> */
> unsigned num_subtitle_areas;
>
> /**
> * Array of subtitle areas, may be empty.
> */
> AVSubtitleArea **subtitle_areas;
>
>
>
> The question was "why this cannot be handled using the buffer
> and data fields?" - there are multiple reasons:
>
> 1. Area Count
>
> In ASS subtitles it's not uncommon that animations are defined
> through subtitle events (regular ass events). This can go as
> far as that dust particles are flying around on the screen and
> each particle has its own subtitle event/line which defines
> how it flies from x to y and how fast, etc.
> Not yet, but in a future update, the ass decoder should be
> improved in a way that it doesn't emit an individual AVFrame
> for each event line but bundles subsequent events together
> when these would have the same start time and duration.
> As a result, we can have AVFrames with dozens or even a hundred
> AVSubtitleArea items in extreme cases.
>
> The buffer and data fields are an array of fixed size (currently
> 8). Increasing to 16 or 32 would not help: there will still be
> cases where this does not suffice.
>
> 2. What exactly should be stored in frame->buf[]?
>
> Should we store the AVSubtitleArea structs as AVBuffer there?
>
> Or should we only store data in those buffers? If yes, which
> data? The subtitle bitmap probably. How about the subtitle
> text - maybe and area has even both. And should we also
> store the palette in some frame->buf[n]?
> If yes, how to keep track of which data is stored in which
> buffer?
> Further, there's a documented convention that requires that
> non-zero buffers are contiguous, i.e. there must not be
> an empty buffer pointer between two other a non-empty buffer
> pointers. This would require to re-arrange the buffers to
> close any gap when some subtitle area data is removed, zeroed
> out or has been converted to another format.
> Closing such gap also require to update any mapping information
> ("which buffer is related to which area?")
>
> That's a lot of tedious work and every API user (or codec/filter
> developer) would need to do it in the right way.
>
>
>
> 3. AVFrame Code Logic Mistmatch
>
> A subtitle frame can have 0 to N areas, which means that a
> frame without any area is still valid. Opposed to that, existing
> (code all over the place in ffmpeg) is treating an AVFrame
> as invalid when the first data buffer is empty,
> i.e. frame->buf[0] != NULL
>
> To accommodate to this situation, subtitle frames are always
> creating a 1-byte buffer for buf[0].
&g
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message-
> From: ffmpeg-devel On Behalf Of
> Anton Khirnov
> Sent: Wednesday, August 31, 2022 3:40 AM
> To: FFmpeg development discussions and patches de...@ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
>
> Quoting Soft Works (2022-08-27 00:48:01)
> > 2. "There's no reason why this cannot be handled using the buffer
> > and data fields"
> >
> > I had explained the reasons and in conversation on IRC, Lynne was
> > no longer insisting on this AFAIR.
>
> I did not see this explanation, can you restate it here?
Sure. Let's look at the AVSubtitleArea struct first:
typedef struct AVSubtitleArea {
enum AVSubtitleType type;
int flags;
int x; ///< top left corner of area.
int y; ///< top left corner of area.
int w; ///< widthof area.
int h; ///< height of area.
int nb_colors; ///< number of colors in bitmap palette (@ref pal).
/**
* Buffers and line sizes for the bitmap of this subtitle.
*
* @{
*/
AVBufferRef *buf[AV_NUM_BUFFER_POINTERS];
int linesize[AV_NUM_BUFFER_POINTERS];
/**
* @}
*/
uint32_t pal[256]; ///< RGBA palette for the bitmap.
char *text;///< 0-terminated plain UTF-8 text
char *ass; ///< 0-terminated ASS/SSA compatible event line.
} AVSubtitleArea;
These structs are stored in AVFrame like this:
/**
* Number of items in the @ref subtitle_areas array.
*/
unsigned num_subtitle_areas;
/**
* Array of subtitle areas, may be empty.
*/
AVSubtitleArea **subtitle_areas;
The question was "why this cannot be handled using the buffer
and data fields?" - there are multiple reasons:
1. Area Count
In ASS subtitles it's not uncommon that animations are defined
through subtitle events (regular ass events). This can go as
far as that dust particles are flying around on the screen and
each particle has its own subtitle event/line which defines
how it flies from x to y and how fast, etc.
Not yet, but in a future update, the ass decoder should be
improved in a way that it doesn't emit an individual AVFrame
for each event line but bundles subsequent events together
when these would have the same start time and duration.
As a result, we can have AVFrames with dozens or even a hundred
AVSubtitleArea items in extreme cases.
The buffer and data fields are an array of fixed size (currently
8). Increasing to 16 or 32 would not help: there will still be
cases where this does not suffice.
2. What exactly should be stored in frame->buf[]?
Should we store the AVSubtitleArea structs as AVBuffer there?
Or should we only store data in those buffers? If yes, which
data? The subtitle bitmap probably. How about the subtitle
text - maybe and area has even both. And should we also
store the palette in some frame->buf[n]?
If yes, how to keep track of which data is stored in which
buffer?
Further, there's a documented convention that requires that
non-zero buffers are contiguous, i.e. there must not be
an empty buffer pointer between two other a non-empty buffer
pointers. This would require to re-arrange the buffers to
close any gap when some subtitle area data is removed, zeroed
out or has been converted to another format.
Closing such gap also require to update any mapping information
("which buffer is related to which area?")
That's a lot of tedious work and every API user (or codec/filter
developer) would need to do it in the right way.
3. AVFrame Code Logic Mistmatch
A subtitle frame can have 0 to N areas, which means that a
frame without any area is still valid. Opposed to that, existing
(code all over the place in ffmpeg) is treating an AVFrame
as invalid when the first data buffer is empty,
i.e. frame->buf[0] != NULL
To accommodate to this situation, subtitle frames are always
creating a 1-byte buffer for buf[0].
When we would want subtitle data to be stored in those buffers,
every developer would need to be aware about the requirement
to have that dummy buffer in the first array element. When something
is to be stored, the dummy buffer would need to be freed first
before storing the actual data.
And when the last buffer gets deleted, API users would need to
know to create a new dummy buffer.
That fixed size array might be a good fit for image data, but
it's not for subtitle data.
The approach in my patchset is simple, intuitive and everybody
can easily work with it without needing any special knowledge:
unsigned num_subtitle_areas;
AVSubtitleArea **subtitle_areas;
IMHO, this is the most suitable pattern for this situation.
> If you claim the other points were addressed I will look at the
> patches
> again.
Oh cool, that would be great!
Thanks,
softworkz
___
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Quoting Soft Works (2022-08-27 00:48:01) > 2. "There's no reason why this cannot be handled using the buffer > and data fields" > > I had explained the reasons and in conversation on IRC, Lynne was > no longer insisting on this AFAIR. I did not see this explanation, can you restate it here? If you claim the other points were addressed I will look at the patches again. -- Anton Khirnov ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Anton Khirnov > Sent: Friday, August 26, 2022 10:47 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Quoting Soft Works (2022-08-25 00:19:33) > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Jean-Baptiste Kempf > > > Sent: Monday, August 22, 2022 4:39 PM > > > To: ffmpeg-devel@ffmpeg.org > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering > 2022 > > > > > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > > > > Almost exactly identical objections to the basic aspects of the > API > > > were > > > > raised independently by me, Lynne, and Hendrik. > > > > IIUC Soft Works still refuses to address them (though it's not > so > > > easy > > > > to tell in a 200-email thread). > > > > > > OK. I lost the lists of objections, then. > > > > > > -- > > > Jean-Baptiste Kempf - President > > > > > > Could everybody who still has any objection PLEASE name it with > reasoning > > and explain in which way it should be resolved? > > Most of the main objections are mentioned in [1]. As far as I can > tell, none of them were adequately addressed. Hi Anton, thanks a lot for your reply. In fact, all of these were addressed long ago. http://lists.ffmpeg.org/pipermail/ffmpeg-devel/2021-December/288894.html That message you referenced is about 4 issues: 1. "Milliseconds?" I have changed fields to int64_t values in AV_TIME_BASE, please see below. 2. "There's no reason why this cannot be handled using the buffer and data fields" I had explained the reasons and in conversation on IRC, Lynne was no longer insisting on this AFAIR. 3. "I'm not going to accept a field which is instantly deprecated." I have reworked this and there is no longer a field that is instantly deprecated. 3. "As we've discussed multiple times, please merge this into the regular frame PTS field. We already have _2_ necessary stard/end fields." I have addressed this as well already. There are no longer 3 but only 2 fields remaining (int64_t in AV_TIME_BASE): AVFrame.subtitle_timing.start_pts AVFrame.subtitle_timing.duration > Frankly, you write too many words. A good argument about something > like > this should fit in a paragraph. Maybe followed by some extended > explanations and clarifications, but the core of it should be quite > short and right at the top, not buried under heaps of introductory > remarks and examples. And if you cannot boil down your argument to a > few words then maybe it's not a very strong one. It's a complicated matter and after I had seen that the situation wasn't understood, I started to get more into detail. I will try to write less text in the future. > Or maybe people just got tired of repeating the same objections to > the > same patches being submitted again and again. Or maybe people didn't realize that the objections were addressed already? > You are the author of this set, it is _your_ job to keep track of > what has and has not been addressed I do that, and the count of unaddressed objections that I'm aware of is zero. That's why I'm asking whether anybody would still have an objection or might not see her/his objection being addressed adequately. Thanks again, softworkz PS: the one thing that was under discussion on IRC was about why the subtitle start_pts cannot be the same as the AVFrame pts and that was the point I was explaining in so much detail. ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Quoting Soft Works (2022-08-25 00:19:33) > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Jean-Baptiste Kempf > > Sent: Monday, August 22, 2022 4:39 PM > > To: ffmpeg-devel@ffmpeg.org > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > > > Almost exactly identical objections to the basic aspects of the API > > were > > > raised independently by me, Lynne, and Hendrik. > > > IIUC Soft Works still refuses to address them (though it's not so > > easy > > > to tell in a 200-email thread). > > > > OK. I lost the lists of objections, then. > > > > -- > > Jean-Baptiste Kempf - President > > > Could everybody who still has any objection PLEASE name it with reasoning > and explain in which way it should be resolved? Most of the main objections are mentioned in [1]. As far as I can tell, none of them were adequately addressed. > I wasn't refusing to make a change, but I have taken a lot of effort to > explain the reasons for that necessity. > I did that in several chats on IRC, on the ML, and recently, I have > written > an article especially to address that concern and better explain the > background: > > https://github.com/softworkz/SubtitleFilteringDemos/issues/1 > > It remained unresponded (but maybe unnoticed?). Sorry, but all explanations I've seen from you are walls of text from which I was never able to extract a solid argument that is not circular and does not leap to unsupported conclusions. And it is not just me, as far as I can tell, none of the others were convinced either. Frankly, you write too many words. A good argument about something like this should fit in a paragraph. Maybe followed by some extended explanations and clarifications, but the core of it should be quite short and right at the top, not buried under heaps of introductory remarks and examples. And if you cannot boil down your argument to a few words then maybe it's not a very strong one. > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > ... > > (though it's not so easy to tell in a 200-email thread) > > Yes that's true. For that reason it is not helpful to talk about > unspecified objections from more than half a year ago. You are the author of this set, it is _your_ job to keep track of what has and has not been addressed. And if you want reviews, you should also try to make things easy to review. > This is not further actionable without having a list of specific objections. > When nobody responds, we need to assume that there aren't any left. Or maybe people just got tired of repeating the same objections to the same patches being submitted again and again. [1] http://lists.ffmpeg.org/pipermail/ffmpeg-devel/2021-December/288894.html Message-Id -- Anton Khirnov ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Jean-Baptiste Kempf > Sent: Monday, August 22, 2022 4:39 PM > To: ffmpeg-devel@ffmpeg.org > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > > Almost exactly identical objections to the basic aspects of the API > were > > raised independently by me, Lynne, and Hendrik. > > IIUC Soft Works still refuses to address them (though it's not so > easy > > to tell in a 200-email thread). > > OK. I lost the lists of objections, then. > > -- > Jean-Baptiste Kempf - President Could everybody who still has any objection PLEASE name it with reasoning and explain in which way it should be resolved? > On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: ... > (though it's not so easy to tell in a 200-email thread) Yes that's true. For that reason it is not helpful to talk about unspecified objections from more than half a year ago. This is not further actionable without having a list of specific objections. When nobody responds, we need to assume that there aren't any left. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
From: ffmpeg-devel on behalf of Soft Works Sent: Tuesday, August 23, 2022 12:08 AM To: FFmpeg development discussions and patches Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 From: ffmpeg-devel on behalf of Anton Khirnov Sent: Monday, August 22, 2022 2:18 PM To: ffmpeg-devel Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 Quoting Jean-Baptiste Kempf (2022-08-21 12:41:20) > > On Sun, 21 Aug 2022, at 11:41, Paul B Mahol wrote: > > We should more forward and merge this considerable subtitle work > > Are there parts of this work that have reach majority consensus? Almost exactly identical objections to the basic aspects of the API were raised independently by me, Lynne, and Hendrik. IIUC Soft Works still refuses to address them (though it's not so easy to tell in a 200-email thread). --- Anton, thanks for the reply. Please correct me if I'm wrong. From my memory and understanding, the one any only remaining point of discussion was the necessity to have a separate field for subtitle start PTS in addition to the AVFrame's PTS field. I wasn't refusing to make a change, but I have taken a lot of effort to explain the reasons for that necessity. I did that in several chats on IRC, on the ML, and recently, I have written an article especially to address that concern and better explain the background: https://github.com/softworkz/SubtitleFilteringDemos/issues/1 It remained unresponded (but maybe unnoticed?). Bottom line is that without having the separate subtitle pts field, the whole patchset cannot work. @Anton: you said the reason for this is because I would have designed it like this. But this not the case. The reason why this field is needed it due to the way how libavfilter is designed to work. The only way to avoid that second field would be to fundamentally rework the scheduling of frames in filter graphs and making changes to a core part like that (which is working pretty well and reliably btw) would impose a huge risk of regressions and incompatibilities (it's more a guarantee for issues rather than an abstract risk only), which doesn't make any sense to do at this time and in this context. My conclusion is that having that one additional field in AVFrame is by far the better option. Please let me know what you think should be done and what kind of concern you have with regards to the additional field in AVFrame. It might not qualify as a top candidate for "API Beauty", but it's none of the worst ones either. Your preceding arguments were based on the assumption that it could easily be avoided. I hope the referenced article helps to understand why it can't (without fundamental changes to libavfilter). Please also let me know whether there are any other objections or desired changes and I'll try to address them. I am in no way refusing to make changes, as long as they are feasible. Thanks, softworkz Other alternatives that were discussed: 1. Move the SubtitleStartTime field to AVSubtitleArea One AVFrame can have multiple AVSubtitleArea instances (bitmaps in case of graphic subs or text events in case of text subs). But all AVSubtitleArea instances of an AVFrame must always have the same start and duration - so it would be semantically incorrect to have those values at the area level (when a frame has multiple areas, with different timings, which value is valid? The ones from area0 and the values from area1 to areaN are irrelevant?) 2. Put the SubtitleStartTime into AVFrame's user/opaque data field This is what Lynne had suggested. It avoids adding an additional field to AVFrame but it blocks the use of the user/opaque field for actual user data in case of subtitle frames. Also, each subtitle decoder, each subtitle encoder and many subtitle filters would need to cast the user/opaque field of AVFrame back and forth to a time value. Neither of the two options does make much sense to me, but when there would be consensus on any of those, then I'd be ok with it and make those changes. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
From: ffmpeg-devel on behalf of Anton Khirnov Sent: Monday, August 22, 2022 2:18 PM To: ffmpeg-devel Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 Quoting Jean-Baptiste Kempf (2022-08-21 12:41:20) > > On Sun, 21 Aug 2022, at 11:41, Paul B Mahol wrote: > > We should more forward and merge this considerable subtitle work > > Are there parts of this work that have reach majority consensus? Almost exactly identical objections to the basic aspects of the API were raised independently by me, Lynne, and Hendrik. IIUC Soft Works still refuses to address them (though it's not so easy to tell in a 200-email thread). --- Anton, thanks for the reply. Please correct me if I'm wrong. From my memory and understanding, the one any only remaining point of discussion was the necessity to have a separate field for subtitle start PTS in addition to the AVFrame's PTS field. I wasn't refusing to make a change, but I have taken a lot of effort to explain the reasons for that necessity. I did that in several chats on IRC, on the ML, and recently, I have written an article especially to address that concern and better explain the background: https://github.com/softworkz/SubtitleFilteringDemos/issues/1 It remained unresponded (but maybe unnoticed?). Bottom line is that without having the separate subtitle pts field, the whole patchset cannot work. @Anton: you said the reason for this is because I would have designed it like this. But this not the case. The reason why this field is needed it due to the way how libavfilter is designed to work. The only way to avoid that second field would be to fundamentally rework the scheduling of frames in filter graphs and making changes to a core part like that (which is working pretty well and reliably btw) would impose a huge risk of regressions and incompatibilities (it's more a guarantee for issues rather than an abstract risk only), which doesn't make any sense to do at this time and in this context. My conclusion is that having that one additional field in AVFrame is by far the better option. Please let me know what you think should be done and what kind of concern you have with regards to the additional field in AVFrame. It might not qualify as a top candidate for "API Beauty", but it's none of the worst ones either. Your preceding arguments were based on the assumption that it could easily be avoided. I hope the referenced article helps to understand why it can't (without fundamental changes to libavfilter). Please also let me know whether there are any other objections or desired changes and I'll try to address them. I am in no way refusing to make changes, as long as they are feasible. Thanks, softworkz PS: Please excuse potentially bad e.mail formatting, I'm off-site ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Mon, 22 Aug 2022, at 14:18, Anton Khirnov wrote: > Almost exactly identical objections to the basic aspects of the API were > raised independently by me, Lynne, and Hendrik. > IIUC Soft Works still refuses to address them (though it's not so easy > to tell in a 200-email thread). OK. I lost the lists of objections, then. -- Jean-Baptiste Kempf - President +33 672 704 734 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Quoting Jean-Baptiste Kempf (2022-08-21 12:41:20) > > On Sun, 21 Aug 2022, at 11:41, Paul B Mahol wrote: > > We should more forward and merge this considerable subtitle work > > Are there parts of this work that have reach majority consensus? Almost exactly identical objections to the basic aspects of the API were raised independently by me, Lynne, and Hendrik. IIUC Soft Works still refuses to address them (though it's not so easy to tell in a 200-email thread). -- Anton Khirnov ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sun, 21 Aug 2022, at 11:41, Paul B Mahol wrote: > We should more forward and merge this considerable subtitle work Are there parts of this work that have reach majority consensus? -- Jean-Baptiste Kempf - President +33 672 704 734 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Paul B Mahol (12022-08-21): > We should more forward and merge this considerable subtitle work instead of > waiting for broken promises from other inactive developers. > Then push to your fork. -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Fri, Aug 12, 2022 at 12:51 AM Soft Works wrote: > > > > -Original Message- > > From: ffmpegagent > > Sent: Saturday, June 25, 2022 11:58 AM > > To: ffmpeg-devel@ffmpeg.org > > Cc: Michael Niedermayer ; Andreas Rheinhardt > > ; Soft Works ; > > Andriy Gelman ; softworkz > > > > Subject: [PATCH v5 00/25] Subtitle Filtering 2022 > > > > > > Subtitle Filtering 2022 > > === > > > > This is a substantial update to the earlier subtitle filtering patch > > series. > > A primary goal has been to address others' concerns as much as > > possible on > > one side and to provide more clarity and control over the way things > > are > > working. Clarity is is specifically important to allow for a better > > understanding of the need for a subtitle start pts value that can be > > different from the frame's pts value. This is done by refactoring the > > subtitle timing fields in AVFrame, adding a frame field to indicate > > repeated > > subtitle frames, and finally the full removal of the heartbeat > > functionality, replaced by a new 'subfeed' filter that provides > > different > > modes for arbitrating subtitle frames in a filter graph. Finally, > > each > > subtitle filter's documentation has been amended by a section > > describing the > > filter's timeline behavior (in v3 update). > > > > > > Subtitle Filtering Demos > > > > > > I published a demonstration of subtitle filtering capabilities with > > OCR, > > text and bitmap subtitle manipulation involved: Demo 1: Text- > > Manipulation > > with Bitmap Subtitles > > [https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo > > 1] > > > As there were no objections in any regard, I hope that we could make > some progress regarding subtitle filtering soon. > > The only recent statement that has been made was something like > > "this may work in a limited number of cases but probably won't be > capable to work for the whole range of real-life situations" > > I understand how easily such statements can induce doubt into the > audience, even on those who tend and try to be neutral in their > assessments. I also think that replying once another time with > something like "no, this isn't true" won't be much impressive. > > > Instead, I'm sharing a video with you, documenting my internal > test runs for text subtitle overlay (with and without subtitle > filtering, with and without hw overlay, with a range of hw accelerations > for encoding and decoding, scaling or no scaling and two different > kinds of source files). > > The diagrams at the right are a rather consumer focused, but still > accurate. Just hwupload/download/hwmap are omitted and implied > by the data changing the "swim lanes". > > https://github.com/softworkz/SubtitleFilteringDemos/tree/master/TestRun1 > > Commands and logs on request. > > We should more forward and merge this considerable subtitle work instead of waiting for broken promises from other inactive developers. > Best wishes, > softworkz > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpegagent > Sent: Saturday, June 25, 2022 11:58 AM > To: ffmpeg-devel@ffmpeg.org > Cc: Michael Niedermayer ; Andreas Rheinhardt > ; Soft Works ; > Andriy Gelman ; softworkz > > Subject: [PATCH v5 00/25] Subtitle Filtering 2022 > > > Subtitle Filtering 2022 > === > > This is a substantial update to the earlier subtitle filtering patch > series. > A primary goal has been to address others' concerns as much as > possible on > one side and to provide more clarity and control over the way things > are > working. Clarity is is specifically important to allow for a better > understanding of the need for a subtitle start pts value that can be > different from the frame's pts value. This is done by refactoring the > subtitle timing fields in AVFrame, adding a frame field to indicate > repeated > subtitle frames, and finally the full removal of the heartbeat > functionality, replaced by a new 'subfeed' filter that provides > different > modes for arbitrating subtitle frames in a filter graph. Finally, > each > subtitle filter's documentation has been amended by a section > describing the > filter's timeline behavior (in v3 update). > > > Subtitle Filtering Demos > > > I published a demonstration of subtitle filtering capabilities with > OCR, > text and bitmap subtitle manipulation involved: Demo 1: Text- > Manipulation > with Bitmap Subtitles > [https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo > 1] As there were no objections in any regard, I hope that we could make some progress regarding subtitle filtering soon. The only recent statement that has been made was something like "this may work in a limited number of cases but probably won't be capable to work for the whole range of real-life situations" I understand how easily such statements can induce doubt into the audience, even on those who tend and try to be neutral in their assessments. I also think that replying once another time with something like "no, this isn't true" won't be much impressive. Instead, I'm sharing a video with you, documenting my internal test runs for text subtitle overlay (with and without subtitle filtering, with and without hw overlay, with a range of hw accelerations for encoding and decoding, scaling or no scaling and two different kinds of source files). The diagrams at the right are a rather consumer focused, but still accurate. Just hwupload/download/hwmap are omitted and implied by the data changing the "swim lanes". https://github.com/softworkz/SubtitleFilteringDemos/tree/master/TestRun1 Commands and logs on request. Best wishes, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Mon, Jul 25, 2022 at 09:01:03PM +0200, Nicolas George wrote: [...] > > If we add a XML parser to FFmpeg. Such a thing would be used by several > > bits and we need to ensure it has redundant maintainership. > > So i think a xml parser needs broad support by developers otherwise we > > run the risk that if it has a single maintainer only and that maintainer > > stops maintaining it that could be annoying to more than just the parser > > itself. > > I cannot remember exactly without re-reading the old thread what the issues > > people had with the xml parser. If it was some component of it or in > > general. > > But i think its mostly a question if theres more than one person who wants > > to maintain it. > > There is nothing to re-read: > > https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/thread.html#296107 > > Nobody answered. Even to send files to make sure they would be > supported. that says "WIP" in the subject. I think some people when they see WIP they skip it and wait for a final version to review. thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB The real ebay dictionary, page 2 "100% positive feedback" - "All either got their money back or didnt complain" "Best seller ever, very honest" - "Seller refunded buyer after failed scam" signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Michael Niedermayer (12022-07-25): > I think many if us, myself included have had un-fun periods in ffmpeg. > This on its own is already bad and we should strive to make project > contributions more fun to all. In my opinion, it would require at least two things: - That the members of the Community Committee step in more proactively in budding conflicts. And take sides: not “both be nice” but “you started it, shut up”. - A person or group of person capable of deciding ahead if a change that will affect every body is worth investing time implementing it. > If we add a XML parser to FFmpeg. Such a thing would be used by several > bits and we need to ensure it has redundant maintainership. > So i think a xml parser needs broad support by developers otherwise we > run the risk that if it has a single maintainer only and that maintainer > stops maintaining it that could be annoying to more than just the parser > itself. > I cannot remember exactly without re-reading the old thread what the issues > people had with the xml parser. If it was some component of it or in general. > But i think its mostly a question if theres more than one person who wants > to maintain it. There is nothing to re-read: https://ffmpeg.org/pipermail/ffmpeg-devel/2022-May/thread.html#296107 Nobody answered. Even to send files to make sure they would be supported. We would need reliable maintainers, you are right. But right now, we rely on libxml2, and that is not ideal: it is not a standard library on non-Linux systems, and even if it seems better nowadays, it used to be a “security issue of the week” kind of library. > If you work on libavfilter, noone will snatch it from you. If you > dont work on it, eventually someone else will attempt to take over and > push libavfilter in the direction he feels best which may or may not > match what you want. I will try to motivate myself to advance the FATE coverage, which is the blocking issue. Thanks, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Ronald S. Bultje (12022-07-25): > We still do this, but currently outside FFmpeg (dav1d). So sadly, it's a > bit more out of the sphere of this mailinglist. So we take risks, and if > AV2-9 comes around, we might again. But we still consider ourselves part of > the FFmpeg community, and I still try to review ffvp9 patches when they > come up every once in a while. Don't lose hope, just find the positive > things and work on them. Ignore the rest. I know I did. I'm having fun > again. > > I don't know if I should offer advice, but my $.02: maybe adding tests (I > don't mean ffmpeg invocations; some people would call this desired > outcomes) might help here. You probably remember how Michael tested patches > pre-FATE: for a particular patch, he'd send an FFmpeg commandline that > either A) gives (undesirable) different output before vs. after patch, or > B) should give some correct/desired output after the patch but (still) > doesn't (and this second would be what I'm inviting you to do). Tests don't > have to be scripted, they can simply be a list of features or behaviors > desired in the new design. It's true that perfection is the enemy of > progress, but I think you're right that we should try to strive for further > improvement if it is within reach, especially for base design or things > with API implications. (FFmpeg's API is a testament to poor design in some > places.) You're likely better at making a comprehensive list than anyone > else. Making the list public/explicit means other people can help > accomplish the full list of features. Thank you for these kind and encouraging words. The fact that the new developments happen in separate project kinds of confirms my impression that the project has become afraid of taking risks. It makes me think of these artists who found a rich patron and, dazzled by the money and luxury, become afraid of offending them and have their art lose its edge. FFmpeg is so eager to please our corporate users, to look “serious”, to look “professional” that sometimes it almost feels like grovelling, and it stifles novelty. Another symptom of this I see is that Michael spends most of his time managing releases and fuzzing and fixing trivial bugs, tasks that are way below his skills and talent, instead of writing new code. Or maybe I am just wrong and that is what he likes to do most. As for me… I am not good at shaving cycles from inner loops, which is the heart of FFmpeg's superiority, I cannot contribute to it. What I think I can contribute (and thank you for suggesting I am not wrong) is in areas of API and architecture. Unfortunately, while support for a new codec or optimization on an existing one will receive no objection on principle, work on API or architecture affect everybody and therefore is subject to much more discussion, possibly to the point of deadly bikeshedding or outright rejection. That makes it hard to invest time in it without some kind of a-priori assurance. Unfortunately my attempts to spark discussions have been met with indifference: https://ffmpeg.org/pipermail/ffmpeg-devel/2020-December/274167.html https://ffmpeg.org/pipermail/ffmpeg-devel/2020-December/thread.html#274167 (I realize I could probably work on internal error messages. As for event loop, the other one that has received positive feedback, I was trying to find a more efficient way out of a small quandary, I should probably go back to it.) Or even worse: https://ffmpeg.org/pipermail/ffmpeg-devel/2021-December/290226.html https://ffmpeg.org/pipermail/ffmpeg-devel/2021-December/thread.html#290226 (A good string API is a pre-requisite for several other projects.) What I think this project need is a leader, or a leading committee: somebody who can decide beforehand if a change in API or architecture is worth it, and therefore worth the time of the developer, with a decision that is binding to the project: when the code is written, other developers may discuss technical details but not reject the change itself. Maybe the technical committee could endorse this role, even though it was not exactly why it was elected. Thanks, -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sun, Jul 24, 2022 at 05:10:17PM +0200, Nicolas George wrote: > I hesitated a long time before replying, but all considering, at this > point expressing what is weighing on my heart cannot make things worse. > > > Michael Niedermayer (12022-07-03): > > What is the timeline for the audio+video merge ? > > I cannot give a timeline because I do not work on FFmpeg on a schedule, > I work on FFmpeg in my free time, for fun. And lately, working on FFmpeg > has been really un-fun. Recently, my contributions have been met with I think many if us, myself included have had un-fun periods in ffmpeg. This on its own is already bad and we should strive to make project contributions more fun to all. > indifference at best (see when I proposed a XML parser six months ago: > nobody cared), outright hostility at worst. Under these circumstances, > whenever I am considering working on something FFmpeg-related, I usually > find something else more fun to do. If we add a XML parser to FFmpeg. Such a thing would be used by several bits and we need to ensure it has redundant maintainership. So i think a xml parser needs broad support by developers otherwise we run the risk that if it has a single maintainer only and that maintainer stops maintaining it that could be annoying to more than just the parser itself. I cannot remember exactly without re-reading the old thread what the issues people had with the xml parser. If it was some component of it or in general. But i think its mostly a question if theres more than one person who wants to maintain it. [...] > It illustrates what it means to be a maintainer. It does not only mean > the task of reviewing and applying bug fixes. The maintainer holds in > their head a knowledge of the code that cannot be fully shared in > writing. The maintainer also holds in their head plans to evolve and > extend the code. > > I have plans for libavfilter. Not just vague ideas, but precise plans on > how to reach the goal. I have plans for subtitles filtering, of course. > But not only. > > I have plans for filtering data packets, so that bistream filters do not > need to have a separate and redundant API and ffmpeg does not need a > separate code path for -c copy. > > I have plans for threading, or more precisely integrating filters in a > global parallelized task and I/O scheduler. > > I have plans for seeking, with the seek target going back from the > output to the input(s). > > I have plans for partial graph reconfiguration, to accommodate format > changes mid-stream and let applications alter filtering in the middle. > > All of this is exciting. I am eager to work on any of this. > > Unfortunately, before it can happen, boring things need to be done. > Parts of the framework of libavfilter are very fragile. Working on any > of these is likely to break cases that were specifically fixed in the > past. > > I can work on boring things if they are necessary to reach the exciting > parts. > > What I cannot do is motivate myself to work on the boring things with > the threat that the exciting things will be snatched under me by an > inferior version from somebody who just refuses to engage with the > boring necessary things. If you work on libavfilter, noone will snatch it from you. If you dont work on it, eventually someone else will attempt to take over and push libavfilter in the direction he feels best which may or may not match what you want. If theres something i can do to make libavfilter work more fun for you please tell me. I think it would be best if people worked a bit more together on this. The current situation where work is split between your vission and this patchset is kind of a bad thing. Its a bit unfortunate noone involved seems to have the carismatic leader skills to pull everyone on his side and then get that side to the point that everyone is happy with the code and neither has a consensus emerged based on technical arguments. I think maybe more a "what is best for the project to move forward" and a less "how can i get my hard work in first" approuch might help but we can also try the technical commiitee of course but iam not sure how much that would really help, it would be better if some consensus would be reached and everyone would then work together on implementing that theres of course also the fork and merge approuch. Each side does whatever they like in their repository and then we at some point compare and merge one or both into ffmpeg. Iam not sure thats a good idea or not. thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Old school: Use the lowest level language in which you can solve the problem conveniently. New school: Use the highest level language in which the latest supercomputer can solve the problem without the user falling asleep waiting. signature.asc Description: PGP signature ___ ffmpeg-devel mailing list
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Hi, On Sun, Jul 24, 2022 at 11:10 PM Nicolas George wrote: > I do not recognize the project I started contributing to more than > fifteen years ago. I do not even recognize the project that boasted the > clever optimization framework that made FFVP9 possible, it has become > increasingly hostile to trying new and more efficient ways of doing > things in favor of a corporate never-take-risks style of coding. > We still do this, but currently outside FFmpeg (dav1d). So sadly, it's a bit more out of the sphere of this mailinglist. So we take risks, and if AV2-9 comes around, we might again. But we still consider ourselves part of the FFmpeg community, and I still try to review ffvp9 patches when they come up every once in a while. Don't lose hope, just find the positive things and work on them. Ignore the rest. I know I did. I'm having fun again. I don't know if I should offer advice, but my $.02: maybe adding tests (I don't mean ffmpeg invocations; some people would call this desired outcomes) might help here. You probably remember how Michael tested patches pre-FATE: for a particular patch, he'd send an FFmpeg commandline that either A) gives (undesirable) different output before vs. after patch, or B) should give some correct/desired output after the patch but (still) doesn't (and this second would be what I'm inviting you to do). Tests don't have to be scripted, they can simply be a list of features or behaviors desired in the new design. It's true that perfection is the enemy of progress, but I think you're right that we should try to strive for further improvement if it is within reach, especially for base design or things with API implications. (FFmpeg's API is a testament to poor design in some places.) You're likely better at making a comprehensive list than anyone else. Making the list public/explicit means other people can help accomplish the full list of features. <3 Ronald ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Sunday, July 24, 2022 5:10 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > I suspect that if I were to do a full review, I would find a few > other flaws. This is a very good example for the kind of behavior I'd kindly ask you to stop with. Do you think you have found a flaw? === => please name it and explain it, ideally asking before calling it a "flaw" (like your one "flaw" wasn't really one as I had laid out) Do you think you COULD find a flaw? === => Go ahead and find one - no need to tell Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Sunday, July 24, 2022 5:10 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > I hesitated a long time before replying, but all considering, at this > point expressing what is weighing on my heart cannot make things > worse. > > > Michael Niedermayer (12022-07-03): > > What is the timeline for the audio+video merge ? > > I cannot give a timeline because I do not work on FFmpeg on a > schedule, > I work on FFmpeg in my free time, for fun. And lately, working on > FFmpeg > has been really un-fun. Recently, my contributions have been met with > indifference at best (see when I proposed a XML parser six months > ago: > nobody cared), outright hostility at worst. Under these > circumstances, > whenever I am considering working on something FFmpeg-related, I > usually > find something else more fun to do. > > I do not recognize the project I started contributing to more than > fifteen years ago. I do not even recognize the project that boasted > the > clever optimization framework that made FFVP9 possible, it has become > increasingly hostile to trying new and more efficient ways of doing > things in favor of a corporate never-take-risks style of coding. I am > more and more often considering giving up and cutting my losses. > > > IIUC this would resolve this deadlock (with extra work adapting the > patchset > > so it would be work for SW adapting it and it would be work for you > finishing > > the merge) > > Also can others help nicolas moving his work forward > > > > What i suggest is to pick a time and then try to finish the merge > before. > > If it succeeds this patchset needs updating and can move forward > without > > this main objection > > OTOH if the time is not hit, we agree that the objection can no > longer be > > used > > My answer as maintainer of the framework of libavfilter is: no. > > Of course, maintainers are not dictators. The members of the project > can > collectively decide otherwise. But I have to warn you about the > consequences. > > First, the issue about negotiation is not he only severe flaw in this > patch series. Negotiation hasn't been implemented for audio+video yet. Neither does that patchset do it for audio+video+subtitles. It is out of scope of this patchset. It can be done later or never, not everybody is a fan of doing so, as comments have shown. Clearly, this is in no way a showstopping reason as you had conceded yourself recently. > I can immediately quote another one: for text > subtitles, > the approach of this proposal to synchronization is to feed > everything > to libass as it comes and see what comes out. It will work on easy > cases, when the subtitles are interleaved with the video or come from > a separate file. - or come from decoded closed captions - or come from graphic subtitles converted with the graphicsub2text filter - or come from the subfeed filter after fixing durations - or come from the subfeed filter ensuring a regular repetition (heartbeat) Subtitle events don't need to come in linear order. Multiple events can have identical start times, subtitle events can overlap. The overlaytextsubs filter is meant to be a direct replacement for the existing subtitles filter, which performs additional opening, parsing and decoding of the source file in parallel, and avoiding that was one of the primary objectives I had for starting development. That's why it was very important for me to preserve the exact same behavior as the overlaytextsubs filter exposes. Other approaches for implementatino are surely possible as well. Traian, who did the text2graphicsub filter had initially an implementation that handled the timing manually instead of letting libass do it, but it turned out that this can quickly become a really complex task, especially when overlapping events or animations are part of the game, so it came down to feeding everything to libass in the end, like the overlaytextsubs filter and the subtitles filters do. The nice thing about having subtitle filtering is that there is no fixed functionality involved where you can argue about right or wrong: anyone is free to contribute another filter which is pursuing a different approach. I would welcome that and there may be cases where an alternative method could be advantageous, but it surely won't be superior in general. > But as soon as a filter will, for example, adjust the > timing to make the subtitles more early, it will just not work. Of > course, it was not tested because this patch series does not even > offer the feature to
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
I hesitated a long time before replying, but all considering, at this point expressing what is weighing on my heart cannot make things worse. Michael Niedermayer (12022-07-03): > What is the timeline for the audio+video merge ? I cannot give a timeline because I do not work on FFmpeg on a schedule, I work on FFmpeg in my free time, for fun. And lately, working on FFmpeg has been really un-fun. Recently, my contributions have been met with indifference at best (see when I proposed a XML parser six months ago: nobody cared), outright hostility at worst. Under these circumstances, whenever I am considering working on something FFmpeg-related, I usually find something else more fun to do. I do not recognize the project I started contributing to more than fifteen years ago. I do not even recognize the project that boasted the clever optimization framework that made FFVP9 possible, it has become increasingly hostile to trying new and more efficient ways of doing things in favor of a corporate never-take-risks style of coding. I am more and more often considering giving up and cutting my losses. > IIUC this would resolve this deadlock (with extra work adapting the patchset > so it would be work for SW adapting it and it would be work for you finishing > the merge) > Also can others help nicolas moving his work forward > > What i suggest is to pick a time and then try to finish the merge before. > If it succeeds this patchset needs updating and can move forward without > this main objection > OTOH if the time is not hit, we agree that the objection can no longer be > used My answer as maintainer of the framework of libavfilter is: no. Of course, maintainers are not dictators. The members of the project can collectively decide otherwise. But I have to warn you about the consequences. First, the issue about negotiation is not he only severe flaw in this patch series. I can immediately quote another one: for text subtitles, the approach of this proposal to synchronization is to feed everything to libass as it comes and see what comes out. It will work on easy cases, when the subtitles are interleaved with the video or come from a separate file. But as soon as a filter will, for example, adjust the timing to make the subtitles more early, it will just not work. Of course, it was not tested because this patch series does not even offer the feature to adjust the time of subtitles, which is frankly ridiculous, it is one of the most obvious thing people might want to do. Note that I did not have to perform a full review of the patch series to find this flaw. I have been preparing to implement subtitles filtering for years now, I know which aspects are tricky and hard to implement properly. I only had to check precisely how it was done. And it turns out it was not done at all. I suspect that if I were to do a full review, I would find a few other flaws. But the author has made painfully clear that they did not respect my expertise in this area, I have no reason to waste my time doing it. It illustrates what it means to be a maintainer. It does not only mean the task of reviewing and applying bug fixes. The maintainer holds in their head a knowledge of the code that cannot be fully shared in writing. The maintainer also holds in their head plans to evolve and extend the code. I have plans for libavfilter. Not just vague ideas, but precise plans on how to reach the goal. I have plans for subtitles filtering, of course. But not only. I have plans for filtering data packets, so that bistream filters do not need to have a separate and redundant API and ffmpeg does not need a separate code path for -c copy. I have plans for threading, or more precisely integrating filters in a global parallelized task and I/O scheduler. I have plans for seeking, with the seek target going back from the output to the input(s). I have plans for partial graph reconfiguration, to accommodate format changes mid-stream and let applications alter filtering in the middle. All of this is exciting. I am eager to work on any of this. Unfortunately, before it can happen, boring things need to be done. Parts of the framework of libavfilter are very fragile. Working on any of these is likely to break cases that were specifically fixed in the past. I can work on boring things if they are necessary to reach the exciting parts. What I cannot do is motivate myself to work on the boring things with the threat that the exciting things will be snatched under me by an inferior version from somebody who just refuses to engage with the boring necessary things. If this patch series gets applied, it will make the boring things a lot harder to do and it will ruin some of the plans I mentioned above. Under these circumstances, do not expect me to work on libavfilter again any time soon, even if it is to apply an obviously valid fix for an exploitable security issue. So, the choice is: - Apply this patch series, find a new maintainer for libavfilter
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sun, 3 Jul 2022, at 12:42, Paul B Mahol wrote: >> But here, the discussion goes nowhere "I'm right", "No, I'm right", "No, >> I'm correct" and will only evolve in insults and slander. >> > Why should I do this work? > > I will just create my own fork. (Already did.) History shows that it's very hard to do a successful FFmpeg fork. Working together is more powerful, IMVHO. -- Jean-Baptiste Kempf - President +33 672 704 734 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sun, Jul 3, 2022 at 7:07 PM Michael Niedermayer wrote: > On Sat, Jul 02, 2022 at 07:18:16PM +0200, Nicolas George wrote: > > Paul B Mahol (12022-07-02): > > > Can this be properly finally be fully reviewed and accepted? > > > > As long as the patch does not have a solution to have all the utility > > filters (setpts, trim, concat, etc.) working with subtitles too without > > more code duplication, the review is just "NAK". > > What is the timeline for the audio+video merge ? > Eternity. > IIUC this would resolve this deadlock (with extra work adapting the > patchset > so it would be work for SW adapting it and it would be work for you > finishing > the merge) > Also can others help nicolas moving his work forward > > What i suggest is to pick a time and then try to finish the merge before. > If it succeeds this patchset needs updating and can move forward without > this main objection > OTOH if the time is not hit, we agree that the objection can no longer be > used > > nothing above is meant as a objection to anything, just my oppinion. Some > thought > i had which i hope could move this forward. > If people agree to anything else iam happy with that too, this mail is only > meant to help! > > thx > > [...] > > -- > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB > > Observe your enemies, for they first find out your faults. -- Antisthenes > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sat, Jul 02, 2022 at 07:18:16PM +0200, Nicolas George wrote: > Paul B Mahol (12022-07-02): > > Can this be properly finally be fully reviewed and accepted? > > As long as the patch does not have a solution to have all the utility > filters (setpts, trim, concat, etc.) working with subtitles too without > more code duplication, the review is just "NAK". What is the timeline for the audio+video merge ? IIUC this would resolve this deadlock (with extra work adapting the patchset so it would be work for SW adapting it and it would be work for you finishing the merge) Also can others help nicolas moving his work forward What i suggest is to pick a time and then try to finish the merge before. If it succeeds this patchset needs updating and can move forward without this main objection OTOH if the time is not hit, we agree that the objection can no longer be used nothing above is meant as a objection to anything, just my oppinion. Some thought i had which i hope could move this forward. If people agree to anything else iam happy with that too, this mail is only meant to help! thx [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Observe your enemies, for they first find out your faults. -- Antisthenes signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sun, Jul 3, 2022 at 9:59 AM Jean-Baptiste Kempf wrote: > Hello Paul, > > On Sat, 2 Jul 2022, at 22:40, Paul B Mahol wrote: > >> I am convinced that doing deduplication afterwards is a better order > >> for getting this done. I'm also convinced that my patchset is pretty > >> [...] > > > > AFAIK only NIcolas is for this merge of different types of filters into > > single filter, > > and filter type negotiation. > > > > Both ideas are very bad. > > If you really feel so, you have a way to unlock this situation: call the > TC, whose precise role is to do work on this. > You can even request a GA vote on this topic, if you feel that more people > should step in. > > But here, the discussion goes nowhere "I'm right", "No, I'm right", "No, > I'm correct" and will only evolve in insults and slander. > > Why should I do this work? I will just create my own fork. (Already did.) > Best, > > -- > Jean-Baptiste Kempf - President > +33 672 704 734 > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Hello Paul, On Sat, 2 Jul 2022, at 22:40, Paul B Mahol wrote: >> I am convinced that doing deduplication afterwards is a better order >> for getting this done. I'm also convinced that my patchset is pretty >> [...] > > AFAIK only NIcolas is for this merge of different types of filters into > single filter, > and filter type negotiation. > > Both ideas are very bad. If you really feel so, you have a way to unlock this situation: call the TC, whose precise role is to do work on this. You can even request a GA vote on this topic, if you feel that more people should step in. But here, the discussion goes nowhere "I'm right", "No, I'm right", "No, I'm correct" and will only evolve in insults and slander. Best, -- Jean-Baptiste Kempf - President +33 672 704 734 ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Matt Zagrabelny (12022-07-02): > I guess your idea of "rudely" is different from mine. I find SW to be > polite. I guess you have not read everything since the beginning. > I am a simpleton that just reads what others write. I find subtitle support > to be desirable and SW's work to be a good step in that direction. It might seem so from the point of view of a user. From my point of view of the person who maintains the framework of libavfilter, support for subtitles is desirable, but SW's patches make my work later significantly harder. -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sat, Jul 2, 2022 at 11:22 PM Nicolas George wrote: > Matt Zagrabelny (12022-07-02): > > Nicolas' arguments seem to be "The perfect is the enemy of the good." > > No, my arguments are: you don't build a third floor when your > foundations are shaky. > > I have posted on this very mailing list a long time ago simple tasks > that need to be done before considering adding more media types for > libavfilter. They have been rudely ignored. > "Simple" tasks that are not necessary at all. > > Are you here to help, or are you here to pour oil on the flame? > > -- > Nicolas George > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Matt Zagrabelny (12022-07-02): > Nicolas' arguments seem to be "The perfect is the enemy of the good." No, my arguments are: you don't build a third floor when your foundations are shaky. I have posted on this very mailing list a long time ago simple tasks that need to be done before considering adding more media types for libavfilter. They have been rudely ignored. Are you here to help, or are you here to pour oil on the flame? -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Paul B Mahol > Sent: Saturday, July 2, 2022 10:40 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Cc: Michael Niedermayer ; Andriy Gelman > ; Andreas Rheinhardt > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > On Sat, Jul 2, 2022 at 10:32 PM Soft Works > wrote: > > > > > > > > -Original Message- > > > From: ffmpeg-devel On Behalf Of > > > Nicolas George > > > Sent: Saturday, July 2, 2022 9:36 PM > > > To: FFmpeg development discussions and patches > > de...@ffmpeg.org> > > > Cc: Michael Niedermayer ; Andriy Gelman > > > ; Andreas Rheinhardt > > > > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering > 2022 > > > > > > Soft Works (12022-07-02): > > > > This can easily be done AFTER my patchset has been merged. > > > > > > With exponentially more work. Out of question. > > > > Previously it would have been about like: > > > > - Merging audio filter code with the video filter code > > (for the filters in question) > > > > Now it will be > > > > - Merging audio and subtitle filter code with the video filter code > > (for the same filters) > > > > TBH, I can't see any exponent here. I think "double work" would be > > closer to the truth and realistically it will be much less than > that > > because the work for merging audio and subtitle code is very > similar, > > so when you have merged audio code in a filter, merging the > subtitle > > part will be very analogous, so in total it would be less than > > double. > > > > And when we look at the required amount of work in total, that > > calculation would only be valid when you would consider the value > > of MY work that I've already done as zero. > > > > I even think that it's a better approach overall to do the > deduplication > > afterwards, because now - with the subtitle filtering patchset - > the > > specific requirements for subtitle filtering are visible on the > table > > and that way, the deduplication can already provision for those > > specific requirements whereas focusing on audio/video only, might > have > > led to do changes that wouldn't accommodate for the needs of > subtitle > > filtering. > > > > I am convinced that doing deduplication afterwards is a better > order > > for getting this done. I'm also convinced that my patchset is > pretty > > solid in the way it does handle subtitles, and I'm further > convinced > > that you know that very well. During all the process I have watched > > very closely, and in several cases where others had objections > about > > things I had done, you kept quiet, presumably because you were the > only > > other one to know why it had to be done that way. Also, you never > > named any specific detail that would be wrong, and I'm sure you > would > > have done if there had been any significant one. > > My impression is that your primary reason for objection is that my > > patchset interferes with your plans and visions you probably had in > > mind for quite a while and I'm very sorry about that. > > But in the end, my patchset doesn't stand in opposition to your > plans, > > it just requires a bit of adaption regarding the order of doing the > work. > > Neither do I stand in opposition to your plans. I respect the > technical > > architecture of libavfilter, especially regarding its simplicity > and > > effectivity compared to other filtering frameworks (like > DirectShow) > > and my interest in Ffmpeg filtering is not limited to subtitles. > > We don't need to be friends, but when you would manage to act and > > communicate in a friendly way, you might gain somebody to help with > > and support your plans in the future and you would also do a favor > > to all readers of the ML by not having them read through despicable > > conversations. > > > > AFAIK only NIcolas is for this merge of different types of filters > into > single filter, > and filter type negotiation. > > Both ideas are very bad. And Nicolas is (still) not (yet) FFmpeg > dictator. I think it makes sense for certain filters like buffer source, buffer sink, null sink, trim, showinfo, copy, delay, repeat, etc. Yet there is no point in making this a pre-condition for subtitle filtering, it's out of scope. sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sat, Jul 2, 2022 at 10:32 PM Soft Works wrote: > > > > -Original Message- > > From: ffmpeg-devel On Behalf Of > > Nicolas George > > Sent: Saturday, July 2, 2022 9:36 PM > > To: FFmpeg development discussions and patches > de...@ffmpeg.org> > > Cc: Michael Niedermayer ; Andriy Gelman > > ; Andreas Rheinhardt > > > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > > > Soft Works (12022-07-02): > > > This can easily be done AFTER my patchset has been merged. > > > > With exponentially more work. Out of question. > > Previously it would have been about like: > > - Merging audio filter code with the video filter code > (for the filters in question) > > Now it will be > > - Merging audio and subtitle filter code with the video filter code > (for the same filters) > > TBH, I can't see any exponent here. I think "double work" would be > closer to the truth and realistically it will be much less than that > because the work for merging audio and subtitle code is very similar, > so when you have merged audio code in a filter, merging the subtitle > part will be very analogous, so in total it would be less than > double. > > And when we look at the required amount of work in total, that > calculation would only be valid when you would consider the value > of MY work that I've already done as zero. > > I even think that it's a better approach overall to do the deduplication > afterwards, because now - with the subtitle filtering patchset - the > specific requirements for subtitle filtering are visible on the table > and that way, the deduplication can already provision for those > specific requirements whereas focusing on audio/video only, might have > led to do changes that wouldn't accommodate for the needs of subtitle > filtering. > > I am convinced that doing deduplication afterwards is a better order > for getting this done. I'm also convinced that my patchset is pretty > solid in the way it does handle subtitles, and I'm further convinced > that you know that very well. During all the process I have watched > very closely, and in several cases where others had objections about > things I had done, you kept quiet, presumably because you were the only > other one to know why it had to be done that way. Also, you never > named any specific detail that would be wrong, and I'm sure you would > have done if there had been any significant one. > My impression is that your primary reason for objection is that my > patchset interferes with your plans and visions you probably had in > mind for quite a while and I'm very sorry about that. > But in the end, my patchset doesn't stand in opposition to your plans, > it just requires a bit of adaption regarding the order of doing the work. > Neither do I stand in opposition to your plans. I respect the technical > architecture of libavfilter, especially regarding its simplicity and > effectivity compared to other filtering frameworks (like DirectShow) > and my interest in Ffmpeg filtering is not limited to subtitles. > We don't need to be friends, but when you would manage to act and > communicate in a friendly way, you might gain somebody to help with > and support your plans in the future and you would also do a favor > to all readers of the ML by not having them read through despicable > conversations. > AFAIK only NIcolas is for this merge of different types of filters into single filter, and filter type negotiation. Both ideas are very bad. And Nicolas is (still) not (yet) FFmpeg dictator. > Best regards, > softworkz > > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Saturday, July 2, 2022 9:36 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Cc: Michael Niedermayer ; Andriy Gelman > ; Andreas Rheinhardt > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Soft Works (12022-07-02): > > This can easily be done AFTER my patchset has been merged. > > With exponentially more work. Out of question. Previously it would have been about like: - Merging audio filter code with the video filter code (for the filters in question) Now it will be - Merging audio and subtitle filter code with the video filter code (for the same filters) TBH, I can't see any exponent here. I think "double work" would be closer to the truth and realistically it will be much less than that because the work for merging audio and subtitle code is very similar, so when you have merged audio code in a filter, merging the subtitle part will be very analogous, so in total it would be less than double. And when we look at the required amount of work in total, that calculation would only be valid when you would consider the value of MY work that I've already done as zero. I even think that it's a better approach overall to do the deduplication afterwards, because now - with the subtitle filtering patchset - the specific requirements for subtitle filtering are visible on the table and that way, the deduplication can already provision for those specific requirements whereas focusing on audio/video only, might have led to do changes that wouldn't accommodate for the needs of subtitle filtering. I am convinced that doing deduplication afterwards is a better order for getting this done. I'm also convinced that my patchset is pretty solid in the way it does handle subtitles, and I'm further convinced that you know that very well. During all the process I have watched very closely, and in several cases where others had objections about things I had done, you kept quiet, presumably because you were the only other one to know why it had to be done that way. Also, you never named any specific detail that would be wrong, and I'm sure you would have done if there had been any significant one. My impression is that your primary reason for objection is that my patchset interferes with your plans and visions you probably had in mind for quite a while and I'm very sorry about that. But in the end, my patchset doesn't stand in opposition to your plans, it just requires a bit of adaption regarding the order of doing the work. Neither do I stand in opposition to your plans. I respect the technical architecture of libavfilter, especially regarding its simplicity and effectivity compared to other filtering frameworks (like DirectShow) and my interest in Ffmpeg filtering is not limited to subtitles. We don't need to be friends, but when you would manage to act and communicate in a friendly way, you might gain somebody to help with and support your plans in the future and you would also do a favor to all readers of the ML by not having them read through despicable conversations. Best regards, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Soft Works (12022-07-02): > This can easily be done AFTER my patchset has been merged. With exponentially more work. Out of question. This is all I have to say to you on the subject. -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Saturday, July 2, 2022 9:17 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Cc: Michael Niedermayer ; Andriy Gelman > ; Andreas Rheinhardt > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Soft Works (12022-07-02): > > The duplication has always existed already between audio and video > > filters. You could have unified and eliminated that duplication for > > years, but you didn't. > > I have been preparing for it for years. You did not help. This can easily be done AFTER my patchset has been merged. Despite all of your negative attitude, I will do my best to assist you in that process later. Thanks, sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: ffmpeg-devel On Behalf Of > Nicolas George > Sent: Saturday, July 2, 2022 9:17 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Cc: Michael Niedermayer ; Andriy Gelman > ; Andreas Rheinhardt > > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Soft Works (12022-07-02): > > The duplication has always existed already between audio and video > > filters. You could have unified and eliminated that duplication for > > years, but you didn't. > > I have been preparing for it for years. You did not help. > > > Even when I would follow that demand, it wouldn't be reasonable, > > because in that case, the patchset would dramatically widen its > > scope and start affecting audio and video as well - which is much > > too huge to get it tackled in a single patchset. > > We will not accept a bad patch just because you have bitten more than > you could chew and reverted to a quick and dirty solution. All you do is keep spitting discrediting keywords, this is just disgusting. sw ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Soft Works (12022-07-02): > The duplication has always existed already between audio and video > filters. You could have unified and eliminated that duplication for > years, but you didn't. I have been preparing for it for years. You did not help. > Even when I would follow that demand, it wouldn't be reasonable, > because in that case, the patchset would dramatically widen its > scope and start affecting audio and video as well - which is much > too huge to get it tackled in a single patchset. We will not accept a bad patch just because you have bitten more than you could chew and reverted to a quick and dirty solution. -- Nicolas George ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
> -Original Message- > From: Nicolas George > Sent: Saturday, July 2, 2022 7:18 PM > To: FFmpeg development discussions and patches de...@ffmpeg.org> > Cc: Michael Niedermayer ; softworkz > ; Andriy Gelman ; > Andreas Rheinhardt > Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 > > Paul B Mahol (12022-07-02): > > Can this be properly finally be fully reviewed and accepted? > > As long as the patch does not have a solution to have all the utility > filters (setpts, trim, concat, etc.) working with subtitles too > without > more code duplication, the review is just "NAK". The latest versions has added support for trim (strim). I strongly disagree on the "code duplication" argument. AFAIU, this is about having similar filters duplicated for each media type, e.g. trim, atrim and strim. The duplication has always existed already between audio and video filters. You could have unified and eliminated that duplication for years, but you didn't. Now that a third media type is being added, it is totally arbitrary to ask for unification and de-duplication of the code. You didn't do it for 2 media types, so why should 3 be the magic number where this is suddenly a requirement? This is an unjustified request. Even when I would follow that demand, it wouldn't be reasonable, because in that case, the patchset would dramatically widen its scope and start affecting audio and video as well - which is much too huge to get it tackled in a single patchset. It has always been possible to de-duplicate between audio and video, and in the future, it will always be possible to de-duplicate between audio, video and subtitles. It might make sense to do so at some point in time, but it is out of the scope of this patchset. And when this is the only objection, then I think that the patchset is actually in really good shape for getting merged. Thanks, softworkz ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
From: Paul B Mahol Sent: Saturday, July 2, 2022 6:39 PM To: FFmpeg development discussions and patches Cc: Michael Niedermayer ; softworkz ; Andriy Gelman ; Andreas Rheinhardt Subject: Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022 On Sat, Jun 25, 2022 at 11:58 AM ffmpegagent mailto:ffmpegag...@gmail.com>> wrote: Subtitle Filtering 2022 === This is a substantial update to the earlier subtitle filtering patch series. A primary goal has been to address others' concerns as much as possible on one side and to provide more clarity and control over the way things are working. Clarity is is specifically important to allow for a better understanding of the need for a subtitle start pts value that can be different from the frame's pts value. This is done by refactoring the subtitle timing fields in AVFrame, adding a frame field to indicate repeated subtitle frames, and finally the full removal of the heartbeat functionality, replaced by a new 'subfeed' filter that provides different modes for arbitrating subtitle frames in a filter graph. Finally, each subtitle filter's documentation has been amended by a section describing the filter's timeline behavior (in v3 update). Subtitle Filtering Demos I published a demonstration of subtitle filtering capabilities with OCR, text and bitmap subtitle manipulation involved: Demo 1: Text-Manipulation with Bitmap Subtitles [https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo1] v5 - Conversion to Graphic Subtitles, and other enhancements * I'm glad to announce that Traian (@tcoza) has joined the project and contributed a new 'text2graphicsub' filter to convert text subtitles to graphic subtitles, which can in turn be encoded as dvd, dvb or x-subs (and any other encoder for graphic subs that might be added in the future). This filter closes the last open "gap" in subtitle processing. * stripstyles filter: now allows very fine-grained control over which ASS style codes should be preserved or stripped * stripstyles: do not drop dialog margin values * subfeed filter: eliminates duplicate frames with duplicate start times when 'fix_overlap' is specified * textmod: do not drop effect values * graphicsub2text: reduce font size jitter * ass_split: add function to selectively preserve elements when splitting * add strim, snull and ssink and further unify subtitle frame handling with audio and video * ffmpeg_filter: get simple filter notation working for subtitles v4 - Quality Improvements = * finally an updated version * includes many improvements from internal testing * all FATE tests passed * all example commands from the docs verified to work * can't list all the detail changes.. * I have left out the extra commits which can be handled separately, just in case somebody wonders why these are missing: * avcodec/webvttenc: Don't encode drawing codes and empty lines * avcodec/webvttenc: convert hard-space tags to * avutil/ass_split: Add parsing of hard-space tags (\h) * avutil/ass_split: Treat all content in curly braces as hidden * avutil/ass_split: Fix ass parsing of style codes with comments v3 - Rebase === due to merge conflicts - apologies. Changes in v2 = * added .gitattributes file to enforce binary diffs for the test refs that cannot be applied when being sent via e-mail * perform filter graph re-init due to subtitle "frame size" change only when the size was unknown before and not set via -canvas_size * overlaytextsubs: Make sure to request frames on the subtitle input * avfilter/splitcc: Start parsing cc data on key frames only * avcodec/webvttenc: Don't encode ass drawing codes and empty lines * stripstyles: fix mem leak * gs2t: improve color detection * gs2t: empty frames must not be skipped * subfeed: fix name * textmod: preserve margins * added .gitattributes file to enforce binary diffs for the test refs that cannot be applied when being sent via e-mail * perform filter graph re-init due to subtitle "frame size" change only when the size was unknown before and not set via -canvas_size * avcodec/dvbsubdec: Fix conditions for fallback to default resolution * Made changes suggested by Andreas * Fixed failing command line reported by Michael Changes from previous version v24: AVFrame === * Removed sub_start_time The start time is now added to the subtitle start_pts during decoding The sub_end_time field is adjusted accordingly * Renamed sub_end_time to duration which it is effectively after removing the start_time * Added a sub-struct 'subtitle_timing' to av frame Contains subtitle_pts renamed to 'subtitle_timing.start_pts' and 'subtitle_timing.duration' * Change both fields to (fixed) time_base AV_TIMEBASE * add repeat_sub field
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Paul B Mahol (12022-07-02): > Could you please mention in relevant thread of those subtitle work where > duplication in code happened? No. Currently, there is no solution, with or without code duplication. -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sat, Jul 2, 2022 at 7:18 PM Nicolas George wrote: > Paul B Mahol (12022-07-02): > > Can this be properly finally be fully reviewed and accepted? > > As long as the patch does not have a solution to have all the utility > filters (setpts, trim, concat, etc.) working with subtitles too without > more code duplication, the review is just "NAK". > Could you please mention in relevant thread of those subtitle work where duplication in code happened? > > -- > Nicolas George > ___ > ffmpeg-devel mailing list > ffmpeg-devel@ffmpeg.org > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel > > To unsubscribe, visit link above, or email > ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe". > ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
Paul B Mahol (12022-07-02): > Can this be properly finally be fully reviewed and accepted? As long as the patch does not have a solution to have all the utility filters (setpts, trim, concat, etc.) working with subtitles too without more code duplication, the review is just "NAK". -- Nicolas George signature.asc Description: PGP signature ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] [PATCH v5 00/25] Subtitle Filtering 2022
On Sat, Jun 25, 2022 at 11:58 AM ffmpegagent wrote: > > Subtitle Filtering 2022 > === > > This is a substantial update to the earlier subtitle filtering patch > series. > A primary goal has been to address others' concerns as much as possible on > one side and to provide more clarity and control over the way things are > working. Clarity is is specifically important to allow for a better > understanding of the need for a subtitle start pts value that can be > different from the frame's pts value. This is done by refactoring the > subtitle timing fields in AVFrame, adding a frame field to indicate > repeated > subtitle frames, and finally the full removal of the heartbeat > functionality, replaced by a new 'subfeed' filter that provides different > modes for arbitrating subtitle frames in a filter graph. Finally, each > subtitle filter's documentation has been amended by a section describing > the > filter's timeline behavior (in v3 update). > > > Subtitle Filtering Demos > > > I published a demonstration of subtitle filtering capabilities with OCR, > text and bitmap subtitle manipulation involved: Demo 1: Text-Manipulation > with Bitmap Subtitles > [https://github.com/softworkz/SubtitleFilteringDemos/tree/master/Demo1] > > > v5 - Conversion to Graphic Subtitles, and other enhancements > > > * I'm glad to announce that Traian (@tcoza) has joined the project and >contributed a new 'text2graphicsub' filter to convert text subtitles to >graphic subtitles, which can in turn be encoded as dvd, dvb or x-subs >(and any other encoder for graphic subs that might be added in the >future). This filter closes the last open "gap" in subtitle processing. > * stripstyles filter: now allows very fine-grained control over which ASS >style codes should be preserved or stripped > * stripstyles: do not drop dialog margin values > * subfeed filter: eliminates duplicate frames with duplicate start times >when 'fix_overlap' is specified > * textmod: do not drop effect values > * graphicsub2text: reduce font size jitter > * ass_split: add function to selectively preserve elements when splitting > * add strim, snull and ssink and further unify subtitle frame handling > with >audio and video > * ffmpeg_filter: get simple filter notation working for subtitles > > > v4 - Quality Improvements > = > > * finally an updated version > * includes many improvements from internal testing > * all FATE tests passed > * all example commands from the docs verified to work > * can't list all the detail changes.. > * I have left out the extra commits which can be handled separately, just >in case somebody wonders why these are missing: >* avcodec/webvttenc: Don't encode drawing codes and empty lines >* avcodec/webvttenc: convert hard-space tags to >* avutil/ass_split: Add parsing of hard-space tags (\h) >* avutil/ass_split: Treat all content in curly braces as hidden >* avutil/ass_split: Fix ass parsing of style codes with comments > > > v3 - Rebase > === > > due to merge conflicts - apologies. > > > Changes in v2 > = > > * added .gitattributes file to enforce binary diffs for the test refs that >cannot be applied when being sent via e-mail > * perform filter graph re-init due to subtitle "frame size" change only >when the size was unknown before and not set via -canvas_size > * overlaytextsubs: Make sure to request frames on the subtitle input > * avfilter/splitcc: Start parsing cc data on key frames only > * avcodec/webvttenc: Don't encode ass drawing codes and empty lines > * stripstyles: fix mem leak > * gs2t: improve color detection > * gs2t: empty frames must not be skipped > * subfeed: fix name > * textmod: preserve margins > * added .gitattributes file to enforce binary diffs for the test refs that >cannot be applied when being sent via e-mail > * perform filter graph re-init due to subtitle "frame size" change only >when the size was unknown before and not set via -canvas_size > * avcodec/dvbsubdec: Fix conditions for fallback to default resolution > * Made changes suggested by Andreas > * Fixed failing command line reported by Michael > > Changes from previous version v24: > > > AVFrame > === > > * Removed sub_start_time The start time is now added to the subtitle >start_pts during decoding The sub_end_time field is adjusted accordingly > * Renamed sub_end_time to duration which it is effectively after removing >the start_time > * Added a sub-struct 'subtitle_timing' to av frame Contains subtitle_pts >renamed to 'subtitle_timing.start_pts' and 'subtitle_timing.duration' > * Change both fields to (fixed) time_base AV_TIMEBASE > * add repeat_sub field provides a clear indication whether a subtitle > frame >is an actual subtitle event or a repeated subtitle frame in a
