Re: [FFmpeg-devel] Flushing while decoding , but need already decoded frames
Michael Henrik Bodenhoff via ffmpeg-devel: > Hi Andrey , > > "when the decoder is set up to ignore B frames" > > How do you do that ? > > We must have missed something while we tried using FFmpeg, because we > couldn't find a way of telling the decoder to ignore B frames, we actually > got so desperate that we considered if it was possible to modify the incoming > encoded stream to make FFmpeg believe that the stream doesn't contain B > frames ☹ > There is AVCodecContext.skip_frame; some decoders (not all, but e.g. H.264) will skip frames based upon this value. But it does not affect the decoder delay. (There is also AV_CODEC_FLAG_LOW_DELAY, but this is honored by even less decoders (only some old ones).) - Andreas ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Flushing while decoding , but need already decoded frames
Hi Andrey ,
"when the decoder is set up to ignore B frames"
How do you do that ?
We must have missed something while we tried using FFmpeg, because we couldn't
find a way of telling the decoder to ignore B frames, we actually got so
desperate that we considered if it was possible to modify the incoming encoded
stream to make FFmpeg believe that the stream doesn't contain B frames ☹
Michael Bodenhoff
Principal Software Engineer
FT01
Email: m...@milestone.dk
-Original Message-
From: ffmpeg-devel On Behalf Of Andrey Turkin
Sent: 24. maj 2024 13:39
To: FFmpeg development discussions and patches
Subject: Re: [FFmpeg-devel] Flushing while decoding , but need already decoded
frames
[You don't often get email from andrey.tur...@gmail.com. Learn why this is
important at https://aka.ms/LearnAboutSenderIdentification ]
Have to say, this issue has been a long grievance of mine. There is no reason
to delay frames when the decoder is set up to ignore B frames as there is no
reordering to be done; ideally this should be zero-delay case (packet goes in,
frame goes out) yet the most common decoders delay frames anyway, as if to
decode B frames. Moreover, with the "new" send/receive API I think there is no
reason to delay frames at all - a single send_packet could decode and queue
multiple frames to be received, so it makes sense to send frames as soon as
possible - yet that is not the case as well.
пт, 24 мая 2024 г. в 13:17, Andreas Rheinhardt :
>
> Michael Henrik Bodenhoff via ffmpeg-devel:
> > Hi ,
> >
> > my team recently had to abandon switching to using FFmpeg from
> > specific decoder implementations (NvDEC, Intel Media SDK , IPP and
> > quite a few codec specific decoders) because of big performance
> > issues because of the way FFmpeg works….. or at least we think it is
> > (we’re FFmpeg noobs )
> >
> > It's actually an issue we also had with Intel Media SDK, leading us to pay
> > Intel to extend Media SDK to do what we needed.
> >
> > Our product is a video surveillance system, and that means we have to
> > decode a LOT of video streams simultaneously.
> >
> > For motion detection we want to only decode keyframes, and skip P and B
> > frames , and that works fine with FFmpeg most of the time, except for when
> > the video stream contains B frames.
> > Without B-Frames it’s really simple (simplified pseudocode) :
> >
> > while(true)
> > {
> > receiveStreamCompleteFrame();
> > If(KeyFrame)
> > {
> > avcodec_send_packet();
> > if(avcodec_receive_frame()==0)
> > {
> >// do motion detection
> > }
> > }
> > }
> >
> > But! with B Frames FFmpeg doesn’t return keyframes when they are decoded,
> > they are kept, and we can’t seem to flush them out. avcodec_flush_buffers
> > allow us to continue to next keyframe, but it doesn’t seem to give us the
> > keyframe we just gave to FFmpeg with avcodec_send_packet.
> >
> > while(true)
> > {
> > receiveStreamCompleteFrame();
> > If(KeyFrame)
> > {
> > avcodec_send_packet();
> > if(avcodec_receive_frame()==0)
> > {
> >// do motion detection
> > }
> > Else
> > {
> > avcodec_flush_buffers();
> > if(avcodec_receive_frame()==0)
> > {
> > // do motion detection
> > }
> > }
> > }
> > }
> >
> > Calling avcodec_receive_frame after calling avcodec_flush_buffer
> > results in -11 and no frame
> >
> > is there anyway around this ? And if not, could FFmpeg be made to have this
> > functionality ?
> >
> > I tried contacting one of the FFmpeg consultants from
> > https://ff/
> > mpeg.org%2Fconsulting.html=05%7C02%7Cmibo%40milestone.dk%7C46bc
> > b557d2194eb8b76d08dc7be63528%7C4744e4b2072d4754be71250b45e049fe%7C0%
> > 7C0%7C638521475866379714%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDA
> > iLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=H
> > V8bPZyvDklo5iGs%2BnrW4uEJ6m%2B9tOExcyvt8W3VxyQ%3D=0 but
> > never got a response
> >
>
> Send your packet with the keyframe, send a NULL packet (to signal
> EOF), then the internally stored frames should be output by
> avcodec_receive_frame(). Then flush the decoder (to be able to send
> new packets to it).
>
> - Andreas
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmp/
> eg.org%2Fmailman%2Flistinfo%2Fffmpeg-devel=05%7C02%7Cmibo%40miles
> tone.dk%7C46bcb557d2194eb8b76d08dc7be63528%7C4744e4b2072d4754be71250b4
> 5e049f
Re: [FFmpeg-devel] Flushing while decoding , but need already decoded frames
Quoting Andrey Turkin (2024-05-24 13:39:15) > Have to say, this issue has been a long grievance of mine. There is no > reason to delay frames when the decoder is set up to ignore B frames > as there is no reordering to be done; This only holds for pre-h264 codecs. -- Anton Khirnov ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Flushing while decoding , but need already decoded frames
Hi, On Fri, May 24, 2024 at 7:39 AM Andrey Turkin wrote: > Have to say, this issue has been a long grievance of mine. There is no > reason to delay frames when the decoder is set up to ignore B frames > as there is no reordering to be done > Frame threading? Ronald ___ ffmpeg-devel mailing list ffmpeg-devel@ffmpeg.org https://ffmpeg.org/mailman/listinfo/ffmpeg-devel To unsubscribe, visit link above, or email ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Flushing while decoding , but need already decoded frames
Have to say, this issue has been a long grievance of mine. There is no
reason to delay frames when the decoder is set up to ignore B frames
as there is no reordering to be done; ideally this should be
zero-delay case (packet goes in, frame goes out) yet the most common
decoders delay frames anyway, as if to decode B frames. Moreover, with
the "new" send/receive API I think there is no reason to delay frames
at all - a single send_packet could decode and queue multiple frames
to be received, so it makes sense to send frames as soon as possible -
yet that is not the case as well.
пт, 24 мая 2024 г. в 13:17, Andreas Rheinhardt :
>
> Michael Henrik Bodenhoff via ffmpeg-devel:
> > Hi ,
> >
> > my team recently had to abandon switching to using FFmpeg from specific
> > decoder implementations (NvDEC, Intel Media SDK , IPP and quite a few codec
> > specific decoders) because of big performance issues because of the way
> > FFmpeg works….. or at least we think it is (we’re FFmpeg noobs )
> >
> > It's actually an issue we also had with Intel Media SDK, leading us to pay
> > Intel to extend Media SDK to do what we needed.
> >
> > Our product is a video surveillance system, and that means we have to
> > decode a LOT of video streams simultaneously.
> >
> > For motion detection we want to only decode keyframes, and skip P and B
> > frames , and that works fine with FFmpeg most of the time, except for when
> > the video stream contains B frames.
> > Without B-Frames it’s really simple (simplified pseudocode) :
> >
> > while(true)
> > {
> > receiveStreamCompleteFrame();
> > If(KeyFrame)
> > {
> > avcodec_send_packet();
> > if(avcodec_receive_frame()==0)
> > {
> >// do motion detection
> > }
> > }
> > }
> >
> > But! with B Frames FFmpeg doesn’t return keyframes when they are decoded,
> > they are kept, and we can’t seem to flush them out. avcodec_flush_buffers
> > allow us to continue to next keyframe, but it doesn’t seem to give us the
> > keyframe we just gave to FFmpeg with avcodec_send_packet.
> >
> > while(true)
> > {
> > receiveStreamCompleteFrame();
> > If(KeyFrame)
> > {
> > avcodec_send_packet();
> > if(avcodec_receive_frame()==0)
> > {
> >// do motion detection
> > }
> > Else
> > {
> > avcodec_flush_buffers();
> > if(avcodec_receive_frame()==0)
> > {
> > // do motion detection
> > }
> > }
> > }
> > }
> >
> > Calling avcodec_receive_frame after calling avcodec_flush_buffer results in
> > -11 and no frame
> >
> > is there anyway around this ? And if not, could FFmpeg be made to have this
> > functionality ?
> >
> > I tried contacting one of the FFmpeg consultants from
> > https://ffmpeg.org/consulting.html but never got a response
> >
>
> Send your packet with the keyframe, send a NULL packet (to signal EOF),
> then the internally stored frames should be output by
> avcodec_receive_frame(). Then flush the decoder (to be able to send new
> packets to it).
>
> - Andreas
>
> ___
> ffmpeg-devel mailing list
> ffmpeg-devel@ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
Re: [FFmpeg-devel] Flushing while decoding , but need already decoded frames
Michael Henrik Bodenhoff via ffmpeg-devel:
> Hi ,
>
> my team recently had to abandon switching to using FFmpeg from specific
> decoder implementations (NvDEC, Intel Media SDK , IPP and quite a few codec
> specific decoders) because of big performance issues because of the way
> FFmpeg works….. or at least we think it is (we’re FFmpeg noobs )
>
> It's actually an issue we also had with Intel Media SDK, leading us to pay
> Intel to extend Media SDK to do what we needed.
>
> Our product is a video surveillance system, and that means we have to decode
> a LOT of video streams simultaneously.
>
> For motion detection we want to only decode keyframes, and skip P and B
> frames , and that works fine with FFmpeg most of the time, except for when
> the video stream contains B frames.
> Without B-Frames it’s really simple (simplified pseudocode) :
>
> while(true)
> {
> receiveStreamCompleteFrame();
> If(KeyFrame)
> {
> avcodec_send_packet();
> if(avcodec_receive_frame()==0)
> {
>// do motion detection
> }
> }
> }
>
> But! with B Frames FFmpeg doesn’t return keyframes when they are decoded,
> they are kept, and we can’t seem to flush them out. avcodec_flush_buffers
> allow us to continue to next keyframe, but it doesn’t seem to give us the
> keyframe we just gave to FFmpeg with avcodec_send_packet.
>
> while(true)
> {
> receiveStreamCompleteFrame();
> If(KeyFrame)
> {
> avcodec_send_packet();
> if(avcodec_receive_frame()==0)
> {
>// do motion detection
> }
> Else
> {
> avcodec_flush_buffers();
> if(avcodec_receive_frame()==0)
> {
> // do motion detection
> }
> }
> }
> }
>
> Calling avcodec_receive_frame after calling avcodec_flush_buffer results in
> -11 and no frame
>
> is there anyway around this ? And if not, could FFmpeg be made to have this
> functionality ?
>
> I tried contacting one of the FFmpeg consultants from
> https://ffmpeg.org/consulting.html but never got a response
>
Send your packet with the keyframe, send a NULL packet (to signal EOF),
then the internally stored frames should be output by
avcodec_receive_frame(). Then flush the decoder (to be able to send new
packets to it).
- Andreas
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
To unsubscribe, visit link above, or email
ffmpeg-devel-requ...@ffmpeg.org with subject "unsubscribe".
[FFmpeg-devel] Flushing while decoding , but need already decoded frames
Hi ,
my team recently had to abandon switching to using FFmpeg from specific decoder
implementations (NvDEC, Intel Media SDK , IPP and quite a few codec specific
decoders) because of big performance issues because of the way FFmpeg works…..
or at least we think it is (we’re FFmpeg noobs )
It's actually an issue we also had with Intel Media SDK, leading us to pay
Intel to extend Media SDK to do what we needed.
Our product is a video surveillance system, and that means we have to decode a
LOT of video streams simultaneously.
For motion detection we want to only decode keyframes, and skip P and B frames
, and that works fine with FFmpeg most of the time, except for when the video
stream contains B frames.
Without B-Frames it’s really simple (simplified pseudocode) :
while(true)
{
receiveStreamCompleteFrame();
If(KeyFrame)
{
avcodec_send_packet();
if(avcodec_receive_frame()==0)
{
// do motion detection
}
}
}
But! with B Frames FFmpeg doesn’t return keyframes when they are decoded, they
are kept, and we can’t seem to flush them out. avcodec_flush_buffers allow us
to continue to next keyframe, but it doesn’t seem to give us the keyframe we
just gave to FFmpeg with avcodec_send_packet.
while(true)
{
receiveStreamCompleteFrame();
If(KeyFrame)
{
avcodec_send_packet();
if(avcodec_receive_frame()==0)
{
// do motion detection
}
Else
{
avcodec_flush_buffers();
if(avcodec_receive_frame()==0)
{
// do motion detection
}
}
}
}
Calling avcodec_receive_frame after calling avcodec_flush_buffer results in -11
and no frame
is there anyway around this ? And if not, could FFmpeg be made to have this
functionality ?
I tried contacting one of the FFmpeg consultants from
https://ffmpeg.org/consulting.html but never got a response
Michael Bodenhoff
Principal Software Engineer
FT01
Email: m...@milestone.dk
[Linkedin Logo][Twitter
Logo][Youtube
Logo][Facebook
Logo]
