Re: [FFmpeg-devel] [PATCH v5 01/11] avformat: Add fifo pseudo-muxer

[Jan Sebechlebsky]

On 08/08/2016 11:55 PM, Marton Balint wrote:



Thanks for your work, and sorry for the delay in the review.

Regards,
Marton
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Thanks for review! Hopefully I've fixed all the mentioned issues.
I'm resending the patch and also affected patches.

Regards,
Jan
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH v5 01/11] avformat: Add fifo pseudo-muxer

[Marton Balint]

On Thu, 4 Aug 2016, sebechlebsky...@gmail.com wrote:


From: Jan Sebechlebsky 

Signed-off-by: Jan Sebechlebsky 


applying the patches with "git am" seems to report some whitespace 
issues (empty lines at the end of files) could you have a look at those?


[...]


--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -1436,6 +1436,96 @@ Specify whether to remove all fragments when finished. 
Default 0 (do not remove)

@end table

+@section fifo
+
+The fifo pseudo-muxer allows to separate encoding from any other muxer by using


... allows the separation of encoding and muxing by using a ...


+first-in-first-out queue and running the actual muxer in a separate thread. 
This
+is especially useful in combination with the @ref{tee} muxer and output to


I get errors for this line when builiding the docs:
doc/muxers.texi:1443: @ref reference to nonexistent node `tee'


+several destinations with different reliability/writing speed/latency.
+
+The behavior of fifo muxer in case of failure can be configured:


The behaviour of the fifo muxer if the queue fills up or if the output 
fails is selectable,



+@itemize @bullet
+
+@item
+output can be transparently restarted with configurable delay between retries
+based on real time or time of the processed stream.
+
+@item
+encoding can be blocked during temporary failure, or continue transparently
+dropping packets in case fifo queue fills up.
+
+@end itemize
+
+@table @option
+
+@item fifo_format
+Specify the format name. Useful if it cannot be guessed from the
+output name suffix.
+
+@item queue_size
+Specify size of the queue (number of packets). Default value is 60.
+
+@item format_opts
+Specify format options for the underlying muxer. Muxer options can be specified
+as a list of @var{key}=@var{value} pairs separated by ':'.
+
+@item drop_pkts_on_overflow @var{bool}
+If set to 1 (true), in case the fifo queue fills up, packets will be dropped
+rather than blocking the encoder. This allows to continue streaming without
+delaying the output, at the cost of ommiting part of the stream. By default


delaying the input you mean?


+this option is set to 0 (false), so in such cases the encoder will be blocked
+until the muxer processes some of the packets and none of them is lost.
+
+@item attempt_recovery @var{bool}
+If failure occurs, attempt to recover the output. This is especially useful
+when used with network output, allows to restart streaming transparently.
+By default this option set to 0 (false).
+
+@item max_recovery_attempts
+Sets maximum number of successive unsucessful recovery attempts after which
+the output fails permanently. Unlimited if set to zero. Default value is 16.
+
+@item recovery_wait_time @var{duration}
+Waiting time before the next recovery attempt after previous unsuccessfull
+recovery attempt. Default value is 5 seconds.
+
+s@item recovery_wait_streamtime @var{bool}
+If set to 0 (false), the real time is used when waiting for the recovery
+attempt (i.e. the recovery will be attempted after at least
+recovery_wait_time seconds).
+If set to 1 (true), the time of the processed stream is taken into account
+instead (i.e. the recovery will be attempted after at least recovery_wait_time
+seconds of the stream is omitted).
+By default, this option is set to 0 (false).
+
+@item recover_any_error @var{bool}
+If set to 1 (true), recovery will be attempted regardless of type of the error
+causing the failure. By default this option is set to 0 (false) and in case of
+certain errors the recovery is not attempted even when @ref{attempt_recovery}


... in case of certain (usually permanent) errors ...

I also get an error when building docs:
doc/muxers.texi:1504: @ref reference to nonexistent node `attempt_recovery'


+is set to 1.
+
+@item restart_with_keyframe @var{bool}
+Specify whether to wait for the keyframe after recovering from
+queue overflow or failure. This option is set to 0 (false) by default.
+
+@end table
+
+@subsection Examples
+
+@itemize
+
+@item
+Stream something to rtmp server, continue processing the stream at real-time
+rate even in case of temporary failure (network outage) and attempt to recover
+streaming every second indefinitely.
+@example
+ffmpeg -re -i ... -c:v libx264 -c:a mp2 -f fifo -fifo_format flv -map 0:v -map 
0:a
+  -block_on_overflow 0 -attempt_recovery 1 -recovery_wait_time 1
+  -max_recovery_attempts 0 rtmp://example.com/live/stream_name
+@end example
+
+@end itemize
+
@section tee



[...]


+/* If >0 recovery will be attempted regardless of error code
+ * (except AVERROR_EXIT, so exit request is never ignored) */
+int recover_any_error;
+
+/* Whether to drop packets in case the queue is full. */
+int drop_pkts_on_overflow;
+
+/* Whether to wait for keyframe when recovering
+ * from failure or queue overflow */
+int restart_with_keyframe;
+
+pthread_mutex_t overflow_flag_lock;
+/* Value > 0 signalizes queue overflow */

[FFmpeg-devel] [PATCH v5 01/11] avformat: Add fifo pseudo-muxer

[sebechlebskyjan]

From: Jan Sebechlebsky 

Signed-off-by: Jan Sebechlebsky 
---
 Changes from the last version of patch:
  - boolean AVOptions are now ints, this was the cause of fate test
segfault reported by Michael

 Changelog|   1 +
 configure|   1 +
 doc/muxers.texi  |  90 +++
 libavformat/Makefile |   1 +
 libavformat/allformats.c |   1 +
 libavformat/fifo.c   | 674 +++
 libavformat/version.h|   2 +-
 7 files changed, 769 insertions(+), 1 deletion(-)
 create mode 100644 libavformat/fifo.c

diff --git a/Changelog b/Changelog
index 0f9b4cf..3f858f1 100644
--- a/Changelog
+++ b/Changelog
@@ -12,6 +12,7 @@ version :
 - 16-bit support in selectivecolor filter
 - OpenH264 decoder wrapper
 - MediaCodec hwaccel
+- fifo muxer
 
 
 version 3.1:
diff --git a/configure b/configure
index 9f5b31f..4651f5f 100755
--- a/configure
+++ b/configure
@@ -2834,6 +2834,7 @@ dv_muxer_select="dvprofile"
 dxa_demuxer_select="riffdec"
 eac3_demuxer_select="ac3_parser"
 f4v_muxer_select="mov_muxer"
+fifo_muxer_deps="pthreads"
 flac_demuxer_select="flac_parser"
 hds_muxer_select="flv_muxer"
 hls_muxer_select="mpegts_muxer"
diff --git a/doc/muxers.texi b/doc/muxers.texi
index 5873269..e2bc290 100644
--- a/doc/muxers.texi
+++ b/doc/muxers.texi
@@ -1436,6 +1436,96 @@ Specify whether to remove all fragments when finished. 
Default 0 (do not remove)
 
 @end table
 
+@section fifo
+
+The fifo pseudo-muxer allows to separate encoding from any other muxer by using
+first-in-first-out queue and running the actual muxer in a separate thread. 
This
+is especially useful in combination with the @ref{tee} muxer and output to
+several destinations with different reliability/writing speed/latency.
+
+The behavior of fifo muxer in case of failure can be configured:
+@itemize @bullet
+
+@item
+output can be transparently restarted with configurable delay between retries
+based on real time or time of the processed stream.
+
+@item
+encoding can be blocked during temporary failure, or continue transparently
+dropping packets in case fifo queue fills up.
+
+@end itemize
+
+@table @option
+
+@item fifo_format
+Specify the format name. Useful if it cannot be guessed from the
+output name suffix.
+
+@item queue_size
+Specify size of the queue (number of packets). Default value is 60.
+
+@item format_opts
+Specify format options for the underlying muxer. Muxer options can be specified
+as a list of @var{key}=@var{value} pairs separated by ':'.
+
+@item drop_pkts_on_overflow @var{bool}
+If set to 1 (true), in case the fifo queue fills up, packets will be dropped
+rather than blocking the encoder. This allows to continue streaming without
+delaying the output, at the cost of ommiting part of the stream. By default
+this option is set to 0 (false), so in such cases the encoder will be blocked
+until the muxer processes some of the packets and none of them is lost.
+
+@item attempt_recovery @var{bool}
+If failure occurs, attempt to recover the output. This is especially useful
+when used with network output, allows to restart streaming transparently.
+By default this option set to 0 (false).
+
+@item max_recovery_attempts
+Sets maximum number of successive unsucessful recovery attempts after which
+the output fails permanently. Unlimited if set to zero. Default value is 16.
+
+@item recovery_wait_time @var{duration}
+Waiting time before the next recovery attempt after previous unsuccessfull
+recovery attempt. Default value is 5 seconds.
+
+s@item recovery_wait_streamtime @var{bool}
+If set to 0 (false), the real time is used when waiting for the recovery
+attempt (i.e. the recovery will be attempted after at least
+recovery_wait_time seconds).
+If set to 1 (true), the time of the processed stream is taken into account
+instead (i.e. the recovery will be attempted after at least recovery_wait_time
+seconds of the stream is omitted).
+By default, this option is set to 0 (false).
+
+@item recover_any_error @var{bool}
+If set to 1 (true), recovery will be attempted regardless of type of the error
+causing the failure. By default this option is set to 0 (false) and in case of
+certain errors the recovery is not attempted even when @ref{attempt_recovery}
+is set to 1.
+
+@item restart_with_keyframe @var{bool}
+Specify whether to wait for the keyframe after recovering from
+queue overflow or failure. This option is set to 0 (false) by default.
+
+@end table
+
+@subsection Examples
+
+@itemize
+
+@item
+Stream something to rtmp server, continue processing the stream at real-time
+rate even in case of temporary failure (network outage) and attempt to recover
+streaming every second indefinitely.
+@example
+ffmpeg -re -i ... -c:v libx264 -c:a mp2 -f fifo -fifo_format flv -map 0:v -map 
0:a
+  -block_on_overflow 0 -attempt_recovery 1 -recovery_wait_time 1
+  -max_recovery_attempts 0 rtmp://example.com/live/stream_name
+@end