Re: [FFmpeg-devel] [PATCH] avfilter/WIP: add streamselect filter

[Clément Bœsch]

On Thu, Nov 19, 2015 at 06:47:07PM +0100, Nicolas George wrote:
> Le nonidi 29 brumaire, an CCXXIV, Clement Boesch a écrit :
> > - I always receive frames from the first input link... I suppose I need
> >   to write a "smart" request_frame callback, but things changed recently
> >   so maybe there are new guidelines for this? Nicolas, can I call for
> >   your help again?
> 
> Nothing new except you do not have to loop: just forward the request on the
> output to the connected input. But you have to decide what to do when an
> output is not connected to any input; EAGAIN comes to mind, but I am not
> entirely sure it will work correctly. For now I would suggest to make it
> unsupported: either an input is connected or it is EOF.
> 

Alright, so if I understand well, all I have to do is implement a
forwarding such as this:

static int request_frame(AVFilterLink *outlink)
{
AVFilterContext *ctx = outlink->src;
StreamSelectContext *s = ctx->priv;
const int outlink_idx = FF_OUTLINK_IDX(outlink);
const int inlink_idx = s->map[outlink_idx];
AVFilterLink *inlink = ctx->inputs[inlink_idx];
return ff_request_frame(inlink);
}

Which indeed seems to do the trick when set for to every output pad
request_frame callback.

Unfortunately, it seems the other stream(s) in "background" (not present
in the mapping) will not be consumed at all (filter_frame() is not called)
so frames are never dropped. As a result, when switching to that
background stream, the frames are delayed by a large margin (the amount of
time the main stream has been in use).

While I still need to add a timing check in the context (to avoid small
non monotonically problems), how would it be possible to continue
consuming the background streams?

-- 
Clément B.


signature.asc
Description: PGP signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

Re: [FFmpeg-devel] [PATCH] avfilter/WIP: add streamselect filter

[Nicolas George]

Le decadi 30 brumaire, an CCXXIV, Clement Boesch a écrit :
> Alright, so if I understand well, all I have to do is implement a
> forwarding such as this:
> 
> static int request_frame(AVFilterLink *outlink)
> {
> AVFilterContext *ctx = outlink->src;
> StreamSelectContext *s = ctx->priv;
> const int outlink_idx = FF_OUTLINK_IDX(outlink);
> const int inlink_idx = s->map[outlink_idx];
> AVFilterLink *inlink = ctx->inputs[inlink_idx];
> return ff_request_frame(inlink);
> }

Indeed. Note: you may want to keep both "in->out" and "out->in" maps in
arrays.

> Unfortunately, it seems the other stream(s) in "background" (not present
> in the mapping) will not be consumed at all (filter_frame() is not called)
> so frames are never dropped. As a result, when switching to that
> background stream, the frames are delayed by a large margin (the amount of
> time the main stream has been in use).

If the corresponding input is connected to a buffersrc fed from a file, that
is perfectly normal.

> While I still need to add a timing check in the context (to avoid small
> non monotonically problems), how would it be possible to continue
> consuming the background streams?

I had not thought about this issue before now. I think it is not possible in
principle. Consider this graph:

  +---+
[v1] >|.  |   ++
  | `>|>[ setpts ×2 ]>||
[v2] >|   |   | vstack |>
  | .>|-->||
[v3] >|´  |   ++
  +---+

With the ×2 setpts, v1 must be consumed twice as fast as v3 (or the other
way around, does not matter). But then, what happens when v2 gets connected
to one of the outputs?

You need to decide the constraints or the user interface to solve this
before you can implement anything.

Regards,

-- 
  Nicolas George


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel

[FFmpeg-devel] [PATCH] avfilter/WIP: add streamselect filter

[Clément Bœsch]

From: Clément Bœsch 

---
No documentation so the concept is simply to map N streams to M other
streams, with the possibility to live configure the filter
(process_command).

For example, let's say you have 2 a/v inputs and you want to live
switch between the two, you will do something like:

  ffmpeg -i ... -i ... -lavfi 
"[0:0][0:1][1:0][1:1]streamselect=in=vava:out=va:map=0 1"

then send commands such as "map 0 1" or "map 2 3" to select one or
another source.

This is a WIP because I have a few issues right now (and also because
mostly untested):

- I always receive frames from the first input link... I suppose I need
  to write a "smart" request_frame callback, but things changed recently
  so maybe there are new guidelines for this? Nicolas, can I call for
  your help again?

- the command input in ffmpeg cli doesn't work anymore here (reproduce
  with: `ffmpeg -f lavfi -i testsrc -f null -` and press 'c': can't type
  anything then). I hacked something around to be able to send commands.

- sendcmd/zmq filters seem to need to be in the filtergraph itself to
  allow forwarding commands, and it doesn't seem possible to fit well
  with "multimedia" filters (we have sendcmd/asendcmd zmq/azmq)
---
 libavfilter/Makefile   |   1 +
 libavfilter/allfilters.c   |   1 +
 libavfilter/avf_streamselect.c | 215 +
 3 files changed, 217 insertions(+)
 create mode 100644 libavfilter/avf_streamselect.c

diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 1f4abeb..f2e6b0d 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -281,6 +281,7 @@ OBJS-$(CONFIG_SHOWSPECTRUM_FILTER)   += 
avf_showspectrum.o
 OBJS-$(CONFIG_SHOWVOLUME_FILTER) += avf_showvolume.o
 OBJS-$(CONFIG_SHOWWAVES_FILTER)  += avf_showwaves.o
 OBJS-$(CONFIG_SHOWWAVESPIC_FILTER)   += avf_showwaves.o
+OBJS-$(CONFIG_STREAMSELECT_FILTER)   += avf_streamselect.o
 
 # multimedia sources
 OBJS-$(CONFIG_AMOVIE_FILTER) += src_movie.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 63b8fdb..cf805df 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -302,6 +302,7 @@ void avfilter_register_all(void)
 REGISTER_FILTER(SHOWVOLUME, showvolume, avf);
 REGISTER_FILTER(SHOWWAVES,  showwaves,  avf);
 REGISTER_FILTER(SHOWWAVESPIC,   showwavespic,   avf);
+REGISTER_FILTER(STREAMSELECT,   streamselect,   avf);
 
 /* multimedia sources */
 REGISTER_FILTER(AMOVIE, amovie, avsrc);
diff --git a/libavfilter/avf_streamselect.c b/libavfilter/avf_streamselect.c
new file mode 100644
index 000..4a424ed
--- /dev/null
+++ b/libavfilter/avf_streamselect.c
@@ -0,0 +1,215 @@
+/*
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/avstring.h"
+#include "libavutil/internal.h"
+#include "libavutil/opt.h"
+#include "avfilter.h"
+#include "audio.h"
+#include "formats.h"
+#include "internal.h"
+#include "video.h"
+
+typedef struct StreamSelectContext {
+const AVClass *class;
+char *in_str;
+char *out_str;
+char *map_str;
+int *map;
+int nb_map;
+} StreamSelectContext;
+
+#define OFFSET(x) offsetof(StreamSelectContext, x)
+#define FLAGS AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_FILTERING_PARAM
+static const AVOption streamselect_options[] = {
+{ "in",  "input streams definition",  OFFSET(in_str),  
AV_OPT_TYPE_STRING, {.str=NULL}, .flags=FLAGS },
+{ "out", "output streams definition", OFFSET(out_str), 
AV_OPT_TYPE_STRING, {.str=NULL}, .flags=FLAGS },
+{ "map", "input indexes to remap to outputs", OFFSET(map_str), 
AV_OPT_TYPE_STRING, {.str=NULL}, .flags=FLAGS },
+{ NULL }
+};
+
+AVFILTER_DEFINE_CLASS(streamselect);
+
+static int filter_frame(AVFilterLink *inlink, AVFrame *frame)
+{
+int i;
+AVFilterContext *ctx = inlink->dst;
+StreamSelectContext *s = ctx->priv;
+const int inlink_idx = FF_INLINK_IDX(inlink);
+
+//av_log(0,0,"inlink %d %p\n", inlink_idx, inlink);
+
+for (i = 0; i < s->nb_map; i++)
+if (s->map[i] == inlink_idx)
+return ff_filter_frame(ctx->outputs[i], frame);
+av_frame_free();
+return 0;
+}
+

Re: [FFmpeg-devel] [PATCH] avfilter/WIP: add streamselect filter

[Nicolas George]

Le nonidi 29 brumaire, an CCXXIV, Clement Boesch a écrit :
> - I always receive frames from the first input link... I suppose I need
>   to write a "smart" request_frame callback, but things changed recently
>   so maybe there are new guidelines for this? Nicolas, can I call for
>   your help again?

Nothing new except you do not have to loop: just forward the request on the
output to the connected input. But you have to decide what to do when an
output is not connected to any input; EAGAIN comes to mind, but I am not
entirely sure it will work correctly. For now I would suggest to make it
unsupported: either an input is connected or it is EOF.

Regards,

-- 
  Nicolas George


signature.asc
Description: Digital signature
___
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel