RE: More videobuf and streaming I/O questions
>This being said, VIDIOC_S_INPUT was designed to switch analog sources. I'm >not >sure it would be the best would to multiplex streams from two digital >sensors >for instance. Even for analog signals, using the ioctl from userspace >usually >results in at least one corrupt frame if you don't synchronize the >switching >to the analog signals, which requires hardware support. > >Can you give us more details about the use case you're thinking about ? This is for supporting interfacing of TVP5148 to DM365 that can mux from two sources. There is an implementation of this in our internal release. Just exploring how this can be ported to upstream. Not done any work yet on this from my side. > >> Does it require 2 buffers per input because every frame period, you have >> multiple frames to queue from the different inputs? > >In this case there's a single video stream, so I don't think it would >require >2 buffers per input. > >> Usually a minimum of 3 buffers are typically required in a SoC case to do >> streaming. Could you share the details if possible? > >-- >Regards, > >Laurent Pinchart >-- >To unsubscribe from this list: send the line "unsubscribe linux-media" in >the body of a message to majord...@vger.kernel.org >More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: More videobuf and streaming I/O questions
Hi, On Friday 26 February 2010 14:05:30 Muralidharan Karicheri wrote: > On Fri, Feb 26, 2010 at 7:04 AM, Mauro Carvalho Chehab wrote: > > Pawel Osciak wrote: > >>> On Tuesday 23 February 2010 08:41:49 Pawel Osciak wrote: > > On Mon, 22 Feb 2010 00:12:18 +0100 > > > Laurent Pinchart wrote: > As for the REQBUF, I've always thought it'd be nice to be able to ask > the driver for the "recommended" number of buffers that should be > used by issuing a REQBUF with count=0... > >>> > >>> How would the driver come up with the number of recommended buffers ? > >> > >> From the top of my head: when encoding a video stream, a codec driver > >> could decide on the minimum number of input frames required (including > >> reference frames, etc.). > >> > >> Or maybe I am missing something, what is your opinion on that? > > > > There are some cases where this feature could be useful. For example, > > there are some devices used for surveillance that have one decoder > > connected to several inputs. For example, several bttv boards have one > > bt848 chip for each 8 inputs. Each input is connected to one camera. The > > minimum recommended number of buffers is 16 (2 per each input). This is > > poorly documented, on some wikis for some of the boards with such usage. > > > > That's said, there's currently a few missing features for surveillance: > > the user software need to manually switch from one input to another, and > > the video buffer metadata doesn't indicate the input. > > > > The better would be to provide a way to let the driver to switch to the > > next camera just after the reception of a new buffer (generally at the > > IRQ time), instead of letting the userspace software to do it at the > > DQBUF. > > This is an interesting use case and I would like to know some details > on this use case. > When you say application manually switch the input, Is it implementing some > kind of input multiplexing during the session (open, stream on - stream off, > close) ? No, applications just call VIDIOC_S_INPUT while the stream is active. > We have encountered a similar use case and I was wondering how this can be > implemented in v4l2 driver. In my understanding, a v4l2 device is not > allowed to switch input while streaming. Switching input while streaming is allowed (if I'm not mistaken), as long as the format and size don't change (not sure about lowering the size, that needs to be double-checked). This being said, VIDIOC_S_INPUT was designed to switch analog sources. I'm not sure it would be the best would to multiplex streams from two digital sensors for instance. Even for analog signals, using the ioctl from userspace usually results in at least one corrupt frame if you don't synchronize the switching to the analog signals, which requires hardware support. Can you give us more details about the use case you're thinking about ? > Does it require 2 buffers per input because every frame period, you have > multiple frames to queue from the different inputs? In this case there's a single video stream, so I don't think it would require 2 buffers per input. > Usually a minimum of 3 buffers are typically required in a SoC case to do > streaming. Could you share the details if possible? -- Regards, Laurent Pinchart -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: More videobuf and streaming I/O questions
Hi Pawel and Mauro, On Friday 26 February 2010 13:04:54 Mauro Carvalho Chehab wrote: > Pawel Osciak wrote: > >> On Tuesday 23 February 2010 08:41:49 Pawel Osciak wrote: > On Mon, 22 Feb 2010 00:12:18 +0100 > >>> > Laurent Pinchart wrote: > >>> As for the REQBUF, I've always thought it'd be nice to be able to ask > >>> the driver for the "recommended" number of buffers that should be used > >>> by issuing a REQBUF with count=0... > >> > >> How would the driver come up with the number of recommended buffers ? > > > > From the top of my head: when encoding a video stream, a codec driver > > could decide on the minimum number of input frames required (including > > reference frames, etc.). Drivers can always raise or lower the number of buffers passed as the VIDIOC_REQBUFS argument, so we already have a way to handle hardware requirements there. If we really need a way to tell the driver "please decide on the number of buffers for me", we could use a flag/magic value for the buffer count instead of using 0. The V4L2 specification clearly states that a count of 0 frees the buffers, and several applications rely on that feature. > > Or maybe I am missing something, what is your opinion on that? > > There are some cases where this feature could be useful. For example, there > are some devices used for surveillance that have one decoder connected to > several inputs. For example, several bttv boards have one bt848 chip for > each 8 inputs. Each input is connected to one camera. The minimum > recommended number of buffers is 16 (2 per each input). Why two per input ? There's a single video stream, buffers are not queued separately for each input. Beside, even if the number of recommended buffers was 2 per input, I would expect applications to know about that. If an application decides to open a single video node and call VIDIOC_S_INPUT during streaming (or configure the driver to do it automatically at IRQ time, which is conceptually similar), the application should be able to compute the required number of buffers. > This is poorly documented, on some wikis for some of the boards with such > usage. > > That's said, there's currently a few missing features for surveillance: the > user software need to manually switch from one input to another, and the > video buffer metadata doesn't indicate the input. There's actually an input field in v4l2_buffer. As far as I know it's only used by an out-of-tree, closed source driver that nobody is using anymore (I'm the one who requested a reserved field to be turned into the input field back then). Now that I'm a bit more knowledgeable about V4L2 and Linux in general, I don't think that's the best way to pass metadata around. The v4l2_buffer structure won't be able to hold all metadata we need in the future. > The better would be to provide a way to let the driver to switch to the > next camera just after the reception of a new buffer (generally at the IRQ > time), instead of letting the userspace software to do it at the DQBUF. That would be an improvement, but even then it might be too late. The only way to handle analog input switching reliably is to synchronize the input switch to the analog signal, and that must be done by the hardware. That kind of feature is not commonly found in cheap bttv boards. -- Regards, Laurent Pinchart -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: More videobuf and streaming I/O questions
On Fri, Feb 26, 2010 at 7:04 AM, Mauro Carvalho Chehab wrote: > Pawel Osciak wrote: >>> On Tuesday 23 February 2010 08:41:49 Pawel Osciak wrote: > On Mon, 22 Feb 2010 00:12:18 +0100 > Laurent Pinchart wrote: As for the REQBUF, I've always thought it'd be nice to be able to ask the driver for the "recommended" number of buffers that should be used by issuing a REQBUF with count=0... >>> How would the driver come up with the number of recommended buffers ? >> >> From the top of my head: when encoding a video stream, a codec driver could >> decide on the minimum number of input frames required (including reference >> frames, etc.). >> >> Or maybe I am missing something, what is your opinion on that? > > There are some cases where this feature could be useful. For example, there > are > some devices used for surveillance that have one decoder connected to several > inputs. For example, several bttv boards have one bt848 chip for each 8 > inputs. > Each input is connected to one camera. The minimum recommended number of > buffers > is 16 (2 per each input). This is poorly documented, on some wikis for some of > the boards with such usage. > > That's said, there's currently a few missing features for surveillance: the > user > software need to manually switch from one input to another, and the video > buffer > metadata doesn't indicate the input. > > The better would be to provide a way to let the driver to switch to the next > camera > just after the reception of a new buffer (generally at the IRQ time), instead > of > letting the userspace software to do it at the DQBUF. > This is an interesting use case and I would like to know some details on this use case. When you say application manually switch the input, Is it implementing some kind of input multiplexing during the session (open, stream on - stream off, close) ? We have encountered a similar use case and I was wondering how this can be implemented in v4l2 driver. In my understanding, a v4l2 device is not allowed to switch input while streaming. Does it require 2 buffers per input because every frame period, you have multiple frames to queue from the different inputs? Usually a minimum of 3 buffers are typically required in a SoC case to do streaming. Could you share the details if possible? Murali > -- > > Cheers, > Mauro > -- > To unsubscribe from this list: send the line "unsubscribe linux-media" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Murali Karicheri mkarich...@gmail.com -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: More videobuf and streaming I/O questions
Pawel Osciak wrote: >> On Tuesday 23 February 2010 08:41:49 Pawel Osciak wrote: On Mon, 22 Feb 2010 00:12:18 +0100 Laurent Pinchart wrote: >>> As for the REQBUF, I've always thought it'd be nice to be able to ask the >>> driver for the "recommended" number of buffers that should be used by >>> issuing a REQBUF with count=0... >> How would the driver come up with the number of recommended buffers ? > > From the top of my head: when encoding a video stream, a codec driver could > decide on the minimum number of input frames required (including reference > frames, etc.). > > Or maybe I am missing something, what is your opinion on that? There are some cases where this feature could be useful. For example, there are some devices used for surveillance that have one decoder connected to several inputs. For example, several bttv boards have one bt848 chip for each 8 inputs. Each input is connected to one camera. The minimum recommended number of buffers is 16 (2 per each input). This is poorly documented, on some wikis for some of the boards with such usage. That's said, there's currently a few missing features for surveillance: the user software need to manually switch from one input to another, and the video buffer metadata doesn't indicate the input. The better would be to provide a way to let the driver to switch to the next camera just after the reception of a new buffer (generally at the IRQ time), instead of letting the userspace software to do it at the DQBUF. -- Cheers, Mauro -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: More videobuf and streaming I/O questions
>On Tuesday 23 February 2010 08:41:49 Pawel Osciak wrote: >> >On Mon, 22 Feb 2010 00:12:18 +0100 >> >Laurent Pinchart wrote: >> As for the REQBUF, I've always thought it'd be nice to be able to ask the >> driver for the "recommended" number of buffers that should be used by >> issuing a REQBUF with count=0... > >How would the driver come up with the number of recommended buffers ? >From the top of my head: when encoding a video stream, a codec driver could decide on the minimum number of input frames required (including reference frames, etc.). Or maybe I am missing something, what is your opinion on that? Best regards -- Pawel Osciak Linux Platform Group Samsung Poland R&D Center -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: More videobuf and streaming I/O questions
Hi Pavel, On Tuesday 23 February 2010 08:41:49 Pawel Osciak wrote: > >On Mon, 22 Feb 2010 00:12:18 +0100 > >Laurent Pinchart wrote: > >> On Saturday 20 February 2010 15:00:21 Hans Verkuil wrote: > >> > 1) The spec mentions that the memory field should be set for > >> > VIDIOC_DQBUF. But videobuf doesn't need it and it makes no sense to > >> > me either unless it is for symmetry with VIDIOC_QBUF. Strictly > >> > speaking QBUF doesn't need it either, but it is a good sanity check. > >> > > >> > Can I remove the statement in the spec that memory should be set > >> > for DQBUF? The alternative is to add a check against the memory > >> > field in videobuf, but that's rather scary. > >> > >> In that case I would remove it for QBUF as well, and state that the > >> memory field must be ignored by drivers (but should they fill it when > >> returning from QBUF/DQBUF ?) > > > >Agree. It seems that the memory field is not useful at all in the struct > >v4l2_buffer if a same process does reqbuf, qbuf, dqbuf and querybuf. > > In the current multi-plane buffer proposal, the memory field is required > in querybuf, qbuf and dqbuf for the v4l2-ioctl.c code to be able to > determine whether the planes array should be copied from/to user along > with the buffer. > Just wanted to add another view to the problem, as multiplanes are accepted > yet of course. Thanks for the reminder. I'm not against keeping the requirement for applications to set the memory field in the v4l2_buffer structure. QBUF and DQBUF should behave the same way though, they should both require the field to be set or not require it at all. > As for the REQBUF, I've always thought it'd be nice to be able to ask the > driver for the "recommended" number of buffers that should be used by > issuing a REQBUF with count=0... How would the driver come up with the number of recommended buffers ? -- Regards, Laurent Pinchart -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: More videobuf and streaming I/O questions
Hi Jean-François, On Monday 22 February 2010 10:47:41 Jean-Francois Moine wrote: > Hi Hans and Laurent, > > On Mon, 22 Feb 2010 00:12:18 +0100 > > Laurent Pinchart wrote: > > On Saturday 20 February 2010 15:00:21 Hans Verkuil wrote: > > > 1) The spec mentions that the memory field should be set for > > > VIDIOC_DQBUF. But videobuf doesn't need it and it makes no sense to > > > me either unless it is for symmetry with VIDIOC_QBUF. Strictly > > > speaking QBUF doesn't need it either, but it is a good sanity check. > > > > > > Can I remove the statement in the spec that memory should be set > > > for DQBUF? The alternative is to add a check against the memory > > > field in videobuf, but that's rather scary. > > > > In that case I would remove it for QBUF as well, and state that the > > memory field must be ignored by drivers (but should they fill it when > > returning from QBUF/DQBUF ?) > > Agree. It seems that the memory field is not useful at all in the struct > v4l2_buffer if a same process does reqbuf, qbuf, dqbuf and querybuf. > > > BTW, I had a pending question. The spec says that streamoff 'removes > all buffers from the incoming and outgoing queues' and return to 'the > same state as after calling VIDIOC_REQBUFS'. For output, there is no > problem. For capture, does this mean that the buffers previously queued > by qbuf are implicitly unqueued (i.e. that qbuf must be done again for > all buffers)? That's correct. > In this case, streamoff does not work with two processes. A first > process is streaming when a second one does streamoff and then > streamon. The first process will stay blocked on polling because no > buffer is queued anymore. It cannot know this fact and the second > process cannot requeue the buffers... I don't think this multiple process use case is valid. The V4L2 streaming API wasn't designed to be used in a multi-thread or multi-process context in the first place. > To work correctly, the spec should say that streamoff discards the > content of the filled buffers and that it requeues these buffers as > empty either in the driver's incoming queue (capture) or outgoing queue > (output). I don't agree. If we did that, buffers couldn't be released after a STREAMOFF. Queued buffers belong to the driver, so to free the buffers applications would have to call VIDIOC_STREAMOFF and then dequeue all buffers. That's not pretty. -- Regards, Laurent Pinchart -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: More videobuf and streaming I/O questions
Hello, >On Mon, 22 Feb 2010 00:12:18 +0100 >Laurent Pinchart wrote: >> On Saturday 20 February 2010 15:00:21 Hans Verkuil wrote: >> > 1) The spec mentions that the memory field should be set for >> > VIDIOC_DQBUF. But videobuf doesn't need it and it makes no sense to >> > me either unless it is for symmetry with VIDIOC_QBUF. Strictly >> > speaking QBUF doesn't need it either, but it is a good sanity check. >> > >> > Can I remove the statement in the spec that memory should be set >> > for DQBUF? The alternative is to add a check against the memory >> > field in videobuf, but that's rather scary. >> >> In that case I would remove it for QBUF as well, and state that the >> memory field must be ignored by drivers (but should they fill it when >> returning from QBUF/DQBUF ?) > >Agree. It seems that the memory field is not useful at all in the struct >v4l2_buffer if a same process does reqbuf, qbuf, dqbuf and querybuf. In the current multi-plane buffer proposal, the memory field is required in querybuf, qbuf and dqbuf for the v4l2-ioctl.c code to be able to determine whether the planes array should be copied from/to user along with the buffer. Just wanted to add another view to the problem, as multiplanes are accepted yet of course. As for the REQBUF, I've always thought it'd be nice to be able to ask the driver for the "recommended" number of buffers that should be used by issuing a REQBUF with count=0... Best regards -- Pawel Osciak Linux Platform Group Samsung Poland R&D Center -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: More videobuf and streaming I/O questions
Hi Hans and Laurent, On Mon, 22 Feb 2010 00:12:18 +0100 Laurent Pinchart wrote: > On Saturday 20 February 2010 15:00:21 Hans Verkuil wrote: > > 1) The spec mentions that the memory field should be set for > > VIDIOC_DQBUF. But videobuf doesn't need it and it makes no sense to > > me either unless it is for symmetry with VIDIOC_QBUF. Strictly > > speaking QBUF doesn't need it either, but it is a good sanity check. > > > > Can I remove the statement in the spec that memory should be set > > for DQBUF? The alternative is to add a check against the memory > > field in videobuf, but that's rather scary. > > In that case I would remove it for QBUF as well, and state that the > memory field must be ignored by drivers (but should they fill it when > returning from QBUF/DQBUF ?) Agree. It seems that the memory field is not useful at all in the struct v4l2_buffer if a same process does reqbuf, qbuf, dqbuf and querybuf. BTW, I had a pending question. The spec says that streamoff 'removes all buffers from the incoming and outgoing queues' and return to 'the same state as after calling VIDIOC_REQBUFS'. For output, there is no problem. For capture, does this mean that the buffers previously queued by qbuf are implicitly unqueued (i.e. that qbuf must be done again for all buffers)? In this case, streamoff does not work with two processes. A first process is streaming when a second one does streamoff and then streamon. The first process will stay blocked on polling because no buffer is queued anymore. It cannot know this fact and the second process cannot requeue the buffers... To work correctly, the spec should say that streamoff discards the content of the filled buffers and that it requeues these buffers as empty either in the driver's incoming queue (capture) or outgoing queue (output). Cheers. -- Ken ar c'hentañ | ** Breizh ha Linux atav! ** Jef | http://moinejf.free.fr/ -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: More videobuf and streaming I/O questions
Hi Hans, On Saturday 20 February 2010 15:00:21 Hans Verkuil wrote: > I have a few more questions regarding the streaming I/O API: > > 1) The spec mentions that the memory field should be set for VIDIOC_DQBUF. > But videobuf doesn't need it and it makes no sense to me either unless it > is for symmetry with VIDIOC_QBUF. Strictly speaking QBUF doesn't need it > either, but it is a good sanity check. > > Can I remove the statement in the spec that memory should be set for DQBUF? > The alternative is to add a check against the memory field in videobuf, but > that's rather scary. In that case I would remove it for QBUF as well, and state that the memory field must be ignored by drivers (but should they fill it when returning from QBUF/DQBUF ?) > 2) What to do with REQBUFS when called with a count of 0? Thinking it over > I agree that it shouldn't do an implicit STREAMOFF. But I do think that it > is useful to allow as a simple check whether the I/O method is supported. REQBUFS(0) should also free allocated buffers (if any). > So a count of 0 will either return an error if streaming is still in > progress or if the proposed I/O method is not supported, otherwise it will > return 0 while leaving count to 0. > > This allows one to use REQBUFS to test which I/O methods are supported by > the driver without having the driver allocating any buffers. > > This will become more important with embedded systems where almost > certainly additional I/O methods will be introduced (in particular > non-contiguous plane support). > > Currently a count of 0 will result in an error in videobuf. > > Note that drivers do not generally check for valid values of the memory > field at the moment. So that is another thing we need to improve. But > before I start working on that, I first want to know exactly how REQBUFS > should work. -- Regards, Laurent Pinchart -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
