Re: Media controller: sysfs vs ioctl
On Monday 21 September 2009 19:23:54 Sakari Ailus wrote: Hans Verkuil wrote: On Sunday 13 September 2009 08:13:04 Nathaniel Kim wrote: 2009. 9. 12., 오전 7:21, Hans Verkuil 작성: Hans, First of all I'm very sorry that I had not enough time to go through your new RFC. I'll checkout right after posting this mail. I think this is a good approach and I also had in my mind that sysfs might be a good method if we could control and monitor through this. Recalling memory when we had a talk in San Francisco, I was frustrated that there is no way to catch events from sort of sub-devices like lens actuator (I mean pizeo motors in camera module). As you know lens actuator is an extremely slow device in comparison with common v4l2 devices we are using and we need to know whether it has succeeded or not in moving to expected position. So I considered sysfs and udev as candidates for catching events from sub-devices. events like success/failure of lens movement, change of status of subdevices. Does anybody experiencing same issue? I think I've seen a lens controller driver in omap3 kernel from TI but not sure how did they control that. My point is that we need a kind of framework to give and event to user space and catching them properly just like udev does. When I was talking to Laurent Pinchart and Sakari and his team at Nokia we discussed just such a framework. It actually exists already, although it is poorly implemented. Look at include/linux/dvb/video.h, struct video_event and ioctl VIDEO_GET_EVENT. It is used in ivtv (ivtv-ioctl.c, look for VIDEO_GET_EVENT). The idea is that you can either call VIDEO_GET_EVENT to wait for an event or use select() and wait for an exception to arrive, and then call VIDEO_GET_EVENT to find which event it was. This is ideal for streaming-related events. In ivtv it is used to report VSYNCs and to report when the MPEG decoder stopped (there is a delay between stopping sending new data to the decoder and when it actually processed all its internal buffers). Laurent is going to look into this to clean it up and present it as a new proper official V4L2 event mechanism. For events completely specific to a subdev I wonder whether it wouldn't be a good idea to use the media controller device for that. I like the select() mechanism since in an application you can just select() on a whole bunch of filehandles. If you can't use select() then you are forced to do awkward coding (e.g. make a separate thread just to handle that other event mechanism). Agree. There's no reasonable way to use video devices here since the events may be connected to non-video related issues --- like the statistics. One possible approach could be allocating a device node for each subdev and use them and leave the media controller device with just the media controller specific ioctls. Then there would be no need to set current subdev nor bind the subdev to file handle either. Just an idea. So with the media controller we can easily let sub-devices notify the media controller when an event is ready and the media controller can then generate an exception. An application can just select() on the mc filehandle. There are two ways of implementing this. One is that the media controller keeps a global queue of pending events and subdevices just queue events to that when they arrive (with some queue size limit to prevent run-away events). With the above arrangement, the events could be easily subdev specific. The mechanism should be generic still, though. So when you call some GET_EVENT type ioctl it should return the ID of the subdevice (aka entity) as well. What makes me slightly uncomfortable is that you still want to use that same ioctl on a normal video node. And the subdev ID has really no meaning there. But making two different ioctls doesn't sit well with me either. The alternative implementation is that the mc will only wait for events from the currently selected sub-device. So if you want to wait on events from different sub-devices, then you have to open the mc multiple times, once for each subdev that you want to receive events from. I think I would probably go for the second implementation because it is consistent with the way ioctls are passed to sub-devices. I like the idea that you can just pass regular V4L2 ioctls to sub-devices. Not all ioctls make sense, obviously (e.g. any of the streaming I/O ioctls), but a surprisingly large number of ioctls can be used in that way. I agree with this. There are just a few ioctls that probably don't make sense (e.g. the steaming related ones). IMO even the format setting ioctls could be nice since the possible input and output formats of the subdevs should be enumerable, too. ENUM_FRAMESIZES and ENUM_FRAMEINTERVALS are missing the v4l2_buf_type, but there are reserved fields... Those two
Re: Media controller: sysfs vs ioctl
Hans Verkuil wrote: On Sunday 13 September 2009 08:13:04 Nathaniel Kim wrote: 2009. 9. 12., 오전 7:21, Hans Verkuil 작성: Hans, First of all I'm very sorry that I had not enough time to go through your new RFC. I'll checkout right after posting this mail. I think this is a good approach and I also had in my mind that sysfs might be a good method if we could control and monitor through this. Recalling memory when we had a talk in San Francisco, I was frustrated that there is no way to catch events from sort of sub-devices like lens actuator (I mean pizeo motors in camera module). As you know lens actuator is an extremely slow device in comparison with common v4l2 devices we are using and we need to know whether it has succeeded or not in moving to expected position. So I considered sysfs and udev as candidates for catching events from sub-devices. events like success/failure of lens movement, change of status of subdevices. Does anybody experiencing same issue? I think I've seen a lens controller driver in omap3 kernel from TI but not sure how did they control that. My point is that we need a kind of framework to give and event to user space and catching them properly just like udev does. When I was talking to Laurent Pinchart and Sakari and his team at Nokia we discussed just such a framework. It actually exists already, although it is poorly implemented. Look at include/linux/dvb/video.h, struct video_event and ioctl VIDEO_GET_EVENT. It is used in ivtv (ivtv-ioctl.c, look for VIDEO_GET_EVENT). The idea is that you can either call VIDEO_GET_EVENT to wait for an event or use select() and wait for an exception to arrive, and then call VIDEO_GET_EVENT to find which event it was. This is ideal for streaming-related events. In ivtv it is used to report VSYNCs and to report when the MPEG decoder stopped (there is a delay between stopping sending new data to the decoder and when it actually processed all its internal buffers). Laurent is going to look into this to clean it up and present it as a new proper official V4L2 event mechanism. For events completely specific to a subdev I wonder whether it wouldn't be a good idea to use the media controller device for that. I like the select() mechanism since in an application you can just select() on a whole bunch of filehandles. If you can't use select() then you are forced to do awkward coding (e.g. make a separate thread just to handle that other event mechanism). Agree. There's no reasonable way to use video devices here since the events may be connected to non-video related issues --- like the statistics. One possible approach could be allocating a device node for each subdev and use them and leave the media controller device with just the media controller specific ioctls. Then there would be no need to set current subdev nor bind the subdev to file handle either. Just an idea. So with the media controller we can easily let sub-devices notify the media controller when an event is ready and the media controller can then generate an exception. An application can just select() on the mc filehandle. There are two ways of implementing this. One is that the media controller keeps a global queue of pending events and subdevices just queue events to that when they arrive (with some queue size limit to prevent run-away events). With the above arrangement, the events could be easily subdev specific. The mechanism should be generic still, though. So when you call some GET_EVENT type ioctl it should return the ID of the subdevice (aka entity) as well. What makes me slightly uncomfortable is that you still want to use that same ioctl on a normal video node. And the subdev ID has really no meaning there. But making two different ioctls doesn't sit well with me either. The alternative implementation is that the mc will only wait for events from the currently selected sub-device. So if you want to wait on events from different sub-devices, then you have to open the mc multiple times, once for each subdev that you want to receive events from. I think I would probably go for the second implementation because it is consistent with the way ioctls are passed to sub-devices. I like the idea that you can just pass regular V4L2 ioctls to sub-devices. Not all ioctls make sense, obviously (e.g. any of the streaming I/O ioctls), but a surprisingly large number of ioctls can be used in that way. I agree with this. There are just a few ioctls that probably don't make sense (e.g. the steaming related ones). IMO even the format setting ioctls could be nice since the possible input and output formats of the subdevs should be enumerable, too. ENUM_FRAMESIZES and ENUM_FRAMEINTERVALS are missing the v4l2_buf_type, but there are reserved fields... -- Sakari Ailus sakari.ai...@maxwell.research.nokia.com -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org
Re: Media controller: sysfs vs ioctl
On Saturday 12 September 2009 00:21:48 Hans Verkuil wrote: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. [snip] What sort of interaction do we need with sub-devices? [snip] 2) Private ioctls. Basically a way to set and get data that is hardware specific from the sub-device. This can be anything from statistics, histogram information, setting resizer coefficients, configuring colorspace converters, whatever. Furthermore, just like the regular V4L2 API it has to be designed with future additions in mind (i.e. it should use something like reserved fields). In my opinion ioctls are ideal for this since they are very flexible and easy to use and efficient. Especially because you can combine multiple fields into one unit. So getting histogram data through an ioctl will also provide the timestamp. And you can both write and read data in one atomic system call. Yes, you can combine data in sysfs reads as well. Although an IORW ioctls is hard to model in sysfs. But whereas an ioctl can just copy a struct from kernel to userspace, for sysfs you have to go through a painful process of parsing and formatting. And not all this data is small. Histogram data can be 10s of kilobytes. 'cat' will typically only read 4 kB at a time, so you will probably have to keep track of how much is read in the read implementation of the attribute. Or does the kernel do that for you? If I'm not mistaken sysfs binary attributes can't be bigger than 4kB in size. [snip] The final part is how to represent the topology. Device nodes and sub-devices can be exposed to sysfs as discussed earlier. Representing both the possible and current links between them is a lot harder. This is especially true for non-v4l device nodes since we cannot just add attributes there. I think this can be done though by providing that information as attributes of an mc sysfs node. That way it remains under control of the v4l core. I was a bit concerned (to say the least) when I started catching up with my e- mails and reading this thread that the discussion would heat up and split developers between two sides. While trying to find arguments to convince people that my side is better (and of course it is, otherwise I would be on the other side :-)) I realized that the whole media controller problem might be understood differently by the two sides. I'll try to shed some light on this in the hope that it will bring the v4l developers together. sysfs was designed to expose kernel objects arranged in a tree-like fashion. It does that pretty well, although one of its weak points is that it can be easily abused. From the kernel point of view, (most of) the various sub-devices in a media device are arranged in a tree of kernel objects. Most of the time we have an I2C controller and various devices sitting on the I2C bus, one or several video devices that sit on some internal bus (usually a SoC internal bus for the most complex and recent platforms), and possibly SPI and other devices as well. Realizing that, as all those sub-devices are already exposed in sysfs in one way or the other, it was tempting to add a few attributes and soft links to solve the media controller problem. However, that solution, even if it might seem simple, misses a very important point. Sub-devices are arranged in a tree-like objects structure from the kernel point of view, but from the media controller point of view they are not. While the kernel cares about kernel objects that are mostly devices on busses in parent-children relationships, the media controller cares about how video is transferred between sub-devices. And those two concepts are totally different. The sub-devices, from the media controller point of view, make an oriented graph of connected nodes. When setting video controls, selecting formats and streaming video, what we care about it how the video will flow from its source (a sensor, a physical connector, memory, whatever) to its sink (same list of possible whatevers). This is what the media controller needs to deal with. We need to expose a connected graph of nodes to userspace, and let userspace access the nodes and the links for various operations. Of course it would be possible to handle that through sysfs, as sub-devices are already exposed there. But let's face it, it wouldn't be practical, efficient or even clean. Hans already mentioned several reasons why using sysfs attributes to replace all media controller ioctls would be cumbersome. I would add that we need to transfer large amounts of aggregated data in some cases (think about statistics), and sysfs attributes are simply not designed for that. There's a 4kB limit for text attributes, and Documentation/filesystems/sysfs.txt states that Attributes should be ASCII text files, preferably with only one value per file. It is noted that it may not be
Re: Media controller: sysfs vs ioctl
From the kernel point of view, (most of) the various sub-devices in a media device are arranged in a tree of kernel objects. Most of the time we have an I2C controller and various devices sitting on the I2C bus, one or several video devices that sit on some internal bus (usually a SoC internal bus for the most complex and recent platforms), and possibly SPI and other devices as well. Realizing that, as all those sub-devices are already exposed in sysfs in one way or the other, it was tempting to add a few attributes and soft links to solve the media controller problem. I just wanted to make one clarification: sub-devices are usually, but not always mapped 1-to-1 to a true kernel device. But sub-devices are an abstract concept and it is possible to have multiple sub-devices exposed by a single kernel device. Or even to have one sub-device covering two or more kernel devices (no such beast exists, but nothing prevents this technically). I know of two instances where the sub-device has no sysfs counterpart. And I expect more will follow. Regards, Hans -- Hans Verkuil - video4linux developer - sponsored by TANDBERG Telecom -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [linux-dvb] Media controller: sysfs vs ioctl
On Tue, Sep 15, 2009 at 2:50 PM, david may david.ma...@ntlworld.com wrote: OC you do realize that IF the DVB/linuxTV devs cant even make a simple and quick LiveCD with a VLC and streaming apps clients and servers connecting to a web sever some were for popping into ANY laptop that happens to be around at the time then theres a problem with mass credibility somewhere how hard can it be to set up a Multicast VLC 224.0.0.1 tunnel over a free IPv6 supplyer to the DVB/LinuxTV servers and Unicast a WVGA or better video stream or four to any VLC clients wanting to follow the feeds , its just a shame theres not easy way to Add an annotation to feeds and/or live transcripts or even skip to a section of video BBC Iplayer style link directly to your favourite scene http://www.bbc.co.uk/blogs/bbcinternet/2009/07/bbc_iplayer_now_lets_you_link.html Steve Hughes in Michael McIntyre's brilliant Comedy Roadshow - see him here: http://bbc.co.uk/i/lbtbg/?t=16m51s these are the things we should be seeing in ALL remote participation feeds and More -- Best regards, david mailto:david.ma...@ntlworld.com David, It really doesn't say anything about the credibility of the developers, other than that perhaps they are already severely overloaded and don't have the cycles to create a setup such as the one you are proposing. This is unlikely to happen not because it's not technically possible, but because the developers have other stuff that is more important to be working on. Devin -- Devin J. Heitmueller - Kernel Labs http://www.kernellabs.com -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Media controller: sysfs vs ioctl
Hi, In our experience, sysfs was useful for simple control mechanism such as enable/disable or displaying statistics or status. But we had received customer complaints that with this approach, these functionality will become unavailable when kernel is built without sysfs as part of size optimization. So if this is really true, I don't think sysfs is the right candidate for MC. Since sysfs is more string oriented, won't it increase the code size when it is used for parsing a lot of variable/value pair to setup device hw configuration ? Besides, most of the application that is written for TI video drivers are based on ioctl and it would make technical support a nightmare with a different API used for device configuration. Murali From: linux-media-ow...@vger.kernel.org [linux-media-ow...@vger.kernel.org] On Behalf Of Mauro Carvalho Chehab [mche...@infradead.org] Sent: Sunday, September 13, 2009 7:31 PM To: wk Cc: Hans Verkuil; linux-media@vger.kernel.org Subject: Re: Media controller: sysfs vs ioctl Em Sun, 13 Sep 2009 17:54:11 +0200 wk handygewinnsp...@gmx.de escreveu: Hans Verkuil schrieb: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. Is it really a good idea to create a dependency to some virtual file system which may go away in future? From time to time some of those seem to go away, for example devfs. Is it really unavoidable to have something in sysfs, something which is really not possible with ioctls? And do you really want to depend on sysfs developers? First of all, both ioctl's and sysfs are part of vfs support. Second: where did you got the wrong information that sysfs would be deprecated? There's no plan to deprecate sysfs, and, since there are lots of kernel-userspace API's depending on sysfs, you can't just remove it. It is completely different from what we had with devfs, where just device names were created there, on a limited way (for example, no directories were allowed at devfs). Yet, before devfs removal, sysfs was added to implement the same features, providing even more functionality. Removing sysfs is as hard as removing ioctl or procfs support on kernel. You may change their internal implementation, but not the userspace API. Btw, if we'll seek for the last internal changes, among those three API's, the more recent internal changes were at fs API where ioctl support is. There, the Kernel big logs were removed. This required a review on all driver locks and changes on almost all v4l/dvb drivers. Also, wanting or not, sysfs is called on every kernel driver, so this dependency already exists. Cheers, Mauro -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
2009. 9. 12., 오전 7:21, Hans Verkuil 작성: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. What is sysfs? (taken from http://lwn.net/Articles/31185/) Sysfs is a virtual filesystem which provides a userspace-visible representation of the device model. The device model and sysfs are sometimes confused with each other, but they are distinct entities. The device model functions just fine without sysfs (but the reverse is not true). Currently both a v4l driver and the device nodes are all represented in sysfs. This is handled automatically by the kernel. Sub-devices are not represented in sysfs since they are not based on struct device. They are v4l-internal structures. Actually, if the subdev represents an i2c device, then that i2c device will be present in sysfs, but not all subdevs are i2c devices. Should we make all sub-devices based on struct device? Currently this is not required. Doing this would probably mean registering a virtual bus, then attaching the sub-device to that. Of course, this only applies to sub-devices that represent something that is not an i2c device (e.g. something internal to the media board like a resizer, or something connected to GPIO pins). If we decide to go with sysfs, then we have to do this. This part shouldn't be too difficult to implement. And also if we do not go with sysfs this might be interesting to do eventually. The media controller topology as I see it should contain the device nodes since the application has to know what device node to open to do the streaming. It should also contain the sub-devices so the application can control them. Is this enough? I think that eventually we also want to show the physical connectors. I left them out (mostly) from the initial media controller proposal, but I suspect that we want those as well eventually. But connectors are definitely not devices. In that respect the entity concept of the media controller is more abstract than sysfs. However, for now I think we can safely assume that sub-devices can be made visible in sysfs. Hans, First of all I'm very sorry that I had not enough time to go through your new RFC. I'll checkout right after posting this mail. I think this is a good approach and I also had in my mind that sysfs might be a good method if we could control and monitor through this. Recalling memory when we had a talk in San Francisco, I was frustrated that there is no way to catch events from sort of sub-devices like lens actuator (I mean pizeo motors in camera module). As you know lens actuator is an extremely slow device in comparison with common v4l2 devices we are using and we need to know whether it has succeeded or not in moving to expected position. So I considered sysfs and udev as candidates for catching events from sub-devices. events like success/failure of lens movement, change of status of subdevices. Does anybody experiencing same issue? I think I've seen a lens controller driver in omap3 kernel from TI but not sure how did they control that. My point is that we need a kind of framework to give and event to user space and catching them properly just like udev does. Cheers, Nate = DongSoo, Nathaniel Kim Engineer Mobile S/W Platform Lab. Digital Media Communications RD Centre Samsung Electronics CO., LTD. e-mail : dongsoo@gmail.com dongsoo45@samsung.com-- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
On Sunday 13 September 2009 08:13:04 Nathaniel Kim wrote: 2009. 9. 12., 오전 7:21, Hans Verkuil 작성: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. What is sysfs? (taken from http://lwn.net/Articles/31185/) Sysfs is a virtual filesystem which provides a userspace-visible representation of the device model. The device model and sysfs are sometimes confused with each other, but they are distinct entities. The device model functions just fine without sysfs (but the reverse is not true). Currently both a v4l driver and the device nodes are all represented in sysfs. This is handled automatically by the kernel. Sub-devices are not represented in sysfs since they are not based on struct device. They are v4l-internal structures. Actually, if the subdev represents an i2c device, then that i2c device will be present in sysfs, but not all subdevs are i2c devices. Should we make all sub-devices based on struct device? Currently this is not required. Doing this would probably mean registering a virtual bus, then attaching the sub-device to that. Of course, this only applies to sub-devices that represent something that is not an i2c device (e.g. something internal to the media board like a resizer, or something connected to GPIO pins). If we decide to go with sysfs, then we have to do this. This part shouldn't be too difficult to implement. And also if we do not go with sysfs this might be interesting to do eventually. The media controller topology as I see it should contain the device nodes since the application has to know what device node to open to do the streaming. It should also contain the sub-devices so the application can control them. Is this enough? I think that eventually we also want to show the physical connectors. I left them out (mostly) from the initial media controller proposal, but I suspect that we want those as well eventually. But connectors are definitely not devices. In that respect the entity concept of the media controller is more abstract than sysfs. However, for now I think we can safely assume that sub-devices can be made visible in sysfs. Hans, First of all I'm very sorry that I had not enough time to go through your new RFC. I'll checkout right after posting this mail. I think this is a good approach and I also had in my mind that sysfs might be a good method if we could control and monitor through this. Recalling memory when we had a talk in San Francisco, I was frustrated that there is no way to catch events from sort of sub-devices like lens actuator (I mean pizeo motors in camera module). As you know lens actuator is an extremely slow device in comparison with common v4l2 devices we are using and we need to know whether it has succeeded or not in moving to expected position. So I considered sysfs and udev as candidates for catching events from sub-devices. events like success/failure of lens movement, change of status of subdevices. Does anybody experiencing same issue? I think I've seen a lens controller driver in omap3 kernel from TI but not sure how did they control that. My point is that we need a kind of framework to give and event to user space and catching them properly just like udev does. When I was talking to Laurent Pinchart and Sakari and his team at Nokia we discussed just such a framework. It actually exists already, although it is poorly implemented. Look at include/linux/dvb/video.h, struct video_event and ioctl VIDEO_GET_EVENT. It is used in ivtv (ivtv-ioctl.c, look for VIDEO_GET_EVENT). The idea is that you can either call VIDEO_GET_EVENT to wait for an event or use select() and wait for an exception to arrive, and then call VIDEO_GET_EVENT to find which event it was. This is ideal for streaming-related events. In ivtv it is used to report VSYNCs and to report when the MPEG decoder stopped (there is a delay between stopping sending new data to the decoder and when it actually processed all its internal buffers). Laurent is going to look into this to clean it up and present it as a new proper official V4L2 event mechanism. For events completely specific to a subdev I wonder whether it wouldn't be a good idea to use the media controller device for that. I like the select() mechanism since in an application you can just select() on a whole bunch of filehandles. If you can't use select() then you are forced to do awkward coding (e.g. make a separate thread just to handle that other event mechanism). So with the media controller we can easily let
Re: Media controller: sysfs vs ioctl
Em Sun, 13 Sep 2009 15:13:04 +0900 Nathaniel Kim dongsoo@gmail.com escreveu: 2009. 9. 12., 오전 7:21, Hans Verkuil 작성: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. What is sysfs? (taken from http://lwn.net/Articles/31185/) Sysfs is a virtual filesystem which provides a userspace-visible representation of the device model. The device model and sysfs are sometimes confused with each other, but they are distinct entities. The device model functions just fine without sysfs (but the reverse is not true). Currently both a v4l driver and the device nodes are all represented in sysfs. This is handled automatically by the kernel. Sub-devices are not represented in sysfs since they are not based on struct device. They are v4l-internal structures. Actually, if the subdev represents an i2c device, then that i2c device will be present in sysfs, but not all subdevs are i2c devices. Should we make all sub-devices based on struct device? Currently this is not required. Doing this would probably mean registering a virtual bus, then attaching the sub-device to that. Of course, this only applies to sub-devices that represent something that is not an i2c device (e.g. something internal to the media board like a resizer, or something connected to GPIO pins). If we decide to go with sysfs, then we have to do this. This part shouldn't be too difficult to implement. And also if we do not go with sysfs this might be interesting to do eventually. The media controller topology as I see it should contain the device nodes since the application has to know what device node to open to do the streaming. It should also contain the sub-devices so the application can control them. Is this enough? I think that eventually we also want to show the physical connectors. I left them out (mostly) from the initial media controller proposal, but I suspect that we want those as well eventually. But connectors are definitely not devices. In that respect the entity concept of the media controller is more abstract than sysfs. However, for now I think we can safely assume that sub-devices can be made visible in sysfs. Hans, First of all I'm very sorry that I had not enough time to go through your new RFC. I'll checkout right after posting this mail. I think this is a good approach and I also had in my mind that sysfs might be a good method if we could control and monitor through this. Recalling memory when we had a talk in San Francisco, I was frustrated that there is no way to catch events from sort of sub-devices like lens actuator (I mean pizeo motors in camera module). As you know lens actuator is an extremely slow device in comparison with common v4l2 devices we are using and we need to know whether it has succeeded or not in moving to expected position. So I considered sysfs and udev as candidates for catching events from sub-devices. events like success/failure of lens movement, change of status of subdevices. Does anybody experiencing same issue? I think I've seen a lens controller driver in omap3 kernel from TI but not sure how did they control that. My point is that we need a kind of framework to give and event to user space and catching them properly just like udev does. Maybe the Kernel event interface could be used for that. Cheers, Nate = DongSoo, Nathaniel Kim Engineer Mobile S/W Platform Lab. Digital Media Communications RD Centre Samsung Electronics CO., LTD. e-mail : dongsoo@gmail.com dongsoo45@samsung.com-- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Cheers, Mauro -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
On Sunday 13 September 2009 15:27:57 Mauro Carvalho Chehab wrote: Em Sun, 13 Sep 2009 15:13:04 +0900 Nathaniel Kim dongsoo@gmail.com escreveu: 2009. 9. 12., 오전 7:21, Hans Verkuil 작성: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. What is sysfs? (taken from http://lwn.net/Articles/31185/) Sysfs is a virtual filesystem which provides a userspace-visible representation of the device model. The device model and sysfs are sometimes confused with each other, but they are distinct entities. The device model functions just fine without sysfs (but the reverse is not true). Currently both a v4l driver and the device nodes are all represented in sysfs. This is handled automatically by the kernel. Sub-devices are not represented in sysfs since they are not based on struct device. They are v4l-internal structures. Actually, if the subdev represents an i2c device, then that i2c device will be present in sysfs, but not all subdevs are i2c devices. Should we make all sub-devices based on struct device? Currently this is not required. Doing this would probably mean registering a virtual bus, then attaching the sub-device to that. Of course, this only applies to sub-devices that represent something that is not an i2c device (e.g. something internal to the media board like a resizer, or something connected to GPIO pins). If we decide to go with sysfs, then we have to do this. This part shouldn't be too difficult to implement. And also if we do not go with sysfs this might be interesting to do eventually. The media controller topology as I see it should contain the device nodes since the application has to know what device node to open to do the streaming. It should also contain the sub-devices so the application can control them. Is this enough? I think that eventually we also want to show the physical connectors. I left them out (mostly) from the initial media controller proposal, but I suspect that we want those as well eventually. But connectors are definitely not devices. In that respect the entity concept of the media controller is more abstract than sysfs. However, for now I think we can safely assume that sub-devices can be made visible in sysfs. Hans, First of all I'm very sorry that I had not enough time to go through your new RFC. I'll checkout right after posting this mail. I think this is a good approach and I also had in my mind that sysfs might be a good method if we could control and monitor through this. Recalling memory when we had a talk in San Francisco, I was frustrated that there is no way to catch events from sort of sub-devices like lens actuator (I mean pizeo motors in camera module). As you know lens actuator is an extremely slow device in comparison with common v4l2 devices we are using and we need to know whether it has succeeded or not in moving to expected position. So I considered sysfs and udev as candidates for catching events from sub-devices. events like success/failure of lens movement, change of status of subdevices. Does anybody experiencing same issue? I think I've seen a lens controller driver in omap3 kernel from TI but not sure how did they control that. My point is that we need a kind of framework to give and event to user space and catching them properly just like udev does. Maybe the Kernel event interface could be used for that. Are you talking about the input event interface? There is no standard kernel way of doing events afaik. Regards, Hans -- Hans Verkuil - video4linux developer - sponsored by TANDBERG Telecom -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
Em Sun, 13 Sep 2009 15:43:02 +0200 Hans Verkuil hverk...@xs4all.nl escreveu: On Sunday 13 September 2009 15:27:57 Mauro Carvalho Chehab wrote: Em Sun, 13 Sep 2009 15:13:04 +0900 Nathaniel Kim dongsoo@gmail.com escreveu: 2009. 9. 12., 오전 7:21, Hans Verkuil 작성: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. What is sysfs? (taken from http://lwn.net/Articles/31185/) Sysfs is a virtual filesystem which provides a userspace-visible representation of the device model. The device model and sysfs are sometimes confused with each other, but they are distinct entities. The device model functions just fine without sysfs (but the reverse is not true). Currently both a v4l driver and the device nodes are all represented in sysfs. This is handled automatically by the kernel. Sub-devices are not represented in sysfs since they are not based on struct device. They are v4l-internal structures. Actually, if the subdev represents an i2c device, then that i2c device will be present in sysfs, but not all subdevs are i2c devices. Should we make all sub-devices based on struct device? Currently this is not required. Doing this would probably mean registering a virtual bus, then attaching the sub-device to that. Of course, this only applies to sub-devices that represent something that is not an i2c device (e.g. something internal to the media board like a resizer, or something connected to GPIO pins). If we decide to go with sysfs, then we have to do this. This part shouldn't be too difficult to implement. And also if we do not go with sysfs this might be interesting to do eventually. The media controller topology as I see it should contain the device nodes since the application has to know what device node to open to do the streaming. It should also contain the sub-devices so the application can control them. Is this enough? I think that eventually we also want to show the physical connectors. I left them out (mostly) from the initial media controller proposal, but I suspect that we want those as well eventually. But connectors are definitely not devices. In that respect the entity concept of the media controller is more abstract than sysfs. However, for now I think we can safely assume that sub-devices can be made visible in sysfs. Hans, First of all I'm very sorry that I had not enough time to go through your new RFC. I'll checkout right after posting this mail. I think this is a good approach and I also had in my mind that sysfs might be a good method if we could control and monitor through this. Recalling memory when we had a talk in San Francisco, I was frustrated that there is no way to catch events from sort of sub-devices like lens actuator (I mean pizeo motors in camera module). As you know lens actuator is an extremely slow device in comparison with common v4l2 devices we are using and we need to know whether it has succeeded or not in moving to expected position. So I considered sysfs and udev as candidates for catching events from sub-devices. events like success/failure of lens movement, change of status of subdevices. Does anybody experiencing same issue? I think I've seen a lens controller driver in omap3 kernel from TI but not sure how did they control that. My point is that we need a kind of framework to give and event to user space and catching them properly just like udev does. Maybe the Kernel event interface could be used for that. Are you talking about the input event interface? There is no standard kernel way of doing events afaik. Yes. It is designed for low-latency report of events, like mouse movements, where you expect that the movement will happen as mouse moves. So, it may work fine also for servo movements. A closer look on it, plus some tests should be done to see if it will work fine for such camera events. Cheers, Mauro -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
Hans Verkuil schrieb: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. Is it really a good idea to create a dependency to some virtual file system which may go away in future? From time to time some of those seem to go away, for example devfs. Is it really unavoidable to have something in sysfs, something which is really not possible with ioctls? And do you really want to depend on sysfs developers? --Winfried -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
On Sunday 13 September 2009 17:54:11 wk wrote: Hans Verkuil schrieb: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. Is it really a good idea to create a dependency to some virtual file system which may go away in future? From time to time some of those seem to go away, for example devfs. Is it really unavoidable to have something in sysfs, something which is really not possible with ioctls? And do you really want to depend on sysfs developers? One other interesting question is: currently the V4L2 API is also used by BSD variants for their video drivers. Our V4L2 header is explicitly dual-licensed to allow this. I don't think that BSD has sysfs. So making the media controller sysfs-based only would make it very hard for them if they ever want to port drivers that rely on that to BSD. Yes, I know that strictly speaking we don't have to care about that, but it is yet another argument against the use of sysfs as far as I am concerned. Regards, Hans -- Hans Verkuil - video4linux developer - sponsored by TANDBERG Telecom -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
Em Sat, 12 Sep 2009 00:21:48 +0200 Hans Verkuil hverk...@xs4all.nl escreveu: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. True. Choosing the better approach is very important since, once merged, we'll need to stick it for a very long time. I saw your proposal of a ioctl-only implementation for the media control. It is important to have a sysfs implementation also to compare. I can do it. However, we are currently in the middle of a merge window, and this one will require even more time than usual, since we have 2 series of patches for soc_camera and for DaVinci/OMAP that depends on arm and omap architecture merge. Also, there are some pending merges that requires some time to analyze, like the ISDB-T/ISDB-S patches and API changes that were proposed for 2.6.32, that requiring the analysis of both Japanese and Brazilian specs and do some tests, and the tuner changes for better handling the i2c gates, and the V4L and DVB specs that we can now merge upstream, as both got converted to DocBook XML 4.1.2 (the same version used upstream). So, during the next two weeks, we'll have enough fun to handle, in order to get our patches merged for 2.6.32. So, unfortunately, I'm afraid that we'll need to give a break on those discussions until the end of the merge window, focusing on merging the patches we have for 2.6.32. Cheers, Mauro -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
On Sat, Sep 12, 2009 at 9:31 AM, Mauro Carvalho Chehab mche...@infradead.org wrote: True. Choosing the better approach is very important since, once merged, we'll need to stick it for a very long time. I saw your proposal of a ioctl-only implementation for the media control. It is important to have a sysfs implementation also to compare. I can do it. However, we are currently in the middle of a merge window, and this one will require even more time than usual, since we have 2 series of patches for soc_camera and for DaVinci/OMAP that depends on arm and omap architecture merge. Also, there are some pending merges that requires some time to analyze, like the ISDB-T/ISDB-S patches and API changes that were proposed for 2.6.32, that requiring the analysis of both Japanese and Brazilian specs and do some tests, and the tuner changes for better handling the i2c gates, and the V4L and DVB specs that we can now merge upstream, as both got converted to DocBook XML 4.1.2 (the same version used upstream). So, during the next two weeks, we'll have enough fun to handle, in order to get our patches merged for 2.6.32. So, unfortunately, I'm afraid that we'll need to give a break on those discussions until the end of the merge window, focusing on merging the patches we have for 2.6.32. Cheers, Mauro Mauro, I respectfully disagree. The original version of this RFC has been pending for almost a year now. Hans has written a prototype implementation. We should strive to get this locked down by the LPC conference. I think we all know that you are busy, but this conversation needs to continue even if you personally do not have the cycles to give it your full attention. There is finally some real momentum behind this initiative, and the lack of this functionality is crippling usability for many, many users. Hi I a new user to tvtime. I can see analog tv with tvtime, but how do I make audio work? Let's finally put this issue to rest. Devin -- Devin J. Heitmueller - Kernel Labs http://www.kernellabs.com -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
Em Sat, 12 Sep 2009 09:41:58 -0400 Devin Heitmueller dheitmuel...@kernellabs.com escreveu: I respectfully disagree. Are you suggesting that we should not submit any patches upstream during this merge window in order to discuss this? Sorry, but this is not right with all the developers that did their homework and submitted changes for 2.6.32. The original version of this RFC has been pending for almost a year now. It was not pending since then. As there were all the framework changes needed, we've agreed on doing the V4L framework changes, that were finally merged at 2.6.30[1], that were required, before proceeding with further discussions. Hans has written a prototype implementation. We should strive to get this locked down by the LPC conference. Why? Nothing stops discussing it there and better prepare a proposal, but, considering all the noise we had after the DVB S2API last year, I don't think we should ever repeat using a conference, where only some of us will be there to approve a proposal. It is the right forum for discussing and better formulate the issues, in order to prepare a proposal, but the decisions should happen at the ML. Hans took a year to prepare RFCv2, and we all know that he was hardly working on implementing what was discussed during the first RFC proposal during all those timeframe. This shows that this is a complex matter. I think we all know that you are busy, but this conversation needs to continue even if you personally do not have the cycles to give it your full attention. It is not only me that has pending tasks, but other developers are also focused on merging their things. For example, Mkrufky already pointed that he is waiting for the merge of the first part of his series, since he needs to send a complementary set of patches. I'm sure that there are other developers that are still finishing some working for the merge or that may need to solve the usual troubles that happens when some patches went upstream via other trees, needing to be backported and tested. There is finally some real momentum behind this initiative, and the lack of this functionality is crippling usability for many, many users. Hi I a new user to tvtime. I can see analog tv with tvtime, but how do I make audio work? Let's finally put this issue to rest. Yes, let's do it for 2.6.33, but it this discussion started too late for 2.6.32. Cheers, Mauro [1] FYI, we had several regression fixes on 2.6.31 due to that - and there are still some unsolved regressions related to them - basically i2c gate and radio breakages yet requiring fixes. Some are documented at Kernel regression list and at bugzilla, others were just reported at ML. As I said a countless number of times, we need to focus fixing the regressions caused by those API changes before doing another series of changes at API. So, if people are with spare time, I suggest a task force to fix the remaining regressions mostly on saa7134. -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
Em Sat, 12 Sep 2009 17:12:35 +0200 Hans Verkuil hverk...@xs4all.nl escreveu: Why? Nothing stops discussing it there and better prepare a proposal, but, considering all the noise we had after the DVB S2API last year, I don't think we should ever repeat using a conference, where only some of us will be there to approve a proposal. It is the right forum for discussing and better formulate the issues, in order to prepare a proposal, but the decisions should happen at the ML. In that particular case you would have had a lot of noise no matter what you did :-) True. The final decisions will indeed be taken here, but the conference is an excellent place to talk face-to-face and to see how well the current proposal will fit actual media hardware. I don't have complete access or knowledge of all the current and upcoming media boards, but there are several TI and Nokia engineers present who can help with that. Also interesting is to see how and if the framework can be extended to dvb. As I've said at #irc, it would be really great if you could find some ways for remote people to participate, maybe setting an audio conference channel. I think we should try to discuss with LPC runners if we can find some ways for it. I would love to discuss this there with you, but this year's budget and logistics didn't allow. For those that will also be in Japan for JLC, we can compliment there with some useful face-to-face discussions. Hans took a year to prepare RFCv2, and we all know that he was hardly working on implementing what was discussed during the first RFC proposal during all those timeframe. This shows that this is a complex matter. Not entirely true, I worked on the necessary building blocks for such a media controller in the past year. There is a reason why it only took me 400-odd lines to get the basic mc support in... Yes, I know. having the drivers using the framework is for sure the first step. Yet, unfortunately, this means that we'll still need to do lots of work with the webcam and dvb drivers for them to use the i2c kernel support and the proper media core. As some webcams has audio input streaming over USB, it is important to extend media controller features also to them. Yes, let's do it for 2.6.33, but it this discussion started too late for 2.6.32. I don't think anyone advocated getting anything merged for 2.6.32. Certainly not me. I'm not even sure whether 2.6.33 is feasible: before we merge anything I'd really like to see it implemented in e.g. omap3, uvcvideo and ivtv at the least. The proof of the pudding is in the eating, and since this is meant to cover a wide range of media boards we should have some idea if theory and practice actually match. Personally I think that the fact that I got an initial version implemented so quickly is very promising. I'm currently trying to get ivtv media-controller-aware. It's probably the most complex driver when it comes to topology that I have access to, so that would be a good test case. The most complex hardware for PC I'm aware is the cx25821. Unfortunately, the driver is currently in bad shape, in terms of CodingStyle (including the removal of large blocks of code that are repeated several times along the driver), needing lots of changes in order to get merged. For those interested, the code is at: http://linuxtv.org/hg/~mchehab/cx25821/ I'll likely do some work on it during this merge window for its inclusion upstream (probably at drivers/staging - since I doubt we'll have enough time to clean it up right now). It has several blocks that can be used for video in and video out. The current driver has support for 8 simultaneous video inputs and 4 simultaneous video output. I'm not sure, but I won't doubt that you can exchange inputs and outputs or even group them. So, this is a good candidate for some media controller tests. I'll try to do it via sysfs, running some tests and post the results. Cheers, Mauro -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
On Sat, 2009-09-12 at 12:54 -0300, Mauro Carvalho Chehab wrote: Em Sat, 12 Sep 2009 17:12:35 +0200 Hans Verkuil hverk...@xs4all.nl escreveu: I'm currently trying to get ivtv media-controller-aware. It's probably the most complex driver when it comes to topology that I have access to, so that would be a good test case. The most complex hardware for PC I'm aware is the cx25821. Unfortunately, the driver is currently in bad shape, in terms of CodingStyle (including the removal of large blocks of code that are repeated several times along the driver), needing lots of changes in order to get merged. For those interested, the code is at: http://linuxtv.org/hg/~mchehab/cx25821/ I'll likely do some work on it during this merge window for its inclusion upstream (probably at drivers/staging - since I doubt we'll have enough time to clean it up right now). It has several blocks that can be used for video in and video out. The current driver has support for 8 simultaneous video inputs and 4 simultaneous video output. I'm not sure, but I won't doubt that you can exchange inputs and outputs or even group them. So, this is a good candidate for some media controller tests. I'll try to do it via sysfs, running some tests and post the results. I read the available specs for that chip when I saw the source code appear in a repo of yours several months ago. The public data sheet is here. http://www.conexant.com/servlets/DownloadServlet/PBR-201499-004.pdf?docid=1501revid=4 The chip looks like it is a good fit for surveillance applications. The block diagram indicates it is essentially a Video (10x CCIR656) and Audio (5x I2S) router, with a pile of GPIOS (48), 3 I2C busses, and support for inbound and outbound DMA channels. The chip also has built in scalers and motion detection. Managing the chip itself doesn't look too complicated, but once intergrated with other devices like compression CODECs, a CX25853 devices, or general purpose microcontrollers, I imagine it could get complex to manage. The reference design brief is here: http://www.conexant.com/servlets/DownloadServlet/RED-202183-001.pdf?docid=2184revid=1 I agree with the coding style problems of the current driver. Regards, Andy Cheers, Mauro -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
Em Sat, 12 Sep 2009 14:48:23 -0400 Andy Walls awa...@radix.net escreveu: On Sat, 2009-09-12 at 12:54 -0300, Mauro Carvalho Chehab wrote: Em Sat, 12 Sep 2009 17:12:35 +0200 Hans Verkuil hverk...@xs4all.nl escreveu: I'm currently trying to get ivtv media-controller-aware. It's probably the most complex driver when it comes to topology that I have access to, so that would be a good test case. The most complex hardware for PC I'm aware is the cx25821. Unfortunately, the driver is currently in bad shape, in terms of CodingStyle (including the removal of large blocks of code that are repeated several times along the driver), needing lots of changes in order to get merged. For those interested, the code is at: http://linuxtv.org/hg/~mchehab/cx25821/ I'll likely do some work on it during this merge window for its inclusion upstream (probably at drivers/staging - since I doubt we'll have enough time to clean it up right now). It has several blocks that can be used for video in and video out. The current driver has support for 8 simultaneous video inputs and 4 simultaneous video output. I'm not sure, but I won't doubt that you can exchange inputs and outputs or even group them. So, this is a good candidate for some media controller tests. I'll try to do it via sysfs, running some tests and post the results. I read the available specs for that chip when I saw the source code appear in a repo of yours several months ago. The public data sheet is here. http://www.conexant.com/servlets/DownloadServlet/PBR-201499-004.pdf?docid=1501revid=4 The chip looks like it is a good fit for surveillance applications. The block diagram indicates it is essentially a Video (10x CCIR656) and Audio (5x I2S) router, with a pile of GPIOS (48), 3 I2C busses, and support for inbound and outbound DMA channels. The chip also has built in scalers and motion detection. Managing the chip itself doesn't look too complicated, but once intergrated with other devices like compression CODECs, a CX25853 devices, or general purpose microcontrollers, I imagine it could get complex to manage. The reference design brief is here: http://www.conexant.com/servlets/DownloadServlet/RED-202183-001.pdf?docid=2184revid=1 Yes, this board is very powerful, and very complex designs are possible with it. I agree with the coding style problems of the current driver. Yes. It will take some time until cleaning it up and letting it done for being at drivers/media. Cheers, Mauro -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Media controller: sysfs vs ioctl
Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. What is sysfs? (taken from http://lwn.net/Articles/31185/) Sysfs is a virtual filesystem which provides a userspace-visible representation of the device model. The device model and sysfs are sometimes confused with each other, but they are distinct entities. The device model functions just fine without sysfs (but the reverse is not true). Currently both a v4l driver and the device nodes are all represented in sysfs. This is handled automatically by the kernel. Sub-devices are not represented in sysfs since they are not based on struct device. They are v4l-internal structures. Actually, if the subdev represents an i2c device, then that i2c device will be present in sysfs, but not all subdevs are i2c devices. Should we make all sub-devices based on struct device? Currently this is not required. Doing this would probably mean registering a virtual bus, then attaching the sub-device to that. Of course, this only applies to sub-devices that represent something that is not an i2c device (e.g. something internal to the media board like a resizer, or something connected to GPIO pins). If we decide to go with sysfs, then we have to do this. This part shouldn't be too difficult to implement. And also if we do not go with sysfs this might be interesting to do eventually. The media controller topology as I see it should contain the device nodes since the application has to know what device node to open to do the streaming. It should also contain the sub-devices so the application can control them. Is this enough? I think that eventually we also want to show the physical connectors. I left them out (mostly) from the initial media controller proposal, but I suspect that we want those as well eventually. But connectors are definitely not devices. In that respect the entity concept of the media controller is more abstract than sysfs. However, for now I think we can safely assume that sub-devices can be made visible in sysfs. The next point is how to access sub-devices. One approach is to create sub-device attributes that can be read and written. The other approach is to use a media controller device node and use ioctls. A third approach would be to create device nodes for each sub-device. This was the original idea I had when they were still called 'media processors'. I no longer think that is a good idea. The problem with that is that it will create even more device nodes (dozens in some cases) in addition to the many streaming device nodes that we already have. We have no choice in the case of streaming, but very few sub-devices (if any) will need to do read/write or streaming. Should this be needed, then it is always possible to create a device node just for that and link it as an output to the sub-device. In 99% of the cases we just need to be able to call ioctl on the sub-device. And we can do that through a media controller device just as well. What sort of interaction do we need with sub-devices? 1) Controls. Most sub-devices will have controls. Some of these controls are the same as are exposed through a v4l device node (brightness, contrast, etc), but others are more advanced controls. E.g. fine-grained gain controls. We have seen this lately in several driver submissions that there are a lot of controls that are hardware specific and that we do not want to expose to the average xawtv-type application. But advanced apps still want to be able to use them. Making such controls private to the sub-device is a good solution. What that means is that they do not appear when you do QUERYCTRL on the video device node, but you can set/get them when dealing directly to the sub-device. If we do this with an ioctl then we can just reuse the existing ioctls for dealing with controls. As you know I am working on a control framework that will move most of the implementation details to the core. It is easy to make this available as well to sub-devices. So calling QUERYCTRL through a media controller will also enumerate all the 'private' controls. In addition, with VIDIOC_S/G_EXT_CTRLS you can set and get multiple controls atomically. Such a control framework can also expose those controls as sysfs attributes. This works fine except for setting multiple controls atomically. E.g. for motor control you will definitely want this. You can do this I guess by implementing an 'all' attribute that will output all the control values when queried and you can write things like: motor_x=100 motor_y=200 to the 'all' attribute. Yuck. Oh, and I don't see how you can implement the information that v4l2_queryctrl and v4l2_querymenu can return in sysfs. Except by creating even more attributes. Note that I would
Re: Media controller: sysfs vs ioctl
On Sat, 2009-09-12 at 00:21 +0200, Hans Verkuil wrote: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. I've wanted to reply earlier but I cannot collect enough time to do proper research, but since you asked this particular question, I happen to have something which may help. I would suggest evaluating a representative proposals by applying Baldwin and Clark's Net Options Value (NOV) metric to a simple system representation. The system to which to apply the metric would comprise: - a representative user space app - a representative v4l-dvb driver - an API concept/suggestion/proposal I think this metric is appropriate to apply, because the NOV is a way to assign value to implementing options (i.e. options in modular systems). An API itself is not a modular system and hard to evaluate in isolation, so it needs to be evaluated in the context of the options it provies to the system designers and maintainers. The NOV boils to simple concepts: 1. a system design has a total value that is its present value plus the value of it's options that can be exploited in the future. 2. an option represents a potential value that may provide a return in the future 3. an option has only a potential value (in the present) 4. an option only yields a return if that option may be exploited in the future. The probability that the option may be exploited needs to be taken into account. 5. an option has costs associated with exploiting it (in the future) I'm not advocating a rigorous computation of the metric for the proposals, but more a qualitative look at the proposals but still using the precise definition of the metric (sorry I don't have a URL handy...). I will note that I think am in agreement with Hans on sysfs. I think the cost of trying to exploit any option provided through sysfs in a userspace apppllication will nullify any technical benefit of said option to the application. Lets say we want to convert an existing app to a Media Controller aware version of that app. There is a cost to do that. Will the API proposal make exploting some options have a large cost? Do some of the options of the API have a low probability of being exploited? Do some of the options of the API provide very low technical benefit? What does the API proposal do to the total value of the system (e.g. an API with no flexibility fixes the total value close to the present value and there is no value to be realized from exploiting options in the future). OK, I hope I've communicated what I mean. I feel like that all may be less than clear. These ideas have come from a confluence of research I've been doing at work, and V4L-DVB work (thinking about Multiproto vs. DVB v5, and the v4l2_subdev IR ops, etc.). Regards, Andy -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Media controller: sysfs vs ioctl
Hi, Am Freitag, den 11.09.2009, 19:01 -0400 schrieb Andy Walls: On Sat, 2009-09-12 at 00:21 +0200, Hans Verkuil wrote: Hi all, I've started this as a new thread to prevent polluting the discussions of the media controller as a concept. First of all, I have no doubt that everything that you can do with an ioctl, you can also do with sysfs and vice versa. That's not the problem here. The problem is deciding which approach is the best. I've wanted to reply earlier but I cannot collect enough time to do proper research, but since you asked this particular question, I happen to have something which may help. I would suggest evaluating a representative proposals by applying Baldwin and Clark's Net Options Value (NOV) metric to a simple system representation. The system to which to apply the metric would comprise: - a representative user space app - a representative v4l-dvb driver - an API concept/suggestion/proposal I think this metric is appropriate to apply, because the NOV is a way to assign value to implementing options (i.e. options in modular systems). An API itself is not a modular system and hard to evaluate in isolation, so it needs to be evaluated in the context of the options it provies to the system designers and maintainers. The NOV boils to simple concepts: 1. a system design has a total value that is its present value plus the value of it's options that can be exploited in the future. 2. an option represents a potential value that may provide a return in the future 3. an option has only a potential value (in the present) 4. an option only yields a return if that option may be exploited in the future. The probability that the option may be exploited needs to be taken into account. 5. an option has costs associated with exploiting it (in the future) I'm not advocating a rigorous computation of the metric for the proposals, but more a qualitative look at the proposals but still using the precise definition of the metric (sorry I don't have a URL handy...). I will note that I think am in agreement with Hans on sysfs. I think the cost of trying to exploit any option provided through sysfs in a userspace apppllication will nullify any technical benefit of said option to the application. Lets say we want to convert an existing app to a Media Controller aware version of that app. There is a cost to do that. Will the API proposal make exploting some options have a large cost? Do some of the options of the API have a low probability of being exploited? Do some of the options of the API provide very low technical benefit? What does the API proposal do to the total value of the system (e.g. an API with no flexibility fixes the total value close to the present value and there is no value to be realized from exploiting options in the future). OK, I hope I've communicated what I mean. I feel like that all may be less than clear. These ideas have come from a confluence of research I've been doing at work, and V4L-DVB work (thinking about Multiproto vs. DVB v5, and the v4l2_subdev IR ops, etc.). Regards, Andy just as a side note. We are not forced to accept any hardware design under all conditions anymore. If there are reasons, we can also tell them to go to Bill and Steve and to pay their fees per device. Cheers, Hermann -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
