Re: [Workshop-2011] Media Subsystem Workshop 2011
Hi Mauro, On Wed, Aug 03, 2011 at 02:21:05PM -0300, Mauro Carvalho Chehab wrote: As already announced, we're continuing the planning for this year's media subsystem workshop. To avoid overriding the main ML with workshop-specifics, a new ML was created: workshop-2...@linuxtv.org I'll also be updating the event page at: http://www.linuxtv.org/events.php Over the one-year period, we had 242 developers contributing to the subsystem. Thank you all for that! Unfortunately, the space there is limited, and we can't affort to have all developers there. Due to that some criteria needed to be applied to create a short list of people that were invited today to participate. The main criteria were to select the developers that did significant contributions for the media subsystem over the last 1 year period, measured in terms of number of commits and changed lines to the kernel drivers/media tree. As the used criteria were the number of kernel patches, userspace-only developers weren't included on the invitations. It would be great to have there open source application developers as well, in order to allow us to tune what's needed from applications point of view. So, if you're leading the development of some V4L and/or DVB open-source application and wants to be there, or you think you can give good contributions for helping to improve the subsystem, please feel free to send us an email. With regards to the themes, we're received, up to now, the following proposals: -+-- THEME| Proposed-by: -+-- Buffer management: snapshot mode | Guennadi Rotation in webcams in tablets while streaming is active | Hans de Goede V4L2 Spec ??? ambiguities fix | Hans Verkuil V4L2 compliance test results | Hans Verkuil Media Controller presentation (probably for Wed, 25) | Laurent Pinchart Workshop summary presentation on Wed, 25 | Mauro Carvalho Chehab -+-- From my side, I also have the following proposals: 1) DVB API consistency - what to do with the audio and video DVB API's that conflict with V4L2 and (somewhat) with ALSA? 2) Multi FE support - How should we handle a frontend with multiple delivery systems like DRX-K frontend? 3) videobuf2 - migration plans for legacy drivers 4) NEC IR decoding - how should we handle 32, 24, and 16 bit protocol variations? Even if you won't be there, please feel free to propose themes for discussion, in order to help us to improve even more the subsystem. Drawing from our recent discussions over e-mail, I would like to add another topic: the V4L2 on desktop vs. embedded systems. The V4L2 is being used as an application interface on desktop systems, but recently as support has been added to complex camera ISPs in embedded systems it is used for a different purpose: it's a much lower level interface for specialised user space which typically contains a middleware layer which provides its own application interface (e.g. GSTphotography). The V4L2 API in the two different kind of systems is exactly the same but its role is different: the hardware drivers are not up to offering an interface suitable for the use by general purpose applications. To run generic purpose applications on such embedded systems, I have promoted the use of libv4l (either plain or with plugins) to provide what is missing from between the V4L2, Media controller and v4l2_subdev interfaces provided by kernel drivers --- which mostly allow controlling the hardware --- and what the general purpose applications need. Much of the missing functionality is usually implemented in algorithm frameworks and libraries that do not fit to kernel space: they are complex and often the algorithms themselves are under very restrictive licenses. There is an upside: the libv4l does contain an automatic exposure and a white balance algorithm which are suitable for some use cases. Defining functionality suitable for general purpose applications at the level of V4L2 requires scores of policy decisions on embedded systems. One of the examples is the pipeline configuration for which the Media controller and v4l2_subdev interfaces are currently being used for. Applications such as Fcam URL:http://fcam.garage.maemo.org/ do need to make these policy decisions by themselves. For this reason, I consider it highly important that the low level hardware control interface is available to the user space applications. I think it is essential for the future support of such embedded devices in the mainline kernel to come to a common agreement on how this kind of systems should be
Re: [Workshop-2011] Media Subsystem Workshop 2011
Em 11-08-2011 07:16, Sakari Ailus escreveu: Hi Mauro, On Wed, Aug 03, 2011 at 02:21:05PM -0300, Mauro Carvalho Chehab wrote: As already announced, we're continuing the planning for this year's media subsystem workshop. To avoid overriding the main ML with workshop-specifics, a new ML was created: workshop-2...@linuxtv.org I'll also be updating the event page at: http://www.linuxtv.org/events.php Over the one-year period, we had 242 developers contributing to the subsystem. Thank you all for that! Unfortunately, the space there is limited, and we can't affort to have all developers there. Due to that some criteria needed to be applied to create a short list of people that were invited today to participate. The main criteria were to select the developers that did significant contributions for the media subsystem over the last 1 year period, measured in terms of number of commits and changed lines to the kernel drivers/media tree. As the used criteria were the number of kernel patches, userspace-only developers weren't included on the invitations. It would be great to have there open source application developers as well, in order to allow us to tune what's needed from applications point of view. So, if you're leading the development of some V4L and/or DVB open-source application and wants to be there, or you think you can give good contributions for helping to improve the subsystem, please feel free to send us an email. With regards to the themes, we're received, up to now, the following proposals: -+-- THEME| Proposed-by: -+-- Buffer management: snapshot mode | Guennadi Rotation in webcams in tablets while streaming is active | Hans de Goede V4L2 Spec ??? ambiguities fix | Hans Verkuil V4L2 compliance test results | Hans Verkuil Media Controller presentation (probably for Wed, 25) | Laurent Pinchart Workshop summary presentation on Wed, 25 | Mauro Carvalho Chehab -+-- From my side, I also have the following proposals: 1) DVB API consistency - what to do with the audio and video DVB API's that conflict with V4L2 and (somewhat) with ALSA? 2) Multi FE support - How should we handle a frontend with multiple delivery systems like DRX-K frontend? 3) videobuf2 - migration plans for legacy drivers 4) NEC IR decoding - how should we handle 32, 24, and 16 bit protocol variations? Even if you won't be there, please feel free to propose themes for discussion, in order to help us to improve even more the subsystem. Drawing from our recent discussions over e-mail, I would like to add another topic: the V4L2 on desktop vs. embedded systems. Topic added to: http://www.linuxtv.org/events.php The V4L2 is being used as an application interface on desktop systems, but recently as support has been added to complex camera ISPs in embedded systems it is used for a different purpose: it's a much lower level interface for specialised user space which typically contains a middleware layer which provides its own application interface (e.g. GSTphotography). The V4L2 API in the two different kind of systems is exactly the same but its role is different: the hardware drivers are not up to offering an interface suitable for the use by general purpose applications. To run generic purpose applications on such embedded systems, I have promoted the use of libv4l (either plain or with plugins) to provide what is missing from between the V4L2, Media controller and v4l2_subdev interfaces provided by kernel drivers --- which mostly allow controlling the hardware --- and what the general purpose applications need. Much of the missing functionality is usually implemented in algorithm frameworks and libraries that do not fit to kernel space: they are complex and often the algorithms themselves are under very restrictive licenses. There is an upside: the libv4l does contain an automatic exposure and a white balance algorithm which are suitable for some use cases. Defining functionality suitable for general purpose applications at the level of V4L2 requires scores of policy decisions on embedded systems. One of the examples is the pipeline configuration for which the Media controller and v4l2_subdev interfaces are currently being used for. Applications such as Fcam URL:http://fcam.garage.maemo.org/ do need to make these policy decisions by themselves. For this reason, I consider it highly important that the low level hardware control interface is available to the user space applications. I think it is essential for the future support of
Re: [Workshop-2011] Media Subsystem Workshop 2011
Hi, On 08/10/2011 02:34 AM, Theodore Kilgore wrote: snip but this is the way how the current discussion feels to me. If we agree on aiming for doing it right then with that comes to me doing a software design from scratch, so without taking into account what is already there. Here, a counter-argument is to point out, as I did in a mail earlier this afternoon, that without taking account what is already there might possibly let one overlook something important. And, no, I am not referring to the userspace-kernelspace problem with this. I am referring to the fact that simply to dump the entire contents of the camera into cache (and to keep it there for quite a while) might not necessarily be a good idea and it had been quite consciously rejected to do that in the design of libgphoto2. Not because it is in userspace, but because to do that eats up and ties up RAM of which one cannot assume there is a surplus. This is an implementation detail which has little to do with the fundamental choice of whether or not we want 2 separate drivers or 1 single driver. In part of the snipped message you called me impatient (no offense taken), my perceived impatience is stemming from what to me feels like we are dancing around the real issue here. The fundamental question is do we want 2 separate drivers or 1 single driver for these devices. Lets answer that first, using all we've learned from the past. But without taking into account that one choice or the other will involve re-doing lots of code, as to me that is a poor argument from a technical pov. snip There are of course limits to the from scratch part, in the end we want this to slot into the existing Linux practices for webcams and stillcams, which means: 1) offering a v4l2 /dev/video# node for streaming; and 2) access to the pictures stored on the camera through libgphoto Taking these 2 constrictions into account, and combining that with my firm believe that the solution to all the device sharing problems is handling both functions in a single driver, I end up with only 1 option: Have a kernel driver which provides both functions of the device, with the streaming exported as a standard v4l2 device, and the stillcam function exported with some to be defined API. Combined with a libgphoto2 portlib and camlib for this new API, so that existing libgphoto2 apps can still access the pictures as if nothing was changed. Well, what I _do_ think is that we need to agree about precisely what is supposed to work and what is not, in an operational sense. But we are still fuzzy about that. For example, you seemed to assert this morning that the webcam functionality needs to be able to preempt any running stillcam app and to grab the camera. Why? Or did I misunderstand you? You've misunderstood me. We need to distinguish between an application having a tie to the device (so having a fd open) and the application doing an actual operation on the device. No application should be able to pre-empt an ongoing operation by another application. Attempting an operation while another operation is ongoing should result in -EBUSY. This differs significantly from what we've currently where: 1) There is no distinguishing going on between an app having a tie and an app actually doing an operation. Only one app can have a fd open 2) Some apps (userspace apps) can pre-empt other apps, taking away their fd and cancelling any ongoing operations The above is what leads me to me still firm believe that having a single driver is the only solution. My reasoning is as follows 1) We cannot count on apps closing the fd when they have no immediate use for the device, iow open != in-use 2) Thus we need to allow both libgphoto2 and v4l2 apps to have the device open at the same time 3) When actual in-use (so an operation is ongoing) attempt by another apps to start an operation will result in -EBUSY 4) 2 + 3 can only be realized by having a single driver Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
Hi, On 08/08/2011 07:39 PM, Theodore Kilgore wrote: On Mon, 8 Aug 2011, Mauro Carvalho Chehab wrote: snip Mauro, In fact none of the currently known and supported cameras are using PTP. All of them are proprietary. They have a rather intimidating set of differences in functionality, too. Namely, some of them have an isochronous endpoint, and some of them rely exclusively upon bulk transport. Some of them have a well developed set of internal capabilities as far as handling still photos are concerned. I mean, such things as the ability to download a single photo, selected at random from the set of photos on the camera, and some do not, requiring that the ability to do this is emulated in software -- by first downloading all previously listed photos and sending the data to /dev/null, then downloading the desired photo and saving it. Some of them permit deletion of individual photos, or all photos, and some do not. For some of them it is even true, as I have previously mentioned, that the USB command string which will delete all photos is the same command used for starting the camera in streaming mode. But the point here is that these cameras are all different from one another, depending upon chipset and even, sometimes, upon firmware or chipset version. The still camera abilities and limitations of all of them are pretty much worked out in libgphoto2. My suggestion would be that the libgphoto2 support libraries for these cameras ought to be left the hell alone, except for some changes in, for example, how the camera is accessed in the first place (through libusb or through a kernel device) in order to address adequately the need to support both modes. I know what is in those libgphoto2 drivers because I wrote them. I can definitely promise that to move all of that functionality over into kernel modules would be a nightmare and would moreover greatly contribute to kernel bloat. You really don't want to go there. I strongly disagree with this. The libgphoto2 camlibs (drivers) for these cameras handle a number of different tasks: 1) Talking to the camera getting binary blobs out of them (be it a PAT or some data) 2) Interpreting said blobs 3) Converting the data parts to pictures doing post processing, etc. I'm not suggesting to move all of this to the kernel driver, we just need to move part 1. to the kernel driver. This is not rocket science. We currently have a really bad situation were drivers are fighting for the same device. The problem here is that these devices are not only one device on the physical level, but also one device on the logical level (IOW they have only 1 usb interface). It is time to quit thinking in band-aides and solve this properly, 1 logical device means it gets 1 driver. This may be an approach which means some more work then others, but I believe in the end that doing it right is worth the effort. As for Mauro's resource locking patches, these won't work because the assume both drivers are active at the same time, which is simply not true. Only 1 driver can be bound to the interface at a time, and when switching from the gspca driver to the usbfs driver, gspca will see an unplug which is indistinguishable from a real device unplug. More over a kernel only solution without libgphoto changes won't solve the problem of a libgphoto app keeping the device open locking out streaming. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
Hi, snip OK, another example. The cameras supported in camlibs/jl2005c do not have webcam ability, but someone could at any time design and market a dualmode which has in stillcam mode the same severe limitation. What limitation? Well, the entire memory of the camera must be dumped, or else the camera jams itself. You can stop dumping in the middle of the operation, but you must continue after that. Suppose that you had ten pictures on the camera and you only wanted to download the first one. Then you can do that and temporarily stop downloading the rest. But while exiting you have to check whether the rest are downloaded or not. And if they are not, then it has to be done, with the data simply thrown in the trash, and then the camera's memory pointer reset before the camera is released. How, one might ask, did anyone produce something so primitive? Well, it is done. Perhaps the money saved thereby was at least in part devoted to producing better optics for the camera. At least, one can hope so. But people did produce those cameras, and people have bought them. But does anyone want to reproduce the code to support this kind of crap in the kernel? And go through all of the hoops required in order to fake the behavior which one woulld expect from a real still camera? It has already been done in camlibs/jl2005c and isn't that enough? This actually is an example where doing a kernel driver would be easier, a kernel driver never exits. So it can simply remember where it was reading (and cache the data it has read sofar). If an app requests picture 10, we read 1-10, cache them and return picture 10 to the app, then the same or another app asks for picture 4, get it from cache, asks for picture 20 read 11-20, etc. Having written code for various small digital picture frames (the keychain models) I know where you are coming from. Trust me I do. Recently I had an interesting bug report, with a corrupt PAT (picture allocation table) turns out that when deleting a picture through the menu inside the frame a different marker gets written to the PAT then when deleting it with the windows software, Fun huh? So yeah duplicating this code is no fun, but it is the only realistic solution which will get us a 100% reliable and robust user experience. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Tue, 9 Aug 2011, Hans de Goede wrote: Hi, On 08/08/2011 07:39 PM, Theodore Kilgore wrote: On Mon, 8 Aug 2011, Mauro Carvalho Chehab wrote: snip Mauro, In fact none of the currently known and supported cameras are using PTP. All of them are proprietary. They have a rather intimidating set of differences in functionality, too. Namely, some of them have an isochronous endpoint, and some of them rely exclusively upon bulk transport. Some of them have a well developed set of internal capabilities as far as handling still photos are concerned. I mean, such things as the ability to download a single photo, selected at random from the set of photos on the camera, and some do not, requiring that the ability to do this is emulated in software -- by first downloading all previously listed photos and sending the data to /dev/null, then downloading the desired photo and saving it. Some of them permit deletion of individual photos, or all photos, and some do not. For some of them it is even true, as I have previously mentioned, that the USB command string which will delete all photos is the same command used for starting the camera in streaming mode. But the point here is that these cameras are all different from one another, depending upon chipset and even, sometimes, upon firmware or chipset version. The still camera abilities and limitations of all of them are pretty much worked out in libgphoto2. My suggestion would be that the libgphoto2 support libraries for these cameras ought to be left the hell alone, except for some changes in, for example, how the camera is accessed in the first place (through libusb or through a kernel device) in order to address adequately the need to support both modes. I know what is in those libgphoto2 drivers because I wrote them. I can definitely promise that to move all of that functionality over into kernel modules would be a nightmare and would moreover greatly contribute to kernel bloat. You really don't want to go there. I strongly disagree with this. The libgphoto2 camlibs (drivers) for these cameras handle a number of different tasks: 1) Talking to the camera getting binary blobs out of them (be it a PAT or some data) 2) Interpreting said blobs 3) Converting the data parts to pictures doing post processing, etc. I'm not suggesting to move all of this to the kernel driver, we just need to move part 1. to the kernel driver. I did not assume otherwise. This is not rocket science. No, but both Adam and I realized, approximately at the same time yesterday afternoon, something which is rather important here. Gphoto is not developed exclusively for Linux. Furthermore, it has a significant user base both on Windows and on MacOS, not to mention BSD. It really isn't nice to be screwing around too much with the way it works. We currently have a really bad situation were drivers are fighting for the same device. The problem here is that these devices are not only one device on the physical level, but also one device on the logical level (IOW they have only 1 usb interface). All true. Which is why I brought the topic up for discussion in the first place and why it now gets on the program of the USB Summit. It is time to quit thinking in band-aides and solve this properly, 1 logical device means it gets 1 driver. This may be an approach which means some more work then others, but I believe in the end that doing it right is worth the effort. Clearly, we agree about doing it right is worth the effort. The whole discussion right now is about what is right. As for Mauro's resource locking patches, these won't work because the assume both drivers are active at the same time, which is simply not true. Only 1 driver can be bound to the interface at a time, and when switching from the gspca driver to the usbfs driver, gspca will see an unplug which is indistinguishable from a real device unplug. Things would not have to happen so, of course. Things did not used to happen so. Presence of kernel support for streaming used to block stillcam access through libusb. Period. End of discussion. The code change in libusb which changes that default behavior is quite recent. It was done because the kernel was *not* addressing the problem at all. That change could presumably be reversed if it were decided that the kernel is going to do the work instead. A POV could be defended, that this behavior of libusb was put in as a stopgap measure because the kernel was not doing its job. In which case the right thing to do is to put the missing functionality into the kernel drivers and take out from libusb the attempt to provide it, when libusb really can't do the job completely. More over a kernel only solution without libgphoto changes won't solve the problem of a libgphoto app keeping the device open locking out streaming. Eh? You really lose me with this
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Tue, 9 Aug 2011, Hans de Goede wrote: Hi, snip OK, another example. The cameras supported in camlibs/jl2005c do not have webcam ability, but someone could at any time design and market a dualmode which has in stillcam mode the same severe limitation. What limitation? Well, the entire memory of the camera must be dumped, or else the camera jams itself. You can stop dumping in the middle of the operation, but you must continue after that. Suppose that you had ten pictures on the camera and you only wanted to download the first one. Then you can do that and temporarily stop downloading the rest. But while exiting you have to check whether the rest are downloaded or not. And if they are not, then it has to be done, with the data simply thrown in the trash, and then the camera's memory pointer reset before the camera is released. How, one might ask, did anyone produce something so primitive? Well, it is done. Perhaps the money saved thereby was at least in part devoted to producing better optics for the camera. At least, one can hope so. But people did produce those cameras, and people have bought them. But does anyone want to reproduce the code to support this kind of crap in the kernel? And go through all of the hoops required in order to fake the behavior which one woulld expect from a real still camera? It has already been done in camlibs/jl2005c and isn't that enough? This actually is an example where doing a kernel driver would be easier, a kernel driver never exits. So it can simply remember where it was reading (and cache the data it has read sofar). If an app requests picture 10, we read 1-10, cache them and return picture 10 to the app, then the same or another app asks for picture 4, get it from cache, asks for picture 20 read 11-20, etc. This, in fact, is the way that the OEM software for most of these cheap cameras works. The camera is dumped, and then raw files for the pictures are created in C:\TEMP. Then the raw files are all processed immediately into viewable pictures, after which thumbnails (which did not previously exist as separate entities) can be created for use in the GUI app. Then, if the user chooses to save certain of the photos, the chosen photos are merely copied to a more permanent location. And when the camera-accessing app is exited, the temporary files are all deleted. Clearly, the OEM approach recommends itself for simplicity. Nevertheless, there is an obvious disadvantage. Namely, *all* of the raw data from the camera needs to be fetched and, as you say, kept in cache. That cache is either going to use RAM, or is going to be based in swap. And not every piece of hardware is a big, honking system with plenty of gigabytes in the RAM slots, and moreover there exist systems with low memory where it is also considered not a good idea to use swap. Precisely because of these realities, the design of libgphoto2 has consciously rejected the approach used in the OEM drivers. Rather, it is a priority to lower the memory footprint by dealing with the data piece by piece. This means, essentially, handling the photos on the camera one at a time. It is worth considering that some of the aforementioned low-powered systems with low quantities of RAM on board, and with no allocated swap space are running Linux these days. Having written code for various small digital picture frames (the keychain models) I know where you are coming from. Trust me I do. Not to worry. I know where you are coming from, too. Trust me I do. Recently I had an interesting bug report, with a corrupt PAT (picture allocation table) turns out that when deleting a picture through the menu inside the frame a different marker gets written to the PAT then when deleting it with the windows software, Fun huh? Yes, of course it is fun. We should not have signed up to do this kind of work if we can't take a joke, right? But, more seriously, there may be some reason why that different character is used -- or OTOH maybe not, and somebody was just being silly. Unfortunately, experience tells me it is probably necessary to figure out which of the two possibilities it is. Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
Hi, On 08/09/2011 07:10 PM, Theodore Kilgore wrote: On Tue, 9 Aug 2011, Hans de Goede wrote: snip No, but both Adam and I realized, approximately at the same time yesterday afternoon, something which is rather important here. Gphoto is not developed exclusively for Linux. Furthermore, it has a significant user base both on Windows and on MacOS, not to mention BSD. It really isn't nice to be screwing around too much with the way it works. Right, so my plan is not to rip out the existing camlibs from libgphoto2, but to instead add a new camlib which talks to /dev/video# nodes which support the new to be defined v4l2 API for this. This camlib will then take precedence over the old libusb based ones when running on a system which has a new enough kernel. On systems without the new enough kernel the matching portdriver won't find any ports, so the camlib will be effectively disabled. On BSD the port driver for this new /dev/video# API and the camlib won't even get compiled. snip It is time to quit thinking in band-aides and solve this properly, 1 logical device means it gets 1 driver. This may be an approach which means some more work then others, but I believe in the end that doing it right is worth the effort. Clearly, we agree about doing it right is worth the effort. The whole discussion right now is about what is right. I'm sorry but I don't get the feeling that the discussion currently is focusing on what is right. To me too much attention is being spend on not throwing away the effort put in the current libgphoto2 camlibs, which I don't like for 2 reasons: 1) It distracts from doing what is right 2) It ignores the fact that a lot has been learned in doing those camlibs, really really a lot. and all that can be re-used in a kernel driver. Let me try to phrase it in a way I think you'll understand. If we agree on doing it right over all other things (such as the fact that doing it right may take a considerable effort). Then this could be an interesting assignment for some of the computer science students I used to be a lecturer for. This assignment could read something like Given the existing situation and knowledge describe all that here, do a re-design for the driverstack for these dual mode cameras, assuming a completely fresh start. Now if I were to give this assignment to a group of students, and they would keep coming back with the but re-doing the camlibs in kernelspace is such a large effort, and we already have them in userspace argument against using one unified driver for these devices, I would give them an F, because they are clearly missing the assuming a completely fresh start part of the assignment. I'm sorry if this sounds a bit harsh, but this is the way how the current discussion feels to me. If we agree on aiming for doing it right then with that comes to me doing a software design from scratch, so without taking into account what is already there. There are of course limits to the from scratch part, in the end we want this to slot into the existing Linux practices for webcams and stillcams, which means: 1) offering a v4l2 /dev/video# node for streaming; and 2) access to the pictures stored on the camera through libgphoto Taking these 2 constrictions into account, and combining that with my firm believe that the solution to all the device sharing problems is handling both functions in a single driver, I end up with only 1 option: Have a kernel driver which provides both functions of the device, with the streaming exported as a standard v4l2 device, and the stillcam function exported with some to be defined API. Combined with a libgphoto2 portlib and camlib for this new API, so that existing libgphoto2 apps can still access the pictures as if nothing was changed. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Tue, 9 Aug 2011, Hans de Goede wrote: Hi, On 08/09/2011 07:10 PM, Theodore Kilgore wrote: On Tue, 9 Aug 2011, Hans de Goede wrote: snip No, but both Adam and I realized, approximately at the same time yesterday afternoon, something which is rather important here. Gphoto is not developed exclusively for Linux. Furthermore, it has a significant user base both on Windows and on MacOS, not to mention BSD. It really isn't nice to be screwing around too much with the way it works. Right, so my plan is not to rip out the existing camlibs from libgphoto2, but to instead add a new camlib which talks to /dev/video# nodes which support the new to be defined v4l2 API for this. This camlib will then take precedence over the old libusb based ones when running on a system which has a new enough kernel. On systems without the new enough kernel the matching portdriver won't find any ports, so the camlib will be effectively disabled. And then, I assume you mean, the old camlib will still work. On BSD the port driver for this new /dev/video# API and the camlib won't even get compiled. snip It is time to quit thinking in band-aides and solve this properly, 1 logical device means it gets 1 driver. This may be an approach which means some more work then others, but I believe in the end that doing it right is worth the effort. Clearly, we agree about doing it right is worth the effort. The whole discussion right now is about what is right. I'm sorry but I don't get the feeling that the discussion currently is focusing on what is right. You are very impatient. To me too much attention is being spend on not throwing away the effort put in the current libgphoto2 camlibs, which I don't like for 2 reasons: 1) It distracts from doing what is right 2) It ignores the fact that a lot has been learned in doing those camlibs, really really a lot. and all that can be re-used in a kernel driver. Note that your two items can contradict or cancel each other out if one is not careful? Let me try to phrase it in a way I think you'll understand. If we agree on doing it right over all other things (such as the fact that doing it right may take a considerable effort). Then this could be an interesting assignment for some of the computer science students I used to be a lecturer for. This assignment could read something like Given the existing situation and knowledge describe all that here, do a re-design for the driverstack for these dual mode cameras, assuming a completely fresh start. Now if I were to give this assignment to a group of students, and they would keep coming back with the but re-doing the camlibs in kernelspace is such a large effort, and we already have them in userspace argument against using one unified driver for these devices, I would give them an F, because they are clearly missing the assuming a completely fresh start part of the assignment. Well, for one thing, Hans, we do not have here any instructor who is giving us an assignment. And nobody is in the position to specify that the assignment says assuming a completely fresh start -- unless Linus happens to be reading this thread and chimes in. Otherwise, unless there is a convincing demonstration that assuming a completely fresh start is an absolute and unavoidable necessity, someone is probably going to disagree. I'm sorry if this sounds a bit harsh, Yes, I am sorry about that, too. but this is the way how the current discussion feels to me. If we agree on aiming for doing it right then with that comes to me doing a software design from scratch, so without taking into account what is already there. Here, a counter-argument is to point out, as I did in a mail earlier this afternoon, that without taking account what is already there might possibly let one overlook something important. And, no, I am not referring to the userspace-kernelspace problem with this. I am referring to the fact that simply to dump the entire contents of the camera into cache (and to keep it there for quite a while) might not necessarily be a good idea and it had been quite consciously rejected to do that in the design of libgphoto2. Not because it is in userspace, but because to do that eats up and ties up RAM of which one cannot assume there is a surplus. Do not misunderstand, though. I am not even going so far as to say that libgphoto2 made the right decision. It certainly has its drawbacks, in that it places severe requirements on someone programming a driver for a really stupid camera. But what I *am* saying is that the issue was anticipated, the issue was faced, and a conscious decision was made. This is the opposite of not anticipating, not facing an issue, and not making any conscious decision. Oh, another example of such lack of deep thought has produced the current crisis, too. I am referring to the amazing decision of some user interface
Re: [Workshop-2011] Media Subsystem Workshop 2011
Hi, On 08/08/2011 12:53 AM, Adam Baker wrote: On Friday 05 August 2011, Hans de Goede wrote: This sounds to be a good theme for the Workshop, or even to KS/2011. Agreed, although we don't need to talk about this for very long, the solution is basically: 1) Define a still image retrieval API for v4l2 devices (there is only 1 interface for both functions on these devices, so only 1 driver, and to me it makes sense to extend the existing drivers to also do still image retrieval). 2) Modify existing kernel v4l2 drivers to provide this API 3) Write a new libgphoto driver which talks this interface (only need to do one driver since all dual mode cams will export the same API). 1) is something to discuss at the workshop. This approach sounds fine as long as you can come up with a definition for the API that covers the existing needs and is extensible when new cameras come along and doesn't create horrible inefficiencies by not matching the way some cameras work. I've only got one example of such a camera and it is a fairly basic one but things I can imagine the API needing to provide are 1) Report number of images on device Make that report highest picture number present call. We want to provide consistent numbers for pictures even if some are deleted, renumbering them on the fly when a picture gets deleted is no good, esp. since multiple apps may be using the device at the same time. So we may have a hole in out numbering, hence my initial proposal of having the following API: int get_max_picture_nr() int is_picture_present(int nr) int get_picture(int nr) int delete_picture(int nr) int delete_all() 2) Select an image to read (for some cameras selecting next may be much more efficient than selecting at random although whether that inefficiency occurs when selecting, when reading image info or when reading image data may vary) 3) Read image information for selected image (resolution, compression type, FOURCC) I have not yet thought about meta-data. But I agree we will need some metadata to convey things like the format of the picture data returned by get_picture (this will be raw data any conversion / post processing will be done in userspace). 4) Read raw image data for selected image 5) Delete individual image (not supported by all cameras) 6) Delete all images (sometimes supported on cameras that don't support individual delete) I'm not sure if any of these cameras support tethered capture but if they do then add Take photo Set resolution That is what the webcam mode is for :) I doubt if any of them support EXIF data, thumbnail images, the ability to upload images to the camera or any sound recording but if they do then those are additional things that gphoto2 would want to be able to do. sound recordings can be handled like pictures but with a different FOURCC code (conveying the contents is audio stored in fmt foo). Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
Em 07-08-2011 23:26, Theodore Kilgore escreveu: (first of two replies to Adam's message; second reply deals with other topics) On Sun, 7 Aug 2011, Adam Baker wrote: On Friday 05 August 2011, Hans de Goede wrote: This sounds to be a good theme for the Workshop, or even to KS/2011. Agreed, although we don't need to talk about this for very long, the solution is basically: 1) Define a still image retrieval API for v4l2 devices (there is only 1 interface for both functions on these devices, so only 1 driver, and to me it makes sense to extend the existing drivers to also do still image retrieval). 2) Modify existing kernel v4l2 drivers to provide this API 3) Write a new libgphoto driver which talks this interface (only need to do one driver since all dual mode cams will export the same API). 1) is something to discuss at the workshop. This approach sounds fine as long as you can come up with a definition for the API that covers the existing needs and is extensible when new cameras come along and doesn't create horrible inefficiencies by not matching the way some cameras work. I've only got one example of such a camera and it is a fairly basic one but things I can imagine the API needing to provide are 1) Report number of images on device 2) Select an image to read (for some cameras selecting next may be much more efficient than selecting at random although whether that inefficiency occurs when selecting, when reading image info or when reading image data may vary) 3) Read image information for selected image (resolution, compression type, FOURCC) 4) Read raw image data for selected image 5) Delete individual image (not supported by all cameras) 6) Delete all images (sometimes supported on cameras that don't support individual delete) I'm not sure if any of these cameras support tethered capture but if they do then add Take photo Set resolution I doubt if any of them support EXIF data, thumbnail images, the ability to upload images to the camera or any sound recording but if they do then those are additional things that gphoto2 would want to be able to do. Adam, Yipe. This looks to me like one inglorious mess. I do not know if it is feasible, or not, but I would wish for something much more simple. Namely, if the camera is not a dual-mode camera then nothing of this is necessary, of course. But if it is a dual-mode camera then the kernel driver is able to hand off the camera to a (libgphoto2-based) userspace driver which can handle all of the gory details of what the camera can do in its role as a still camera. This would imply that there is a device which libgphoto2 can access, presumably another device which is distinct from /dev/videoX, lets call it right now /dev/camX just to give it a name during the discussion. So then what happens ought to be something like the following: 1. Camera is plugged in, detected, and kernel module is fired up. Then either 2a. A streaming app is started. Then, upon request from outside the kernel, the /dev/videoX is locked in and /dev/camX is locked out. The camera streams until told to quit streaming, and in the meantime any access to /dev/camX is not permitted. When the streaming is turned off, the lock is released. or 2b. A stillcam app is started. Then similar to 2a, but the locking is reversed. I think that this kind of thing would keep life simple. As I understand what Hans is envisioning, it is pretty much along the same lines, too. It would mean, of course, that the way that libgphoto2 would access one of these cameras would be directly to access the /dev/camX provided by the kernel, and not to use libusb. But that can be done, I think. As I mentioned before, Hans has written several libgphoto2 drivers for digital picture frames which are otherwise seen as USB mass storage devices. Something similar would have to be done with dual-mode cameras. I will send a second reply to this message, which deals in particular with the list of abilities you outlined above. The point is, the situation as to that list of abilities is more chaotic than is generally realized. And when people are laying plans they really need to be aware of that. From what I understood from your proposal, /dev/camX would be providing a libusb-like interface, right? If so, then, I'd say that we should just use the current libusb infrastructure. All we need is a way to lock libusb access when another driver is using the same USB interface. Hans and Adam's proposal is to actually create a /dev/camX node that will give fs-like access to the pictures. As the data access to the cameras generally use PTP (or a PTP-like protocol), probably one driver will handle several different types of cameras, so, we'll end by having one different driver for PTP than the V4L driver. In other words, part of libgphoto2 code will be moved into the Kernel, to allow
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Mon, 8 Aug 2011, Adam Baker wrote: Further testing reveals the situation is more complex than I first thought - the behaviour I get depends upon whether what gets plugged in is a full speed or a high speed device. After I've run the test of running gphoto whilst streaming from a supported dual mode camera, lsusb fails to recognise a high speed device plugged into the port the camera was plugged into (it works fine if plugged in elsewhere) and lsusb hangs if I plug in a new low speed or full speed device. When I get some time I'll see if I can recreate the problem using libusb with a totally different device. Looking around my pile of USB bits for something full speed with a kernel driver I've got a PL2303 serial port. Would that be a good choice to test with? I have no idea. But the symptoms you describe are indicative of a hardware problem, not a driver bug. Just for reference with a full speed device I see the messages below in dmesg with the second one only appearing when I do lsusb [10832.128039] usb 3-2: new full speed USB device using uhci_hcd and address 34 [10847.240031] usb 3-2: device descriptor read/64, error -110 and with a high speed device I see a continuous stream of [11079.820097] usb 1-4: new high speed USB device using ehci_hcd and address 103 [11079.888355] hub 1-0:1.0: unable to enumerate USB device on port 4 [11080.072377] hub 1-0:1.0: unable to enumerate USB device on port 4 [11080.312053] usb 1-4: new high speed USB device using ehci_hcd and address 105 [11080.380418] hub 1-0:1.0: unable to enumerate USB device on port 4 [11080.620030] usb 1-4: new high speed USB device using ehci_hcd and address 106 [11080.688322] hub 1-0:1.0: unable to enumerate USB device on port 4 The dmesg log is relatively uninformative unless you enable CONFIG_USB_DEBUG in the kernel build. Have you tried running these tests on a different computer, preferably one using a different chipset? Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Sun, 7 Aug 2011, Theodore Kilgore wrote: This indirectly answers my question, above, about whatever device there may or may not be. What I get from this, and also from a bit of snooping around, is that there is not any dev that gets created in order to be accessed by libusb. Just an entry under /proc/bus/usb, which AFAICT is at most a pseudo-device. Thanks. Nowadays, most distributions create device nodes under /dev/bus/usb. A few also support the old /proc/bus/usb files. So, Alan, what do you think is the best way to go about the problem? The camera can act as a stillcam or as a webcam. The problem is to provide access to both, with equal facility (and of course to lock out access to whichever action is not currently in use, if the other one is). The current situation with libusb does not cut it, as among other things it currently does only half the job and seemingly cannot address the locking problem. Hans suggests to create two explicit devices, /dev/video (as already done and something like /dev/cam. Then access webcam function as now and stillcam function with libgphoto2, as now, but through /dev/cam instead of through libusb. This would seem to me to solve all the problems, but at the expense of some work. Can you think of something more clever? I'm not familiar with the MTP protocol used in the stillcam mode, or how feasible it would be to implement that protocol in a kernel driver. Maybe a good compromise would be to create a kind of stub driver that could negotiate the device access while still delegating most of the real work to userspace. This could become a bigger problem if this kind of design becomes an ongoing trend. To do what Hans was suggesting, today we have to merge two separate drivers... then tomorrow we would have to merge two others and then later on even more. Before you know it, we would end up with a single gigantic kernel driver to manage every USB device! Obviously not a sustainable approach in the long run. Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Mon, 8 Aug 2011, Mauro Carvalho Chehab wrote: Em 07-08-2011 23:26, Theodore Kilgore escreveu: (first of two replies to Adam's message; second reply deals with other topics) On Sun, 7 Aug 2011, Adam Baker wrote: On Friday 05 August 2011, Hans de Goede wrote: This sounds to be a good theme for the Workshop, or even to KS/2011. Agreed, although we don't need to talk about this for very long, the solution is basically: 1) Define a still image retrieval API for v4l2 devices (there is only 1 interface for both functions on these devices, so only 1 driver, and to me it makes sense to extend the existing drivers to also do still image retrieval). 2) Modify existing kernel v4l2 drivers to provide this API 3) Write a new libgphoto driver which talks this interface (only need to do one driver since all dual mode cams will export the same API). 1) is something to discuss at the workshop. This approach sounds fine as long as you can come up with a definition for the API that covers the existing needs and is extensible when new cameras come along and doesn't create horrible inefficiencies by not matching the way some cameras work. I've only got one example of such a camera and it is a fairly basic one but things I can imagine the API needing to provide are 1) Report number of images on device 2) Select an image to read (for some cameras selecting next may be much more efficient than selecting at random although whether that inefficiency occurs when selecting, when reading image info or when reading image data may vary) 3) Read image information for selected image (resolution, compression type, FOURCC) 4) Read raw image data for selected image 5) Delete individual image (not supported by all cameras) 6) Delete all images (sometimes supported on cameras that don't support individual delete) I'm not sure if any of these cameras support tethered capture but if they do then add Take photo Set resolution I doubt if any of them support EXIF data, thumbnail images, the ability to upload images to the camera or any sound recording but if they do then those are additional things that gphoto2 would want to be able to do. Adam, Yipe. This looks to me like one inglorious mess. I do not know if it is feasible, or not, but I would wish for something much more simple. Namely, if the camera is not a dual-mode camera then nothing of this is necessary, of course. But if it is a dual-mode camera then the kernel driver is able to hand off the camera to a (libgphoto2-based) userspace driver which can handle all of the gory details of what the camera can do in its role as a still camera. This would imply that there is a device which libgphoto2 can access, presumably another device which is distinct from /dev/videoX, lets call it right now /dev/camX just to give it a name during the discussion. So then what happens ought to be something like the following: 1. Camera is plugged in, detected, and kernel module is fired up. Then either 2a. A streaming app is started. Then, upon request from outside the kernel, the /dev/videoX is locked in and /dev/camX is locked out. The camera streams until told to quit streaming, and in the meantime any access to /dev/camX is not permitted. When the streaming is turned off, the lock is released. or 2b. A stillcam app is started. Then similar to 2a, but the locking is reversed. I think that this kind of thing would keep life simple. As I understand what Hans is envisioning, it is pretty much along the same lines, too. It would mean, of course, that the way that libgphoto2 would access one of these cameras would be directly to access the /dev/camX provided by the kernel, and not to use libusb. But that can be done, I think. As I mentioned before, Hans has written several libgphoto2 drivers for digital picture frames which are otherwise seen as USB mass storage devices. Something similar would have to be done with dual-mode cameras. I will send a second reply to this message, which deals in particular with the list of abilities you outlined above. The point is, the situation as to that list of abilities is more chaotic than is generally realized. And when people are laying plans they really need to be aware of that. From what I understood from your proposal, /dev/camX would be providing a libusb-like interface, right? If so, then, I'd say that we should just use the current libusb infrastructure. All we need is a way to lock libusb access when another driver is using the same USB interface. Hans and Adam's proposal is to actually create a /dev/camX node that will give fs-like access to the pictures. As the data access to the cameras generally use PTP (or a PTP-like protocol), probably one driver will handle several
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Mon, 8 Aug 2011, Alan Stern wrote: On Sun, 7 Aug 2011, Theodore Kilgore wrote: This indirectly answers my question, above, about whatever device there may or may not be. What I get from this, and also from a bit of snooping around, is that there is not any dev that gets created in order to be accessed by libusb. Just an entry under /proc/bus/usb, which AFAICT is at most a pseudo-device. Thanks. Nowadays, most distributions create device nodes under /dev/bus/usb. A few also support the old /proc/bus/usb files. What does this mean, exactly, in practice? You are right that I have the /dev/bus/usb/ files but does everybody have them, these days? So, Alan, what do you think is the best way to go about the problem? The camera can act as a stillcam or as a webcam. The problem is to provide access to both, with equal facility (and of course to lock out access to whichever action is not currently in use, if the other one is). The current situation with libusb does not cut it, as among other things it currently does only half the job and seemingly cannot address the locking problem. Alan, I do not know if you have actually followed what has been going on. One of the things which has been happening is that we have a real, immediate mess on our hands, in that some folks have put together distros which insist upon automatically mounting all stilcams whenever they are plugged in, thereby destroying the ability of any dual-mode camera to function as a webcam. This brilliant move seems to have been conceived in sin, or at least in total ignorance or disregard of the underlying problems, but it has been done and sold to the public as a convenience. So it seems to me that the infrastructural problem needs attention. Hans suggests to create two explicit devices, /dev/video (as already done and something like /dev/cam. Then access webcam function as now and stillcam function with libgphoto2, as now, but through /dev/cam instead of through libusb. This would seem to me to solve all the problems, but at the expense of some work. Can you think of something more clever? I'm not familiar with the MTP protocol used in the stillcam mode, or how feasible it would be to implement that protocol in a kernel driver. What? You, too? :-/ As I said to Mauro, all of the currently affected cameras that I know of are proprietary. And proprietary means what it says. It means that the protocol depends upon the controller chip in the camera, not upon any standard. And not just the commands, but also the methodologies and approaches which underlie the command structure can differ wildly. Maybe a good compromise would be to create a kind of stub driver that could negotiate the device access while still delegating most of the real work to userspace. Hooray. This appears to me to be a very good solution. This could become a bigger problem if this kind of design becomes an ongoing trend. To do what Hans was suggesting, today we have to merge two separate drivers... Actually, the current count is even worse. It is four drivers, not two. See previous remarks about proprietary protocols. then tomorrow we would have to merge two others and then later on even more. Before you know it, we would end up with a single gigantic kernel driver to manage every USB device! More exactly, a gillion kernel drivers, each one to control each proprietary device, and then, I suppose, one ring to rule them all ... But that is just as bad as your nightmare, so it makes no difference in the end, does it? Obviously not a sustainable approach in the long run. I agree approximately 120% with this. Let's think of a more clever way. If we get the basic idea right, it really ought not to be too terribly difficult. Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
Em 08-08-2011 14:39, Theodore Kilgore escreveu: On Mon, 8 Aug 2011, Mauro Carvalho Chehab wrote: Em 07-08-2011 23:26, Theodore Kilgore escreveu: (first of two replies to Adam's message; second reply deals with other topics) On Sun, 7 Aug 2011, Adam Baker wrote: On Friday 05 August 2011, Hans de Goede wrote: This sounds to be a good theme for the Workshop, or even to KS/2011. Agreed, although we don't need to talk about this for very long, the solution is basically: 1) Define a still image retrieval API for v4l2 devices (there is only 1 interface for both functions on these devices, so only 1 driver, and to me it makes sense to extend the existing drivers to also do still image retrieval). 2) Modify existing kernel v4l2 drivers to provide this API 3) Write a new libgphoto driver which talks this interface (only need to do one driver since all dual mode cams will export the same API). 1) is something to discuss at the workshop. This approach sounds fine as long as you can come up with a definition for the API that covers the existing needs and is extensible when new cameras come along and doesn't create horrible inefficiencies by not matching the way some cameras work. I've only got one example of such a camera and it is a fairly basic one but things I can imagine the API needing to provide are 1) Report number of images on device 2) Select an image to read (for some cameras selecting next may be much more efficient than selecting at random although whether that inefficiency occurs when selecting, when reading image info or when reading image data may vary) 3) Read image information for selected image (resolution, compression type, FOURCC) 4) Read raw image data for selected image 5) Delete individual image (not supported by all cameras) 6) Delete all images (sometimes supported on cameras that don't support individual delete) I'm not sure if any of these cameras support tethered capture but if they do then add Take photo Set resolution I doubt if any of them support EXIF data, thumbnail images, the ability to upload images to the camera or any sound recording but if they do then those are additional things that gphoto2 would want to be able to do. Adam, Yipe. This looks to me like one inglorious mess. I do not know if it is feasible, or not, but I would wish for something much more simple. Namely, if the camera is not a dual-mode camera then nothing of this is necessary, of course. But if it is a dual-mode camera then the kernel driver is able to hand off the camera to a (libgphoto2-based) userspace driver which can handle all of the gory details of what the camera can do in its role as a still camera. This would imply that there is a device which libgphoto2 can access, presumably another device which is distinct from /dev/videoX, lets call it right now /dev/camX just to give it a name during the discussion. So then what happens ought to be something like the following: 1. Camera is plugged in, detected, and kernel module is fired up. Then either 2a. A streaming app is started. Then, upon request from outside the kernel, the /dev/videoX is locked in and /dev/camX is locked out. The camera streams until told to quit streaming, and in the meantime any access to /dev/camX is not permitted. When the streaming is turned off, the lock is released. or 2b. A stillcam app is started. Then similar to 2a, but the locking is reversed. I think that this kind of thing would keep life simple. As I understand what Hans is envisioning, it is pretty much along the same lines, too. It would mean, of course, that the way that libgphoto2 would access one of these cameras would be directly to access the /dev/camX provided by the kernel, and not to use libusb. But that can be done, I think. As I mentioned before, Hans has written several libgphoto2 drivers for digital picture frames which are otherwise seen as USB mass storage devices. Something similar would have to be done with dual-mode cameras. I will send a second reply to this message, which deals in particular with the list of abilities you outlined above. The point is, the situation as to that list of abilities is more chaotic than is generally realized. And when people are laying plans they really need to be aware of that. From what I understood from your proposal, /dev/camX would be providing a libusb-like interface, right? If so, then, I'd say that we should just use the current libusb infrastructure. All we need is a way to lock libusb access when another driver is using the same USB interface. Hans and Adam's proposal is to actually create a /dev/camX node that will give fs-like access to the pictures. As the data access to the cameras generally use PTP (or a PTP-like protocol), probably one driver will handle several different types of cameras, so, we'll end by having one
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Mon, 8 Aug 2011, Theodore Kilgore wrote: On Mon, 8 Aug 2011, Alan Stern wrote: On Sun, 7 Aug 2011, Theodore Kilgore wrote: This indirectly answers my question, above, about whatever device there may or may not be. What I get from this, and also from a bit of snooping around, is that there is not any dev that gets created in order to be accessed by libusb. Just an entry under /proc/bus/usb, which AFAICT is at most a pseudo-device. Thanks. Nowadays, most distributions create device nodes under /dev/bus/usb. A few also support the old /proc/bus/usb files. What does this mean, exactly, in practice? You are right that I have the /dev/bus/usb/ files but does everybody have them, these days? Pretty much everybody using udev should have them, which means pretty much every desktop system. ... Maybe a good compromise would be to create a kind of stub driver that could negotiate the device access while still delegating most of the real work to userspace. Hooray. This appears to me to be a very good solution. I'm not so sure. It would require vast changes to the userspace program, for example. I agree approximately 120% with this. Let's think of a more clever way. If we get the basic idea right, it really ought not to be too terribly difficult. The method Hans suggested was rather clunky. It also required drivers to know when the device was in use, which may be okay for a video driver but is not so practical for usb-storage (although to be fair, I suspect usb-storage wouldn't need to be involved). And it required kernel drivers to inform user programs somehow when they want to get control of the device back, which is not the sort of thing drivers normally have to do. Even if we could come up with a way to let the video driver somehow share ownership of the device with usbfs, we'd still have to set up a protocol for deciding who was in charge at any given time. Would it be okay for the userspace program simply to say I want control now and I'm done, you can have control back? For that matter, what should the video driver do if the user program crashes or hangs while in charge of the device? Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Mon, 8 Aug 2011, Mauro Carvalho Chehab wrote: Em 08-08-2011 14:39, Theodore Kilgore escreveu: On Mon, 8 Aug 2011, Mauro Carvalho Chehab wrote: Em 07-08-2011 23:26, Theodore Kilgore escreveu: (first of two replies to Adam's message; second reply deals with other topics) In summary, there are currently two proposals: 1) a resource lock for USB interface between V4L and libusb; 2) a PTP-like USB driver, plus a resource lock between V4L and the PTP-like driver. The same resource lock may also be implemented at libusb, in order to avoid concurrency. As you said that streaming on some cameras may delete all pictures from it, I suspect that (2) is the best alternative. Thanks, Mauro Mauro, In fact none of the currently known and supported cameras are using PTP. All of them are proprietary. They have a rather intimidating set of differences in functionality, too. Namely, some of them have an isochronous endpoint, and some of them rely exclusively upon bulk transport. Some of them have a well developed set of internal capabilities as far as handling still photos are concerned. I mean, such things as the ability to download a single photo, selected at random from the set of photos on the camera, and some do not, requiring that the ability to do this is emulated in software -- by first downloading all previously listed photos and sending the data to /dev/null, then downloading the desired photo and saving it. Some of them permit deletion of individual photos, or all photos, and some do not. For some of them it is even true, as I have previously mentioned, that the USB command string which will delete all photos is the same command used for starting the camera in streaming mode. But the point here is that these cameras are all different from one another, depending upon chipset and even, sometimes, upon firmware or chipset version. The still camera abilities and limitations of all of them are pretty much worked out in libgphoto2. My suggestion would be that the libgphoto2 support libraries for these cameras ought to be left the hell alone, except for some changes in, for example, how the camera is accessed in the first place (through libusb or through a kernel device) in order to address adequately the need to support both modes. I know what is in those libgphoto2 drivers because I wrote them. I can definitely promise that to move all of that functionality over into kernel modules would be a nightmare and would moreover greatly contribute to kernel bloat. You really don't want to go there. As to whether to use libusb or not to use libusb: It would be very nice to be able to keep using libusb to get access to these cameras, as then no change in the existing stillcam drivers would be required at all. Furthermore, if it were possible to solve all of the associated locking problems and to do it this way, it would be something that could be generalized to any analogous situation. This would be very nice. I can also imagine, of course, that such an approach might require changes in libusb. For example, the current ability of libusb itself to switch off a kernel device might possibly be a step in the wrong direction, and it might possibly be needed to move that function, somehow, out of libusb and into the kernel support for affected hardware. In the alternative, it ought to be possible for a libgphoto2 driver to hook up directly to a kernel-created device without going through libusb, and, as I have said in earlier messages, some of our driver code (for digital picture frames in particular) does just that. Then, whatever /dev entries and associated locking problems are needed could be handled by the kernel, and libgphoto2 talks to the device. But if things are done this way I strongly suggest that as little of the internals of the libgphoto2 driver are put in the kernel as it is possible to do. Be very economical about that, else there will be a big mess. Doing an specific libusb-like approach just for those cams seems to be the wrong direction, as such driver would be just a fork of an already existing code. I'm all against duplicating it. Well, in practice the fork would presumably be carried out by yours truly. Presumably with the advice and help of concerned parties. too. Since I am involved on both the kernel side and the libgphoto2 side of the support for the same cameras, it would certainly shorten the lines of communication at the very least. Therefore this is not infeasible. So, either we need to move the code from libgphoto2 to kernel As I said, I think you don't want to do that. or work into an approach that will make libusb (or an appropriate substitute) to return -EBUSY when a driver like V4L is in usage, and vice-versa. I never took a look on how libusb works. It seems
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Mon, 8 Aug 2011, Alan Stern wrote: On Mon, 8 Aug 2011, Theodore Kilgore wrote: On Mon, 8 Aug 2011, Alan Stern wrote: On Sun, 7 Aug 2011, Theodore Kilgore wrote: This indirectly answers my question, above, about whatever device there may or may not be. What I get from this, and also from a bit of snooping around, is that there is not any dev that gets created in order to be accessed by libusb. Just an entry under /proc/bus/usb, which AFAICT is at most a pseudo-device. Thanks. Nowadays, most distributions create device nodes under /dev/bus/usb. A few also support the old /proc/bus/usb files. What does this mean, exactly, in practice? You are right that I have the /dev/bus/usb/ files but does everybody have them, these days? Pretty much everybody using udev should have them, which means pretty much every desktop system. OK, so far so good. ... Maybe a good compromise would be to create a kind of stub driver that could negotiate the device access while still delegating most of the real work to userspace. Hooray. This appears to me to be a very good solution. I'm not so sure. It would require vast changes to the userspace program, for example. Such as? I agree approximately 120% with this. Let's think of a more clever way. If we get the basic idea right, it really ought not to be too terribly difficult. The method Hans suggested was rather clunky. If it involves moving practically all of the gory details of the support of stillcam mode for individual dual-mode cameras into the kernel, then it certainly appears clunky to me, too. It also required drivers to know when the device was in use, which may be okay for a video driver but is not so practical for usb-storage (although to be fair, I suspect usb-storage wouldn't need to be involved). Yes, I can see that. Usb-storage is, essentially, in use while the device is attached, and that has to be true because the device is a storage device. And alas, not all storage devices even get mounted, so one cannot decide whether the device is in use just by checking whether or not something on it is mounted ... And it required kernel drivers to inform user programs somehow when they want to get control of the device back, Why, exactly? I mean, fundamentally we have two functionalities of the device which are accessed, at the user level, by two userspace programs. One of them gets the still photos off the camera, and the other one gets the video stream. Perhaps we just need a method for saying No! to either one of those apps if the other one is using the camera? which is not the sort of thing drivers normally have to do. Even if we could come up with a way to let the video driver somehow share ownership of the device with usbfs, we'd still have to set up a protocol for deciding who was in charge at any given time. Would it be okay for the userspace program simply to say I want control now and I'm done, you can have control back? Actually, I would expect that if one program is accessing the device then the other one can't, and this works the same in both directions. Unless you think that what you described is better? For that matter, what should the video driver do if the user program crashes or hangs while in charge of the device? Good one. Commit seppuku? Seriously, though, what should it do if a video streaming userspace program crashes or hangs? Probably, the same thing should happen as it should if a photo-getting program crashes or hangs. Namely whatever needs to be done in order to prevent some kind of catastrophe ought to be implemented. Incidentally, I think that in some respects the fact that webcam support is in the kernel and stillcam support is in userspace is a red herring. The fundamental problem is a piece of hardware which does two different kinds of things which require two different kinds of support. Further to narrow this, though, it is hardware which needs to be usable for either function at any time, which does distinguish it from such things as a one-shot loading of firmware as happens with mass storage USB modems. Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
Em 08-08-2011 16:32, Theodore Kilgore escreveu: Doing an specific libusb-like approach just for those cams seems to be the wrong direction, as such driver would be just a fork of an already existing code. I'm all against duplicating it. Well, in practice the fork would presumably be carried out by yours truly. Presumably with the advice and help of concerned parties. too. Since I am involved on both the kernel side and the libgphoto2 side of the support for the same cameras, it would certainly shorten the lines of communication at the very least. Therefore this is not infeasible. Forking the code just because we have something special is the wrong thing to do (TM). I would not like to fork V4L core code due to some special need, but instead to add some glue there to cover the extra case. Maintaining a fork is bad in long term, as the same fixes/changes will likely be needed on both copies. Adding some sort of resource locking like the example I've pointed seems easy and will work just fine. So, either we need to move the code from libgphoto2 to kernel As I said, I think you don't want to do that. I don't have a strong opinion about that ATM. Both approaches have advantages and disadvantages. or work into an approach that will make libusb (or an appropriate substitute) Something like what Hans proposed makes sense on my eyes, as an appropriate substitute, but it seems that this is exactly what you don't want. I can really see two alternatives there: 1) keep the libusb API, e. g. the driver for data access is on userspace, and a char device allows to communicate with USB in a transparent way; 2) create an API for camera, like Hans/Adam proposal. If we take (1), we should just use the already existing Kernel infrastructure, plus a resource locking, to put the USB device into exclusive access mode. to return -EBUSY when a driver like V4L is in usage, and vice-versa. I never took a look on how libusb works. It seems that the logic for it is at drivers/usb/core/devio.c. Assuming that this is correct driver for libusb, the locking patch would be similar to the enclosed one. Of course, more work will be needed. For example, in the specific case of devices where starting stream will clean the memory data, the V4L driver will need some additional logic to detect if the memory is filled and not allowing stream (or requiring CAP_SYS_ADMIN, returning -EPERM otherwise). Yes, this is probably a good idea in any event. Agreed. As far as I know, this would affect just one kernel driver. A complication is that it is only some of the cameras supported by that driver, and they need to be detected. Yes. NOTE: As the problem also happens with some PCI devices, instead of adding such locking schema at usb_device, it seems better to bind whatever solution into struct device. Interesting comment. The problem with PCI devices is not exactly the same, but I tried to think on a way that could also work for those issues. Eventually, when actually implementing the code, we may come to a conclusion that this is the right thing to do, or to decide to address those cases with a different solution. The issue we have (and that it is bus-agnostic), is that some resources depend on or are mutually exclusive of another resource. For example, considering a single-tuner device that supports both analog and digital TV. Due to the analog TV, such device needs to have an ALSA module. However, accessing the ALSA input depends on having the hardware in analog mode, as, on almost all supported hardware, there's no MPEG decoder inside it. So, accessing the alsa device should return -EBUSY, if the device is on digital mode. On the other hand, as the device has just one tuner, the digital mode driver can't be used simultaneously with the analog mode one. So, what I'm seeing is that we need some kernel way to describe hardware resource dependencies. Regards, Mauro -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Monday 08 August 2011, Mauro Carvalho Chehab wrote: Well, in practice the fork would presumably be carried out by yours truly. Presumably with the advice and help of concerned parties. too. Since I am involved on both the kernel side and the libgphoto2 side of the support for the same cameras, it would certainly shorten the lines of communication at the very least. Therefore this is not infeasible. Forking the code just because we have something special is the wrong thing to do (TM). I would not like to fork V4L core code due to some special need, but instead to add some glue there to cover the extra case. Maintaining a fork is bad in long term, as the same fixes/changes will likely be needed on both copies. Unfortunately there is some difficulty with libusb in that respect. libgphoto relies upon libusb-0.1 becuase it is cross platform and Win32 support in libusb-1.0 is only just being integrated. The libusb developers consider the libusb-0.1 API frozen and are not willing to extend it to address our problem. libusb doesn't expose the file descriptor it uses to talk to the underlying device so it is hard to extend the interface without forking libusb (The best hope I can think of at the moment is to get the distros to accept a patch for it to add the extra required API call(s) and for libgphoto to use the extra features in that patch if it detects it is supported at compile time). Adam Baker -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Monday 08 August 2011, Mauro Carvalho Chehab wrote: I will send a second reply to this message, which deals in particular with the list of abilities you outlined above. The point is, the situation as to that list of abilities is more chaotic than is generally realized. And when people are laying plans they really need to be aware of that. From what I understood from your proposal, /dev/camX would be providing a libusb-like interface, right? If so, then, I'd say that we should just use the current libusb infrastructure. All we need is a way to lock libusb access when another driver is using the same USB interface. I think adding the required features to libusb is in general the correct approach however some locking may be needed in the kernel regardless to ensure a badly behaved libusb or libusb user can't corrupt kernel state. Hans and Adam's proposal is to actually create a /dev/camX node that will give fs-like access to the pictures. As the data access to the cameras generally use PTP (or a PTP-like protocol), probably one driver will handle several different types of cameras, so, we'll end by having one different driver for PTP than the V4L driver. I'm not advocating this approach, my post was intended as a straw man to allow the advantages and disadvantages of such an approach to be considered by all concerned. I suspected it would be excessively complex but I don't know enough about the various cameras to be certain. Adam -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Mon, 8 Aug 2011, Theodore Kilgore wrote: Maybe a good compromise would be to create a kind of stub driver that could negotiate the device access while still delegating most of the real work to userspace. Hooray. This appears to me to be a very good solution. I'm not so sure. It would require vast changes to the userspace program, for example. Such as? Such as completely rewriting the USB interface. You wouldn't be able to use libusb, for example. The method Hans suggested was rather clunky. If it involves moving practically all of the gory details of the support of stillcam mode for individual dual-mode cameras into the kernel, then it certainly appears clunky to me, too. It also required drivers to know when the device was in use, which may be okay for a video driver but is not so practical for usb-storage (although to be fair, I suspect usb-storage wouldn't need to be involved). Yes, I can see that. Usb-storage is, essentially, in use while the device is attached, and that has to be true because the device is a storage device. And alas, not all storage devices even get mounted, so one cannot decide whether the device is in use just by checking whether or not something on it is mounted ... And it required kernel drivers to inform user programs somehow when they want to get control of the device back, Why, exactly? Don't ask me, ask Hans! :-) I mean, fundamentally we have two functionalities of the device which are accessed, at the user level, by two userspace programs. One of them gets the still photos off the camera, and the other one gets the video stream. Perhaps we just need a method for saying No! to either one of those apps if the other one is using the camera? That's basically what I suggested below. which is not the sort of thing drivers normally have to do. Even if we could come up with a way to let the video driver somehow share ownership of the device with usbfs, we'd still have to set up a protocol for deciding who was in charge at any given time. Would it be okay for the userspace program simply to say I want control now and I'm done, you can have control back? Actually, I would expect that if one program is accessing the device then the other one can't, and this works the same in both directions. Unless you think that what you described is better? When a program uses libgphoto2, how is the kernel supposed to know when the program is busy accessing the device? The kernel can't just ask the program. Incidentally, I think that in some respects the fact that webcam support is in the kernel and stillcam support is in userspace is a red herring. No, this has some significant implications. In particular, there's no good way for the kernel driver to ask the userspace driver if it is busy. If both drivers were in the kernel, this would be easy to arrange. Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Mon, 8 Aug 2011, Adam Baker wrote: On Monday 08 August 2011, Mauro Carvalho Chehab wrote: Well, in practice the fork would presumably be carried out by yours truly. Presumably with the advice and help of concerned parties. too. Since I am involved on both the kernel side and the libgphoto2 side of the support for the same cameras, it would certainly shorten the lines of communication at the very least. Therefore this is not infeasible. Forking the code just because we have something special is the wrong thing to do (TM). I would not like to fork V4L core code due to some special need, but instead to add some glue there to cover the extra case. Maintaining a fork is bad in long term, as the same fixes/changes will likely be needed on both copies. Unfortunately there is some difficulty with libusb in that respect. libgphoto relies upon libusb-0.1 becuase it is cross platform and Win32 support in libusb-1.0 is only just being integrated. The libusb developers consider the libusb-0.1 API frozen and are not willing to extend it to address our problem. libusb doesn't expose the file descriptor it uses to talk to the underlying device so it is hard to extend the interface without forking libusb (The best hope I can think of at the moment is to get the distros to accept a patch for it to add the extra required API call(s) and for libgphoto to use the extra features in that patch if it detects it is supported at compile time). Adam, Yes, you are quite correct about this. I was just on the way out of the house and remembered that this problem exists, decided to re-connect and add this point to the witches' brew that we are working on. What struck me was not the Windows support, though, it was the Mac support. And a number of people run Gphoto stuff on Mac, too. That just reinforces your point, of course. Gphoto is explicitly cross-platform. It is developed on Linux but it is supposed to compile on anyone's C compiler and run on any hardware platform or operating system which has available the minimal support require to make it work. You are right. We, basically, can not screw with the internals of libgphoto2. At the outside, one can not go to the point where any changes would break the support for other platforms. Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Mon, 8 Aug 2011, Adam Baker wrote: On Monday 08 August 2011, Mauro Carvalho Chehab wrote: I will send a second reply to this message, which deals in particular with the list of abilities you outlined above. The point is, the situation as to that list of abilities is more chaotic than is generally realized. And when people are laying plans they really need to be aware of that. From what I understood from your proposal, /dev/camX would be providing a libusb-like interface, right? If so, then, I'd say that we should just use the current libusb infrastructure. All we need is a way to lock libusb access when another driver is using the same USB interface. I think adding the required features to libusb is in general the correct approach however some locking may be needed in the kernel regardless to ensure a badly behaved libusb or libusb user can't corrupt kernel state. Hans and Adam's proposal is to actually create a /dev/camX node that will give fs-like access to the pictures. As the data access to the cameras generally use PTP (or a PTP-like protocol), probably one driver will handle several different types of cameras, so, we'll end by having one different driver for PTP than the V4L driver. I'm not advocating this approach, my post was intended as a straw man to allow the advantages and disadvantages of such an approach to be considered by all concerned. I suspected it would be excessively complex but I don't know enough about the various cameras to be certain. Fair enough. Go and have a look at the code in the various subdirectories of libgphoto2/camlibs, and you will see. Also consider that some of those subdirectories do not support currently-supported dual-mode cameras, but some of the ways of doing things that are present there could be applied to any dual-mode camera in the future. A prime example of what I mean can be seen in camlibs/aox. Those cameras are very old now and they probably will never be fully supported. They can download plain bitmap photos, or they can use some kind of compression which is not figured out. They can, as I recall, be run as webcams, too, and then they will presumably use that weird compression. But what is immediately interesting is that in still mode there is no allocation table, or at least none is downloaded. Everything about how many images and what kind of images and what size are they can be read out of a downloaded allocation table on most cameras, but not on these. No. One has to send a sequence of commands and parse the responses to them in order to get the information. I merely mention this example in order to point up the actual complexity of the situation, and the necessity not to make sweeping assumptions about how the camera is supposed to work. Be assured, that already happened when Gphoto was set up, and it made some of these cameras rather hard to support. Why? Well, it was set up with the assumption that all still cameras will do X, and Y, and Z. But be assured that someone either has or will design a still camera which is not capable of doing X, nor Y, nor Z, nor, even, all three of them, at least not in the way envisioned in someone's grand design. OK, another example. The cameras supported in camlibs/jl2005c do not have webcam ability, but someone could at any time design and market a dualmode which has in stillcam mode the same severe limitation. What limitation? Well, the entire memory of the camera must be dumped, or else the camera jams itself. You can stop dumping in the middle of the operation, but you must continue after that. Suppose that you had ten pictures on the camera and you only wanted to download the first one. Then you can do that and temporarily stop downloading the rest. But while exiting you have to check whether the rest are downloaded or not. And if they are not, then it has to be done, with the data simply thrown in the trash, and then the camera's memory pointer reset before the camera is released. How, one might ask, did anyone produce something so primitive? Well, it is done. Perhaps the money saved thereby was at least in part devoted to producing better optics for the camera. At least, one can hope so. But people did produce those cameras, and people have bought them. But does anyone want to reproduce the code to support this kind of crap in the kernel? And go through all of the hoops required in order to fake the behavior which one woulld expect from a real still camera? It has already been done in camlibs/jl2005c and isn't that enough? Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Friday 05 August 2011, Hans de Goede wrote: This sounds to be a good theme for the Workshop, or even to KS/2011. Agreed, although we don't need to talk about this for very long, the solution is basically: 1) Define a still image retrieval API for v4l2 devices (there is only 1 interface for both functions on these devices, so only 1 driver, and to me it makes sense to extend the existing drivers to also do still image retrieval). 2) Modify existing kernel v4l2 drivers to provide this API 3) Write a new libgphoto driver which talks this interface (only need to do one driver since all dual mode cams will export the same API). 1) is something to discuss at the workshop. This approach sounds fine as long as you can come up with a definition for the API that covers the existing needs and is extensible when new cameras come along and doesn't create horrible inefficiencies by not matching the way some cameras work. I've only got one example of such a camera and it is a fairly basic one but things I can imagine the API needing to provide are 1) Report number of images on device 2) Select an image to read (for some cameras selecting next may be much more efficient than selecting at random although whether that inefficiency occurs when selecting, when reading image info or when reading image data may vary) 3) Read image information for selected image (resolution, compression type, FOURCC) 4) Read raw image data for selected image 5) Delete individual image (not supported by all cameras) 6) Delete all images (sometimes supported on cameras that don't support individual delete) I'm not sure if any of these cameras support tethered capture but if they do then add Take photo Set resolution I doubt if any of them support EXIF data, thumbnail images, the ability to upload images to the camera or any sound recording but if they do then those are additional things that gphoto2 would want to be able to do. Regards Adam -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Sun, 7 Aug 2011, Adam Baker wrote: I've addec Hans de Geode and linux-usb to the CC as this response picks up on a related discussion about the usb mini summit. On Friday 05 August 2011, Theodore Kilgore wrote: If you can solve the locking problem between devices in the kernel then it shouldn't matter if one of the kernel devices is the generic device that is used to support libusb. Hmmm. Perhaps not. While we are on the topic, what exactly do you mean by the generic device that is used to support libusb. Which device is that, exactly? The file drivers/usb/core/devio.c registers itself as a driver called usb_device which is used to provide all of the device drivers that live under /proc/bus/usb Let's get things correct. The driver is called usbfs, not usb_device. The things that live under /proc/bus/usb are files representing USB devices, not device drivers. If you look in there for the code to handle the ioctl() USBDEVFS_DISCONNECT then you will find the code that is called when you make a usb_detach_kernel_driver_np() call through libusb. That code, according to the documentation and my testing needs to acquire a lock before it calls usb_driver_release_interface(). Based on my testing to date (using cheese to start a camera streaming and then gphoto2 -L to trigger the disconnect ioctl) I would suggest that the fact it doesn't is a kernel bug that needs fixing What makes you think the code doesn't acquire the lock? (Hint: Look at usbdev_do_ioctl() instead of proc_ioctl().) regardless of whether there is any user space solution to camera mode switching because that code could potentially get called on any in use USB device and if it does even thing like lsusb don't work correctly afterwards and completely unrelated devices don't work if they are later plugged into the same USB port. That's a rather incomprehensible run-on sentence, but as near as I can tell, it is wrong. Alan Stern -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Monday 08 August 2011, Alan Stern wrote: On Sun, 7 Aug 2011, Adam Baker wrote: I've addec Hans de Geode and linux-usb to the CC as this response picks up on a related discussion about the usb mini summit. On Friday 05 August 2011, Theodore Kilgore wrote: If you can solve the locking problem between devices in the kernel then it shouldn't matter if one of the kernel devices is the generic device that is used to support libusb. Hmmm. Perhaps not. While we are on the topic, what exactly do you mean by the generic device that is used to support libusb. Which device is that, exactly? The file drivers/usb/core/devio.c registers itself as a driver called usb_device which is used to provide all of the device drivers that live under /proc/bus/usb Let's get things correct. The driver is called usbfs, not usb_device. The things that live under /proc/bus/usb are files representing USB devices, not device drivers. OK, I was taking the name from the register_chrdev_region call which is ambiguous as to whether that is a driver name or just a name to associate with the devices the driver creates. Hopefully I at least gave enough info to make my meaning clear. If you look in there for the code to handle the ioctl() USBDEVFS_DISCONNECT then you will find the code that is called when you make a usb_detach_kernel_driver_np() call through libusb. That code, according to the documentation and my testing needs to acquire a lock before it calls usb_driver_release_interface(). Based on my testing to date (using cheese to start a camera streaming and then gphoto2 -L to trigger the disconnect ioctl) I would suggest that the fact it doesn't is a kernel bug that needs fixing What makes you think the code doesn't acquire the lock? (Hint: Look at usbdev_do_ioctl() instead of proc_ioctl().) My assumption is based on observed behaviour rather than looking at the code. regardless of whether there is any user space solution to camera mode switching because that code could potentially get called on any in use USB device and if it does even thing like lsusb don't work correctly afterwards and completely unrelated devices don't work if they are later plugged into the same USB port. That's a rather incomprehensible run-on sentence, but as near as I can tell, it is wrong. Further testing reveals the situation is more complex than I first thought - the behaviour I get depends upon whether what gets plugged in is a full speed or a high speed device. After I've run the test of running gphoto whilst streaming from a supported dual mode camera, lsusb fails to recognise a high speed device plugged into the port the camera was plugged into (it works fine if plugged in elsewhere) and lsusb hangs if I plug in a new low speed or full speed device. When I get some time I'll see if I can recreate the problem using libusb with a totally different device. Looking around my pile of USB bits for something full speed with a kernel driver I've got a PL2303 serial port. Would that be a good choice to test with? Just for reference with a full speed device I see the messages below in dmesg with the second one only appearing when I do lsusb [10832.128039] usb 3-2: new full speed USB device using uhci_hcd and address 34 [10847.240031] usb 3-2: device descriptor read/64, error -110 and with a high speed device I see a continuous stream of [11079.820097] usb 1-4: new high speed USB device using ehci_hcd and address 103 [11079.888355] hub 1-0:1.0: unable to enumerate USB device on port 4 [11080.072377] hub 1-0:1.0: unable to enumerate USB device on port 4 [11080.312053] usb 1-4: new high speed USB device using ehci_hcd and address 105 [11080.380418] hub 1-0:1.0: unable to enumerate USB device on port 4 [11080.620030] usb 1-4: new high speed USB device using ehci_hcd and address 106 [11080.688322] hub 1-0:1.0: unable to enumerate USB device on port 4 Adam Baker -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
(first of two replies to Adam's message; second reply deals with other topics) On Sun, 7 Aug 2011, Adam Baker wrote: On Friday 05 August 2011, Hans de Goede wrote: This sounds to be a good theme for the Workshop, or even to KS/2011. Agreed, although we don't need to talk about this for very long, the solution is basically: 1) Define a still image retrieval API for v4l2 devices (there is only 1 interface for both functions on these devices, so only 1 driver, and to me it makes sense to extend the existing drivers to also do still image retrieval). 2) Modify existing kernel v4l2 drivers to provide this API 3) Write a new libgphoto driver which talks this interface (only need to do one driver since all dual mode cams will export the same API). 1) is something to discuss at the workshop. This approach sounds fine as long as you can come up with a definition for the API that covers the existing needs and is extensible when new cameras come along and doesn't create horrible inefficiencies by not matching the way some cameras work. I've only got one example of such a camera and it is a fairly basic one but things I can imagine the API needing to provide are 1) Report number of images on device 2) Select an image to read (for some cameras selecting next may be much more efficient than selecting at random although whether that inefficiency occurs when selecting, when reading image info or when reading image data may vary) 3) Read image information for selected image (resolution, compression type, FOURCC) 4) Read raw image data for selected image 5) Delete individual image (not supported by all cameras) 6) Delete all images (sometimes supported on cameras that don't support individual delete) I'm not sure if any of these cameras support tethered capture but if they do then add Take photo Set resolution I doubt if any of them support EXIF data, thumbnail images, the ability to upload images to the camera or any sound recording but if they do then those are additional things that gphoto2 would want to be able to do. Adam, Yipe. This looks to me like one inglorious mess. I do not know if it is feasible, or not, but I would wish for something much more simple. Namely, if the camera is not a dual-mode camera then nothing of this is necessary, of course. But if it is a dual-mode camera then the kernel driver is able to hand off the camera to a (libgphoto2-based) userspace driver which can handle all of the gory details of what the camera can do in its role as a still camera. This would imply that there is a device which libgphoto2 can access, presumably another device which is distinct from /dev/videoX, lets call it right now /dev/camX just to give it a name during the discussion. So then what happens ought to be something like the following: 1. Camera is plugged in, detected, and kernel module is fired up. Then either 2a. A streaming app is started. Then, upon request from outside the kernel, the /dev/videoX is locked in and /dev/camX is locked out. The camera streams until told to quit streaming, and in the meantime any access to /dev/camX is not permitted. When the streaming is turned off, the lock is released. or 2b. A stillcam app is started. Then similar to 2a, but the locking is reversed. I think that this kind of thing would keep life simple. As I understand what Hans is envisioning, it is pretty much along the same lines, too. It would mean, of course, that the way that libgphoto2 would access one of these cameras would be directly to access the /dev/camX provided by the kernel, and not to use libusb. But that can be done, I think. As I mentioned before, Hans has written several libgphoto2 drivers for digital picture frames which are otherwise seen as USB mass storage devices. Something similar would have to be done with dual-mode cameras. I will send a second reply to this message, which deals in particular with the list of abilities you outlined above. The point is, the situation as to that list of abilities is more chaotic than is generally realized. And when people are laying plans they really need to be aware of that. Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Sun, 7 Aug 2011, Alan Stern wrote: On Sun, 7 Aug 2011, Adam Baker wrote: I've addec Hans de Geode and linux-usb to the CC as this response picks up on a related discussion about the usb mini summit. On Friday 05 August 2011, Theodore Kilgore wrote: If you can solve the locking problem between devices in the kernel then it shouldn't matter if one of the kernel devices is the generic device that is used to support libusb. Hmmm. Perhaps not. While we are on the topic, what exactly do you mean by the generic device that is used to support libusb. Which device is that, exactly? The file drivers/usb/core/devio.c registers itself as a driver called usb_device which is used to provide all of the device drivers that live under /proc/bus/usb Let's get things correct. The driver is called usbfs, not usb_device. The things that live under /proc/bus/usb are files representing USB devices, not device drivers. This indirectly answers my question, above, about whatever device there may or may not be. What I get from this, and also from a bit of snooping around, is that there is not any dev that gets created in order to be accessed by libusb. Just an entry under /proc/bus/usb, which AFAICT is at most a pseudo-device. Thanks. So, Alan, what do you think is the best way to go about the problem? The camera can act as a stillcam or as a webcam. The problem is to provide access to both, with equal facility (and of course to lock out access to whichever action is not currently in use, if the other one is). The current situation with libusb does not cut it, as among other things it currently does only half the job and seemingly cannot address the locking problem. Hans suggests to create two explicit devices, /dev/video (as already done and something like /dev/cam. Then access webcam function as now and stillcam function with libgphoto2, as now, but through /dev/cam instead of through libusb. This would seem to me to solve all the problems, but at the expense of some work. Can you think of something more clever? Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
Hi all, On 08/04/2011 02:34 PM, Mauro Carvalho Chehab wrote: Em 03-08-2011 20:20, Theodore Kilgore escreveu: snip snip Yes, that kind of thing is an obvious problem. Actually, though, it may be that this had just better not happen. For some of the hardware that I know of, it could be a real problem no matter what approach would be taken. For example, certain specific dual-mode cameras will delete all data stored on the camera if the camera is fired up in webcam mode. To drop Gphoto suddenly in order to do the videoconf call would, on such cameras, result in the automatic deletion of all photos on the camera even if those photos had not yet been downloaded. Presumably, one would not want to do that. So, in other words, the Kernel driver should return -EBUSY if on such cameras, if there are photos stored on them, and someone tries to stream. Agreed. IMO, the right solution is to work on a proper snapshot mode, in kernelspace, and moving the drivers that have already a kernelspace out of Gphoto. Well, the problem with that is, a still camera and a webcam are entirely different beasts. Still photos stored in the memory of an external device, waiting to be downloaded, are not snapshots. Thus, access to those still photos is not access to snapshots. Things are not that simple. Yes, stored photos require a different API, as Hans pointed. I think that some cameras just export them as a USB storage. Erm, that is not what I tried to say, or do you mean another Hans? snip snip If I understood you well, there are 4 possible ways: 1) UVC + USB mass storage; 2) UVC + Vendor Class mass storage; 3) Vendor Class video + USB mass storage; 4) Vendor Class video + Vendor Class mass storage. Actually the cameras Theodore and I are talking about here all fall into category 4. I expect devices which do any of 1-3 to properly use different interfaces for this, actually the different class specifications mandate that they use different interfaces for this. This sounds to be a good theme for the Workshop, or even to KS/2011. Agreed, although we don't need to talk about this for very long, the solution is basically: 1) Define a still image retrieval API for v4l2 devices (there is only 1 interface for both functions on these devices, so only 1 driver, and to me it makes sense to extend the existing drivers to also do still image retrieval). 2) Modify existing kernel v4l2 drivers to provide this API 3) Write a new libgphoto driver which talks this interface (only need to do one driver since all dual mode cams will export the same API). 1) is something to discuss at the workshop. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Fri, 5 Aug 2011, Hans de Goede wrote: Hi all, On 08/04/2011 02:34 PM, Mauro Carvalho Chehab wrote: Em 03-08-2011 20:20, Theodore Kilgore escreveu: snip snip Yes, that kind of thing is an obvious problem. Actually, though, it may be that this had just better not happen. For some of the hardware that I know of, it could be a real problem no matter what approach would be taken. For example, certain specific dual-mode cameras will delete all data stored on the camera if the camera is fired up in webcam mode. To drop Gphoto suddenly in order to do the videoconf call would, on such cameras, result in the automatic deletion of all photos on the camera even if those photos had not yet been downloaded. Presumably, one would not want to do that. So, in other words, the Kernel driver should return -EBUSY if on such cameras, if there are photos stored on them, and someone tries to stream. Agreed. Here, too. Not only that, but also -EBUSY needs to be returned if streaming is being done and someone tries to download photos (cf. yesterday's exchange between me and Adam Baker, where it was definitely established that this currently leads to bad stuff happening). IMO, the right solution is to work on a proper snapshot mode, in kernelspace, and moving the drivers that have already a kernelspace out of Gphoto. Well, the problem with that is, a still camera and a webcam are entirely different beasts. Still photos stored in the memory of an external device, waiting to be downloaded, are not snapshots. Thus, access to those still photos is not access to snapshots. Things are not that simple. Yes, stored photos require a different API, as Hans pointed. I think that some cameras just export them as a USB storage. Erm, that is not what I tried to say, or do you mean another Hans? For the record, this one didn't come from me, either. :-) snip snip If I understood you well, there are 4 possible ways: 1) UVC + USB mass storage; 2) UVC + Vendor Class mass storage; 3) Vendor Class video + USB mass storage; 4) Vendor Class video + Vendor Class mass storage. Actually the cameras Theodore and I are talking about here all fall into category 4. Currently true, yes. I expect devices which do any of 1-3 to properly use different interfaces for this, actually the different class specifications mandate that they use different interfaces for this. As is well known, *everybody* obeys the class specifications, too. Always did, and always will. And Linus says that he got the original kernel from the Tooth Fairy, and because he said that we all believe him. The point being, trouble will very likely come along. I think Mauro is right at least to consider the possibility. This sounds to be a good theme for the Workshop, or even to KS/2011. Agreed, although we don't need to talk about this for very long, the solution is basically: 1) Define a still image retrieval API for v4l2 devices (there is only 1 interface for both functions on these devices, True so only 1 driver, and to me it makes sense to extend the existing drivers to also do still image retrieval). 2) Modify existing kernel v4l2 drivers to provide this API 3) Write a new libgphoto driver which talks this interface (only need to do one driver since all dual mode cams will export the same API). Yes, we pretty much agree that this is probably a good way to proceed. However, my curiosity is aroused by something that Adam mentioned yesterday. Namely If you can solve the locking problem between devices in the kernel then it shouldn't matter if one of the kernel devices is the generic device that is used to support libusb. I am not completely sure of what he meant here. I am not intimately conversant with the internals of libusb. However, is there something here which could be used constructively? Could things be set up so that, for example, the kernel module hands the generic device over to libusb? If it were possible to do things that way, it might be the most minimally disruptive approach of all, since it might not require much if any changes in libgphoto2 access to cameras. 1) is something to discuss at the workshop. Regards, Hans Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
Hi, On 08/03/2011 10:36 PM, Mauro Carvalho Chehab wrote: Em 03-08-2011 16:53, Theodore Kilgore escreveu: snip snip Mauro, Not saying that you need to change the program for this session to deal with this topic, but an old and vexing problem is dual-mode devices. It is an issue which needs some kind of unified approach, and, in my opinion, consensus about policy and methodology. As a very good example if this problem, several of the cameras that I have supported as GSPCA devices in their webcam modality are also still cameras and are supported, as still cameras, in Gphoto. This can cause a collision between driver software in userspace which functions with libusb, and on the other hand with a kernel driver which tries to grab the device. Recent attempts to deal with this problem involve the incorporation of code in libusb which disables a kernel module that has already grabbed the device, allowing the userspace driver to function. This has made life a little bit easier for some people, but not for everybody. For, the device needs to be re-plugged in order to re-activate the kernel support. But some of the user-friencly desktop setups used by some distros will automatically start up a dual-mode camera with a gphoto-based program, thereby making it impossible for the camera to be used as a webcam unless the user goes for a crash course in how to disable the feature which has been so thoughtfully (thoughtlessly?) provided. As the problem is not confined to cameras but also affects some other devices, such as DSL modems which have a partition on them and are thus seen as Mass Storage devices, perhaps it is time to try to find a systematic approach to problems like this. There are of course several possible approaches. 1. A kernel module should handle everything related to connecting up the hardware. In that case, the existing userspace driver has to be modified to use the kernel module instead of libusb. Those who support this option would say that it gets everything under the control of the kernel, where it belongs. OTOG, the possible result is to create a minor mess in projects like Gphoto. 2. The kernel module should be abolished, and all of its functionality moved to userspace. This would of course involve difficulties approximately equivalent to item 1. An advantage, in the eyes of some, would be to cut down on the yet-another-driver-for-yet-another-piece-of-peculiar-hardware syndrome which obviously contributes to an in principle unlimited increase in the size of the kernel codebase. A disadvantage would be that it would create some disruption in webcam support. 3. A further modification to libusb reactivates the kernel module automatically, as soon as the userspace app which wanted to access the device through a libusb-based driver library is closed. This seems attractive, but it has certain deficiencies as well. One of them is that it can not necessarily provide a smooth and informative user experience, since circumstances can occur in which something appears to go wrong, but the user gets no clear message saying what the problem is. In other words, it is a patchwork solution which only slightly refines the current patchwork solution in libusb, which is in itself only a slight improvement on the original, unaddressed problem. 4. ??? Several people are interested in this problem, but not much progress has been made at this time. I think that the topic ought to be put somehow on the front burner so that lots of people will try to think of the best way to handle it. Many eyes, and all that. Not saying change your schedule, as I said. Have a nice conference. I wish I could attend. But I do hope by this message to raise some general concern about this problem. That's an interesting issue. A solution like (3) is a little bit out of scope, as it is a pure userspace (or a mixed userspace USB stack) solution. Technically speaking, letting the same device being handled by either an userspace or a kernelspace driver doesn't seem smart to me, due to: - Duplicated efforts to maintain both drivers; - It is hard to sync a kernel driver with an userspace driver, as you've pointed. So, we're between (1) or (2). Moving the solution entirely to userspace will have, additionally, the problem of having two applications trying to access the same hardware using two different userspace instances (for example, an incoming videoconf call while Gphoto is opened, assuming that such videoconf call would also have an userspace driver). IMO, the right solution is to work on a proper snapshot mode, in kernelspace, and moving the drivers that have already a kernelspace out of Gphoto. I agree that solution 1) so all the driver bits in kernelspace is the right solution. This is unrelated to snapshot mode though, snapshot mode is all about taking live snapshots. Where as in this case we are downloading pictures which have already been taken (perhaps days ago) from device memory. What we need
Re: [Workshop-2011] Media Subsystem Workshop 2011
Em 04-08-2011 08:39, Hans de Goede escreveu: Hi, On 08/03/2011 10:36 PM, Mauro Carvalho Chehab wrote: Em 03-08-2011 16:53, Theodore Kilgore escreveu: snip snip Mauro, Not saying that you need to change the program for this session to deal with this topic, but an old and vexing problem is dual-mode devices. It is an issue which needs some kind of unified approach, and, in my opinion, consensus about policy and methodology. As a very good example if this problem, several of the cameras that I have supported as GSPCA devices in their webcam modality are also still cameras and are supported, as still cameras, in Gphoto. This can cause a collision between driver software in userspace which functions with libusb, and on the other hand with a kernel driver which tries to grab the device. Recent attempts to deal with this problem involve the incorporation of code in libusb which disables a kernel module that has already grabbed the device, allowing the userspace driver to function. This has made life a little bit easier for some people, but not for everybody. For, the device needs to be re-plugged in order to re-activate the kernel support. But some of the user-friencly desktop setups used by some distros will automatically start up a dual-mode camera with a gphoto-based program, thereby making it impossible for the camera to be used as a webcam unless the user goes for a crash course in how to disable the feature which has been so thoughtfully (thoughtlessly?) provided. As the problem is not confined to cameras but also affects some other devices, such as DSL modems which have a partition on them and are thus seen as Mass Storage devices, perhaps it is time to try to find a systematic approach to problems like this. There are of course several possible approaches. 1. A kernel module should handle everything related to connecting up the hardware. In that case, the existing userspace driver has to be modified to use the kernel module instead of libusb. Those who support this option would say that it gets everything under the control of the kernel, where it belongs. OTOG, the possible result is to create a minor mess in projects like Gphoto. 2. The kernel module should be abolished, and all of its functionality moved to userspace. This would of course involve difficulties approximately equivalent to item 1. An advantage, in the eyes of some, would be to cut down on the yet-another-driver-for-yet-another-piece-of-peculiar-hardware syndrome which obviously contributes to an in principle unlimited increase in the size of the kernel codebase. A disadvantage would be that it would create some disruption in webcam support. 3. A further modification to libusb reactivates the kernel module automatically, as soon as the userspace app which wanted to access the device through a libusb-based driver library is closed. This seems attractive, but it has certain deficiencies as well. One of them is that it can not necessarily provide a smooth and informative user experience, since circumstances can occur in which something appears to go wrong, but the user gets no clear message saying what the problem is. In other words, it is a patchwork solution which only slightly refines the current patchwork solution in libusb, which is in itself only a slight improvement on the original, unaddressed problem. 4. ??? Several people are interested in this problem, but not much progress has been made at this time. I think that the topic ought to be put somehow on the front burner so that lots of people will try to think of the best way to handle it. Many eyes, and all that. Not saying change your schedule, as I said. Have a nice conference. I wish I could attend. But I do hope by this message to raise some general concern about this problem. That's an interesting issue. A solution like (3) is a little bit out of scope, as it is a pure userspace (or a mixed userspace USB stack) solution. Technically speaking, letting the same device being handled by either an userspace or a kernelspace driver doesn't seem smart to me, due to: - Duplicated efforts to maintain both drivers; - It is hard to sync a kernel driver with an userspace driver, as you've pointed. So, we're between (1) or (2). Moving the solution entirely to userspace will have, additionally, the problem of having two applications trying to access the same hardware using two different userspace instances (for example, an incoming videoconf call while Gphoto is opened, assuming that such videoconf call would also have an userspace driver). IMO, the right solution is to work on a proper snapshot mode, in kernelspace, and moving the drivers that have already a kernelspace out of Gphoto. I agree that solution 1) so all the driver bits in kernelspace is the right solution. This is unrelated to snapshot mode though, snapshot mode is all about taking live snapshots. Where as in this
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Thu, 04 Aug 2011 09:40:18 -0300 Mauro Carvalho Chehab mche...@redhat.com wrote: What we need for this is a simple API (new v4l ioctl's I guess) for the stillcam mode of these dual mode cameras (stillcam + webcam). So that the webcam drivers can grow code to also allow access to the stored pictures, which were taken in standalone (iow not connected to usb) stillcam mode. This API does not need to be terribly complex. AFAIK all of the currently supported dual cam cameras don't have filenames only picture numbers, so the API could consist of a simple, get highest picture nr, is picture X present (some slots may contain deleted pictures), get picture X, delete picture X, delete all API. That sounds to work. I would map it on a way close to the controls API (or like the DVB FE_[GET|SET]_PROPERTY API), as this would make easier to expand it in the future, if we start to see webcams with file names or other things like that. I did not follow all the thread, but I was wondering about an other solution: what about offering both USB mass storage and webcam accesses? When a dual-mode webcam is plugged in, the driver creates two devices, the video device /dev/videox and the volume /dev/sdx. When the webcam is opened, the volume cannot be mounted. When the volume is mounted, the webcam cannot be opened. There is no need for a specific API. As Mauro said: For those, we may eventually need some sort of locking between the USB storage and V4L. That's all. By where am I wrong? -- Ken ar c'hentañ | ** Breizh ha Linux atav! ** Jef | http://moinejf.free.fr/ -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
(re-adding all from the original CC-list, please, don't drop anyone) On Thu, 4 Aug 2011, Jean-Francois Moine wrote: On Thu, 04 Aug 2011 09:40:18 -0300 Mauro Carvalho Chehab mche...@redhat.com wrote: What we need for this is a simple API (new v4l ioctl's I guess) for the stillcam mode of these dual mode cameras (stillcam + webcam). So that the webcam drivers can grow code to also allow access to the stored pictures, which were taken in standalone (iow not connected to usb) stillcam mode. This API does not need to be terribly complex. AFAIK all of the currently supported dual cam cameras don't have filenames only picture numbers, so the API could consist of a simple, get highest picture nr, is picture X present (some slots may contain deleted pictures), get picture X, delete picture X, delete all API. That sounds to work. I would map it on a way close to the controls API (or like the DVB FE_[GET|SET]_PROPERTY API), as this would make easier to expand it in the future, if we start to see webcams with file names or other things like that. I did not follow all the thread, but I was wondering about an other solution: what about offering both USB mass storage and webcam accesses? When a dual-mode webcam is plugged in, the driver creates two devices, the video device /dev/videox and the volume /dev/sdx. When the webcam is opened, the volume cannot be mounted. When the volume is mounted, the webcam cannot be opened. There is no need for a specific API. As Mauro said: For those, we may eventually need some sort of locking between the USB storage and V4L. That's all. By where am I wrong? That'd also be my understanding. There are already several standard ways to access data on still cameras: mass-storage, PTP, MTP, why invent Yet Another One? Just learn to share a device between several existing drivers. Thanks Guennadi --- Guennadi Liakhovetski, Ph.D. Freelance Open-Source Software Developer http://www.open-technology.de/ -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Thursday 04 August 2011, Theodore Kilgore wrote: As far as I know, /dev/sdx signifies a device which is accessible by something like the USB mass storage protocols, at the very least. So, if that fits the camera, fine. But most of the cameras in question are Class Proprietary. Thus, not in any way standard mass storage devices. Then it is probably better not to call the new device by that name unless that name really fits. Probably, it would be better to have /dev/cam or /dev/stillcam, or something like that. Correct and that is why this idea doesn't work - /dev/sdx needs to be a block device that can have a file system on it. These cameras don't have a traditional file system and there is a lot of code in gphoto to support all the different types of camera. There does exist the possibility of a relatively simple fix - If libusb include a usb_reattach_kernel_driver_np call to go with the usb_detach_kernel_driver_np then once gphoto had finished with the device it could restore the kernel driver and webcam mode would work. Unfortunately the libusb devs don't want to support it in the 0.1 version of libusb that everyone uses and the reattach function needs knowledge of libusb internals to work reliably. I did come up with a hack that sort of worked but I never worked out how to clean it up to be acceptable to go upstream. http://old.nabble.com/Re-attaching-USB-kernel-drivers-detached-by-libgphoto2- td22978838.html http://libusb.6.n5.nabble.com/re-attaching-after-usb-detach-kernel-driver-np- td6068.html Adam Baker -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Thursday 04 August 2011, Mauro Carvalho Chehab wrote: That'd also be my understanding. There are already several standard ways to access data on still cameras: mass-storage, PTP, MTP, why invent Yet Another One? Just learn to share a device between several existing drivers. For those that can export data into some fs-like way, this may be the better way. It seems that gvfs does something like that. I've no idea how easy or difficult would be to write Kernel driver for it. As I understand it gvfs uses libgphoto2 and fuse and it is the interface libghoto2 that is the problem. libgphoto2 contains lots of the same sort of code to handle strange data formats from the camera as libv4l so I don't think we want to be moving that code back into the kernel.(The old out of kernel driver for sq905 before Theodore and I rewrote it contained code to do Bayer decoding and gamma correction that was copied from libgphoto2). Regards Adam Baker -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
Em 04-08-2011 18:38, Adam Baker escreveu: On Thursday 04 August 2011, Mauro Carvalho Chehab wrote: That'd also be my understanding. There are already several standard ways to access data on still cameras: mass-storage, PTP, MTP, why invent Yet Another One? Just learn to share a device between several existing drivers. For those that can export data into some fs-like way, this may be the better way. It seems that gvfs does something like that. I've no idea how easy or difficult would be to write Kernel driver for it. As I understand it gvfs uses libgphoto2 and fuse and it is the interface libghoto2 that is the problem. libgphoto2 contains lots of the same sort of code to handle strange data formats from the camera as libv4l so I don't think we want to be moving that code back into the kernel.(The old out of kernel driver for sq905 before Theodore and I rewrote it contained code to do Bayer decoding and gamma correction that was copied from libgphoto2). I don't think we should move the entire libgphoto2 to kernel. For sure, format conversions don't belong to Kernelspace. We just need to move the file handling (e. g. PTP/MTP) to a kernel driver. Something might be needed for libgphoto2 to know what is the format of the images inside the filesystem, but this could be just mapped as a file extension. Regards Adam Baker -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Thu, 4 Aug 2011, Adam Baker wrote: On Thursday 04 August 2011, Theodore Kilgore wrote: As far as I know, /dev/sdx signifies a device which is accessible by something like the USB mass storage protocols, at the very least. So, if that fits the camera, fine. But most of the cameras in question are Class Proprietary. Thus, not in any way standard mass storage devices. Then it is probably better not to call the new device by that name unless that name really fits. Probably, it would be better to have /dev/cam or /dev/stillcam, or something like that. Correct and that is why this idea doesn't work - /dev/sdx needs to be a block device that can have a file system on it. These cameras don't have a traditional file system and there is a lot of code in gphoto to support all the different types of camera. There does exist the possibility of a relatively simple fix - If libusb include a usb_reattach_kernel_driver_np call to go with the usb_detach_kernel_driver_np then once gphoto had finished with the device it could restore the kernel driver and webcam mode would work. Unfortunately the libusb devs don't want to support it in the 0.1 version of libusb that everyone uses and the reattach function needs knowledge of libusb internals to work reliably. I did come up with a hack that sort of worked but I never worked out how to clean it up to be acceptable to go upstream. http://old.nabble.com/Re-attaching-USB-kernel-drivers-detached-by-libgphoto2- td22978838.html http://libusb.6.n5.nabble.com/re-attaching-after-usb-detach-kernel-driver-np- td6068.html Adam Baker Adam, (without looking at the details of your code) I agree that something like fixing libusb to reattach a kernel driver would partially alleviate the immediate problem of dual-mode cameras. 1. It would provide immediate relief to those people who are afflicted with the shortsightedness of some of the user friendly distros, which have set up a rule that every camera supported by Gphoto will be opened for download of photos as soon as it ls plugged in and the result is that no dual-mode camera can be used in webcam mode -- unless the user knows how to go and fix the mess. 2. It would solve a lot of existing problems for lots of other people. Therefore, I have favored this approach myself, sometimes in the past. The problems, as I see it (partly after some education from people like Hans de Goede), are the following: 1. No locking, and no error-handling. -- What if the user is downloading photos and gets a videoconference telephone call? What if the user, just for fun, starts up a webcam app, at the same time? Well, you might say, it can't start up because the /dev/video is disabled so we are home free on that one. But then -- What if it is the other way around, and the webcam interface is active, and the user (or some idiot automated software like what I mentioned above!) decides to start up the stillcam apps? What then? Does libusb just cut off the /dev/videoX device in the middle of things? 2. This adaptation to libusb solves the specific problem of handling dual-mode hardware for which one of the modes is handled by the kernel and the other mode is handled in userspace, through libusb. The further refinement of libusb addresses only this problem, not the general problem of dual-mode or triple-mode hardware, in the case that all functionality of the hardware is addressed through the kernel. Therefore, your solution ends up being a partial cure to a general problem, not a general cure for a general problem. Further, it is much easier to solve the locking issues which arise if the basic access to the hardware is through the kernel for all of its functionality. Thus, while originally favoring your approach, my position is at this point more in the direction that something needs to be done about this at the level of the kernel. As I said, others have convinced me of this, mainly Hans, because at first I thought your way of doing it was plenty good enough. Thanks for joining the debate, Adam, even though I just gave an opinion that you don't have the most optimal solution. I think that this problem has gone on long enough, and we all need to get together and fix it. Seriously. Cheers, Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Thu, 4 Aug 2011, Adam Baker wrote: On Thursday 04 August 2011, Mauro Carvalho Chehab wrote: That'd also be my understanding. There are already several standard ways to access data on still cameras: mass-storage, PTP, MTP, why invent Yet Another One? Just learn to share a device between several existing drivers. For those that can export data into some fs-like way, this may be the better way. It seems that gvfs does something like that. I've no idea how easy or difficult would be to write Kernel driver for it. As I understand it gvfs uses libgphoto2 and fuse and it is the interface libghoto2 that is the problem. This is correct. Except that the problem is not in libgphoto2 per se, but is at an even lower level. It could be said that the problem is in libusb, because libghoto2 uses libusb. So maybe the solution is to fix up libusb. Or, as I have come recently to think, maybe not. In any event, neither use nor avoidance of gvfs has much of anything to do with the problem at hand. But the problem exists with it or without it. libgphoto2 contains lots of the same sort of code to handle strange data formats from the camera as libv4l so I don't think we want to be moving that code back into the kernel.(The old out of kernel driver for sq905 before Theodore and I rewrote it contained code to do Bayer decoding and gamma correction that was copied from libgphoto2). This is all very much true. Moreover, I do not think that anyone has the idea to put any of that kind of code back into the kernel. But, just in case that anyone is thinking of possible overlap between what is done in libv4l and libgphoto2, someone should point out that things like Bayer demosaicing and gamma correction are not necessarily done the same way in the two libraries. Why is that? Well, it is true because one of the libraries supports streaming and the other one supports still cameras. Thus, the Bayer demosaicing functions in libv4l are optimized for speed, which will directly affect the frames per second rate. The Bayer demosaicing functions in libusb are intended to process image data from still cameras. For a still camera, frame rate is irrelevant and meaningless. Therefore the priority is, or ought to be, to get the best possible image out of the downloaded image data. Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Thursday 04 August 2011, Theodore Kilgore wrote: On Thu, 4 Aug 2011, Adam Baker wrote: On Thursday 04 August 2011, Theodore Kilgore wrote: As far as I know, /dev/sdx signifies a device which is accessible by something like the USB mass storage protocols, at the very least. So, if that fits the camera, fine. But most of the cameras in question are Class Proprietary. Thus, not in any way standard mass storage devices. Then it is probably better not to call the new device by that name unless that name really fits. Probably, it would be better to have /dev/cam or /dev/stillcam, or something like that. Correct and that is why this idea doesn't work - /dev/sdx needs to be a block device that can have a file system on it. These cameras don't have a traditional file system and there is a lot of code in gphoto to support all the different types of camera. There does exist the possibility of a relatively simple fix - If libusb include a usb_reattach_kernel_driver_np call to go with the usb_detach_kernel_driver_np then once gphoto had finished with the device it could restore the kernel driver and webcam mode would work. Unfortunately the libusb devs don't want to support it in the 0.1 version of libusb that everyone uses and the reattach function needs knowledge of libusb internals to work reliably. I did come up with a hack that sort of worked but I never worked out how to clean it up to be acceptable to go upstream. http://old.nabble.com/Re-attaching-USB-kernel-drivers-detached-by-libgpho to2- td22978838.html http://libusb.6.n5.nabble.com/re-attaching-after-usb-detach-kernel-driver -np- td6068.html Adam Baker Adam, (without looking at the details of your code) I agree that something like fixing libusb to reattach a kernel driver would partially alleviate the immediate problem of dual-mode cameras. 1. It would provide immediate relief to those people who are afflicted with the shortsightedness of some of the user friendly distros, which have set up a rule that every camera supported by Gphoto will be opened for download of photos as soon as it ls plugged in and the result is that no dual-mode camera can be used in webcam mode -- unless the user knows how to go and fix the mess. 2. It would solve a lot of existing problems for lots of other people. Therefore, I have favored this approach myself, sometimes in the past. The problems, as I see it (partly after some education from people like Hans de Goede), are the following: 1. No locking, and no error-handling. -- What if the user is downloading photos and gets a videoconference telephone call? What if the user, just for fun, starts up a webcam app, at the same time? Well, you might say, it can't start up because the /dev/video is disabled so we are home free on that one. But then -- What if it is the other way around, and the webcam interface is active, and the user (or some idiot automated software like what I mentioned above!) decides to start up the stillcam apps? What then? Does libusb just cut off the /dev/videoX device in the middle of things? It does look as though there might be an issue here - the IOCTL that libusb uses calls usb_driver_release_interface in drivers/core/usb/devio.c, the definition of that function says Callers must own the device lock but as far as I can see it won't and a quick test running gphoto2 -L while streaming video does indicate it is making a severe mess of things. 2. This adaptation to libusb solves the specific problem of handling dual-mode hardware for which one of the modes is handled by the kernel and the other mode is handled in userspace, through libusb. The further refinement of libusb addresses only this problem, not the general problem of dual-mode or triple-mode hardware, in the case that all functionality of the hardware is addressed through the kernel. Therefore, your solution ends up being a partial cure to a general problem, not a general cure for a general problem. Further, it is much easier to solve the locking issues which arise if the basic access to the hardware is through the kernel for all of its functionality. If you can solve the locking problem between devices in the kernel then it shouldn't matter if one of the kernel devices is the generic device that is used to support libusb. Thus, while originally favoring your approach, my position is at this point more in the direction that something needs to be done about this at the level of the kernel. As I said, others have convinced me of this, mainly Hans, because at first I thought your way of doing it was plenty good enough. Thanks for joining the debate, Adam, even though I just gave an opinion that you don't have the most optimal solution. I think that this problem has gone on long enough, and we all need to get together and fix it. That it has gone on long enough is something I can't argue
Re: [Workshop-2011] Media Subsystem Workshop 2011
On Fri, 5 Aug 2011, Adam Baker wrote: On Thursday 04 August 2011, Theodore Kilgore wrote: On Thu, 4 Aug 2011, Adam Baker wrote: On Thursday 04 August 2011, Theodore Kilgore wrote: As far as I know, /dev/sdx signifies a device which is accessible by something like the USB mass storage protocols, at the very least. So, if that fits the camera, fine. But most of the cameras in question are Class Proprietary. Thus, not in any way standard mass storage devices. Then it is probably better not to call the new device by that name unless that name really fits. Probably, it would be better to have /dev/cam or /dev/stillcam, or something like that. Correct and that is why this idea doesn't work - /dev/sdx needs to be a block device that can have a file system on it. These cameras don't have a traditional file system and there is a lot of code in gphoto to support all the different types of camera. There does exist the possibility of a relatively simple fix - If libusb include a usb_reattach_kernel_driver_np call to go with the usb_detach_kernel_driver_np then once gphoto had finished with the device it could restore the kernel driver and webcam mode would work. Unfortunately the libusb devs don't want to support it in the 0.1 version of libusb that everyone uses and the reattach function needs knowledge of libusb internals to work reliably. I did come up with a hack that sort of worked but I never worked out how to clean it up to be acceptable to go upstream. http://old.nabble.com/Re-attaching-USB-kernel-drivers-detached-by-libgpho to2- td22978838.html http://libusb.6.n5.nabble.com/re-attaching-after-usb-detach-kernel-driver -np- td6068.html Adam Baker Adam, (without looking at the details of your code) I agree that something like fixing libusb to reattach a kernel driver would partially alleviate the immediate problem of dual-mode cameras. 1. It would provide immediate relief to those people who are afflicted with the shortsightedness of some of the user friendly distros, which have set up a rule that every camera supported by Gphoto will be opened for download of photos as soon as it ls plugged in and the result is that no dual-mode camera can be used in webcam mode -- unless the user knows how to go and fix the mess. 2. It would solve a lot of existing problems for lots of other people. Therefore, I have favored this approach myself, sometimes in the past. The problems, as I see it (partly after some education from people like Hans de Goede), are the following: 1. No locking, and no error-handling. -- What if the user is downloading photos and gets a videoconference telephone call? What if the user, just for fun, starts up a webcam app, at the same time? Well, you might say, it can't start up because the /dev/video is disabled so we are home free on that one. But then -- What if it is the other way around, and the webcam interface is active, and the user (or some idiot automated software like what I mentioned above!) decides to start up the stillcam apps? What then? Does libusb just cut off the /dev/videoX device in the middle of things? It does look as though there might be an issue here - the IOCTL that libusb uses calls usb_driver_release_interface in drivers/core/usb/devio.c, the definition of that function says Callers must own the device lock but as far as I can see it won't and a quick test running gphoto2 -L while streaming video does indicate it is making a severe mess of things. I haven't tried it myself. It is one of those things that all by myself I would never have thought of testing. But I am certainly not surprised at the result. 2. This adaptation to libusb solves the specific problem of handling dual-mode hardware for which one of the modes is handled by the kernel and the other mode is handled in userspace, through libusb. The further refinement of libusb addresses only this problem, not the general problem of dual-mode or triple-mode hardware, in the case that all functionality of the hardware is addressed through the kernel. Therefore, your solution ends up being a partial cure to a general problem, not a general cure for a general problem. Further, it is much easier to solve the locking issues which arise if the basic access to the hardware is through the kernel for all of its functionality. If you can solve the locking problem between devices in the kernel then it shouldn't matter if one of the kernel devices is the generic device that is used to support libusb. Hmmm. Perhaps not. While we are on the topic, what exactly do you mean by the generic device that is used to support libusb. Which device is that, exactly? Thus, while originally favoring your approach, my position is at this point more in the direction that something needs to be done
