Re: [RFC] How to pass camera Orientation to userspace
Actually, this happens and is happening! I got OT. I just needed to vent! Sorry for the spam. For the sensor mounting, I think the cam knows how the sensor is mounted and therefor the driver knows (driver - first abstraction layer between software and hardware). Therefor the drive has to report the sensor orientation to user space (read only). Thomas PS: May be, I am OT all ready. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Tue, 24 Feb 2009, Mauro Carvalho Chehab wrote: On Mon, 23 Feb 2009, kilg...@banach.math.auburn.edu wrote: big snip Theodore, You're considering just one subset of the V4L usages: notebook webcams. Actually, the sq905 cameras are not notebook webcams. They are cheap, consumer entry level dual-mode cameras. They can be used as hand-held still cameras, to shoot still photos, and they can also be used as webcams. When sold, they usually came with some kind of mounting device that could hold them rigidly for webcam use. There are lots of similar cameras. Mercifully, not all of those others have the problems of the sq905, which have led to the present impasse. If you think about non-notebook webcams [1] or about security cams, you'll see that the only way for you to have the flipping information is inside some userspace file. However, there are obvious differences. For those cameras the question might well come up about how to control the movement of the camera, or, at least, to be aware of which way the camera is pointing. For these, the topic is an inherent property of the particular model of the camera -- or a defect, if someone wants to say so. Since the property is not determined by USB number, it is inherently impossible outside of the module to create a table which contains the needed information. My intention here was to re-focus attention on the original problem which brought up the current discussion, and to that end the problem must be clearly understood. To have the flipping information inside some userspace file might solve some other problem and may be a generally very good idea. But it will not, can not, and never will be able to solve this problem. For example, I have here one video input bttv board with 16 cameras, connected on 4 bttv chips. I have also a Pelco camera, that has just one support socket. Depending on the place you mount it, the camera has to be rotated by 180 degrees. It can also be useful to mount it rotated by 90 degrees. Good. So one needs external controls and userspace tools. Did I ever say that such things should never be done? No. All I said was that there is a problem, presently on the table, and those kinds of things are not, can not be, never were, and never will be solutions for _this_ problem. After mounting the cameras, no matter what apps you are using (a streaming video app to broadcast it via internet, a security app, etc), the rotation information for that input, on that particular PCI, bus won't change. As we've standardized the VIDIOC_QUERYCAP bus_info, now, the information for the camera at input 3 of bttv hardware at PCI addres 03:00.3 is unique. It is also unique the same information for a notebook mounted webcam (since the USB bus ID won't change for that devices). Errrm... Again, the cameras in question here are not notebook mounted webcams. So, if we standardize where this information is stored, for example, at /etc/libv4l and/or at ~/.libv4lrc, and let libv4l handle such info, this will be something consistent for all applications using libv4l. Other apps that might not want to trust on libv4l can also read the info at the same file. Sorry, this will not work here. It may solve some other problem, but not this problem. Or, if one wants to store the information there, I don't care, really, but then there needs to be a way to get the information from the module, where it is, and get written into said table, which is where you want it, and this needs to happen every time an sq905 camera gets plugged in -- without pestering the user about the matter every time that such a camera gets hooked up. Comparison: I have tossed a coin. Is it going to come up heads? Or tails? It is possible to know which, because the coin has been tossed. It would not even be cheating to look at it, or allow someone who did look at it, to pass to us the information. But we are not going to look, and if someone tells us we will not listen because we have not agreed on what language to use for communication. Instead, we will put a guess about the outcome into a table. We will make it a nice table, which can be revised using nice GUI tools, so it is easy for the user. So if our guess is wrong let the user fix it. Then next time we toss the coin the table entry will be right because either it was right before, or now someone fixed it??? So, I really think that this should be part of the approach. I was not even addressing what should or should not be part of the approach to some other problems. My point was that such discussion is not germane to the problem of how to pass on the correct orientation of the sensor, for the sq905 cameras. There are lots of other problems out there to solve. No denying that. Also an overview is often very helpful. Also trying to visualize what might be needed in the future is helpful. All of this can be extremely helpful. But not everyone can see or
Re: [RFC] How to pass camera Orientation to userspace
On Tue, 24 Feb 2009, kilg...@banach.math.auburn.edu wrote: On Tue, 24 Feb 2009, Mauro Carvalho Chehab wrote: On Mon, 23 Feb 2009, kilg...@banach.math.auburn.edu wrote: big snip Theodore, You're considering just one subset of the V4L usages: notebook webcams. Actually, the sq905 cameras are not notebook webcams. They are cheap, consumer entry level dual-mode cameras. They can be used as hand-held still cameras, to shoot still photos, and they can also be used as webcams. When sold, they usually came with some kind of mounting device that could hold them rigidly for webcam use. There are lots of similar cameras. Mercifully, not all of those others have the problems of the sq905, which have led to the present impasse. If you think about non-notebook webcams [1] or about security cams, you'll see that the only way for you to have the flipping information is inside some userspace file. However, there are obvious differences. For those cameras the question might well come up about how to control the movement of the camera, or, at least, to be aware of which way the camera is pointing. For these, the topic is an inherent property of the particular model of the camera -- or a defect, if someone wants to say so. Since the property is not determined by USB number, it is inherently impossible outside of the module to create a table which contains the needed information. My intention here was to re-focus attention on the original problem which brought up the current discussion, and to that end the problem must be clearly understood. To have the flipping information inside some userspace file might solve some other problem and may be a generally very good idea. But it will not, can not, and never will be able to solve this problem. For example, I have here one video input bttv board with 16 cameras, connected on 4 bttv chips. I have also a Pelco camera, that has just one support socket. Depending on the place you mount it, the camera has to be rotated by 180 degrees. It can also be useful to mount it rotated by 90 degrees. Good. So one needs external controls and userspace tools. Did I ever say that such things should never be done? No. All I said was that there is a problem, presently on the table, and those kinds of things are not, can not be, never were, and never will be solutions for _this_ problem. After mounting the cameras, no matter what apps you are using (a streaming video app to broadcast it via internet, a security app, etc), the rotation information for that input, on that particular PCI, bus won't change. As we've standardized the VIDIOC_QUERYCAP bus_info, now, the information for the camera at input 3 of bttv hardware at PCI addres 03:00.3 is unique. It is also unique the same information for a notebook mounted webcam (since the USB bus ID won't change for that devices). Errrm... Again, the cameras in question here are not notebook mounted webcams. So, if we standardize where this information is stored, for example, at /etc/libv4l and/or at ~/.libv4lrc, and let libv4l handle such info, this will be something consistent for all applications using libv4l. Other apps that might not want to trust on libv4l can also read the info at the same file. Sorry, this will not work here. It may solve some other problem, but not this problem. Or, if one wants to store the information there, I don't care, really, but then there needs to be a way to get the information from the module, where it is, and get written into said table, which is where you want it, and this needs to happen every time an sq905 camera gets plugged in -- without pestering the user about the matter every time that such a camera gets hooked up. Comparison: I have tossed a coin. Is it going to come up heads? Or tails? It is possible to know which, because the coin has been tossed. It would not even be cheating to look at it, or allow someone who did look at it, to pass to us the information. But we are not going to look, and if someone tells us we will not listen because we have not agreed on what language to use for communication. Instead, we will put a guess about the outcome into a table. We will make it a nice table, which can be revised using nice GUI tools, so it is easy for the user. So if our guess is wrong let the user fix it. Then next time we toss the coin the table entry will be right because either it was right before, or now someone fixed it??? So, I really think that this should be part of the approach. I was not even addressing what should or should not be part of the approach to some other problems. My point was that such discussion is not germane to the problem of how to pass on the correct orientation of the sensor, for the sq905 cameras. There are lots of other problems out there to solve. No denying that. Also an overview is often very helpful. Also trying to visualize what might be needed in the future is
Re: [RFC] How to pass camera Orientation to userspace
On Tue, 24 Feb 2009, Mauro Carvalho Chehab wrote: On Tue, 24 Feb 2009, kilg...@banach.math.auburn.edu wrote: On Tue, 24 Feb 2009, Mauro Carvalho Chehab wrote: On Mon, 23 Feb 2009, kilg...@banach.math.auburn.edu wrote: big snip Theodore, You're considering just one subset of the V4L usages: notebook webcams. Actually, the sq905 cameras are not notebook webcams. They are cheap, consumer entry level dual-mode cameras. They can be used as hand-held still cameras, to shoot still photos, and they can also be used as webcams. When sold, they usually came with some kind of mounting device that could hold them rigidly for webcam use. There are lots of similar cameras. Mercifully, not all of those others have the problems of the sq905, which have led to the present impasse. If you think about non-notebook webcams [1] or about security cams, you'll see that the only way for you to have the flipping information is inside some userspace file. However, there are obvious differences. For those cameras the question might well come up about how to control the movement of the camera, or, at least, to be aware of which way the camera is pointing. For these, the topic is an inherent property of the particular model of the camera -- or a defect, if someone wants to say so. Since the property is not determined by USB number, it is inherently impossible outside of the module to create a table which contains the needed information. My intention here was to re-focus attention on the original problem which brought up the current discussion, and to that end the problem must be clearly understood. To have the flipping information inside some userspace file might solve some other problem and may be a generally very good idea. But it will not, can not, and never will be able to solve this problem. For example, I have here one video input bttv board with 16 cameras, connected on 4 bttv chips. I have also a Pelco camera, that has just one support socket. Depending on the place you mount it, the camera has to be rotated by 180 degrees. It can also be useful to mount it rotated by 90 degrees. Good. So one needs external controls and userspace tools. Did I ever say that such things should never be done? No. All I said was that there is a problem, presently on the table, and those kinds of things are not, can not be, never were, and never will be solutions for _this_ problem. After mounting the cameras, no matter what apps you are using (a streaming video app to broadcast it via internet, a security app, etc), the rotation information for that input, on that particular PCI, bus won't change. As we've standardized the VIDIOC_QUERYCAP bus_info, now, the information for the camera at input 3 of bttv hardware at PCI addres 03:00.3 is unique. It is also unique the same information for a notebook mounted webcam (since the USB bus ID won't change for that devices). Errrm... Again, the cameras in question here are not notebook mounted webcams. So, if we standardize where this information is stored, for example, at /etc/libv4l and/or at ~/.libv4lrc, and let libv4l handle such info, this will be something consistent for all applications using libv4l. Other apps that might not want to trust on libv4l can also read the info at the same file. Sorry, this will not work here. It may solve some other problem, but not this problem. Or, if one wants to store the information there, I don't care, really, but then there needs to be a way to get the information from the module, where it is, and get written into said table, which is where you want it, and this needs to happen every time an sq905 camera gets plugged in -- without pestering the user about the matter every time that such a camera gets hooked up. Comparison: I have tossed a coin. Is it going to come up heads? Or tails? It is possible to know which, because the coin has been tossed. It would not even be cheating to look at it, or allow someone who did look at it, to pass to us the information. But we are not going to look, and if someone tells us we will not listen because we have not agreed on what language to use for communication. Instead, we will put a guess about the outcome into a table. We will make it a nice table, which can be revised using nice GUI tools, so it is easy for the user. So if our guess is wrong let the user fix it. Then next time we toss the coin the table entry will be right because either it was right before, or now someone fixed it??? So, I really think that this should be part of the approach. I was not even addressing what should or should not be part of the approach to some other problems. My point was that such discussion is not germane to the problem of how to pass on the correct orientation of the sensor, for the sq905 cameras. There are lots of other problems out there to solve. No denying that. Also an overview is often very helpful. Also trying to
Re: [RFC] How to pass camera Orientation to userspace
Also an overview is often very helpful. Also trying to visualize what might be needed in the future is helpful. All of this can be extremely helpful. But not everyone can see or imagine every possible thing. For example, it seems that some of the best minds in the business are stunned when confronted with the fact that some manufacturer of cheap electronics in Taiwan has produced a lot of mass-market cameras with the sensors turned upside down, along with some other cameras having the same USB ID with different sensors, which act a bit differently. Clearly, if such a thing happened once it can happen again. So how to deal with such a problem? Actually, this happens and is happening! Just step back a get an other view. These consumer products are manly produced for the Windoz audience. After introduction of Win XP the consumer where told that USB device will run out of the box in Win XP, which is sometimes true, but . But on all (Windowz) Webcams (are Linux Webcams available?) I buy, I find a sticker which tells me to first insert the driver CD before connecting the cam to the PC. When you do, like instructed, your cam works like you expected! Evan the USB ID is the same like the other webcam from the other vendor, you are (more or less) forced to install the driver from this particular vendor, you get a new driver! Doesn't matter if the sensor is mounted upside down, the new driver takes care about this. So, it looks like the cam in the Windowz World just works because you were forced to install the driver from the CD. So I guess the Windoz diver just knows more then the USB ID. In the Linux World most of the drive are re-engineered, we don't know how to detect how the sensor is mounted, do we? Actually, what I try to say, is that only the cam can know how the sensor is mounted. Thus, the kernel module has to provide this information to user space (by query the hardware). The pivot is an other thing. Thomas -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Wed, 25 Feb 2009, Thomas Kaiser wrote: Also an overview is often very helpful. Also trying to visualize what might be needed in the future is helpful. All of this can be extremely helpful. But not everyone can see or imagine every possible thing. For example, it seems that some of the best minds in the business are stunned when confronted with the fact that some manufacturer of cheap electronics in Taiwan has produced a lot of mass-market cameras with the sensors turned upside down, along with some other cameras having the same USB ID with different sensors, which act a bit differently. Clearly, if such a thing happened once it can happen again. So how to deal with such a problem? Actually, this happens and is happening! Just step back a get an other view. I had plenty of other views. SQ905, SQ905C, MR97310, SN9C2028, and one or two more. And, oh yes, the GT8290 chip on which Grandtech went broke, which was intended for mass storage still cameras but had a 32-bit CBW instead of a 31-bit CBW (off by one error and nobody caught it). These consumer products are manly produced for the Windoz audience. Very true. Usually, it even says on the package that it will not work on Mac. But since Linux is unmentionable they did not say anything about it. So we make it work anyway. After introduction of Win XP the consumer where told that USB device will run out of the box in Win XP, Not all devices. Perhaps what they really meant is now you don't need to install two drivers if you have two Mass Storage Transparent Scsi Bulk Transport flash drives. To that extent, it seems that they were truthful. There are lots of proprietary devices out there for which AFAICT the drivers are still not included in Windows. Some of these are unfinished projects of mine, too, such as the JL2005C cameras. The big bugbear with those is the compression algorithm. It is a horror. which is sometimes true, but . But on all (Windowz) Webcams (are Linux Webcams available?) I buy, I find a sticker which tells me to first insert the driver CD before connecting the cam to the PC. When you do, like instructed, your cam works like you expected! Of course. Hardware will not work without a driver. Evan the USB ID is the same like the other webcam from the other vendor, you are (more or less) forced to install the driver from this particular vendor, you get a new driver! Not true. Windows, even back in the days of Windows 98, searched for the Vendor:Product number to look for the driver. There were several places to search for the number. For example, the INF files. And I have a box full of SQ905 cameras for which I will personally guarantee that every one of them will work on Windows 98 with the Windows driver from the CD that came with any other one. Now, if the picture is always facing in the right direction, that is another question, naturally. Doesn't matter if the sensor is mounted upside down, the new driver takes care about this. So, it looks like the cam in the Windowz World just works because you were forced to install the driver from the CD. And while we are on that topic, I could definitely assert that not always did the manufacturer get this right, either. I, recall, developed the original SQ905 stillcam driver for Linux. I got letters from users and sample photos, too, where the photos came out of the camera and lettering on a sign visible in the photo was bass ackwards. They were cheap cameras, and the people who sold some of them were just a little bit sloppy. Their virtue, as cheap cameras go, was comparatively good optics in some of them. I have seen lots of cameras about the same price, which had much more features and much worse pictures. So I guess the Windoz diver just knows more then the USB ID. No, not really. See above. In the Linux World most of the drive are re-engineered, we don't know how to detect how the sensor is mounted, do we? Well, yes, we do. And that is what this discussion was about. How to use that knowledge constructively while writing a kernel driver. Actually, what I try to say, is that only the cam can know how the sensor is mounted. Thus, the kernel module has to provide this information to user space (by query the hardware). Well, that is more like it. Yes, one does have to ask the camera. But the camera will tell its answer truthfully. And this is before any streaming and image processing has started, too. What a deal. The pivot is an other thing. Very true. And worth paying attention to. But it is not the same issue. Thomas Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Wed, 25 Feb 2009, Mauro Carvalho Chehab wrote: On Tue, 24 Feb 2009 20:12:00 -0600 (CST) kilg...@banach.math.auburn.edu wrote: For sure we need to have a way for retrieving this information for devices like the sq905 cameras, where the information can't be currently be determined by userspace. Yes, indeed. Except for just one word, currently. It does not fit here. Yes, it fits, since, after implementing the API, the userspace will have this information by using the agreed API. If so, IMO, the better approach is to use a flag at the v4l2_input, as already discussed in this thread. OK. As it seems to be a consensus that the better is to use a flag inside v4l2_input, could you please provide us an RFC patch to implement it and update the V4L2 spec? With respect, I think that it would be better to consult with some of the other people who are most closely involved in the issue. This would include Jean-Francois (one obviously does not want to come up with some scheme which would interfere with other camera drivers, and he has a good overview on that) and Hans de Goede (v4l will use the new method) and Adam Baker. I will be happy to participate in discussions with them and work something up. Thank you very much for helping to come to an apparent consensus about what kind of characteristics that the solution has to have. I appreciate your attention to the issue, and at this point I hope for a good and a fairly quick resolution. Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
really big snip So, what do these two deep questions, which confound the assembled wisdom of an entire list of Linux video developers, have to do with tables in userspace? None that I can see, unless someone wants to provide a mechanism for the information, having been collected in the module, to be available to the table in userspace. I'm not saying that userspace tables would solve all problems. I'm just saying that this should be part of the solution. For sure we need to have a way for retrieving this information for devices like the sq905 cameras, where the information can't be currently be determined by userspace. In the case of sq905, this information is static, right? If so, IMO, the better approach is to use a flag at the v4l2_input, as already discussed in this thread. Yes we all seem to agree on this. Adam, since you started this thread can you write a small RFC with that solution worked out with proposed videodev2.h changes? Note that I'm only talking about input flags for the orientation problem and not the pivotting problem. I think the pivotting problem may need some more discussion. But since we all seem to be in agreement wrt to orientation problem and specifically the sq905 problem, lets do one more RFC, then everyone does a +1 to that and we move forward with this as a solution for the orientation problem. Regards, Hans p.s. For the pivotting problem I'm tending towards a special control class which contains read-only controls which are really camera properties. This will allow us to cope with any granularity of pivoting sensors. This could then also be used for in example aperture. But lets start a new thread for that. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
Trent Piepho wrote: On Mon, 23 Feb 2009, Hans de Goede wrote: Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. First, I'd like to say I agree with most that the installed orientation of the camera sensor really is a different concept than the current value of a gravity sensor. It's not necessary, and maybe not even desirable, to handle them in the same way. I do not see the advantage of using reserved bits instead of controls. The are a limited number of reserved bits. In some structures there are only a few left. They will run out. Then what? Packing non-standard sensor attributes and camera sensor meta-data into a few reserved bits is not a sustainable policy. Controls on the other card are not limited and won't run out. Yes but these things are *not* controls, end of discussion. The control API is for controls, not to stuff all kind of cruft in. All kind of cruft belongs in the reserved bits of whatever field it can be stuffed in? Not whatever field, these are input properties which happen to also be pretty binary so putting them in the input flags field makes plenty of sense. What is the difference? Why does it matter? Performance? Maintenance? Is there something that's not possible? I do not find end of discussion to be a very convincing argument. Well they are not controls, that is the difference, the control interface is for controls (and only for controls, end of discussion if you ask me). These are not controls but properties, they do not have a default min and max value, Camera pivot sensor ranges from 0 to 270. How is that not a min and max? they have only one *unchanging* value, there is nothing the application can Camera sensors don't have an unchanging value. And who says scan order can't change? Suppose the camera returns raw bayer format data top to bottom, but if you request yuv then an image processing section needs to kick in and that returns the data bottom to top. Yes, because hardware designers like throwing away lots of transistors to memory so they are going to put memory in the controller to buffer an entire frame and then scan out the memory buffer in different order then the sensor gave them the data, so they cannot do FIFO, so they will actually need 2 frames of memory. If the sensor is soldered upside down on the PCB that is a very much unchanging value, and an input property if you ask me. So new proposal: use 2 bits in the input flags to indicate if the input is hardwired vflipped and/or hflipped. Create a new class of controls for querying possible changing camera properties like pivoting and aperture. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Sat, 21 Feb 2009 12:53:57 +0100 Hans Verkuil hverk...@xs4all.nl wrote: Hi Adam, Sorry for the late reply, it's been very busy. Me too. 1) Reuse the existing HFLIP and VFLIP controls, marking them as read-only Pros : No change needed to videodev2.h Cons: It is confusing to have controls that have a subtly different meaning if they are read only. Existing apps that support those controls might get confused. Would require polling to support the case of a camera being turned toward / away from the user while streaming. Reusing an existing control for a different meaning seems wrong. What happens when some cam has the capability of doing hardware flipping, and have the cam flipped? 2) Introduce a new orientation control (possibly in a new CAMERA_PROPERTIES class) Pros: libv4l can easily tell if the driver supports the control. Cons: It is really a property, not a control so calling it a control is wrong. Controls add lots of overhead in terms of driver code. Would require polling to support the case of a camera being turned toward / away from the user while streaming. I think this could be a good idea, but I agree with Hans Verkuil comments: since this is a characteristics for a given input, using a control here would mean that the driver will return it based on the selected input. Seems a little messy. Also, a mounted characteristics of the device, is not really a sensor, as Hans de Goede pointed. 3) Use an extra couple of bits in V4L2_BUF_FLAGS Pros: Simple to implement. Can change per frame without needing polling. Cons: Doesn't work for non libv4l apps that try to use the read() interface. Can't easily identify drivers that don't support it (does 0 mean not rotated or just not implemented). Can only be read when streaming (does that matter?) I think that matters, yes. I don't think this is a good idea. The metadata at the frame polling is meant to return the stream info. We shouldn't mix sensor position here. 4) Use some reserved bits from the v4l2_capability structure Pros: Less overhead than controls. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Can't easily identify drivers that don't support it. 5) Use some reserved bits from the v4l2_input structure (or possibly the status word but that is normally only valid for current input) Pros: Less overhead than controls. Could support multiple sensors in one camera if such a beast exists. What does exist is devices with a video input (e.g. composite) and a camera input: each input will have different flags. Since these vflip/hflip properties do not change they can be enumerated in advance and you know what each input supports. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Polling applies only to the bits that tell the orientation of the camera. See below for a discussion of this. Analog tv does polling for signal strength, since userspace apps do mute and stops presenting video, if the signal is too weak. IMO, a similar mechanism should be used by pivoting. IMO, this would be better addressed as a property of v4l2_input. So, I think that (5) is better than (4). Can't easily identify drivers that don't support it. Not too difficult to add through the use of a capability bit. Either in v4l2_input or (perhaps) v4l2_capability. Another Pro is that this approach will also work for v4l2_output in the case of, say, rotated LCD displays. Using camera orientation bits in v4l2_buffer while capturing will work, but using similar bits for output will fail since the data is going in the wrong direction. The interest in detecting if a driver provides this informnation is to allow libv4l to know when it should use the driver provided information and when it should use its internal table (which needs to be retained for backward compatibility). With no detection capability the driver provided info should be ignored for USB IDs in the built in table. Thoughts please There is a case that we should think: at libv4l, we may need to override the default orientation, by a custom one. For example: Surveillance systems have cameras mounted on a fixed position. Depending on the camera, and the desired position, some cameras may needed to be mounted rotated (the same case also applies to some embedded hardware like ATM machines, where a webcam maybe mounted with 180 degrees, due to hardware constraints). Ok, this is nothing that kernel needs to handle, but, at userspace, we need to have a file where the user could edit and store the camera position, to override whatever we have in kernel. Is polling bad in this case? It is not something that needs immediate attention IMHO. The overhead for checking once every X seconds is quite low. Furthermore, it is only needed on devices that cannot do v/hflipping in
Re: [RFC] How to pass camera Orientation to userspace
On Monday 23 February 2009, Mauro Carvalho Chehab wrote: On Sat, 21 Feb 2009 12:53:57 +0100 Hans Verkuil hverk...@xs4all.nl wrote: Hi Adam, Sorry for the late reply, it's been very busy. Me too. 1) Reuse the existing HFLIP and VFLIP controls, marking them as read-only Pros : No change needed to videodev2.h Cons: It is confusing to have controls that have a subtly different meaning if they are read only. Existing apps that support those controls might get confused. Would require polling to support the case of a camera being turned toward / away from the user while streaming. Reusing an existing control for a different meaning seems wrong. What happens when some cam has the capability of doing hardware flipping, and have the cam flipped? I thought that case had already been agreed, implement the flip controls but set the flip flag in the hardware to the opposite of what the control says. 4) Use some reserved bits from the v4l2_capability structure Pros: Less overhead than controls. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Can't easily identify drivers that don't support it. 5) Use some reserved bits from the v4l2_input structure (or possibly the status word but that is normally only valid for current input) Pros: Less overhead than controls. Could support multiple sensors in one camera if such a beast exists. What does exist is devices with a video input (e.g. composite) and a camera input: each input will have different flags. Since these vflip/hflip properties do not change they can be enumerated in advance and you know what each input supports. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Polling applies only to the bits that tell the orientation of the camera. See below for a discussion of this. Analog tv does polling for signal strength, since userspace apps do mute and stops presenting video, if the signal is too weak. IMO, a similar mechanism should be used by pivoting. IMO, this would be better addressed as a property of v4l2_input. So, I think that (5) is better than (4). Can't easily identify drivers that don't support it. Not too difficult to add through the use of a capability bit. Either in v4l2_input or (perhaps) v4l2_capability. Another Pro is that this approach will also work for v4l2_output in the case of, say, rotated LCD displays. Using camera orientation bits in v4l2_buffer while capturing will work, but using similar bits for output will fail since the data is going in the wrong direction. The interest in detecting if a driver provides this informnation is to allow libv4l to know when it should use the driver provided information and when it should use its internal table (which needs to be retained for backward compatibility). With no detection capability the driver provided info should be ignored for USB IDs in the built in table. Thoughts please There is a case that we should think: at libv4l, we may need to override the default orientation, by a custom one. For example: Surveillance systems have cameras mounted on a fixed position. Depending on the camera, and the desired position, some cameras may needed to be mounted rotated (the same case also applies to some embedded hardware like ATM machines, where a webcam maybe mounted with 180 degrees, due to hardware constraints). Agreed, Hans de Geode pointed out the similar case that 2 laptops may use the same camera but one mount it upside down so hardware info unrelated to the camera indicates the orientation. Ok, this is nothing that kernel needs to handle, but, at userspace, we need to have a file where the user could edit and store the camera position, to override whatever we have in kernel. Unfortunately what that doesn't address is the problem that first started this discussion. A camera where the orientation information is contained in the USB messages from the camera so the driver is the only thing that can reasonably access it. Note also that for sensor orientation I doubt that 90 or 270 degrees rotation will be seen but I do know that the case of data being flipped on just one axis does exist. Is polling bad in this case? It is not something that needs immediate attention IMHO. The overhead for checking once every X seconds is quite low. Furthermore, it is only needed on devices that cannot do v/hflipping in hardware. An alternative is to put some effort in a proper event interface. There is one implemented in include/linux/dvb/video.h and used by ivtv for video decoding. The idea is that the application registers events it wants to receive, and whenever such an event arrives the select() call will exit with a high-prio event (exception). The application then checks what happened. The video.h
Re: [RFC] How to pass camera Orientation to userspace
On Mon, 23 Feb 2009, Adam Baker wrote: On Monday 23 February 2009, Mauro Carvalho Chehab wrote: On Sat, 21 Feb 2009 12:53:57 +0100 Hans Verkuil hverk...@xs4all.nl wrote: Hi Adam, Sorry for the late reply, it's been very busy. Me too. big snip The interest in detecting if a driver provides this informnation is to allow libv4l to know when it should use the driver provided information and when it should use its internal table (which needs to be retained for backward compatibility). With no detection capability the driver provided info should be ignored for USB IDs in the built in table. snip Ok, this is nothing that kernel needs to handle, but, at userspace, we need to have a file where the user could edit and store the camera position, to override whatever we have in kernel. Unfortunately what that doesn't address is the problem that first started this discussion. A camera where the orientation information is contained in the USB messages from the camera so the driver is the only thing that can reasonably access it. Alas, this is so true. What started the entire discussion about passing the info about sensor orientation is a set of cameras all of which have the same Vendor:Product ID but require different handling of the data (vflip and hflip, or just vflip) depending upon information which can only be obtained by communication with the camera, *not* by just knowing its Vendor:Product ID. Therefore, it is a little bit disheartening to see discussions -- again -- which come back to some kind of internal table in V4L, or which come back to things like have a file where the user could edit and store the camera position, to override whatever we have in kernel. Repeating the obvious, which apparently still needs to be repeated because not all of the participants in the discussion get it: 1. The internal table in V4L could not handle this problem, because the internal table would be based upon what information? The USB Vendor:Product number? For that matter, no other table could work, either. Well, actually, a table could work. It would have to be inside the module supporting the camera, and the matter of which table entry corresponds to what camera would have to be settled by passing a USB command to the camera and then parsing the camera's response. So now the question is how to get the information out of the module, which can only be collected and analysed inside the module. 2. The have a file where the user could edit idea may seem attractive to some, because it shoves the whole problem of agreeing on the appropriate way to get needed information out of a kernel module and into userspace onto someone not present during the current discussion, the user. However, this is not a solution, and thinking about it just a little bit ought to make that totally obvious. This is a strongly worded statement, so I will proceed to explain why the matter is so obvious. Let us assume the very best, and assume that every app which is V4L conformant has a one time initialization step and creates a directory $HOME/.app containing stored settings. So we might have a file called $HOME/.app/sq905. This file gets automatically written when the hapless user hooks up his sq905 camera the first time, and has to go through a choice routine to decide which side of the frame is up and which side is to the left and to the right. One could even take serious steps to make this otherwise unnecessary and silly sequence to be as really nice and user friendly as it could possibly be, and set this all up to be done with a sequence of mouse clicks and then the file, very kindly, gets written automatically. Those of you who are thinking that this file a user could edit is the way to go are, presumably, thinking along these lines. Well there are at least four things which are obviously wrong with this solution: 2.1. The user is forced to deal with something which the user should not even have to confront. The user is called upon to remedy an omission and a deficiency which was ignored at a lower level, because a bunch of developers could not come together on a reasonable course of action. Well, some don't like to see this one, so there are three more reasons. 2.2 Every camera is going to require a file for itself in $HOME/.app even if there is nothing that the user needs to do. Many of the supported cameras need nothing of the kind, so this would be kind of silly. 2.3 The user has two apps for dealing with webcams. So now the user needs to have another directory called $HOME/.app2 with similar files in it? 2.4 (and this one is the worst of all) The user has two cameras which are both powered by the same kernel module, and the two cameras need two different things done. Now what??? Both cameras can not simultaneously have a valid entry in $HOME/.app/sq905 which tells what to do with the data out of the camera. Because what is right for one of them is wrong
Re: [RFC] How to pass camera Orientation to userspace
Hans Verkuil wrote: Hi Adam, Sorry for the late reply, it's been very busy. Same here :) On Wednesday 18 February 2009 01:30:52 Adam Baker wrote: (linux-omap included in distribution as lots of omap systems include cameras so this could be relevant there.) Background A number of the webcams now supported by v4l have sensors that are mounted upside down. Up to now this has been handled by having a table in libv4l of the USB IDs of affected cameras. This approach however fails to address two known cases (and probably more as yet unknown ones) where the USB ID is insufficient to determine the sensor orientation. In one of those cases (SQ-905) USB commands must be issued to the camera at probe time) to determine what sensor is fitted and in the other case (Genesys gl860) the camera can be pointed towards or away from the user and it swaps orientation when it is changed. It is possible that there are cameras that can use gravity sensors or similar to report how they are being held but such user driven orientation which may be intended for creative effect should probably be separated from this hardware related issue. Yes, I strongly agree with this. +1 3) Use an extra couple of bits in V4L2_BUF_FLAGS Pros: Simple to implement. Can change per frame without needing polling. Cons: Doesn't work for non libv4l apps that try to use the read() interface. Can't easily identify drivers that don't support it (does 0 mean not rotated or just not implemented). Can only be read when streaming (does that matter?) I think that matters, yes. I too can see it being usefull to get the orientation when not streaming. 4) Use some reserved bits from the v4l2_capability structure Pros: Less overhead than controls. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Can't easily identify drivers that don't support it. 5) Use some reserved bits from the v4l2_input structure (or possibly the status word but that is normally only valid for current input) Pros: Less overhead than controls. Could support multiple sensors in one camera if such a beast exists. What does exist is devices with a video input (e.g. composite) and a camera input: each input will have different flags. Since these vflip/hflip properties do not change they can be enumerated in advance and you know what each input supports. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Polling applies only to the bits that tell the orientation of the camera. See below for a discussion of this. Didn't we agree to separate orietation (as in sensor mounted upside down) and the user being able to rotate the camera (which i believe we called pivotting). For the orientation case, which is the case with the sq905X, we do not need to poll, as for pivotting, here is my proposal: Have both orientation and pivotting flags in the input flags. The pivotting flags will indicate the pivotting state at the moment of doing the ioctl (which may change later), so these indeed may need to be polled, unlike the orientation flags which will never change. Can't easily identify drivers that don't support it. Not too difficult to add through the use of a capability bit. Either in v4l2_input or (perhaps) v4l2_capability. +1 Another Pro is that this approach will also work for v4l2_output in the case of, say, rotated LCD displays. Using camera orientation bits in v4l2_buffer while capturing will work, but using similar bits for output will fail since the data is going in the wrong direction. The interest in detecting if a driver provides this informnation is to allow libv4l to know when it should use the driver provided information and when it should use its internal table (which needs to be retained for backward compatibility). With no detection capability the driver provided info should be ignored for USB IDs in the built in table. Thoughts please Is polling bad in this case? It is not something that needs immediate attention IMHO. The overhead for checking once every X seconds is quite low. Agreed, but it still is something which should be avoided if possible, implementing polling also means adding a lot of IMHO unneeded code on the userspace side. I would prefer to make the realtime pivotting state available to the app by adding flags containing the pivotting state at frame capture to the v4l2_buf flags. But if people dislike this, libv4l can simple poll the input now and then. Furthermore, it is only needed on devices that cannot do v/hflipping in hardware. Erm, no, since we decided we want to differentiate between sensor orientation and camera pivotting, I think the driver should not automagically do v/hflipping in the pivot case, this should be left up to userspace. Maybe the user actually wants to have an upside down picture ? An alternative is to put some effort in a
Re: [RFC] How to pass camera Orientation to userspace
On Sunday 22 February 2009 12:17:58 Hans de Goede wrote: Hans Verkuil wrote: Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Polling applies only to the bits that tell the orientation of the camera. See below for a discussion of this. Didn't we agree to separate orietation (as in sensor mounted upside down) and the user being able to rotate the camera (which i believe we called pivotting). Yes, I wrote this more as a clarification since I thought that the original text didn't make this distinction clearly enough. For the orientation case, which is the case with the sq905X, we do not need to poll, as for pivotting, here is my proposal: Have both orientation and pivotting flags in the input flags. The pivotting flags will indicate the pivotting state at the moment of doing the ioctl (which may change later), so these indeed may need to be polled, unlike the orientation flags which will never change. Ack. I propose to take one of the reserved fields from v4l2_input and either turn it into two __u16 fields (capabilities and flags) or one __u32 field (flags). I'm not sure whether the alignment of two __u16 fields will be done correctly on 32 and 64 bit systems since v4l2_input cannot change in size. Something to test first. Can't easily identify drivers that don't support it. Not too difficult to add through the use of a capability bit. Either in v4l2_input or (perhaps) v4l2_capability. +1 Another Pro is that this approach will also work for v4l2_output in the case of, say, rotated LCD displays. Using camera orientation bits in v4l2_buffer while capturing will work, but using similar bits for output will fail since the data is going in the wrong direction. The interest in detecting if a driver provides this informnation is to allow libv4l to know when it should use the driver provided information and when it should use its internal table (which needs to be retained for backward compatibility). With no detection capability the driver provided info should be ignored for USB IDs in the built in table. Thoughts please Is polling bad in this case? It is not something that needs immediate attention IMHO. The overhead for checking once every X seconds is quite low. Agreed, but it still is something which should be avoided if possible, implementing polling also means adding a lot of IMHO unneeded code on the userspace side. I would prefer to make the realtime pivotting state available to the app by adding flags containing the pivotting state at frame capture to the v4l2_buf flags. But if people dislike this, libv4l can simple poll the input now and then. I think this should be prototyped. Compare polling vs. putting the state in the v4l2_buf flags and see if that makes a lot of difference in the user experience compared to polling once a second. If it clearly improves things for the user, then I have no objections to adding bits to v4l2_buf. Furthermore, it is only needed on devices that cannot do v/hflipping in hardware. Erm, no, since we decided we want to differentiate between sensor orientation and camera pivotting, I think the driver should not automagically do v/hflipping in the pivot case, this should be left up to userspace. Maybe the user actually wants to have an upside down picture ? Sorry, you're right. I'd forgotten that. An alternative is to put some effort in a proper event interface. There is one implemented in include/linux/dvb/video.h and used by ivtv for video decoding. The idea is that the application registers events it wants to receive, and whenever such an event arrives the select() call will exit with a high-prio event (exception). The application then checks what happened. No not another event interface please. Once could argue that we should export the pivotting info through the generic linux input system, but not a v4l specific event interface please. Actually I think making the pivotting sensor available through the generic input system in addition to making it available through input flags would be nice to have. Just like we need to make all those take a picture now buttons on webcams available through the generic input system. Agreed. I'm not up to date when it comes to the state of event interfaces in linux, and this is a solution for the longer term anyway. I guess someone needs to pick this up and research it to see what works best. Regards, Hans -- Hans Verkuil - video4linux developer - sponsored by TANDBERG -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
Hans Verkuil wrote: On Sunday 22 February 2009 12:17:58 Hans de Goede wrote: snipped a large part we agree upon (hurray!) Agreed, but it still is something which should be avoided if possible, implementing polling also means adding a lot of IMHO unneeded code on the userspace side. I would prefer to make the realtime pivotting state available to the app by adding flags containing the pivotting state at frame capture to the v4l2_buf flags. But if people dislike this, libv4l can simple poll the input now and then. I think this should be prototyped. Compare polling vs. putting the state in the v4l2_buf flags and see if that makes a lot of difference in the user experience compared to polling once a second. If it clearly improves things for the user, then I have no objections to adding bits to v4l2_buf. This is were the difficulties start, doing this 1 per second requires making clock() calls and comparing timestamps (always tricky to get right). We could do this once every X frames, but framerates can vary wildly with webcams. My biggest concern is not the user experience, but the fact that needing to poll uglyfies the userspace code. However thinking about this more, I guess we could just poll every frame. That makes the polling code simple and should give a good user experience. snip of more things we agree up on Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
kilg...@banach.math.auburn.edu wrote: snip Hans and Adam, I am not sure how it fits into the above discussion, but perhaps it is relevant to point out that flags can be toggled. Here is what I mean: Suppose that we have two flags 01 and 10 (i.e. 2), and 01 signifies VFLIP and 10 signifies HFLIP. Then for an ordinary camera in ordinary position these are initialized as 00. If the ordinary camera is turned in some funny way (and it is possible to know that) then one or both of these flags gets turned off. But if it is a funny camera like some of the SQ905s the initial values are 1 and 1, because the sensor is in fact mounted upside down. Now, suppose that there is some camera in the future which, just like this, has the sensor upside down, and suppose that said hypothetical camera also has the ability to know that it has been turned over so what was upside down is now right side up. Well, all that one has to do is to flip the two bits from whatever they were to have instead the opposite values! Observe that this would take care of the orientation problem both for cameras which had the sensor mounted right in the first place, and for cameras which had the sensor mounted wrong in the first place. Just use the same two bits to describe the sensor orientation, and if there is any reason (based upon some ability to know that the camera orientation is now different) that the orientation should change, then just flip the bits as appropriate. Then it would be the job of the support module to provide proper initial values only for these bits, and everything else could be done later on, in userspace. Theodore Kilgore Theodore, We want to be able to differentiate between a cam which has its sensor mounted upside down, and a cam which can be pivotted and happens to be upside down at the moment, in case of any upside down mounted sensor, we will always want to compentsate, in case of a pivotting camera wether we compensate or not could be a user preference. So in you example of an upside down mounted sensor in a pivotting encasing and the encasing is pivotted 180 degrees we would have the hflip and vflip bits set for sensor orientation and we would have the pivotted 180 degrees bit set. If the user has choosen to compensate for pivotting the default, we would do nothing. But it is important to be able to differentiate between the 2. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Sun, 22 Feb 2009, Hans de Goede wrote: kilg...@banach.math.auburn.edu wrote: snip Hans and Adam, I am not sure how it fits into the above discussion, but perhaps it is relevant to point out that flags can be toggled. Here is what I mean: Suppose that we have two flags 01 and 10 (i.e. 2), and 01 signifies VFLIP and 10 signifies HFLIP. Then for an ordinary camera in ordinary position these are initialized as 00. If the ordinary camera is turned in some funny way (and it is possible to know that) then one or both of these flags gets turned off. But if it is a funny camera like some of the SQ905s the initial values are 1 and 1, because the sensor is in fact mounted upside down. Now, suppose that there is some camera in the future which, just like this, has the sensor upside down, and suppose that said hypothetical camera also has the ability to know that it has been turned over so what was upside down is now right side up. Well, all that one has to do is to flip the two bits from whatever they were to have instead the opposite values! Observe that this would take care of the orientation problem both for cameras which had the sensor mounted right in the first place, and for cameras which had the sensor mounted wrong in the first place. Just use the same two bits to describe the sensor orientation, and if there is any reason (based upon some ability to know that the camera orientation is now different) that the orientation should change, then just flip the bits as appropriate. Then it would be the job of the support module to provide proper initial values only for these bits, and everything else could be done later on, in userspace. Theodore Kilgore Theodore, We want to be able to differentiate between a cam which has its sensor mounted upside down, and a cam which can be pivotted and happens to be upside down at the moment, in case of any upside down mounted sensor, we will always want to compentsate, in case of a pivotting camera wether we compensate or not could be a user preference. So in you example of an upside down mounted sensor in a pivotting encasing and the encasing is pivotted 180 degrees we would have the hflip and vflip bits set for sensor orientation and we would have the pivotted 180 degrees bit set. If the user has choosen to compensate for pivotting the default, we would do nothing. But it is important to be able to differentiate between the 2. Hans, I am not sure if we are talking past each other, or what. But I was pointing out that the initial values of two bits can indicate the default orientation of the sensor, and this can be done permanently in the module, which transmits the initial setting of those two bits to anything up the line which is interested or curious to know those initial values. The information in those two bits will definitely tell whether the sensor is mounted upside down in the camera. For example, if it is mounted upside down, then they are both set in the module to on and exported therefrom. But if the sensor is mounted correctly, then both of them are set to off and similarly exported. Now if any application for any reason (such as knowing that the camera is upside down or is pointing in the opposite direction, or into a mirror) wants to change the defaults, all it has to do is to toggle the bits. But, hmmm. Perhaps there is the question about how the app knows that the camera is upside down or is pointed in another direction. If the camera has a gyroscope inside, for example, then it could be the camera which needs to tell to the app about the current orientation, or else the app would not have any way to know ... Is this the problem, then? For that kind of thing, one might need more than two bits in order to pass the needed information. Incidentally, I obviously agree about the need to pass the initial sensor orientation. So anything I say about this should be taken in that context. Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Sunday 22 February 2009, Hans de Goede wrote: We want to be able to differentiate between a cam which has its sensor mounted upside down, and a cam which can be pivotted and happens to be upside down at the moment, in case of any upside down mounted sensor, we will always want to compentsate, in case of a pivotting camera wether we compensate or not could be a user preference. If we take Olivier Lorin's gl-860 case though, how do we define what is the normal orientation and what is pivoted, it is likely we'd just decide the direction where the sensor output is the right way up is normal and the other is pivoted and then what info have you got from having multiple flags. In order to explain what I mean it is probably best to refer to rotations in terms of pitch, yaw and roll (as per the definitions at http://en.wikipedia.org/wiki/Flight_dynamics) where the forward direction is the shooting direction. When still cameras are fitted with gravity sensors they are normally set up with the intention of measuring roll and will often get confused by 90 degrees of pitch. If a laptop is fitted with a camera that can either record the user or the view looking away from the user then the camera needs to be able to either pitch or yaw but not roll. If the camera yaws then no correction is needed but if it pitches then the resulting image needs rotating to be the correct way up (and in that scenario it is improbable that the user doesn't want the correction applied). Other than the fact that one needs correcting and the other doesn't these options appear identical to the user and so manufacturers provide one or the other but not both. If a video camera had a roll sensor (or even, as a believe some specialist tripods can manage, a full set of roll, pitch and yaw measurements) then a substantially different mechanism is needed to provide access to that data but in the absence of anyone having access to such equipment I don't think we can design the interface now. Adam -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
kilg...@banach.math.auburn.edu wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: kilg...@banach.math.auburn.edu wrote: snip Hans and Adam, I am not sure how it fits into the above discussion, but perhaps it is relevant to point out that flags can be toggled. Here is what I mean: Suppose that we have two flags 01 and 10 (i.e. 2), and 01 signifies VFLIP and 10 signifies HFLIP. Then for an ordinary camera in ordinary position these are initialized as 00. If the ordinary camera is turned in some funny way (and it is possible to know that) then one or both of these flags gets turned off. But if it is a funny camera like some of the SQ905s the initial values are 1 and 1, because the sensor is in fact mounted upside down. Now, suppose that there is some camera in the future which, just like this, has the sensor upside down, and suppose that said hypothetical camera also has the ability to know that it has been turned over so what was upside down is now right side up. Well, all that one has to do is to flip the two bits from whatever they were to have instead the opposite values! Observe that this would take care of the orientation problem both for cameras which had the sensor mounted right in the first place, and for cameras which had the sensor mounted wrong in the first place. Just use the same two bits to describe the sensor orientation, and if there is any reason (based upon some ability to know that the camera orientation is now different) that the orientation should change, then just flip the bits as appropriate. Then it would be the job of the support module to provide proper initial values only for these bits, and everything else could be done later on, in userspace. Theodore Kilgore Theodore, We want to be able to differentiate between a cam which has its sensor mounted upside down, and a cam which can be pivotted and happens to be upside down at the moment, in case of any upside down mounted sensor, we will always want to compentsate, in case of a pivotting camera wether we compensate or not could be a user preference. So in you example of an upside down mounted sensor in a pivotting encasing and the encasing is pivotted 180 degrees we would have the hflip and vflip bits set for sensor orientation and we would have the pivotted 180 degrees bit set. If the user has choosen to compensate for pivotting the default, we would do nothing. But it is important to be able to differentiate between the 2. Hans, I am not sure if we are talking past each other, or what. But I was pointing out that the initial values of two bits can indicate the default orientation of the sensor, and this can be done permanently in the module, which transmits the initial setting of those two bits to anything up the line which is interested or curious to know those initial values. The information in those two bits will definitely tell whether the sensor is mounted upside down in the camera. For example, if it is mounted upside down, then they are both set in the module to on and exported therefrom. But if the sensor is mounted correctly, then both of them are set to off and similarly exported. Now if any application for any reason (such as knowing that the camera is upside down or is pointing in the opposite direction, or into a mirror) wants to change the defaults, all it has to do is to toggle the bits. But, hmmm. Perhaps there is the question about how the app knows that the camera is upside down or is pointed in another direction. If the camera has a gyroscope inside, for example, then it could be the camera which needs to tell to the app about the current orientation, or else the app would not have any way to know ... Is this the problem, then? For that kind of thing, one might need more than two bits in order to pass the needed information. Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Sun, 22 Feb 2009, Hans de Goede wrote: big snip Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. Regards, Hans Well, of course the bits are in a struct which is communicates information one way from the camera to userspace. But userspace can do what it deems appropriate with those bits. My point was that if userspace wants to turn the camera upside down in software, all it has to do is to negate those bits. For that purpose, it does not matter whether the bits were originally set on (indicating that the sensor is upside down in the camera) or whether they were set off (indicating the sensor is right side up in the camera). Now, if it is a question of passing along a changing camera orientation, it is obvious that has to be done with some other mechanism. So, unless my not terribly profound observation about the ability to negate bits was confusing, we are not in disagreement. Theodore Kilgore -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. First, I'd like to say I agree with most that the installed orientation of the camera sensor really is a different concept than the current value of a gravity sensor. It's not necessary, and maybe not even desirable, to handle them in the same way. I do not see the advantage of using reserved bits instead of controls. The are a limited number of reserved bits. In some structures there are only a few left. They will run out. Then what? Packing non-standard sensor attributes and camera sensor meta-data into a few reserved bits is not a sustainable policy. Controls on the other card are not limited and won't run out. Yes but these things are *not* controls, end of discussion. The control API is for controls, not to stuff all kind of cruft in. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Sun, 22 Feb 2009, Hans de Goede wrote: Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. First, I'd like to say I agree with most that the installed orientation of the camera sensor really is a different concept than the current value of a gravity sensor. It's not necessary, and maybe not even desirable, to handle them in the same way. I do not see the advantage of using reserved bits instead of controls. The are a limited number of reserved bits. In some structures there are only a few left. They will run out. Then what? Packing non-standard sensor attributes and camera sensor meta-data into a few reserved bits is not a sustainable policy. Controls on the other card are not limited and won't run out. Currently reserved bits and previously reserved but now in use bits look exactly the same. How does an app know if the bits are valid or not? More reserved bits? The control API is already designed to allow for enumeration of controls. Reserved bits have no meta-data to document them. An app sees that some bits in the input struct which were reserved when it was written are now set. What does this tell the app? Nothing. What can the app tell the user? Nothing. The control API provides for control meta-data. We have a means of enumeration, name, min, max, step size, and various flags to tell us about a control that wasn't already handled. Does this new gravity sensor have 45 degree resolution? The control's step size can show this even if only 90 degree rotations had been considered when gravity sensor support was first added to some other driver. At the very least, the app (and various existing utilities) can provide information about the controls to the user. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Sunday 22 February 2009 23:54:42 Hans de Goede wrote: Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. First, I'd like to say I agree with most that the installed orientation of the camera sensor really is a different concept than the current value of a gravity sensor. It's not necessary, and maybe not even desirable, to handle them in the same way. I do not see the advantage of using reserved bits instead of controls. The are a limited number of reserved bits. In some structures there are only a few left. They will run out. Then what? Packing non-standard sensor attributes and camera sensor meta-data into a few reserved bits is not a sustainable policy. Controls on the other card are not limited and won't run out. Yes but these things are *not* controls, end of discussion. The control API is for controls, not to stuff all kind of cruft in. I agree, these are not controls. There is an option to use the current status field. There are enough bits free, that's not the problem. But the spec is explicit about the fact that these bits apply to the current input only, and that's not true for these new bits. We can change the spec in this regard of course, but then you have to document each bit of the status field whether it is valid for the current input only, or also if this isn't the current input. It's all a bit messy. In addition, there are 4 reserved fields here and it is the first time in a very long time that we actually need one. And after all, that's why they are there in the first place. I see three options: 1) stuff them into the status field after all. 2) take one of the reserved fields and make it a single 'flags' field. 3) take one of the reserved fields and make it a u16 capabilities and u16 flags field. Trent does have a point that we need to be careful not to add fields without a good reason. Choosing option 1 fits the bill, and the orientation also fits the 'status' name. Only the sensor mount orientation is not really a status. Although with some creative naming we might come close :-) Hmm, let's see: V4L2_IN_ST_HAS_SENSOR_INFO 0x0010 V4L2_IN_ST_SENSOR_HFLIPPED 0x0020 V4L2_IN_ST_SENSOR_VFLIPPED 0x0040 V4L2_IN_ST_HAS_PIVOT_INFO 0x1000 V4L2_IN_ST_PIVOT_0 0x V4L2_IN_ST_PIVOT_90 0x2000 V4L2_IN_ST_PIVOT_1800x4000 V4L2_IN_ST_PIVOT_2700x6000 V4L2_IN_ST_PIVOT_MSK0x6000 Actually, that's not too bad. Regards, Hans -- Hans Verkuil - video4linux developer - sponsored by TANDBERG -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. First, I'd like to say I agree with most that the installed orientation of the camera sensor really is a different concept than the current value of a gravity sensor. It's not necessary, and maybe not even desirable, to handle them in the same way. I do not see the advantage of using reserved bits instead of controls. The are a limited number of reserved bits. In some structures there are only a few left. They will run out. Then what? Packing non-standard sensor attributes and camera sensor meta-data into a few reserved bits is not a sustainable policy. Controls on the other card are not limited and won't run out. Yes but these things are *not* controls, end of discussion. The control API is for controls, not to stuff all kind of cruft in. All kind of cruft belongs in the reserved bits of whatever field it can be stuffed in? Not whatever field, these are input properties which happen to also be pretty binary so putting them in the input flags field makes plenty of sense. What is the difference? Why does it matter? Performance? Maintenance? Is there something that's not possible? I do not find end of discussion to be a very convincing argument. Well they are not controls, that is the difference, the control interface is for controls (and only for controls, end of discussion if you ask me). These are not controls but properties, they do not have a default min and max value, they have only one *unchanging* value, there is nothing the application can control / change. It has been suggested to make them readonly, but that does not fix the ugliness. A proper written v4l2 application will enumerate all the controls, and then the user will see a grayed out control saying: your cam is upside down what is there to control ? will this be a grayed out slider? or a grayed out checkbox your cam is upside down, or maybe a not grayed out dropdown: where the user can select: my sensor is upside down, I deny my sensor is upside down, I don't care my sensor is upside down, WTF is this doing in my webcam control panel?, nwod edispu si rosnes yM Do you know I have an idea, lets get rid of the S2 API for DVB and put all that in controls too. Oh, and think like standards for video formats, surely that can be a control too, and ... and, ... Yes we can stuff almost anything in a control, that does not make it a good idea. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Mon, 23 Feb 2009, Hans Verkuil wrote: On Sunday 22 February 2009 23:54:42 Hans de Goede wrote: Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. First, I'd like to say I agree with most that the installed orientation of the camera sensor really is a different concept than the current value of a gravity sensor. It's not necessary, and maybe not even desirable, to handle them in the same way. I do not see the advantage of using reserved bits instead of controls. The are a limited number of reserved bits. In some structures there are only a few left. They will run out. Then what? Packing non-standard sensor attributes and camera sensor meta-data into a few reserved bits is not a sustainable policy. Controls on the other card are not limited and won't run out. Yes but these things are *not* controls, end of discussion. The control API is for controls, not to stuff all kind of cruft in. I agree, these are not controls. There is an option to use the current status field. There are enough bits free, that's not the problem. But the spec is explicit about the fact that these bits apply to the current input only, and that's not true for these new bits. We can change the spec in this regard of course, but then you have to document each bit of the status field whether it is valid for the current input only, or also if this isn't the current input. It's all a bit messy. In addition, there are 4 reserved fields here and it is the first time in a very long time that we actually need one. And after all, that's why they are there in the first place. v4l2_capability: 5 of 32 cap bits left, 4 reserved words v4l2_fmtdesc: 31 flag bits, 4 reserved words v4l2_buffer: 22 flags bits, 1 reserved word v4l2_framebuffer: 25 cap bits, 26 flag bits, no reserved words v4l2_input: 4 reserved words v4l2_output: 4 reserved words v4l2_tuner: 27 cap bits, 4 reserved words Trent does have a point that we need to be careful not to add fields without a good reason. Choosing option 1 fits the bill, and the orientation also fits the 'status' name. Only the sensor mount orientation is not really a status. Although with some creative naming we might come close :-) Hmm, let's see: V4L2_IN_ST_HAS_SENSOR_INFO0x0010 V4L2_IN_ST_SENSOR_HFLIPPED0x0020 V4L2_IN_ST_SENSOR_VFLIPPED0x0040 V4L2_IN_ST_HAS_PIVOT_INFO 0x1000 V4L2_IN_ST_PIVOT_00x V4L2_IN_ST_PIVOT_90 0x2000 V4L2_IN_ST_PIVOT_180 0x4000 V4L2_IN_ST_PIVOT_270 0x6000 V4L2_IN_ST_PIVOT_MSK 0x6000 One of my other points what the controls include meta-data. What happens when someone has an orientation sensor with 45 degree resolution? A control would just change the step size from 90 to 45. Existing software would already know what it means. With this method you have to add: V4L2_IN_ST_HAS_PIVOT_45_INFO0x8000 V4L2_IN_ST_PIVOT_45 0x0001 V4L2_IN_ST_PIVOT_1350x00012000 V4L2_IN_ST_PIVOT_2250x00014000 V4L2_IN_ST_PIVOT_3150x00016000 V4L2_IN_ST_PIVOT_45_MSK 0x00016000 Interpreting the pivot bits becomes more fun now. Existing software can do nothing with these extra bits. It can tell the user nothing. Next, add camera aperture. f/0.7 to f/64 in both half stop and third stop increments. Getting low on bits already. Suppose I have just three or four sensors on my camera. That's not all that much. A typical digital camera has more than that. They're probably connected by something slow, like I2C. Maybe non-trivial calculations are needed to use the data, like most demod's SNR registers. Each time I query the input the driver must query and calculate the value of ALL the sensors. Even though I only can about one of them. Controls can be queried individually. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Mon, 23 Feb 2009, Hans de Goede wrote: Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. First, I'd like to say I agree with most that the installed orientation of the camera sensor really is a different concept than the current value of a gravity sensor. It's not necessary, and maybe not even desirable, to handle them in the same way. I do not see the advantage of using reserved bits instead of controls. The are a limited number of reserved bits. In some structures there are only a few left. They will run out. Then what? Packing non-standard sensor attributes and camera sensor meta-data into a few reserved bits is not a sustainable policy. Controls on the other card are not limited and won't run out. Yes but these things are *not* controls, end of discussion. The control API is for controls, not to stuff all kind of cruft in. All kind of cruft belongs in the reserved bits of whatever field it can be stuffed in? Not whatever field, these are input properties which happen to also be pretty binary so putting them in the input flags field makes plenty of sense. What is the difference? Why does it matter? Performance? Maintenance? Is there something that's not possible? I do not find end of discussion to be a very convincing argument. Well they are not controls, that is the difference, the control interface is for controls (and only for controls, end of discussion if you ask me). These are not controls but properties, they do not have a default min and max value, Camera pivot sensor ranges from 0 to 270. How is that not a min and max? they have only one *unchanging* value, there is nothing the application can Camera sensors don't have an unchanging value. And who says scan order can't change? Suppose the camera returns raw bayer format data top to bottom, but if you request yuv then an image processing section needs to kick in and that returns the data bottom to top. control / change. It has been suggested to make them readonly, but that does not fix the ugliness. A proper written v4l2 application will enumerate all the controls, and then the user will see a grayed out control saying: your cam is upside down what is there to control ? will this be a grayed out slider? or a grayed out checkbox your cam is upside down, or maybe a not grayed out dropdown: where the user can select: my sensor is upside down, I deny my sensor is upside down, I don't care my sensor is upside down, WTF is this doing in my webcam control panel?, nwod edispu si rosnes yM Why is there a read-only flag for controls if the concept is so mind blowing to users? Have there been complaints about it? Do you know I have an idea, lets get rid of the S2 API for DVB and put all that in controls too. Oh, and think like standards for video formats, surely that can be a control too, and ... and, ... Good point. The S2 API is much more like the control interface than the previous API. Enumerated attributes which can be set one at a time or in groups. More can be added. There is some meta data about them. The old API used the a limited number of fixed structs, a few reserved bits, no meta-data, and a query/set everything at once API. I think the camera meta-data and camera sensor API should look more like S2. -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
On Monday 23 February 2009 00:56:40 Trent Piepho wrote: On Mon, 23 Feb 2009, Hans Verkuil wrote: On Sunday 22 February 2009 23:54:42 Hans de Goede wrote: Trent Piepho wrote: On Sun, 22 Feb 2009, Hans de Goede wrote: Yes that is what we are talking about, the camera having a gravity switch (usually nothing as advanced as a gyroscope). Also the bits we are talking about are in a struct which communicates information one way, from the camera to userspace, so there is no way to clear the bits to make the camera do something. First, I'd like to say I agree with most that the installed orientation of the camera sensor really is a different concept than the current value of a gravity sensor. It's not necessary, and maybe not even desirable, to handle them in the same way. I do not see the advantage of using reserved bits instead of controls. The are a limited number of reserved bits. In some structures there are only a few left. They will run out. Then what? Packing non-standard sensor attributes and camera sensor meta-data into a few reserved bits is not a sustainable policy. Controls on the other card are not limited and won't run out. Yes but these things are *not* controls, end of discussion. The control API is for controls, not to stuff all kind of cruft in. I agree, these are not controls. There is an option to use the current status field. There are enough bits free, that's not the problem. But the spec is explicit about the fact that these bits apply to the current input only, and that's not true for these new bits. We can change the spec in this regard of course, but then you have to document each bit of the status field whether it is valid for the current input only, or also if this isn't the current input. It's all a bit messy. In addition, there are 4 reserved fields here and it is the first time in a very long time that we actually need one. And after all, that's why they are there in the first place. v4l2_capability: 5 of 32 cap bits left, 4 reserved words v4l2_fmtdesc: 31 flag bits, 4 reserved words v4l2_buffer: 22 flags bits, 1 reserved word v4l2_framebuffer: 25 cap bits, 26 flag bits, no reserved words v4l2_input: 4 reserved words v4l2_output: 4 reserved words v4l2_tuner: 27 cap bits, 4 reserved words Trent does have a point that we need to be careful not to add fields without a good reason. Choosing option 1 fits the bill, and the orientation also fits the 'status' name. Only the sensor mount orientation is not really a status. Although with some creative naming we might come close :-) Hmm, let's see: V4L2_IN_ST_HAS_SENSOR_INFO 0x0010 V4L2_IN_ST_SENSOR_HFLIPPED 0x0020 V4L2_IN_ST_SENSOR_VFLIPPED 0x0040 V4L2_IN_ST_HAS_PIVOT_INFO 0x1000 V4L2_IN_ST_PIVOT_0 0x V4L2_IN_ST_PIVOT_90 0x2000 V4L2_IN_ST_PIVOT_1800x4000 V4L2_IN_ST_PIVOT_2700x6000 V4L2_IN_ST_PIVOT_MSK0x6000 One of my other points what the controls include meta-data. What happens when someone has an orientation sensor with 45 degree resolution? A control would just change the step size from 90 to 45. Existing software would already know what it means. With this method you have to add: V4L2_IN_ST_HAS_PIVOT_45_INFO 0x8000 V4L2_IN_ST_PIVOT_45 0x0001 V4L2_IN_ST_PIVOT_135 0x00012000 V4L2_IN_ST_PIVOT_225 0x00014000 V4L2_IN_ST_PIVOT_315 0x00016000 V4L2_IN_ST_PIVOT_45_MSK 0x00016000 Interpreting the pivot bits becomes more fun now. Existing software can do nothing with these extra bits. It can tell the user nothing. These bits deal exclusively with the sensor position and for the sole purpose of deciding whether the image has to be rotated in hard/software to get it the right-way up. In most cases this is limited to rotating 180 degrees, but omap has some memory management tricks to do the 90/270 degree cases as well. This is not the same as having a full positioning 2D or 3D system in a camera that can give you exact angles. I do not think that should be part of v4l2. There isn't anything that v4l2 can do with that. An application might use this information to setup special transformations to do such rotates, and if the hardware can do that then that would be part of an effects API or something like that. With non-90 degree angle you just run into a totally different set of problems that are definitely out-of-scope. Next, add camera aperture. f/0.7 to f/64 in both half stop and third stop increments. Getting low on bits already. Suppose I have just three or four sensors on my camera. That's not all that much. A typical digital camera has more than that. They're probably connected by something slow, like I2C. Maybe non-trivial calculations are needed to
Re: [RFC] How to pass camera Orientation to userspace
Hi Adam, Sorry for the late reply, it's been very busy. On Wednesday 18 February 2009 01:30:52 Adam Baker wrote: (linux-omap included in distribution as lots of omap systems include cameras so this could be relevant there.) Background A number of the webcams now supported by v4l have sensors that are mounted upside down. Up to now this has been handled by having a table in libv4l of the USB IDs of affected cameras. This approach however fails to address two known cases (and probably more as yet unknown ones) where the USB ID is insufficient to determine the sensor orientation. In one of those cases (SQ-905) USB commands must be issued to the camera at probe time) to determine what sensor is fitted and in the other case (Genesys gl860) the camera can be pointed towards or away from the user and it swaps orientation when it is changed. It is possible that there are cameras that can use gravity sensors or similar to report how they are being held but such user driven orientation which may be intended for creative effect should probably be separated from this hardware related issue. Yes, I strongly agree with this. Most if not all of the cameras affected by this problem produce video formats that are not widely supported by applications. They are therefore all likely to be normally used in conjunction with libv4l to perform format conversion and so it makes sense to retain the actual flipping operation in libv4l. libv4l provides a capability via LD_PRELOAD to attach itself to unmodified binaries. It is likely that whatever solution is chosen there will end up being cases that have to be handled in libv4l as external information (such as laptop model ID) is the only mechanism to distinguish otherwise identical cameras. So far libv4l only supports the case of data needing a 180 degree rotation but there is known to exist hardware which requires a VFLIP and it is possible some hardware requires just HFLIP so all of those cases should be supported. While libv4l will be the main user of this information, it should not matter for the chosen API whether it is libv4l or another application, possibly a custom application running on an embedded system. There have been a number of inconclusive discussions on this subject on the v4l / linux-media mailing lists over the last few months offering many options for how to pass this information across which I will list below and hopefully a preferred solution can be selected from them 1) Reuse the existing HFLIP and VFLIP controls, marking them as read-only Pros : No change needed to videodev2.h Cons: It is confusing to have controls that have a subtly different meaning if they are read only. Existing apps that support those controls might get confused. Would require polling to support the case of a camera being turned toward / away from the user while streaming. 2) Introduce a new orientation control (possibly in a new CAMERA_PROPERTIES class) Pros: libv4l can easily tell if the driver supports the control. Cons: It is really a property, not a control so calling it a control is wrong. Controls add lots of overhead in terms of driver code. Would require polling to support the case of a camera being turned toward / away from the user while streaming. 3) Use an extra couple of bits in V4L2_BUF_FLAGS Pros: Simple to implement. Can change per frame without needing polling. Cons: Doesn't work for non libv4l apps that try to use the read() interface. Can't easily identify drivers that don't support it (does 0 mean not rotated or just not implemented). Can only be read when streaming (does that matter?) I think that matters, yes. 4) Use some reserved bits from the v4l2_capability structure Pros: Less overhead than controls. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Can't easily identify drivers that don't support it. 5) Use some reserved bits from the v4l2_input structure (or possibly the status word but that is normally only valid for current input) Pros: Less overhead than controls. Could support multiple sensors in one camera if such a beast exists. What does exist is devices with a video input (e.g. composite) and a camera input: each input will have different flags. Since these vflip/hflip properties do not change they can be enumerated in advance and you know what each input supports. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Polling applies only to the bits that tell the orientation of the camera. See below for a discussion of this. Can't easily identify drivers that don't support it. Not too difficult to add through the use of a capability bit. Either in v4l2_input or (perhaps) v4l2_capability. Another Pro is that this approach will also work for v4l2_output in the case of, say, rotated LCD displays. Using camera orientation bits
Re: [RFC] How to pass camera Orientation to userspace
DongSoo(Nathaniel) Kim wrote: Hello Adam, I've been thinking exactly the same issue not usb but SoC based camera. I have no idea about how usb cameras work but I am quite curious about is it really possible to make proper orientation with only querying camera driver. Because in case of SoC based camera device, many of camera ISPs are supporting VFLIP, HFLIP register on their own, and we can read current orientation by reading those registers. But the problem is ISP's registers are set as not flipped at all but it physically mounted upside down, because the H/W vendor has packed the camera module upside down. (it sounds ridiculous but happens sometimes) That happens a lot with webcams too. Given that these SoC systems will come with some board specific config anyways, all that is needed is to pass some boardconfig in to the camera driver (through platform data for example) which tells the camera driver that on this board the sensor is mounted upside down. So in that case when we query orientation of camera, it returns not flipped vertically or horizontally at all but actually it turns out to be upside down. Actually we are setting camera device to be flipped for default in that case. Ack, but the right thing to do is not to set the vflip and hflip video4linux2 controls on by default, but to invert their meaning, so when the sensor is upside down, the hflip and vflip controls as seen by the application through the v4l2 API will report not flipping, but the hwcontrols will actually be set to flipping, and when an app enables flipping at the v4l2 API level it will actually gets disables at the HW level, this way the upside downness is 100% hidden from userspace. So your problem does not need any of the new API we are working on. The new API is for when the hardware cannot flip and we need to tell userspace to correct for this in software. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
Hello Hans. I went too far. Just forgot what we can do through porting job. You are right. That should be board specific item. Cheers, Nate 2009. 02. 18, 오후 11:36, Hans de Goede 작성: DongSoo(Nathaniel) Kim wrote: Hello Adam, I've been thinking exactly the same issue not usb but SoC based camera. I have no idea about how usb cameras work but I am quite curious about is it really possible to make proper orientation with only querying camera driver. Because in case of SoC based camera device, many of camera ISPs are supporting VFLIP, HFLIP register on their own, and we can read current orientation by reading those registers. But the problem is ISP's registers are set as not flipped at all but it physically mounted upside down, because the H/W vendor has packed the camera module upside down. (it sounds ridiculous but happens sometimes) That happens a lot with webcams too. Given that these SoC systems will come with some board specific config anyways, all that is needed is to pass some boardconfig in to the camera driver (through platform data for example) which tells the camera driver that on this board the sensor is mounted upside down. So in that case when we query orientation of camera, it returns not flipped vertically or horizontally at all but actually it turns out to be upside down. Actually we are setting camera device to be flipped for default in that case. Ack, but the right thing to do is not to set the vflip and hflip video4linux2 controls on by default, but to invert their meaning, so when the sensor is upside down, the hflip and vflip controls as seen by the application through the v4l2 API will report not flipping, but the hwcontrols will actually be set to flipping, and when an app enables flipping at the v4l2 API level it will actually gets disables at the HW level, this way the upside downness is 100% hidden from userspace. So your problem does not need any of the new API we are working on. The new API is for when the hardware cannot flip and we need to tell userspace to correct for this in software. Regards, Hans -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC] How to pass camera Orientation to userspace
(linux-omap included in distribution as lots of omap systems include cameras so this could be relevant there.) Background A number of the webcams now supported by v4l have sensors that are mounted upside down. Up to now this has been handled by having a table in libv4l of the USB IDs of affected cameras. This approach however fails to address two known cases (and probably more as yet unknown ones) where the USB ID is insufficient to determine the sensor orientation. In one of those cases (SQ-905) USB commands must be issued to the camera at probe time) to determine what sensor is fitted and in the other case (Genesys gl860) the camera can be pointed towards or away from the user and it swaps orientation when it is changed. It is possible that there are cameras that can use gravity sensors or similar to report how they are being held but such user driven orientation which may be intended for creative effect should probably be separated from this hardware related issue. Most if not all of the cameras affected by this problem produce video formats that are not widely supported by applications. They are therefore all likely to be normally used in conjunction with libv4l to perform format conversion and so it makes sense to retain the actual flipping operation in libv4l. libv4l provides a capability via LD_PRELOAD to attach itself to unmodified binaries. It is likely that whatever solution is chosen there will end up being cases that have to be handled in libv4l as external information (such as laptop model ID) is the only mechanism to distinguish otherwise identical cameras. So far libv4l only supports the case of data needing a 180 degree rotation but there is known to exist hardware which requires a VFLIP and it is possible some hardware requires just HFLIP so all of those cases should be supported. There have been a number of inconclusive discussions on this subject on the v4l / linux-media mailing lists over the last few months offering many options for how to pass this information across which I will list below and hopefully a preferred solution can be selected from them 1) Reuse the existing HFLIP and VFLIP controls, marking them as read-only Pros : No change needed to videodev2.h Cons: It is confusing to have controls that have a subtly different meaning if they are read only. Existing apps that support those controls might get confused. Would require polling to support the case of a camera being turned toward / away from the user while streaming. 2) Introduce a new orientation control (possibly in a new CAMERA_PROPERTIES class) Pros: libv4l can easily tell if the driver supports the control. Cons: It is really a property, not a control so calling it a control is wrong. Controls add lots of overhead in terms of driver code. Would require polling to support the case of a camera being turned toward / away from the user while streaming. 3) Use an extra couple of bits in V4L2_BUF_FLAGS Pros: Simple to implement. Can change per frame without needing polling. Cons: Doesn't work for non libv4l apps that try to use the read() interface. Can't easily identify drivers that don't support it (does 0 mean not rotated or just not implemented). Can only be read when streaming (does that matter?) 4) Use some reserved bits from the v4l2_capability structure Pros: Less overhead than controls. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Can't easily identify drivers that don't support it. 5) Use some reserved bits from the v4l2_input structure (or possibly the status word but that is normally only valid for current input) Pros: Less overhead than controls. Could support multiple sensors in one camera if such a beast exists. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Can't easily identify drivers that don't support it. The interest in detecting if a driver provides this informnation is to allow libv4l to know when it should use the driver provided information and when it should use its internal table (which needs to be retained for backward compatibility). With no detection capability the driver provided info should be ignored for USB IDs in the built in table. Thoughts please Adam Baker -- To unsubscribe from this list: send the line unsubscribe linux-omap in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC] How to pass camera Orientation to userspace
Hello Adam, I've been thinking exactly the same issue not usb but SoC based camera. I have no idea about how usb cameras work but I am quite curious about is it really possible to make proper orientation with only querying camera driver. Because in case of SoC based camera device, many of camera ISPs are supporting VFLIP, HFLIP register on their own, and we can read current orientation by reading those registers. But the problem is ISP's registers are set as not flipped at all but it physically mounted upside down, because the H/W vendor has packed the camera module upside down. (it sounds ridiculous but happens sometimes) So in that case when we query orientation of camera, it returns not flipped vertically or horizontally at all but actually it turns out to be upside down. Actually we are setting camera device to be flipped for default in that case. I guess in USB camera case that kind of situation is not happening, but for SoC camera it should be a tough job to find out exact way of determine the orientation of camera automatically. Cheers, Nate On Wed, Feb 18, 2009 at 9:30 AM, Adam Baker li...@baker-net.org.uk wrote: (linux-omap included in distribution as lots of omap systems include cameras so this could be relevant there.) Background A number of the webcams now supported by v4l have sensors that are mounted upside down. Up to now this has been handled by having a table in libv4l of the USB IDs of affected cameras. This approach however fails to address two known cases (and probably more as yet unknown ones) where the USB ID is insufficient to determine the sensor orientation. In one of those cases (SQ-905) USB commands must be issued to the camera at probe time) to determine what sensor is fitted and in the other case (Genesys gl860) the camera can be pointed towards or away from the user and it swaps orientation when it is changed. It is possible that there are cameras that can use gravity sensors or similar to report how they are being held but such user driven orientation which may be intended for creative effect should probably be separated from this hardware related issue. Most if not all of the cameras affected by this problem produce video formats that are not widely supported by applications. They are therefore all likely to be normally used in conjunction with libv4l to perform format conversion and so it makes sense to retain the actual flipping operation in libv4l. libv4l provides a capability via LD_PRELOAD to attach itself to unmodified binaries. It is likely that whatever solution is chosen there will end up being cases that have to be handled in libv4l as external information (such as laptop model ID) is the only mechanism to distinguish otherwise identical cameras. So far libv4l only supports the case of data needing a 180 degree rotation but there is known to exist hardware which requires a VFLIP and it is possible some hardware requires just HFLIP so all of those cases should be supported. There have been a number of inconclusive discussions on this subject on the v4l / linux-media mailing lists over the last few months offering many options for how to pass this information across which I will list below and hopefully a preferred solution can be selected from them 1) Reuse the existing HFLIP and VFLIP controls, marking them as read-only Pros : No change needed to videodev2.h Cons: It is confusing to have controls that have a subtly different meaning if they are read only. Existing apps that support those controls might get confused. Would require polling to support the case of a camera being turned toward / away from the user while streaming. 2) Introduce a new orientation control (possibly in a new CAMERA_PROPERTIES class) Pros: libv4l can easily tell if the driver supports the control. Cons: It is really a property, not a control so calling it a control is wrong. Controls add lots of overhead in terms of driver code. Would require polling to support the case of a camera being turned toward / away from the user while streaming. 3) Use an extra couple of bits in V4L2_BUF_FLAGS Pros: Simple to implement. Can change per frame without needing polling. Cons: Doesn't work for non libv4l apps that try to use the read() interface. Can't easily identify drivers that don't support it (does 0 mean not rotated or just not implemented). Can only be read when streaming (does that matter?) 4) Use some reserved bits from the v4l2_capability structure Pros: Less overhead than controls. Cons: Would require polling to support the case of a camera being turned toward / away from the user while streaming. Can't easily identify drivers that don't support it. 5) Use some reserved bits from the v4l2_input structure (or possibly the status word but that is normally only valid for current input) Pros: Less overhead than controls. Could support multiple sensors in one camera if such a beast exists. Cons: Would require polling
