Re: [PATCH/RFC v3 0/7] Videobuf2 framework
On Wednesday, October 20, 2010 08:41:06 Marek Szyprowski wrote: Hello, As I promissed I continue the development of the VideoBuf2 at Samsung until Pawel finds some spare time to help us. This is a third version of the framework. Besides the minor bugfixes here and there I've added a complete read() callback emulator. This emulator provides 2 types of read() operation - 'streaming' and 'one shot'. It is suitable to replace both videobuf_read_stream() and videobuf_read_one() methods from the old videobuf. One thing I never understood: what is the point of supporting 'one shot' read mode? Why not support just streaming? Does anyone know? Another question: how hard is it to support write mode as well? I think vb2 should support both. I suspect that once we have a read emulator it isn't difficult to make a write emulator too. A last remark: the locking has changed recently in videobuf due to the work done on eliminating the BKL. It's probably a good idea to incorporate those changes as well in vb2. Regards, Hans Taking into account the size of the patches and the number of lines I've changed, I've decided to keep the Pawel signed-off attribute and authorship to correctly credit him. If this is against kernel rules, feel free to let me know. Best regards -- Marek Szyprowski Samsung Poland RD Center Changes since V2: = - added read() emulator (see patch #5/7) - fixed lack of parentheses in macro definitions (caused side effects in some places) - added a separate check for VM_READ or VM_WRITE in vb2_mmap() - added vb2_is_streaming(), vb2_lock and vb2_unlock inlines - updated vivi driver with the new read() emulator Changes since V1: = - removed drv_lock, added start_streaming and stop_streaming callbacks Here is the original Videobuf2 introduction prepared by Pawel: === These patches add a new driver framework for Video for Linux 2 driver - Videobuf2. Videobuf2 is intended as a replacement for videobuf, the current driver framework, which will be referred to as videobuf1 for the remainder of this document. What is videobuf2? Videobuf2 is a Video for Linux 2 API-compatible driver framework for multimedia devices. It acts as an intermediate layer between userspace applications and device drivers. It also provides low-level, modular memory management functions for drivers. Videobuf2 eases driver development, reduces drivers' code size and aids in proper and consistent implementation of V4L2 API in drivers. Videobuf2 memory management backend is fully modular. This allows custom memory management routines for devices and platforms with non-standard memory management requirements to be plugged in, without changing the high-level buffer management functions and API. The framework provides: - implementations of streaming I/O V4L2 ioctls and file operations - high-level video buffer, video queue and state management functions - video buffer memory allocation and management Why a new framework? There have been many discussions in the V4L2 community about the feasibility of writing a new framework, as opposed to fixing the existing one. It has been agreed though that: - videobuf1 has major flaws and an attempt to fix it would end up in rewriting most of the code - many drivers depend on videobuf1 and since the changes would be major, an effort to adapt and test them all would not be realistically possible Due to the problems with videobuf most new drivers cannot use it. This leads to code replication and overcomplicated drivers. What is wrong with videobuf1? There are many problems with the current videobuf implementation. During a V4L2 mini-summit in Helsinki in June 2010, two presentations were delivered on this topic: - Laurent Pinchart videobuf - the good, the bad and the ugly http://linuxtv.org/downloads/presentations/summit_jun_2010/20100614-v4l2_summit-videobuf.pdf - Pawel Osciak Future of the videobuf framework http://linuxtv.org/downloads/presentations/summit_jun_2010/Videobuf_Helsinki_June2010.pdf These presentations highlighted many problems with videobuf. The most prominent include: - V4L2 API violations and wrong memory management design - it is impossible to pause streaming (buffers are freed on streamoff) - VIDIOC_REQBUFS(0) does not free memory - it is impossible to reallocate memory with VIDIOC_REQBUFS - video memory is allocated on mmap, qbuf or even on page fault, freed on unmap, streamoff or explicitly by drivers - per-buffer waitqueues - not extensible enough and thus not ready for new platforms and uses, especially considering embedded multimedia
Re: [PATCH/RFC v3 0/7] Videobuf2 framework
On Wed, Oct 20, 2010 at 4:14 PM, Hans Verkuil hverk...@xs4all.nl wrote: On Wednesday, October 20, 2010 08:41:06 Marek Szyprowski wrote: Hello, As I promissed I continue the development of the VideoBuf2 at Samsung until Pawel finds some spare time to help us. This is a third version of the framework. Besides the minor bugfixes here and there I've added a complete read() callback emulator. This emulator provides 2 types of read() operation - 'streaming' and 'one shot'. It is suitable to replace both videobuf_read_stream() and videobuf_read_one() methods from the old videobuf. One thing I never understood: what is the point of supporting 'one shot' read mode? Why not support just streaming? Does anyone know? Another question: how hard is it to support write mode as well? I think vb2 should support both. I suspect that once we have a read emulator it isn't difficult to make a write emulator too. One thing consideration is that with this implementation we can't meet merge window for 2.6.37. How do you think? Thank you, Kyungmin Park A last remark: the locking has changed recently in videobuf due to the work done on eliminating the BKL. It's probably a good idea to incorporate those changes as well in vb2. Regards, Hans Taking into account the size of the patches and the number of lines I've changed, I've decided to keep the Pawel signed-off attribute and authorship to correctly credit him. If this is against kernel rules, feel free to let me know. Best regards -- Marek Szyprowski Samsung Poland RD Center Changes since V2: = - added read() emulator (see patch #5/7) - fixed lack of parentheses in macro definitions (caused side effects in some places) - added a separate check for VM_READ or VM_WRITE in vb2_mmap() - added vb2_is_streaming(), vb2_lock and vb2_unlock inlines - updated vivi driver with the new read() emulator Changes since V1: = - removed drv_lock, added start_streaming and stop_streaming callbacks Here is the original Videobuf2 introduction prepared by Pawel: === These patches add a new driver framework for Video for Linux 2 driver - Videobuf2. Videobuf2 is intended as a replacement for videobuf, the current driver framework, which will be referred to as videobuf1 for the remainder of this document. What is videobuf2? Videobuf2 is a Video for Linux 2 API-compatible driver framework for multimedia devices. It acts as an intermediate layer between userspace applications and device drivers. It also provides low-level, modular memory management functions for drivers. Videobuf2 eases driver development, reduces drivers' code size and aids in proper and consistent implementation of V4L2 API in drivers. Videobuf2 memory management backend is fully modular. This allows custom memory management routines for devices and platforms with non-standard memory management requirements to be plugged in, without changing the high-level buffer management functions and API. The framework provides: - implementations of streaming I/O V4L2 ioctls and file operations - high-level video buffer, video queue and state management functions - video buffer memory allocation and management Why a new framework? There have been many discussions in the V4L2 community about the feasibility of writing a new framework, as opposed to fixing the existing one. It has been agreed though that: - videobuf1 has major flaws and an attempt to fix it would end up in rewriting most of the code - many drivers depend on videobuf1 and since the changes would be major, an effort to adapt and test them all would not be realistically possible Due to the problems with videobuf most new drivers cannot use it. This leads to code replication and overcomplicated drivers. What is wrong with videobuf1? There are many problems with the current videobuf implementation. During a V4L2 mini-summit in Helsinki in June 2010, two presentations were delivered on this topic: - Laurent Pinchart videobuf - the good, the bad and the ugly http://linuxtv.org/downloads/presentations/summit_jun_2010/20100614-v4l2_summit-videobuf.pdf - Pawel Osciak Future of the videobuf framework http://linuxtv.org/downloads/presentations/summit_jun_2010/Videobuf_Helsinki_June2010.pdf These presentations highlighted many problems with videobuf. The most prominent include: - V4L2 API violations and wrong memory management design - it is impossible to pause streaming (buffers are freed on streamoff) - VIDIOC_REQBUFS(0) does not free memory - it is impossible to reallocate memory with VIDIOC_REQBUFS - video memory is allocated on mmap, qbuf or even on page fault,
RE: [PATCH/RFC v3 0/7] Videobuf2 framework
Hello, On Wednesday, October 20, 2010 9:15 AM wrote: On Wednesday, October 20, 2010 08:41:06 Marek Szyprowski wrote: Hello, As I promissed I continue the development of the VideoBuf2 at Samsung until Pawel finds some spare time to help us. This is a third version of the framework. Besides the minor bugfixes here and there I've added a complete read() callback emulator. This emulator provides 2 types of read() operation - 'streaming' and 'one shot'. It is suitable to replace both videobuf_read_stream() and videobuf_read_one() methods from the old videobuf. One thing I never understood: what is the point of supporting 'one shot' read mode? Why not support just streaming? Does anyone know? I can imagine that some simple cameras that capture pure JPG frames might want to use 'one shot' mode. This enables easier scripting and things like 'cat /dev/video capture.jpg' working. If you think that 'one shot' mode should be removed - let me know. Another question: how hard is it to support write mode as well? I think vb2 should support both. I suspect that once we have a read emulator it isn't difficult to make a write emulator too. Well, that's possible. If you really think that write() emulator is also required, I can implement both. This shouldn't be much work. A last remark: the locking has changed recently in videobuf due to the work done on eliminating the BKL. It's probably a good idea to incorporate those changes as well in vb2. Yes, I noticed that there are a lot of changes in for-2.6.37 staging tree, I will try to get through them and update relevant parts of vb2. The current version the vb2 patches is based on 2.6.36-rc8. Kernel tree with vb2 patches (and the required multiplane patches) will be available in a few hours on: http://git.infradead.org/users/kmpark/linux-2.6-samsung/shortlog/refs/heads/v4l/vb2 Best regards -- Marek Szyprowski Samsung Poland RD Center -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH/RFC v3 0/7] Videobuf2 framework
Hello, On Wednesday, October 20, 2010 9:15 AM wrote: On Wednesday, October 20, 2010 08:41:06 Marek Szyprowski wrote: Hello, As I promissed I continue the development of the VideoBuf2 at Samsung until Pawel finds some spare time to help us. This is a third version of the framework. Besides the minor bugfixes here and there I've added a complete read() callback emulator. This emulator provides 2 types of read() operation - 'streaming' and 'one shot'. It is suitable to replace both videobuf_read_stream() and videobuf_read_one() methods from the old videobuf. One thing I never understood: what is the point of supporting 'one shot' read mode? Why not support just streaming? Does anyone know? I can imagine that some simple cameras that capture pure JPG frames might want to use 'one shot' mode. This enables easier scripting and things like 'cat /dev/video capture.jpg' working. If you think that 'one shot' mode should be removed - let me know. I did a grep for videobuf_read_one in the sources and the only ones that use it are bttv, saa7134, zr364xx, cx88 and cx23885. AFAIK zr364xx is the only webcam driver in this set. Mauro, do you know if there are any guidelines on what drivers are supposed to use? Right now it seems random as to what drivers use. The only constant I see is that vbi and compressed video (e.g. mpeg) streaming is always using read_stream. Raw video seems to depend purely on what the driver developer chose. Another question: how hard is it to support write mode as well? I think vb2 should support both. I suspect that once we have a read emulator it isn't difficult to make a write emulator too. Well, that's possible. If you really think that write() emulator is also required, I can implement both. This shouldn't be much work. If it is not much work, then we should do that. The reason write wasn't present before is simply that few drivers supported output streaming. But that is changing these days so write support would definitely be a good idea. Regards, Hans A last remark: the locking has changed recently in videobuf due to the work done on eliminating the BKL. It's probably a good idea to incorporate those changes as well in vb2. Yes, I noticed that there are a lot of changes in for-2.6.37 staging tree, I will try to get through them and update relevant parts of vb2. The current version the vb2 patches is based on 2.6.36-rc8. Kernel tree with vb2 patches (and the required multiplane patches) will be available in a few hours on: http://git.infradead.org/users/kmpark/linux-2.6-samsung/shortlog/refs/heads/v4l/vb2 Best regards -- Marek Szyprowski Samsung Poland RD Center -- Hans Verkuil - video4linux developer - sponsored by TANDBERG, part of Cisco -- To unsubscribe from this list: send the line unsubscribe linux-media in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH/RFC v3 0/7] Videobuf2 framework
On Wed, 2010-10-20 at 16:18 +0900, Kyungmin Park wrote: On Wed, Oct 20, 2010 at 4:14 PM, Hans Verkuil hverk...@xs4all.nl wrote: On Wednesday, October 20, 2010 08:41:06 Marek Szyprowski wrote: Hello, As I promissed I continue the development of the VideoBuf2 at Samsung until Pawel finds some spare time to help us. This is a third version of the framework. Besides the minor bugfixes here and there I've added a complete read() callback emulator. This emulator provides 2 types of read() operation - 'streaming' and 'one shot'. It is suitable to replace both videobuf_read_stream() and videobuf_read_one() methods from the old videobuf. One thing I never understood: what is the point of supporting 'one shot' read mode? Why not support just streaming? Does anyone know? Another question: how hard is it to support write mode as well? I think vb2 should support both. I suspect that once we have a read emulator it isn't difficult to make a write emulator too. One thing consideration is that with this implementation we can't meet merge window for 2.6.37. How do you think? Thank you, Kyungmin Park I'll agree that we should not let requests for features not currently used by user-space impede a functional replacement from being committed. I'm assuming the original videobuf never had write() support or that no userspace apps used it. My $0.02. Regards, Andy A last remark: the locking has changed recently in videobuf due to the work done on eliminating the BKL. It's probably a good idea to incorporate those changes as well in vb2. Regards, Hans Taking into account the size of the patches and the number of lines I've changed, I've decided to keep the Pawel signed-off attribute and authorship to correctly credit him. If this is against kernel rules, feel free to let me know. Best regards -- Marek Szyprowski Samsung Poland RD Center Changes since V2: = - added read() emulator (see patch #5/7) - fixed lack of parentheses in macro definitions (caused side effects in some places) - added a separate check for VM_READ or VM_WRITE in vb2_mmap() - added vb2_is_streaming(), vb2_lock and vb2_unlock inlines - updated vivi driver with the new read() emulator Changes since V1: = - removed drv_lock, added start_streaming and stop_streaming callbacks Here is the original Videobuf2 introduction prepared by Pawel: === These patches add a new driver framework for Video for Linux 2 driver - Videobuf2. Videobuf2 is intended as a replacement for videobuf, the current driver framework, which will be referred to as videobuf1 for the remainder of this document. What is videobuf2? Videobuf2 is a Video for Linux 2 API-compatible driver framework for multimedia devices. It acts as an intermediate layer between userspace applications and device drivers. It also provides low-level, modular memory management functions for drivers. Videobuf2 eases driver development, reduces drivers' code size and aids in proper and consistent implementation of V4L2 API in drivers. Videobuf2 memory management backend is fully modular. This allows custom memory management routines for devices and platforms with non-standard memory management requirements to be plugged in, without changing the high-level buffer management functions and API. The framework provides: - implementations of streaming I/O V4L2 ioctls and file operations - high-level video buffer, video queue and state management functions - video buffer memory allocation and management Why a new framework? There have been many discussions in the V4L2 community about the feasibility of writing a new framework, as opposed to fixing the existing one. It has been agreed though that: - videobuf1 has major flaws and an attempt to fix it would end up in rewriting most of the code - many drivers depend on videobuf1 and since the changes would be major, an effort to adapt and test them all would not be realistically possible Due to the problems with videobuf most new drivers cannot use it. This leads to code replication and overcomplicated drivers. What is wrong with videobuf1? There are many problems with the current videobuf implementation. During a V4L2 mini-summit in Helsinki in June 2010, two presentations were delivered on this topic: - Laurent Pinchart videobuf - the good, the bad and the ugly http://linuxtv.org/downloads/presentations/summit_jun_2010/20100614-v4l2_summit-videobuf.pdf - Pawel Osciak Future of the videobuf framework
Re: [PATCH/RFC v3 0/7] Videobuf2 framework
On Wed, 2010-10-20 at 16:18 +0900, Kyungmin Park wrote: On Wed, Oct 20, 2010 at 4:14 PM, Hans Verkuil hverk...@xs4all.nl wrote: On Wednesday, October 20, 2010 08:41:06 Marek Szyprowski wrote: Hello, As I promissed I continue the development of the VideoBuf2 at Samsung until Pawel finds some spare time to help us. This is a third version of the framework. Besides the minor bugfixes here and there I've added a complete read() callback emulator. This emulator provides 2 types of read() operation - 'streaming' and 'one shot'. It is suitable to replace both videobuf_read_stream() and videobuf_read_one() methods from the old videobuf. One thing I never understood: what is the point of supporting 'one shot' read mode? Why not support just streaming? Does anyone know? Another question: how hard is it to support write mode as well? I think vb2 should support both. I suspect that once we have a read emulator it isn't difficult to make a write emulator too. One thing consideration is that with this implementation we can't meet merge window for 2.6.37. How do you think? Thank you, Kyungmin Park I'll agree that we should not let requests for features not currently used by user-space impede a functional replacement from being committed. I'm assuming the original videobuf never had write() support or that no userspace apps used it. My $0.02. I certainly never expected to see videobuf2 to appear in 2.6.37. I'm hoping for 2.6.38, but it requires that videobuf-dma-sg is also ported to vb2. The main reason why I am pushing for write now is that it is easy to add since Marek just implemented the read version. So he has all the knowledge needed to implement this still in his head. It always bugged me that read was supported in videobuf but not write. We can do better in videobuf2. I think that it would be a good idea (just my opinion) to merge videobuf2 early in the next 2.6.38 kernel cycle, even if dma-sg isn't implemented yet, provided we can get that missing piece in before that kernel cycle ends. I think that is definitely doable. Regards, Hans Regards, Andy A last remark: the locking has changed recently in videobuf due to the work done on eliminating the BKL. It's probably a good idea to incorporate those changes as well in vb2. Regards, Hans Taking into account the size of the patches and the number of lines I've changed, I've decided to keep the Pawel signed-off attribute and authorship to correctly credit him. If this is against kernel rules, feel free to let me know. Best regards -- Marek Szyprowski Samsung Poland RD Center Changes since V2: = - added read() emulator (see patch #5/7) - fixed lack of parentheses in macro definitions (caused side effects in some places) - added a separate check for VM_READ or VM_WRITE in vb2_mmap() - added vb2_is_streaming(), vb2_lock and vb2_unlock inlines - updated vivi driver with the new read() emulator Changes since V1: = - removed drv_lock, added start_streaming and stop_streaming callbacks Here is the original Videobuf2 introduction prepared by Pawel: === These patches add a new driver framework for Video for Linux 2 driver - Videobuf2. Videobuf2 is intended as a replacement for videobuf, the current driver framework, which will be referred to as videobuf1 for the remainder of this document. What is videobuf2? Videobuf2 is a Video for Linux 2 API-compatible driver framework for multimedia devices. It acts as an intermediate layer between userspace applications and device drivers. It also provides low-level, modular memory management functions for drivers. Videobuf2 eases driver development, reduces drivers' code size and aids in proper and consistent implementation of V4L2 API in drivers. Videobuf2 memory management backend is fully modular. This allows custom memory management routines for devices and platforms with non-standard memory management requirements to be plugged in, without changing the high-level buffer management functions and API. The framework provides: - implementations of streaming I/O V4L2 ioctls and file operations - high-level video buffer, video queue and state management functions - video buffer memory allocation and management Why a new framework? There have been many discussions in the V4L2 community about the feasibility of writing a new framework, as opposed to fixing the existing one. It has been agreed though that: - videobuf1 has major flaws and an attempt to fix it would end up in rewriting most of the code - many drivers depend on videobuf1 and since the changes would be
Re: [PATCH/RFC v3 0/7] Videobuf2 framework
On Wed, Oct 20, 2010 at 9:28 PM, Hans Verkuil hverk...@xs4all.nl wrote: On Wed, 2010-10-20 at 16:18 +0900, Kyungmin Park wrote: On Wed, Oct 20, 2010 at 4:14 PM, Hans Verkuil hverk...@xs4all.nl wrote: On Wednesday, October 20, 2010 08:41:06 Marek Szyprowski wrote: Hello, As I promissed I continue the development of the VideoBuf2 at Samsung until Pawel finds some spare time to help us. This is a third version of the framework. Besides the minor bugfixes here and there I've added a complete read() callback emulator. This emulator provides 2 types of read() operation - 'streaming' and 'one shot'. It is suitable to replace both videobuf_read_stream() and videobuf_read_one() methods from the old videobuf. One thing I never understood: what is the point of supporting 'one shot' read mode? Why not support just streaming? Does anyone know? Another question: how hard is it to support write mode as well? I think vb2 should support both. I suspect that once we have a read emulator it isn't difficult to make a write emulator too. One thing consideration is that with this implementation we can't meet merge window for 2.6.37. How do you think? Thank you, Kyungmin Park I'll agree that we should not let requests for features not currently used by user-space impede a functional replacement from being committed. I'm assuming the original videobuf never had write() support or that no userspace apps used it. My $0.02. I certainly never expected to see videobuf2 to appear in 2.6.37. I'm hoping for 2.6.38, but it requires that videobuf-dma-sg is also ported to vb2. then how about to merge it as migration plan from videobuf to videobuf2 at 2.6.37. Since there are some users and modules such as MFC depends on videobuf2. If it's merged at 2.6.38 it has to wait about 2~3 month again. and videobuf-dma-sg also have to wait until that time. I think give a chance to test videobuf2 at kernel 2.6.37 and then use it at 2.6.38 finally. Thank you, Kyungmin Park The main reason why I am pushing for write now is that it is easy to add since Marek just implemented the read version. So he has all the knowledge needed to implement this still in his head. It always bugged me that read was supported in videobuf but not write. We can do better in videobuf2. I think that it would be a good idea (just my opinion) to merge videobuf2 early in the next 2.6.38 kernel cycle, even if dma-sg isn't implemented yet, provided we can get that missing piece in before that kernel cycle ends. I think that is definitely doable. Regards, Hans Regards, Andy A last remark: the locking has changed recently in videobuf due to the work done on eliminating the BKL. It's probably a good idea to incorporate those changes as well in vb2. Regards, Hans Taking into account the size of the patches and the number of lines I've changed, I've decided to keep the Pawel signed-off attribute and authorship to correctly credit him. If this is against kernel rules, feel free to let me know. Best regards -- Marek Szyprowski Samsung Poland RD Center Changes since V2: = - added read() emulator (see patch #5/7) - fixed lack of parentheses in macro definitions (caused side effects in some places) - added a separate check for VM_READ or VM_WRITE in vb2_mmap() - added vb2_is_streaming(), vb2_lock and vb2_unlock inlines - updated vivi driver with the new read() emulator Changes since V1: = - removed drv_lock, added start_streaming and stop_streaming callbacks Here is the original Videobuf2 introduction prepared by Pawel: === These patches add a new driver framework for Video for Linux 2 driver - Videobuf2. Videobuf2 is intended as a replacement for videobuf, the current driver framework, which will be referred to as videobuf1 for the remainder of this document. What is videobuf2? Videobuf2 is a Video for Linux 2 API-compatible driver framework for multimedia devices. It acts as an intermediate layer between userspace applications and device drivers. It also provides low-level, modular memory management functions for drivers. Videobuf2 eases driver development, reduces drivers' code size and aids in proper and consistent implementation of V4L2 API in drivers. Videobuf2 memory management backend is fully modular. This allows custom memory management routines for devices and platforms with non-standard memory management requirements to be plugged in, without changing the high-level buffer management functions and API. The framework provides: - implementations of streaming I/O V4L2 ioctls and file operations - high-level video buffer, video queue and state management functions - video buffer
