Re: [PATCH] gpu: ipu-v3: Allow negative offsets for interlaced scanning
On 06/20/2018 01:54 AM, Philipp Zabel wrote: Hi Steve, On Tue, 2018-06-19 at 18:30 -0700, Steve Longerbeam wrote: I've found some time to diagnose the behavior of interweave with B/T line swapping (to support interlaced-bt) with planar formats. There are a couple problems (one known and one unknown): 1. This requires 32 pixel alignment to meet the IDMAC 8-byte alignment of the planar U/V buffer offsets, and 32 pixel alignment precludes capturing raw NTSC/PAL at 720 pixel line stride. What needs to be aligned to multiples of 32 pixels? I thought 8 pixel width alignment should be good enough for NV12/NV16, for which luma and chroma strides are equal to the width in pixels, and 16 pixel alignment for YUV420/YVU420/YUV422P, where chroma stride is half the width in pixels. I see where the problem is now. I was basing my mistaken 32 pixel alignment from a read of the U_OFFSET/V_OFFSET macros in ipu-cpmem.c: u_offset = pix->width * pix->height + pix->width * y / 4 + x / 2 But you can probably see the bug now. This does not produce a correct offset for odd values of y. It should read: u_offset = pix->width * pix->height + pix->width * (y / 2) / 2 + x / 2 With that fix, interweave line swap with planar 4:2:0 is working now. That includes YUV420, YVU420, and NV12. NV16 is also working after programming SLUV with double the chroma line stride. 2. Even with 32 pixel aligned frames, for example by using the prpenc scaler to generate 704 pixel strides from 720, the colors are still wrong when capturing interlaced-bt. As a side note, we can't properly scale seq-tb/bt direct input from the CSI vertically with the IC, as the bottom line of the first field will be blended with the top line of the second field... I thought for sure this must be because we also need to double the SLUV line strides in addition to doubling SLY line stride. But I tried this and the results are that it works only for YUV 4:2:2. For 4:2:0 it causes system hard lockups. (Aside note: interweave without line swap apparently has never worked for 4:2:2, even when doubling SLUV, so it's quite bizarre to me why 4:2:2 interweave _with_ line swap _does_ work after doubling SLUV). When you say 4:2:2 you only mean YUV422P, not NV16 or YUYV/UYVY ? Correct, I meant planar YUV422P. For these reasons I think we should disallow interlaced-bt with planar formats. Does that include NV12/NV16? Capturing to NV12 could be desirable if at all possible, because the result can be encoded by the CODA. The other formats are bandwidth inefficient anyway. Never mind, I found the bug described above in the U_OFFSET/V_OFFSET macros. In summary, at this point all planar formats are working with interlaced bt and tb, except for YUV422P. Steve
cron job: media_tree daily build: ERRORS
This message is generated daily by a cron job that builds media_tree for the kernels and architectures in the list below. Results of the daily build of media_tree: date: Thu Jun 21 05:00:13 CEST 2018 media-tree git hash:f2809d20b9250c675fca8268a0f6274277cca7ff media_build git hash: 26d102795c91f8593a4f74f96b955f9a8b81dbc3 v4l-utils git hash: c3b46c2c53d7d815a53c902cfb2ddd96c3732c5b gcc version:i686-linux-gcc (GCC) 8.1.0 sparse version: 0.5.2 smatch version: 0.5.1 host hardware: x86_64 host os:4.16.0-1-amd64 linux-git-arm-at91: OK linux-git-arm-davinci: OK linux-git-arm-multi: OK linux-git-arm-pxa: OK linux-git-arm-stm32: OK linux-git-arm64: OK linux-git-i686: OK linux-git-mips: OK linux-git-powerpc64: OK linux-git-sh: OK linux-git-x86_64: OK Check COMPILE_TEST: OK linux-2.6.36.4-i686: OK linux-2.6.36.4-x86_64: OK linux-2.6.37.6-i686: OK linux-2.6.37.6-x86_64: OK linux-2.6.38.8-i686: OK linux-2.6.38.8-x86_64: OK linux-2.6.39.4-i686: OK linux-2.6.39.4-x86_64: OK linux-3.0.101-i686: OK linux-3.0.101-x86_64: OK linux-3.1.10-i686: OK linux-3.1.10-x86_64: OK linux-3.2.101-i686: OK linux-3.2.101-x86_64: OK linux-3.3.8-i686: OK linux-3.3.8-x86_64: OK linux-3.4.113-i686: OK linux-3.4.113-x86_64: OK linux-3.5.7-i686: OK linux-3.5.7-x86_64: OK linux-3.6.11-i686: OK linux-3.6.11-x86_64: OK linux-3.7.10-i686: OK linux-3.7.10-x86_64: OK linux-3.8.13-i686: OK linux-3.8.13-x86_64: OK linux-3.9.11-i686: OK linux-3.9.11-x86_64: OK linux-3.10.108-i686: OK linux-3.10.108-x86_64: OK linux-3.11.10-i686: OK linux-3.11.10-x86_64: OK linux-3.12.74-i686: OK linux-3.12.74-x86_64: OK linux-3.13.11-i686: OK linux-3.13.11-x86_64: OK linux-3.14.79-i686: OK linux-3.14.79-x86_64: OK linux-3.15.10-i686: OK linux-3.15.10-x86_64: OK linux-3.16.56-i686: OK linux-3.16.56-x86_64: OK linux-3.17.8-i686: OK linux-3.17.8-x86_64: OK linux-3.18.102-i686: OK linux-3.18.102-x86_64: OK linux-3.19.8-i686: OK linux-3.19.8-x86_64: OK linux-4.0.9-i686: OK linux-4.0.9-x86_64: OK linux-4.1.51-i686: OK linux-4.1.51-x86_64: OK linux-4.2.8-i686: OK linux-4.2.8-x86_64: OK linux-4.3.6-i686: OK linux-4.3.6-x86_64: OK linux-4.4.109-i686: OK linux-4.4.109-x86_64: OK linux-4.5.7-i686: OK linux-4.5.7-x86_64: OK linux-4.6.7-i686: OK linux-4.6.7-x86_64: OK linux-4.7.10-i686: OK linux-4.7.10-x86_64: OK linux-4.8.17-i686: OK linux-4.8.17-x86_64: OK linux-4.9.91-i686: OK linux-4.9.91-x86_64: OK linux-4.10.17-i686: OK linux-4.10.17-x86_64: OK linux-4.11.12-i686: OK linux-4.11.12-x86_64: OK linux-4.12.14-i686: OK linux-4.12.14-x86_64: OK linux-4.13.16-i686: OK linux-4.13.16-x86_64: OK linux-4.14.42-i686: OK linux-4.14.42-x86_64: OK linux-4.15.14-i686: OK linux-4.15.14-x86_64: OK linux-4.16.8-i686: OK linux-4.16.8-x86_64: OK linux-4.17.2-i686: OK linux-4.17.2-x86_64: OK linux-4.18-rc1-i686: ERRORS linux-4.18-rc1-x86_64: ERRORS apps: OK spec-git: OK sparse: WARNINGS Detailed results are available here: http://www.xs4all.nl/~hverkuil/logs/Thursday.log Full logs are available here: http://www.xs4all.nl/~hverkuil/logs/Thursday.tar.bz2 The Media Infrastructure API from this daily build is here: http://www.xs4all.nl/~hverkuil/spec/index.html
[no subject]
Could you act as the beneficiary to claim 9.8M USD that my bank wants to consifacte? I will give you 50% and Every documentation would be put in placed. Mr Ubaithullah from Hong Kong.
RE: [PATCH v6 02/12] intel-ipu3: Add user space API definitions
Hi, Tomasz,
Thank you for the time spent to review this long file.
> -Original Message-
> From: Tomasz Figa [mailto:tf...@chromium.org]
> Sent: Sunday, June 17, 2018 11:09 PM
> To: Zhi, Yong
> Cc: Linux Media Mailing List ; Sakari Ailus
> ; Mani, Rajmohan
> ; Toivonen, Tuukka
> ; Hu, Jerry W ; Zheng,
> Jian Xu
> Subject: Re: [PATCH v6 02/12] intel-ipu3: Add user space API definitions
>
> Hi Yong,
>
> On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi wrote:
> >
> > Define the structures and macros to be used by public.
> >
> > Signed-off-by: Yong Zhi
> > Signed-off-by: Rajmohan Mani
> > ---
> > include/uapi/linux/intel-ipu3.h | 1403
> > +++
> > 1 file changed, 1403 insertions(+)
> > create mode 100644 include/uapi/linux/intel-ipu3.h
> >
>
> Since we'll need 1 more resend with latest fixes from Chromium tree and
> recently posted documentation anyway, let me do some more nitpicking
> inline, so we can end up with slightly cleaner code. :)
>
> > diff --git a/include/uapi/linux/intel-ipu3.h
> > b/include/uapi/linux/intel-ipu3.h new file mode 100644 index
> > ..694ef0c8d7a7
> > --- /dev/null
> > +++ b/include/uapi/linux/intel-ipu3.h
> > @@ -0,0 +1,1403 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/* Copyright (C) 2018 Intel Corporation */
> > +
> > +#ifndef __IPU3_UAPI_H
> > +#define __IPU3_UAPI_H
> > +
> > +#include
> > +
> > +#define IPU3_UAPI_ISP_VEC_ELEMS64
> > +
> > +#define IMGU_ABI_PAD __aligned(IPU3_UAPI_ISP_WORD_BYTES)
>
> This seems unused.
Ack, will remove.
>
> > +#define IPU3_ALIGN
> __attribute__((aligned(IPU3_UAPI_ISP_WORD_BYTES)))
>
> Any reason to mix both __aligned() and __attribute__((aligned()))?
>
> > +
> > +#define IPU3_UAPI_ISP_WORD_BYTES 32
>
> It would make sense to define this above IPU3_ALIGN(), which references it.
>
Sure.
> > +#define IPU3_UAPI_MAX_STRIPES 2
> > +
> > +/*** ipu3_uapi_stats_3a ***/
> > +
> > +#define IPU3_UAPI_MAX_BUBBLE_SIZE 10
> > +
> > +#define IPU3_UAPI_AE_COLORS4
> > +#define IPU3_UAPI_AE_BINS 256
> > +
> > +#define IPU3_UAPI_AWB_MD_ITEM_SIZE 8
> > +#define IPU3_UAPI_AWB_MAX_SETS 60
> > +#define IPU3_UAPI_AWB_SET_SIZE 0x500
>
> Why not just decimal 1280?
Ok, will change above and similar places to decimal expression.
>
> > +#define IPU3_UAPI_AWB_SPARE_FOR_BUBBLES \
> > + (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
> > +IPU3_UAPI_AWB_MD_ITEM_SIZE)
> > +#define IPU3_UAPI_AWB_MAX_BUFFER_SIZE \
> > + (IPU3_UAPI_AWB_MAX_SETS * \
> > +(IPU3_UAPI_AWB_SET_SIZE +
> IPU3_UAPI_AWB_SPARE_FOR_BUBBLES))
> > +
> > +#define IPU3_UAPI_AF_MAX_SETS 24
> > +#define IPU3_UAPI_AF_MD_ITEM_SIZE 4
> > +#define IPU3_UAPI_AF_SPARE_FOR_BUBBLES \
> > + (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
> > +IPU3_UAPI_AF_MD_ITEM_SIZE)
> > +#define IPU3_UAPI_AF_Y_TABLE_SET_SIZE 0x80
>
> Why not just decimal 128?
Ack.
>
> > +#define IPU3_UAPI_AF_Y_TABLE_MAX_SIZE \
> > + (IPU3_UAPI_AF_MAX_SETS * \
> > +(IPU3_UAPI_AF_Y_TABLE_SET_SIZE +
> IPU3_UAPI_AF_SPARE_FOR_BUBBLES) * \
> > +IPU3_UAPI_MAX_STRIPES)
> > +
> > +#define IPU3_UAPI_AWB_FR_MAX_SETS 24
> > +#define IPU3_UAPI_AWB_FR_MD_ITEM_SIZE 8
> > +#define IPU3_UAPI_AWB_FR_BAYER_TBL_SIZE0x100
>
> Why not just decimal 256?
Ack.
>
> > +#define IPU3_UAPI_AWB_FR_SPARE_FOR_BUBBLES \
> > + (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
> > +IPU3_UAPI_AWB_FR_MD_ITEM_SIZE) #define
> > +IPU3_UAPI_AWB_FR_BAYER_TABLE_MAX_SIZE \
> > + (IPU3_UAPI_AWB_FR_MAX_SETS * \
> > + (IPU3_UAPI_AWB_FR_BAYER_TBL_SIZE + \
> > +IPU3_UAPI_AWB_FR_SPARE_FOR_BUBBLES) *
> IPU3_UAPI_MAX_STRIPES)
> [snip]
> > +struct ipu3_uapi_af_filter_config {
> > + struct {
> > + __u8 a1;
> > + __u8 a2;
> > + __u8 a3;
> > + __u8 a4;
> > + } y1_coeff_0;
> > + struct {
> > + __u8 a5;
> > + __u8 a6;
> > + __u8 a7;
> > + __u8 a8;
> > + } y1_coeff_1;
> > + struct {
> > + __u8 a9;
> > + __u8 a10;
> > + __u8 a11;
> > + __u8 a12;
> > + } y1_coeff_2;
>
> Why these aren't just __u8 y1_coeff[12]?
Yes, we can combine them together.
>
> > +
> > + __u32 y1_sign_vec;
> > +
> > + struct {
> > + __u8 a1;
> > + __u8 a2;
> > + __u8 a3;
> > + __u8 a4;
> > + } y2_coeff_0;
> > + struct {
> > + __u8 a5;
> > + __u8 a6;
>
Re: [PATCH 1/2] media: add helpers for memory-to-memory media controller
Hi Ezequiel,
I love your patch! Yet something to improve:
[auto build test ERROR on linuxtv-media/master]
[also build test ERROR on v4.18-rc1 next-20180620]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system]
url:
https://github.com/0day-ci/linux/commits/Ezequiel-Garcia/media-add-helpers-for-memory-to-memory-media-controller/20180621-050216
base: git://linuxtv.org/media_tree.git master
config: x86_64-randconfig-x012-201824 (attached as .config)
compiler: gcc-7 (Debian 7.3.0-16) 7.3.0
reproduce:
# save the attached .config to linux build tree
make ARCH=x86_64
All errors (new ones prefixed by >>):
drivers/media/v4l2-core/v4l2-mem2mem.c: In function
'v4l2_m2m_unregister_media_controller':
>> drivers/media/v4l2-core/v4l2-mem2mem.c:624:34: error: 'struct v4l2_m2m_dev'
>> has no member named 'intf_devnode'
media_remove_intf_links(_dev->intf_devnode->intf);
^~
drivers/media/v4l2-core/v4l2-mem2mem.c:625:30: error: 'struct v4l2_m2m_dev'
has no member named 'intf_devnode'
media_devnode_remove(m2m_dev->intf_devnode);
^~
>> drivers/media/v4l2-core/v4l2-mem2mem.c:627:35: error: 'struct v4l2_m2m_dev'
>> has no member named 'source'
media_entity_remove_links(m2m_dev->source);
^~
>> drivers/media/v4l2-core/v4l2-mem2mem.c:628:36: error: 'struct v4l2_m2m_dev'
>> has no member named 'sink'
media_entity_remove_links(_dev->sink);
^~
>> drivers/media/v4l2-core/v4l2-mem2mem.c:629:36: error: 'struct v4l2_m2m_dev'
>> has no member named 'proc'
media_entity_remove_links(_dev->proc);
^~
drivers/media/v4l2-core/v4l2-mem2mem.c:630:40: error: 'struct v4l2_m2m_dev'
has no member named 'source'
media_device_unregister_entity(m2m_dev->source);
^~
drivers/media/v4l2-core/v4l2-mem2mem.c:631:41: error: 'struct v4l2_m2m_dev'
has no member named 'sink'
media_device_unregister_entity(_dev->sink);
^~
drivers/media/v4l2-core/v4l2-mem2mem.c:632:41: error: 'struct v4l2_m2m_dev'
has no member named 'proc'
media_device_unregister_entity(_dev->proc);
^~
vim +624 drivers/media/v4l2-core/v4l2-mem2mem.c
621
622 void v4l2_m2m_unregister_media_controller(struct v4l2_m2m_dev *m2m_dev)
623 {
> 624 media_remove_intf_links(_dev->intf_devnode->intf);
> 625 media_devnode_remove(m2m_dev->intf_devnode);
626
> 627 media_entity_remove_links(m2m_dev->source);
> 628 media_entity_remove_links(_dev->sink);
> 629 media_entity_remove_links(_dev->proc);
630 media_device_unregister_entity(m2m_dev->source);
631 media_device_unregister_entity(_dev->sink);
632 media_device_unregister_entity(_dev->proc);
633 }
634 EXPORT_SYMBOL_GPL(v4l2_m2m_unregister_media_controller);
635
---
0-DAY kernel test infrastructureOpen Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
.config.gz
Description: application/gzip
Re: Software-only image processing for Intel "complex" cameras
Hi! > > On Nokia N900, I have similar problems as Intel IPU3 hardware. > > > > Meeting notes say that pure software implementation is not fast > > enough, but that it may be useful for debugging. It would be also > > useful for me on N900, and probably useful for processing "raw" > > images > > from digital cameras. > > > > There is sensor part, and memory-to-memory part, right? What is > > the format of data from the sensor part? What operations would be > > expensive on the CPU? If we did everthing on the CPU, what would be > > maximum resolution where we could still manage it in real time? > > The IPU3 sensor produce a vendor specific form of bayer. If we manage > to implement support for this format, it would likely be done in > software. I don't think anyone can answer your other questions has no > one have ever implemented this, hence measure performance. I believe Intel has some estimates. What is the maximum resolution of camera in the current Dell systems? Thanks, Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html signature.asc Description: Digital signature
Re: Software-only image processing for Intel "complex" cameras
Le mercredi 20 juin 2018 à 22:38 +0200, Pavel Machek a écrit : > Hi! > > On Nokia N900, I have similar problems as Intel IPU3 hardware. > > Meeting notes say that pure software implementation is not fast > enough, but that it may be useful for debugging. It would be also > useful for me on N900, and probably useful for processing "raw" > images > from digital cameras. > > There is sensor part, and memory-to-memory part, right? What is > the format of data from the sensor part? What operations would be > expensive on the CPU? If we did everthing on the CPU, what would be > maximum resolution where we could still manage it in real time? The IPU3 sensor produce a vendor specific form of bayer. If we manage to implement support for this format, it would likely be done in software. I don't think anyone can answer your other questions has no one have ever implemented this, hence measure performance. > > Would it be possible to get access to machine with IPU3, or would > there be someone willing to test libv4l2 patches? > > Thanks and best regards, > > > Pavel signature.asc Description: This is a digitally signed message part
Software-only image processing for Intel "complex" cameras
Hi! On Nokia N900, I have similar problems as Intel IPU3 hardware. Meeting notes say that pure software implementation is not fast enough, but that it may be useful for debugging. It would be also useful for me on N900, and probably useful for processing "raw" images from digital cameras. There is sensor part, and memory-to-memory part, right? What is the format of data from the sensor part? What operations would be expensive on the CPU? If we did everthing on the CPU, what would be maximum resolution where we could still manage it in real time? Would it be possible to get access to machine with IPU3, or would there be someone willing to test libv4l2 patches? Thanks and best regards, Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html signature.asc Description: Digital signature
Re: [PATCH v2 2/3] dt-bindings: ov5640: Add "rotation" property
On Mon, Jun 18, 2018 at 12:29:18PM +0200, Hugues Fruchet wrote: > Add the rotation property to the list of optional properties > for the OV5640 camera sensor. > > Signed-off-by: Hugues Fruchet > --- > Documentation/devicetree/bindings/media/i2c/ov5640.txt | 5 + > 1 file changed, 5 insertions(+) Reviewed-by: Rob Herring
[PATCH 0/2] Memory-to-memory media controller topology
As discussed on IRC, memory-to-memory need to be modeled properly in order to be supported by the media controller framework, and thus to support the Request API. The topology looks like this: Device topology - entity 1: source (1 pad, 1 link) type Node subtype V4L flags 0 pad0: Source <- "proc":0 [ENABLED,IMMUTABLE] - entity 3: proc (2 pads, 2 links) type Node subtype Unknown flags 0 pad0: Source -> "source":0 [ENABLED,IMMUTABLE] pad1: Sink <- "sink":0 [ENABLED,IMMUTABLE] - entity 6: sink (1 pad, 1 link) type Node subtype V4L flags 0 pad0: Sink -> "proc":1 [ENABLED,IMMUTABLE] The first commit introduces a register/unregister API, that creates/destroys all the entities and pads needed, and links them. The second commit uses this API to support the vim2m driver. Ezequiel Garcia (1): media: add helpers for memory-to-memory media controller Hans Verkuil (1): vim2m: add media device drivers/media/platform/vim2m.c | 41 +- drivers/media/v4l2-core/v4l2-dev.c | 13 +- drivers/media/v4l2-core/v4l2-mem2mem.c | 176 + include/media/v4l2-mem2mem.h | 5 + include/uapi/linux/media.h | 3 + 5 files changed, 229 insertions(+), 9 deletions(-) -- 2.17.1
[PATCH 2/2] vim2m: add media device
From: Hans Verkuil
Request API requires a media node. Add one to the vim2m driver so we can
use requests with it.
Signed-off-by: Hans Verkuil
Signed-off-by: Ezequiel Garcia
---
drivers/media/platform/vim2m.c | 41 ++
1 file changed, 37 insertions(+), 4 deletions(-)
diff --git a/drivers/media/platform/vim2m.c b/drivers/media/platform/vim2m.c
index 065483e62db4..da13a8927f3f 100644
--- a/drivers/media/platform/vim2m.c
+++ b/drivers/media/platform/vim2m.c
@@ -140,6 +140,9 @@ static struct vim2m_fmt *find_format(struct v4l2_format *f)
struct vim2m_dev {
struct v4l2_device v4l2_dev;
struct video_device vfd;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_device mdev;
+#endif
atomic_tnum_inst;
struct mutexdev_mutex;
@@ -1016,7 +1019,7 @@ static int vim2m_probe(struct platform_device *pdev)
ret = video_register_device(vfd, VFL_TYPE_GRABBER, 0);
if (ret) {
v4l2_err(>v4l2_dev, "Failed to register video device\n");
- goto unreg_dev;
+ goto unreg_v4l2;
}
video_set_drvdata(vfd, dev);
@@ -1031,15 +1034,39 @@ static int vim2m_probe(struct platform_device *pdev)
if (IS_ERR(dev->m2m_dev)) {
v4l2_err(>v4l2_dev, "Failed to init mem2mem device\n");
ret = PTR_ERR(dev->m2m_dev);
- goto err_m2m;
+ goto unreg_dev;
+ }
+
+#ifdef CONFIG_MEDIA_CONTROLLER
+ dev->mdev.dev = >dev;
+ strlcpy(dev->mdev.model, "vim2m", sizeof(dev->mdev.model));
+ media_device_init(>mdev);
+ dev->v4l2_dev.mdev = >mdev;
+
+ ret = v4l2_m2m_register_media_controller(dev->m2m_dev,
+ vfd, MEDIA_ENT_F_PROC_VIDEO_SCALER);
+ if (ret) {
+ v4l2_err(>v4l2_dev, "Failed to init mem2mem media
controller\n");
+ goto unreg_m2m;
}
+ ret = media_device_register(>mdev);
+ if (ret) {
+ v4l2_err(>v4l2_dev, "Failed to register mem2mem media
device\n");
+ goto unreg_m2m_mc;
+ }
+#endif
return 0;
-err_m2m:
+#ifdef CONFIG_MEDIA_CONTROLLER
+unreg_m2m_mc:
+ v4l2_m2m_unregister_media_controller(dev->m2m_dev);
+unreg_m2m:
v4l2_m2m_release(dev->m2m_dev);
- video_unregister_device(>vfd);
+#endif
unreg_dev:
+ video_unregister_device(>vfd);
+unreg_v4l2:
v4l2_device_unregister(>v4l2_dev);
return ret;
@@ -1050,6 +1077,12 @@ static int vim2m_remove(struct platform_device *pdev)
struct vim2m_dev *dev = platform_get_drvdata(pdev);
v4l2_info(>v4l2_dev, "Removing " MEM2MEM_NAME);
+
+#ifdef CONFIG_MEDIA_CONTROLLER
+ media_device_unregister(>mdev);
+ v4l2_m2m_unregister_media_controller(dev->m2m_dev);
+ media_device_cleanup(>mdev);
+#endif
v4l2_m2m_release(dev->m2m_dev);
del_timer_sync(>timer);
video_unregister_device(>vfd);
--
2.17.1
[PATCH 1/2] media: add helpers for memory-to-memory media controller
A memory-to-memory pipeline device consists in three
entities: two DMA engine and one video processing entities.
The DMA engine entities are linked to a V4L interface.
This commit add a new v4l2_m2m_{un}register_media_controller
API to register this topology.
For instance, a typical mem2mem device topology would
look like this:
Device topology
- entity 1: source (1 pad, 1 link)
type Node subtype V4L flags 0
pad0: Source
<- "proc":0 [ENABLED,IMMUTABLE]
- entity 3: proc (2 pads, 2 links)
type Node subtype Unknown flags 0
pad0: Source
-> "source":0 [ENABLED,IMMUTABLE]
pad1: Sink
<- "sink":0 [ENABLED,IMMUTABLE]
- entity 6: sink (1 pad, 1 link)
type Node subtype V4L flags 0
pad0: Sink
-> "proc":1 [ENABLED,IMMUTABLE]
Suggested-by: Laurent Pinchart
Suggested-by: Hans Verkuil
Signed-off-by: Ezequiel Garcia
---
drivers/media/v4l2-core/v4l2-dev.c | 13 +-
drivers/media/v4l2-core/v4l2-mem2mem.c | 176 +
include/media/v4l2-mem2mem.h | 5 +
include/uapi/linux/media.h | 3 +
4 files changed, 192 insertions(+), 5 deletions(-)
diff --git a/drivers/media/v4l2-core/v4l2-dev.c
b/drivers/media/v4l2-core/v4l2-dev.c
index 4ffd7d60a901..c1996d73ca74 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -202,7 +202,7 @@ static void v4l2_device_release(struct device *cd)
mutex_unlock(_lock);
#if defined(CONFIG_MEDIA_CONTROLLER)
- if (v4l2_dev->mdev) {
+ if (v4l2_dev->mdev && vdev->vfl_dir != VFL_DIR_M2M) {
/* Remove interfaces and interface links */
media_devnode_remove(vdev->intf_devnode);
if (vdev->entity.function != MEDIA_ENT_F_UNKNOWN)
@@ -733,19 +733,22 @@ static void determine_valid_ioctls(struct video_device
*vdev)
BASE_VIDIOC_PRIVATE);
}
-static int video_register_media_controller(struct video_device *vdev, int type)
+static int video_register_media_controller(struct video_device *vdev)
{
#if defined(CONFIG_MEDIA_CONTROLLER)
u32 intf_type;
int ret;
- if (!vdev->v4l2_dev->mdev)
+ /* Memory-to-memory devices are more complex and use
+* their own function to register its mc entities.
+*/
+ if (!vdev->v4l2_dev->mdev || vdev->vfl_dir == VFL_DIR_M2M)
return 0;
vdev->entity.obj_type = MEDIA_ENTITY_TYPE_VIDEO_DEVICE;
vdev->entity.function = MEDIA_ENT_F_UNKNOWN;
- switch (type) {
+ switch (vdev->vfl_type) {
case VFL_TYPE_GRABBER:
intf_type = MEDIA_INTF_T_V4L_VIDEO;
vdev->entity.function = MEDIA_ENT_F_IO_V4L;
@@ -993,7 +996,7 @@ int __video_register_device(struct video_device *vdev,
v4l2_device_get(vdev->v4l2_dev);
/* Part 5: Register the entity. */
- ret = video_register_media_controller(vdev, type);
+ ret = video_register_media_controller(vdev);
/* Part 6: Activate this minor. The char device can now be used. */
set_bit(V4L2_FL_REGISTERED, >flags);
diff --git a/drivers/media/v4l2-core/v4l2-mem2mem.c
b/drivers/media/v4l2-core/v4l2-mem2mem.c
index c4f963d96a79..c9ee141c2b33 100644
--- a/drivers/media/v4l2-core/v4l2-mem2mem.c
+++ b/drivers/media/v4l2-core/v4l2-mem2mem.c
@@ -17,9 +17,11 @@
#include
#include
+#include
#include
#include
#include
+#include
#include
#include
@@ -50,6 +52,19 @@ module_param(debug, bool, 0644);
* offsets but for different queues */
#define DST_QUEUE_OFF_BASE (1 << 30)
+enum v4l2_m2m_entity_type {
+ MEM2MEM_ENT_TYPE_SOURCE,
+ MEM2MEM_ENT_TYPE_SINK,
+ MEM2MEM_ENT_TYPE_PROC,
+ MEM2MEM_ENT_TYPE_MAX
+};
+
+static const char * const m2m_entity_name[] = {
+ "source",
+ "sink",
+ "proc"
+};
+
/**
* struct v4l2_m2m_dev - per-device context
@@ -60,6 +75,15 @@ module_param(debug, bool, 0644);
*/
struct v4l2_m2m_dev {
struct v4l2_m2m_ctx *curr_ctx;
+#ifdef CONFIG_MEDIA_CONTROLLER
+ struct media_entity *source;
+ struct media_padsource_pad;
+ struct media_entity sink;
+ struct media_padsink_pad;
+ struct media_entity proc;
+ struct media_padproc_pads[2];
+ struct media_intf_devnode *intf_devnode;
+#endif
struct list_headjob_queue;
spinlock_t job_spinlock;
@@ -595,6 +619,158 @@ int v4l2_m2m_mmap(struct file *file, struct v4l2_m2m_ctx
*m2m_ctx,
}
EXPORT_SYMBOL(v4l2_m2m_mmap);
+void v4l2_m2m_unregister_media_controller(struct v4l2_m2m_dev *m2m_dev)
+{
+ media_remove_intf_links(_dev->intf_devnode->intf);
+ media_devnode_remove(m2m_dev->intf_devnode);
+
+ media_entity_remove_links(m2m_dev->source);
+ media_entity_remove_links(_dev->sink);
+
Re: [RFC 0/2] Memory-to-memory media controller topology
On Sat, 2018-06-16 at 20:31 -0400, Nicolas Dufresne wrote: > Le vendredi 15 juin 2018 à 17:05 -0300, Ezequiel Garcia a écrit : > > > Will the end result have "device node name /dev/..." on both > > > entity > > > 1 > > > and 6 ? > > > > No. There is just one devnode /dev/videoX, which is accepts > > both CAPTURE and OUTPUT directions. > > My question is more ifthe dev node path will be provided somehow, > because it's not displayed in this topologyé > AFAIU, the device node associated to each media interface object can be discovered via the G_TOPOLOGY[1] ioctl. User gets the major:minor tuple, which can be used to get the node path. [1] https://linuxtv.org/downloads/v4l-dvb-apis/uapi/mediactl/media-ioc- g-topology.html
[PATCH v5 04/17] omap3isp: Add vb2_queue lock
vb2_queue locks is now mandatory. Add it, remove driver ad-hoc locks,
and implement wait_{prepare, finish}.
Also, remove stream_lock mutex. Sicen the ioctls operations
are protected by the queue mutex, the stream_lock mutex is
not needed.
Signed-off-by: Ezequiel Garcia
---
drivers/media/platform/omap3isp/ispvideo.c | 65 --
drivers/media/platform/omap3isp/ispvideo.h | 1 -
2 files changed, 11 insertions(+), 55 deletions(-)
diff --git a/drivers/media/platform/omap3isp/ispvideo.c
b/drivers/media/platform/omap3isp/ispvideo.c
index 9d228eac24ea..f835aeb9ddc5 100644
--- a/drivers/media/platform/omap3isp/ispvideo.c
+++ b/drivers/media/platform/omap3isp/ispvideo.c
@@ -496,6 +496,8 @@ static const struct vb2_ops isp_video_queue_ops = {
.buf_prepare = isp_video_buffer_prepare,
.buf_queue = isp_video_buffer_queue,
.start_streaming = isp_video_start_streaming,
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
};
/*
@@ -628,11 +630,8 @@ void omap3isp_video_resume(struct isp_video *video, int
continuous)
{
struct isp_buffer *buf = NULL;
- if (continuous && video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE) {
- mutex_lock(>queue_lock);
+ if (continuous && video->type == V4L2_BUF_TYPE_VIDEO_CAPTURE)
vb2_discard_done(video->queue);
- mutex_unlock(>queue_lock);
- }
if (!list_empty(>dmaqueue)) {
buf = list_first_entry(>dmaqueue,
@@ -909,13 +908,8 @@ isp_video_reqbufs(struct file *file, void *fh, struct
v4l2_requestbuffers *rb)
{
struct isp_video_fh *vfh = to_isp_video_fh(fh);
struct isp_video *video = video_drvdata(file);
- int ret;
-
- mutex_lock(>queue_lock);
- ret = vb2_reqbufs(>queue, rb);
- mutex_unlock(>queue_lock);
- return ret;
+ return vb2_reqbufs(>queue, rb);
}
static int
@@ -923,13 +917,8 @@ isp_video_querybuf(struct file *file, void *fh, struct
v4l2_buffer *b)
{
struct isp_video_fh *vfh = to_isp_video_fh(fh);
struct isp_video *video = video_drvdata(file);
- int ret;
- mutex_lock(>queue_lock);
- ret = vb2_querybuf(>queue, b);
- mutex_unlock(>queue_lock);
-
- return ret;
+ return vb2_querybuf(>queue, b);
}
static int
@@ -937,13 +926,8 @@ isp_video_qbuf(struct file *file, void *fh, struct
v4l2_buffer *b)
{
struct isp_video_fh *vfh = to_isp_video_fh(fh);
struct isp_video *video = video_drvdata(file);
- int ret;
- mutex_lock(>queue_lock);
- ret = vb2_qbuf(>queue, b);
- mutex_unlock(>queue_lock);
-
- return ret;
+ return vb2_qbuf(>queue, b);
}
static int
@@ -951,13 +935,8 @@ isp_video_dqbuf(struct file *file, void *fh, struct
v4l2_buffer *b)
{
struct isp_video_fh *vfh = to_isp_video_fh(fh);
struct isp_video *video = video_drvdata(file);
- int ret;
- mutex_lock(>queue_lock);
- ret = vb2_dqbuf(>queue, b, file->f_flags & O_NONBLOCK);
- mutex_unlock(>queue_lock);
-
- return ret;
+ return vb2_dqbuf(>queue, b, file->f_flags & O_NONBLOCK);
}
static int isp_video_check_external_subdevs(struct isp_video *video,
@@ -1096,8 +1075,6 @@ isp_video_streamon(struct file *file, void *fh, enum
v4l2_buf_type type)
if (type != video->type)
return -EINVAL;
- mutex_lock(>stream_lock);
-
/* Start streaming on the pipeline. No link touching an entity in the
* pipeline can be activated or deactivated once streaming is started.
*/
@@ -1106,7 +1083,7 @@ isp_video_streamon(struct file *file, void *fh, enum
v4l2_buf_type type)
ret = media_entity_enum_init(>ent_enum, >isp->media_dev);
if (ret)
- goto err_enum_init;
+ return ret;
/* TODO: Implement PM QoS */
pipe->l3_ick = clk_get_rate(video->isp->clock[ISP_CLK_L3_ICK]);
@@ -1158,14 +1135,10 @@ isp_video_streamon(struct file *file, void *fh, enum
v4l2_buf_type type)
atomic_set(>frame_number, -1);
pipe->field = vfh->format.fmt.pix.field;
- mutex_lock(>queue_lock);
ret = vb2_streamon(>queue, type);
- mutex_unlock(>queue_lock);
if (ret < 0)
goto err_check_format;
- mutex_unlock(>stream_lock);
-
return 0;
err_check_format:
@@ -1184,9 +1157,6 @@ isp_video_streamon(struct file *file, void *fh, enum
v4l2_buf_type type)
media_entity_enum_cleanup(>ent_enum);
-err_enum_init:
- mutex_unlock(>stream_lock);
-
return ret;
}
@@ -1203,15 +1173,10 @@ isp_video_streamoff(struct file *file, void *fh, enum
v4l2_buf_type type)
if (type != video->type)
return -EINVAL;
- mutex_lock(>stream_lock);
-
/* Make sure we're not streaming yet. */
- mutex_lock(>queue_lock);
streaming = vb2_is_streaming(>queue);
-
Re: [PATCH v3] media: ov5640: fix frame interval enumeration
Hi Hugues,
I love your patch! Perhaps something to improve:
[auto build test WARNING on linuxtv-media/master]
[also build test WARNING on v4.18-rc1 next-20180619]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system]
url:
https://github.com/0day-ci/linux/commits/Hugues-Fruchet/media-ov5640-fix-frame-interval-enumeration/20180620-175405
base: git://linuxtv.org/media_tree.git master
reproduce:
# apt-get install sparse
make ARCH=x86_64 allmodconfig
make C=1 CF=-D__CHECK_ENDIAN__
sparse warnings: (new ones prefixed by >>)
>> drivers/media/i2c/ov5640.c:1394:14: sparse: incorrect type in assignment
>> (different base types) @@expected struct ov5640_mode_info const *mode @@
>>got ststruct ov5640_mode_info const *mode @@
drivers/media/i2c/ov5640.c:1394:14:expected struct ov5640_mode_info
const *mode
drivers/media/i2c/ov5640.c:1394:14:got struct ov5640_mode_info const (
* )[9]
vim +1394 drivers/media/i2c/ov5640.c
1387
1388 static const struct ov5640_mode_info *
1389 ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
1390 int width, int height, bool nearest)
1391 {
1392 const struct ov5640_mode_info *mode;
1393
> 1394 mode = v4l2_find_nearest_size(ov5640_mode_data[fr],
1395ARRAY_SIZE(ov5640_mode_data[fr]),
1396hact, vact,
1397width, height);
1398
1399 if (!mode ||
1400 (!nearest && (mode->hact != width || mode->vact != height)))
1401 return NULL;
1402
1403 return mode;
1404 }
1405
---
0-DAY kernel test infrastructureOpen Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
Re: [PATCH] gpu: ipu-v3: Allow negative offsets for interlaced scanning
Hi Steve, On Tue, 2018-06-19 at 18:30 -0700, Steve Longerbeam wrote: > Hi Philipp, Krzysztof, > > On 06/15/2018 01:33 AM, Krzysztof Hałasa wrote: > > Steve Longerbeam writes: > > > > > Right, the selection of interweave is moved to the capture devices, > > > so the following will enable interweave: > > > > > > v4l2-ctl -dN --set-fmt-video=field=interlaced_tb > > > > and > > > > > So the patch to adv7180 needs to be modified to report # field lines. > > > > > > Try the following: > > > > > > --- a/drivers/media/i2c/adv7180.c > > > +++ b/drivers/media/i2c/adv7180.c > > > > With this patch, fix-csi-interlaced.3 seems to work for me. > > "ipu2_csi1":2 reports [fmt:AYUV32/720x576 field:seq-tb], but the > > /dev/videoX shows (when requested) 720 x 576 NV12 interlaced, top field > > first, and I'm getting valid output. > > > > Thanks for your work. > > I've found some time to diagnose the behavior of interweave with B/T line > swapping (to support interlaced-bt) with planar formats. > > There are a couple problems (one known and one unknown): > > 1. This requires 32 pixel alignment to meet the IDMAC 8-byte alignment > of the planar U/V buffer offsets, and 32 pixel alignment precludes > capturing raw NTSC/PAL at 720 pixel line stride. What needs to be aligned to multiples of 32 pixels? I thought 8 pixel width alignment should be good enough for NV12/NV16, for which luma and chroma strides are equal to the width in pixels, and 16 pixel alignment for YUV420/YVU420/YUV422P, where chroma stride is half the width in pixels. > 2. Even with 32 pixel aligned frames, for example by using the prpenc scaler > to generate 704 pixel strides from 720, the colors are still wrong when > capturing interlaced-bt. As a side note, we can't properly scale seq-tb/bt direct input from the CSI vertically with the IC, as the bottom line of the first field will be blended with the top line of the second field... > I thought for sure this must be because we > also > need to double the SLUV line strides in addition to doubling SLY > line stride. > But I tried this and the results are that it works only for YUV > 4:2:2. For 4:2:0 > it causes system hard lockups. (Aside note: interweave without line > swap > apparently has never worked for 4:2:2, even when doubling SLUV, so it's > quite bizarre to me why 4:2:2 interweave _with_ line swap _does_ work > after doubling SLUV). When you say 4:2:2 you only mean YUV422P, not NV16 or YUYV/UYVY ? > For these reasons I think we should disallow interlaced-bt with planar > formats. Does that include NV12/NV16? Capturing to NV12 could be desirable if at all possible, because the result can be encoded by the CODA. The other formats are bandwidth inefficient anyway. > If the user needs NTSC interlaced capture with planar, the fields can be > swapped at > the CSI, by selecting seq-tb at the CSI source pad, which allows for > interlaced-tb > at the capture interface, which doesn't require interweave line swapping. regards Philipp
[PATCH v3] media: ov5640: fix frame interval enumeration
Driver must reject frame interval enumeration of unsupported resolution.
This was detected by v4l2-compliance format ioctl test:
v4l2-compliance Format ioctls:
info: found 2 frameintervals for pixel format 4745504a and size 176x144
fail: v4l2-test-formats.cpp(123):
found frame intervals for invalid size 177x144
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: FAIL
Signed-off-by: Hugues Fruchet
---
version 2:
- revisit patch according to Mauro comments:
See https://www.mail-archive.com/linux-media@vger.kernel.org/msg127380.html
version 3:
- revisit patch using v4l2_find_nearest_size() helper as per Sakari
suggestion:
See https://www.mail-archive.com/linux-media@vger.kernel.org/msg128186.html
drivers/media/i2c/ov5640.c | 34 --
1 file changed, 16 insertions(+), 18 deletions(-)
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index f6e40cc..e4b68e3 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -1389,24 +1389,16 @@ static int ov5640_set_timings(struct ov5640_dev *sensor,
ov5640_find_mode(struct ov5640_dev *sensor, enum ov5640_frame_rate fr,
int width, int height, bool nearest)
{
- const struct ov5640_mode_info *mode = NULL;
- int i;
-
- for (i = OV5640_NUM_MODES - 1; i >= 0; i--) {
- mode = _mode_data[fr][i];
-
- if (!mode->reg_data)
- continue;
+ const struct ov5640_mode_info *mode;
- if ((nearest && mode->hact <= width &&
-mode->vact <= height) ||
- (!nearest && mode->hact == width &&
-mode->vact == height))
- break;
- }
+ mode = v4l2_find_nearest_size(ov5640_mode_data[fr],
+ ARRAY_SIZE(ov5640_mode_data[fr]),
+ hact, vact,
+ width, height);
- if (nearest && i < 0)
- mode = _mode_data[fr][0];
+ if (!mode ||
+ (!nearest && (mode->hact != width || mode->vact != height)))
+ return NULL;
return mode;
}
@@ -2435,8 +2427,14 @@ static int ov5640_s_frame_interval(struct v4l2_subdev
*sd,
sensor->current_fr = frame_rate;
sensor->frame_interval = fi->interval;
- sensor->current_mode = ov5640_find_mode(sensor, frame_rate, mode->hact,
- mode->vact, true);
+ mode = ov5640_find_mode(sensor, frame_rate, mode->hact,
+ mode->vact, true);
+ if (!mode) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ sensor->current_mode = mode;
sensor->pending_mode_change = true;
out:
mutex_unlock(>lock);
--
1.9.1
