Re: VIVID/VIMC and media fuzzing

[Helen Koike]

Hi Dmitry,

On 10/31/18 9:49 AM, Helen Koike wrote:
> Hi Dmitry,
> 
> On 10/31/18 7:46 AM, Hans Verkuil wrote:
>> On 10/30/2018 03:02 PM, Dmitry Vyukov wrote:
>>> Hello Helen and linux-media,
>>>
>>> I've attended your talk "Shifting Media App Development into High
>>> Gear" on OSS Summit last week and approached you with some questions
>>> if/how this can be used for kernel testing. Thanks, turn out to be a
>>> very useful talk!
> 
> Great, I am  glad it was useful :)
> 
>>>
>>> I am working on syzkaller/syzbot, continuous kernel fuzzing system:
>>> https://github.com/google/syzkaller
>>> https://github.com/google/syzkaller/blob/master/docs/syzbot.md
>>> https://syzkaller.appspot.com
>>>
>>> After simply enabling CONFIG_VIDEO_VIMC, CONFIG_VIDEO_VIM2M,
>>> CONFIG_VIDEO_VIVID, CONFIG_VIDEO_VICODEC syzbot has found 8 bugs in
>>> media subsystem in just 24 hours:
>>>
>>> KASAN: use-after-free Read in vb2_mmap
>>> https://groups.google.com/forum/#!msg/syzkaller-bugs/XGGH69jMWQ0/S8vfxgEmCgAJ
>>>
>>> KASAN: use-after-free Write in __vb2_cleanup_fileio
>>> https://groups.google.com/forum/#!msg/syzkaller-bugs/qKKhsZVPo3o/P6AB2of2CQAJ
>>>
>>> WARNING in __vb2_queue_cancel
>>> https://groups.google.com/forum/#!msg/syzkaller-bugs/S29GU_NtfPY/ZvAz8UDtCQAJ
>>>
>>> divide error in vivid_vid_cap_s_dv_timings
>>> https://groups.google.com/forum/#!msg/syzkaller-bugs/GwF5zGBCfyg/wnuWmW_sCQAJ
>>
>> Should be fixed by https://patchwork.linuxtv.org/patch/52641/
>>
>>>
>>> KASAN: use-after-free Read in wake_up_if_idle
>>> https://groups.google.com/forum/#!msg/syzkaller-bugs/aBWb_yV1kiI/sWQO63fkCQAJ
>>>
>>> KASAN: use-after-free Read in __vb2_perform_fileio
>>> https://groups.google.com/forum/#!msg/syzkaller-bugs/MdFCZHz0LUQ/qSK_bFbcCQAJ
>>>
>>> INFO: task hung in vivid_stop_generating_vid_cap
>>> https://groups.google.com/forum/#!msg/syzkaller-bugs/F_KFW6PVyTA/wTBeHLfTCQAJ
>>>
>>> KASAN: null-ptr-deref Write in kthread_stop
>>> https://groups.google.com/forum/#!msg/syzkaller-bugs/u0AGnYvSlf4/fUiyfA_TCQAJ
>>
>> These last two should be fixed by https://patchwork.linuxtv.org/patch/52640/
>>
>> Haven't figured out the others yet (hope to get back to that next week).
>>
>>>
>>> Based on this I think if we put more effort into media fuzzing, it
>>> will be able to find dozens more.
>>
>> Yeah, this is good stuff. Thank you for setting this up.
> 
> Agreed, Dmitry thank you for doing this.
> 
>>
>>>
>>> syzkaller needs descriptions of kernel interfaces to efficiently cover
>>> a subsystem. For example, see:
>>> https://github.com/google/syzkaller/blob/master/sys/linux/uinput.txt
>>> Hopefully you can read it without much explanation, it basically
>>> states that there is that node in /dev and here are ioctls and other
>>> syscalls that are relevant for this device and here are types of
>>> arguments and layout of involved data structures.
>>>
>>> Turned we actually have such descriptions for /dev/video* and 
>>> /dev/v4l-subdev*:
>>> https://github.com/google/syzkaller/blob/master/sys/linux/video4linux.txt
>>> But we don't have anything for /dev/media*, fuzzer merely knows that
>>> it can open the device:
>>> https://github.com/google/syzkaller/blob/12b38f22c18c6109a5cc1c0238d015eef121b9b7/sys/linux/sys.txt#L479
>>> and then it will just blindly execute completely random workload on
>>> it, e.g. most likely it won't be able to come up with a proper complex
>>> structure layout for some ioctls. And I am actually not completely
>>> sure about completeness and coverage of video4linux.txt descriptions
>>> too as they were contributed by somebody interested in android
>>> testing.
>>
>> A quick look suggests that it is based on the 4.9 videodev2.h, which ain't
>> too bad. There are some differences between the 4.20 videodev2.h and the
>> 4.9, but not too many.
>>
>>>
>>> I wonder if somebody knowledgeable in /dev/media interface be willing
>>> to contribute additional descriptions?
>>
>> We'll have to wait for 4.20-rc1 to be released since there are important
>> additions to the media API. I can probably come up with something, I'm
>> just not sure when I get around to it. Ping me in three weeks time if you
>> haven't heard from me.
>>
>>>
>>> We also have code cov

Re: VIVID/VIMC and media fuzzing

[Helen Koike]

Hi Dmitry,

On 10/31/18 7:46 AM, Hans Verkuil wrote:
> On 10/30/2018 03:02 PM, Dmitry Vyukov wrote:
>> Hello Helen and linux-media,
>>
>> I've attended your talk "Shifting Media App Development into High
>> Gear" on OSS Summit last week and approached you with some questions
>> if/how this can be used for kernel testing. Thanks, turn out to be a
>> very useful talk!

Great, I am  glad it was useful :)

>>
>> I am working on syzkaller/syzbot, continuous kernel fuzzing system:
>> https://github.com/google/syzkaller
>> https://github.com/google/syzkaller/blob/master/docs/syzbot.md
>> https://syzkaller.appspot.com
>>
>> After simply enabling CONFIG_VIDEO_VIMC, CONFIG_VIDEO_VIM2M,
>> CONFIG_VIDEO_VIVID, CONFIG_VIDEO_VICODEC syzbot has found 8 bugs in
>> media subsystem in just 24 hours:
>>
>> KASAN: use-after-free Read in vb2_mmap
>> https://groups.google.com/forum/#!msg/syzkaller-bugs/XGGH69jMWQ0/S8vfxgEmCgAJ
>>
>> KASAN: use-after-free Write in __vb2_cleanup_fileio
>> https://groups.google.com/forum/#!msg/syzkaller-bugs/qKKhsZVPo3o/P6AB2of2CQAJ
>>
>> WARNING in __vb2_queue_cancel
>> https://groups.google.com/forum/#!msg/syzkaller-bugs/S29GU_NtfPY/ZvAz8UDtCQAJ
>>
>> divide error in vivid_vid_cap_s_dv_timings
>> https://groups.google.com/forum/#!msg/syzkaller-bugs/GwF5zGBCfyg/wnuWmW_sCQAJ
> 
> Should be fixed by https://patchwork.linuxtv.org/patch/52641/
> 
>>
>> KASAN: use-after-free Read in wake_up_if_idle
>> https://groups.google.com/forum/#!msg/syzkaller-bugs/aBWb_yV1kiI/sWQO63fkCQAJ
>>
>> KASAN: use-after-free Read in __vb2_perform_fileio
>> https://groups.google.com/forum/#!msg/syzkaller-bugs/MdFCZHz0LUQ/qSK_bFbcCQAJ
>>
>> INFO: task hung in vivid_stop_generating_vid_cap
>> https://groups.google.com/forum/#!msg/syzkaller-bugs/F_KFW6PVyTA/wTBeHLfTCQAJ
>>
>> KASAN: null-ptr-deref Write in kthread_stop
>> https://groups.google.com/forum/#!msg/syzkaller-bugs/u0AGnYvSlf4/fUiyfA_TCQAJ
> 
> These last two should be fixed by https://patchwork.linuxtv.org/patch/52640/
> 
> Haven't figured out the others yet (hope to get back to that next week).
> 
>>
>> Based on this I think if we put more effort into media fuzzing, it
>> will be able to find dozens more.
> 
> Yeah, this is good stuff. Thank you for setting this up.

Agreed, Dmitry thank you for doing this.

> 
>>
>> syzkaller needs descriptions of kernel interfaces to efficiently cover
>> a subsystem. For example, see:
>> https://github.com/google/syzkaller/blob/master/sys/linux/uinput.txt
>> Hopefully you can read it without much explanation, it basically
>> states that there is that node in /dev and here are ioctls and other
>> syscalls that are relevant for this device and here are types of
>> arguments and layout of involved data structures.
>>
>> Turned we actually have such descriptions for /dev/video* and 
>> /dev/v4l-subdev*:
>> https://github.com/google/syzkaller/blob/master/sys/linux/video4linux.txt
>> But we don't have anything for /dev/media*, fuzzer merely knows that
>> it can open the device:
>> https://github.com/google/syzkaller/blob/12b38f22c18c6109a5cc1c0238d015eef121b9b7/sys/linux/sys.txt#L479
>> and then it will just blindly execute completely random workload on
>> it, e.g. most likely it won't be able to come up with a proper complex
>> structure layout for some ioctls. And I am actually not completely
>> sure about completeness and coverage of video4linux.txt descriptions
>> too as they were contributed by somebody interested in android
>> testing.
> 
> A quick look suggests that it is based on the 4.9 videodev2.h, which ain't
> too bad. There are some differences between the 4.20 videodev2.h and the
> 4.9, but not too many.
> 
>>
>> I wonder if somebody knowledgeable in /dev/media interface be willing
>> to contribute additional descriptions?
> 
> We'll have to wait for 4.20-rc1 to be released since there are important
> additions to the media API. I can probably come up with something, I'm
> just not sure when I get around to it. Ping me in three weeks time if you
> haven't heard from me.
> 
>>
>> We also have code coverage reports with the coverage fuzzer achieved
>> so far. Here in the Cover column:
>> https://syzkaller.appspot.com/#managers
>> e.g. this one (but note this is a ~80MB html file):
>> https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
>> This can be used to assess e.g. v4l coverage. But I don't know what's
>> coverable in general from syscalls and what's coverable via the stub
>> drivers in particular. So some expertise from media developers would
>> be helpful too.
> 
> The four virtual drivers should give pretty decent coverage of the core
> code. Are you able to test with a 32-bit syzkaller application on a 64-bit
> kernel as well? That way the compat32 code is tested.
> 
>>
>> Do I understand it correctly that when a process opens /dev/video* or
>> /dev/media* it gets a private instance of the device? In particular,
>> if several processes test this in parallel, will they collide? Or they
>> 

Re: [PATCH vicodec] media: pvrusb2: replace `printk` with `pr_*`

[Helen Koike]

Hi Dafna,

Thanks for you patch.

On 10/6/18 4:21 PM, Dafna Hirschfeld wrote:
> Replace calls to `printk` with the appropriate `pr_*`
> macro.
> 
> Signed-off-by: Dafna Hirschfeld 
> ---
>  drivers/media/usb/pvrusb2/pvrusb2-hdw.c  |  8 
>  drivers/media/usb/pvrusb2/pvrusb2-i2c-core.c | 14 ++
>  drivers/media/usb/pvrusb2/pvrusb2-main.c |  4 ++--
>  drivers/media/usb/pvrusb2/pvrusb2-v4l2.c |  4 ++--

Could you also update drivers/media/usb/pvrusb2/pvrusb2-debug.h please?
So we can get rid of all the printk in this driver.

>  4 files changed, 14 insertions(+), 16 deletions(-)
> 
> diff --git a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c 
> b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
> index a8519da0020b..7702285c1519 100644
> --- a/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
> +++ b/drivers/media/usb/pvrusb2/pvrusb2-hdw.c
> @@ -3293,12 +3293,12 @@ void pvr2_hdw_trigger_module_log(struct pvr2_hdw *hdw)
>   int nr = pvr2_hdw_get_unit_number(hdw);
>   LOCK_TAKE(hdw->big_lock);
>   do {
> - printk(KERN_INFO "pvrusb2: =  START STATUS CARD 
> #%d  =\n", nr);
> + pr_info("pvrusb2: =  START STATUS CARD #%d  
> =\n", nr);
>   v4l2_device_call_all(>v4l2_dev, 0, core, log_status);
>   pvr2_trace(PVR2_TRACE_INFO,"cx2341x config:");
>   cx2341x_log_status(>enc_ctl_state, "pvrusb2");
>   pvr2_hdw_state_log_state(hdw);
> - printk(KERN_INFO "pvrusb2: ==  END STATUS CARD 
> #%d  ==\n", nr);
> + pr_info("pvrusb2: ==  END STATUS CARD #%d  
> ==\n", nr);
>   } while (0);
>   LOCK_GIVE(hdw->big_lock);
>  }
> @@ -4851,7 +4851,7 @@ static void pvr2_hdw_state_log_state(struct pvr2_hdw 
> *hdw)
>   for (idx = 0; ; idx++) {
>   ccnt = pvr2_hdw_report_unlocked(hdw,idx,buf,sizeof(buf));
>   if (!ccnt) break;
> - printk(KERN_INFO "%s %.*s\n",hdw->name,ccnt,buf);
> + pr_info("%s %.*s\n", hdw->name, ccnt, buf);
>   }
>   ccnt = pvr2_hdw_report_clients(hdw, buf, sizeof(buf));
>   if (ccnt >= sizeof(buf))
> @@ -4863,7 +4863,7 @@ static void pvr2_hdw_state_log_state(struct pvr2_hdw 
> *hdw)
>   while ((lcnt + ucnt < ccnt) && (buf[lcnt + ucnt] != '\n')) {
>   lcnt++;
>   }
> - printk(KERN_INFO "%s %.*s\n", hdw->name, lcnt, buf + ucnt);
> + pr_info("%s %.*s\n", hdw->name, lcnt, buf + ucnt);
>   ucnt += lcnt + 1;
>   }
>  }> diff --git a/drivers/media/usb/pvrusb2/pvrusb2-i2c-core.c
b/drivers/media/usb/pvrusb2/pvrusb2-i2c-core.c
> index ec7d32759e39..06c7e875fa0f 100644
> --- a/drivers/media/usb/pvrusb2/pvrusb2-i2c-core.c
> +++ b/drivers/media/usb/pvrusb2/pvrusb2-i2c-core.c
> @@ -478,8 +478,7 @@ static int pvr2_i2c_xfer(struct i2c_adapter *i2c_adap,
>   unsigned int idx,offs,cnt;
>   for (idx = 0; idx < num; idx++) {
>   cnt = msgs[idx].len;
> - printk(KERN_INFO
> -"pvrusb2 i2c xfer %u/%u: addr=0x%x len=%d %s",
> + pr_info("pvrusb2 i2c xfer %u/%u: addr=0x%x len=%d %s",
>  idx+1,num,
>  msgs[idx].addr,
>  cnt,
> @@ -501,8 +500,7 @@ static int pvr2_i2c_xfer(struct i2c_adapter *i2c_adap,
>   printk(KERN_CONT "\n");
>   }
>   if (!num) {
> - printk(KERN_INFO
> -"pvrusb2 i2c xfer null transfer result=%d\n",
> + pr_info("pvrusb2 i2c xfer null transfer result=%d\n",
>  ret);
>   }
>   }
> @@ -542,14 +540,14 @@ static int do_i2c_probe(struct pvr2_hdw *hdw, int addr)
>  static void do_i2c_scan(struct pvr2_hdw *hdw)
>  {
>   int i;
> - printk(KERN_INFO "%s: i2c scan beginning\n", hdw->name);
> + pr_info("%s: i2c scan beginning\n", hdw->name);
>   for (i = 0; i < 128; i++) {
>   if (do_i2c_probe(hdw, i)) {
> - printk(KERN_INFO "%s: i2c scan: found device @ 0x%x\n",
> + pr_info("%s: i2c scan: found device @ 0x%x\n",
>  hdw->name, i);
>   }
>   }
> - printk(KERN_INFO "%s: i2c scan done.\n", hdw->name);
> + pr_info("%s: i2c scan done.\n", hdw->name);
>  }
>  
>  static void pvr2_i2c_register_ir(struct pvr2_hdw *hdw)
> @@ -612,7 +610,7 @@ void pvr2_i2c_core_init(struct pvr2_hdw *hdw)
>  
>   /* However, deal with various special cases for 24xxx hardware. */
>   if (ir_mode[hdw->unit_number] == 0) {
> - printk(KERN_INFO "%s: IR disabled\n",hdw->name);
> + pr_info("%s: IR disabled\n", hdw->name);
>   hdw->i2c_func[0x18] = i2c_black_hole;
>   } 

Re: [ANN] Draft Agenda for the media summit on Thursday Oct 25th in Edinburgh

[Helen Koike]

Hi Hans and Mauro,


On 9/24/18 11:42 AM, Hans Verkuil wrote:
> Hi all,
> 
> We are organizing a media mini-summit on Thursday October 25th in
> Edinburgh, Edinburgh International Conference Centre.
> 
> If you plan to attend, please let Mauro know. It is open for all, but
> we have a limited number of seats.

I believe I also I selected attendance when registering for the
conference. Please add my name too.

> 
> Name of the room for the summit: TBD
> 
> Currently known attendees (please add/remove names as needed):
> 
> Sakari Ailus 
> Mauro Carvalho Chehab 
> Ezequiel Garcia 
> Michael Ira Krufky 
> Laurent Pinchart 
> Ricardo Ribalda Delgado 
> Hans Verkuil 
> Sean Young 
> 
> Agenda (First draft!)
> =
> 
> General remarks: the given start/end times for the various topics are
> approximate since it is always hard to predict how long a discussion will 
> take.
> If people are attending other summits and those conflict with specific media
> topics they want to be part of, then let me know and we can rearrange the
> schedule to (hopefully) accommodate that.
> 
> 9:00-9:15: Introduction (Hans Verkuil)
> 
> 9:15-9:30: Status of the HDMI CEC kernel support (Hans Verkuil)
>   Give a quick overview of the status: what has been merged, what is
>   still pending, what is under development.
> 
> 9:30-9:45: Save/restore controls from MTD (Ricardo Ribalda Delgado)
>   Industrial/Scientific sensors usually come with very extensive
>   calibration information such as: per column gain, list of dead
>   pixels, temperature sensor offset... etc
> 
>   We are saving that information on an flash device that is located
>   by the sensor.
> 
>   Show how we are integrating that calibration flash with v4l2-ctrl.
>   And if this feature is useful for someone else and upstream it.
> 
> 9:45-11:00: Complex Cameras (Mauro Carvalho Chehab)
>   I expect that we could have something to discuss there about complex
>   cameras. So, I'd reserve a 50 mins slot for it.
> 
>   The idea is to discuss about the undergoing work with complex camera
>   development is happening.
> 
>   As we're working to merge request API, another topic for discussion
>   is how to add support for requests on it (or on a separate but related
>   library).
> 
> 11:00-11:15: Break
> 
> 11:15-12:00: Automated Testing (Ezequiel Garcia)
>   There is a lot of discussion going on around testing,
>   so it's a good opportunity for us to talk about our
>   current testing infrastructure.
> 
>   We are already doing a good job with v4l2-compliance.
>   Can we do more?
> 
> Lunch
> 
> 13:30-14:30: Stateless Codec userspace (Hans Verkuil)
>   Support for stateless codecs and Request API should be merged for
>   4.20, and the next step is to discuss how to organize the userspace
>   support.
> 
>   Hopefully by the time the media summit starts we'll have some better
>   ideas of what we want in this area.
> 
> 14:30-15:15: Which ioctls should be replaced with better versions? (Hans 
> Verkuil)
>   Some parts of the V4L2 API are awkward to use and I think it would be
>   a good idea to look at possible candidates for that.
> 
>   Examples are the ioctls that use struct v4l2_buffer: the multiplanar 
> support is
>   really horrible, and writing code to support both single and 
> multiplanar is hard.
>   We are also running out of fields and the timeval isn't y2038 compliant.
> 
>   A proof-of-concept is here:
> 
>   
> https://git.linuxtv.org/hverkuil/media_tree.git/commit/?h=v4l2-buffer=a95549df06d9900f3559afdbb9da06bd4b22d1f3
> 
>   It's a bit old, but it gives a good impression of what I have in mind.
> 
>   Another candidate is 
> VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL/VIDIOC_ENUM_FRAMEINTERVALS:
>   expressing frame intervals as a fraction is really awkward and so is 
> the fact
>   that the subdev and 'normal' ioctls are not the same.
> 
>   Discuss what possible other ioctls are candidates for a refresh.
> 
> 15:15-15:30: Break
> 
> 15:30-16:00: Discuss the media development process
>   Since we are all here, discuss any issues there may be with the media
>   subsystem development process. Anything to improve?
> 
> 16:00-16:15: Wrap up
>   Create action items (and who will take care of them) if needed.
>   Summarize and conclude the day.
> 
> End of the day: Key Signing Party
> 
> Regards,
> 
>   Hans
> 

Thanks
Helen

Re: Question regarding optimizing pipeline in Vimc

[Helen Koike]

On 8/22/18 3:49 AM, Hans Verkuil wrote:
> On 08/22/2018 05:35 AM, Helen Koike wrote:
>> Hello,
>>
>> One of the discussions we had when developing Vimc, was regarding
>> optimizing image generation.
>> The ideia was to generate the images directly in the capture instead
>> of propagating through the pipeline (to make things faster).
>> But my question is: if this optimization is on, and if there is a
>> greyscaler filter in the middle of the pipeline, do you expect to see
>> a grey image with this optimization?
> 
> Yes.
> 
>> Or if we just generate a dummy
>> image (with the right size format) at the end of the pipeline, would
>> it be ok? (I am asking because it doesn't sound that simple to
>> propagate the image transformation made by each entity in the pipe)
> 
> No, that would not be OK.
> 
> My basic idea was that you use a TPG state structure that contains the
> desired output: the sensor starts with e.g. 720p using some bayer pixelformat,
> the debayer module replaces the pixelformat with e.g. PIX_FMT_RGB32, a
> grayscale filter replaces it with PI_FMT_GREY, and that's what the TPG for the
> video device eventually will use to generate the video.
> 
> This assumes of course that all the vimc blocks only do operations that can
> be handled by the TPG. Depending on what the blocks will do the TPG might need
> to be extended if a feature is missing.
> 
> Regards,
> 
>   Hans
> 
>> Or do you have any other thing in mind?
>>
>> Thanks
>> Helen
>>
> 

Thanks Hans,

We'll be working on that soon.

Helen

Question regarding optimizing pipeline in Vimc

[Helen Koike]

Hello,

One of the discussions we had when developing Vimc, was regarding
optimizing image generation.
The ideia was to generate the images directly in the capture instead
of propagating through the pipeline (to make things faster).
But my question is: if this optimization is on, and if there is a
greyscaler filter in the middle of the pipeline, do you expect to see
a grey image with this optimization? Or if we just generate a dummy
image (with the right size format) at the end of the pipeline, would
it be ok? (I am asking because it doesn't sound that simple to
propagate the image transformation made by each entity in the pipe)
Or do you have any other thing in mind?

Thanks
Helen

Re: [PATCH] vimc: use correct subdev functions

[Helen Koike]

Hi Hans,

On 02/07/2018 03:06 PM, Hans Verkuil wrote:

Instead of calling everything a MEDIA_ENT_F_ATV_DECODER, pick the
correct functions for these blocks.


Nice, thanks for the patch



Signed-off-by: Hans Verkuil <hans.verk...@cisco.com>



Acked-by: Helen Koike <helen.ko...@collabora.com>


---
diff --git a/drivers/media/platform/vimc/vimc-debayer.c 
b/drivers/media/platform/vimc/vimc-debayer.c
index 4d663e89d33f..6e10b63ba9ec 100644
--- a/drivers/media/platform/vimc/vimc-debayer.c
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -533,7 +533,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct 
device *master,
/* Initialize ved and sd */
ret = vimc_ent_sd_register(>ved, >sd, v4l2_dev,
   pdata->entity_name,
-  MEDIA_ENT_F_ATV_DECODER, 2,
+  MEDIA_ENT_F_PROC_VIDEO_PIXEL_ENC_CONV, 2,
   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
   MEDIA_PAD_FL_SOURCE},
   _deb_ops);
diff --git a/drivers/media/platform/vimc/vimc-scaler.c 
b/drivers/media/platform/vimc/vimc-scaler.c
index e1602e0bc230..e583ec7a91da 100644
--- a/drivers/media/platform/vimc/vimc-scaler.c
+++ b/drivers/media/platform/vimc/vimc-scaler.c
@@ -395,7 +395,7 @@ static int vimc_sca_comp_bind(struct device *comp, struct 
device *master,
/* Initialize ved and sd */
ret = vimc_ent_sd_register(>ved, >sd, v4l2_dev,
   pdata->entity_name,
-  MEDIA_ENT_F_ATV_DECODER, 2,
+  MEDIA_ENT_F_PROC_VIDEO_SCALER, 2,
   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
   MEDIA_PAD_FL_SOURCE},
   _sca_ops);
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 457e211514c6..7d9fa9ccdb0e 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -378,7 +378,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct 
device *master,
/* Initialize ved and sd */
ret = vimc_ent_sd_register(>ved, >sd, v4l2_dev,
   pdata->entity_name,
-  MEDIA_ENT_F_ATV_DECODER, 1,
+  MEDIA_ENT_F_CAM_SENSOR, 1,
   (const unsigned long[1]) 
{MEDIA_PAD_FL_SOURCE},
   _sen_ops);
if (ret)

Re: [PATCH] v4l2-subdev: without controls return -ENOTTY

[Helen Koike]

Hi Hans

On 02/02/2018 11:05 AM, Hans Verkuil wrote:

If the subdev did not define any controls, then return -ENOTTY if
userspace attempts to call these ioctls.

The control framework functions will return -EINVAL, not -ENOTTY if
vfh->ctrl_handler is NULL.

Several of these framework functions are also called directly from
drivers, so I don't want to change the error code there.


Right, I see, thanks for the patch



Found with vimc and v4l2-compliance.

Signed-off-by: Hans Verkuil <hans.verk...@cisco.com>


Acked-by: Helen Koike <helen.ko...@collabora.com>


---
diff --git a/drivers/media/v4l2-core/v4l2-subdev.c 
b/drivers/media/v4l2-core/v4l2-subdev.c
index 43fefa73e0a3..be7a19272614 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -187,27 +187,43 @@ static long subdev_do_ioctl(struct file *file, unsigned 
int cmd, void *arg)

switch (cmd) {
case VIDIOC_QUERYCTRL:
+   if (!vfh->ctrl_handler)
+   return -ENOTTY;
return v4l2_queryctrl(vfh->ctrl_handler, arg);

case VIDIOC_QUERY_EXT_CTRL:
+   if (!vfh->ctrl_handler)
+   return -ENOTTY;
return v4l2_query_ext_ctrl(vfh->ctrl_handler, arg);

case VIDIOC_QUERYMENU:
+   if (!vfh->ctrl_handler)
+   return -ENOTTY;
return v4l2_querymenu(vfh->ctrl_handler, arg);

case VIDIOC_G_CTRL:
+   if (!vfh->ctrl_handler)
+   return -ENOTTY;
return v4l2_g_ctrl(vfh->ctrl_handler, arg);

case VIDIOC_S_CTRL:
+   if (!vfh->ctrl_handler)
+   return -ENOTTY;
return v4l2_s_ctrl(vfh, vfh->ctrl_handler, arg);

case VIDIOC_G_EXT_CTRLS:
+   if (!vfh->ctrl_handler)
+   return -ENOTTY;
return v4l2_g_ext_ctrls(vfh->ctrl_handler, arg);

case VIDIOC_S_EXT_CTRLS:
+   if (!vfh->ctrl_handler)
+   return -ENOTTY;
return v4l2_s_ext_ctrls(vfh, vfh->ctrl_handler, arg);

case VIDIOC_TRY_EXT_CTRLS:
+   if (!vfh->ctrl_handler)
+   return -ENOTTY;
return v4l2_try_ext_ctrls(vfh->ctrl_handler, arg);

case VIDIOC_DQEVENT:

Re: [PATCH] vimc: fix control event handling

[Helen Koike]

Hi Hans,

Thanks for the patch

On 02/02/2018 11:00 AM, Hans Verkuil wrote:

The sensor subdev didn't handle control events. Add support for this.
Found with v4l2-compliance.

Signed-off-by: Hans Verkuil <hans.verk...@cisco.com>


Acked-by: Helen Koike <helen.ko...@collabora.com>


---
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 9d63c84a9876..617415c224fe 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -434,7 +434,9 @@ int vimc_ent_sd_register(struct vimc_ent_device *ved,
v4l2_set_subdevdata(sd, ved);

/* Expose this subdev to user space */
-   sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+   sd->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+   if (sd->ctrl_handler)
+   sd->flags |= V4L2_SUBDEV_FL_HAS_EVENTS;

/* Initialize the media entity */
ret = media_entity_pads_init(>entity, num_pads, ved->pads);
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 457e211514c6..54184cd9e0ff 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -23,6 +23,7 @@
  #include 
  #include 
  #include 
+#include 
  #include 
  #include 

@@ -284,11 +285,18 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int 
enable)
return 0;
  }

+static struct v4l2_subdev_core_ops vimc_sen_core_ops = {
+   .log_status = v4l2_ctrl_subdev_log_status,
+   .subscribe_event = v4l2_ctrl_subdev_subscribe_event,
+   .unsubscribe_event = v4l2_event_subdev_unsubscribe,
+};
+
  static const struct v4l2_subdev_video_ops vimc_sen_video_ops = {
.s_stream = vimc_sen_s_stream,
  };

  static const struct v4l2_subdev_ops vimc_sen_ops = {
+   .core = _sen_core_ops,
.pad = _sen_pad_ops,
.video = _sen_video_ops,
  };

Re: [PATCH] vimc: add test_pattern and h/vflip controls to the sensor

[Helen Koike]

Hi Hans,

On 2017-11-06 08:19 AM, Hans Verkuil wrote:
> Hi Helen,
> 
> On 09/27/2017 08:30 PM, Helen Koike wrote:
>> Hi Hans,
>>
>> Thanks for your patch and sorry for my late reply.
> 
> Sorry for my late reply to your reply :-)
> 
>> Please see my comments and questions below
>>
>> On 2017-07-28 07:23 AM, Hans Verkuil wrote:
>>> Add support for the test_pattern control and the h/vflip controls.
>>>
>>> This makes it possible to switch to more interesting test patterns and to
>>> test control handling in v4l-subdevs.
>>>
>>> There are more tpg-related controls that can be added, but this is a good
>>> start.
>>>
>>> Signed-off-by: Hans Verkuil <hans.verk...@cisco.com>
>>> ---
>>> diff --git a/drivers/media/platform/vimc/vimc-common.h 
>>> b/drivers/media/platform/vimc/vimc-common.h
>>> index dca528a316e7..2e9981b18166 100644
>>> --- a/drivers/media/platform/vimc/vimc-common.h
>>> +++ b/drivers/media/platform/vimc/vimc-common.h
>>> @@ -22,6 +22,11 @@
>>>  #include 
>>>  #include 
>>>
>>> +/* VIMC-specific controls */
>>> +#define VIMC_CID_VIMC_BASE (0x00f0 | 0xf000)
>>> +#define VIMC_CID_VIMC_CLASS(0x00f0 | 1)
>>
>> Why this values, shouldn't we use a derivative from
>> V4L2_CID_PRIVATE_BASE for custom controls? Or can we use random values?
> 
> The values are taken from vivid which uses the same scheme. These controls
> deal with how the virtual driver emulates things, and I prefer not to make
> these control IDs part of the public API so we can be a bit more flexible in
> the future. It's a design choice which worked well for vivid.
> 
>>
>>> +#define VIMC_CID_TEST_PATTERN  (VIMC_CID_VIMC_BASE + 0)
>>> +
>>>  #define VIMC_FRAME_MAX_WIDTH 4096
>>>  #define VIMC_FRAME_MAX_HEIGHT 2160
>>>  #define VIMC_FRAME_MIN_WIDTH 16
>>> diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
>>> b/drivers/media/platform/vimc/vimc-sensor.c
>>> index 615c2b18dcfc..532097566b27 100644
>>> --- a/drivers/media/platform/vimc/vimc-sensor.c
>>> +++ b/drivers/media/platform/vimc/vimc-sensor.c
>>> @@ -22,6 +22,7 @@
>>>  #include 
>>>  #include 
>>>  #include 
>>> +#include 
>>>  #include 
>>>  #include 
>>>
>>> @@ -38,6 +39,7 @@ struct vimc_sen_device {
>>> u8 *frame;
>>> /* The active format */
>>> struct v4l2_mbus_framefmt mbus_format;
>>> +   struct v4l2_ctrl_handler hdl;
>>>  };
>>>
>>>  static const struct v4l2_mbus_framefmt fmt_default = {
>>> @@ -291,6 +293,31 @@ static const struct v4l2_subdev_ops vimc_sen_ops = {
>>> .video = _sen_video_ops,
>>>  };
>>>
>>> +static int vimc_sen_s_ctrl(struct v4l2_ctrl *ctrl)
>>> +{
>>> +   struct vimc_sen_device *vsen =
>>> +   container_of(ctrl->handler, struct vimc_sen_device, hdl);
>>> +
>>> +   switch (ctrl->id) {
>>> +   case VIMC_CID_TEST_PATTERN:
>>> +   tpg_s_pattern(>tpg, ctrl->val);
>>> +   break;
>>> +   case V4L2_CID_HFLIP:
>>> +   tpg_s_hflip(>tpg, ctrl->val);
>>> +   break;
>>> +   case V4L2_CID_VFLIP:
>>> +   tpg_s_vflip(>tpg, ctrl->val);
>>> +   break;
>>> +   default:
>>> +   return -EINVAL;
>>> +   }
>>> +   return 0;
>>> +}
>>> +
>>> +static const struct v4l2_ctrl_ops vimc_sen_ctrl_ops = {
>>> +   .s_ctrl = vimc_sen_s_ctrl,
>>> +};
>>> +
>>>  static void vimc_sen_comp_unbind(struct device *comp, struct device 
>>> *master,
>>>  void *master_data)
>>>  {
>>> @@ -299,10 +326,29 @@ static void vimc_sen_comp_unbind(struct device *comp, 
>>> struct device *master,
>>> container_of(ved, struct vimc_sen_device, ved);
>>>
>>> vimc_ent_sd_unregister(ved, >sd);
>>> +   v4l2_ctrl_handler_free(>hdl);
>>> tpg_free(>tpg);
>>> kfree(vsen);
>>>  }
>>>
>>> +/* Image Processing Controls */
>>> +static const struct v4l2_ctrl_config vimc_sen_ctrl_class = {
>>> +   .ops = _sen_ctrl_ops,
>>> +   .flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_WRITE_ONLY,
>>
>> I was wondering if it is really necessary to spec

Re: [PATCH] [media] vimc: Fix return value check in vimc_add_subdevs()

[Helen Koike]

Hello,


On 2017-10-11 08:16 AM, Wei Yongjun wrote:
> In case of error, the function platform_device_register_data() returns
> ERR_PTR() and never returns NULL. The NULL test in the return value check
> should be replaced with IS_ERR().
> 
> Signed-off-by: Wei Yongjun <weiyongj...@huawei.com>
> ---
>  drivers/media/platform/vimc/vimc-core.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/media/platform/vimc/vimc-core.c 
> b/drivers/media/platform/vimc/vimc-core.c
> index 51c0eee..fe088a9 100644
> --- a/drivers/media/platform/vimc/vimc-core.c
> +++ b/drivers/media/platform/vimc/vimc-core.c
> @@ -267,11 +267,12 @@ static struct component_match *vimc_add_subdevs(struct 
> vimc_device *vimc)
>   PLATFORM_DEVID_AUTO,
>   ,
>   sizeof(pdata));
> - if (!vimc->subdevs[i]) {
> + if (IS_ERR(vimc->subdevs[i])) {
> + match = ERR_CAST(vimc->subdevs[i]);
>   while (--i >= 0)
>   platform_device_unregister(vimc->subdevs[i]);
>  
> - return ERR_PTR(-ENOMEM);
> + return match;
>   }
>  
>   component_match_add(>pdev.dev, , vimc_comp_compare,
> 
> 
> 

Nice catch, thanks, looks good to me

Acked-by: Helen Koike <helen.ko...@collabora.com>

Re: [media] vimc: API proposal, configuring the topology from user space

[Helen Koike]

Hi Hans,

Thanks for your review.

On 2017-07-28 10:39 AM, Hans Verkuil wrote:
> Hi Helen,
> 
> Finally after way too long I found some time to review this. See my comments
> below.
> 
> On 04/11/2017 12:53 AM, Helen Koike wrote:
>>
>> Hi,
>>
>> Continuing the discussion about the API of the vimc driver, I made some 
>> changes
>> based on the previous comments, please see below and let me know your 
>> opinion about it.
>>
>> Helen
>>
>> /***
>> Configfs considerations:
>> /
>> Informal definitions:
>>  subsystem: the root driver folder in user space (/configfs/vimc)
>>  item: aka a folder in user space
>>  attributes: aka files in the folder
>>  group: aka a folder that can contain subfolders (parent and child 
>> relation)
>>  default group: aka a subfolder that is created automatically when the 
>> "parent" folder is created
>>  it is not considered a child in terms of rmdir
>>
>> * Performing rmdir in a group will fail if it contain children that are 
>> not default groups, i.e, if the
>> folder contain subfolders that are default group, then it can be removed 
>> with rmdir, if the
>> subfolders were created with mkdir, then rmdir in the parent will fail.
>>
>> * Configfs has the notion of committable item but it is not implemented 
>> yet. A committable item is an item
>> that can be in one of two parent folders called: live and pending. The 
>> idea is to create and modify the item
>> in the pending directory and then to move the item through a rename to 
>> the live directory where
>> it can't be modified. This seems to be a nice feature for vimc, but as 
>> it is not available yet the
>> proposal below won't be based on this.
>>
>> * Groups can be dynamically created/destroyed by the driver whenever it 
>> wants. Afaik attributes can only
>> be created when the group or item is created and symlinks can only be 
>> create from user space, i.e, the
>> driver don't know how to create/destroy attributes or symlinks in anytime.
>>
>> /***
>> The API:
>> /
>>
>> In short, a topology like this one: http://goo.gl/Y7eUfu
>> Would look like this filesystem tree: https://goo.gl/mEOmOf
> 
> This mentions 'Yellow' lines, but since you dropped symlinks these no
> longer exist. You probably need to update the legend.
> 

Yes, thanks for noticing, I'll update when including it in the docs of
the API.

>>
>> v3 core changes:
>> - I removed the use of symlinks as I wans't able to see how to do it nicely.
>> - I use the names of the folders created by user space to retrieve 
>> information at mkdir time
>> - hotplug file in each entity
>> - hotplug file in each device
>> - reset file in each device
>>
>> * The /configfs/vimc subsystem
>> empty when the driver is loaded
> 
> I'm not sure about that. I think it would make sense that vimc when loaded
> would make one instance, unless otherwise told via a module option.

I don't think it really matters, the instance would be in unplugged
state anyway and it would need to be configured, but sure, we can add a
n_devs param as you proposed, no problem.

> 
> Something like this (taken from vivid):
> 
> parm:   n_devs: number of driver instances to create (uint)
> 
> By default this is 1, but can also be 0, 2, 3, etc.
> 
>>
>> * Create a device
>> Userspace can create a new vimc device with:
>>
>>  $ mkdir /configfs/vimc/any_name
>>  Example:
>>  $ mkdir /configfs/vimc/vimc0
>>  $ ls -l /configfs/vimc/vimc0
>>  hotplug
>>  reset
>>  entities/
>>  links/
>>
>> entities/ and links/ folder are default groups, thus they don't prevent 
>> rmdir vimc0/, but
>> rmdir will fail if it has any child inside entities/ or links/.
>> hotplug is used to plug and unplug the device, it can read "plugged" or 
>> "unplugged" and user can
>> write "plug" or "unplug" to change its state.
> 
> I would also support writing "plugged" and "unplugged". I.e. support both 
> variants.
> 

So it matches the values that you can read from the file, thanks for
this suggestion.

>> Changing hotplug state will never fail as the configfs tree will always 
>> be in a valid state.
>> reset is used to easily destroy all the topology without the need to 
>> walk through all the children
>> to perform 

Re: [PATCH] vimc: add test_pattern and h/vflip controls to the sensor

[Helen Koike]

Hi Hans,

Thanks for your patch and sorry for my late reply.
Please see my comments and questions below

On 2017-07-28 07:23 AM, Hans Verkuil wrote:
> Add support for the test_pattern control and the h/vflip controls.
> 
> This makes it possible to switch to more interesting test patterns and to
> test control handling in v4l-subdevs.
> 
> There are more tpg-related controls that can be added, but this is a good
> start.
> 
> Signed-off-by: Hans Verkuil 
> ---
> diff --git a/drivers/media/platform/vimc/vimc-common.h 
> b/drivers/media/platform/vimc/vimc-common.h
> index dca528a316e7..2e9981b18166 100644
> --- a/drivers/media/platform/vimc/vimc-common.h
> +++ b/drivers/media/platform/vimc/vimc-common.h
> @@ -22,6 +22,11 @@
>  #include 
>  #include 
> 
> +/* VIMC-specific controls */
> +#define VIMC_CID_VIMC_BASE   (0x00f0 | 0xf000)
> +#define VIMC_CID_VIMC_CLASS  (0x00f0 | 1)

Why this values, shouldn't we use a derivative from
V4L2_CID_PRIVATE_BASE for custom controls? Or can we use random values?

> +#define VIMC_CID_TEST_PATTERN(VIMC_CID_VIMC_BASE + 0)
> +
>  #define VIMC_FRAME_MAX_WIDTH 4096
>  #define VIMC_FRAME_MAX_HEIGHT 2160
>  #define VIMC_FRAME_MIN_WIDTH 16
> diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
> b/drivers/media/platform/vimc/vimc-sensor.c
> index 615c2b18dcfc..532097566b27 100644
> --- a/drivers/media/platform/vimc/vimc-sensor.c
> +++ b/drivers/media/platform/vimc/vimc-sensor.c
> @@ -22,6 +22,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
> 
> @@ -38,6 +39,7 @@ struct vimc_sen_device {
>   u8 *frame;
>   /* The active format */
>   struct v4l2_mbus_framefmt mbus_format;
> + struct v4l2_ctrl_handler hdl;
>  };
> 
>  static const struct v4l2_mbus_framefmt fmt_default = {
> @@ -291,6 +293,31 @@ static const struct v4l2_subdev_ops vimc_sen_ops = {
>   .video = _sen_video_ops,
>  };
> 
> +static int vimc_sen_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> + struct vimc_sen_device *vsen =
> + container_of(ctrl->handler, struct vimc_sen_device, hdl);
> +
> + switch (ctrl->id) {
> + case VIMC_CID_TEST_PATTERN:
> + tpg_s_pattern(>tpg, ctrl->val);
> + break;
> + case V4L2_CID_HFLIP:
> + tpg_s_hflip(>tpg, ctrl->val);
> + break;
> + case V4L2_CID_VFLIP:
> + tpg_s_vflip(>tpg, ctrl->val);
> + break;
> + default:
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> +static const struct v4l2_ctrl_ops vimc_sen_ctrl_ops = {
> + .s_ctrl = vimc_sen_s_ctrl,
> +};
> +
>  static void vimc_sen_comp_unbind(struct device *comp, struct device *master,
>void *master_data)
>  {
> @@ -299,10 +326,29 @@ static void vimc_sen_comp_unbind(struct device *comp, 
> struct device *master,
>   container_of(ved, struct vimc_sen_device, ved);
> 
>   vimc_ent_sd_unregister(ved, >sd);
> + v4l2_ctrl_handler_free(>hdl);
>   tpg_free(>tpg);
>   kfree(vsen);
>  }
> 
> +/* Image Processing Controls */
> +static const struct v4l2_ctrl_config vimc_sen_ctrl_class = {
> + .ops = _sen_ctrl_ops,
> + .flags = V4L2_CTRL_FLAG_READ_ONLY | V4L2_CTRL_FLAG_WRITE_ONLY,

I was wondering if it is really necessary to specify the ops and flags
in the class, as this is seems to me a meta control for the other
controls to be grouped in a class.

> + .id = VIMC_CID_VIMC_CLASS,
> + .name = "VIMC Controls",
> + .type = V4L2_CTRL_TYPE_CTRL_CLASS,
> +};
> +
> +static const struct v4l2_ctrl_config vimc_sen_ctrl_test_pattern = {
> + .ops = _sen_ctrl_ops,
> + .id = VIMC_CID_TEST_PATTERN,
> + .name = "Test Pattern",
> + .type = V4L2_CTRL_TYPE_MENU,
> + .max = TPG_PAT_NOISE,
> + .qmenu = tpg_pattern_strings,
> +};
> +
>  static int vimc_sen_comp_bind(struct device *comp, struct device *master,
> void *master_data)
>  {
> @@ -316,6 +362,20 @@ static int vimc_sen_comp_bind(struct device *comp, 
> struct device *master,
>   if (!vsen)
>   return -ENOMEM;
> 
> + v4l2_ctrl_handler_init(>hdl, 4);
> +
> + v4l2_ctrl_new_custom(>hdl, _sen_ctrl_class, NULL);
> + v4l2_ctrl_new_custom(>hdl, _sen_ctrl_test_pattern, NULL);
> + v4l2_ctrl_new_std(>hdl, _sen_ctrl_ops,
> + V4L2_CID_VFLIP, 0, 1, 1, 0);
> + v4l2_ctrl_new_std(>hdl, _sen_ctrl_ops,
> + V4L2_CID_HFLIP, 0, 1, 1, 0);

Shouldn't we test the return values of the above functions? Or maybe not
because we should know what we are doing and this doesn't depend on the
user space.

> + vsen->sd.ctrl_handler = >hdl;
> + if (vsen->hdl.error) {
> + ret = vsen->hdl.error;
> + goto err_free_vsen;
> + }
> +
>   /* Initialize ved and sd */
>   ret = vimc_ent_sd_register(>ved, >sd, v4l2_dev,
>  

Re: [PATCH] [media] vimc: set id_table for platform drivers

[Helen Koike]

Hi,

Thanks for the patch

On 2017-07-14 05:58 AM, Javier Martinez Canillas wrote:

The vimc platform drivers define a platform device ID table but these
are not set to the .id_table field in the platform driver structure.

So the platform device ID table is only used to fill the aliases in
the module but are not used for matching (works because the platform
subsystem fallbacks to the driver's name if no .id_table is set).

But this also means that the platform device ID table isn't used if
the driver is built-in, which leads to the following build warning:

This causes the following build warnings when the driver is built-in:

drivers/media/platform/vimc//vimc-capture.c:528:40: warning: 
‘vimc_cap_driver_ids’ defined but not used [-Wunused-const-variable=]
  static const struct platform_device_id vimc_cap_driver_ids[] = {
 ^~~
drivers/media/platform/vimc//vimc-debayer.c:588:40: warning: 
‘vimc_deb_driver_ids’ defined but not used [-Wunused-const-variable=]
  static const struct platform_device_id vimc_deb_driver_ids[] = {
 ^~~
drivers/media/platform/vimc//vimc-scaler.c:442:40: warning: 
‘vimc_sca_driver_ids’ defined but not used [-Wunused-const-variable=]
  static const struct platform_device_id vimc_sca_driver_ids[] = {
 ^~~
drivers/media/platform/vimc//vimc-sensor.c:376:40: warning: 
‘vimc_sen_driver_ids’ defined but not used [-Wunused-const-variable=]
  static const struct platform_device_id vimc_sen_driver_ids[] = {
 ^~~

Reported-by: Mauro Carvalho Chehab <mche...@s-opensource.com>
Suggested-by: Sakari Ailus <sakari.ai...@iki.fi>
Signed-off-by: Javier Martinez Canillas <javi...@redhat.com>


Seems good to me.

Reviewed-by: Helen Koike <helen.ko...@collabora.com>



---

  drivers/media/platform/vimc/vimc-capture.c | 15 ---
  drivers/media/platform/vimc/vimc-debayer.c | 15 ---
  drivers/media/platform/vimc/vimc-scaler.c  | 15 ---
  drivers/media/platform/vimc/vimc-sensor.c  | 15 ---
  4 files changed, 32 insertions(+), 28 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 14cb32e21130..88a1e5670c72 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -517,21 +517,22 @@ static int vimc_cap_remove(struct platform_device *pdev)
return 0;
  }
  
+static const struct platform_device_id vimc_cap_driver_ids[] = {

+   {
+   .name   = VIMC_CAP_DRV_NAME,
+   },
+   { }
+};
+
  static struct platform_driver vimc_cap_pdrv = {
.probe  = vimc_cap_probe,
.remove = vimc_cap_remove,
+   .id_table   = vimc_cap_driver_ids,
.driver = {
.name   = VIMC_CAP_DRV_NAME,
},
  };
  
-static const struct platform_device_id vimc_cap_driver_ids[] = {

-   {
-   .name   = VIMC_CAP_DRV_NAME,
-   },
-   { }
-};
-
  module_platform_driver(vimc_cap_pdrv);
  
  MODULE_DEVICE_TABLE(platform, vimc_cap_driver_ids);

diff --git a/drivers/media/platform/vimc/vimc-debayer.c 
b/drivers/media/platform/vimc/vimc-debayer.c
index 35b15bd4d61d..033a131f67af 100644
--- a/drivers/media/platform/vimc/vimc-debayer.c
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -577,21 +577,22 @@ static int vimc_deb_remove(struct platform_device *pdev)
return 0;
  }
  
+static const struct platform_device_id vimc_deb_driver_ids[] = {

+   {
+   .name   = VIMC_DEB_DRV_NAME,
+   },
+   { }
+};
+
  static struct platform_driver vimc_deb_pdrv = {
.probe  = vimc_deb_probe,
.remove = vimc_deb_remove,
+   .id_table   = vimc_deb_driver_ids,
.driver = {
.name   = VIMC_DEB_DRV_NAME,
},
  };
  
-static const struct platform_device_id vimc_deb_driver_ids[] = {

-   {
-   .name   = VIMC_DEB_DRV_NAME,
-   },
-   { }
-};
-
  module_platform_driver(vimc_deb_pdrv);
  
  MODULE_DEVICE_TABLE(platform, vimc_deb_driver_ids);

diff --git a/drivers/media/platform/vimc/vimc-scaler.c 
b/drivers/media/platform/vimc/vimc-scaler.c
index fe77505d2679..0a3e086e12f3 100644
--- a/drivers/media/platform/vimc/vimc-scaler.c
+++ b/drivers/media/platform/vimc/vimc-scaler.c
@@ -431,21 +431,22 @@ static int vimc_sca_remove(struct platform_device *pdev)
return 0;
  }
  
+static const struct platform_device_id vimc_sca_driver_ids[] = {

+   {
+   .name   = VIMC_SCA_DRV_NAME,
+   },
+   { }
+};
+
  static struct platform_driver vimc_sca_pdrv = {
.probe  = vimc_sca_probe,
.remove = vimc_sca_remove,
+   .id_table

Re: [PATCH v2] [media] v4l2: add V4L2_CAP_IO_MC

[Helen Koike]

Hi Hans,

Thanks for reviewing this

On 2017-06-19 08:15 AM, Hans Verkuil wrote:

On 06/14/2017 06:50 AM, Helen Koike wrote:

Add V4L2_CAP_IO_MC to be used in struct v4l2_capability to indicate that
input and output are controlled by the Media Controller instead of V4L2
API.
When this flag is set, ioctls for get, set and enum input and outputs
are automatically enabled and programmed to call helper function.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2::
- replace the type by capability
- erase V4L2_INPUT_TYPE_DEFAULT
- also consider output
- plug helpers in the ops automatically so drivers doesn't need
to set it by hand
- update docs
- commit message and title
---
  Documentation/media/uapi/v4l/vidioc-querycap.rst |  3 +
  Documentation/media/videodev2.h.rst.exceptions   |  1 +
  drivers/media/v4l2-core/v4l2-dev.c   | 35 +++--
  drivers/media/v4l2-core/v4l2-ioctl.c | 91
++--
  include/uapi/linux/videodev2.h   |  2 +
  5 files changed, 120 insertions(+), 12 deletions(-)

diff --git a/Documentation/media/uapi/v4l/vidioc-querycap.rst
b/Documentation/media/uapi/v4l/vidioc-querycap.rst
index 12e0d9a..2bd1223 100644
--- a/Documentation/media/uapi/v4l/vidioc-querycap.rst
+++ b/Documentation/media/uapi/v4l/vidioc-querycap.rst
@@ -252,6 +252,9 @@ specification the ioctl returns an ``EINVAL``
error code.
  * - ``V4L2_CAP_TOUCH``
- 0x1000
- This is a touch device.
+* - ``V4L2_CAP_IO_MC``
+  - 0x2000
+  - This device has its inputs and outputs controller by the
Media Controller


controller -> controlled


Sorry, I'll remember to use a spell checker next time



But I would rephrase this a bit:

- The inputs and/or outputs of this device are controlled by the
Media Controller
  (see: ).



Sure, much better.
In this document, almost all the flags start with "The device 
supports..." or "The device has...", I was thinking to do something similar.



  * - ``V4L2_CAP_DEVICE_CAPS``
- 0x8000
- The driver fills the ``device_caps`` field. This capability can
diff --git a/Documentation/media/videodev2.h.rst.exceptions
b/Documentation/media/videodev2.h.rst.exceptions
index a5cb0a8..0b48cd0 100644
--- a/Documentation/media/videodev2.h.rst.exceptions
+++ b/Documentation/media/videodev2.h.rst.exceptions
@@ -159,6 +159,7 @@ replace define V4L2_CAP_ASYNCIO device-capabilities
  replace define V4L2_CAP_STREAMING device-capabilities
  replace define V4L2_CAP_DEVICE_CAPS device-capabilities
  replace define V4L2_CAP_TOUCH device-capabilities
+replace define V4L2_CAP_IO_MC device-capabilities
# V4L2 pix flags
  replace define V4L2_PIX_FMT_PRIV_MAGIC :c:type:`v4l2_pix_format`
diff --git a/drivers/media/v4l2-core/v4l2-dev.c
b/drivers/media/v4l2-core/v4l2-dev.c
index c647ba6..0f272fe 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -688,22 +688,34 @@ static void determine_valid_ioctls(struct
video_device *vdev)
  SET_VALID_IOCTL(ops, VIDIOC_G_STD, vidioc_g_std);
  if (is_rx) {
  SET_VALID_IOCTL(ops, VIDIOC_QUERYSTD, vidioc_querystd);
-SET_VALID_IOCTL(ops, VIDIOC_ENUMINPUT, vidioc_enum_input);
-SET_VALID_IOCTL(ops, VIDIOC_G_INPUT, vidioc_g_input);
-SET_VALID_IOCTL(ops, VIDIOC_S_INPUT, vidioc_s_input);
  SET_VALID_IOCTL(ops, VIDIOC_ENUMAUDIO, vidioc_enumaudio);
  SET_VALID_IOCTL(ops, VIDIOC_G_AUDIO, vidioc_g_audio);
  SET_VALID_IOCTL(ops, VIDIOC_S_AUDIO, vidioc_s_audio);
  SET_VALID_IOCTL(ops, VIDIOC_QUERY_DV_TIMINGS,
vidioc_query_dv_timings);
  SET_VALID_IOCTL(ops, VIDIOC_S_EDID, vidioc_s_edid);
+if (vdev->device_caps & V4L2_CAP_IO_MC) {
+set_bit(_IOC_NR(VIDIOC_ENUMINPUT), valid_ioctls);
+set_bit(_IOC_NR(VIDIOC_G_INPUT), valid_ioctls);
+set_bit(_IOC_NR(VIDIOC_S_INPUT), valid_ioctls);
+} else {
+SET_VALID_IOCTL(ops, VIDIOC_ENUMINPUT,
vidioc_enum_input);
+SET_VALID_IOCTL(ops, VIDIOC_G_INPUT, vidioc_g_input);
+SET_VALID_IOCTL(ops, VIDIOC_S_INPUT, vidioc_s_input);
+}
  }
  if (is_tx) {
-SET_VALID_IOCTL(ops, VIDIOC_ENUMOUTPUT, vidioc_enum_output);
-SET_VALID_IOCTL(ops, VIDIOC_G_OUTPUT, vidioc_g_output);
-SET_VALID_IOCTL(ops, VIDIOC_S_OUTPUT, vidioc_s_output);
  SET_VALID_IOCTL(ops, VIDIOC_ENUMAUDOUT, vidioc_enumaudout);
  SET_VALID_IOCTL(ops, VIDIOC_G_AUDOUT, vidioc_g_audout);
  SET_VALID_IOCTL(ops, VIDIOC_S_AUDOUT, vidioc_s_audout);
+if (vdev->device_caps & V4L2_CAP_IO_MC) {
+set_bit(_IOC_NR(VIDIOC_ENUMOUTPUT), valid_ioctls);
+set_bit(_IOC_NR(VIDIOC_G_OUTPUT), vali

[PATCH v5 04/12] [media] vimc: common: Add vimc_pipeline_s_stream helper

[Helen Koike]

Move the vimc_cap_pipeline_s_stream from the vimc-cap.c to vimc-common.c
as this core will be reused by other subdevices to activate the stream
in their directly connected nodes

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5: None
Changes in v4: None
Changes in v3:
[media] vimc: Add vimc_pipeline_s_stream in the core
- add it in vimc-common instead of vimc-core
- rename commit with "common" tag

Changes in v2:
[media] vimc: Add vimc_pipeline_s_stream in the core
- Use is_media_entity_v4l2_subdev instead of comparing with the old
entity->type
- Fix comments style
- add kernel-docs
- call s_stream across all sink pads


---
 drivers/media/platform/vimc/vimc-capture.c | 29 ++-
 drivers/media/platform/vimc/vimc-common.c  | 32 ++
 drivers/media/platform/vimc/vimc-common.h  | 11 ++
 3 files changed, 45 insertions(+), 27 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 9adb06d..93f6a09 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -132,31 +132,6 @@ static void vimc_cap_return_all_buffers(struct 
vimc_cap_device *vcap,
spin_unlock(>qlock);
 }
 
-static int vimc_cap_pipeline_s_stream(struct vimc_cap_device *vcap, int enable)
-{
-   struct v4l2_subdev *sd;
-   struct media_pad *pad;
-   int ret;
-
-   /* Start the stream in the subdevice direct connected */
-   pad = media_entity_remote_pad(>vdev.entity.pads[0]);
-
-   /*
-* if it is a raw node from vimc-core, there is nothing to activate
-* TODO: remove this when there are no more raw nodes in the
-* core and return error instead
-*/
-   if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
-   return 0;
-
-   sd = media_entity_to_v4l2_subdev(pad->entity);
-   ret = v4l2_subdev_call(sd, video, s_stream, enable);
-   if (ret && ret != -ENOIOCTLCMD)
-   return ret;
-
-   return 0;
-}
-
 static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
@@ -173,7 +148,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, 
unsigned int count)
}
 
/* Enable streaming from the pipe */
-   ret = vimc_cap_pipeline_s_stream(vcap, 1);
+   ret = vimc_pipeline_s_stream(>vdev.entity, 1);
if (ret) {
media_pipeline_stop(entity);
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
@@ -192,7 +167,7 @@ static void vimc_cap_stop_streaming(struct vb2_queue *vq)
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
 
/* Disable streaming from the pipe */
-   vimc_cap_pipeline_s_stream(vcap, 0);
+   vimc_pipeline_s_stream(>vdev.entity, 0);
 
/* Stop the media pipeline */
media_pipeline_stop(>vdev.entity);
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 3afbabd..f809a9d 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -220,6 +220,38 @@ struct media_pad *vimc_pads_init(u16 num_pads, const 
unsigned long *pads_flag)
return pads;
 }
 
+int vimc_pipeline_s_stream(struct media_entity *ent, int enable)
+{
+   struct v4l2_subdev *sd;
+   struct media_pad *pad;
+   unsigned int i;
+   int ret;
+
+   for (i = 0; i < ent->num_pads; i++) {
+   if (ent->pads[i].flags & MEDIA_PAD_FL_SOURCE)
+   continue;
+
+   /* Start the stream in the subdevice direct connected */
+   pad = media_entity_remote_pad(>pads[i]);
+
+   /*
+* if this is a raw node from vimc-core, then there is
+* nothing to activate
+* TODO: remove this when there are no more raw nodes in the
+* core and return error instead
+*/
+   if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
+   continue;
+
+   sd = media_entity_to_v4l2_subdev(pad->entity);
+   ret = v4l2_subdev_call(sd, video, s_stream, enable);
+   if (ret && ret != -ENOIOCTLCMD)
+   return ret;
+   }
+
+   return 0;
+}
+
 static const struct media_entity_operations vimc_ent_sd_mops = {
.link_validate = v4l2_subdev_link_validate,
 };
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 9ec361c..73e7e94 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -97,6 +97,17 @@ static inline void vimc_pads_cleanup(struct media_pad *pads)
 

[PATCH v5 07/12] [media] vimc: sen: Support several image formats

[Helen Koike]

Allow user space to change the image format as the frame size, the
media bus pixel format, colorspace, quantization, field YCbCr encoding
and the transfer function

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5: None
Changes in v4:
[media] vimc: sen: Support several image formats
- use vimc_colorimetry_clamp macro
- replace V4L2_COLORSPACE_SRGB by V4L2_COLORSPACE_DEFAULT in the
default format struct

Changes in v3:
[media] vimc: sen: Support several image formats
- remove support for V4L2_FIELD_ALTERNATE (left as TODO for now)
- clamp image size to an even dimension for height and width
- set default values for colorimetry using _DEFAULT macro
- reset all values of colorimetry to _DEFAULT if user tries to
set an invalid colorspace

Changes in v2:
[media] vimc: sen: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- add init_cfg to initialize try_fmt
- reorder code in vimc_sen_set_fmt
- allow user space to change all fields from struct v4l2_mbus_framefmt
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- merge with patch for the enum_mbus_code and enum_frame_size
- change commit message
- add vimc_pix_map_by_index
- rename MIN/MAX macros
- check set_fmt default parameters for quantization, colorspace 
...media] vimc: sen: Support several image formats


---
 drivers/media/platform/vimc/vimc-common.c |   8 ++
 drivers/media/platform/vimc/vimc-common.h |  12 +++
 drivers/media/platform/vimc/vimc-sensor.c | 130 +++---
 3 files changed, 121 insertions(+), 29 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 6ad77fd..b698055 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -144,6 +144,14 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
},
 };
 
+const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i)
+{
+   if (i >= ARRAY_SIZE(vimc_pix_map_list))
+   return NULL;
+
+   return _pix_map_list[i];
+}
+
 const struct vimc_pix_map *vimc_pix_map_by_code(u32 code)
 {
unsigned int i;
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 43483ee..fb3463c 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -22,6 +22,11 @@
 #include 
 #include 
 
+#define VIMC_FRAME_MAX_WIDTH 4096
+#define VIMC_FRAME_MAX_HEIGHT 2160
+#define VIMC_FRAME_MIN_WIDTH 16
+#define VIMC_FRAME_MIN_HEIGHT 16
+
 /**
  * struct vimc_colorimetry_clamp - Adjust colorimetry parameters
  *
@@ -139,6 +144,13 @@ static inline void vimc_pads_cleanup(struct media_pad 
*pads)
 int vimc_pipeline_s_stream(struct media_entity *ent, int enable);
 
 /**
+ * vimc_pix_map_by_index - get vimc_pix_map struct by its index
+ *
+ * @i: index of the vimc_pix_map struct in vimc_pix_map_list
+ */
+const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i);
+
+/**
  * vimc_pix_map_by_code - get vimc_pix_map struct by media bus code
  *
  * @code:  media bus format code defined by MEDIA_BUS_FMT_* macros
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 6386ac1..d4f9705 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -24,8 +24,6 @@
 
 #include "vimc-sensor.h"
 
-#define VIMC_SEN_FRAME_MAX_WIDTH 4096
-
 struct vimc_sen_device {
struct vimc_ent_device ved;
struct v4l2_subdev sd;
@@ -36,18 +34,39 @@ struct vimc_sen_device {
struct v4l2_mbus_framefmt mbus_format;
 };
 
+static const struct v4l2_mbus_framefmt fmt_default = {
+   .width = 640,
+   .height = 480,
+   .code = MEDIA_BUS_FMT_RGB888_1X24,
+   .field = V4L2_FIELD_NONE,
+   .colorspace = V4L2_COLORSPACE_DEFAULT,
+};
+
+static int vimc_sen_init_cfg(struct v4l2_subdev *sd,
+struct v4l2_subdev_pad_config *cfg)
+{
+   unsigned int i;
+
+   for (i = 0; i < sd->entity.num_pads; i++) {
+   struct v4l2_mbus_framefmt *mf;
+
+   mf = v4l2_subdev_get_try_format(sd, cfg, i);
+   *mf = fmt_default;
+   }
+
+   return 0;
+}
+
 static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
   struct v4l2_subdev_pad_config *cfg,
   struct v4l2_subdev_mbus_code_enum *code)
 {
-   struct vimc_sen_device *vsen =
-   container_of(sd, struct vimc_sen_device, sd);
+   const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
 
-   /* TODO: Add support for other codes */
-   if (co

[PATCH v5 06/12] [media] vimc: common: Add vimc_colorimetry_clamp

[Helen Koike]

Colorimetry value will always be checked in the same way. Adding a
helper macro for that

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5: None
Changes in v4:
[media] vimc: common: Add vimc_colorimetry_clamp
- this is a new patch in the series

Changes in v3: None
Changes in v2: None


---
 drivers/media/platform/vimc/vimc-common.h | 26 ++
 1 file changed, 26 insertions(+)

diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 60ebde2..43483ee 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -23,6 +23,32 @@
 #include 
 
 /**
+ * struct vimc_colorimetry_clamp - Adjust colorimetry parameters
+ *
+ * @fmt:   the pointer to struct v4l2_pix_format or
+ * struct v4l2_mbus_framefmt
+ *
+ * Entities must check if colorimetry given by the userspace is valid, if not
+ * then set them as DEFAULT
+ */
+#define vimc_colorimetry_clamp(fmt)\
+do {   \
+   if ((fmt)->colorspace == V4L2_COLORSPACE_DEFAULT\
+   || (fmt)->colorspace > V4L2_COLORSPACE_DCI_P3) {\
+   (fmt)->colorspace = V4L2_COLORSPACE_DEFAULT;\
+   (fmt)->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;  \
+   (fmt)->quantization = V4L2_QUANTIZATION_DEFAULT;\
+   (fmt)->xfer_func = V4L2_XFER_FUNC_DEFAULT;  \
+   }   \
+   if ((fmt)->ycbcr_enc > V4L2_YCBCR_ENC_SMPTE240M)\
+   (fmt)->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;  \
+   if ((fmt)->quantization > V4L2_QUANTIZATION_LIM_RANGE)  \
+   (fmt)->quantization = V4L2_QUANTIZATION_DEFAULT;\
+   if ((fmt)->xfer_func > V4L2_XFER_FUNC_SMPTE2084)\
+   (fmt)->xfer_func = V4L2_XFER_FUNC_DEFAULT;  \
+} while (0)
+
+/**
  * struct vimc_pix_map - maps media bus code with v4l2 pixel format
  *
  * @code:  media bus format code defined by MEDIA_BUS_FMT_* macros
-- 
2.7.4

[PATCH v5 01/12] [media] vimc: sen: Integrate the tpg on the sensor

[Helen Koike]

Initialize the test pattern generator on the sensor
Generate a colored bar image instead of a grey one

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5: None
Changes in v4: None
Changes in v3:
[media] vimc: sen: Integrate the tpg on the sensor
- Declare frame_size as a local variable
- Set tpg frame format before starting kthread
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: propagate error from kthread_stop
- coding style when calling tpg_s_bytesperline
- s/vimc_thread_sen/vimc_sen_tpg_thread
- fix multiline comment
- remove V4L2_FIELD_ALTERNATE from tpg_s_field
- remove V4L2_STD_PAL from tpg_fill_plane_buffer

Changes in v2:
[media] vimc: sen: Integrate the tpg on the sensor
- Fix include location
- Select V4L2_TPG in Kconfig
- configure tpg on streamon only
- rm BUG_ON
- coding style


---
 drivers/media/platform/vimc/Kconfig   |  1 +
 drivers/media/platform/vimc/vimc-sensor.c | 64 ---
 2 files changed, 52 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/vimc/Kconfig 
b/drivers/media/platform/vimc/Kconfig
index a18f635..71c9fe7 100644
--- a/drivers/media/platform/vimc/Kconfig
+++ b/drivers/media/platform/vimc/Kconfig
@@ -2,6 +2,7 @@ config VIDEO_VIMC
tristate "Virtual Media Controller Driver (VIMC)"
depends on VIDEO_DEV && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
select VIDEOBUF2_VMALLOC
+   select VIDEO_V4L2_TPG
default n
---help---
  Skeleton driver for Virtual Media Controller
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 591f6a4..2e83487 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -20,17 +20,20 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "vimc-sensor.h"
 
+#define VIMC_SEN_FRAME_MAX_WIDTH 4096
+
 struct vimc_sen_device {
struct vimc_ent_device ved;
struct v4l2_subdev sd;
+   struct tpg_data tpg;
struct task_struct *kthread_sen;
u8 *frame;
/* The active format */
struct v4l2_mbus_framefmt mbus_format;
-   int frame_size;
 };
 
 static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
@@ -84,6 +87,24 @@ static int vimc_sen_get_fmt(struct v4l2_subdev *sd,
return 0;
 }
 
+static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
+{
+   const struct vimc_pix_map *vpix =
+   vimc_pix_map_by_code(vsen->mbus_format.code);
+
+   tpg_reset_source(>tpg, vsen->mbus_format.width,
+vsen->mbus_format.height, vsen->mbus_format.field);
+   tpg_s_bytesperline(>tpg, 0, vsen->mbus_format.width * vpix->bpp);
+   tpg_s_buf_height(>tpg, vsen->mbus_format.height);
+   tpg_s_fourcc(>tpg, vpix->pixelformat);
+   /* TODO: add support for V4L2_FIELD_ALTERNATE */
+   tpg_s_field(>tpg, vsen->mbus_format.field, false);
+   tpg_s_colorspace(>tpg, vsen->mbus_format.colorspace);
+   tpg_s_ycbcr_enc(>tpg, vsen->mbus_format.ycbcr_enc);
+   tpg_s_quantization(>tpg, vsen->mbus_format.quantization);
+   tpg_s_xfer_func(>tpg, vsen->mbus_format.xfer_func);
+}
+
 static const struct v4l2_subdev_pad_ops vimc_sen_pad_ops = {
.enum_mbus_code = vimc_sen_enum_mbus_code,
.enum_frame_size= vimc_sen_enum_frame_size,
@@ -97,7 +118,7 @@ static const struct media_entity_operations vimc_sen_mops = {
.link_validate = v4l2_subdev_link_validate,
 };
 
-static int vimc_thread_sen(void *data)
+static int vimc_sen_tpg_thread(void *data)
 {
struct vimc_sen_device *vsen = data;
unsigned int i;
@@ -110,7 +131,7 @@ static int vimc_thread_sen(void *data)
if (kthread_should_stop())
break;
 
-   memset(vsen->frame, 100, vsen->frame_size);
+   tpg_fill_plane_buffer(>tpg, 0, 0, vsen->frame);
 
/* Send the frame to all source pads */
for (i = 0; i < vsen->sd.entity.num_pads; i++)
@@ -132,26 +153,31 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int 
enable)
 
if (enable) {
const struct vimc_pix_map *vpix;
+   unsigned int frame_size;
 
if (vsen->kthread_sen)
-   return -EINVAL;
+   /* tpg is already executing */
+   return 0;
 
/* Calculate the frame size */
vpix = vimc_pix_map_by_code(vsen->mbus_format.code);
-   vsen->frame_size = vsen->mbus_format.width * vpix->bpp *
-   

[PATCH v5 08/12] [media] vimc: cap: Support several image formats

[Helen Koike]

Allow user space to change the image format as the frame size, the
pixel format, colorspace, quantization, field YCbCr encoding
and the transfer function

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5: None
Changes in v4:
[media] vimc: cap: Support several image formats
- add vimc_colorimetry_clamp macro
- replace V4L2_COLORSPACE_SRGB by V4L2_COLORSPACE_DEFAULT in the
default format struct

Changes in v3:
[media] vimc: cap: Support several image formats
- use *_DEFAULT macros for colorimetry in the default format
- clamp height and width of the image by an even value
- is user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- remove V4L2_FMT_FLAG_COMPRESSED from vimc_cap_enum_fmt_vid_cap
- remove V4L2_BUF_TYPE_VIDEO_CAPTURE from vimc_cap_enum_fmt_vid_cap
- increase step_width and step_height to 2 instead of 1
- remove link validate function, use the one in vimc-common.c

Changes in v2:
[media] vimc: cap: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- allow user space to change all fields from struct v4l2_pix_format
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- link_validate and try_fmt: also checks colospace, quantization, 
field, xfer_func, ycbcr_enc
- add struct v4l2_pix_format fmt_default
- add enum_framesizes
- enum_fmt_vid_cap: enumerate all formats from vimc_pix_map_table
- add mode dev_dbg


---
 drivers/media/platform/vimc/vimc-capture.c | 117 +
 1 file changed, 101 insertions(+), 16 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 5bdecd1..359f59e 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -40,6 +40,14 @@ struct vimc_cap_device {
struct media_pipeline pipe;
 };
 
+static const struct v4l2_pix_format fmt_default = {
+   .width = 640,
+   .height = 480,
+   .pixelformat = V4L2_PIX_FMT_RGB24,
+   .field = V4L2_FIELD_NONE,
+   .colorspace = V4L2_COLORSPACE_DEFAULT,
+};
+
 struct vimc_cap_buffer {
/*
 * struct vb2_v4l2_buffer must be the first element
@@ -73,7 +81,7 @@ static void vimc_cap_get_format(struct vimc_ent_device *ved,
*fmt = vcap->format;
 }
 
-static int vimc_cap_fmt_vid_cap(struct file *file, void *priv,
+static int vimc_cap_g_fmt_vid_cap(struct file *file, void *priv,
  struct v4l2_format *f)
 {
struct vimc_cap_device *vcap = video_drvdata(file);
@@ -83,16 +91,98 @@ static int vimc_cap_fmt_vid_cap(struct file *file, void 
*priv,
return 0;
 }
 
+static int vimc_cap_try_fmt_vid_cap(struct file *file, void *priv,
+   struct v4l2_format *f)
+{
+   struct v4l2_pix_format *format = >fmt.pix;
+   const struct vimc_pix_map *vpix;
+
+   format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
+   VIMC_FRAME_MAX_WIDTH) & ~1;
+   format->height = clamp_t(u32, format->height, VIMC_FRAME_MIN_HEIGHT,
+VIMC_FRAME_MAX_HEIGHT) & ~1;
+
+   /* Don't accept a pixelformat that is not on the table */
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   if (!vpix) {
+   format->pixelformat = fmt_default.pixelformat;
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   }
+   /* TODO: Add support for custom bytesperline values */
+   format->bytesperline = format->width * vpix->bpp;
+   format->sizeimage = format->bytesperline * format->height;
+
+   if (format->field == V4L2_FIELD_ANY)
+   format->field = fmt_default.field;
+
+   vimc_colorimetry_clamp(format);
+
+   return 0;
+}
+
+static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+   struct vimc_cap_device *vcap = video_drvdata(file);
+
+   /* Do not change the format while stream is on */
+   if (vb2_is_busy(>queue))
+   return -EBUSY;
+
+   vimc_cap_try_fmt_vid_cap(file, priv, f);
+
+   dev_dbg(vcap->vdev.v4l2_dev->dev, "%s: format update: "
+   "old:%dx%d (0x%x, %d, %d, %d, %d) "
+   "new:%dx%d (0x%x, %d, %d, %d, %d)\n", vcap->vdev.name,
+   /* old */
+   vcap->format.width, vcap->format.height,
+   vcap->format.pixelformat, vcap->format.colorspace,
+   vcap->format.quantization, vcap->format.xfer_func,
+   vcap->format.ycbcr_enc,
+   /* new */
+

[PATCH v5 02/12] [media] vimc: Move common code from the core

[Helen Koike]

Remove helper functions from vimc-core and add it in vimc-common to
clean up the core.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5: None
Changes in v4: None
Changes in v3:
[media] vimc: Move common code from the core
- This is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/Makefile   |   2 +-
 drivers/media/platform/vimc/vimc-capture.h |   2 +-
 drivers/media/platform/vimc/vimc-common.c  | 221 +
 .../platform/vimc/{vimc-core.h => vimc-common.h}   |   7 +-
 drivers/media/platform/vimc/vimc-core.c| 205 +--
 drivers/media/platform/vimc/vimc-sensor.h  |   2 +-
 6 files changed, 229 insertions(+), 210 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc-common.c
 rename drivers/media/platform/vimc/{vimc-core.h => vimc-common.h} (96%)

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index c45195e..6b6ddf4 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,3 +1,3 @@
-vimc-objs := vimc-core.o vimc-capture.o vimc-sensor.o
+vimc-objs := vimc-core.o vimc-capture.o vimc-common.o vimc-sensor.o
 
 obj-$(CONFIG_VIDEO_VIMC) += vimc.o
diff --git a/drivers/media/platform/vimc/vimc-capture.h 
b/drivers/media/platform/vimc/vimc-capture.h
index 581a813..7e5c707 100644
--- a/drivers/media/platform/vimc/vimc-capture.h
+++ b/drivers/media/platform/vimc/vimc-capture.h
@@ -18,7 +18,7 @@
 #ifndef _VIMC_CAPTURE_H_
 #define _VIMC_CAPTURE_H_
 
-#include "vimc-core.h"
+#include "vimc-common.h"
 
 struct vimc_ent_device *vimc_cap_create(struct v4l2_device *v4l2_dev,
const char *const name,
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
new file mode 100644
index 000..42f779a
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -0,0 +1,221 @@
+/*
+ * vimc-common.c Virtual Media Controller Driver
+ *
+ * Copyright (C) 2015-2017 Helen Koike <helen.fornaz...@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include "vimc-common.h"
+
+static const struct vimc_pix_map vimc_pix_map_list[] = {
+   /* TODO: add all missing formats */
+
+   /* RGB formats */
+   {
+   .code = MEDIA_BUS_FMT_BGR888_1X24,
+   .pixelformat = V4L2_PIX_FMT_BGR24,
+   .bpp = 3,
+   },
+   {
+   .code = MEDIA_BUS_FMT_RGB888_1X24,
+   .pixelformat = V4L2_PIX_FMT_RGB24,
+   .bpp = 3,
+   },
+   {
+   .code = MEDIA_BUS_FMT_ARGB_1X32,
+   .pixelformat = V4L2_PIX_FMT_ARGB32,
+   .bpp = 4,
+   },
+
+   /* Bayer formats */
+   {
+   .code = MEDIA_BUS_FMT_SBGGR8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SBGGR8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGBRG8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SGBRG8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGRBG8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SGRBG8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SRGGB8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SRGGB8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SBGGR10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SBGGR10,
+   .bpp = 2,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGBRG10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SGBRG10,
+   .bpp = 2,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGRBG10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SGRBG10,
+   .bpp = 2,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SRGGB10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SRGGB10,
+   .bpp = 2,
+   },
+
+   /* 10bit raw bayer a-law compressed to 8 bits */
+   {
+   .code = MEDIA_BUS_FMT_SBGGR10_ALAW8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SBGGR10ALAW8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGBRG10_ALAW8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SGBRG10ALAW8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT

[PATCH v5 05/12] [media] vimc: common: Add vimc_link_validate

[Helen Koike]

All links will be checked in the same way. Adding a helper function for
that

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5: None
Changes in v4:
[media] vimc: common: Add vimc_link_validate
- remove vimc_fmt_pix_to_mbus(), replaced by
v4l2_fill_mbus_format()
- remove EXPORT_SYMBOL(vimc_link_validate), not necessary in
this patch, moved to submodules patch
- Fix multi-line comment style
- If colorspace is set to DEFAULT, then assume all the other
colorimetry parameters are also DEFAULT

Changes in v3:
[media] vimc: common: Add vimc_link_validate
- this is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/vimc-capture.c |  78 +++
 drivers/media/platform/vimc/vimc-common.c  | 121 -
 drivers/media/platform/vimc/vimc-common.h  |  14 
 3 files changed, 145 insertions(+), 68 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 93f6a09..5bdecd1 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -64,6 +64,15 @@ static int vimc_cap_querycap(struct file *file, void *priv,
return 0;
 }
 
+static void vimc_cap_get_format(struct vimc_ent_device *ved,
+   struct v4l2_pix_format *fmt)
+{
+   struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+   ved);
+
+   *fmt = vcap->format;
+}
+
 static int vimc_cap_fmt_vid_cap(struct file *file, void *priv,
  struct v4l2_format *f)
 {
@@ -231,74 +240,8 @@ static const struct vb2_ops vimc_cap_qops = {
.wait_finish= vb2_ops_wait_finish,
 };
 
-/*
- * NOTE: this function is a copy of v4l2_subdev_link_validate_get_format
- * maybe the v4l2 function should be public
- */
-static int vimc_cap_v4l2_subdev_link_validate_get_format(struct media_pad *pad,
-   struct v4l2_subdev_format *fmt)
-{
-   struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(pad->entity);
-
-   fmt->which = V4L2_SUBDEV_FORMAT_ACTIVE;
-   fmt->pad = pad->index;
-
-   return v4l2_subdev_call(sd, pad, get_fmt, NULL, fmt);
-}
-
-static int vimc_cap_link_validate(struct media_link *link)
-{
-   struct v4l2_subdev_format source_fmt;
-   const struct vimc_pix_map *vpix;
-   struct vimc_cap_device *vcap = container_of(link->sink->entity,
-   struct vimc_cap_device,
-   vdev.entity);
-   struct v4l2_pix_format *sink_fmt = >format;
-   int ret;
-
-   /*
-* if it is a raw node from vimc-core, ignore the link for now
-* TODO: remove this when there are no more raw nodes in the
-* core and return error instead
-*/
-   if (link->source->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
-   return 0;
-
-   /* Get the the format of the subdev */
-   ret = vimc_cap_v4l2_subdev_link_validate_get_format(link->source,
-   _fmt);
-   if (ret)
-   return ret;
-
-   dev_dbg(vcap->vdev.v4l2_dev->dev,
-   "%s: link validate formats src:%dx%d %d sink:%dx%d %d\n",
-   vcap->vdev.name,
-   source_fmt.format.width, source_fmt.format.height,
-   source_fmt.format.code,
-   sink_fmt->width, sink_fmt->height,
-   sink_fmt->pixelformat);
-
-   /* The width, height and code must match. */
-   vpix = vimc_pix_map_by_pixelformat(sink_fmt->pixelformat);
-   if (source_fmt.format.width != sink_fmt->width
-   || source_fmt.format.height != sink_fmt->height
-   || vpix->code != source_fmt.format.code)
-   return -EPIPE;
-
-   /*
-* The field order must match, or the sink field order must be NONE
-* to support interlaced hardware connected to bridges that support
-* progressive formats only.
-*/
-   if (source_fmt.format.field != sink_fmt->field &&
-   sink_fmt->field != V4L2_FIELD_NONE)
-   return -EPIPE;
-
-   return 0;
-}
-
 static const struct media_entity_operations vimc_cap_mops = {
-   .link_validate  = vimc_cap_link_validate,
+   .link_validate  = vimc_link_validate,
 };
 
 static void vimc_cap_destroy(struct vimc_ent_device *ved)
@@ -434,6 +377,7 @@ struct vimc_ent_device *vimc_cap_create(struct v4l2_device 
*v4l2_dev,
vcap->ved.destroy = vimc_cap_destroy;
vcap->ved.ent = >vdev.entity;
vcap->ved.process_frame = vimc_cap_process_frame;
+   vcap->ved.vdev_get_fo

[PATCH v5 09/12] [media] vimc: Subdevices as modules

[Helen Koike]

Change the core structure for adding subdevices in the topology.
Instead of calling the specific create function for each subdevice,
inject a child platform_device with the driver's name.
Each type of node in the topology (sensor, capture, debayer, scaler)
will register a platform_driver with the corresponding name through the
component subsystem.
Implementing a new subdevice type doesn't require vimc-core to be altered.

This facilitates future implementation of dynamic entities, where
hotpluging an entity in the topology is just a matter of
registering/unregistering a platform_device in the system.
It also facilitates other implementations of different nodes without
touching the core code and remove the need of a header file for each
type of node.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5:
[media] vimc: Subdevices as modules
- Fix vimc_add_subdevs in rollback case. The loop variable can
be negative, remove 'unsigned' from the loop variable and fix
warning from smatch tool

Changes in v4:
[media] vimc: Subdevices as modules
- Rebase without [media] vimc: Optimize frame generation the through
pipe
- s/EXPORT_SYMBOL/EXPORT_SYMBOL_GPL
- add struct vimc_platform_data to pass the entity's name to the
sudmodule
- Fix comment about vimc-input (remove vimc-output comment)

Changes in v3:
[media] vimc: Subdevices as modules
- This is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/Makefile   |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  99 ---
 drivers/media/platform/vimc/vimc-capture.h |  28 --
 drivers/media/platform/vimc/vimc-common.c  |  38 +--
 drivers/media/platform/vimc/vimc-common.h  |  29 ++-
 drivers/media/platform/vimc/vimc-core.c| 405 +++--
 drivers/media/platform/vimc/vimc-sensor.c  |  93 +--
 drivers/media/platform/vimc/vimc-sensor.h  |  28 --
 8 files changed, 339 insertions(+), 388 deletions(-)
 delete mode 100644 drivers/media/platform/vimc/vimc-capture.h
 delete mode 100644 drivers/media/platform/vimc/vimc-sensor.h

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index 6b6ddf4..0e5d5ce 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,3 +1,6 @@
-vimc-objs := vimc-core.o vimc-capture.o vimc-common.o vimc-sensor.o
+vimc-objs := vimc-core.o
+vimc_capture-objs := vimc-capture.o
+vimc_common-objs := vimc-common.o
+vimc_sensor-objs := vimc-sensor.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 359f59e..14cb32e 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -15,15 +15,21 @@
  *
  */
 
+#include 
+#include 
+#include 
 #include 
 #include 
 #include 
 
-#include "vimc-capture.h"
+#include "vimc-common.h"
+
+#define VIMC_CAP_DRV_NAME "vimc-capture"
 
 struct vimc_cap_device {
struct vimc_ent_device ved;
struct video_device vdev;
+   struct device *dev;
struct v4l2_pix_format format;
struct vb2_queue queue;
struct list_head buf_list;
@@ -131,7 +137,7 @@ static int vimc_cap_s_fmt_vid_cap(struct file *file, void 
*priv,
 
vimc_cap_try_fmt_vid_cap(file, priv, f);
 
-   dev_dbg(vcap->vdev.v4l2_dev->dev, "%s: format update: "
+   dev_dbg(vcap->dev, "%s: format update: "
"old:%dx%d (0x%x, %d, %d, %d, %d) "
"new:%dx%d (0x%x, %d, %d, %d, %d)\n", vcap->vdev.name,
/* old */
@@ -309,8 +315,7 @@ static int vimc_cap_buffer_prepare(struct vb2_buffer *vb)
unsigned long size = vcap->format.sizeimage;
 
if (vb2_plane_size(vb, 0) < size) {
-   dev_err(vcap->vdev.v4l2_dev->dev,
-   "%s: buffer too small (%lu < %lu)\n",
+   dev_err(vcap->dev, "%s: buffer too small (%lu < %lu)\n",
vcap->vdev.name, vb2_plane_size(vb, 0), size);
return -EINVAL;
}
@@ -335,8 +340,10 @@ static const struct media_entity_operations vimc_cap_mops 
= {
.link_validate  = vimc_link_validate,
 };
 
-static void vimc_cap_destroy(struct vimc_ent_device *ved)
+static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
+void *master_data)
 {
+   struct vimc_ent_device *ved = dev_get_drvdata(comp);
struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
ved);
 
@@ -385,42 +392,35 @@ static void vimc_cap_process_frame(struct vimc_ent_device 
*ved,
  

[PATCH v5 10/12] [media] vimc: deb: Add debayer filter

[Helen Koike]

Implement the debayer filter and integrate it with the core

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5:
[media] vimc: deb: Add debayer filter
- delare vimc_deb_video_ops as static, remove sparse warning

Changes in v4:
[media] vimc: deb: Add debayer filter
- Rebase without [media] vimc: Optimize frame generation through
pipe
- use vimc_colorimetry_clamp
- replace V4L2_COLORSPACE_SRGB by V4L2_COLORSPACE_DEFAULT in the
default format struct
- use struct vimc_platform_data to retrieve the entity's name

Changes in v3:
[media] vimc: deb: Add debayer filter
- Declare frame_size as a local variable
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: add ret variable to propagate return errors
- structure code to be a module, use platform_driver and component 
system
- fix multiline comment
- s/thought/through
- s/RGB/RGB888
- clamp height and width of the image by an even value
- if user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- uset _DEFAULT for colorimetry in the default format

Changes in v2:
[media] vimc: deb: Add debayer filter
- Using MEDIA_ENT_F_ATV_DECODER in function
- remove v4l2_dev and dev from vimc_deb_device struct
- src fmt propagates from the sink
- coding style
- remove redundant else if statements
- check end of enum and remove BUG_ON
- enum frame size with min and max values
- set/try fmt
- remove unecessary include freezer.h
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add deb_mean_win_size as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg


---
 drivers/media/platform/vimc/Makefile   |   4 +-
 drivers/media/platform/vimc/vimc-common.h  |   2 +
 drivers/media/platform/vimc/vimc-debayer.c | 601 +
 3 files changed, 606 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/vimc/vimc-debayer.c

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index 0e5d5ce..4fba8ef 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,6 +1,8 @@
 vimc-objs := vimc-core.o
 vimc_capture-objs := vimc-capture.o
 vimc_common-objs := vimc-common.o
+vimc_debayer-objs := vimc-debayer.o
 vimc_sensor-objs := vimc-sensor.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc_sensor.o
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc-debayer.o 
\
+   vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index a9c1cfd..25ba752 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -27,6 +27,8 @@
 #define VIMC_FRAME_MIN_WIDTH 16
 #define VIMC_FRAME_MIN_HEIGHT 16
 
+#define VIMC_FRAME_INDEX(lin, col, width, bpp) ((lin * width + col) * bpp)
+
 /**
  * struct vimc_colorimetry_clamp - Adjust colorimetry parameters
  *
diff --git a/drivers/media/platform/vimc/vimc-debayer.c 
b/drivers/media/platform/vimc/vimc-debayer.c
new file mode 100644
index 000..35b15bd
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -0,0 +1,601 @@
+/*
+ * vimc-debayer.c Virtual Media Controller Driver
+ *
+ * Copyright (C) 2015-2017 Helen Koike <helen.fornaz...@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "vimc-common.h"
+
+#define VIMC_DEB_DRV_NAME "vimc-debayer"
+
+static unsigned int deb_mean_win_size = 3;
+module_param(deb_mean_win_size, uint, );
+MODULE_PARM_DESC(deb_mean_win_size, " the window size to calculate the mean.\n"
+   "NOTE: the window size need to be an odd number, as the main pixel "
+   "stays in the center of the window, otherwise the next odd number "
+   "is considered");
+
+#define IS_SINK(pad) (!pad)
+#define IS_SRC(pad)  (pad)
+
+enum vimc_deb_rgb_colors {
+  

[PATCH v5 11/12] [media] vimc: sca: Add scaler

[Helen Koike]

Implement scaler and integrated with the core

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5:
[media] vimc: sca: Add scaler
- declare vimc_sca_video_ops as static, remove sparse warning

Changes in v4:
[media] vimc: sca: Add scaler
- use vimc_colorimetry_clamp
- replace V4L2_COLORSPACE_SRGB by V4L2_COLORSPACE_DEFAULT in the
default format struct
- use struct vimc_platform_data to retrieve the entity's name

Changes in v3:
[media] vimc: sca: Add scaler
- Declare frame_size as a local variable
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: add ret variable to propagate return errors
- structure code to be a module, use platform_driver and component 
system
- s/thought/through
- clamp height and width of the image by an even value
- if user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- uset _DEFAULT for colorimetry in the default format

Changes in v2:
[media] vimc: sca: Add scaler
- Add function MEDIA_ENT_F_IO_V4L
- remove v4l2_dev and dev
- s/sink_mbus_fmt/sink_fmt
- remove BUG_ON, remove redundant if else, rewrite TODO, check end of 
enum
- rm src_width/height, enum fsize with min and max values
- set/try fmt
- remove unecessary include freezer.h
- core: add bayer boolean in pixel table
- coding style
- fix bug in enum frame size
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add sca_mult as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg


---
 drivers/media/platform/vimc/Makefile  |   3 +-
 drivers/media/platform/vimc/vimc-common.c |  27 ++
 drivers/media/platform/vimc/vimc-common.h |   1 +
 drivers/media/platform/vimc/vimc-scaler.c | 455 ++
 4 files changed, 485 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/vimc/vimc-scaler.c

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index 4fba8ef..68c5d98 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -2,7 +2,8 @@ vimc-objs := vimc-core.o
 vimc_capture-objs := vimc-capture.o
 vimc_common-objs := vimc-common.o
 vimc_debayer-objs := vimc-debayer.o
+vimc_scaler-objs := vimc-scaler.o
 vimc_sensor-objs := vimc-sensor.o
 
 obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc-debayer.o 
\
-   vimc_sensor.o
+   vimc_scaler.o vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index da7f2b7..9d63c84 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -20,6 +20,10 @@
 
 #include "vimc-common.h"
 
+/*
+ * NOTE: non-bayer formats need to come first (necessary for enum_mbus_code
+ * in the scaler)
+ */
 static const struct vimc_pix_map vimc_pix_map_list[] = {
/* TODO: add all missing formats */
 
@@ -28,16 +32,19 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
.code = MEDIA_BUS_FMT_BGR888_1X24,
.pixelformat = V4L2_PIX_FMT_BGR24,
.bpp = 3,
+   .bayer = false,
},
{
.code = MEDIA_BUS_FMT_RGB888_1X24,
.pixelformat = V4L2_PIX_FMT_RGB24,
.bpp = 3,
+   .bayer = false,
},
{
.code = MEDIA_BUS_FMT_ARGB_1X32,
.pixelformat = V4L2_PIX_FMT_ARGB32,
.bpp = 4,
+   .bayer = false,
},
 
/* Bayer formats */
@@ -45,41 +52,49 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
.code = MEDIA_BUS_FMT_SBGGR8_1X8,
.pixelformat = V4L2_PIX_FMT_SBGGR8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGBRG8_1X8,
.pixelformat = V4L2_PIX_FMT_SGBRG8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGRBG8_1X8,
.pixelformat = V4L2_PIX_FMT_SGRBG8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SRGGB8_1X8,
.pixelformat = V4L2_PIX_FMT_SRGGB8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SBGGR10_1X10,
.pixelformat = V4L2_PIX_FMT_SBGGR10,
.bpp = 2,
+

[PATCH v5 12/12] [media] vimc: sen: Declare vimc_sen_video_ops as static

[Helen Koike]

Declare vimc_sen_video_ops as static, remove warning from sparse tool

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5:
[media] vimc: sen: Declare vimc_sen_video_ops as static
- This is a new patch in the series

Changes in v4: None
Changes in v3: None
Changes in v2: None


---
 drivers/media/platform/vimc/vimc-sensor.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 5ea7b08..ebdbbe8 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -282,7 +282,7 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int 
enable)
return 0;
 }
 
-struct v4l2_subdev_video_ops vimc_sen_video_ops = {
+static struct v4l2_subdev_video_ops vimc_sen_video_ops = {
.s_stream = vimc_sen_s_stream,
 };
 
-- 
2.7.4

[PATCH v5 03/12] [media] vimc: common: Add vimc_ent_sd_* helper

[Helen Koike]

As all the subdevices in the topology will be initialized in the same
way, to avoid code repetition the vimc_ent_sd_{register, unregister}
helper functions were created

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v5: None
Changes in v4: None
Changes in v3:
[media] vimc: common: Add vimc_ent_sd_* helper
- add it in vimc-common.c instead in vimc-core.c
- fix vimc_ent_sd_register, use function parameter to assign
sd->entity.function instead of using a fixed value
- rename commit to add the "common" tag

Changes in v2:
[media] vimc: Add vimc_ent_sd_* helper functions
- Comments in vimc_ent_sd_init
- Update vimc_ent_sd_init with upstream code as media_entity_pads_init
(instead of media_entity_init), entity->function intead of entity->type
- Add missing vimc_pads_cleanup in vimc_ent_sd_cleanup
- remove subdevice v4l2_dev and dev fields
- change unregister order in vimc_ent_sd_cleanup
- rename vimc_ent_sd_{init,cleanup} to vimc_ent_sd_{register,unregister}
- remove struct vimc_ent_subdevice, use ved and sd directly
- don't impose struct vimc_sen_device to declare ved and sd struct first
- add kernel docs


---
 drivers/media/platform/vimc/vimc-common.c | 66 +++
 drivers/media/platform/vimc/vimc-common.h | 39 ++
 drivers/media/platform/vimc/vimc-sensor.c | 58 +--
 3 files changed, 114 insertions(+), 49 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 42f779a..3afbabd 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -219,3 +219,69 @@ struct media_pad *vimc_pads_init(u16 num_pads, const 
unsigned long *pads_flag)
 
return pads;
 }
+
+static const struct media_entity_operations vimc_ent_sd_mops = {
+   .link_validate = v4l2_subdev_link_validate,
+};
+
+int vimc_ent_sd_register(struct vimc_ent_device *ved,
+struct v4l2_subdev *sd,
+struct v4l2_device *v4l2_dev,
+const char *const name,
+u32 function,
+u16 num_pads,
+const unsigned long *pads_flag,
+const struct v4l2_subdev_ops *sd_ops,
+void (*sd_destroy)(struct vimc_ent_device *))
+{
+   int ret;
+
+   /* Allocate the pads */
+   ved->pads = vimc_pads_init(num_pads, pads_flag);
+   if (IS_ERR(ved->pads))
+   return PTR_ERR(ved->pads);
+
+   /* Fill the vimc_ent_device struct */
+   ved->destroy = sd_destroy;
+   ved->ent = >entity;
+
+   /* Initialize the subdev */
+   v4l2_subdev_init(sd, sd_ops);
+   sd->entity.function = function;
+   sd->entity.ops = _ent_sd_mops;
+   sd->owner = THIS_MODULE;
+   strlcpy(sd->name, name, sizeof(sd->name));
+   v4l2_set_subdevdata(sd, ved);
+
+   /* Expose this subdev to user space */
+   sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+
+   /* Initialize the media entity */
+   ret = media_entity_pads_init(>entity, num_pads, ved->pads);
+   if (ret)
+   goto err_clean_pads;
+
+   /* Register the subdev with the v4l2 and the media framework */
+   ret = v4l2_device_register_subdev(v4l2_dev, sd);
+   if (ret) {
+   dev_err(v4l2_dev->dev,
+   "%s: subdev register failed (err=%d)\n",
+   name, ret);
+   goto err_clean_m_ent;
+   }
+
+   return 0;
+
+err_clean_m_ent:
+   media_entity_cleanup(>entity);
+err_clean_pads:
+   vimc_pads_cleanup(ved->pads);
+   return ret;
+}
+
+void vimc_ent_sd_unregister(struct vimc_ent_device *ved, struct v4l2_subdev 
*sd)
+{
+   v4l2_device_unregister_subdev(sd);
+   media_entity_cleanup(ved->ent);
+   vimc_pads_cleanup(ved->pads);
+}
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 00d3da4..9ec361c 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -110,4 +110,43 @@ const struct vimc_pix_map *vimc_pix_map_by_code(u32 code);
  */
 const struct vimc_pix_map *vimc_pix_map_by_pixelformat(u32 pixelformat);
 
+/**
+ * vimc_ent_sd_register - initialize and register a subdev node
+ *
+ * @ved:   the vimc_ent_device struct to be initialize
+ * @sd:the v4l2_subdev struct to be initialize and registered
+ * @v4l2_dev:  the v4l2 device to register the v4l2_subdev
+ * @name:  name of the sub-device. Please notice that the name must be
+ * unique.
+ * @function:  media entity function defined by MEDIA_ENT_F_* macros
+ * @num_pads:  number of pads to ini

[PATCH v5 00/12] [media]: vimc: Virtual Media Control VPU's

[Helen Koike]

include freezer.h
- core: add bayer boolean in pixel table
- coding style
- fix bug in enum frame size
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add sca_mult as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
    - add more dev_dbg

Helen Koike (12):
  [media] vimc: sen: Integrate the tpg on the sensor
  [media] vimc: Move common code from the core
  [media] vimc: common: Add vimc_ent_sd_* helper
  [media] vimc: common: Add vimc_pipeline_s_stream helper
  [media] vimc: common: Add vimc_link_validate
  [media] vimc: common: Add vimc_colorimetry_clamp
  [media] vimc: sen: Support several image formats
  [media] vimc: cap: Support several image formats
  [media] vimc: Subdevices as modules
  [media] vimc: deb: Add debayer filter
  [media] vimc: sca: Add scaler
  [media] vimc: sen: Declare vimc_sen_video_ops as static

 drivers/media/platform/vimc/Kconfig|   1 +
 drivers/media/platform/vimc/Makefile   |  10 +-
 drivers/media/platform/vimc/vimc-capture.c | 321 ---
 drivers/media/platform/vimc/vimc-capture.h |  28 --
 drivers/media/platform/vimc/vimc-common.c  | 473 ++
 drivers/media/platform/vimc/vimc-common.h  | 229 +++
 drivers/media/platform/vimc/vimc-core.c| 610 -
 drivers/media/platform/vimc/vimc-core.h| 112 --
 drivers/media/platform/vimc/vimc-debayer.c | 601 
 drivers/media/platform/vimc/vimc-scaler.c  | 455 +
 drivers/media/platform/vimc/vimc-sensor.c  | 321 ++-
 drivers/media/platform/vimc/vimc-sensor.h  |  28 --
 12 files changed, 2325 insertions(+), 864 deletions(-)
 delete mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-common.c
 create mode 100644 drivers/media/platform/vimc/vimc-common.h
 delete mode 100644 drivers/media/platform/vimc/vimc-core.h
 create mode 100644 drivers/media/platform/vimc/vimc-debayer.c
 create mode 100644 drivers/media/platform/vimc/vimc-scaler.c
 delete mode 100644 drivers/media/platform/vimc/vimc-sensor.h

-- 
2.7.4

[PATCH v2] [media] v4l2: add V4L2_CAP_IO_MC

[Helen Koike]

Add V4L2_CAP_IO_MC to be used in struct v4l2_capability to indicate that
input and output are controlled by the Media Controller instead of V4L2
API.
When this flag is set, ioctls for get, set and enum input and outputs
are automatically enabled and programmed to call helper function.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2::
- replace the type by capability
- erase V4L2_INPUT_TYPE_DEFAULT
- also consider output
- plug helpers in the ops automatically so drivers doesn't need
to set it by hand
- update docs
- commit message and title
---
 Documentation/media/uapi/v4l/vidioc-querycap.rst |  3 +
 Documentation/media/videodev2.h.rst.exceptions   |  1 +
 drivers/media/v4l2-core/v4l2-dev.c   | 35 +++--
 drivers/media/v4l2-core/v4l2-ioctl.c | 91 ++--
 include/uapi/linux/videodev2.h   |  2 +
 5 files changed, 120 insertions(+), 12 deletions(-)

diff --git a/Documentation/media/uapi/v4l/vidioc-querycap.rst 
b/Documentation/media/uapi/v4l/vidioc-querycap.rst
index 12e0d9a..2bd1223 100644
--- a/Documentation/media/uapi/v4l/vidioc-querycap.rst
+++ b/Documentation/media/uapi/v4l/vidioc-querycap.rst
@@ -252,6 +252,9 @@ specification the ioctl returns an ``EINVAL`` error code.
 * - ``V4L2_CAP_TOUCH``
   - 0x1000
   - This is a touch device.
+* - ``V4L2_CAP_IO_MC``
+  - 0x2000
+  - This device has its inputs and outputs controller by the Media 
Controller
 * - ``V4L2_CAP_DEVICE_CAPS``
   - 0x8000
   - The driver fills the ``device_caps`` field. This capability can
diff --git a/Documentation/media/videodev2.h.rst.exceptions 
b/Documentation/media/videodev2.h.rst.exceptions
index a5cb0a8..0b48cd0 100644
--- a/Documentation/media/videodev2.h.rst.exceptions
+++ b/Documentation/media/videodev2.h.rst.exceptions
@@ -159,6 +159,7 @@ replace define V4L2_CAP_ASYNCIO device-capabilities
 replace define V4L2_CAP_STREAMING device-capabilities
 replace define V4L2_CAP_DEVICE_CAPS device-capabilities
 replace define V4L2_CAP_TOUCH device-capabilities
+replace define V4L2_CAP_IO_MC device-capabilities
 
 # V4L2 pix flags
 replace define V4L2_PIX_FMT_PRIV_MAGIC :c:type:`v4l2_pix_format`
diff --git a/drivers/media/v4l2-core/v4l2-dev.c 
b/drivers/media/v4l2-core/v4l2-dev.c
index c647ba6..0f272fe 100644
--- a/drivers/media/v4l2-core/v4l2-dev.c
+++ b/drivers/media/v4l2-core/v4l2-dev.c
@@ -688,22 +688,34 @@ static void determine_valid_ioctls(struct video_device 
*vdev)
SET_VALID_IOCTL(ops, VIDIOC_G_STD, vidioc_g_std);
if (is_rx) {
SET_VALID_IOCTL(ops, VIDIOC_QUERYSTD, vidioc_querystd);
-   SET_VALID_IOCTL(ops, VIDIOC_ENUMINPUT, 
vidioc_enum_input);
-   SET_VALID_IOCTL(ops, VIDIOC_G_INPUT, vidioc_g_input);
-   SET_VALID_IOCTL(ops, VIDIOC_S_INPUT, vidioc_s_input);
SET_VALID_IOCTL(ops, VIDIOC_ENUMAUDIO, 
vidioc_enumaudio);
SET_VALID_IOCTL(ops, VIDIOC_G_AUDIO, vidioc_g_audio);
SET_VALID_IOCTL(ops, VIDIOC_S_AUDIO, vidioc_s_audio);
SET_VALID_IOCTL(ops, VIDIOC_QUERY_DV_TIMINGS, 
vidioc_query_dv_timings);
SET_VALID_IOCTL(ops, VIDIOC_S_EDID, vidioc_s_edid);
+   if (vdev->device_caps & V4L2_CAP_IO_MC) {
+   set_bit(_IOC_NR(VIDIOC_ENUMINPUT), 
valid_ioctls);
+   set_bit(_IOC_NR(VIDIOC_G_INPUT), valid_ioctls);
+   set_bit(_IOC_NR(VIDIOC_S_INPUT), valid_ioctls);
+   } else {
+   SET_VALID_IOCTL(ops, VIDIOC_ENUMINPUT, 
vidioc_enum_input);
+   SET_VALID_IOCTL(ops, VIDIOC_G_INPUT, 
vidioc_g_input);
+   SET_VALID_IOCTL(ops, VIDIOC_S_INPUT, 
vidioc_s_input);
+   }
}
if (is_tx) {
-   SET_VALID_IOCTL(ops, VIDIOC_ENUMOUTPUT, 
vidioc_enum_output);
-   SET_VALID_IOCTL(ops, VIDIOC_G_OUTPUT, vidioc_g_output);
-   SET_VALID_IOCTL(ops, VIDIOC_S_OUTPUT, vidioc_s_output);
SET_VALID_IOCTL(ops, VIDIOC_ENUMAUDOUT, 
vidioc_enumaudout);
SET_VALID_IOCTL(ops, VIDIOC_G_AUDOUT, vidioc_g_audout);
SET_VALID_IOCTL(ops, VIDIOC_S_AUDOUT, vidioc_s_audout);
+   if (vdev->device_caps & V4L2_CAP_IO_MC) {
+   set_bit(_IOC_NR(VIDIOC_ENUMOUTPUT), 
valid_ioctls);
+   set_bit(_IOC_NR(VIDIOC_G_OUTPUT), valid_ioctls);
+   set_bit(_IOC_NR(VIDIOC_S_OUTPUT), valid_ioctls);
+   } else {
+   SET_VALID_IOCTL(op

[PATCH v4 01/11] [media] vimc: sen: Integrate the tpg on the sensor

[Helen Koike]

Initialize the test pattern generator on the sensor
Generate a colored bar image instead of a grey one

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4: None
Changes in v3:
[media] vimc: sen: Integrate the tpg on the sensor
- Declare frame_size as a local variable
- Set tpg frame format before starting kthread
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: propagate error from kthread_stop
- coding style when calling tpg_s_bytesperline
- s/vimc_thread_sen/vimc_sen_tpg_thread
- fix multiline comment
- remove V4L2_FIELD_ALTERNATE from tpg_s_field
- remove V4L2_STD_PAL from tpg_fill_plane_buffer

Changes in v2:
[media] vimc: sen: Integrate the tpg on the sensor
- Fix include location
- Select V4L2_TPG in Kconfig
- configure tpg on streamon only
- rm BUG_ON
- coding style


---
 drivers/media/platform/vimc/Kconfig   |  1 +
 drivers/media/platform/vimc/vimc-sensor.c | 64 ---
 2 files changed, 52 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/vimc/Kconfig 
b/drivers/media/platform/vimc/Kconfig
index a18f635..71c9fe7 100644
--- a/drivers/media/platform/vimc/Kconfig
+++ b/drivers/media/platform/vimc/Kconfig
@@ -2,6 +2,7 @@ config VIDEO_VIMC
tristate "Virtual Media Controller Driver (VIMC)"
depends on VIDEO_DEV && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
select VIDEOBUF2_VMALLOC
+   select VIDEO_V4L2_TPG
default n
---help---
  Skeleton driver for Virtual Media Controller
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 591f6a4..2e83487 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -20,17 +20,20 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "vimc-sensor.h"
 
+#define VIMC_SEN_FRAME_MAX_WIDTH 4096
+
 struct vimc_sen_device {
struct vimc_ent_device ved;
struct v4l2_subdev sd;
+   struct tpg_data tpg;
struct task_struct *kthread_sen;
u8 *frame;
/* The active format */
struct v4l2_mbus_framefmt mbus_format;
-   int frame_size;
 };
 
 static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
@@ -84,6 +87,24 @@ static int vimc_sen_get_fmt(struct v4l2_subdev *sd,
return 0;
 }
 
+static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
+{
+   const struct vimc_pix_map *vpix =
+   vimc_pix_map_by_code(vsen->mbus_format.code);
+
+   tpg_reset_source(>tpg, vsen->mbus_format.width,
+vsen->mbus_format.height, vsen->mbus_format.field);
+   tpg_s_bytesperline(>tpg, 0, vsen->mbus_format.width * vpix->bpp);
+   tpg_s_buf_height(>tpg, vsen->mbus_format.height);
+   tpg_s_fourcc(>tpg, vpix->pixelformat);
+   /* TODO: add support for V4L2_FIELD_ALTERNATE */
+   tpg_s_field(>tpg, vsen->mbus_format.field, false);
+   tpg_s_colorspace(>tpg, vsen->mbus_format.colorspace);
+   tpg_s_ycbcr_enc(>tpg, vsen->mbus_format.ycbcr_enc);
+   tpg_s_quantization(>tpg, vsen->mbus_format.quantization);
+   tpg_s_xfer_func(>tpg, vsen->mbus_format.xfer_func);
+}
+
 static const struct v4l2_subdev_pad_ops vimc_sen_pad_ops = {
.enum_mbus_code = vimc_sen_enum_mbus_code,
.enum_frame_size= vimc_sen_enum_frame_size,
@@ -97,7 +118,7 @@ static const struct media_entity_operations vimc_sen_mops = {
.link_validate = v4l2_subdev_link_validate,
 };
 
-static int vimc_thread_sen(void *data)
+static int vimc_sen_tpg_thread(void *data)
 {
struct vimc_sen_device *vsen = data;
unsigned int i;
@@ -110,7 +131,7 @@ static int vimc_thread_sen(void *data)
if (kthread_should_stop())
break;
 
-   memset(vsen->frame, 100, vsen->frame_size);
+   tpg_fill_plane_buffer(>tpg, 0, 0, vsen->frame);
 
/* Send the frame to all source pads */
for (i = 0; i < vsen->sd.entity.num_pads; i++)
@@ -132,26 +153,31 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int 
enable)
 
if (enable) {
const struct vimc_pix_map *vpix;
+   unsigned int frame_size;
 
if (vsen->kthread_sen)
-   return -EINVAL;
+   /* tpg is already executing */
+   return 0;
 
/* Calculate the frame size */
vpix = vimc_pix_map_by_code(vsen->mbus_format.code);
-   vsen->frame_size = vsen->mbus_format.width * vpix->bpp *
-  vsen-

[PATCH v4 02/11] [media] vimc: Move common code from the core

[Helen Koike]

Remove helper functions from vimc-core and add it in vimc-common to
clean up the core.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4: None
Changes in v3:
[media] vimc: Move common code from the core
- This is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/Makefile   |   2 +-
 drivers/media/platform/vimc/vimc-capture.h |   2 +-
 drivers/media/platform/vimc/vimc-common.c  | 221 +
 .../platform/vimc/{vimc-core.h => vimc-common.h}   |   7 +-
 drivers/media/platform/vimc/vimc-core.c| 205 +--
 drivers/media/platform/vimc/vimc-sensor.h  |   2 +-
 6 files changed, 229 insertions(+), 210 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc-common.c
 rename drivers/media/platform/vimc/{vimc-core.h => vimc-common.h} (96%)

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index c45195e..6b6ddf4 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,3 +1,3 @@
-vimc-objs := vimc-core.o vimc-capture.o vimc-sensor.o
+vimc-objs := vimc-core.o vimc-capture.o vimc-common.o vimc-sensor.o
 
 obj-$(CONFIG_VIDEO_VIMC) += vimc.o
diff --git a/drivers/media/platform/vimc/vimc-capture.h 
b/drivers/media/platform/vimc/vimc-capture.h
index 581a813..7e5c707 100644
--- a/drivers/media/platform/vimc/vimc-capture.h
+++ b/drivers/media/platform/vimc/vimc-capture.h
@@ -18,7 +18,7 @@
 #ifndef _VIMC_CAPTURE_H_
 #define _VIMC_CAPTURE_H_
 
-#include "vimc-core.h"
+#include "vimc-common.h"
 
 struct vimc_ent_device *vimc_cap_create(struct v4l2_device *v4l2_dev,
const char *const name,
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
new file mode 100644
index 000..42f779a
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -0,0 +1,221 @@
+/*
+ * vimc-common.c Virtual Media Controller Driver
+ *
+ * Copyright (C) 2015-2017 Helen Koike <helen.fornaz...@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include "vimc-common.h"
+
+static const struct vimc_pix_map vimc_pix_map_list[] = {
+   /* TODO: add all missing formats */
+
+   /* RGB formats */
+   {
+   .code = MEDIA_BUS_FMT_BGR888_1X24,
+   .pixelformat = V4L2_PIX_FMT_BGR24,
+   .bpp = 3,
+   },
+   {
+   .code = MEDIA_BUS_FMT_RGB888_1X24,
+   .pixelformat = V4L2_PIX_FMT_RGB24,
+   .bpp = 3,
+   },
+   {
+   .code = MEDIA_BUS_FMT_ARGB_1X32,
+   .pixelformat = V4L2_PIX_FMT_ARGB32,
+   .bpp = 4,
+   },
+
+   /* Bayer formats */
+   {
+   .code = MEDIA_BUS_FMT_SBGGR8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SBGGR8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGBRG8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SGBRG8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGRBG8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SGRBG8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SRGGB8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SRGGB8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SBGGR10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SBGGR10,
+   .bpp = 2,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGBRG10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SGBRG10,
+   .bpp = 2,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGRBG10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SGRBG10,
+   .bpp = 2,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SRGGB10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SRGGB10,
+   .bpp = 2,
+   },
+
+   /* 10bit raw bayer a-law compressed to 8 bits */
+   {
+   .code = MEDIA_BUS_FMT_SBGGR10_ALAW8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SBGGR10ALAW8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGBRG10_ALAW8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SGBRG10ALAW8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGRBG10_ALAW8_1X8,
+   

[PATCH v4 03/11] [media] vimc: common: Add vimc_ent_sd_* helper

[Helen Koike]

As all the subdevices in the topology will be initialized in the same
way, to avoid code repetition the vimc_ent_sd_{register, unregister}
helper functions were created

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4: None
Changes in v3:
[media] vimc: common: Add vimc_ent_sd_* helper
- add it in vimc-common.c instead in vimc-core.c
- fix vimc_ent_sd_register, use function parameter to assign
sd->entity.function instead of using a fixed value
- rename commit to add the "common" tag

Changes in v2:
[media] vimc: Add vimc_ent_sd_* helper functions
- Comments in vimc_ent_sd_init
- Update vimc_ent_sd_init with upstream code as media_entity_pads_init
(instead of media_entity_init), entity->function intead of entity->type
- Add missing vimc_pads_cleanup in vimc_ent_sd_cleanup
- remove subdevice v4l2_dev and dev fields
- change unregister order in vimc_ent_sd_cleanup
- rename vimc_ent_sd_{init,cleanup} to vimc_ent_sd_{register,unregister}
- remove struct vimc_ent_subdevice, use ved and sd directly
- don't impose struct vimc_sen_device to declare ved and sd struct first
- add kernel docs


---
 drivers/media/platform/vimc/vimc-common.c | 66 +++
 drivers/media/platform/vimc/vimc-common.h | 39 ++
 drivers/media/platform/vimc/vimc-sensor.c | 58 +--
 3 files changed, 114 insertions(+), 49 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 42f779a..3afbabd 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -219,3 +219,69 @@ struct media_pad *vimc_pads_init(u16 num_pads, const 
unsigned long *pads_flag)
 
return pads;
 }
+
+static const struct media_entity_operations vimc_ent_sd_mops = {
+   .link_validate = v4l2_subdev_link_validate,
+};
+
+int vimc_ent_sd_register(struct vimc_ent_device *ved,
+struct v4l2_subdev *sd,
+struct v4l2_device *v4l2_dev,
+const char *const name,
+u32 function,
+u16 num_pads,
+const unsigned long *pads_flag,
+const struct v4l2_subdev_ops *sd_ops,
+void (*sd_destroy)(struct vimc_ent_device *))
+{
+   int ret;
+
+   /* Allocate the pads */
+   ved->pads = vimc_pads_init(num_pads, pads_flag);
+   if (IS_ERR(ved->pads))
+   return PTR_ERR(ved->pads);
+
+   /* Fill the vimc_ent_device struct */
+   ved->destroy = sd_destroy;
+   ved->ent = >entity;
+
+   /* Initialize the subdev */
+   v4l2_subdev_init(sd, sd_ops);
+   sd->entity.function = function;
+   sd->entity.ops = _ent_sd_mops;
+   sd->owner = THIS_MODULE;
+   strlcpy(sd->name, name, sizeof(sd->name));
+   v4l2_set_subdevdata(sd, ved);
+
+   /* Expose this subdev to user space */
+   sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+
+   /* Initialize the media entity */
+   ret = media_entity_pads_init(>entity, num_pads, ved->pads);
+   if (ret)
+   goto err_clean_pads;
+
+   /* Register the subdev with the v4l2 and the media framework */
+   ret = v4l2_device_register_subdev(v4l2_dev, sd);
+   if (ret) {
+   dev_err(v4l2_dev->dev,
+   "%s: subdev register failed (err=%d)\n",
+   name, ret);
+   goto err_clean_m_ent;
+   }
+
+   return 0;
+
+err_clean_m_ent:
+   media_entity_cleanup(>entity);
+err_clean_pads:
+   vimc_pads_cleanup(ved->pads);
+   return ret;
+}
+
+void vimc_ent_sd_unregister(struct vimc_ent_device *ved, struct v4l2_subdev 
*sd)
+{
+   v4l2_device_unregister_subdev(sd);
+   media_entity_cleanup(ved->ent);
+   vimc_pads_cleanup(ved->pads);
+}
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 00d3da4..9ec361c 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -110,4 +110,43 @@ const struct vimc_pix_map *vimc_pix_map_by_code(u32 code);
  */
 const struct vimc_pix_map *vimc_pix_map_by_pixelformat(u32 pixelformat);
 
+/**
+ * vimc_ent_sd_register - initialize and register a subdev node
+ *
+ * @ved:   the vimc_ent_device struct to be initialize
+ * @sd:the v4l2_subdev struct to be initialize and registered
+ * @v4l2_dev:  the v4l2 device to register the v4l2_subdev
+ * @name:  name of the sub-device. Please notice that the name must be
+ * unique.
+ * @function:  media entity function defined by MEDIA_ENT_F_* macros
+ * @num_pads:  number of pads to initialize
+ * @pads

[PATCH v4 04/11] [media] vimc: common: Add vimc_pipeline_s_stream helper

[Helen Koike]

Move the vimc_cap_pipeline_s_stream from the vimc-cap.c to vimc-common.c
as this core will be reused by other subdevices to activate the stream
in their directly connected nodes

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4: None
Changes in v3:
[media] vimc: Add vimc_pipeline_s_stream in the core
- add it in vimc-common instead of vimc-core
- rename commit with "common" tag

Changes in v2:
[media] vimc: Add vimc_pipeline_s_stream in the core
- Use is_media_entity_v4l2_subdev instead of comparing with the old
entity->type
- Fix comments style
- add kernel-docs
- call s_stream across all sink pads


---
 drivers/media/platform/vimc/vimc-capture.c | 29 ++-
 drivers/media/platform/vimc/vimc-common.c  | 32 ++
 drivers/media/platform/vimc/vimc-common.h  | 11 ++
 3 files changed, 45 insertions(+), 27 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 9adb06d..93f6a09 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -132,31 +132,6 @@ static void vimc_cap_return_all_buffers(struct 
vimc_cap_device *vcap,
spin_unlock(>qlock);
 }
 
-static int vimc_cap_pipeline_s_stream(struct vimc_cap_device *vcap, int enable)
-{
-   struct v4l2_subdev *sd;
-   struct media_pad *pad;
-   int ret;
-
-   /* Start the stream in the subdevice direct connected */
-   pad = media_entity_remote_pad(>vdev.entity.pads[0]);
-
-   /*
-* if it is a raw node from vimc-core, there is nothing to activate
-* TODO: remove this when there are no more raw nodes in the
-* core and return error instead
-*/
-   if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
-   return 0;
-
-   sd = media_entity_to_v4l2_subdev(pad->entity);
-   ret = v4l2_subdev_call(sd, video, s_stream, enable);
-   if (ret && ret != -ENOIOCTLCMD)
-   return ret;
-
-   return 0;
-}
-
 static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
@@ -173,7 +148,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, 
unsigned int count)
}
 
/* Enable streaming from the pipe */
-   ret = vimc_cap_pipeline_s_stream(vcap, 1);
+   ret = vimc_pipeline_s_stream(>vdev.entity, 1);
if (ret) {
media_pipeline_stop(entity);
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
@@ -192,7 +167,7 @@ static void vimc_cap_stop_streaming(struct vb2_queue *vq)
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
 
/* Disable streaming from the pipe */
-   vimc_cap_pipeline_s_stream(vcap, 0);
+   vimc_pipeline_s_stream(>vdev.entity, 0);
 
/* Stop the media pipeline */
media_pipeline_stop(>vdev.entity);
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 3afbabd..f809a9d 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -220,6 +220,38 @@ struct media_pad *vimc_pads_init(u16 num_pads, const 
unsigned long *pads_flag)
return pads;
 }
 
+int vimc_pipeline_s_stream(struct media_entity *ent, int enable)
+{
+   struct v4l2_subdev *sd;
+   struct media_pad *pad;
+   unsigned int i;
+   int ret;
+
+   for (i = 0; i < ent->num_pads; i++) {
+   if (ent->pads[i].flags & MEDIA_PAD_FL_SOURCE)
+   continue;
+
+   /* Start the stream in the subdevice direct connected */
+   pad = media_entity_remote_pad(>pads[i]);
+
+   /*
+* if this is a raw node from vimc-core, then there is
+* nothing to activate
+* TODO: remove this when there are no more raw nodes in the
+* core and return error instead
+*/
+   if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
+   continue;
+
+   sd = media_entity_to_v4l2_subdev(pad->entity);
+   ret = v4l2_subdev_call(sd, video, s_stream, enable);
+   if (ret && ret != -ENOIOCTLCMD)
+   return ret;
+   }
+
+   return 0;
+}
+
 static const struct media_entity_operations vimc_ent_sd_mops = {
.link_validate = v4l2_subdev_link_validate,
 };
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 9ec361c..73e7e94 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -97,6 +97,17 @@ static inline void vimc_pads_cleanup(struct media_pad *pads)
 

[PATCH v4 05/11] [media] vimc: common: Add vimc_link_validate

[Helen Koike]

All links will be checked in the same way. Adding a helper function for
that

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4:
[media] vimc: common: Add vimc_link_validate
- remove vimc_fmt_pix_to_mbus(), replaced by
v4l2_fill_mbus_format()
- remove EXPORT_SYMBOL(vimc_link_validate), not necessary in
this patch, moved to submodules patch
- Fix multi-line comment style
- If colorspace is set to DEFAULT, then assume all the other
colorimetry parameters are also DEFAULT

Changes in v3:
[media] vimc: common: Add vimc_link_validate
- this is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/vimc-capture.c |  78 +++
 drivers/media/platform/vimc/vimc-common.c  | 121 -
 drivers/media/platform/vimc/vimc-common.h  |  14 
 3 files changed, 145 insertions(+), 68 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 93f6a09..5bdecd1 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -64,6 +64,15 @@ static int vimc_cap_querycap(struct file *file, void *priv,
return 0;
 }
 
+static void vimc_cap_get_format(struct vimc_ent_device *ved,
+   struct v4l2_pix_format *fmt)
+{
+   struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+   ved);
+
+   *fmt = vcap->format;
+}
+
 static int vimc_cap_fmt_vid_cap(struct file *file, void *priv,
  struct v4l2_format *f)
 {
@@ -231,74 +240,8 @@ static const struct vb2_ops vimc_cap_qops = {
.wait_finish= vb2_ops_wait_finish,
 };
 
-/*
- * NOTE: this function is a copy of v4l2_subdev_link_validate_get_format
- * maybe the v4l2 function should be public
- */
-static int vimc_cap_v4l2_subdev_link_validate_get_format(struct media_pad *pad,
-   struct v4l2_subdev_format *fmt)
-{
-   struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(pad->entity);
-
-   fmt->which = V4L2_SUBDEV_FORMAT_ACTIVE;
-   fmt->pad = pad->index;
-
-   return v4l2_subdev_call(sd, pad, get_fmt, NULL, fmt);
-}
-
-static int vimc_cap_link_validate(struct media_link *link)
-{
-   struct v4l2_subdev_format source_fmt;
-   const struct vimc_pix_map *vpix;
-   struct vimc_cap_device *vcap = container_of(link->sink->entity,
-   struct vimc_cap_device,
-   vdev.entity);
-   struct v4l2_pix_format *sink_fmt = >format;
-   int ret;
-
-   /*
-* if it is a raw node from vimc-core, ignore the link for now
-* TODO: remove this when there are no more raw nodes in the
-* core and return error instead
-*/
-   if (link->source->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
-   return 0;
-
-   /* Get the the format of the subdev */
-   ret = vimc_cap_v4l2_subdev_link_validate_get_format(link->source,
-   _fmt);
-   if (ret)
-   return ret;
-
-   dev_dbg(vcap->vdev.v4l2_dev->dev,
-   "%s: link validate formats src:%dx%d %d sink:%dx%d %d\n",
-   vcap->vdev.name,
-   source_fmt.format.width, source_fmt.format.height,
-   source_fmt.format.code,
-   sink_fmt->width, sink_fmt->height,
-   sink_fmt->pixelformat);
-
-   /* The width, height and code must match. */
-   vpix = vimc_pix_map_by_pixelformat(sink_fmt->pixelformat);
-   if (source_fmt.format.width != sink_fmt->width
-   || source_fmt.format.height != sink_fmt->height
-   || vpix->code != source_fmt.format.code)
-   return -EPIPE;
-
-   /*
-* The field order must match, or the sink field order must be NONE
-* to support interlaced hardware connected to bridges that support
-* progressive formats only.
-*/
-   if (source_fmt.format.field != sink_fmt->field &&
-   sink_fmt->field != V4L2_FIELD_NONE)
-   return -EPIPE;
-
-   return 0;
-}
-
 static const struct media_entity_operations vimc_cap_mops = {
-   .link_validate  = vimc_cap_link_validate,
+   .link_validate  = vimc_link_validate,
 };
 
 static void vimc_cap_destroy(struct vimc_ent_device *ved)
@@ -434,6 +377,7 @@ struct vimc_ent_device *vimc_cap_create(struct v4l2_device 
*v4l2_dev,
vcap->ved.destroy = vimc_cap_destroy;
vcap->ved.ent = >vdev.entity;
vcap->ved.process_frame = vimc_cap_process_frame;
+   vcap->ved.vdev_get_format = vimc_cap_

[PATCH v4 10/11] [media] vimc: deb: Add debayer filter

[Helen Koike]

Implement the debayer filter and integrate it with the core

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4:
[media] vimc: deb: Add debayer filter
- Rebase without [media] vimc: Optimize frame generation through
pipe
- use vimc_colorimetry_clamp
- replace V4L2_COLORSPACE_SRGB by V4L2_COLORSPACE_DEFAULT in the
default format struct
- use struct vimc_platform_data to retrieve the entity's name

Changes in v3:
[media] vimc: deb: Add debayer filter
- Declare frame_size as a local variable
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: add ret variable to propagate return errors
- structure code to be a module, use platform_driver and component 
system
- fix multiline comment
- s/thought/through
- s/RGB/RGB888
- clamp height and width of the image by an even value
- if user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- uset _DEFAULT for colorimetry in the default format

Changes in v2:
[media] vimc: deb: Add debayer filter
- Using MEDIA_ENT_F_ATV_DECODER in function
- remove v4l2_dev and dev from vimc_deb_device struct
- src fmt propagates from the sink
- coding style
- remove redundant else if statements
- check end of enum and remove BUG_ON
- enum frame size with min and max values
- set/try fmt
- remove unecessary include freezer.h
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add deb_mean_win_size as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg


---
 drivers/media/platform/vimc/Makefile   |   4 +-
 drivers/media/platform/vimc/vimc-common.h  |   2 +
 drivers/media/platform/vimc/vimc-debayer.c | 601 +
 3 files changed, 606 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/vimc/vimc-debayer.c

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index 0e5d5ce..4fba8ef 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,6 +1,8 @@
 vimc-objs := vimc-core.o
 vimc_capture-objs := vimc-capture.o
 vimc_common-objs := vimc-common.o
+vimc_debayer-objs := vimc-debayer.o
 vimc_sensor-objs := vimc-sensor.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc_sensor.o
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc-debayer.o 
\
+   vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index a9c1cfd..25ba752 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -27,6 +27,8 @@
 #define VIMC_FRAME_MIN_WIDTH 16
 #define VIMC_FRAME_MIN_HEIGHT 16
 
+#define VIMC_FRAME_INDEX(lin, col, width, bpp) ((lin * width + col) * bpp)
+
 /**
  * struct vimc_colorimetry_clamp - Adjust colorimetry parameters
  *
diff --git a/drivers/media/platform/vimc/vimc-debayer.c 
b/drivers/media/platform/vimc/vimc-debayer.c
new file mode 100644
index 000..e23a5b9
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -0,0 +1,601 @@
+/*
+ * vimc-debayer.c Virtual Media Controller Driver
+ *
+ * Copyright (C) 2015-2017 Helen Koike <helen.fornaz...@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "vimc-common.h"
+
+#define VIMC_DEB_DRV_NAME "vimc-debayer"
+
+static unsigned int deb_mean_win_size = 3;
+module_param(deb_mean_win_size, uint, );
+MODULE_PARM_DESC(deb_mean_win_size, " the window size to calculate the mean.\n"
+   "NOTE: the window size need to be an odd number, as the main pixel "
+   "stays in the center of the window, otherwise the next odd number "
+   "is considered");
+
+#define IS_SINK(pad) (!pad)
+#define IS_SRC(pad)  (pad)
+
+enum vimc_deb_rgb_colors {
+   VIMC_DEB_RED = 0,
+   VIMC_DEB_GREEN = 1,
+   VIMC_DEB_BLUE = 2,
+};
+
+struct vimc_deb_pix_map {
+

[PATCH v4 11/11] [media] vimc: sca: Add scaler

[Helen Koike]

Implement scaler and integrated with the core

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4:
[media] vimc: sca: Add scaler
- use vimc_colorimetry_clamp
- replace V4L2_COLORSPACE_SRGB by V4L2_COLORSPACE_DEFAULT in the
default format struct
- use struct vimc_platform_data to retrieve the entity's name

Changes in v3:
[media] vimc: sca: Add scaler
- Declare frame_size as a local variable
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: add ret variable to propagate return errors
- structure code to be a module, use platform_driver and component 
system
- s/thought/through
- clamp height and width of the image by an even value
- if user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- uset _DEFAULT for colorimetry in the default format

Changes in v2:
[media] vimc: sca: Add scaler
- Add function MEDIA_ENT_F_IO_V4L
- remove v4l2_dev and dev
- s/sink_mbus_fmt/sink_fmt
- remove BUG_ON, remove redundant if else, rewrite TODO, check end of 
enum
- rm src_width/height, enum fsize with min and max values
- set/try fmt
- remove unecessary include freezer.h
- core: add bayer boolean in pixel table
- coding style
- fix bug in enum frame size
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add sca_mult as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg


---
 drivers/media/platform/vimc/Makefile  |   3 +-
 drivers/media/platform/vimc/vimc-common.c |  27 ++
 drivers/media/platform/vimc/vimc-common.h |   1 +
 drivers/media/platform/vimc/vimc-scaler.c | 455 ++
 4 files changed, 485 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/vimc/vimc-scaler.c

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index 4fba8ef..68c5d98 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -2,7 +2,8 @@ vimc-objs := vimc-core.o
 vimc_capture-objs := vimc-capture.o
 vimc_common-objs := vimc-common.o
 vimc_debayer-objs := vimc-debayer.o
+vimc_scaler-objs := vimc-scaler.o
 vimc_sensor-objs := vimc-sensor.o
 
 obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc-debayer.o 
\
-   vimc_sensor.o
+   vimc_scaler.o vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index da7f2b7..9d63c84 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -20,6 +20,10 @@
 
 #include "vimc-common.h"
 
+/*
+ * NOTE: non-bayer formats need to come first (necessary for enum_mbus_code
+ * in the scaler)
+ */
 static const struct vimc_pix_map vimc_pix_map_list[] = {
/* TODO: add all missing formats */
 
@@ -28,16 +32,19 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
.code = MEDIA_BUS_FMT_BGR888_1X24,
.pixelformat = V4L2_PIX_FMT_BGR24,
.bpp = 3,
+   .bayer = false,
},
{
.code = MEDIA_BUS_FMT_RGB888_1X24,
.pixelformat = V4L2_PIX_FMT_RGB24,
.bpp = 3,
+   .bayer = false,
},
{
.code = MEDIA_BUS_FMT_ARGB_1X32,
.pixelformat = V4L2_PIX_FMT_ARGB32,
.bpp = 4,
+   .bayer = false,
},
 
/* Bayer formats */
@@ -45,41 +52,49 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
.code = MEDIA_BUS_FMT_SBGGR8_1X8,
.pixelformat = V4L2_PIX_FMT_SBGGR8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGBRG8_1X8,
.pixelformat = V4L2_PIX_FMT_SGBRG8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGRBG8_1X8,
.pixelformat = V4L2_PIX_FMT_SGRBG8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SRGGB8_1X8,
.pixelformat = V4L2_PIX_FMT_SRGGB8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SBGGR10_1X10,
.pixelformat = V4L2_PIX_FMT_SBGGR10,
.bpp = 2,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGBRG10_1X10,

[PATCH v4 06/11] [media] vimc: common: Add vimc_colorimetry_clamp

[Helen Koike]

Colorimetry value will always be checked in the same way. Adding a
helper macro for that

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4:
[media] vimc: common: Add vimc_colorimetry_clamp
- this is a new patch in the series

Changes in v3: None
Changes in v2: None


---
 drivers/media/platform/vimc/vimc-common.h | 26 ++
 1 file changed, 26 insertions(+)

diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 60ebde2..43483ee 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -23,6 +23,32 @@
 #include 
 
 /**
+ * struct vimc_colorimetry_clamp - Adjust colorimetry parameters
+ *
+ * @fmt:   the pointer to struct v4l2_pix_format or
+ * struct v4l2_mbus_framefmt
+ *
+ * Entities must check if colorimetry given by the userspace is valid, if not
+ * then set them as DEFAULT
+ */
+#define vimc_colorimetry_clamp(fmt)\
+do {   \
+   if ((fmt)->colorspace == V4L2_COLORSPACE_DEFAULT\
+   || (fmt)->colorspace > V4L2_COLORSPACE_DCI_P3) {\
+   (fmt)->colorspace = V4L2_COLORSPACE_DEFAULT;\
+   (fmt)->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;  \
+   (fmt)->quantization = V4L2_QUANTIZATION_DEFAULT;\
+   (fmt)->xfer_func = V4L2_XFER_FUNC_DEFAULT;  \
+   }   \
+   if ((fmt)->ycbcr_enc > V4L2_YCBCR_ENC_SMPTE240M)\
+   (fmt)->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;  \
+   if ((fmt)->quantization > V4L2_QUANTIZATION_LIM_RANGE)  \
+   (fmt)->quantization = V4L2_QUANTIZATION_DEFAULT;\
+   if ((fmt)->xfer_func > V4L2_XFER_FUNC_SMPTE2084)\
+   (fmt)->xfer_func = V4L2_XFER_FUNC_DEFAULT;  \
+} while (0)
+
+/**
  * struct vimc_pix_map - maps media bus code with v4l2 pixel format
  *
  * @code:  media bus format code defined by MEDIA_BUS_FMT_* macros
-- 
2.7.4

[PATCH v4 08/11] [media] vimc: cap: Support several image formats

[Helen Koike]

Allow user space to change the image format as the frame size, the
pixel format, colorspace, quantization, field YCbCr encoding
and the transfer function

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4:
[media] vimc: cap: Support several image formats
- add vimc_colorimetry_clamp macro
- replace V4L2_COLORSPACE_SRGB by V4L2_COLORSPACE_DEFAULT in the
default format struct

Changes in v3:
[media] vimc: cap: Support several image formats
- use *_DEFAULT macros for colorimetry in the default format
- clamp height and width of the image by an even value
- is user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- remove V4L2_FMT_FLAG_COMPRESSED from vimc_cap_enum_fmt_vid_cap
- remove V4L2_BUF_TYPE_VIDEO_CAPTURE from vimc_cap_enum_fmt_vid_cap
- increase step_width and step_height to 2 instead of 1
- remove link validate function, use the one in vimc-common.c

Changes in v2:
[media] vimc: cap: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- allow user space to change all fields from struct v4l2_pix_format
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- link_validate and try_fmt: also checks colospace, quantization, 
field, xfer_func, ycbcr_enc
- add struct v4l2_pix_format fmt_default
- add enum_framesizes
- enum_fmt_vid_cap: enumerate all formats from vimc_pix_map_table
- add mode dev_dbg


---
 drivers/media/platform/vimc/vimc-capture.c | 117 +
 1 file changed, 101 insertions(+), 16 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 5bdecd1..359f59e 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -40,6 +40,14 @@ struct vimc_cap_device {
struct media_pipeline pipe;
 };
 
+static const struct v4l2_pix_format fmt_default = {
+   .width = 640,
+   .height = 480,
+   .pixelformat = V4L2_PIX_FMT_RGB24,
+   .field = V4L2_FIELD_NONE,
+   .colorspace = V4L2_COLORSPACE_DEFAULT,
+};
+
 struct vimc_cap_buffer {
/*
 * struct vb2_v4l2_buffer must be the first element
@@ -73,7 +81,7 @@ static void vimc_cap_get_format(struct vimc_ent_device *ved,
*fmt = vcap->format;
 }
 
-static int vimc_cap_fmt_vid_cap(struct file *file, void *priv,
+static int vimc_cap_g_fmt_vid_cap(struct file *file, void *priv,
  struct v4l2_format *f)
 {
struct vimc_cap_device *vcap = video_drvdata(file);
@@ -83,16 +91,98 @@ static int vimc_cap_fmt_vid_cap(struct file *file, void 
*priv,
return 0;
 }
 
+static int vimc_cap_try_fmt_vid_cap(struct file *file, void *priv,
+   struct v4l2_format *f)
+{
+   struct v4l2_pix_format *format = >fmt.pix;
+   const struct vimc_pix_map *vpix;
+
+   format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
+   VIMC_FRAME_MAX_WIDTH) & ~1;
+   format->height = clamp_t(u32, format->height, VIMC_FRAME_MIN_HEIGHT,
+VIMC_FRAME_MAX_HEIGHT) & ~1;
+
+   /* Don't accept a pixelformat that is not on the table */
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   if (!vpix) {
+   format->pixelformat = fmt_default.pixelformat;
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   }
+   /* TODO: Add support for custom bytesperline values */
+   format->bytesperline = format->width * vpix->bpp;
+   format->sizeimage = format->bytesperline * format->height;
+
+   if (format->field == V4L2_FIELD_ANY)
+   format->field = fmt_default.field;
+
+   vimc_colorimetry_clamp(format);
+
+   return 0;
+}
+
+static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+   struct vimc_cap_device *vcap = video_drvdata(file);
+
+   /* Do not change the format while stream is on */
+   if (vb2_is_busy(>queue))
+   return -EBUSY;
+
+   vimc_cap_try_fmt_vid_cap(file, priv, f);
+
+   dev_dbg(vcap->vdev.v4l2_dev->dev, "%s: format update: "
+   "old:%dx%d (0x%x, %d, %d, %d, %d) "
+   "new:%dx%d (0x%x, %d, %d, %d, %d)\n", vcap->vdev.name,
+   /* old */
+   vcap->format.width, vcap->format.height,
+   vcap->format.pixelformat, vcap->format.colorspace,
+   vcap->format.quantization, vcap->format.xfer_func,
+   vcap->format.ycbcr_enc,
+   /* new */
+   f->fmt.pix.

[PATCH v4 09/11] [media] vimc: Subdevices as modules

[Helen Koike]

Change the core structure for adding subdevices in the topology.
Instead of calling the specific create function for each subdevice,
inject a child platform_device with the driver's name.
Each type of node in the topology (sensor, capture, debayer, scaler)
will register a platform_driver with the corresponding name through the
component subsystem.
Implementing a new subdevice type doesn't require vimc-core to be altered.

This facilitates future implementation of dynamic entities, where
hotpluging an entity in the topology is just a matter of
registering/unregistering a platform_device in the system.
It also facilitates other implementations of different nodes without
touching the core code and remove the need of a header file for each
type of node.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4:
[media] vimc: Subdevices as modules
- Rebase without [media] vimc: Optimize frame generation the through
pipe
- s/EXPORT_SYMBOL/EXPORT_SYMBOL_GPL
- add struct vimc_platform_data to pass the entity's name to the
sudmodule
- Fix comment about vimc-input (remove vimc-output comment)

Changes in v3:
[media] vimc: Subdevices as modules
- This is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/Makefile   |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  99 ---
 drivers/media/platform/vimc/vimc-capture.h |  28 --
 drivers/media/platform/vimc/vimc-common.c  |  38 +--
 drivers/media/platform/vimc/vimc-common.h  |  29 ++-
 drivers/media/platform/vimc/vimc-core.c| 405 +++--
 drivers/media/platform/vimc/vimc-sensor.c  |  93 +--
 drivers/media/platform/vimc/vimc-sensor.h  |  28 --
 8 files changed, 339 insertions(+), 388 deletions(-)
 delete mode 100644 drivers/media/platform/vimc/vimc-capture.h
 delete mode 100644 drivers/media/platform/vimc/vimc-sensor.h

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index 6b6ddf4..0e5d5ce 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,3 +1,6 @@
-vimc-objs := vimc-core.o vimc-capture.o vimc-common.o vimc-sensor.o
+vimc-objs := vimc-core.o
+vimc_capture-objs := vimc-capture.o
+vimc_common-objs := vimc-common.o
+vimc_sensor-objs := vimc-sensor.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 359f59e..14cb32e 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -15,15 +15,21 @@
  *
  */
 
+#include 
+#include 
+#include 
 #include 
 #include 
 #include 
 
-#include "vimc-capture.h"
+#include "vimc-common.h"
+
+#define VIMC_CAP_DRV_NAME "vimc-capture"
 
 struct vimc_cap_device {
struct vimc_ent_device ved;
struct video_device vdev;
+   struct device *dev;
struct v4l2_pix_format format;
struct vb2_queue queue;
struct list_head buf_list;
@@ -131,7 +137,7 @@ static int vimc_cap_s_fmt_vid_cap(struct file *file, void 
*priv,
 
vimc_cap_try_fmt_vid_cap(file, priv, f);
 
-   dev_dbg(vcap->vdev.v4l2_dev->dev, "%s: format update: "
+   dev_dbg(vcap->dev, "%s: format update: "
"old:%dx%d (0x%x, %d, %d, %d, %d) "
"new:%dx%d (0x%x, %d, %d, %d, %d)\n", vcap->vdev.name,
/* old */
@@ -309,8 +315,7 @@ static int vimc_cap_buffer_prepare(struct vb2_buffer *vb)
unsigned long size = vcap->format.sizeimage;
 
if (vb2_plane_size(vb, 0) < size) {
-   dev_err(vcap->vdev.v4l2_dev->dev,
-   "%s: buffer too small (%lu < %lu)\n",
+   dev_err(vcap->dev, "%s: buffer too small (%lu < %lu)\n",
vcap->vdev.name, vb2_plane_size(vb, 0), size);
return -EINVAL;
}
@@ -335,8 +340,10 @@ static const struct media_entity_operations vimc_cap_mops 
= {
.link_validate  = vimc_link_validate,
 };
 
-static void vimc_cap_destroy(struct vimc_ent_device *ved)
+static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
+void *master_data)
 {
+   struct vimc_ent_device *ved = dev_get_drvdata(comp);
struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
ved);
 
@@ -385,42 +392,35 @@ static void vimc_cap_process_frame(struct vimc_ent_device 
*ved,
vb2_buffer_done(_buf->vb2.vb2_buf, VB2_BUF_STATE_DONE);
 }
 
-struct vimc_ent_device *vimc_cap_create(struct v4l2_device *v4l2_dev,
-   const char *const name,
-

[PATCH v4 07/11] [media] vimc: sen: Support several image formats

[Helen Koike]

Allow user space to change the image format as the frame size, the
media bus pixel format, colorspace, quantization, field YCbCr encoding
and the transfer function

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v4:
[media] vimc: sen: Support several image formats
- use vimc_colorimetry_clamp macro
- replace V4L2_COLORSPACE_SRGB by V4L2_COLORSPACE_DEFAULT in the
default format struct

Changes in v3:
[media] vimc: sen: Support several image formats
- remove support for V4L2_FIELD_ALTERNATE (left as TODO for now)
- clamp image size to an even dimension for height and width
- set default values for colorimetry using _DEFAULT macro
- reset all values of colorimetry to _DEFAULT if user tries to
set an invalid colorspace

Changes in v2:
[media] vimc: sen: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- add init_cfg to initialize try_fmt
- reorder code in vimc_sen_set_fmt
- allow user space to change all fields from struct v4l2_mbus_framefmt
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- merge with patch for the enum_mbus_code and enum_frame_size
- change commit message
- add vimc_pix_map_by_index
- rename MIN/MAX macros
- check set_fmt default parameters for quantization, colorspace 
...media] vimc: sen: Support several image formats


---
 drivers/media/platform/vimc/vimc-common.c |   8 ++
 drivers/media/platform/vimc/vimc-common.h |  12 +++
 drivers/media/platform/vimc/vimc-sensor.c | 130 +++---
 3 files changed, 121 insertions(+), 29 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 6ad77fd..b698055 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -144,6 +144,14 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
},
 };
 
+const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i)
+{
+   if (i >= ARRAY_SIZE(vimc_pix_map_list))
+   return NULL;
+
+   return _pix_map_list[i];
+}
+
 const struct vimc_pix_map *vimc_pix_map_by_code(u32 code)
 {
unsigned int i;
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 43483ee..fb3463c 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -22,6 +22,11 @@
 #include 
 #include 
 
+#define VIMC_FRAME_MAX_WIDTH 4096
+#define VIMC_FRAME_MAX_HEIGHT 2160
+#define VIMC_FRAME_MIN_WIDTH 16
+#define VIMC_FRAME_MIN_HEIGHT 16
+
 /**
  * struct vimc_colorimetry_clamp - Adjust colorimetry parameters
  *
@@ -139,6 +144,13 @@ static inline void vimc_pads_cleanup(struct media_pad 
*pads)
 int vimc_pipeline_s_stream(struct media_entity *ent, int enable);
 
 /**
+ * vimc_pix_map_by_index - get vimc_pix_map struct by its index
+ *
+ * @i: index of the vimc_pix_map struct in vimc_pix_map_list
+ */
+const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i);
+
+/**
  * vimc_pix_map_by_code - get vimc_pix_map struct by media bus code
  *
  * @code:  media bus format code defined by MEDIA_BUS_FMT_* macros
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 6386ac1..d4f9705 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -24,8 +24,6 @@
 
 #include "vimc-sensor.h"
 
-#define VIMC_SEN_FRAME_MAX_WIDTH 4096
-
 struct vimc_sen_device {
struct vimc_ent_device ved;
struct v4l2_subdev sd;
@@ -36,18 +34,39 @@ struct vimc_sen_device {
struct v4l2_mbus_framefmt mbus_format;
 };
 
+static const struct v4l2_mbus_framefmt fmt_default = {
+   .width = 640,
+   .height = 480,
+   .code = MEDIA_BUS_FMT_RGB888_1X24,
+   .field = V4L2_FIELD_NONE,
+   .colorspace = V4L2_COLORSPACE_DEFAULT,
+};
+
+static int vimc_sen_init_cfg(struct v4l2_subdev *sd,
+struct v4l2_subdev_pad_config *cfg)
+{
+   unsigned int i;
+
+   for (i = 0; i < sd->entity.num_pads; i++) {
+   struct v4l2_mbus_framefmt *mf;
+
+   mf = v4l2_subdev_get_try_format(sd, cfg, i);
+   *mf = fmt_default;
+   }
+
+   return 0;
+}
+
 static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
   struct v4l2_subdev_pad_config *cfg,
   struct v4l2_subdev_mbus_code_enum *code)
 {
-   struct vimc_sen_device *vsen =
-   container_of(sd, struct vimc_sen_device, sd);
+   const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
 
-   /* TODO: Add support for other codes */
-   if (code->index)
+   if (!vpix)

[PATCH v4 00/11] [media]: vimc: Virtual Media Control VPU's

[Helen Koike]

- check set_fmt default parameters for quantization, colorspace ...
    - add more dev_dbg

Helen Koike (11):
  [media] vimc: sen: Integrate the tpg on the sensor
  [media] vimc: Move common code from the core
  [media] vimc: common: Add vimc_ent_sd_* helper
  [media] vimc: common: Add vimc_pipeline_s_stream helper
  [media] vimc: common: Add vimc_link_validate
  [media] vimc: common: Add vimc_colorimetry_clamp
  [media] vimc: sen: Support several image formats
  [media] vimc: cap: Support several image formats
  [media] vimc: Subdevices as modules
  [media] vimc: deb: Add debayer filter
  [media] vimc: sca: Add scaler

 drivers/media/platform/vimc/Kconfig|   1 +
 drivers/media/platform/vimc/Makefile   |  10 +-
 drivers/media/platform/vimc/vimc-capture.c | 321 ---
 drivers/media/platform/vimc/vimc-capture.h |  28 --
 drivers/media/platform/vimc/vimc-common.c  | 473 ++
 drivers/media/platform/vimc/vimc-common.h  | 229 +++
 drivers/media/platform/vimc/vimc-core.c| 610 -
 drivers/media/platform/vimc/vimc-core.h| 112 --
 drivers/media/platform/vimc/vimc-debayer.c | 601 
 drivers/media/platform/vimc/vimc-scaler.c  | 455 +
 drivers/media/platform/vimc/vimc-sensor.c  | 319 ++-
 drivers/media/platform/vimc/vimc-sensor.h  |  28 --
 12 files changed, 2324 insertions(+), 863 deletions(-)
 delete mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-common.c
 create mode 100644 drivers/media/platform/vimc/vimc-common.h
 delete mode 100644 drivers/media/platform/vimc/vimc-core.h
 create mode 100644 drivers/media/platform/vimc/vimc-debayer.c
 create mode 100644 drivers/media/platform/vimc/vimc-scaler.c
 delete mode 100644 drivers/media/platform/vimc/vimc-sensor.h

-- 
2.7.4

Re: [RFC PATCH v3 09/11] [media] vimc: Subdevices as modules

[Helen Koike]

Hi Hans,

On 2017-06-13 03:49 AM, Hans Verkuil wrote:

On 06/12/2017 10:35 PM, Helen Koike wrote:

Hi Hans,

Thank you for your review. Please check my comments below

On 2017-06-12 07:37 AM, Hans Verkuil wrote:

On 06/03/2017 04:58 AM, Helen Koike wrote:

+static struct component_match *vimc_add_subdevs(struct vimc_device
*vimc)
+{
+struct component_match *match = NULL;
+unsigned int i;
+
+for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
+dev_dbg(>pdev.dev, "new pdev for %s\n",
+vimc->pipe_cfg->ents[i].drv);
+
+/*
+ * TODO: check if using platform_data is indeed the best
way to
+ * pass the name to the driver or if we should add the drv
name
+ * in the platform_device_id table
+ */


Didn't you set the drv name in the platform_device_id table already?


I refer to the name of the entity, there is the name that identifies the
driver as "vimc-sensor" that is set in the platform_device_id table but
there is also the custom name of the entity e.g. "My Sensor A" that I
need to inform to the vimc-sensor driver.


Ah, so in the TODO you mean:

"the best way to pass the entity name to the driver"


Yes



I got confused there.


Sorry about that, I'll improve this comment (if we don't decide to 
remove it)




But in that case I still don't get what you mean with "add the drv name
in the platform_device_id table". Do you mean "entity name" there as
well?


Yes, it is because there is a member of the platform_device_id table 
called driver_data and I was wondering if this would be more 
appropriated then using the platform_data.


Under Documentation/driver-model/platform.txt it is written:
"In many cases, the memory and IRQ resources associated with the 
platform device are not enough to let the device's driver work.  Board 
setup code

will often provide additional information using the device's platform_data
field to hold additional information."

So I thought that using platform_data to pass the entity's name to the 
driver would be acceptable as it seems to be a "Board setup code" for vimc








Using platform_data feels like an abuse to be honest.


Another option would be to make the vimc-sensor driver to populate the
entity name automatically as "Sensor x", where x could be the entity
number, but I don't think this is a good option.


Why not? Well, probably not the entity number, but a simple instance
counter would do fine (i.e. Sensor 1, 2, 3...).


Because I usually use tests scripts that configure the right pad image 
format to a specific entity's name using media-ctl, I would like an 
assurance that what identifies the entity doesn't change (that doesn't 
depend on the order they are inserted), and I thought that applications 
in userspace might want the same.




It can be made fancier later with dynamic reconfiguration where you
might want to use the first unused instance number.


We could use configfs for that, but I was wondering what the names of 
the folders that represent an entity would mean, if the user do

$ mkdir vimc-sensor-foo
$ mkdir vimc-sensor-bar
Then foo and bar would unused names as the entities would be first 
created under the names "Sensor 1" and "Sensor 2", I think it would be 
nice if they were created as "Sensor foo" and "Sensor bar".








Creating these components here makes sense. Wouldn't it also make sense
to use
v4l2_async to wait until they have all been bound? It would more closely
emulate
standard drivers. Apologies if I misunderstand what is happening here.


I am using linux/component.h for that, when all devices are present and
all modules are loaded, the component.h system brings up the core by
calling vimc_comp_bind() function, which calls component_bind_all() to
call the binding function of each module, then it finishes registering
the topology.
If any one of the components or module is unload, the component system
takes down the entire topology calling component_unbind_all which calls
the unbind functions from each module.
This makes sure that the media device, subdevices and video device are
only registered in the v4l2 system if all the modules are loaded.

I wans't familiar with v4l2-async.h, but from a quick look it seems that
it only works with struct v4l2_subdev, but I'll also need for struct
video_device (for the capture node for example).
And also, if a module is missing we would have vimc partially
registered, e.g. the debayer could be registered at /dev/subdevX but the
sensor not yet and the media wouldn't be ready, I am not sure if this is
a problem though.

Maybe we can use component.h for now, then I can implement
v4l2_async_{un}register_video_device and migrate to v4l2-sync.h latter.
What do you think?


That's OK. The v4l2-async mechanism precedes the component API. We should
probably investigate moving over to the 

Re: [RFC PATCH v3 09/11] [media] vimc: Subdevices as modules

[Helen Koike]

Hi Hans,

Thank you for your review. Please check my comments below

On 2017-06-12 07:37 AM, Hans Verkuil wrote:

On 06/03/2017 04:58 AM, Helen Koike wrote:

Change the core structure for adding subdevices in the topology.
Instead of calling the specific create function for each subdevice,
inject a child platform_device with the driver's name.
Each type of node in the topology (sensor, capture, debayer, scaler)
will register a platform_driver with the corresponding name through the
component subsystem.
Implementing a new subdevice type doesn't require vimc-core to be
altered.

This facilitates future implementation of dynamic entities, where
hotpluging an entity in the topology is just a matter of
registering/unregistering a platform_device in the system.
It also facilitates other implementations of different nodes without
touching the core code and remove the need of a header file for each
type of node.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: Subdevices as modules
- This is a new patch in the series

Changes in v2: None


---
  drivers/media/platform/vimc/Makefile   |   7 +-
  drivers/media/platform/vimc/vimc-capture.c |  96 ---
  drivers/media/platform/vimc/vimc-capture.h |  28 --
  drivers/media/platform/vimc/vimc-common.c  |  37 ++-
  drivers/media/platform/vimc/vimc-common.h  |  14 +-
  drivers/media/platform/vimc/vimc-core.c| 406
+++--
  drivers/media/platform/vimc/vimc-sensor.c  |  91 +--
  drivers/media/platform/vimc/vimc-sensor.h  |  28 --
  8 files changed, 320 insertions(+), 387 deletions(-)
  delete mode 100644 drivers/media/platform/vimc/vimc-capture.h
  delete mode 100644 drivers/media/platform/vimc/vimc-sensor.h

diff --git a/drivers/media/platform/vimc/Makefile
b/drivers/media/platform/vimc/Makefile
index 6b6ddf4..0e5d5ce 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,3 +1,6 @@
-vimc-objs := vimc-core.o vimc-capture.o vimc-common.o vimc-sensor.o
+vimc-objs := vimc-core.o
+vimc_capture-objs := vimc-capture.o
+vimc_common-objs := vimc-common.o
+vimc_sensor-objs := vimc-sensor.o
  -obj-$(CONFIG_VIDEO_VIMC) += vimc.o
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o
vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-capture.c
b/drivers/media/platform/vimc/vimc-capture.c
index b5da0ea..633d99a 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -15,18 +15,24 @@
   *
   */
  +#include 
  #include 
  #include 
+#include 
+#include 
  #include 
  #include 
  #include 
  #include 
  -#include "vimc-capture.h"
+#include "vimc-common.h"
+
+#define VIMC_CAP_DRV_NAME "vimc-capture"
struct vimc_cap_device {
  struct vimc_ent_device ved;
  struct video_device vdev;
+struct device *dev;
  struct v4l2_pix_format format;
  struct vb2_queue queue;
  struct list_head buf_list;
@@ -150,7 +156,7 @@ static int vimc_cap_s_fmt_vid_cap(struct file
*file, void *priv,
vimc_cap_try_fmt_vid_cap(file, priv, f);
  -dev_dbg(vcap->vdev.v4l2_dev->dev, "%s: format update: "
+dev_dbg(vcap->dev, "%s: format update: "
  "old:%dx%d (0x%x, %d, %d, %d, %d) "
  "new:%dx%d (0x%x, %d, %d, %d, %d)\n", vcap->vdev.name,
  /* old */
@@ -365,8 +371,7 @@ static int vimc_cap_start_streaming(struct
vb2_queue *vq, unsigned int count)
  vcap->kthread_cap = kthread_run(vimc_cap_tpg_thread, vcap,
  "%s-cap", vcap->vdev.v4l2_dev->name);
  if (IS_ERR(vcap->kthread_cap)) {
-dev_err(vcap->vdev.v4l2_dev->dev,
-"%s: kernel_thread() failed\n",
+dev_err(vcap->dev, "%s: kernel_thread() failed\n",
  vcap->vdev.name);
  ret = PTR_ERR(vcap->kthread_cap);
  goto err_tpg_free;
@@ -443,8 +448,7 @@ static int vimc_cap_buffer_prepare(struct
vb2_buffer *vb)
  unsigned long size = vcap->format.sizeimage;
if (vb2_plane_size(vb, 0) < size) {
-dev_err(vcap->vdev.v4l2_dev->dev,
-"%s: buffer too small (%lu < %lu)\n",
+dev_err(vcap->dev, "%s: buffer too small (%lu < %lu)\n",
  vcap->vdev.name, vb2_plane_size(vb, 0), size);
  return -EINVAL;
  }
@@ -469,8 +473,10 @@ static const struct media_entity_operations
vimc_cap_mops = {
  .link_validate= vimc_link_validate,
  };
  -static void vimc_cap_destroy(struct vimc_ent_device *ved)
+static void vimc_cap_comp_unbind(struct device *comp, struct device
*master,
+ void *master_data)
  {
+struct vimc_ent_device *ved = dev_get_drvdata(comp);
  struct vimc_cap_device *vcap = container_of(ved, struct
vimc_cap_device,
   

Re: [RFC PATCH v3 08/11] [media] vimc: Optimize frame generation through the pipe

[Helen Koike]

Hi Hans,

Thank you for your review

On 2017-06-12 07:03 AM, Hans Verkuil wrote:

On 06/03/2017 04:58 AM, Helen Koike wrote:

Add a parameter called vsen_tpg, if true then vimc will work as before:
frames will be generated in the sensor nodes then propagated through the
pipe and processed by each node until a capture node is reached.
If vsen_tpg is false, then the frame is not generated by the sensor, but
directly from the capture node, thus saving intermediate memory buffers
and process time, allowing a higher frame rate.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: Optimize frame generation through the pipe
- This is a new patch in the series

Changes in v2: None


---
  drivers/media/platform/vimc/vimc-capture.c | 178
+
  drivers/media/platform/vimc/vimc-common.h  |   2 +
  drivers/media/platform/vimc/vimc-sensor.c  |  30 -
  3 files changed, 156 insertions(+), 54 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c
b/drivers/media/platform/vimc/vimc-capture.c
index e943267..b5da0ea 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -15,7 +15,10 @@
   *
   */
  +#include 
+#include 
  #include 
+#include 
  #include 
  #include 
  @@ -38,6 +41,8 @@ struct vimc_cap_device {
  struct mutex lock;
  u32 sequence;
  struct media_pipeline pipe;
+struct tpg_data tpg;
+struct task_struct *kthread_cap;
  };
static const struct v4l2_pix_format fmt_default = {
@@ -246,6 +251,91 @@ static void vimc_cap_return_all_buffers(struct
vimc_cap_device *vcap,
  spin_unlock(>qlock);
  }
  +static void vimc_cap_process_frame(struct vimc_ent_device *ved,
+   struct media_pad *sink, const void *frame)
+{
+struct vimc_cap_device *vcap = container_of(ved, struct
vimc_cap_device,
+ved);
+struct vimc_cap_buffer *vimc_buf;
+void *vbuf;
+
+spin_lock(>qlock);
+
+/* Get the first entry of the list */
+vimc_buf = list_first_entry_or_null(>buf_list,
+typeof(*vimc_buf), list);
+if (!vimc_buf) {
+spin_unlock(>qlock);
+return;
+}
+
+/* Remove this entry from the list */
+list_del(_buf->list);
+
+spin_unlock(>qlock);
+
+/* Fill the buffer */
+vimc_buf->vb2.vb2_buf.timestamp = ktime_get_ns();
+vimc_buf->vb2.sequence = vcap->sequence++;
+vimc_buf->vb2.field = vcap->format.field;
+
+vbuf = vb2_plane_vaddr(_buf->vb2.vb2_buf, 0);
+
+if (sink)
+memcpy(vbuf, frame, vcap->format.sizeimage);
+else
+tpg_fill_plane_buffer(>tpg, V4L2_STD_PAL, 0, vbuf);
+
+/* Set it as ready */
+vb2_set_plane_payload(_buf->vb2.vb2_buf, 0,
+  vcap->format.sizeimage);
+vb2_buffer_done(_buf->vb2.vb2_buf, VB2_BUF_STATE_DONE);
+}
+
+
+static int vimc_cap_tpg_thread(void *data)
+{
+struct vimc_cap_device *vcap = data;
+
+set_freezable();
+set_current_state(TASK_UNINTERRUPTIBLE);
+
+for (;;) {
+try_to_freeze();
+if (kthread_should_stop())
+break;
+
+vimc_cap_process_frame(>ved, NULL, NULL);
+
+/* 60 frames per second */
+schedule_timeout(HZ/60);
+}
+
+return 0;
+}
+
+static void vimc_cap_tpg_s_format(struct vimc_cap_device *vcap)
+{
+const struct vimc_pix_map *vpix =
+vimc_pix_map_by_pixelformat(vcap->format.pixelformat);
+
+tpg_reset_source(>tpg, vcap->format.width,
vcap->format.height,
+ vcap->format.field);
+tpg_s_bytesperline(>tpg, 0, vcap->format.width * vpix->bpp);
+tpg_s_buf_height(>tpg, vcap->format.height);
+tpg_s_fourcc(>tpg, vpix->pixelformat);
+/*
+ * TODO: check why the tpg_s_field need this third argument if
+ * it is already receiving the field
+ */
+tpg_s_field(>tpg, vcap->format.field,
+vcap->format.field == V4L2_FIELD_ALTERNATE);


I thought I explained that earlier? When in alternate field mode the
format.field
value will be TOP or BOTTOM, but never ALTERNATE. So tpg_s_field needs
to know if
it will create TOP or BOTTOM fields only, or if they will be alternating.


Yes, sorry about that. I removed from the vimc_sensor but I forgot it 
here. I am dropping this patch in the series as I found another bug when 
multiple links are enabled going to the same sink pad. I'll re-work in 
this optimization and re-send another version in the future in a 
different patch series.




Since we don't support ALTERNATE anyway, just pass false as the third
argument and
drop the comment.


+tpg_s_colorspace(>tpg, vcap->format.colorspace);
+tpg_s_ycbcr_enc(>tpg, vcap->format.ycbcr_enc);
+tpg_s_quantization(>tpg, vcap->format.quantization);
+tpg_s_xfer_func(>tpg, vcap->format.xfer_func);
+}
+
  static in

Re: [RFC PATCH v3 05/11] [media] vimc: common: Add vimc_link_validate

[Helen Koike]

Hi Hans,

Thanks for your review, just a question below

On 2017-06-12 06:50 AM, Hans Verkuil wrote:

On 06/03/2017 04:58 AM, Helen Koike wrote:

All links will be checked in the same way. Adding a helper function for
that

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: common: Add vimc_link_validate
- this is a new patch in the series

Changes in v2: None


---
  drivers/media/platform/vimc/vimc-capture.c |  78 +++---
  drivers/media/platform/vimc/vimc-common.c  | 124
-
  drivers/media/platform/vimc/vimc-common.h  |  14 
  3 files changed, 148 insertions(+), 68 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c
b/drivers/media/platform/vimc/vimc-capture.c
index 93f6a09..5bdecd1 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -64,6 +64,15 @@ static int vimc_cap_querycap(struct file *file,
void *priv,
  return 0;
  }
  +static void vimc_cap_get_format(struct vimc_ent_device *ved,
+struct v4l2_pix_format *fmt)
+{
+struct vimc_cap_device *vcap = container_of(ved, struct
vimc_cap_device,
+ved);
+
+*fmt = vcap->format;
+}
+
  static int vimc_cap_fmt_vid_cap(struct file *file, void *priv,
struct v4l2_format *f)
  {
@@ -231,74 +240,8 @@ static const struct vb2_ops vimc_cap_qops = {
  .wait_finish= vb2_ops_wait_finish,
  };
  -/*
- * NOTE: this function is a copy of v4l2_subdev_link_validate_get_format
- * maybe the v4l2 function should be public
- */
-static int vimc_cap_v4l2_subdev_link_validate_get_format(struct
media_pad *pad,
-struct v4l2_subdev_format *fmt)
-{
-struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(pad->entity);
-
-fmt->which = V4L2_SUBDEV_FORMAT_ACTIVE;
-fmt->pad = pad->index;
-
-return v4l2_subdev_call(sd, pad, get_fmt, NULL, fmt);
-}
-
-static int vimc_cap_link_validate(struct media_link *link)
-{
-struct v4l2_subdev_format source_fmt;
-const struct vimc_pix_map *vpix;
-struct vimc_cap_device *vcap = container_of(link->sink->entity,
-struct vimc_cap_device,
-vdev.entity);
-struct v4l2_pix_format *sink_fmt = >format;
-int ret;
-
-/*
- * if it is a raw node from vimc-core, ignore the link for now
- * TODO: remove this when there are no more raw nodes in the
- * core and return error instead
- */
-if (link->source->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
-return 0;
-
-/* Get the the format of the subdev */
-ret = vimc_cap_v4l2_subdev_link_validate_get_format(link->source,
-_fmt);
-if (ret)
-return ret;
-
-dev_dbg(vcap->vdev.v4l2_dev->dev,
-"%s: link validate formats src:%dx%d %d sink:%dx%d %d\n",
-vcap->vdev.name,
-source_fmt.format.width, source_fmt.format.height,
-source_fmt.format.code,
-sink_fmt->width, sink_fmt->height,
-sink_fmt->pixelformat);
-
-/* The width, height and code must match. */
-vpix = vimc_pix_map_by_pixelformat(sink_fmt->pixelformat);
-if (source_fmt.format.width != sink_fmt->width
-|| source_fmt.format.height != sink_fmt->height
-|| vpix->code != source_fmt.format.code)
-return -EPIPE;
-
-/*
- * The field order must match, or the sink field order must be NONE
- * to support interlaced hardware connected to bridges that support
- * progressive formats only.
- */
-if (source_fmt.format.field != sink_fmt->field &&
-sink_fmt->field != V4L2_FIELD_NONE)
-return -EPIPE;
-
-return 0;
-}
-
  static const struct media_entity_operations vimc_cap_mops = {
-.link_validate= vimc_cap_link_validate,
+.link_validate= vimc_link_validate,
  };
static void vimc_cap_destroy(struct vimc_ent_device *ved)
@@ -434,6 +377,7 @@ struct vimc_ent_device *vimc_cap_create(struct
v4l2_device *v4l2_dev,
  vcap->ved.destroy = vimc_cap_destroy;
  vcap->ved.ent = >vdev.entity;
  vcap->ved.process_frame = vimc_cap_process_frame;
+vcap->ved.vdev_get_format = vimc_cap_get_format;
/* Initialize the video_device struct */
  vdev = >vdev;
diff --git a/drivers/media/platform/vimc/vimc-common.c
b/drivers/media/platform/vimc/vimc-common.c
index f809a9d..83d4251 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -252,8 +252,130 @@ int vimc_pipeline_s_stream(struct media_entity
*ent, int enable)
  return 0;
  }
  +static void vimc_fmt_pix_to_mbus(struct v4l2_mbus_framefmt *mfmt,
+ struct v4l2_pix_format *pfmt)
+{
+const struct vimc_pix_map *vpix =
+vimc_pix_map_by_pixelformat(pf

[PATCH v2] [media] v4l2-subdev: check colorimetry in link validate

[Helen Koike]

colorspace, ycbcr_enc, quantization and xfer_func must match
across the link.
Check if they match in v4l2_subdev_link_validate_default
unless they are set as _DEFAULT.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Hi,

As discussed previously, I added a warn message instead of returning
error to give drivers some time to adapt.
But the problem is that: as the format is set by userspace, it is
possible that userspace will set the wrong format at pads and see these
messages when there is no error in the driver's code at all (or maybe
this is not a problem, just noise in the log).

Helen
---
 drivers/media/v4l2-core/v4l2-subdev.c | 38 +++
 1 file changed, 38 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c 
b/drivers/media/v4l2-core/v4l2-subdev.c
index da78497..1a642c7 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -508,6 +508,44 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev 
*sd,
|| source_fmt->format.code != sink_fmt->format.code)
return -EPIPE;
 
+   /*
+* TODO: return -EPIPE instead of printing a warning in the following
+* checks. As colorimetry properties were added after most of the
+* drivers, only a warning was added to avoid potential regressions
+*/
+
+   /* colorspace match. */
+   if (source_fmt->format.colorspace != sink_fmt->format.colorspace)
+   dev_warn(sd->v4l2_dev->dev,
+"colorspace doesn't match in link 
\"%s\":%d->\"%s\":%d\n",
+   link->source->entity->name, link->source->index,
+   link->sink->entity->name, link->sink->index);
+
+   /* Colorimetry must match if they are not set to DEFAULT */
+   if (source_fmt->format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT
+   && sink_fmt->format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT
+   && source_fmt->format.ycbcr_enc != sink_fmt->format.ycbcr_enc)
+   dev_warn(sd->v4l2_dev->dev,
+"YCbCr encoding doesn't match in link 
\"%s\":%d->\"%s\":%d\n",
+   link->source->entity->name, link->source->index,
+   link->sink->entity->name, link->sink->index);
+
+   if (source_fmt->format.quantization != V4L2_QUANTIZATION_DEFAULT
+   && sink_fmt->format.quantization != V4L2_QUANTIZATION_DEFAULT
+   && source_fmt->format.quantization != sink_fmt->format.quantization)
+   dev_warn(sd->v4l2_dev->dev,
+"quantization doesn't match in link 
\"%s\":%d->\"%s\":%d\n",
+   link->source->entity->name, link->source->index,
+   link->sink->entity->name, link->sink->index);
+
+   if (source_fmt->format.xfer_func != V4L2_XFER_FUNC_DEFAULT
+   && sink_fmt->format.xfer_func != V4L2_XFER_FUNC_DEFAULT
+   && source_fmt->format.xfer_func != sink_fmt->format.xfer_func)
+   dev_warn(sd->v4l2_dev->dev,
+"transfer function doesn't match in link 
\"%s\":%d->\"%s\":%d\n",
+   link->source->entity->name, link->source->index,
+   link->sink->entity->name, link->sink->index);
+
/* The field order must match, or the sink field order must be NONE
 * to support interlaced hardware connected to bridges that support
 * progressive formats only.
-- 
2.7.4

Re: [RFC PATCH] [media] v4l2-subdev: check colorimetry in link validate

[Helen Koike]

Hi

On 2017-06-08 08:41 AM, Mauro Carvalho Chehab wrote:

Em Tue, 6 Jun 2017 19:15:34 -0300
Helen Koike <helen.ko...@collabora.com> escreveu:


Hi Sakari,


Thanks for replying

On 2017-05-31 03:31 AM, Sakari Ailus wrote:

Hi Helen,

On Tue, May 30, 2017 at 04:08:08PM -0300, Helen Koike wrote:

colorspace, ycbcr_enc, quantization and xfer_func must match across the
link.
Check if they match in v4l2_subdev_link_validate_default unless they are
set as _DEFAULT.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Hi,

I think we should validate colorimetry as having different colorimetry
across a link doesn't make sense.
But I am confused about what to do when they are set to _DEFAULT, what
do you think?


These fields were added at various points over the course of the past three
years or so. User space code that was written before that will certainly not
set anything and I'm not sure many drivers care about these fields nor they
are relevant for many formats. In practice that means that they are very
likely zero in many cases.


If they are set to zero then they won't be affected by this patch.



Driver changes will probably be necessary for removing the explicit checks
for the default value.


At least in the drivers/media tree I couldn't find many drivers that
implement its own link_validate, there is only
platform/omap3isp/ispccdc.c and platform/omap3isp/ispresizer.c that
implements a custom value, but from a quick look it doesn't seems that
they will be affected.



The same applies to checking the colour space: drivers should enforce using
the correct colour space before the check can be merged. I might move that
to a separate patch.


I am not sure if I got what you mean. If driver don't care about
colourspace then probably it will be set to zero and won't be affected
by this patch, if colourspace is different across the link then the user
space must set both ends to the same colourspace.


I guess what Sakari is concerned about is to avoid regressions.

Colorimetry properties were added after the addition of most of
the drivers. Adding a mandatory link validation may break drivers
that don't set it right. Even on new drivers, this may not be ok.

Just to give you an example, this week I just applied a patch fixing
colorimetry handling at the coda driver:

https://git.linuxtv.org/media_tree.git/commit/?id=b14ac545688d8cc4b2b707d71d106799ad476964

If a change like that would have been applied before such fix, it
could be breaking coda.


If I understand correct, coda doesn't have subdevs or links so it 
wouldn't be affected by this patch.




So, before adding such patch, we need to check how existing drivers are
setting colorimetry fields, to be sure that the ones that don't
touch it won't break.


In my view, if they don't touch colorimetry they shouldn't be affected 
by this patch as values would be zero, and there are very few drivers 
that implement a custom link validation under driver/media tree, and 
from a quick look it doesn't seems to change colorimetry values.




Perhaps one alternative would be to write a patchset that would,
instead, print a warning at dmesg, and let it be upstream for a
while, to give people time to check if the colorimetry logic is
ok at the subdevs.


Agreed, lets start with that, I'll re-send this patch which this change.

Thanks
Helen

Re: [RFC PATCH] [media] v4l2-subdev: check colorimetry in link validate

[Helen Koike]

Hi Sakari,


Thanks for replying

On 2017-05-31 03:31 AM, Sakari Ailus wrote:

Hi Helen,

On Tue, May 30, 2017 at 04:08:08PM -0300, Helen Koike wrote:

colorspace, ycbcr_enc, quantization and xfer_func must match across the
link.
Check if they match in v4l2_subdev_link_validate_default unless they are
set as _DEFAULT.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Hi,

I think we should validate colorimetry as having different colorimetry
across a link doesn't make sense.
But I am confused about what to do when they are set to _DEFAULT, what
do you think?


These fields were added at various points over the course of the past three
years or so. User space code that was written before that will certainly not
set anything and I'm not sure many drivers care about these fields nor they
are relevant for many formats. In practice that means that they are very
likely zero in many cases.


If they are set to zero then they won't be affected by this patch.



Driver changes will probably be necessary for removing the explicit checks
for the default value.


At least in the drivers/media tree I couldn't find many drivers that 
implement its own link_validate, there is only 
platform/omap3isp/ispccdc.c and platform/omap3isp/ispresizer.c that 
implements a custom value, but from a quick look it doesn't seems that 
they will be affected.




The same applies to checking the colour space: drivers should enforce using
the correct colour space before the check can be merged. I might move that
to a separate patch.


I am not sure if I got what you mean. If driver don't care about 
colourspace then probably it will be set to zero and won't be affected 
by this patch, if colourspace is different across the link then the user 
space must set both ends to the same colourspace.






Thanks
---
 drivers/media/v4l2-core/v4l2-subdev.c | 21 +++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c 
b/drivers/media/v4l2-core/v4l2-subdev.c
index da78497..784ae92 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -502,10 +502,27 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev 
*sd,
  struct v4l2_subdev_format *source_fmt,
  struct v4l2_subdev_format *sink_fmt)
 {
-   /* The width, height and code must match. */
+   /* The width, height, code and colorspace must match. */
if (source_fmt->format.width != sink_fmt->format.width
|| source_fmt->format.height != sink_fmt->format.height
-   || source_fmt->format.code != sink_fmt->format.code)
+   || source_fmt->format.code != sink_fmt->format.code
+   || source_fmt->format.colorspace != sink_fmt->format.colorspace)
+   return -EPIPE;
+
+   /* Colorimetry must match if they are not set to DEFAULT */
+   if (source_fmt->format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT
+   && sink_fmt->format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT
+   && source_fmt->format.ycbcr_enc != sink_fmt->format.ycbcr_enc)
+   return -EPIPE;
+
+   if (source_fmt->format.quantization != V4L2_QUANTIZATION_DEFAULT
+   && sink_fmt->format.quantization != V4L2_QUANTIZATION_DEFAULT
+   && source_fmt->format.quantization != sink_fmt->format.quantization)
+   return -EPIPE;
+
+   if (source_fmt->format.xfer_func != V4L2_XFER_FUNC_DEFAULT
+   && sink_fmt->format.xfer_func != V4L2_XFER_FUNC_DEFAULT
+   && source_fmt->format.xfer_func != sink_fmt->format.xfer_func)
return -EPIPE;

/* The field order must match, or the sink field order must be NONE




LN

Re: [PATCH RFC 1/2] [media] v4l2: add V4L2_INPUT_TYPE_DEFAULT

[Helen Koike]

Hi All,

Just reviving this discussion

On 2017-04-07 06:53 AM, Laurent Pinchart wrote:

Hi Hans,

On Friday 07 Apr 2017 11:46:48 Hans Verkuil wrote:

On 04/04/2017 03:22 PM, Sakari Ailus wrote:

On Mon, Apr 03, 2017 at 12:11:54PM -0300, Helen Koike wrote:

On 2017-03-31 06:57 AM, Mauro Carvalho Chehab wrote:

Em Fri, 31 Mar 2017 10:29:04 +0200 Hans Verkuil escreveu:

On 30/03/17 18:02, Helen Koike wrote:

Add V4L2_INPUT_TYPE_DEFAULT and helpers functions for input ioctls to
be used when no inputs are available in the device

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
drivers/media/v4l2-core/v4l2-ioctl.c | 27 +++
include/media/v4l2-ioctl.h   | 26 ++
include/uapi/linux/videodev2.h   |  1 +
3 files changed, 54 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c index 0c3f238..ccaf04b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2573,6 +2573,33 @@ struct mutex *v4l2_ioctl_get_lock(struct
video_device *vdev, unsigned cmd)
return vdev->lock;
}

+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+ struct v4l2_input *i)
+{
+   if (i->index > 0)
+   return -EINVAL;
+
+   memset(i, 0, sizeof(*i));
+   i->type = V4L2_INPUT_TYPE_DEFAULT;
+   strlcpy(i->name, "Default", sizeof(i->name));
+
+   return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_enum_input_default);
+
+int v4l2_ioctl_g_input_default(struct file *file, void *priv,
unsigned int *i)
+{
+   *i = 0;
+   return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_g_input_default);
+
+int v4l2_ioctl_s_input_default(struct file *file, void *priv,
unsigned int i)
+{
+   return i ? -EINVAL : 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_s_input_default);
+
/* Common ioctl debug function. This function can be used by
   external ioctl messages as well as internal V4L ioctl */

void v4l_printk_ioctl(const char *prefix, unsigned int cmd)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index 6cd94e5..accc470 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -652,6 +652,32 @@ struct video_device;
 */

struct mutex *v4l2_ioctl_get_lock(struct video_device *vdev, unsigned
int cmd);
+
+/**
+ * v4l2_ioctl_enum_input_default - v4l2 ioctl helper for
VIDIOC_ENUM_INPUT ioctl
+ *
+ * Plug this function in vidioc_enum_input field of the struct
v4l2_ioctl_ops to
+ * enumerate a single input as V4L2_INPUT_TYPE_DEFAULT
+ */
+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+ struct v4l2_input *i);
+
+/**
+ * v4l2_ioctl_g_input_default - v4l2 ioctl helper for VIDIOC_G_INPUT
ioctl
+ *
+ * Plug this function in vidioc_g_input field of the struct
v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_g_input_default(struct file *file, void *priv,
unsigned int *i);
+
+/**
+ * v4l2_ioctl_s_input_default - v4l2 ioctl helper for VIDIOC_S_INPUT
ioctl
+ *
+ * Plug this function in vidioc_s_input field of the struct
v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_s_input_default(struct file *file, void *priv,
unsigned int i);
+
/* names for fancy debug output */
extern const char *v4l2_field_names[];
extern const char *v4l2_type_names[];
diff --git a/include/uapi/linux/videodev2.h
b/include/uapi/linux/videodev2.h index 316be62..c10bbde 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1477,6 +1477,7 @@ struct v4l2_input {
};

/*  Values for the 'type' field */
+#define V4L2_INPUT_TYPE_DEFAULT0


I don't think we should add a new type here.


I second that. Just replied the same thing on a comment from Sakari to
patch 2/2.


The whole point of this exercise is to
allow existing apps to work, and existing apps expect a TYPE_CAMERA.

BTW, don't read to much in the term 'CAMERA': it's really a catch all
for any video stream, whether it is from a sensor, composite input,
HDMI, etc.

The description for V4L2_INPUT_TYPE_CAMERA in the spec is hopelessly
out of date :-(


Yeah, we always used "CAMERA" to mean NOT_TUNER.


Rather than creating a new type I would add a new V4L2_IN_CAP_MC
capability that indicates that this input is controlled via the media
controller. That makes much more sense and it wouldn't potentially
break applications.

Exactly the same can be done for outputs as well: add V4L2_OUT_CAP_MC
and use V4L2_OUTPUT_TYPE_ANALOG as the output type (again, a horrible
outdated name and the spec is again out of date).


I don't see any sense on distinguishing IN and OUT for MC. I mean:
should
we ever allow that any driver to have their inputs controlled via V4L2
API,
and their outputs controlled via MC (or vice-versa)? I don't think so.

Either all device inputs/outputs are controlled via V4L2 or via MC. So,
let's ca

Re: [RFC PATCH v3 08/11] [media] vimc: Optimize frame generation through the pipe

[Helen Koike]

On 2017-06-02 11:58 PM, Helen Koike wrote:

Add a parameter called vsen_tpg, if true then vimc will work as before:
frames will be generated in the sensor nodes then propagated through the
pipe and processed by each node until a capture node is reached.
If vsen_tpg is false, then the frame is not generated by the sensor, but
directly from the capture node, thus saving intermediate memory buffers
and process time, allowing a higher frame rate.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: Optimize frame generation through the pipe
- This is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/vimc-capture.c | 178 +
 drivers/media/platform/vimc/vimc-common.h  |   2 +
 drivers/media/platform/vimc/vimc-sensor.c  |  30 -
 3 files changed, 156 insertions(+), 54 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index e943267..b5da0ea 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -15,7 +15,10 @@
  *
  */

+#include 
+#include 
 #include 
+#include 
 #include 
 #include 

@@ -38,6 +41,8 @@ struct vimc_cap_device {
struct mutex lock;
u32 sequence;
struct media_pipeline pipe;
+   struct tpg_data tpg;
+   struct task_struct *kthread_cap;
 };

 static const struct v4l2_pix_format fmt_default = {
@@ -246,6 +251,91 @@ static void vimc_cap_return_all_buffers(struct 
vimc_cap_device *vcap,
spin_unlock(>qlock);
 }

+static void vimc_cap_process_frame(struct vimc_ent_device *ved,
+  struct media_pad *sink, const void *frame)
+{
+   struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+   ved);
+   struct vimc_cap_buffer *vimc_buf;
+   void *vbuf;
+
+   spin_lock(>qlock);
+
+   /* Get the first entry of the list */
+   vimc_buf = list_first_entry_or_null(>buf_list,
+   typeof(*vimc_buf), list);
+   if (!vimc_buf) {
+   spin_unlock(>qlock);
+   return;
+   }
+
+   /* Remove this entry from the list */
+   list_del(_buf->list);
+
+   spin_unlock(>qlock);
+
+   /* Fill the buffer */
+   vimc_buf->vb2.vb2_buf.timestamp = ktime_get_ns();
+   vimc_buf->vb2.sequence = vcap->sequence++;
+   vimc_buf->vb2.field = vcap->format.field;
+
+   vbuf = vb2_plane_vaddr(_buf->vb2.vb2_buf, 0);
+
+   if (sink)
+   memcpy(vbuf, frame, vcap->format.sizeimage);
+   else
+   tpg_fill_plane_buffer(>tpg, V4L2_STD_PAL, 0, vbuf);
+
+   /* Set it as ready */
+   vb2_set_plane_payload(_buf->vb2.vb2_buf, 0,
+ vcap->format.sizeimage);
+   vb2_buffer_done(_buf->vb2.vb2_buf, VB2_BUF_STATE_DONE);
+}
+
+
+static int vimc_cap_tpg_thread(void *data)
+{
+   struct vimc_cap_device *vcap = data;
+
+   set_freezable();
+   set_current_state(TASK_UNINTERRUPTIBLE);
+
+   for (;;) {
+   try_to_freeze();
+   if (kthread_should_stop())
+   break;
+
+   vimc_cap_process_frame(>ved, NULL, NULL);
+
+   /* 60 frames per second */
+   schedule_timeout(HZ/60);
+   }
+
+   return 0;
+}
+
+static void vimc_cap_tpg_s_format(struct vimc_cap_device *vcap)
+{
+   const struct vimc_pix_map *vpix =
+   vimc_pix_map_by_pixelformat(vcap->format.pixelformat);
+
+   tpg_reset_source(>tpg, vcap->format.width, vcap->format.height,
+vcap->format.field);
+   tpg_s_bytesperline(>tpg, 0, vcap->format.width * vpix->bpp);
+   tpg_s_buf_height(>tpg, vcap->format.height);
+   tpg_s_fourcc(>tpg, vpix->pixelformat);
+   /*
+* TODO: check why the tpg_s_field need this third argument if
+* it is already receiving the field
+*/
+   tpg_s_field(>tpg, vcap->format.field,
+   vcap->format.field == V4L2_FIELD_ALTERNATE);
+   tpg_s_colorspace(>tpg, vcap->format.colorspace);
+   tpg_s_ycbcr_enc(>tpg, vcap->format.ycbcr_enc);
+   tpg_s_quantization(>tpg, vcap->format.quantization);
+   tpg_s_xfer_func(>tpg, vcap->format.xfer_func);
+}
+
 static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
@@ -256,20 +346,43 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, 
unsigned int count)

/* Start the media pipeline */
ret = media_pipeline_start(entity, >pipe);
-   if (ret) {
-   vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
-   retu

Re: [RFC PATCH v3 00/11] [media]: vimc: Virtual Media Control VPU's

[Helen Koike]

I forgot to mention that this patch series is also available here 
https://github.com/helen-fornazier/opw-staging/tree/z/sent/vimc/vpu/v3


On 2017-06-02 11:58 PM, Helen Koike wrote:

This patch series improves the current video processing units in vimc
(by adding more controls to the sensor and capture node, allowing the
user to configure different frame formats) and also adds a debayer
and a scaler node.
The debayer transforms the bayer format image received in its sink pad
to a bayer format by averaging the pixels within a mean window.
The scaler only scales up the image for now.

In this version I added an optimization where the image can be generated
direct in the capture device instead of being generated in the sensor
and processed by each node in the topology.
I also changed the approach to implement each node of the topology as a
submodule to make the code component oriented, where new components
won't need to touch vimc-core and won't need a header file.
Please, let me know your view regarding this new approach.

Changes in v3:
[media] vimc: sen: Integrate the tpg on the sensor
- Declare frame_size as a local variable
- Set tpg frame format before starting kthread
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: propagate error from kthread_stop
- coding style when calling tpg_s_bytesperline
- s/vimc_thread_sen/vimc_sen_tpg_thread
- fix multiline comment
[media] vimc: Move common code from the core
- This is a new patch in the series
[media] vimc: Add vimc_ent_sd_* helper functions
- add it in vimc-common.c instead in vimc-core.c
- fix vimc_ent_sd_register, use function parameter to assign
sd->entity.function instead of using a fixed value
- rename commit to add the "common" tag
[media] vimc: Add vimc_pipeline_s_stream in the core
- add it in vimc-common instead of vimc-core
- rename commit with "common" tag
[media] vimc: common: Add vimc_link_validate
- this is a new patch in the series
[media] vimc: sen: Support several image formats
- remove support for V4L2_FIELD_ALTERNATE (left as TODO for now)
- clamp image size to an even dimension for height and width
- set default values for colorimetry using _DEFAULT macro
- reset all values of colorimetry to _DEFAULT if user tries to
set an invalid colorspace
[media] vimc: cap: Support several image formats
- use *_DEFAULT macros for colorimetry in the default format
- clamp height and width of the image by an even value
- is user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- remove V4L2_FMT_FLAG_COMPRESSED from vimc_cap_enum_fmt_vid_cap
- remove V4L2_BUF_TYPE_VIDEO_CAPTURE from vimc_cap_enum_fmt_vid_cap
- increase step_width and step_height to 2 instead of 1
- remove link validate function, use the one in vimc-common.c
[media] vimc: Optimize frame generation through the pipe
- This is a new patch in the series
[media] vimc: Subdevices as modules
- This is a new patch in the series
[media] vimc: deb: Add debayer filter
- Declare frame_size as a local variable
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: add ret variable to propagate return errors
- structure code to be a module, use platform_driver and component 
system
- fix multiline comment
- s/thought/through
- s/RGB/RGB888
- clamp height and width of the image by an even value
- if user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- uset _DEFAULT for colorimetry in the default format
[media] vimc: sca: Add scaler
- Declare frame_size as a local variable
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: add ret variable to propagate return errors
- structure code to be a module, use platform_driver and component 
system
- s/thought/through
- clamp height and width of the image by an even value
- if user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- uset _DEFAULT for colorimetry in the default format

Changes in v2:
[media] vimc: sen: Integrate the tpg on the sensor
- Fix include location
- Select V4L2_TPG in Kconfig
- configure tpg on streamon only
- rm BUG_ON
- coding style
- remove V4L2_FIELD_ALTERNATE from tpg_s_field
- remove V4L2_STD_PAL from tpg_fill_plane_buffer
[media] vimc: Add vimc_ent_sd_* helper functions
- Comme

Re: [media] vimc: API proposal, configuring the topology from user space

[Helen Koike]

ping

On 2017-04-10 07:53 PM, Helen Koike wrote:


Hi,

Continuing the discussion about the API of the vimc driver, I made some
changes
based on the previous comments, please see below and let me know your
opinion about it.

Helen

/***
Configfs considerations:
/
Informal definitions:
subsystem: the root driver folder in user space (/configfs/vimc)
item: aka a folder in user space
attributes: aka files in the folder
group: aka a folder that can contain subfolders (parent and child
relation)
default group: aka a subfolder that is created automatically when
the "parent" folder is created
it is not considered a child in terms of rmdir

* Performing rmdir in a group will fail if it contain children that are
not default groups, i.e, if the
folder contain subfolders that are default group, then it can be removed
with rmdir, if the
subfolders were created with mkdir, then rmdir in the parent will fail.

* Configfs has the notion of committable item but it is not implemented
yet. A committable item is an item
that can be in one of two parent folders called: live and pending. The
idea is to create and modify the item
in the pending directory and then to move the item through a rename to
the live directory where
it can't be modified. This seems to be a nice feature for vimc, but as
it is not available yet the
proposal below won't be based on this.

* Groups can be dynamically created/destroyed by the driver whenever it
wants. Afaik attributes can only
be created when the group or item is created and symlinks can only be
create from user space, i.e, the
driver don't know how to create/destroy attributes or symlinks in anytime.

/***
The API:
/

In short, a topology like this one: http://goo.gl/Y7eUfu
Would look like this filesystem tree: https://goo.gl/mEOmOf

v3 core changes:
- I removed the use of symlinks as I wans't able to see how to do it
nicely.
- I use the names of the folders created by user space to retrieve
information at mkdir time
- hotplug file in each entity
- hotplug file in each device
- reset file in each device

* The /configfs/vimc subsystem
empty when the driver is loaded

* Create a device
Userspace can create a new vimc device with:

$ mkdir /configfs/vimc/any_name
Example:
$ mkdir /configfs/vimc/vimc0
$ ls -l /configfs/vimc/vimc0
hotplug
reset
entities/
links/

entities/ and links/ folder are default groups, thus they don't prevent
rmdir vimc0/, but
rmdir will fail if it has any child inside entities/ or links/.
hotplug is used to plug and unplug the device, it can read "plugged" or
"unplugged" and user can
write "plug" or "unplug" to change its state.
Changing hotplug state will never fail as the configfs tree will always
be in a valid state.
reset is used to easily destroy all the topology without the need to
walk through all the children
to perform rmdir, writing 1 to reset file will set hotplug to
"unplugged" and erase all folders
under entities/ and links/.

* Create an entity
Userspace can create a new entity with:

$ mkdir /configfs/vimc/vimc0/entities/:
Example:
$ mkdir /configfs/vimc/vimc0/entities/sensor:SensorA
$ ls -l /configfs/vimc/vimc0/entities/sensor:SensorA
hotplug
pad:source:0/

The name of the folder needs to be in the format : or it
will be rejected, this allows the
creation of the right pads according to its role at mkdir time,
eliminating the previously proposed role
and name files.
hotplug is used to plug and unplug the hw block, it can read "plugged"
or "unplugged" and user can
write "plug" or "unplug" to change its state. As we don't support this
yet in the media core, changing it
will only be allowed if /configfs/vimc/vimc0/hotplug is "unplugged".
hotplug file is "unplugged" by default.
Pads will be created as default groups with the name in the format
pad:: and it
will be an empty folder.
If the hw block supports different number of pads, we could expose two
files:
sinks
sources
where the user space can write the desired number of sink and source
pads and the driver will dynamically
create the folders pad::

* Create a link
User space can create a link between two entities with:

$ mkdir
/configfs/vimc/vimc0/links/:->:

Example:
$ mkdir /configfs/vimc/vimc0/links/DebayerA:1->Scaler:0
$ ls -l /configfs/vimc/vimc0/links/DebayerA:1->Scaler:0
flags

mkdir will be rejected if folder is not on the format
:->:.
mkdir will be rejected if either  or 
are not found in /configfs/vimc/vimc0/entities/
The link will only be created if both entities are in "plugged" state.
When an entity is removed from /configfs/vimc/vimc0/entities/ with
rmdir, its corresponding link folders at
/configfs/vimc/vimc0/links will be automatically removed.
If one of the e

[RFC PATCH v3 04/11] [media] vimc: common: Add vimc_pipeline_s_stream helper

[Helen Koike]

Move the vimc_cap_pipeline_s_stream from the vimc-cap.c to vimc-common.c
as this core will be reused by other subdevices to activate the stream
in their directly connected nodes

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: Add vimc_pipeline_s_stream in the core
- add it in vimc-common instead of vimc-core
- rename commit with "common" tag

Changes in v2:
[media] vimc: Add vimc_pipeline_s_stream in the core
- Use is_media_entity_v4l2_subdev instead of comparing with the old
entity->type
- Fix comments style
- add kernel-docs
- call s_stream across all sink pads


---
 drivers/media/platform/vimc/vimc-capture.c | 29 ++-
 drivers/media/platform/vimc/vimc-common.c  | 32 ++
 drivers/media/platform/vimc/vimc-common.h  | 11 ++
 3 files changed, 45 insertions(+), 27 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 9adb06d..93f6a09 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -132,31 +132,6 @@ static void vimc_cap_return_all_buffers(struct 
vimc_cap_device *vcap,
spin_unlock(>qlock);
 }
 
-static int vimc_cap_pipeline_s_stream(struct vimc_cap_device *vcap, int enable)
-{
-   struct v4l2_subdev *sd;
-   struct media_pad *pad;
-   int ret;
-
-   /* Start the stream in the subdevice direct connected */
-   pad = media_entity_remote_pad(>vdev.entity.pads[0]);
-
-   /*
-* if it is a raw node from vimc-core, there is nothing to activate
-* TODO: remove this when there are no more raw nodes in the
-* core and return error instead
-*/
-   if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
-   return 0;
-
-   sd = media_entity_to_v4l2_subdev(pad->entity);
-   ret = v4l2_subdev_call(sd, video, s_stream, enable);
-   if (ret && ret != -ENOIOCTLCMD)
-   return ret;
-
-   return 0;
-}
-
 static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
@@ -173,7 +148,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, 
unsigned int count)
}
 
/* Enable streaming from the pipe */
-   ret = vimc_cap_pipeline_s_stream(vcap, 1);
+   ret = vimc_pipeline_s_stream(>vdev.entity, 1);
if (ret) {
media_pipeline_stop(entity);
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
@@ -192,7 +167,7 @@ static void vimc_cap_stop_streaming(struct vb2_queue *vq)
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
 
/* Disable streaming from the pipe */
-   vimc_cap_pipeline_s_stream(vcap, 0);
+   vimc_pipeline_s_stream(>vdev.entity, 0);
 
/* Stop the media pipeline */
media_pipeline_stop(>vdev.entity);
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 3afbabd..f809a9d 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -220,6 +220,38 @@ struct media_pad *vimc_pads_init(u16 num_pads, const 
unsigned long *pads_flag)
return pads;
 }
 
+int vimc_pipeline_s_stream(struct media_entity *ent, int enable)
+{
+   struct v4l2_subdev *sd;
+   struct media_pad *pad;
+   unsigned int i;
+   int ret;
+
+   for (i = 0; i < ent->num_pads; i++) {
+   if (ent->pads[i].flags & MEDIA_PAD_FL_SOURCE)
+   continue;
+
+   /* Start the stream in the subdevice direct connected */
+   pad = media_entity_remote_pad(>pads[i]);
+
+   /*
+* if this is a raw node from vimc-core, then there is
+* nothing to activate
+* TODO: remove this when there are no more raw nodes in the
+* core and return error instead
+*/
+   if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
+   continue;
+
+   sd = media_entity_to_v4l2_subdev(pad->entity);
+   ret = v4l2_subdev_call(sd, video, s_stream, enable);
+   if (ret && ret != -ENOIOCTLCMD)
+   return ret;
+   }
+
+   return 0;
+}
+
 static const struct media_entity_operations vimc_ent_sd_mops = {
.link_validate = v4l2_subdev_link_validate,
 };
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 9ec361c..73e7e94 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -97,6 +97,17 @@ static inline void vimc_pads_cleanup(struct media_pad *pads)
 }
 
 /**
+ * vimc_pipeline_s_stream -

[RFC PATCH v3 03/11] [media] vimc: common: Add vimc_ent_sd_* helper

[Helen Koike]

As all the subdevices in the topology will be initialized in the same
way, to avoid code repetition the vimc_ent_sd_{register, unregister}
helper functions were created

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: Add vimc_ent_sd_* helper functions
- add it in vimc-common.c instead in vimc-core.c
- fix vimc_ent_sd_register, use function parameter to assign
sd->entity.function instead of using a fixed value
- rename commit to add the "common" tag

Changes in v2:
[media] vimc: Add vimc_ent_sd_* helper functions
- Comments in vimc_ent_sd_init
- Update vimc_ent_sd_init with upstream code as media_entity_pads_init
(instead of media_entity_init), entity->function intead of entity->type
- Add missing vimc_pads_cleanup in vimc_ent_sd_cleanup
- remove subdevice v4l2_dev and dev fields
- change unregister order in vimc_ent_sd_cleanup
- rename vimc_ent_sd_{init,cleanup} to vimc_ent_sd_{register,unregister}
- remove struct vimc_ent_subdevice, use ved and sd directly
- don't impose struct vimc_sen_device to declare ved and sd struct first
- add kernel docs


---
 drivers/media/platform/vimc/vimc-common.c | 66 +++
 drivers/media/platform/vimc/vimc-common.h | 39 ++
 drivers/media/platform/vimc/vimc-sensor.c | 58 +--
 3 files changed, 114 insertions(+), 49 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 42f779a..3afbabd 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -219,3 +219,69 @@ struct media_pad *vimc_pads_init(u16 num_pads, const 
unsigned long *pads_flag)
 
return pads;
 }
+
+static const struct media_entity_operations vimc_ent_sd_mops = {
+   .link_validate = v4l2_subdev_link_validate,
+};
+
+int vimc_ent_sd_register(struct vimc_ent_device *ved,
+struct v4l2_subdev *sd,
+struct v4l2_device *v4l2_dev,
+const char *const name,
+u32 function,
+u16 num_pads,
+const unsigned long *pads_flag,
+const struct v4l2_subdev_ops *sd_ops,
+void (*sd_destroy)(struct vimc_ent_device *))
+{
+   int ret;
+
+   /* Allocate the pads */
+   ved->pads = vimc_pads_init(num_pads, pads_flag);
+   if (IS_ERR(ved->pads))
+   return PTR_ERR(ved->pads);
+
+   /* Fill the vimc_ent_device struct */
+   ved->destroy = sd_destroy;
+   ved->ent = >entity;
+
+   /* Initialize the subdev */
+   v4l2_subdev_init(sd, sd_ops);
+   sd->entity.function = function;
+   sd->entity.ops = _ent_sd_mops;
+   sd->owner = THIS_MODULE;
+   strlcpy(sd->name, name, sizeof(sd->name));
+   v4l2_set_subdevdata(sd, ved);
+
+   /* Expose this subdev to user space */
+   sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+
+   /* Initialize the media entity */
+   ret = media_entity_pads_init(>entity, num_pads, ved->pads);
+   if (ret)
+   goto err_clean_pads;
+
+   /* Register the subdev with the v4l2 and the media framework */
+   ret = v4l2_device_register_subdev(v4l2_dev, sd);
+   if (ret) {
+   dev_err(v4l2_dev->dev,
+   "%s: subdev register failed (err=%d)\n",
+   name, ret);
+   goto err_clean_m_ent;
+   }
+
+   return 0;
+
+err_clean_m_ent:
+   media_entity_cleanup(>entity);
+err_clean_pads:
+   vimc_pads_cleanup(ved->pads);
+   return ret;
+}
+
+void vimc_ent_sd_unregister(struct vimc_ent_device *ved, struct v4l2_subdev 
*sd)
+{
+   v4l2_device_unregister_subdev(sd);
+   media_entity_cleanup(ved->ent);
+   vimc_pads_cleanup(ved->pads);
+}
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 00d3da4..9ec361c 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -110,4 +110,43 @@ const struct vimc_pix_map *vimc_pix_map_by_code(u32 code);
  */
 const struct vimc_pix_map *vimc_pix_map_by_pixelformat(u32 pixelformat);
 
+/**
+ * vimc_ent_sd_register - initialize and register a subdev node
+ *
+ * @ved:   the vimc_ent_device struct to be initialize
+ * @sd:the v4l2_subdev struct to be initialize and registered
+ * @v4l2_dev:  the v4l2 device to register the v4l2_subdev
+ * @name:  name of the sub-device. Please notice that the name must be
+ * unique.
+ * @function:  media entity function defined by MEDIA_ENT_F_* macros
+ * @num_pads:  number of pads to initialize
+ * @pads_flag: flags to u

[RFC PATCH v3 01/11] [media] vimc: sen: Integrate the tpg on the sensor

[Helen Koike]

Initialize the test pattern generator on the sensor
Generate a colored bar image instead of a grey one

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: sen: Integrate the tpg on the sensor
- Declare frame_size as a local variable
- Set tpg frame format before starting kthread
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: propagate error from kthread_stop
- coding style when calling tpg_s_bytesperline
- s/vimc_thread_sen/vimc_sen_tpg_thread
- fix multiline comment

Changes in v2:
[media] vimc: sen: Integrate the tpg on the sensor
- Fix include location
- Select V4L2_TPG in Kconfig
- configure tpg on streamon only
- rm BUG_ON
- coding style
- remove V4L2_FIELD_ALTERNATE from tpg_s_field
- remove V4L2_STD_PAL from tpg_fill_plane_buffer


---
 drivers/media/platform/vimc/Kconfig   |  1 +
 drivers/media/platform/vimc/vimc-sensor.c | 64 ---
 2 files changed, 52 insertions(+), 13 deletions(-)

diff --git a/drivers/media/platform/vimc/Kconfig 
b/drivers/media/platform/vimc/Kconfig
index a18f635..71c9fe7 100644
--- a/drivers/media/platform/vimc/Kconfig
+++ b/drivers/media/platform/vimc/Kconfig
@@ -2,6 +2,7 @@ config VIDEO_VIMC
tristate "Virtual Media Controller Driver (VIMC)"
depends on VIDEO_DEV && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
select VIDEOBUF2_VMALLOC
+   select VIDEO_V4L2_TPG
default n
---help---
  Skeleton driver for Virtual Media Controller
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 591f6a4..2e83487 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -20,17 +20,20 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "vimc-sensor.h"
 
+#define VIMC_SEN_FRAME_MAX_WIDTH 4096
+
 struct vimc_sen_device {
struct vimc_ent_device ved;
struct v4l2_subdev sd;
+   struct tpg_data tpg;
struct task_struct *kthread_sen;
u8 *frame;
/* The active format */
struct v4l2_mbus_framefmt mbus_format;
-   int frame_size;
 };
 
 static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
@@ -84,6 +87,24 @@ static int vimc_sen_get_fmt(struct v4l2_subdev *sd,
return 0;
 }
 
+static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
+{
+   const struct vimc_pix_map *vpix =
+   vimc_pix_map_by_code(vsen->mbus_format.code);
+
+   tpg_reset_source(>tpg, vsen->mbus_format.width,
+vsen->mbus_format.height, vsen->mbus_format.field);
+   tpg_s_bytesperline(>tpg, 0, vsen->mbus_format.width * vpix->bpp);
+   tpg_s_buf_height(>tpg, vsen->mbus_format.height);
+   tpg_s_fourcc(>tpg, vpix->pixelformat);
+   /* TODO: add support for V4L2_FIELD_ALTERNATE */
+   tpg_s_field(>tpg, vsen->mbus_format.field, false);
+   tpg_s_colorspace(>tpg, vsen->mbus_format.colorspace);
+   tpg_s_ycbcr_enc(>tpg, vsen->mbus_format.ycbcr_enc);
+   tpg_s_quantization(>tpg, vsen->mbus_format.quantization);
+   tpg_s_xfer_func(>tpg, vsen->mbus_format.xfer_func);
+}
+
 static const struct v4l2_subdev_pad_ops vimc_sen_pad_ops = {
.enum_mbus_code = vimc_sen_enum_mbus_code,
.enum_frame_size= vimc_sen_enum_frame_size,
@@ -97,7 +118,7 @@ static const struct media_entity_operations vimc_sen_mops = {
.link_validate = v4l2_subdev_link_validate,
 };
 
-static int vimc_thread_sen(void *data)
+static int vimc_sen_tpg_thread(void *data)
 {
struct vimc_sen_device *vsen = data;
unsigned int i;
@@ -110,7 +131,7 @@ static int vimc_thread_sen(void *data)
if (kthread_should_stop())
break;
 
-   memset(vsen->frame, 100, vsen->frame_size);
+   tpg_fill_plane_buffer(>tpg, 0, 0, vsen->frame);
 
/* Send the frame to all source pads */
for (i = 0; i < vsen->sd.entity.num_pads; i++)
@@ -132,26 +153,31 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int 
enable)
 
if (enable) {
const struct vimc_pix_map *vpix;
+   unsigned int frame_size;
 
if (vsen->kthread_sen)
-   return -EINVAL;
+   /* tpg is already executing */
+   return 0;
 
/* Calculate the frame size */
vpix = vimc_pix_map_by_code(vsen->mbus_format.code);
-   vsen->frame_size = vsen->mbus_format.width * vpix->bpp *
-  vsen-

[RFC PATCH v3 02/11] [media] vimc: Move common code from the core

[Helen Koike]

Remove helper functions from vimc-core and add it in vimc-common to
clean up the core.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: Move common code from the core
- This is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/Makefile   |   2 +-
 drivers/media/platform/vimc/vimc-capture.h |   2 +-
 drivers/media/platform/vimc/vimc-common.c  | 221 +
 .../platform/vimc/{vimc-core.h => vimc-common.h}   |   7 +-
 drivers/media/platform/vimc/vimc-core.c| 205 +--
 drivers/media/platform/vimc/vimc-sensor.h  |   2 +-
 6 files changed, 229 insertions(+), 210 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc-common.c
 rename drivers/media/platform/vimc/{vimc-core.h => vimc-common.h} (96%)

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index c45195e..6b6ddf4 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,3 +1,3 @@
-vimc-objs := vimc-core.o vimc-capture.o vimc-sensor.o
+vimc-objs := vimc-core.o vimc-capture.o vimc-common.o vimc-sensor.o
 
 obj-$(CONFIG_VIDEO_VIMC) += vimc.o
diff --git a/drivers/media/platform/vimc/vimc-capture.h 
b/drivers/media/platform/vimc/vimc-capture.h
index 581a813..7e5c707 100644
--- a/drivers/media/platform/vimc/vimc-capture.h
+++ b/drivers/media/platform/vimc/vimc-capture.h
@@ -18,7 +18,7 @@
 #ifndef _VIMC_CAPTURE_H_
 #define _VIMC_CAPTURE_H_
 
-#include "vimc-core.h"
+#include "vimc-common.h"
 
 struct vimc_ent_device *vimc_cap_create(struct v4l2_device *v4l2_dev,
const char *const name,
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
new file mode 100644
index 000..42f779a
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -0,0 +1,221 @@
+/*
+ * vimc-common.c Virtual Media Controller Driver
+ *
+ * Copyright (C) 2015-2017 Helen Koike <helen.fornaz...@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include "vimc-common.h"
+
+static const struct vimc_pix_map vimc_pix_map_list[] = {
+   /* TODO: add all missing formats */
+
+   /* RGB formats */
+   {
+   .code = MEDIA_BUS_FMT_BGR888_1X24,
+   .pixelformat = V4L2_PIX_FMT_BGR24,
+   .bpp = 3,
+   },
+   {
+   .code = MEDIA_BUS_FMT_RGB888_1X24,
+   .pixelformat = V4L2_PIX_FMT_RGB24,
+   .bpp = 3,
+   },
+   {
+   .code = MEDIA_BUS_FMT_ARGB_1X32,
+   .pixelformat = V4L2_PIX_FMT_ARGB32,
+   .bpp = 4,
+   },
+
+   /* Bayer formats */
+   {
+   .code = MEDIA_BUS_FMT_SBGGR8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SBGGR8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGBRG8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SGBRG8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGRBG8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SGRBG8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SRGGB8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SRGGB8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SBGGR10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SBGGR10,
+   .bpp = 2,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGBRG10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SGBRG10,
+   .bpp = 2,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGRBG10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SGRBG10,
+   .bpp = 2,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SRGGB10_1X10,
+   .pixelformat = V4L2_PIX_FMT_SRGGB10,
+   .bpp = 2,
+   },
+
+   /* 10bit raw bayer a-law compressed to 8 bits */
+   {
+   .code = MEDIA_BUS_FMT_SBGGR10_ALAW8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SBGGR10ALAW8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGBRG10_ALAW8_1X8,
+   .pixelformat = V4L2_PIX_FMT_SGBRG10ALAW8,
+   .bpp = 1,
+   },
+   {
+   .code = MEDIA_BUS_FMT_SGRBG10_ALAW8_1X8,
+   .pixelformat = V4L

[RFC PATCH v3 06/11] [media] vimc: sen: Support several image formats

[Helen Koike]

Allow user space to change the image format as the frame size, the
media bus pixel format, colorspace, quantization, field YCbCr encoding
and the transfer function

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: sen: Support several image formats
- remove support for V4L2_FIELD_ALTERNATE (left as TODO for now)
- clamp image size to an even dimension for height and width
- set default values for colorimetry using _DEFAULT macro
- reset all values of colorimetry to _DEFAULT if user tries to
set an invalid colorspace

Changes in v2:
[media] vimc: sen: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- add init_cfg to initialize try_fmt
- reorder code in vimc_sen_set_fmt
- allow user space to change all fields from struct v4l2_mbus_framefmt
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- merge with patch for the enum_mbus_code and enum_frame_size
- change commit message
- add vimc_pix_map_by_index
- rename MIN/MAX macros
- check set_fmt default parameters for quantization, colorspace 
...media] vimc: sen: Support several image formats


---
 drivers/media/platform/vimc/vimc-common.c |   8 ++
 drivers/media/platform/vimc/vimc-common.h |  12 +++
 drivers/media/platform/vimc/vimc-sensor.c | 145 --
 3 files changed, 136 insertions(+), 29 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index 83d4251..ff59e09 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -144,6 +144,14 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
},
 };
 
+const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i)
+{
+   if (i >= ARRAY_SIZE(vimc_pix_map_list))
+   return NULL;
+
+   return _pix_map_list[i];
+}
+
 const struct vimc_pix_map *vimc_pix_map_by_code(u32 code)
 {
unsigned int i;
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 60ebde2..2189fd6 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -22,6 +22,11 @@
 #include 
 #include 
 
+#define VIMC_FRAME_MAX_WIDTH 4096
+#define VIMC_FRAME_MAX_HEIGHT 2160
+#define VIMC_FRAME_MIN_WIDTH 16
+#define VIMC_FRAME_MIN_HEIGHT 16
+
 /**
  * struct vimc_pix_map - maps media bus code with v4l2 pixel format
  *
@@ -113,6 +118,13 @@ static inline void vimc_pads_cleanup(struct media_pad 
*pads)
 int vimc_pipeline_s_stream(struct media_entity *ent, int enable);
 
 /**
+ * vimc_pix_map_by_index - get vimc_pix_map struct by its index
+ *
+ * @i: index of the vimc_pix_map struct in vimc_pix_map_list
+ */
+const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i);
+
+/**
  * vimc_pix_map_by_code - get vimc_pix_map struct by media bus code
  *
  * @code:  media bus format code defined by MEDIA_BUS_FMT_* macros
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 6386ac1..90c41c6 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -24,8 +24,6 @@
 
 #include "vimc-sensor.h"
 
-#define VIMC_SEN_FRAME_MAX_WIDTH 4096
-
 struct vimc_sen_device {
struct vimc_ent_device ved;
struct v4l2_subdev sd;
@@ -36,18 +34,39 @@ struct vimc_sen_device {
struct v4l2_mbus_framefmt mbus_format;
 };
 
+static const struct v4l2_mbus_framefmt fmt_default = {
+   .width = 640,
+   .height = 480,
+   .code = MEDIA_BUS_FMT_RGB888_1X24,
+   .field = V4L2_FIELD_NONE,
+   .colorspace = V4L2_COLORSPACE_SRGB,
+};
+
+static int vimc_sen_init_cfg(struct v4l2_subdev *sd,
+struct v4l2_subdev_pad_config *cfg)
+{
+   unsigned int i;
+
+   for (i = 0; i < sd->entity.num_pads; i++) {
+   struct v4l2_mbus_framefmt *mf;
+
+   mf = v4l2_subdev_get_try_format(sd, cfg, i);
+   *mf = fmt_default;
+   }
+
+   return 0;
+}
+
 static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
   struct v4l2_subdev_pad_config *cfg,
   struct v4l2_subdev_mbus_code_enum *code)
 {
-   struct vimc_sen_device *vsen =
-   container_of(sd, struct vimc_sen_device, sd);
+   const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
 
-   /* TODO: Add support for other codes */
-   if (code->index)
+   if (!vpix)
return -EINVAL;
 
-   code->code = vsen->mbus_format.code;
+   code->code = vpix->code;
 
return 0;
 }
@@ -56,33 +75,34 @@ static int vimc

[RFC PATCH v3 07/11] [media] vimc: cap: Support several image formats

[Helen Koike]

Allow user space to change the image format as the frame size, the
pixel format, colorspace, quantization, field YCbCr encoding
and the transfer function

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: cap: Support several image formats
- use *_DEFAULT macros for colorimetry in the default format
- clamp height and width of the image by an even value
- is user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- remove V4L2_FMT_FLAG_COMPRESSED from vimc_cap_enum_fmt_vid_cap
- remove V4L2_BUF_TYPE_VIDEO_CAPTURE from vimc_cap_enum_fmt_vid_cap
- increase step_width and step_height to 2 instead of 1
- remove link validate function, use the one in vimc-common.c

Changes in v2:
[media] vimc: cap: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- allow user space to change all fields from struct v4l2_pix_format
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- link_validate and try_fmt: also checks colospace, quantization, 
field, xfer_func, ycbcr_enc
- add struct v4l2_pix_format fmt_default
- add enum_framesizes
- enum_fmt_vid_cap: enumerate all formats from vimc_pix_map_table
- add mode dev_dbg


---
 drivers/media/platform/vimc/vimc-capture.c | 131 +
 1 file changed, 115 insertions(+), 16 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 5bdecd1..e943267 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -40,6 +40,14 @@ struct vimc_cap_device {
struct media_pipeline pipe;
 };
 
+static const struct v4l2_pix_format fmt_default = {
+   .width = 640,
+   .height = 480,
+   .pixelformat = V4L2_PIX_FMT_RGB24,
+   .field = V4L2_FIELD_NONE,
+   .colorspace = V4L2_COLORSPACE_SRGB,
+};
+
 struct vimc_cap_buffer {
/*
 * struct vb2_v4l2_buffer must be the first element
@@ -73,7 +81,7 @@ static void vimc_cap_get_format(struct vimc_ent_device *ved,
*fmt = vcap->format;
 }
 
-static int vimc_cap_fmt_vid_cap(struct file *file, void *priv,
+static int vimc_cap_g_fmt_vid_cap(struct file *file, void *priv,
  struct v4l2_format *f)
 {
struct vimc_cap_device *vcap = video_drvdata(file);
@@ -83,16 +91,112 @@ static int vimc_cap_fmt_vid_cap(struct file *file, void 
*priv,
return 0;
 }
 
+static int vimc_cap_try_fmt_vid_cap(struct file *file, void *priv,
+   struct v4l2_format *f)
+{
+   struct v4l2_pix_format *format = >fmt.pix;
+   const struct vimc_pix_map *vpix;
+
+   format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
+   VIMC_FRAME_MAX_WIDTH) & ~1;
+   format->height = clamp_t(u32, format->height, VIMC_FRAME_MIN_HEIGHT,
+VIMC_FRAME_MAX_HEIGHT) & ~1;
+
+   /* Don't accept a pixelformat that is not on the table */
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   if (!vpix) {
+   format->pixelformat = fmt_default.pixelformat;
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   }
+   /* TODO: Add support for custom bytesperline values */
+   format->bytesperline = format->width * vpix->bpp;
+   format->sizeimage = format->bytesperline * format->height;
+
+   if (format->field == V4L2_FIELD_ANY)
+   format->field = fmt_default.field;
+
+   if (format->colorspace == V4L2_COLORSPACE_DEFAULT)
+   format->colorspace = fmt_default.colorspace;
+
+   /* Check if values are out of range */
+   if (format->colorspace > V4L2_COLORSPACE_DCI_P3) {
+   format->colorspace = fmt_default.colorspace;
+   format->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+   format->quantization = V4L2_QUANTIZATION_DEFAULT;
+   format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+   }
+   if (format->ycbcr_enc > V4L2_YCBCR_ENC_SMPTE240M)
+   format->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT;
+   if (format->quantization > V4L2_QUANTIZATION_LIM_RANGE)
+   format->quantization = V4L2_QUANTIZATION_DEFAULT;
+   if (format->xfer_func > V4L2_XFER_FUNC_SMPTE2084)
+   format->xfer_func = V4L2_XFER_FUNC_DEFAULT;
+   return 0;
+}
+
+static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+   struct vimc_cap_device *vcap = video_drvdata(file);
+
+   /* Do not change the format while stream is on */
+   if (vb2_is_busy(>queue))
+ 

[RFC PATCH v3 08/11] [media] vimc: Optimize frame generation through the pipe

[Helen Koike]

Add a parameter called vsen_tpg, if true then vimc will work as before:
frames will be generated in the sensor nodes then propagated through the
pipe and processed by each node until a capture node is reached.
If vsen_tpg is false, then the frame is not generated by the sensor, but
directly from the capture node, thus saving intermediate memory buffers
and process time, allowing a higher frame rate.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: Optimize frame generation through the pipe
- This is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/vimc-capture.c | 178 +
 drivers/media/platform/vimc/vimc-common.h  |   2 +
 drivers/media/platform/vimc/vimc-sensor.c  |  30 -
 3 files changed, 156 insertions(+), 54 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index e943267..b5da0ea 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -15,7 +15,10 @@
  *
  */
 
+#include 
+#include 
 #include 
+#include 
 #include 
 #include 
 
@@ -38,6 +41,8 @@ struct vimc_cap_device {
struct mutex lock;
u32 sequence;
struct media_pipeline pipe;
+   struct tpg_data tpg;
+   struct task_struct *kthread_cap;
 };
 
 static const struct v4l2_pix_format fmt_default = {
@@ -246,6 +251,91 @@ static void vimc_cap_return_all_buffers(struct 
vimc_cap_device *vcap,
spin_unlock(>qlock);
 }
 
+static void vimc_cap_process_frame(struct vimc_ent_device *ved,
+  struct media_pad *sink, const void *frame)
+{
+   struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+   ved);
+   struct vimc_cap_buffer *vimc_buf;
+   void *vbuf;
+
+   spin_lock(>qlock);
+
+   /* Get the first entry of the list */
+   vimc_buf = list_first_entry_or_null(>buf_list,
+   typeof(*vimc_buf), list);
+   if (!vimc_buf) {
+   spin_unlock(>qlock);
+   return;
+   }
+
+   /* Remove this entry from the list */
+   list_del(_buf->list);
+
+   spin_unlock(>qlock);
+
+   /* Fill the buffer */
+   vimc_buf->vb2.vb2_buf.timestamp = ktime_get_ns();
+   vimc_buf->vb2.sequence = vcap->sequence++;
+   vimc_buf->vb2.field = vcap->format.field;
+
+   vbuf = vb2_plane_vaddr(_buf->vb2.vb2_buf, 0);
+
+   if (sink)
+   memcpy(vbuf, frame, vcap->format.sizeimage);
+   else
+   tpg_fill_plane_buffer(>tpg, V4L2_STD_PAL, 0, vbuf);
+
+   /* Set it as ready */
+   vb2_set_plane_payload(_buf->vb2.vb2_buf, 0,
+ vcap->format.sizeimage);
+   vb2_buffer_done(_buf->vb2.vb2_buf, VB2_BUF_STATE_DONE);
+}
+
+
+static int vimc_cap_tpg_thread(void *data)
+{
+   struct vimc_cap_device *vcap = data;
+
+   set_freezable();
+   set_current_state(TASK_UNINTERRUPTIBLE);
+
+   for (;;) {
+   try_to_freeze();
+   if (kthread_should_stop())
+   break;
+
+   vimc_cap_process_frame(>ved, NULL, NULL);
+
+   /* 60 frames per second */
+   schedule_timeout(HZ/60);
+   }
+
+   return 0;
+}
+
+static void vimc_cap_tpg_s_format(struct vimc_cap_device *vcap)
+{
+   const struct vimc_pix_map *vpix =
+   vimc_pix_map_by_pixelformat(vcap->format.pixelformat);
+
+   tpg_reset_source(>tpg, vcap->format.width, vcap->format.height,
+vcap->format.field);
+   tpg_s_bytesperline(>tpg, 0, vcap->format.width * vpix->bpp);
+   tpg_s_buf_height(>tpg, vcap->format.height);
+   tpg_s_fourcc(>tpg, vpix->pixelformat);
+   /*
+* TODO: check why the tpg_s_field need this third argument if
+* it is already receiving the field
+*/
+   tpg_s_field(>tpg, vcap->format.field,
+   vcap->format.field == V4L2_FIELD_ALTERNATE);
+   tpg_s_colorspace(>tpg, vcap->format.colorspace);
+   tpg_s_ycbcr_enc(>tpg, vcap->format.ycbcr_enc);
+   tpg_s_quantization(>tpg, vcap->format.quantization);
+   tpg_s_xfer_func(>tpg, vcap->format.xfer_func);
+}
+
 static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
@@ -256,20 +346,43 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, 
unsigned int count)
 
/* Start the media pipeline */
ret = media_pipeline_start(entity, >pipe);
-   if (ret) {
-   vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
-   return ret;
-   }
+   if (ret)
+  

[RFC PATCH v3 10/11] [media] vimc: deb: Add debayer filter

[Helen Koike]

Implement the debayer filter and integrate it with the core

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: deb: Add debayer filter
- Declare frame_size as a local variable
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: add ret variable to propagate return errors
- structure code to be a module, use platform_driver and component 
system
- fix multiline comment
- s/thought/through
- s/RGB/RGB888
- clamp height and width of the image by an even value
- if user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- uset _DEFAULT for colorimetry in the default format

Changes in v2:
[media] vimc: deb: Add debayer filter
- Using MEDIA_ENT_F_ATV_DECODER in function
- remove v4l2_dev and dev from vimc_deb_device struct
- src fmt propagates from the sink
- coding style
- remove redundant else if statements
- check end of enum and remove BUG_ON
- enum frame size with min and max values
- set/try fmt
- remove unecessary include freezer.h
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add deb_mean_win_size as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg


---
 drivers/media/platform/vimc/Makefile   |   4 +-
 drivers/media/platform/vimc/vimc-common.h  |   2 +
 drivers/media/platform/vimc/vimc-debayer.c | 615 +
 3 files changed, 620 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/vimc/vimc-debayer.c

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index 0e5d5ce..4fba8ef 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,6 +1,8 @@
 vimc-objs := vimc-core.o
 vimc_capture-objs := vimc-capture.o
 vimc_common-objs := vimc-common.o
+vimc_debayer-objs := vimc-debayer.o
 vimc_sensor-objs := vimc-sensor.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc_sensor.o
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc-debayer.o 
\
+   vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-common.h 
b/drivers/media/platform/vimc/vimc-common.h
index 27c9b8c..c63d51f 100644
--- a/drivers/media/platform/vimc/vimc-common.h
+++ b/drivers/media/platform/vimc/vimc-common.h
@@ -29,6 +29,8 @@
 
 #define VIMC_PIPE_OPT 1
 
+#define VIMC_FRAME_INDEX(lin, col, width, bpp) ((lin * width + col) * bpp)
+
 /**
  * struct vimc_pix_map - maps media bus code with v4l2 pixel format
  *
diff --git a/drivers/media/platform/vimc/vimc-debayer.c 
b/drivers/media/platform/vimc/vimc-debayer.c
new file mode 100644
index 000..9f9604c
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -0,0 +1,615 @@
+/*
+ * vimc-debayer.c Virtual Media Controller Driver
+ *
+ * Copyright (C) 2015-2017 Helen Koike <helen.fornaz...@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "vimc-common.h"
+
+#define VIMC_DEB_DRV_NAME "vimc-debayer"
+
+static unsigned int deb_mean_win_size = 3;
+module_param(deb_mean_win_size, uint, );
+MODULE_PARM_DESC(deb_mean_win_size, " the window size to calculate the mean.\n"
+   "NOTE: the window size need to be an odd number, as the main pixel "
+   "stays in the center of the window, otherwise the next odd number "
+   "is considered");
+
+#define IS_SINK(pad) (!pad)
+#define IS_SRC(pad)  (pad)
+
+enum vimc_deb_rgb_colors {
+   VIMC_DEB_RED = 0,
+   VIMC_DEB_GREEN = 1,
+   VIMC_DEB_BLUE = 2,
+};
+
+struct vimc_deb_pix_map {
+   u32 code;
+   enum vimc_deb_rgb_colors order[2][2];
+};
+
+struct vimc_deb_device {
+   struct vimc_ent_device ved;
+   struct v4l2_subdev sd;
+   struct device *dev;
+   /* The active format */
+   struct v4l2_mbus_framefmt sink_fmt;
+   u32 src_code;
+   void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
+   unsigned int

[RFC PATCH v3 09/11] [media] vimc: Subdevices as modules

[Helen Koike]

Change the core structure for adding subdevices in the topology.
Instead of calling the specific create function for each subdevice,
inject a child platform_device with the driver's name.
Each type of node in the topology (sensor, capture, debayer, scaler)
will register a platform_driver with the corresponding name through the
component subsystem.
Implementing a new subdevice type doesn't require vimc-core to be altered.

This facilitates future implementation of dynamic entities, where
hotpluging an entity in the topology is just a matter of
registering/unregistering a platform_device in the system.
It also facilitates other implementations of different nodes without
touching the core code and remove the need of a header file for each
type of node.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: Subdevices as modules
- This is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/Makefile   |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  96 ---
 drivers/media/platform/vimc/vimc-capture.h |  28 --
 drivers/media/platform/vimc/vimc-common.c  |  37 ++-
 drivers/media/platform/vimc/vimc-common.h  |  14 +-
 drivers/media/platform/vimc/vimc-core.c| 406 +++--
 drivers/media/platform/vimc/vimc-sensor.c  |  91 +--
 drivers/media/platform/vimc/vimc-sensor.h  |  28 --
 8 files changed, 320 insertions(+), 387 deletions(-)
 delete mode 100644 drivers/media/platform/vimc/vimc-capture.h
 delete mode 100644 drivers/media/platform/vimc/vimc-sensor.h

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index 6b6ddf4..0e5d5ce 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,3 +1,6 @@
-vimc-objs := vimc-core.o vimc-capture.o vimc-common.o vimc-sensor.o
+vimc-objs := vimc-core.o
+vimc_capture-objs := vimc-capture.o
+vimc_common-objs := vimc-common.o
+vimc_sensor-objs := vimc-sensor.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index b5da0ea..633d99a 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -15,18 +15,24 @@
  *
  */
 
+#include 
 #include 
 #include 
+#include 
+#include 
 #include 
 #include 
 #include 
 #include 
 
-#include "vimc-capture.h"
+#include "vimc-common.h"
+
+#define VIMC_CAP_DRV_NAME "vimc-capture"
 
 struct vimc_cap_device {
struct vimc_ent_device ved;
struct video_device vdev;
+   struct device *dev;
struct v4l2_pix_format format;
struct vb2_queue queue;
struct list_head buf_list;
@@ -150,7 +156,7 @@ static int vimc_cap_s_fmt_vid_cap(struct file *file, void 
*priv,
 
vimc_cap_try_fmt_vid_cap(file, priv, f);
 
-   dev_dbg(vcap->vdev.v4l2_dev->dev, "%s: format update: "
+   dev_dbg(vcap->dev, "%s: format update: "
"old:%dx%d (0x%x, %d, %d, %d, %d) "
"new:%dx%d (0x%x, %d, %d, %d, %d)\n", vcap->vdev.name,
/* old */
@@ -365,8 +371,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, 
unsigned int count)
vcap->kthread_cap = kthread_run(vimc_cap_tpg_thread, vcap,
"%s-cap", vcap->vdev.v4l2_dev->name);
if (IS_ERR(vcap->kthread_cap)) {
-   dev_err(vcap->vdev.v4l2_dev->dev,
-   "%s: kernel_thread() failed\n",
+   dev_err(vcap->dev, "%s: kernel_thread() failed\n",
vcap->vdev.name);
ret = PTR_ERR(vcap->kthread_cap);
goto err_tpg_free;
@@ -443,8 +448,7 @@ static int vimc_cap_buffer_prepare(struct vb2_buffer *vb)
unsigned long size = vcap->format.sizeimage;
 
if (vb2_plane_size(vb, 0) < size) {
-   dev_err(vcap->vdev.v4l2_dev->dev,
-   "%s: buffer too small (%lu < %lu)\n",
+   dev_err(vcap->dev, "%s: buffer too small (%lu < %lu)\n",
vcap->vdev.name, vb2_plane_size(vb, 0), size);
return -EINVAL;
}
@@ -469,8 +473,10 @@ static const struct media_entity_operations vimc_cap_mops 
= {
.link_validate  = vimc_link_validate,
 };
 
-static void vimc_cap_destroy(struct vimc_ent_device *ved)
+static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
+void *master_data)
 {
+   struct vimc_ent_device *ved = dev_get_drvdata(comp);
struct vimc_cap_device *vcap = container_of(ved, struct v

[RFC PATCH v3 11/11] [media] vimc: sca: Add scaler

[Helen Koike]

Implement scaler and integrated with the core

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: sca: Add scaler
- Declare frame_size as a local variable
- s_stream(sd, 1): return 0 if stream is already enabled
- s_stream(sd, 0): return 0 if stream is already disabled
- s_stream: add ret variable to propagate return errors
- structure code to be a module, use platform_driver and component 
system
- s/thought/through
- clamp height and width of the image by an even value
- if user try to set colorspace to an invalid format, set all
colorimetry parameters to _DEFAULT
- uset _DEFAULT for colorimetry in the default format

Changes in v2:
[media] vimc: sca: Add scaler
- Add function MEDIA_ENT_F_IO_V4L
- remove v4l2_dev and dev
- s/sink_mbus_fmt/sink_fmt
- remove BUG_ON, remove redundant if else, rewrite TODO, check end of 
enum
- rm src_width/height, enum fsize with min and max values
- set/try fmt
- remove unecessary include freezer.h
- core: add bayer boolean in pixel table
- coding style
- fix bug in enum frame size
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add sca_mult as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg


---
 drivers/media/platform/vimc/Makefile  |   3 +-
 drivers/media/platform/vimc/vimc-common.c |  27 ++
 drivers/media/platform/vimc/vimc-common.h |   1 +
 drivers/media/platform/vimc/vimc-scaler.c | 469 ++
 4 files changed, 499 insertions(+), 1 deletion(-)
 create mode 100644 drivers/media/platform/vimc/vimc-scaler.c

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index 4fba8ef..68c5d98 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -2,7 +2,8 @@ vimc-objs := vimc-core.o
 vimc_capture-objs := vimc-capture.o
 vimc_common-objs := vimc-common.o
 vimc_debayer-objs := vimc-debayer.o
+vimc_scaler-objs := vimc-scaler.o
 vimc_sensor-objs := vimc-sensor.o
 
 obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc_capture.o vimc_common.o vimc-debayer.o 
\
-   vimc_sensor.o
+   vimc_scaler.o vimc_sensor.o
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index d5ed387..68887d7 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -20,6 +20,10 @@
 
 #include "vimc-common.h"
 
+/*
+ * NOTE: non-bayer formats need to come first (necessary for enum_mbus_code
+ * in the scaler)
+ */
 static const struct vimc_pix_map vimc_pix_map_list[] = {
/* TODO: add all missing formats */
 
@@ -28,16 +32,19 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
.code = MEDIA_BUS_FMT_BGR888_1X24,
.pixelformat = V4L2_PIX_FMT_BGR24,
.bpp = 3,
+   .bayer = false,
},
{
.code = MEDIA_BUS_FMT_RGB888_1X24,
.pixelformat = V4L2_PIX_FMT_RGB24,
.bpp = 3,
+   .bayer = false,
},
{
.code = MEDIA_BUS_FMT_ARGB_1X32,
.pixelformat = V4L2_PIX_FMT_ARGB32,
.bpp = 4,
+   .bayer = false,
},
 
/* Bayer formats */
@@ -45,41 +52,49 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
.code = MEDIA_BUS_FMT_SBGGR8_1X8,
.pixelformat = V4L2_PIX_FMT_SBGGR8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGBRG8_1X8,
.pixelformat = V4L2_PIX_FMT_SGBRG8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGRBG8_1X8,
.pixelformat = V4L2_PIX_FMT_SGRBG8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SRGGB8_1X8,
.pixelformat = V4L2_PIX_FMT_SRGGB8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SBGGR10_1X10,
.pixelformat = V4L2_PIX_FMT_SBGGR10,
.bpp = 2,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGBRG10_1X10,
.pixelformat = V4L2_PIX_FMT_SGBRG10,
.bpp = 2,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGRBG10_1X10,
.pixelformat = V4L2_PIX_FMT_SGRBG10,
.bpp = 2,
+

[RFC PATCH v3 05/11] [media] vimc: common: Add vimc_link_validate

[Helen Koike]

All links will be checked in the same way. Adding a helper function for
that

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v3:
[media] vimc: common: Add vimc_link_validate
- this is a new patch in the series

Changes in v2: None


---
 drivers/media/platform/vimc/vimc-capture.c |  78 +++---
 drivers/media/platform/vimc/vimc-common.c  | 124 -
 drivers/media/platform/vimc/vimc-common.h  |  14 
 3 files changed, 148 insertions(+), 68 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 93f6a09..5bdecd1 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -64,6 +64,15 @@ static int vimc_cap_querycap(struct file *file, void *priv,
return 0;
 }
 
+static void vimc_cap_get_format(struct vimc_ent_device *ved,
+   struct v4l2_pix_format *fmt)
+{
+   struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+   ved);
+
+   *fmt = vcap->format;
+}
+
 static int vimc_cap_fmt_vid_cap(struct file *file, void *priv,
  struct v4l2_format *f)
 {
@@ -231,74 +240,8 @@ static const struct vb2_ops vimc_cap_qops = {
.wait_finish= vb2_ops_wait_finish,
 };
 
-/*
- * NOTE: this function is a copy of v4l2_subdev_link_validate_get_format
- * maybe the v4l2 function should be public
- */
-static int vimc_cap_v4l2_subdev_link_validate_get_format(struct media_pad *pad,
-   struct v4l2_subdev_format *fmt)
-{
-   struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(pad->entity);
-
-   fmt->which = V4L2_SUBDEV_FORMAT_ACTIVE;
-   fmt->pad = pad->index;
-
-   return v4l2_subdev_call(sd, pad, get_fmt, NULL, fmt);
-}
-
-static int vimc_cap_link_validate(struct media_link *link)
-{
-   struct v4l2_subdev_format source_fmt;
-   const struct vimc_pix_map *vpix;
-   struct vimc_cap_device *vcap = container_of(link->sink->entity,
-   struct vimc_cap_device,
-   vdev.entity);
-   struct v4l2_pix_format *sink_fmt = >format;
-   int ret;
-
-   /*
-* if it is a raw node from vimc-core, ignore the link for now
-* TODO: remove this when there are no more raw nodes in the
-* core and return error instead
-*/
-   if (link->source->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
-   return 0;
-
-   /* Get the the format of the subdev */
-   ret = vimc_cap_v4l2_subdev_link_validate_get_format(link->source,
-   _fmt);
-   if (ret)
-   return ret;
-
-   dev_dbg(vcap->vdev.v4l2_dev->dev,
-   "%s: link validate formats src:%dx%d %d sink:%dx%d %d\n",
-   vcap->vdev.name,
-   source_fmt.format.width, source_fmt.format.height,
-   source_fmt.format.code,
-   sink_fmt->width, sink_fmt->height,
-   sink_fmt->pixelformat);
-
-   /* The width, height and code must match. */
-   vpix = vimc_pix_map_by_pixelformat(sink_fmt->pixelformat);
-   if (source_fmt.format.width != sink_fmt->width
-   || source_fmt.format.height != sink_fmt->height
-   || vpix->code != source_fmt.format.code)
-   return -EPIPE;
-
-   /*
-* The field order must match, or the sink field order must be NONE
-* to support interlaced hardware connected to bridges that support
-* progressive formats only.
-*/
-   if (source_fmt.format.field != sink_fmt->field &&
-   sink_fmt->field != V4L2_FIELD_NONE)
-   return -EPIPE;
-
-   return 0;
-}
-
 static const struct media_entity_operations vimc_cap_mops = {
-   .link_validate  = vimc_cap_link_validate,
+   .link_validate  = vimc_link_validate,
 };
 
 static void vimc_cap_destroy(struct vimc_ent_device *ved)
@@ -434,6 +377,7 @@ struct vimc_ent_device *vimc_cap_create(struct v4l2_device 
*v4l2_dev,
vcap->ved.destroy = vimc_cap_destroy;
vcap->ved.ent = >vdev.entity;
vcap->ved.process_frame = vimc_cap_process_frame;
+   vcap->ved.vdev_get_format = vimc_cap_get_format;
 
/* Initialize the video_device struct */
vdev = >vdev;
diff --git a/drivers/media/platform/vimc/vimc-common.c 
b/drivers/media/platform/vimc/vimc-common.c
index f809a9d..83d4251 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -252,8 +252,130 @@ int vimc_pipeline_s_stream(struct media_entity *ent, int 
enable)
return 

[RFC PATCH v3 00/11] [media]: vimc: Virtual Media Control VPU's

[Helen Koike]

gt;type
- Add missing vimc_pads_cleanup in vimc_ent_sd_cleanup
- remove subdevice v4l2_dev and dev fields
- change unregister order in vimc_ent_sd_cleanup
- rename vimc_ent_sd_{init,cleanup} to vimc_ent_sd_{register,unregister}
- remove struct vimc_ent_subdevice, use ved and sd directly
- don't impose struct vimc_sen_device to declare ved and sd struct first
- add kernel docs
[media] vimc: Add vimc_pipeline_s_stream in the core
- Use is_media_entity_v4l2_subdev instead of comparing with the old
entity->type
- Fix comments style
- add kernel-docs
- call s_stream across all sink pads
[media] vimc: sen: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- add init_cfg to initialize try_fmt
- reorder code in vimc_sen_set_fmt
- allow user space to change all fields from struct v4l2_mbus_framefmt
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- merge with patch for the enum_mbus_code and enum_frame_size
- change commit message
- add vimc_pix_map_by_index
- rename MIN/MAX macros
- check set_fmt default parameters for quantization, colorspace 
...media] vimc: sen: Support several image formats
[media] vimc: cap: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- allow user space to change all fields from struct v4l2_pix_format
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- link_validate and try_fmt: also checks colospace, quantization, 
field, xfer_func, ycbcr_enc
- add struct v4l2_pix_format fmt_default
- add enum_framesizes
- enum_fmt_vid_cap: enumerate all formats from vimc_pix_map_table
- add mode dev_dbg
[media] vimc: deb: Add debayer filter
- Using MEDIA_ENT_F_ATV_DECODER in function
- remove v4l2_dev and dev from vimc_deb_device struct
- src fmt propagates from the sink
- coding style
- remove redundant else if statements
- check end of enum and remove BUG_ON
- enum frame size with min and max values
- set/try fmt
- remove unecessary include freezer.h
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add deb_mean_win_size as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg
[media] vimc: sca: Add scaler
- Add function MEDIA_ENT_F_IO_V4L
- remove v4l2_dev and dev
- s/sink_mbus_fmt/sink_fmt
- remove BUG_ON, remove redundant if else, rewrite TODO, check end of 
enum
- rm src_width/height, enum fsize with min and max values
- set/try fmt
- remove unecessary include freezer.h
- core: add bayer boolean in pixel table
- coding style
- fix bug in enum frame size
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add sca_mult as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
    - add more dev_dbg

Helen Koike (11):
  [media] vimc: sen: Integrate the tpg on the sensor
  [media] vimc: Move common code from the core
  [media] vimc: common: Add vimc_ent_sd_* helper
  [media] vimc: common: Add vimc_pipeline_s_stream helper
  [media] vimc: common: Add vimc_link_validate
  [media] vimc: sen: Support several image formats
  [media] vimc: cap: Support several image formats
  [media] vimc: Optimize frame generation through the pipe
  [media] vimc: Subdevices as modules
  [media] vimc: deb: Add debayer filter
  [media] vimc: sca: Add scaler

 drivers/media/platform/vimc/Kconfig|   1 +
 drivers/media/platform/vimc/Makefile   |  10 +-
 drivers/media/platform/vimc/vimc-capture.c | 488 ++--
 drivers/media/platform/vimc/vimc-capture.h |  28 -
 drivers/media/platform/vimc/vimc-common.c  | 475 
 .../platform/vimc/{vimc-core.h => vimc-common.h}   |  88 ++-
 drivers/media/platform/vimc/vimc-core.c| 611 ++--
 drivers/media/platform/vimc/vimc-debayer.c | 615 +
 drivers/media/platform/vimc/vimc-scaler.c  | 469 
 drivers/media/platform/vimc/vimc-sensor.c  | 348 
 drivers/media/platform/vimc/vimc-sensor.h  |  28 -
 11 files changed, 2370 insertions(+), 791 deletions(-)
 delete mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-common.c
 rename drivers/media/platform/vimc/{vimc-core.h =

[RFC PATCH] [media] v4l2-subdev: check colorimetry in link validate

[Helen Koike]

colorspace, ycbcr_enc, quantization and xfer_func must match across the
link.
Check if they match in v4l2_subdev_link_validate_default unless they are
set as _DEFAULT.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Hi,

I think we should validate colorimetry as having different colorimetry
across a link doesn't make sense.
But I am confused about what to do when they are set to _DEFAULT, what
do you think?

Thanks
---
 drivers/media/v4l2-core/v4l2-subdev.c | 21 +++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c 
b/drivers/media/v4l2-core/v4l2-subdev.c
index da78497..784ae92 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -502,10 +502,27 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev 
*sd,
  struct v4l2_subdev_format *source_fmt,
  struct v4l2_subdev_format *sink_fmt)
 {
-   /* The width, height and code must match. */
+   /* The width, height, code and colorspace must match. */
if (source_fmt->format.width != sink_fmt->format.width
|| source_fmt->format.height != sink_fmt->format.height
-   || source_fmt->format.code != sink_fmt->format.code)
+   || source_fmt->format.code != sink_fmt->format.code
+   || source_fmt->format.colorspace != sink_fmt->format.colorspace)
+   return -EPIPE;
+
+   /* Colorimetry must match if they are not set to DEFAULT */
+   if (source_fmt->format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT
+   && sink_fmt->format.ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT
+   && source_fmt->format.ycbcr_enc != sink_fmt->format.ycbcr_enc)
+   return -EPIPE;
+
+   if (source_fmt->format.quantization != V4L2_QUANTIZATION_DEFAULT
+   && sink_fmt->format.quantization != V4L2_QUANTIZATION_DEFAULT
+   && source_fmt->format.quantization != sink_fmt->format.quantization)
+   return -EPIPE;
+
+   if (source_fmt->format.xfer_func != V4L2_XFER_FUNC_DEFAULT
+   && sink_fmt->format.xfer_func != V4L2_XFER_FUNC_DEFAULT
+   && source_fmt->format.xfer_func != sink_fmt->format.xfer_func)
return -EPIPE;
 
/* The field order must match, or the sink field order must be NONE
-- 
2.7.4

Re: [PATCH v2 5/7] [media] vimc: cap: Support several image formats

[Helen Koike]

Hi Hans,

Thank you for your review. I just have a question for this one.

On 2017-05-08 08:53 AM, Hans Verkuil wrote:

On 04/08/2017 12:37 AM, Helen Koike wrote:

Allow user space to change the image format as the frame size, the
pixel format, colorspace, quantization, field YCbCr encoding
and the transfer function

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2:
[media] vimc: cap: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- allow user space to change all fields from struct v4l2_pix_format
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- link_validate and try_fmt: also checks colospace, quantization, 
field, xfer_func, ycbcr_enc
- add struct v4l2_pix_format fmt_default
- add enum_framesizes
- enum_fmt_vid_cap: enumerate all formats from vimc_pix_map_table
- add mode dev_dbg


---
 drivers/media/platform/vimc/vimc-capture.c | 167 -
 1 file changed, 139 insertions(+), 28 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 93f6a09..a6441f7 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -40,6 +40,16 @@ struct vimc_cap_device {
struct media_pipeline pipe;
 };

+static const struct v4l2_pix_format fmt_default = {
+   .width = 640,
+   .height = 480,
+   .pixelformat = V4L2_PIX_FMT_RGB24,
+   .field = V4L2_FIELD_NONE,
+   .colorspace = V4L2_COLORSPACE_SRGB,
+   .quantization = V4L2_QUANTIZATION_FULL_RANGE,
+   .xfer_func = V4L2_XFER_FUNC_SRGB,


I actually think we should keep .quantization and .xfer_func to 0 (DEFAULT).
It's what most drivers will do. Same for the previous patch (I didn't mention
it there).


+};
+
 struct vimc_cap_buffer {
/*
 * struct vb2_v4l2_buffer must be the first element
@@ -64,7 +74,7 @@ static int vimc_cap_querycap(struct file *file, void *priv,
return 0;
 }

-static int vimc_cap_fmt_vid_cap(struct file *file, void *priv,
+static int vimc_cap_g_fmt_vid_cap(struct file *file, void *priv,
  struct v4l2_format *f)
 {
struct vimc_cap_device *vcap = video_drvdata(file);
@@ -74,16 +84,112 @@ static int vimc_cap_fmt_vid_cap(struct file *file, void 
*priv,
return 0;
 }

+static int vimc_cap_try_fmt_vid_cap(struct file *file, void *priv,
+   struct v4l2_format *f)
+{
+   struct v4l2_pix_format *format = >fmt.pix;
+   const struct vimc_pix_map *vpix;
+
+   format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
+   VIMC_FRAME_MAX_WIDTH);
+   format->height = clamp_t(u32, format->height, VIMC_FRAME_MIN_HEIGHT,
+VIMC_FRAME_MAX_HEIGHT);
+
+   /* Don't accept a pixelformat that is not on the table */
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   if (!vpix) {
+   format->pixelformat = fmt_default.pixelformat;
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   }
+   /* TODO: Add support for custom bytesperline values */
+   format->bytesperline = format->width * vpix->bpp;
+   format->sizeimage = format->bytesperline * format->height;
+
+   if (format->field == V4L2_FIELD_ANY)
+   format->field = fmt_default.field;
+
+   /* Check if values are out of range */
+   if (format->colorspace == V4L2_COLORSPACE_DEFAULT
+   || format->colorspace > V4L2_COLORSPACE_DCI_P3)
+   format->colorspace = fmt_default.colorspace;
+   if (format->ycbcr_enc == V4L2_YCBCR_ENC_DEFAULT
+   || format->ycbcr_enc > V4L2_YCBCR_ENC_SMPTE240M)
+   format->ycbcr_enc = fmt_default.ycbcr_enc;
+   if (format->quantization == V4L2_QUANTIZATION_DEFAULT
+   || format->quantization > V4L2_QUANTIZATION_LIM_RANGE)
+   format->quantization = fmt_default.quantization;
+   if (format->xfer_func == V4L2_XFER_FUNC_DEFAULT
+   || format->xfer_func > V4L2_XFER_FUNC_SMPTE2084)
+   format->xfer_func = fmt_default.xfer_func;


Same comments in the previous patch regarding width/height and the colorspace
information apply here as well.


+
+   return 0;
+}
+
+static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+   struct vimc_cap_device *vcap = video_drvdata(file);
+
+   /* Do not change the format while stream is on */
+   if (vb2_is_busy(>queue))
+   return -EBUSY;
+
+   vimc_cap_try_fmt_vid_cap(file, priv, f);
+
+   dev_dbg(vcap->vdev.v4l2_dev->dev, "%s: format update: "

Re: please help with uninstall

[Helen Koike]

Hi,

On 2017-04-12 08:00 AM, Milan Čížek wrote:

How to remove your product from my kernel? I tried make rmmod
rminstall but this message stills in my syslog.
Sorry for question, I'm linux newbie.

[   15.753993] WARNING: You are using an experimental version of the
media stack.
As the driver is backported to an older kernel, it
doesn't offer
enough quality for its usage in production.
Use it with care.
   Latest git patches (needed if you report a bug to
linux-media@vger.kernel.org):
427ae153c65ad7a08288d86baf99000569627d03 [media]
bq/c-qcam, w9966, pms: move to staging in preparation for removal
ea2e813e8cc3492c951b9895724fd47187e04a6f [media]
tlg2300: move to staging in preparation for removal
c1d9e03d4ef47de60b414fa25f05f9c867f43c5a [media]
vino/saa7191: move to staging in preparation for removal

Milan



This message doesn't seem to be an error, after running rmmod does the 
driver still shows with lsmod?.
If you want to prevent a module to load at boot time you can add the 
driver in the blacklist file, its location depends on your distribution, 
check this example: 
http://askubuntu.com/questions/110341/how-to-blacklist-kernel-modules
Or you can recompile the kernel without this module, I can point you to 
some guides if you want to go to this direction.


Helen

[media] vimc: API proposal, configuring the topology from user space

[Helen Koike]

Hi,

Continuing the discussion about the API of the vimc driver, I made some 
changes
based on the previous comments, please see below and let me know your 
opinion about it.


Helen

/***
Configfs considerations:
/
Informal definitions:
subsystem: the root driver folder in user space (/configfs/vimc)
item: aka a folder in user space
attributes: aka files in the folder
group: aka a folder that can contain subfolders (parent and child 
relation)
	default group: aka a subfolder that is created automatically when the 
"parent" folder is created

it is not considered a child in terms of rmdir

* Performing rmdir in a group will fail if it contain children that are 
not default groups, i.e, if the
folder contain subfolders that are default group, then it can be removed 
with rmdir, if the

subfolders were created with mkdir, then rmdir in the parent will fail.

* Configfs has the notion of committable item but it is not implemented 
yet. A committable item is an item
that can be in one of two parent folders called: live and pending. The 
idea is to create and modify the item
in the pending directory and then to move the item through a rename to 
the live directory where
it can't be modified. This seems to be a nice feature for vimc, but as 
it is not available yet the

proposal below won't be based on this.

* Groups can be dynamically created/destroyed by the driver whenever it 
wants. Afaik attributes can only
be created when the group or item is created and symlinks can only be 
create from user space, i.e, the

driver don't know how to create/destroy attributes or symlinks in anytime.

/***
The API:
/

In short, a topology like this one: http://goo.gl/Y7eUfu
Would look like this filesystem tree: https://goo.gl/mEOmOf

v3 core changes:
- I removed the use of symlinks as I wans't able to see how to do it nicely.
- I use the names of the folders created by user space to retrieve 
information at mkdir time

- hotplug file in each entity
- hotplug file in each device
- reset file in each device

* The /configfs/vimc subsystem
empty when the driver is loaded

* Create a device
Userspace can create a new vimc device with:

$ mkdir /configfs/vimc/any_name
Example:
$ mkdir /configfs/vimc/vimc0
$ ls -l /configfs/vimc/vimc0
hotplug
reset
entities/
links/

entities/ and links/ folder are default groups, thus they don't prevent 
rmdir vimc0/, but

rmdir will fail if it has any child inside entities/ or links/.
hotplug is used to plug and unplug the device, it can read "plugged" or 
"unplugged" and user can

write "plug" or "unplug" to change its state.
Changing hotplug state will never fail as the configfs tree will always 
be in a valid state.
reset is used to easily destroy all the topology without the need to 
walk through all the children
to perform rmdir, writing 1 to reset file will set hotplug to 
"unplugged" and erase all folders

under entities/ and links/.

* Create an entity
Userspace can create a new entity with:

$ mkdir /configfs/vimc/vimc0/entities/:
Example:
$ mkdir /configfs/vimc/vimc0/entities/sensor:SensorA
$ ls -l /configfs/vimc/vimc0/entities/sensor:SensorA
hotplug
pad:source:0/

The name of the folder needs to be in the format : or it 
will be rejected, this allows the
creation of the right pads according to its role at mkdir time, 
eliminating the previously proposed role

and name files.
hotplug is used to plug and unplug the hw block, it can read "plugged" 
or "unplugged" and user can
write "plug" or "unplug" to change its state. As we don't support this 
yet in the media core, changing it

will only be allowed if /configfs/vimc/vimc0/hotplug is "unplugged".
hotplug file is "unplugged" by default.
Pads will be created as default groups with the name in the format 
pad:: and it

will be an empty folder.
If the hw block supports different number of pads, we could expose two 
files:

sinks
sources
where the user space can write the desired number of sink and source 
pads and the driver will dynamically

create the folders pad::

* Create a link
User space can create a link between two entities with:

	$ mkdir 
/configfs/vimc/vimc0/links/:->:

Example:
$ mkdir /configfs/vimc/vimc0/links/DebayerA:1->Scaler:0
$ ls -l /configfs/vimc/vimc0/links/DebayerA:1->Scaler:0
flags

mkdir will be rejected if folder is not on the format 
:->:.
mkdir will be rejected if either  or  
are not found in /configfs/vimc/vimc0/entities/

The link will only be created if both entities are in "plugged" state.
When an entity is removed from /configfs/vimc/vimc0/entities/ with 
rmdir, its corresponding link folders at

/configfs/vimc/vimc0/links will be automatically removed.
If one of the entities changes from "plugged" to "unplugged", the link 

[PATCH v2 5/7] [media] vimc: cap: Support several image formats

[Helen Koike]

Allow user space to change the image format as the frame size, the
pixel format, colorspace, quantization, field YCbCr encoding
and the transfer function

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2:
[media] vimc: cap: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- allow user space to change all fields from struct v4l2_pix_format
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- link_validate and try_fmt: also checks colospace, quantization, 
field, xfer_func, ycbcr_enc
- add struct v4l2_pix_format fmt_default
- add enum_framesizes
- enum_fmt_vid_cap: enumerate all formats from vimc_pix_map_table
- add mode dev_dbg


---
 drivers/media/platform/vimc/vimc-capture.c | 167 -
 1 file changed, 139 insertions(+), 28 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 93f6a09..a6441f7 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -40,6 +40,16 @@ struct vimc_cap_device {
struct media_pipeline pipe;
 };
 
+static const struct v4l2_pix_format fmt_default = {
+   .width = 640,
+   .height = 480,
+   .pixelformat = V4L2_PIX_FMT_RGB24,
+   .field = V4L2_FIELD_NONE,
+   .colorspace = V4L2_COLORSPACE_SRGB,
+   .quantization = V4L2_QUANTIZATION_FULL_RANGE,
+   .xfer_func = V4L2_XFER_FUNC_SRGB,
+};
+
 struct vimc_cap_buffer {
/*
 * struct vb2_v4l2_buffer must be the first element
@@ -64,7 +74,7 @@ static int vimc_cap_querycap(struct file *file, void *priv,
return 0;
 }
 
-static int vimc_cap_fmt_vid_cap(struct file *file, void *priv,
+static int vimc_cap_g_fmt_vid_cap(struct file *file, void *priv,
  struct v4l2_format *f)
 {
struct vimc_cap_device *vcap = video_drvdata(file);
@@ -74,16 +84,112 @@ static int vimc_cap_fmt_vid_cap(struct file *file, void 
*priv,
return 0;
 }
 
+static int vimc_cap_try_fmt_vid_cap(struct file *file, void *priv,
+   struct v4l2_format *f)
+{
+   struct v4l2_pix_format *format = >fmt.pix;
+   const struct vimc_pix_map *vpix;
+
+   format->width = clamp_t(u32, format->width, VIMC_FRAME_MIN_WIDTH,
+   VIMC_FRAME_MAX_WIDTH);
+   format->height = clamp_t(u32, format->height, VIMC_FRAME_MIN_HEIGHT,
+VIMC_FRAME_MAX_HEIGHT);
+
+   /* Don't accept a pixelformat that is not on the table */
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   if (!vpix) {
+   format->pixelformat = fmt_default.pixelformat;
+   vpix = vimc_pix_map_by_pixelformat(format->pixelformat);
+   }
+   /* TODO: Add support for custom bytesperline values */
+   format->bytesperline = format->width * vpix->bpp;
+   format->sizeimage = format->bytesperline * format->height;
+
+   if (format->field == V4L2_FIELD_ANY)
+   format->field = fmt_default.field;
+
+   /* Check if values are out of range */
+   if (format->colorspace == V4L2_COLORSPACE_DEFAULT
+   || format->colorspace > V4L2_COLORSPACE_DCI_P3)
+   format->colorspace = fmt_default.colorspace;
+   if (format->ycbcr_enc == V4L2_YCBCR_ENC_DEFAULT
+   || format->ycbcr_enc > V4L2_YCBCR_ENC_SMPTE240M)
+   format->ycbcr_enc = fmt_default.ycbcr_enc;
+   if (format->quantization == V4L2_QUANTIZATION_DEFAULT
+   || format->quantization > V4L2_QUANTIZATION_LIM_RANGE)
+   format->quantization = fmt_default.quantization;
+   if (format->xfer_func == V4L2_XFER_FUNC_DEFAULT
+   || format->xfer_func > V4L2_XFER_FUNC_SMPTE2084)
+   format->xfer_func = fmt_default.xfer_func;
+
+   return 0;
+}
+
+static int vimc_cap_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+   struct vimc_cap_device *vcap = video_drvdata(file);
+
+   /* Do not change the format while stream is on */
+   if (vb2_is_busy(>queue))
+   return -EBUSY;
+
+   vimc_cap_try_fmt_vid_cap(file, priv, f);
+
+   dev_dbg(vcap->vdev.v4l2_dev->dev, "%s: format update: "
+   "old:%dx%d (0x%x, %d, %d, %d, %d) "
+   "new:%dx%d (0x%x, %d, %d, %d, %d)\n", vcap->vdev.name,
+   /* old */
+   vcap->format.width, vcap->format.height,
+   vcap->format.pixelformat, vcap->format.colorspace,
+   vcap->format.quantization, vcap->format.xfer_func,
+   vcap->format.ycbcr_enc,
+   /* 

[PATCH v2 6/7] [media] vimc: deb: Add debayer filter

[Helen Koike]

Implement the debayer filter and integrate it with the core

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2:
[media] vimc: deb: Add debayer filter
- Using MEDIA_ENT_F_ATV_DECODER in function
- remove v4l2_dev and dev from vimc_deb_device struct
- src fmt propagates from the sink
- coding style
- remove redundant else if statements
- check end of enum and remove BUG_ON
- enum frame size with min and max values
- set/try fmt
- remove unecessary include freezer.h
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add deb_mean_win_size as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg


---
 drivers/media/platform/vimc/Makefile   |   2 +-
 drivers/media/platform/vimc/vimc-core.c|   6 +-
 drivers/media/platform/vimc/vimc-core.h|   2 +
 drivers/media/platform/vimc/vimc-debayer.c | 573 +
 drivers/media/platform/vimc/vimc-debayer.h |  28 ++
 5 files changed, 609 insertions(+), 2 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc-debayer.c
 create mode 100644 drivers/media/platform/vimc/vimc-debayer.h

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index c45195e..a6708f9 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,3 +1,3 @@
-vimc-objs := vimc-core.o vimc-capture.o vimc-sensor.o
+vimc-objs := vimc-core.o vimc-capture.o vimc-debayer.o vimc-sensor.o
 
 obj-$(CONFIG_VIDEO_VIMC) += vimc.o
diff --git a/drivers/media/platform/vimc/vimc-core.c 
b/drivers/media/platform/vimc/vimc-core.c
index bc4b1bb..51cbbf6 100644
--- a/drivers/media/platform/vimc/vimc-core.c
+++ b/drivers/media/platform/vimc/vimc-core.c
@@ -23,6 +23,7 @@
 
 #include "vimc-capture.h"
 #include "vimc-core.h"
+#include "vimc-debayer.h"
 #include "vimc-sensor.h"
 
 #define VIMC_PDEV_NAME "vimc"
@@ -637,9 +638,12 @@ static int vimc_device_register(struct vimc_device *vimc)
create_func = vimc_cap_create;
break;
 
+   case VIMC_ENT_NODE_DEBAYER:
+   create_func = vimc_deb_create;
+   break;
+
/* TODO: Instantiate the specific topology node */
case VIMC_ENT_NODE_INPUT:
-   case VIMC_ENT_NODE_DEBAYER:
case VIMC_ENT_NODE_SCALER:
default:
/*
diff --git a/drivers/media/platform/vimc/vimc-core.h 
b/drivers/media/platform/vimc/vimc-core.h
index 2146672..2e621fe 100644
--- a/drivers/media/platform/vimc/vimc-core.h
+++ b/drivers/media/platform/vimc/vimc-core.h
@@ -26,6 +26,8 @@
 #define VIMC_FRAME_MIN_WIDTH 16
 #define VIMC_FRAME_MIN_HEIGHT 16
 
+#define VIMC_FRAME_INDEX(lin, col, width, bpp) ((lin * width + col) * bpp)
+
 /**
  * struct vimc_pix_map - maps media bus code with v4l2 pixel format
  *
diff --git a/drivers/media/platform/vimc/vimc-debayer.c 
b/drivers/media/platform/vimc/vimc-debayer.c
new file mode 100644
index 000..24e5952
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -0,0 +1,573 @@
+/*
+ * vimc-debayer.c Virtual Media Controller Driver
+ *
+ * Copyright (C) 2015-2017 Helen Koike <helen.fornaz...@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include 
+#include 
+#include 
+#include 
+
+#include "vimc-debayer.h"
+
+static unsigned int deb_mean_win_size = 3;
+module_param(deb_mean_win_size, uint, );
+MODULE_PARM_DESC(deb_mean_win_size, " the window size to calculate the mean.\n"
+   "NOTE: the window size need to be an odd number, as the main pixel "
+   "stays in the center of the window, otherwise the next odd number "
+   "is considered");
+
+#define IS_SINK(pad) (!pad)
+#define IS_SRC(pad)  (pad)
+
+enum vimc_deb_rgb_colors {
+   VIMC_DEB_RED = 0,
+   VIMC_DEB_GREEN = 1,
+   VIMC_DEB_BLUE = 2,
+};
+
+struct vimc_deb_pix_map {
+   u32 code;
+   enum vimc_deb_rgb_colors order[2][2];
+};
+
+struct vimc_deb_device {
+   struct vimc_ent_device ved;
+   struct v4l2_subdev sd;
+   /* The active format */
+   struct v4l2_mbus_fra

[PATCH v2 7/7] [media] vimc: sca: Add scaler

[Helen Koike]

Implement scaler and integrated with the core

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2:
[media] vimc: sca: Add scaler
- Add function MEDIA_ENT_F_IO_V4L
- remove v4l2_dev and dev
- s/sink_mbus_fmt/sink_fmt
- remove BUG_ON, remove redundant if else, rewrite TODO, check end of 
enum
- rm src_width/height, enum fsize with min and max values
- set/try fmt
- remove unecessary include freezer.h
- core: add bayer boolean in pixel table
- coding style
- fix bug in enum frame size
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add sca_mult as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg


---
 drivers/media/platform/vimc/Makefile  |   3 +-
 drivers/media/platform/vimc/vimc-core.c   |  33 ++-
 drivers/media/platform/vimc/vimc-core.h   |   1 +
 drivers/media/platform/vimc/vimc-scaler.c | 426 ++
 drivers/media/platform/vimc/vimc-scaler.h |  28 ++
 5 files changed, 489 insertions(+), 2 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc-scaler.c
 create mode 100644 drivers/media/platform/vimc/vimc-scaler.h

diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
index a6708f9..f13a594 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,3 +1,4 @@
-vimc-objs := vimc-core.o vimc-capture.o vimc-debayer.o vimc-sensor.o
+vimc-objs := vimc-core.o vimc-capture.o vimc-debayer.o vimc-scaler.o \
+   vimc-sensor.o
 
 obj-$(CONFIG_VIDEO_VIMC) += vimc.o
diff --git a/drivers/media/platform/vimc/vimc-core.c 
b/drivers/media/platform/vimc/vimc-core.c
index 51cbbf6..3a04db2 100644
--- a/drivers/media/platform/vimc/vimc-core.c
+++ b/drivers/media/platform/vimc/vimc-core.c
@@ -24,6 +24,7 @@
 #include "vimc-capture.h"
 #include "vimc-core.h"
 #include "vimc-debayer.h"
+#include "vimc-scaler.h"
 #include "vimc-sensor.h"
 
 #define VIMC_PDEV_NAME "vimc"
@@ -198,6 +199,10 @@ static const struct vimc_pipeline_config pipe_cfg = {
 
 /* -- 
*/
 
+/*
+ * NOTE: non-bayer formats need to come first (necessary for enum_mbus_code
+ * in the scaler)
+ */
 static const struct vimc_pix_map vimc_pix_map_list[] = {
/* TODO: add all missing formats */
 
@@ -206,16 +211,19 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
.code = MEDIA_BUS_FMT_BGR888_1X24,
.pixelformat = V4L2_PIX_FMT_BGR24,
.bpp = 3,
+   .bayer = false,
},
{
.code = MEDIA_BUS_FMT_RGB888_1X24,
.pixelformat = V4L2_PIX_FMT_RGB24,
.bpp = 3,
+   .bayer = false,
},
{
.code = MEDIA_BUS_FMT_ARGB_1X32,
.pixelformat = V4L2_PIX_FMT_ARGB32,
.bpp = 4,
+   .bayer = false,
},
 
/* Bayer formats */
@@ -223,41 +231,49 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
.code = MEDIA_BUS_FMT_SBGGR8_1X8,
.pixelformat = V4L2_PIX_FMT_SBGGR8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGBRG8_1X8,
.pixelformat = V4L2_PIX_FMT_SGBRG8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGRBG8_1X8,
.pixelformat = V4L2_PIX_FMT_SGRBG8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SRGGB8_1X8,
.pixelformat = V4L2_PIX_FMT_SRGGB8,
.bpp = 1,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SBGGR10_1X10,
.pixelformat = V4L2_PIX_FMT_SBGGR10,
.bpp = 2,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGBRG10_1X10,
.pixelformat = V4L2_PIX_FMT_SGBRG10,
.bpp = 2,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SGRBG10_1X10,
.pixelformat = V4L2_PIX_FMT_SGRBG10,
.bpp = 2,
+   .bayer = true,
},
{
.code = MEDIA_BUS_FMT_SRGGB10_1X10,
.pixelformat = V4L2_PIX_FMT_SRGGB10,
.bpp = 2,
+   .bayer = true,
},
 
/* 10bit raw bayer a-law compressed to 8 bits */
@@ -265,21 +281,25 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
.co

[PATCH v2 2/7] [media] vimc: Add vimc_ent_sd_* helper functions

[Helen Koike]

As all the subdevices in the topology will be initialized in the same
way, to avoid code repetition the vimc_ent_sd_{register, unregister}
helper functions were created

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2:
[media] vimc: Add vimc_ent_sd_* helper functions
- Comments in vimc_ent_sd_init
- Update vimc_ent_sd_init with upstream code as media_entity_pads_init
(instead of media_entity_init), entity->function intead of entity->type
- Add missing vimc_pads_cleanup in vimc_ent_sd_cleanup
- remove subdevice v4l2_dev and dev fields
- change unregister order in vimc_ent_sd_cleanup
- rename vimc_ent_sd_{init,cleanup} to vimc_ent_sd_{register,unregister}
- remove struct vimc_ent_subdevice, use ved and sd directly
- don't impose struct vimc_sen_device to declare ved and sd struct first
- add kernel docs


---
 drivers/media/platform/vimc/vimc-core.c   | 66 +++
 drivers/media/platform/vimc/vimc-core.h   | 39 ++
 drivers/media/platform/vimc/vimc-sensor.c | 58 +--
 3 files changed, 114 insertions(+), 49 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-core.c 
b/drivers/media/platform/vimc/vimc-core.c
index bc107da..15fa311 100644
--- a/drivers/media/platform/vimc/vimc-core.c
+++ b/drivers/media/platform/vimc/vimc-core.c
@@ -416,6 +416,72 @@ struct media_pad *vimc_pads_init(u16 num_pads, const 
unsigned long *pads_flag)
return pads;
 }
 
+static const struct media_entity_operations vimc_ent_sd_mops = {
+   .link_validate = v4l2_subdev_link_validate,
+};
+
+int vimc_ent_sd_register(struct vimc_ent_device *ved,
+struct v4l2_subdev *sd,
+struct v4l2_device *v4l2_dev,
+const char *const name,
+u32 function,
+u16 num_pads,
+const unsigned long *pads_flag,
+const struct v4l2_subdev_ops *sd_ops,
+void (*sd_destroy)(struct vimc_ent_device *))
+{
+   int ret;
+
+   /* Allocate the pads */
+   ved->pads = vimc_pads_init(num_pads, pads_flag);
+   if (IS_ERR(ved->pads))
+   return PTR_ERR(ved->pads);
+
+   /* Fill the vimc_ent_device struct */
+   ved->destroy = sd_destroy;
+   ved->ent = >entity;
+
+   /* Initialize the subdev */
+   v4l2_subdev_init(sd, sd_ops);
+   sd->entity.function = MEDIA_ENT_F_CAM_SENSOR;
+   sd->entity.ops = _ent_sd_mops;
+   sd->owner = THIS_MODULE;
+   strlcpy(sd->name, name, sizeof(sd->name));
+   v4l2_set_subdevdata(sd, ved);
+
+   /* Expose this subdev to user space */
+   sd->flags = V4L2_SUBDEV_FL_HAS_DEVNODE;
+
+   /* Initialize the media entity */
+   ret = media_entity_pads_init(>entity, num_pads, ved->pads);
+   if (ret)
+   goto err_clean_pads;
+
+   /* Register the subdev with the v4l2 and the media framework */
+   ret = v4l2_device_register_subdev(v4l2_dev, sd);
+   if (ret) {
+   dev_err(v4l2_dev->dev,
+   "%s: subdev register failed (err=%d)\n",
+   name, ret);
+   goto err_clean_m_ent;
+   }
+
+   return 0;
+
+err_clean_m_ent:
+   media_entity_cleanup(>entity);
+err_clean_pads:
+   vimc_pads_cleanup(ved->pads);
+   return ret;
+}
+
+void vimc_ent_sd_unregister(struct vimc_ent_device *ved, struct v4l2_subdev 
*sd)
+{
+   v4l2_device_unregister_subdev(sd);
+   media_entity_cleanup(ved->ent);
+   vimc_pads_cleanup(ved->pads);
+}
+
 /*
  * TODO: remove this function when all the
  * entities specific code are implemented
diff --git a/drivers/media/platform/vimc/vimc-core.h 
b/drivers/media/platform/vimc/vimc-core.h
index 4525d23..92c4729 100644
--- a/drivers/media/platform/vimc/vimc-core.h
+++ b/drivers/media/platform/vimc/vimc-core.h
@@ -109,4 +109,43 @@ const struct vimc_pix_map *vimc_pix_map_by_code(u32 code);
  */
 const struct vimc_pix_map *vimc_pix_map_by_pixelformat(u32 pixelformat);
 
+/**
+ * vimc_ent_sd_register - initialize and register a subdev node
+ *
+ * @ved:   the vimc_ent_device struct to be initialize
+ * @sd:the v4l2_subdev struct to be initialize and registered
+ * @v4l2_dev:  the v4l2 device to register the v4l2_subdev
+ * @name:  name of the sub-device. Please notice that the name must be
+ * unique.
+ * @function:  media entity function defined by MEDIA_ENT_F_* macros
+ * @num_pads:  number of pads to initialize
+ * @pads_flag: flags to use in each pad
+ * @sd_ops:pointer to  v4l2_subdev_ops.
+ * @sd_destroy:callback to destroy the node
+ *
+ * Helper function initialize and register the struct vimc_ent_device and 
struct
+ * v4l2_subdev which

[PATCH v2 0/7] [media]: vimc: Virtual Media Control VPU's

[Helen Koike]

This patch series improves the current video processing units in vimc
(by adding more controls to the sensor and capture node, allowing the
user to configure different frame formats) and also adds a debayer
and a scaler node.
The debayer transforms the bayer format image received in its sink pad
to a bayer format by averaging the pixels within a mean window.
The scaler only scales up the image for now.

This patch series depends on commit "[media] vimc: Virtual Media Controller 
core, capture and sensor"
and it is available at:
https://github.com/helen-fornazier/opw-staging/tree/z/sent/vimc/vpu/v2

Changes in v2:
[media] vimc: sen: Integrate the tpg on the sensor
- Fix include location
- Select V4L2_TPG in Kconfig
- configure tpg on streamon only
- rm BUG_ON
- coding style
[media] vimc: Add vimc_ent_sd_* helper functions
- Comments in vimc_ent_sd_init
- Update vimc_ent_sd_init with upstream code as media_entity_pads_init
(instead of media_entity_init), entity->function intead of entity->type
- Add missing vimc_pads_cleanup in vimc_ent_sd_cleanup
- remove subdevice v4l2_dev and dev fields
- change unregister order in vimc_ent_sd_cleanup
- rename vimc_ent_sd_{init,cleanup} to vimc_ent_sd_{register,unregister}
- remove struct vimc_ent_subdevice, use ved and sd directly
- don't impose struct vimc_sen_device to declare ved and sd struct first
- add kernel docs
[media] vimc: Add vimc_pipeline_s_stream in the core
- Use is_media_entity_v4l2_subdev instead of comparing with the old
entity->type
- Fix comments style
- add kernel-docs
- call s_stream across all sink pads
[media] vimc: sen: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- add init_cfg to initialize try_fmt
- reorder code in vimc_sen_set_fmt
- allow user space to change all fields from struct v4l2_mbus_framefmt
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- merge with patch for the enum_mbus_code and enum_frame_size
- change commit message
- add vimc_pix_map_by_index
- rename MIN/MAX macros
- check set_fmt default parameters for quantization, colorspace ...
[media] vimc: cap: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- allow user space to change all fields from struct v4l2_pix_format
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- link_validate and try_fmt: also checks colospace, quantization, 
field, xfer_func, ycbcr_enc
- add struct v4l2_pix_format fmt_default
- add enum_framesizes
- enum_fmt_vid_cap: enumerate all formats from vimc_pix_map_table
- add mode dev_dbg
[media] vimc: deb: Add debayer filter
- Using MEDIA_ENT_F_ATV_DECODER in function
- remove v4l2_dev and dev from vimc_deb_device struct
- src fmt propagates from the sink
- coding style
- remove redundant else if statements
- check end of enum and remove BUG_ON
- enum frame size with min and max values
- set/try fmt
- remove unecessary include freezer.h
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add deb_mean_win_size as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg
[media] vimc: sca: Add scaler
- Add function MEDIA_ENT_F_IO_V4L
- remove v4l2_dev and dev
- s/sink_mbus_fmt/sink_fmt
- remove BUG_ON, remove redundant if else, rewrite TODO, check end of 
enum
- rm src_width/height, enum fsize with min and max values
- set/try fmt
- remove unecessary include freezer.h
- core: add bayer boolean in pixel table
- coding style
- fix bug in enum frame size
- check pad types on create
- return EBUSY when trying to set the format while stream is on
- remove vsd struct
- add IS_SRC and IS_SINK macros
- add sca_mult as a parameter of the module
- check set_fmt default parameters for quantization, colorspace ...
- add more dev_dbg

Helen Koike (7):
  [media] vimc: sen: Integrate the tpg on the sensor
  [media] vimc: Add vimc_ent_sd_* helper functions
  [media] vimc: Add vimc_pipeline_s_stream in the core
  [media] vimc: sen: Support several image formats
  [media] vimc: cap: Support several image formats
  [media] vimc: deb: Add debayer filter
  [media] vimc: sca: Add scaler

 drivers/media/platform/vimc/Kconfig|   1 +
 drivers/media/platform/vimc/Makefile   |   3 +-
 drivers/media/platform/vimc/v

[PATCH v2 1/7] [media] vimc: sen: Integrate the tpg on the sensor

[Helen Koike]

Initialize the test pattern generator on the sensor
Generate a colored bar image instead of a grey one

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2:
[media] vimc: sen: Integrate the tpg on the sensor
- Fix include location
- Select V4L2_TPG in Kconfig
- configure tpg on streamon only
- rm BUG_ON
- coding style


---
 drivers/media/platform/vimc/Kconfig   |  1 +
 drivers/media/platform/vimc/vimc-sensor.c | 43 +--
 2 files changed, 42 insertions(+), 2 deletions(-)

diff --git a/drivers/media/platform/vimc/Kconfig 
b/drivers/media/platform/vimc/Kconfig
index dd285fa..df124d4 100644
--- a/drivers/media/platform/vimc/Kconfig
+++ b/drivers/media/platform/vimc/Kconfig
@@ -2,6 +2,7 @@ config VIDEO_VIMC
tristate "Virtual Media Controller Driver (VIMC)"
depends on VIDEO_DEV && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
select VIDEOBUF2_VMALLOC
+   select VIDEO_V4L2_TPG
default n
---help---
  Skeleton driver for Virtual Media Controller
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index 591f6a4..9154322 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -20,12 +20,16 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "vimc-sensor.h"
 
+#define VIMC_SEN_FRAME_MAX_WIDTH 4096
+
 struct vimc_sen_device {
struct vimc_ent_device ved;
struct v4l2_subdev sd;
+   struct tpg_data tpg;
struct task_struct *kthread_sen;
u8 *frame;
/* The active format */
@@ -84,6 +88,28 @@ static int vimc_sen_get_fmt(struct v4l2_subdev *sd,
return 0;
 }
 
+static void vimc_sen_tpg_s_format(struct vimc_sen_device *vsen)
+{
+   const struct vimc_pix_map *vpix =
+   vimc_pix_map_by_code(vsen->mbus_format.code);
+
+   tpg_reset_source(>tpg, vsen->mbus_format.width,
+vsen->mbus_format.height, vsen->mbus_format.field);
+   tpg_s_bytesperline(>tpg, 0,
+  vsen->mbus_format.width * vpix->bpp);
+   tpg_s_buf_height(>tpg, vsen->mbus_format.height);
+   tpg_s_fourcc(>tpg, vpix->pixelformat);
+   /* TODO: check why the tpg_s_field need this third argument if
+* it is already receiving the field
+*/
+   tpg_s_field(>tpg, vsen->mbus_format.field,
+   vsen->mbus_format.field == V4L2_FIELD_ALTERNATE);
+   tpg_s_colorspace(>tpg, vsen->mbus_format.colorspace);
+   tpg_s_ycbcr_enc(>tpg, vsen->mbus_format.ycbcr_enc);
+   tpg_s_quantization(>tpg, vsen->mbus_format.quantization);
+   tpg_s_xfer_func(>tpg, vsen->mbus_format.xfer_func);
+}
+
 static const struct v4l2_subdev_pad_ops vimc_sen_pad_ops = {
.enum_mbus_code = vimc_sen_enum_mbus_code,
.enum_frame_size= vimc_sen_enum_frame_size,
@@ -110,7 +136,7 @@ static int vimc_thread_sen(void *data)
if (kthread_should_stop())
break;
 
-   memset(vsen->frame, 100, vsen->frame_size);
+   tpg_fill_plane_buffer(>tpg, V4L2_STD_PAL, 0, vsen->frame);
 
/* Send the frame to all source pads */
for (i = 0; i < vsen->sd.entity.num_pads; i++)
@@ -159,6 +185,9 @@ static int vimc_sen_s_stream(struct v4l2_subdev *sd, int 
enable)
vsen->frame = NULL;
return PTR_ERR(vsen->kthread_sen);
}
+
+   /* configure the test pattern generator */
+   vimc_sen_tpg_s_format(vsen);
} else {
if (!vsen->kthread_sen)
return -EINVAL;
@@ -189,6 +218,7 @@ static void vimc_sen_destroy(struct vimc_ent_device *ved)
struct vimc_sen_device *vsen =
container_of(ved, struct vimc_sen_device, ved);
 
+   tpg_free(>tpg);
v4l2_device_unregister_subdev(>sd);
media_entity_cleanup(ved->ent);
kfree(vsen);
@@ -254,17 +284,26 @@ struct vimc_ent_device *vimc_sen_create(struct 
v4l2_device *v4l2_dev,
vsen->mbus_format.quantization = V4L2_QUANTIZATION_FULL_RANGE;
vsen->mbus_format.xfer_func = V4L2_XFER_FUNC_SRGB;
 
+   /* Initialize the test pattern generator */
+   tpg_init(>tpg, vsen->mbus_format.width,
+vsen->mbus_format.height);
+   ret = tpg_alloc(>tpg, VIMC_SEN_FRAME_MAX_WIDTH);
+   if (ret)
+   goto err_clean_m_ent;
+
/* Register the subdev with the v4l2 and the media framework */
ret = v4l2_device_register_subdev(v4l2_dev, >sd);
if (ret) {
dev_err(vsen->sd.v4l2_dev->dev,
"%s: subdev

[PATCH v2 4/7] [media] vimc: sen: Support several image formats

[Helen Koike]

Allow user space to change the image format as the frame size, the
media bus pixel format, colorspace, quantization, field YCbCr encoding
and the transfer function

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2:
[media] vimc: sen: Support several image formats
- this is a new commit in the serie (the old one was splitted in two)
- add init_cfg to initialize try_fmt
- reorder code in vimc_sen_set_fmt
- allow user space to change all fields from struct v4l2_mbus_framefmt
  (e.g. colospace, quantization, field, xfer_func, ycbcr_enc)
- merge with patch for the enum_mbus_code and enum_frame_size
- change commit message
- add vimc_pix_map_by_index
- rename MIN/MAX macros
- check set_fmt default parameters for quantization, colorspace ...


---
 drivers/media/platform/vimc/vimc-core.c   |   8 ++
 drivers/media/platform/vimc/vimc-core.h   |  12 +++
 drivers/media/platform/vimc/vimc-sensor.c | 143 --
 3 files changed, 134 insertions(+), 29 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-core.c 
b/drivers/media/platform/vimc/vimc-core.c
index 7c23735..bc4b1bb 100644
--- a/drivers/media/platform/vimc/vimc-core.c
+++ b/drivers/media/platform/vimc/vimc-core.c
@@ -324,6 +324,14 @@ static const struct vimc_pix_map vimc_pix_map_list[] = {
},
 };
 
+const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i)
+{
+   if (i >= ARRAY_SIZE(vimc_pix_map_list))
+   return NULL;
+
+   return _pix_map_list[i];
+}
+
 const struct vimc_pix_map *vimc_pix_map_by_code(u32 code)
 {
unsigned int i;
diff --git a/drivers/media/platform/vimc/vimc-core.h 
b/drivers/media/platform/vimc/vimc-core.h
index 8c3d401..2146672 100644
--- a/drivers/media/platform/vimc/vimc-core.h
+++ b/drivers/media/platform/vimc/vimc-core.h
@@ -21,6 +21,11 @@
 #include 
 #include 
 
+#define VIMC_FRAME_MAX_WIDTH 4096
+#define VIMC_FRAME_MAX_HEIGHT 2160
+#define VIMC_FRAME_MIN_WIDTH 16
+#define VIMC_FRAME_MIN_HEIGHT 16
+
 /**
  * struct vimc_pix_map - maps media bus code with v4l2 pixel format
  *
@@ -107,6 +112,13 @@ static inline void vimc_pads_cleanup(struct media_pad 
*pads)
 int vimc_pipeline_s_stream(struct media_entity *ent, int enable);
 
 /**
+ * vimc_pix_map_by_index - get vimc_pix_map struct by its index
+ *
+ * @i: index of the vimc_pix_map struct in vimc_pix_map_list
+ */
+const struct vimc_pix_map *vimc_pix_map_by_index(unsigned int i);
+
+/**
  * vimc_pix_map_by_code - get vimc_pix_map struct by media bus code
  *
  * @code:  media bus format code defined by MEDIA_BUS_FMT_* macros
diff --git a/drivers/media/platform/vimc/vimc-sensor.c 
b/drivers/media/platform/vimc/vimc-sensor.c
index abb2172..c86b4e6 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -24,8 +24,6 @@
 
 #include "vimc-sensor.h"
 
-#define VIMC_SEN_FRAME_MAX_WIDTH 4096
-
 struct vimc_sen_device {
struct vimc_ent_device ved;
struct v4l2_subdev sd;
@@ -37,18 +35,41 @@ struct vimc_sen_device {
int frame_size;
 };
 
+static const struct v4l2_mbus_framefmt fmt_default = {
+   .width = 640,
+   .height = 480,
+   .code = MEDIA_BUS_FMT_RGB888_1X24,
+   .field = V4L2_FIELD_NONE,
+   .colorspace = V4L2_COLORSPACE_SRGB,
+   .quantization = V4L2_QUANTIZATION_FULL_RANGE,
+   .xfer_func = V4L2_XFER_FUNC_SRGB,
+};
+
+static int vimc_sen_init_cfg(struct v4l2_subdev *sd,
+struct v4l2_subdev_pad_config *cfg)
+{
+   unsigned int i;
+
+   for (i = 0; i < sd->entity.num_pads; i++) {
+   struct v4l2_mbus_framefmt *mf;
+
+   mf = v4l2_subdev_get_try_format(sd, cfg, i);
+   *mf = fmt_default;
+   }
+
+   return 0;
+}
+
 static int vimc_sen_enum_mbus_code(struct v4l2_subdev *sd,
   struct v4l2_subdev_pad_config *cfg,
   struct v4l2_subdev_mbus_code_enum *code)
 {
-   struct vimc_sen_device *vsen =
-   container_of(sd, struct vimc_sen_device, sd);
+   const struct vimc_pix_map *vpix = vimc_pix_map_by_index(code->index);
 
-   /* TODO: Add support for other codes */
-   if (code->index)
+   if (!vpix)
return -EINVAL;
 
-   code->code = vsen->mbus_format.code;
+   code->code = vpix->code;
 
return 0;
 }
@@ -57,33 +78,34 @@ static int vimc_sen_enum_frame_size(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_frame_size_enum *fse)
 {
-   struct vimc_sen_device *vsen =
-   container_of(sd, struct vimc_sen_device, sd);
+   const struct vimc_pix_map *vpix;
 
-   /* TODO: Add support

[PATCH v2 3/7] [media] vimc: Add vimc_pipeline_s_stream in the core

[Helen Koike]

Move the vimc_cap_pipeline_s_stream from the vimc-cap.c to vimc-core.c
as this core will be reused by other subdevices to activate the stream
in their directly connected nodes

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Changes in v2:
[media] vimc: Add vimc_pipeline_s_stream in the core
- Use is_media_entity_v4l2_subdev instead of comparing with the old
entity->type
- Fix comments style
- add kernel-docs
- call s_stream across all sink pads


---
 drivers/media/platform/vimc/vimc-capture.c | 29 ++-
 drivers/media/platform/vimc/vimc-core.c| 32 ++
 drivers/media/platform/vimc/vimc-core.h| 11 ++
 3 files changed, 45 insertions(+), 27 deletions(-)

diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
index 9adb06d..93f6a09 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -132,31 +132,6 @@ static void vimc_cap_return_all_buffers(struct 
vimc_cap_device *vcap,
spin_unlock(>qlock);
 }
 
-static int vimc_cap_pipeline_s_stream(struct vimc_cap_device *vcap, int enable)
-{
-   struct v4l2_subdev *sd;
-   struct media_pad *pad;
-   int ret;
-
-   /* Start the stream in the subdevice direct connected */
-   pad = media_entity_remote_pad(>vdev.entity.pads[0]);
-
-   /*
-* if it is a raw node from vimc-core, there is nothing to activate
-* TODO: remove this when there are no more raw nodes in the
-* core and return error instead
-*/
-   if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
-   return 0;
-
-   sd = media_entity_to_v4l2_subdev(pad->entity);
-   ret = v4l2_subdev_call(sd, video, s_stream, enable);
-   if (ret && ret != -ENOIOCTLCMD)
-   return ret;
-
-   return 0;
-}
-
 static int vimc_cap_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
@@ -173,7 +148,7 @@ static int vimc_cap_start_streaming(struct vb2_queue *vq, 
unsigned int count)
}
 
/* Enable streaming from the pipe */
-   ret = vimc_cap_pipeline_s_stream(vcap, 1);
+   ret = vimc_pipeline_s_stream(>vdev.entity, 1);
if (ret) {
media_pipeline_stop(entity);
vimc_cap_return_all_buffers(vcap, VB2_BUF_STATE_QUEUED);
@@ -192,7 +167,7 @@ static void vimc_cap_stop_streaming(struct vb2_queue *vq)
struct vimc_cap_device *vcap = vb2_get_drv_priv(vq);
 
/* Disable streaming from the pipe */
-   vimc_cap_pipeline_s_stream(vcap, 0);
+   vimc_pipeline_s_stream(>vdev.entity, 0);
 
/* Stop the media pipeline */
media_pipeline_stop(>vdev.entity);
diff --git a/drivers/media/platform/vimc/vimc-core.c 
b/drivers/media/platform/vimc/vimc-core.c
index 15fa311..7c23735 100644
--- a/drivers/media/platform/vimc/vimc-core.c
+++ b/drivers/media/platform/vimc/vimc-core.c
@@ -396,6 +396,38 @@ static void vimc_device_unregister(struct vimc_device 
*vimc)
media_device_cleanup(>mdev);
 }
 
+int vimc_pipeline_s_stream(struct media_entity *ent, int enable)
+{
+   struct v4l2_subdev *sd;
+   struct media_pad *pad;
+   unsigned int i;
+   int ret;
+
+   for (i = 0; i < ent->num_pads; i++) {
+   if (ent->pads[i].flags & MEDIA_PAD_FL_SOURCE)
+   continue;
+
+   /* Start the stream in the subdevice direct connected */
+   pad = media_entity_remote_pad(>pads[i]);
+
+   /*
+* if this is a raw node from vimc-core, then there is
+* nothing to activate
+* TODO: remove this when there are no more raw nodes in the
+* core and return error instead
+*/
+   if (pad->entity->obj_type == MEDIA_ENTITY_TYPE_BASE)
+   continue;
+
+   sd = media_entity_to_v4l2_subdev(pad->entity);
+   ret = v4l2_subdev_call(sd, video, s_stream, enable);
+   if (ret && ret != -ENOIOCTLCMD)
+   return ret;
+   }
+
+   return 0;
+}
+
 /* Helper function to allocate and initialize pads */
 struct media_pad *vimc_pads_init(u16 num_pads, const unsigned long *pads_flag)
 {
diff --git a/drivers/media/platform/vimc/vimc-core.h 
b/drivers/media/platform/vimc/vimc-core.h
index 92c4729..8c3d401 100644
--- a/drivers/media/platform/vimc/vimc-core.h
+++ b/drivers/media/platform/vimc/vimc-core.h
@@ -96,6 +96,17 @@ static inline void vimc_pads_cleanup(struct media_pad *pads)
 }
 
 /**
+ * vimc_pipeline_s_stream - start stream through the pipeline
+ *
+ * @ent:   the pointer to struct media_entity for the node
+ * @enable:1 to start the stream and 0 to stop
+ *
+ 

[PATCH v10] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
Reviewed-by: Hans Verkuil <hans.verk...@cisco.com>

---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/z/sent/vimc/skel/v10

Changes since v9:
- reuturn -EPIPE instead of -EINVAL in link_validate when formats don't
match
- remove check for vb2_is_busy in vimc_cap_process_frame as it can cause
a dead lock with stop_streaming ioctl
- add vb2_ioctl_prepare_buf in vimc_cap_ioctl_ops
- add reviewd-by tag from Hans
- update MAINTAINERS file

Changes since v8:
- fix vimc_sen_enum_mbus_code: return EINVAL if code->index > 0
- remove input ioctls from vimc-capture.c as v4l2 intends to have default
input ioctls in the core
- change kernel-docs for vimc_pix_map_by_* so the line fits in 80
characters

Changes since v7:
- remove left over union in struct vimc_ent_device
- remove v4l2_dev pointer from vimc_sen_device structure
- remove unused dev parameter from vimc_propagate_frame()
- remove struct device *dev from struct vimc_cap_device and vimc_sen_device
- in vimc_sen_create: call media_entity_pads_init() after v4l2_subdev_init()
to avoid double initialization of vsen->sd.entity.name
- in vimc_sen_destroy: move media_entity_cleanup to be after 
v4l2_device_unregister_subdev
- rename video_device back to vdev instead of vd
- adjust copyright with range 2015-2017
- add node names in dev_err prints for vimc-capture.c and vimc-sensor.c
- remove prefix "cap" in dev_dbg as it already uses the name of the node

Changes since v6:
- add kernel-docs in vimc-core.h
- reorder list_del call in vimc_cap_return_all_buffers()
- call media_pipeline_stop in vimc_cap_start_streaming when fail
- remove DMA comment (left over from the sample driver)
- remove vb2_set_plane_payload call in vimc_cap_buffer_prepare
- remove format verification in vimc_cap_link_validate
- set vimc_pix_map_list as static
- use MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN in vimc_raw_create()
- register media device after creating the topology and unregister it before 
destroying the topology
- replace devm_kzalloc by kzalloc for allocating struct vimc_device
- uset TASK_UNINTERRUPTIBLE for vimc_thread_sen
- do not allow the creation of a sensor with no pads
- add more verbose description in Kconfig
- change copyright to 2017
- arrange includes: number before letters
- remove v4l2_dev pointer from vimc_cap_device structure
- coding style adjustments
- remove entity variable in vimc_cap_pipeline_s_stream
- declare and assign variables in vimc_cap_link_validate
- declare vimc_dev_release() and vimc_pdev closer to vimc_init()
- remove pad check in vimc_sen_enum_{mbus_code,frame_size} that is already 
performed by v4l2-subdev.c
- fix multiline comments to start with /*
- reorder variable declaration in functions
- remove pad and subdevice type check in vimc_cap_pipeline_s_stream
- add a note that sensor nodes can can have more then one source pad
- remove the use of entity->use_count in the core and attach the ved structure 
in sd or vd, use
  ent->obj_type to know which structure (sd or vd) to use
- rename vdev (video_device) to vd to be similar to sd (subdev)

Changes since v5:
- Fix message "Entity type for entity Sensor A was not initialized!"
  by initializing the sensor entity.function after the calling
  v4l2_subded_init
- populate device_caps in vimc_cap_create instead of in
  vimc_cap_querycap
- Fix typo in vimc-core.c s/de/the

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or v4l2_dev->dev is null
- vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use 

Re: [PATCH] [media] media-entity: only call dev_dbg_obj if mdev is not NULL

[Helen Koike]

Hi Sakari,

On 2017-04-07 04:40 AM, Sakari Ailus wrote:

Hi Helen,

On Thu, Apr 06, 2017 at 04:32:00PM -0300, Helen Koike wrote:

Fix kernel Oops NULL pointer deference
Call dev_dbg_obj only after checking if gobj->mdev is not NULL

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
 drivers/media/media-entity.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 5640ca2..bc44193 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -199,12 +199,12 @@ void media_gobj_create(struct media_device *mdev,

 void media_gobj_destroy(struct media_gobj *gobj)
 {
-   dev_dbg_obj(__func__, gobj);
-
/* Do nothing if the object is not linked. */
if (gobj->mdev == NULL)
return;

+   dev_dbg_obj(__func__, gobj);
+
gobj->mdev->topology_version++;

/* Remove the object from mdev list */


Where is media_gobj_destroy() called with an object with NULL mdev?

I do not object to the change, but would like to know because I don't think
it's supposed to happen.



This happens when media_device_unregister(mdev) is called before 
unregistering the subdevices v4l2_device_unregister_subdev(sd) (which 
should be possible).


v4l2_device_unregister_subdev(sd) ends up calling v4l2_device_release() 
that calls media_device_unregister_entity() again (previously called by 
media_device_unregister(mdev)


Helen



There are issues though, until the patches fixing object referencing are
finished and merged. Unfortunately I haven't been able to work on those
recently, will pick them up again soon...

[PATCH] [media] media-entity: only call dev_dbg_obj if mdev is not NULL

[Helen Koike]

Fix kernel Oops NULL pointer deference
Call dev_dbg_obj only after checking if gobj->mdev is not NULL

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
 drivers/media/media-entity.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 5640ca2..bc44193 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -199,12 +199,12 @@ void media_gobj_create(struct media_device *mdev,
 
 void media_gobj_destroy(struct media_gobj *gobj)
 {
-   dev_dbg_obj(__func__, gobj);
-
/* Do nothing if the object is not linked. */
if (gobj->mdev == NULL)
return;
 
+   dev_dbg_obj(__func__, gobj);
+
gobj->mdev->topology_version++;
 
/* Remove the object from mdev list */
-- 
2.7.4

[PATCH] [media] media-entity: only call dev_dbg_obj if mdev is not NULL

[Helen Koike]

Fix kernel Oops NULL pointer deference
Call dev_dbg_obj only after checking if gobj->mdev is not NULL

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
 drivers/media/media-entity.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
index 5640ca2..bc44193 100644
--- a/drivers/media/media-entity.c
+++ b/drivers/media/media-entity.c
@@ -199,12 +199,12 @@ void media_gobj_create(struct media_device *mdev,
 
 void media_gobj_destroy(struct media_gobj *gobj)
 {
-   dev_dbg_obj(__func__, gobj);
-
/* Do nothing if the object is not linked. */
if (gobj->mdev == NULL)
return;
 
+   dev_dbg_obj(__func__, gobj);
+
gobj->mdev->topology_version++;
 
/* Remove the object from mdev list */
-- 
2.7.4

Re: [PATCH v9] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

Hi,

There are two points below that I'll change in v10. I'll wait for your 
comments in the rest of the code so I can send all the requested changes 
in v10


On 2017-04-03 07:16 PM, Helen Koike wrote:

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/vimc/devel/v8

Changes since v8:
- fix vimc_sen_enum_mbus_code: return EINVAL if code->index > 0
- remove input ioctls from vimc-capture.c as v4l2 intends to have default
input ioctls in the core
- change kernel-docs for vimc_pix_map_by_* so the line fits in 80
characters

Changes since v7:
- remove left over union in struct vimc_ent_device
- remove v4l2_dev pointer from vimc_sen_device structure
- remove unused dev parameter from vimc_propagate_frame()
- remove struct device *dev from struct vimc_cap_device and vimc_sen_device
- in vimc_sen_create: call media_entity_pads_init() after v4l2_subdev_init()
to avoid double initialization of vsen->sd.entity.name
- in vimc_sen_destroy: move media_entity_cleanup to be after 
v4l2_device_unregister_subdev
- rename video_device back to vdev instead of vd
- adjust copyright with range 2015-2017
- add node names in dev_err prints for vimc-capture.c and vimc-sensor.c
- remove prefix "cap" in dev_dbg as it already uses the name of the node

Changes since v6:
- add kernel-docs in vimc-core.h
- reorder list_del call in vimc_cap_return_all_buffers()
- call media_pipeline_stop in vimc_cap_start_streaming when fail
- remove DMA comment (left over from the sample driver)
- remove vb2_set_plane_payload call in vimc_cap_buffer_prepare
- remove format verification in vimc_cap_link_validate
- set vimc_pix_map_list as static
- use MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN in vimc_raw_create()
- register media device after creating the topology and unregister it before 
destroying the topology
- replace devm_kzalloc by kzalloc for allocating struct vimc_device
- uset TASK_UNINTERRUPTIBLE for vimc_thread_sen
- do not allow the creation of a sensor with no pads
- add more verbose description in Kconfig
- change copyright to 2017
- arrange includes: number before letters
- remove v4l2_dev pointer from vimc_cap_device structure
- coding style adjustments
- remove entity variable in vimc_cap_pipeline_s_stream
- declare and assign variables in vimc_cap_link_validate
- declare vimc_dev_release() and vimc_pdev closer to vimc_init()
- remove pad check in vimc_sen_enum_{mbus_code,frame_size} that is already 
performed by v4l2-subdev.c
- fix multiline comments to start with /*
- reorder variable declaration in functions
- remove pad and subdevice type check in vimc_cap_pipeline_s_stream
- add a note that sensor nodes can can have more then one source pad
- remove the use of entity->use_count in the core and attach the ved structure 
in sd or vd, use
  ent->obj_type to know which structure (sd or vd) to use
- rename vdev (video_device) to vd to be similar to sd (subdev)

Changes since v5:
- Fix message "Entity type for entity Sensor A was not initialized!"
  by initializing the sensor entity.function after the calling
  v4l2_subded_init
- populate device_caps in vimc_cap_create instead of in
  vimc_cap_querycap
- Fix typo in vimc-core.c s/de/the

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or v4l2_dev->dev is null
- vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to k

[PATCH v9] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/vimc/devel/v8

Changes since v8:
- fix vimc_sen_enum_mbus_code: return EINVAL if code->index > 0
- remove input ioctls from vimc-capture.c as v4l2 intends to have default
input ioctls in the core
- change kernel-docs for vimc_pix_map_by_* so the line fits in 80
characters

Changes since v7:
- remove left over union in struct vimc_ent_device
- remove v4l2_dev pointer from vimc_sen_device structure
- remove unused dev parameter from vimc_propagate_frame()
- remove struct device *dev from struct vimc_cap_device and vimc_sen_device
- in vimc_sen_create: call media_entity_pads_init() after v4l2_subdev_init()
to avoid double initialization of vsen->sd.entity.name
- in vimc_sen_destroy: move media_entity_cleanup to be after 
v4l2_device_unregister_subdev
- rename video_device back to vdev instead of vd
- adjust copyright with range 2015-2017
- add node names in dev_err prints for vimc-capture.c and vimc-sensor.c
- remove prefix "cap" in dev_dbg as it already uses the name of the node

Changes since v6:
- add kernel-docs in vimc-core.h
- reorder list_del call in vimc_cap_return_all_buffers()
- call media_pipeline_stop in vimc_cap_start_streaming when fail
- remove DMA comment (left over from the sample driver)
- remove vb2_set_plane_payload call in vimc_cap_buffer_prepare
- remove format verification in vimc_cap_link_validate
- set vimc_pix_map_list as static
- use MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN in vimc_raw_create()
- register media device after creating the topology and unregister it before 
destroying the topology
- replace devm_kzalloc by kzalloc for allocating struct vimc_device
- uset TASK_UNINTERRUPTIBLE for vimc_thread_sen
- do not allow the creation of a sensor with no pads
- add more verbose description in Kconfig
- change copyright to 2017
- arrange includes: number before letters
- remove v4l2_dev pointer from vimc_cap_device structure
- coding style adjustments
- remove entity variable in vimc_cap_pipeline_s_stream
- declare and assign variables in vimc_cap_link_validate
- declare vimc_dev_release() and vimc_pdev closer to vimc_init()
- remove pad check in vimc_sen_enum_{mbus_code,frame_size} that is already 
performed by v4l2-subdev.c
- fix multiline comments to start with /*
- reorder variable declaration in functions
- remove pad and subdevice type check in vimc_cap_pipeline_s_stream
- add a note that sensor nodes can can have more then one source pad
- remove the use of entity->use_count in the core and attach the ved structure 
in sd or vd, use
  ent->obj_type to know which structure (sd or vd) to use
- rename vdev (video_device) to vd to be similar to sd (subdev)

Changes since v5:
- Fix message "Entity type for entity Sensor A was not initialized!"
  by initializing the sensor entity.function after the calling
  v4l2_subded_init
- populate device_caps in vimc_cap_create instead of in
  vimc_cap_querycap
- Fix typo in vimc-core.c s/de/the

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or v4l2_dev->dev is null
- vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to keep track of entities instead of the old
entity->id
- Replace media_entity_init by media_entity_pads_init
---
 drivers/media/platform/Kconfig |   2 +
 drivers/media/platform/Makefile|   1 +
 d

Re: [PATCH RFC 1/2] [media] v4l2: add V4L2_INPUT_TYPE_DEFAULT

[Helen Koike]

Hi,

On 2017-03-31 06:57 AM, Mauro Carvalho Chehab wrote:

Em Fri, 31 Mar 2017 10:29:04 +0200
Hans Verkuil <hverk...@xs4all.nl> escreveu:


On 30/03/17 18:02, Helen Koike wrote:

Add V4L2_INPUT_TYPE_DEFAULT and helpers functions for input ioctls to be
used when no inputs are available in the device

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 27 +++
 include/media/v4l2-ioctl.h   | 26 ++
 include/uapi/linux/videodev2.h   |  1 +
 3 files changed, 54 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 0c3f238..ccaf04b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2573,6 +2573,33 @@ struct mutex *v4l2_ioctl_get_lock(struct video_device 
*vdev, unsigned cmd)
return vdev->lock;
 }

+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+ struct v4l2_input *i)
+{
+   if (i->index > 0)
+   return -EINVAL;
+
+   memset(i, 0, sizeof(*i));
+   i->type = V4L2_INPUT_TYPE_DEFAULT;
+   strlcpy(i->name, "Default", sizeof(i->name));
+
+   return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_enum_input_default);
+
+int v4l2_ioctl_g_input_default(struct file *file, void *priv, unsigned int *i)
+{
+   *i = 0;
+   return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_g_input_default);
+
+int v4l2_ioctl_s_input_default(struct file *file, void *priv, unsigned int i)
+{
+   return i ? -EINVAL : 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_s_input_default);
+
 /* Common ioctl debug function. This function can be used by
external ioctl messages as well as internal V4L ioctl */
 void v4l_printk_ioctl(const char *prefix, unsigned int cmd)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index 6cd94e5..accc470 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -652,6 +652,32 @@ struct video_device;
  */
 struct mutex *v4l2_ioctl_get_lock(struct video_device *vdev, unsigned int cmd);

+
+/**
+ * v4l2_ioctl_enum_input_default - v4l2 ioctl helper for VIDIOC_ENUM_INPUT 
ioctl
+ *
+ * Plug this function in vidioc_enum_input field of the struct v4l2_ioctl_ops 
to
+ * enumerate a single input as V4L2_INPUT_TYPE_DEFAULT
+ */
+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+ struct v4l2_input *i);
+
+/**
+ * v4l2_ioctl_g_input_default - v4l2 ioctl helper for VIDIOC_G_INPUT ioctl
+ *
+ * Plug this function in vidioc_g_input field of the struct v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_g_input_default(struct file *file, void *priv, unsigned int *i);
+
+/**
+ * v4l2_ioctl_s_input_default - v4l2 ioctl helper for VIDIOC_S_INPUT ioctl
+ *
+ * Plug this function in vidioc_s_input field of the struct v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_s_input_default(struct file *file, void *priv, unsigned int i);
+
 /* names for fancy debug output */
 extern const char *v4l2_field_names[];
 extern const char *v4l2_type_names[];
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 316be62..c10bbde 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1477,6 +1477,7 @@ struct v4l2_input {
 };

 /*  Values for the 'type' field */
+#define V4L2_INPUT_TYPE_DEFAULT0


I don't think we should add a new type here.


I second that. Just replied the same thing on a comment from Sakari to
patch 2/2.


The whole point of this exercise is to
allow existing apps to work, and existing apps expect a TYPE_CAMERA.

BTW, don't read to much in the term 'CAMERA': it's really a catch all for any 
video
stream, whether it is from a sensor, composite input, HDMI, etc.

The description for V4L2_INPUT_TYPE_CAMERA in the spec is hopelessly out of 
date :-(


Yeah, we always used "CAMERA" to mean NOT_TUNER.


Rather than creating a new type I would add a new V4L2_IN_CAP_MC capability that
indicates that this input is controlled via the media controller. That makes 
much
more sense and it wouldn't potentially break applications.

Exactly the same can be done for outputs as well: add V4L2_OUT_CAP_MC and use
V4L2_OUTPUT_TYPE_ANALOG as the output type (again, a horrible outdated name and 
the
spec is again out of date).


I don't see any sense on distinguishing IN and OUT for MC. I mean: should
we ever allow that any driver to have their inputs controlled via V4L2 API,
and their outputs controlled via MC (or vice-versa)? I don't think so.

Either all device inputs/outputs are controlled via V4L2 or via MC. So,
let's call it just V4L2_CAP_MC.


Regarding the name: should we use the name stored in struct video_device 
instead?
That might be more descriptive.


Makes sense to me.


Alternatively use something 

Re: [PATCH RFC 1/2] [media] v4l2: add V4L2_INPUT_TYPE_DEFAULT

[Helen Koike]

On 2017-03-30 11:39 PM, Helen Koike wrote:

Hi Laurent,

Thanks for reviewing

On 2017-03-30 04:56 PM, Laurent Pinchart wrote:

Hi Helen,

Thank you for the patch.

On Thursday 30 Mar 2017 13:02:17 Helen Koike wrote:

Add V4L2_INPUT_TYPE_DEFAULT and helpers functions for input ioctls to be
used when no inputs are available in the device

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 27 +++
 include/media/v4l2-ioctl.h   | 26 ++
 include/uapi/linux/videodev2.h   |  1 +
 3 files changed, 54 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c index 0c3f238..ccaf04b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2573,6 +2573,33 @@ struct mutex *v4l2_ioctl_get_lock(struct
video_device
*vdev, unsigned cmd) return vdev->lock;
 }

+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+  struct v4l2_input *i)
+{
+if (i->index > 0)
+return -EINVAL;
+
+memset(i, 0, sizeof(*i));
+i->type = V4L2_INPUT_TYPE_DEFAULT;
+strlcpy(i->name, "Default", sizeof(i->name));
+
+return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_enum_input_default);


V4L2 tends to use EXPORT_SYMBOL_GPL.


The whole v4l2-ioctl.c file is using EXPORT_SYMBOL instead of
EXPORT_SYMBOL_GPL, should we change it all to EXPORT_SYMBOL_GPL then (in
another patch) ?



What would you think about calling those default functions directly
from the
core when the input ioctl handlers are not set ? You wouldn't need to
modify
drivers.


Sure, I'll add them in ops inside __video_register_device when it
validates the ioctls


I just realize I can not simply override struct v4l2_ioctl_ops as it is 
declared as a const inside strut video_device. I'll call those default 
functions only when the ioctls are handled to not modify vdev->ops.







+
+int v4l2_ioctl_g_input_default(struct file *file, void *priv,
unsigned int
*i) +{
+*i = 0;
+return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_g_input_default);
+
+int v4l2_ioctl_s_input_default(struct file *file, void *priv,
unsigned int
i) +{
+return i ? -EINVAL : 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_s_input_default);
+
 /* Common ioctl debug function. This function can be used by
external ioctl messages as well as internal V4L ioctl */
 void v4l_printk_ioctl(const char *prefix, unsigned int cmd)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index 6cd94e5..accc470 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -652,6 +652,32 @@ struct video_device;
  */
 struct mutex *v4l2_ioctl_get_lock(struct video_device *vdev,
unsigned int
cmd);

+
+/**
+ * v4l2_ioctl_enum_input_default - v4l2 ioctl helper for
VIDIOC_ENUM_INPUT
ioctl + *
+ * Plug this function in vidioc_enum_input field of the struct
v4l2_ioctl_ops to + * enumerate a single input as
V4L2_INPUT_TYPE_DEFAULT
+ */
+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+  struct v4l2_input *i);
+
+/**
+ * v4l2_ioctl_g_input_default - v4l2 ioctl helper for VIDIOC_G_INPUT
ioctl
+ *
+ * Plug this function in vidioc_g_input field of the struct
v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_g_input_default(struct file *file, void *priv,
unsigned int
*i); +
+/**
+ * v4l2_ioctl_s_input_default - v4l2 ioctl helper for VIDIOC_S_INPUT
ioctl
+ *
+ * Plug this function in vidioc_s_input field of the struct
v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_s_input_default(struct file *file, void *priv,
unsigned int
i); +
 /* names for fancy debug output */
 extern const char *v4l2_field_names[];
 extern const char *v4l2_type_names[];
diff --git a/include/uapi/linux/videodev2.h
b/include/uapi/linux/videodev2.h
index 316be62..c10bbde 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1477,6 +1477,7 @@ struct v4l2_input {
 };

 /*  Values for the 'type' field */
+#define V4L2_INPUT_TYPE_DEFAULT0
 #define V4L2_INPUT_TYPE_TUNER1
 #define V4L2_INPUT_TYPE_CAMERA2
 #define V4L2_INPUT_TYPE_TOUCH3




Helen


Helen

Re: [PATCH RFC 1/2] [media] v4l2: add V4L2_INPUT_TYPE_DEFAULT

[Helen Koike]

Hi Laurent,

Thanks for reviewing

On 2017-03-30 04:56 PM, Laurent Pinchart wrote:

Hi Helen,

Thank you for the patch.

On Thursday 30 Mar 2017 13:02:17 Helen Koike wrote:

Add V4L2_INPUT_TYPE_DEFAULT and helpers functions for input ioctls to be
used when no inputs are available in the device

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 27 +++
 include/media/v4l2-ioctl.h   | 26 ++
 include/uapi/linux/videodev2.h   |  1 +
 3 files changed, 54 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c index 0c3f238..ccaf04b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2573,6 +2573,33 @@ struct mutex *v4l2_ioctl_get_lock(struct video_device
*vdev, unsigned cmd) return vdev->lock;
 }

+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+ struct v4l2_input *i)
+{
+   if (i->index > 0)
+   return -EINVAL;
+
+   memset(i, 0, sizeof(*i));
+   i->type = V4L2_INPUT_TYPE_DEFAULT;
+   strlcpy(i->name, "Default", sizeof(i->name));
+
+   return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_enum_input_default);


V4L2 tends to use EXPORT_SYMBOL_GPL.


The whole v4l2-ioctl.c file is using EXPORT_SYMBOL instead of 
EXPORT_SYMBOL_GPL, should we change it all to EXPORT_SYMBOL_GPL then (in 
another patch) ?




What would you think about calling those default functions directly from the
core when the input ioctl handlers are not set ? You wouldn't need to modify
drivers.


Sure, I'll add them in ops inside __video_register_device when it 
validates the ioctls





+
+int v4l2_ioctl_g_input_default(struct file *file, void *priv, unsigned int
*i) +{
+   *i = 0;
+   return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_g_input_default);
+
+int v4l2_ioctl_s_input_default(struct file *file, void *priv, unsigned int
i) +{
+   return i ? -EINVAL : 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_s_input_default);
+
 /* Common ioctl debug function. This function can be used by
external ioctl messages as well as internal V4L ioctl */
 void v4l_printk_ioctl(const char *prefix, unsigned int cmd)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index 6cd94e5..accc470 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -652,6 +652,32 @@ struct video_device;
  */
 struct mutex *v4l2_ioctl_get_lock(struct video_device *vdev, unsigned int
cmd);

+
+/**
+ * v4l2_ioctl_enum_input_default - v4l2 ioctl helper for VIDIOC_ENUM_INPUT
ioctl + *
+ * Plug this function in vidioc_enum_input field of the struct
v4l2_ioctl_ops to + * enumerate a single input as V4L2_INPUT_TYPE_DEFAULT
+ */
+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+ struct v4l2_input *i);
+
+/**
+ * v4l2_ioctl_g_input_default - v4l2 ioctl helper for VIDIOC_G_INPUT ioctl
+ *
+ * Plug this function in vidioc_g_input field of the struct v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_g_input_default(struct file *file, void *priv, unsigned int
*i); +
+/**
+ * v4l2_ioctl_s_input_default - v4l2 ioctl helper for VIDIOC_S_INPUT ioctl
+ *
+ * Plug this function in vidioc_s_input field of the struct v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_s_input_default(struct file *file, void *priv, unsigned int
i); +
 /* names for fancy debug output */
 extern const char *v4l2_field_names[];
 extern const char *v4l2_type_names[];
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 316be62..c10bbde 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1477,6 +1477,7 @@ struct v4l2_input {
 };

 /*  Values for the 'type' field */
+#define V4L2_INPUT_TYPE_DEFAULT0
 #define V4L2_INPUT_TYPE_TUNER  1
 #define V4L2_INPUT_TYPE_CAMERA 2
 #define V4L2_INPUT_TYPE_TOUCH  3




Helen

[PATCH 2/2] [media] docs-rst: add V4L2_INPUT_TYPE_DEFAULT

[Helen Koike]

add documentation for V4L2_INPUT_TYPE_DEFAULT

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
 Documentation/media/uapi/v4l/vidioc-enuminput.rst | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/media/uapi/v4l/vidioc-enuminput.rst 
b/Documentation/media/uapi/v4l/vidioc-enuminput.rst
index 17aaaf9..0237e10 100644
--- a/Documentation/media/uapi/v4l/vidioc-enuminput.rst
+++ b/Documentation/media/uapi/v4l/vidioc-enuminput.rst
@@ -112,6 +112,9 @@ at index zero, incrementing by one until the driver returns 
``EINVAL``.
 :stub-columns: 0
 :widths:   3 1 4
 
+* - ``V4L2_INPUT_TYPE_DEFAULT``
+  - 0
+  - This is the default value returned when no input is supported.
 * - ``V4L2_INPUT_TYPE_TUNER``
   - 1
   - This input uses a tuner (RF demodulator).
-- 
2.7.4

[PATCH RFC 1/2] [media] v4l2: add V4L2_INPUT_TYPE_DEFAULT

[Helen Koike]

Add V4L2_INPUT_TYPE_DEFAULT and helpers functions for input ioctls to be
used when no inputs are available in the device

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
 drivers/media/v4l2-core/v4l2-ioctl.c | 27 +++
 include/media/v4l2-ioctl.h   | 26 ++
 include/uapi/linux/videodev2.h   |  1 +
 3 files changed, 54 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c 
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 0c3f238..ccaf04b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -2573,6 +2573,33 @@ struct mutex *v4l2_ioctl_get_lock(struct video_device 
*vdev, unsigned cmd)
return vdev->lock;
 }
 
+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+ struct v4l2_input *i)
+{
+   if (i->index > 0)
+   return -EINVAL;
+
+   memset(i, 0, sizeof(*i));
+   i->type = V4L2_INPUT_TYPE_DEFAULT;
+   strlcpy(i->name, "Default", sizeof(i->name));
+
+   return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_enum_input_default);
+
+int v4l2_ioctl_g_input_default(struct file *file, void *priv, unsigned int *i)
+{
+   *i = 0;
+   return 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_g_input_default);
+
+int v4l2_ioctl_s_input_default(struct file *file, void *priv, unsigned int i)
+{
+   return i ? -EINVAL : 0;
+}
+EXPORT_SYMBOL(v4l2_ioctl_s_input_default);
+
 /* Common ioctl debug function. This function can be used by
external ioctl messages as well as internal V4L ioctl */
 void v4l_printk_ioctl(const char *prefix, unsigned int cmd)
diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h
index 6cd94e5..accc470 100644
--- a/include/media/v4l2-ioctl.h
+++ b/include/media/v4l2-ioctl.h
@@ -652,6 +652,32 @@ struct video_device;
  */
 struct mutex *v4l2_ioctl_get_lock(struct video_device *vdev, unsigned int cmd);
 
+
+/**
+ * v4l2_ioctl_enum_input_default - v4l2 ioctl helper for VIDIOC_ENUM_INPUT 
ioctl
+ *
+ * Plug this function in vidioc_enum_input field of the struct v4l2_ioctl_ops 
to
+ * enumerate a single input as V4L2_INPUT_TYPE_DEFAULT
+ */
+int v4l2_ioctl_enum_input_default(struct file *file, void *priv,
+ struct v4l2_input *i);
+
+/**
+ * v4l2_ioctl_g_input_default - v4l2 ioctl helper for VIDIOC_G_INPUT ioctl
+ *
+ * Plug this function in vidioc_g_input field of the struct v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_g_input_default(struct file *file, void *priv, unsigned int *i);
+
+/**
+ * v4l2_ioctl_s_input_default - v4l2 ioctl helper for VIDIOC_S_INPUT ioctl
+ *
+ * Plug this function in vidioc_s_input field of the struct v4l2_ioctl_ops
+ * when using v4l2_ioctl_enum_input_default
+ */
+int v4l2_ioctl_s_input_default(struct file *file, void *priv, unsigned int i);
+
 /* names for fancy debug output */
 extern const char *v4l2_field_names[];
 extern const char *v4l2_type_names[];
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 316be62..c10bbde 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -1477,6 +1477,7 @@ struct v4l2_input {
 };
 
 /*  Values for the 'type' field */
+#define V4L2_INPUT_TYPE_DEFAULT0
 #define V4L2_INPUT_TYPE_TUNER  1
 #define V4L2_INPUT_TYPE_CAMERA 2
 #define V4L2_INPUT_TYPE_TOUCH  3
-- 
2.7.4

Re: [PATCH v7] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

Hi Sakari,

On 2017-03-26 10:31 AM, Sakari Ailus wrote:

Hi Helen,

...

+static int vimc_cap_enum_input(struct file *file, void *priv,
+  struct v4l2_input *i)
+{
+   /* We only have one input */
+   if (i->index > 0)
+   return -EINVAL;
+
+   i->type = V4L2_INPUT_TYPE_CAMERA;
+   strlcpy(i->name, "VIMC capture", sizeof(i->name));
+
+   return 0;
+}
+
+static int vimc_cap_g_input(struct file *file, void *priv, unsigned int *i)
+{
+   /* We only have one input */
+   *i = 0;
+   return 0;
+}
+
+static int vimc_cap_s_input(struct file *file, void *priv, unsigned int i)
+{
+   /* We only have one input */
+   return i ? -EINVAL : 0;
+}


You can drop the input IOCTLs altogether here. If you had e.g. a TV
tuner, it'd be the TV tuner driver's responsibility to implement them.



input IOCTLs seems to be mandatory from v4l2-compliance when capability 
V4L2_CAP_VIDEO_CAPTURE is set (which is the case):


https://git.linuxtv.org/v4l-utils.git/tree/utils/v4l2-compliance/v4l2-test-input-output.cpp#n418

https://git.linuxtv.org/v4l-utils.git/tree/utils/v4l2-compliance/v4l2-compliance.cpp#n989

Helen

[PATCH v8] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/vimc/devel/v8

Changes since v7:
- remove left over union in struct vimc_ent_device
- remove v4l2_dev pointer from vimc_sen_device structure
- remove unused dev parameter from vimc_propagate_frame()
- remove struct device *dev from struct vimc_cap_device and vimc_sen_device
- in vimc_sen_create: call media_entity_pads_init() after v4l2_subdev_init()
to avoid double initialization of vsen->sd.entity.name
- in vimc_sen_destroy: move media_entity_cleanup to be after 
v4l2_device_unregister_subdev
- rename video_device back to vdev instead of vd
- adjust copyright with range 2015-2017
- add node names in dev_err prints for vimc-capture.c and vimc-sensor.c
- remove prefix "cap" in dev_dbg as it already uses the name of the node

Changes since v6:
- add kernel-docs in vimc-core.h
- reorder list_del call in vimc_cap_return_all_buffers()
- call media_pipeline_stop in vimc_cap_start_streaming when fail
- remove DMA comment (left over from the sample driver)
- remove vb2_set_plane_payload call in vimc_cap_buffer_prepare
- remove format verification in vimc_cap_link_validate
- set vimc_pix_map_list as static
- use MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN in vimc_raw_create()
- register media device after creating the topology and unregister it before 
destroying the topology
- replace devm_kzalloc by kzalloc for allocating struct vimc_device
- uset TASK_UNINTERRUPTIBLE for vimc_thread_sen
- do not allow the creation of a sensor with no pads
- add more verbose description in Kconfig
- change copyright to 2017
- arrange includes: number before letters
- remove v4l2_dev pointer from vimc_cap_device structure
- coding style adjustments
- remove entity variable in vimc_cap_pipeline_s_stream
- declare and assign variables in vimc_cap_link_validate
- declare vimc_dev_release() and vimc_pdev closer to vimc_init()
- remove pad check in vimc_sen_enum_{mbus_code,frame_size} that is already 
performed by v4l2-subdev.c
- fix multiline comments to start with /*
- reorder variable declaration in functions
- remove pad and subdevice type check in vimc_cap_pipeline_s_stream
- add a note that sensor nodes can can have more then one source pad
- remove the use of entity->use_count in the core and attach the ved structure 
in sd or vd, use
  ent->obj_type to know which structure (sd or vd) to use
- rename vdev (video_device) to vd to be similar to sd (subdev)

Changes since v5:
- Fix message "Entity type for entity Sensor A was not initialized!"
  by initializing the sensor entity.function after the calling
  v4l2_subded_init
- populate device_caps in vimc_cap_create instead of in
  vimc_cap_querycap
- Fix typo in vimc-core.c s/de/the

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or v4l2_dev->dev is null
- vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to keep track of entities instead of the old
entity->id
- Replace media_entity_init by media_entity_pads_init
---
 drivers/media/platform/Kconfig |   2 +
 drivers/media/platform/Makefile|   1 +
 drivers/media/platform/vimc/Kconfig|  14 +
 drivers/media/platform/vimc/Makefile   |   3 +
 drivers/media/platform/vimc/vimc-capture.c | 535 ++
 drivers/media/platform/vimc/vimc-capture.h |  28 ++
 drivers/media/platform/vimc/vimc-co

Re: [PATCH v7] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

On 2017-03-25 02:11 PM, Helen Koike wrote:

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/vimc/devel/v7

Changes since v6:
- add kernel-docs in vimc-core.h
- reorder list_del call in vimc_cap_return_all_buffers()
- call media_pipeline_stop in vimc_cap_start_streaming when fail
- remove DMA comment (left over from the sample driver)
- remove vb2_set_plane_payload call in vimc_cap_buffer_prepare
- remove format verification in vimc_cap_link_validate
- set vimc_pix_map_list as static
- use MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN in vimc_raw_create()
- register media device after creating the topology and unregister it
before destroying the topology
- replace devm_kzalloc by kzalloc for allocating struct vimc_device
- uset TASK_UNINTERRUPTIBLE for vimc_thread_sen
- do not allow the creation of a sensor with no pads
- add more verbose description in Kconfig
- change copyright to 2017
- arrange includes: number before letters
- remove v4l2_dev pointer from vimc_cap_device structure
- coding style adjustments
- remove entity variable in vimc_cap_pipeline_s_stream
- declare and assign variables in vimc_cap_link_validate
- declare vimc_dev_release() and vimc_pdev closer to vimc_init()
- remove pad check in vimc_sen_enum_{mbus_code,frame_size} that is
already performed by v4l2-subdev.c
- fix multiline comments to start with /*
- reorder variable declaration in functions
- remove pad and subdevice type check in vimc_cap_pipeline_s_stream
- add a note that sensor nodes can can have more then one source pad
- remove the use of entity->use_count in the core and attach the ved
structure in sd or vd, use
  ent->obj_type to know which structure (sd or vd) to use
- rename vdev (video_device) to vd to be similar to sd (subdev)

Changes since v5:
- Fix message "Entity type for entity Sensor A was not initialized!"
  by initializing the sensor entity.function after the calling
  v4l2_subded_init
- populate device_caps in vimc_cap_create instead of in
  vimc_cap_querycap
- Fix typo in vimc-core.c s/de/the

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or
v4l2_dev->dev is null
- vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to keep track of entities instead of the old
entity->id
- Replace media_entity_init by media_entity_pads_init
---
 drivers/media/platform/Kconfig |   2 +
 drivers/media/platform/Makefile|   1 +
 drivers/media/platform/vimc/Kconfig|  14 +
 drivers/media/platform/vimc/Makefile   |   3 +
 drivers/media/platform/vimc/vimc-capture.c | 536 ++
 drivers/media/platform/vimc/vimc-capture.h |  28 ++
 drivers/media/platform/vimc/vimc-core.c| 696
+
 drivers/media/platform/vimc/vimc-core.h| 123 +
 drivers/media/platform/vimc/vimc-sensor.c  | 279 
 drivers/media/platform/vimc/vimc-sensor.h  |  28 ++
 10 files changed, 1710 insertions(+)
 create mode 100644 drivers/media/platform/vimc/Kconfig
 create mode 100644 drivers/media/platform/vimc/Makefile
 create mode 100644 drivers/media/platform/vimc/vimc-capture.c
 create mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-core.c
 create mode 100644 drivers/media/platform/vimc/vimc-core.h
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.c
 create mode 100

[PATCH v7] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/vimc/devel/v7

Changes since v6:
- add kernel-docs in vimc-core.h
- reorder list_del call in vimc_cap_return_all_buffers()
- call media_pipeline_stop in vimc_cap_start_streaming when fail
- remove DMA comment (left over from the sample driver)
- remove vb2_set_plane_payload call in vimc_cap_buffer_prepare
- remove format verification in vimc_cap_link_validate
- set vimc_pix_map_list as static
- use MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN in vimc_raw_create()
- register media device after creating the topology and unregister it before 
destroying the topology
- replace devm_kzalloc by kzalloc for allocating struct vimc_device
- uset TASK_UNINTERRUPTIBLE for vimc_thread_sen
- do not allow the creation of a sensor with no pads
- add more verbose description in Kconfig
- change copyright to 2017
- arrange includes: number before letters
- remove v4l2_dev pointer from vimc_cap_device structure
- coding style adjustments
- remove entity variable in vimc_cap_pipeline_s_stream
- declare and assign variables in vimc_cap_link_validate
- declare vimc_dev_release() and vimc_pdev closer to vimc_init()
- remove pad check in vimc_sen_enum_{mbus_code,frame_size} that is already 
performed by v4l2-subdev.c
- fix multiline comments to start with /*
- reorder variable declaration in functions
- remove pad and subdevice type check in vimc_cap_pipeline_s_stream
- add a note that sensor nodes can can have more then one source pad
- remove the use of entity->use_count in the core and attach the ved structure 
in sd or vd, use
  ent->obj_type to know which structure (sd or vd) to use
- rename vdev (video_device) to vd to be similar to sd (subdev)

Changes since v5:
- Fix message "Entity type for entity Sensor A was not initialized!"
  by initializing the sensor entity.function after the calling
  v4l2_subded_init
- populate device_caps in vimc_cap_create instead of in
  vimc_cap_querycap
- Fix typo in vimc-core.c s/de/the

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or v4l2_dev->dev is null
- vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to keep track of entities instead of the old
entity->id
- Replace media_entity_init by media_entity_pads_init
---
 drivers/media/platform/Kconfig |   2 +
 drivers/media/platform/Makefile|   1 +
 drivers/media/platform/vimc/Kconfig|  14 +
 drivers/media/platform/vimc/Makefile   |   3 +
 drivers/media/platform/vimc/vimc-capture.c | 536 ++
 drivers/media/platform/vimc/vimc-capture.h |  28 ++
 drivers/media/platform/vimc/vimc-core.c| 696 +
 drivers/media/platform/vimc/vimc-core.h| 123 +
 drivers/media/platform/vimc/vimc-sensor.c  | 279 
 drivers/media/platform/vimc/vimc-sensor.h  |  28 ++
 10 files changed, 1710 insertions(+)
 create mode 100644 drivers/media/platform/vimc/Kconfig
 create mode 100644 drivers/media/platform/vimc/Makefile
 create mode 100644 drivers/media/platform/vimc/vimc-capture.c
 create mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-core.c
 create mode 100644 drivers/media/platform/vimc/vimc-core.h
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.c
 create mode 100644 drivers/media/platform/vimc/vimc-

Re: [PATCH v6] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

Hi Sakari,

Thanks for your review, I have some questions below:

On 2017-01-25 11:03 AM, Sakari Ailus wrote:

Hi Helen,

My apologies for the long review time!

Please see my comments below.

On Sun, Sep 04, 2016 at 05:02:18PM -0300, Helen Koike wrote:

From: Helen Fornazier <helen.fornaz...@gmail.com>

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/vimc/devel/vpu

Changes since v5:
- Fix message "Entity type for entity Sensor A was not initialized!"
  by initializing the sensor entity.function after the calling
  v4l2_subded_init
- populate device_caps in vimc_cap_create instead of in
  vimc_cap_querycap
- Fix typo in vimc-core.c s/de/the

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or v4l2_dev->dev is null
- vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to keep track of entities instead of the old
entity->id
- Replace media_entity_init by media_entity_pads_init
---
 drivers/media/platform/Kconfig |   2 +
 drivers/media/platform/Makefile|   1 +
 drivers/media/platform/vimc/Kconfig|   7 +
 drivers/media/platform/vimc/Makefile   |   3 +
 drivers/media/platform/vimc/vimc-capture.c | 553 ++
 drivers/media/platform/vimc/vimc-capture.h |  28 ++
 drivers/media/platform/vimc/vimc-core.c| 600 +
 drivers/media/platform/vimc/vimc-core.h|  57 +++
 drivers/media/platform/vimc/vimc-sensor.c  | 279 ++
 drivers/media/platform/vimc/vimc-sensor.h  |  28 ++
 10 files changed, 1558 insertions(+)
 create mode 100644 drivers/media/platform/vimc/Kconfig
 create mode 100644 drivers/media/platform/vimc/Makefile
 create mode 100644 drivers/media/platform/vimc/vimc-capture.c
 create mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-core.c
 create mode 100644 drivers/media/platform/vimc/vimc-core.h
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.c
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 46f14dd..4a0577f 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -329,6 +329,8 @@ menuconfig V4L_TEST_DRIVERS

 if V4L_TEST_DRIVERS

+source "drivers/media/platform/vimc/Kconfig"
+
 source "drivers/media/platform/vivid/Kconfig"

 config VIDEO_VIM2M
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 536d1d8..dd4f658 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -12,6 +12,7 @@ obj-$(CONFIG_VIDEO_OMAP3) += omap3isp/

 obj-$(CONFIG_VIDEO_VIU) += fsl-viu.o

+obj-$(CONFIG_VIDEO_VIMC)   += vimc/
 obj-$(CONFIG_VIDEO_VIVID)  += vivid/
 obj-$(CONFIG_VIDEO_VIM2M)  += vim2m.o

diff --git a/drivers/media/platform/vimc/Kconfig 
b/drivers/media/platform/vimc/Kconfig
new file mode 100644
index 000..b48819c
--- /dev/null
+++ b/drivers/media/platform/vimc/Kconfig
@@ -0,0 +1,7 @@
+config VIDEO_VIMC
+   tristate "Virtual Media Controller Driver (VIMC)"
+   depends on VIDEO_DEV && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+   select VIDEOBUF2_VMALLOC
+   default n
+   ---help---
+ Skeleton driver for Virtual Me

Re: [PATCH v6] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

Hi Hans,

On 2017-03-10 10:08 AM, Hans Verkuil wrote:

Hi Helen,

On 11/01/17 02:30, Helen Koike wrote:
>
> Thank you for the review, I'll update the patch accordingly and re-submit it.
>
> Helen

Do you know when you have a v7 ready?


Thanks for your interest. I don't have the next version entirely ready yet but 
I'll work on it to send this next version asap and the other patches I have on 
top of it as well.



We really need a vimc driver so people without hardware can actually fiddle 
around
with the MC.

See also my rant here (not directed at you!):

https://www.spinics.net/lists/kernel/msg2462009.html

Regards,

Hans


Helen

Re: [PATCH v6] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

Hi Laurent,

On 2017-01-10 04:54 PM, Laurent Pinchart wrote:

Hi Helen,

(CC'ing Sakari as there's a question specifically for him)

Thank you for the patch, and so sorry for the late review.

On Sunday 04 Sep 2016 17:02:18 Helen Koike wrote:

From: Helen Fornazier <helen.fornaz...@gmail.com>

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>


I've reviewed the whole patch but haven't had time to test it. I've also
skipped the items marked as TODO or FIXME as they're obviously not ready yet
:-) Overall this looks good to me, all the issues are minor.


---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/vimc/devel/vpu

Changes since v5:
- Fix message "Entity type for entity Sensor A was not initialized!"
  by initializing the sensor entity.function after the calling
  v4l2_subded_init
- populate device_caps in vimc_cap_create instead of in
  vimc_cap_querycap
- Fix typo in vimc-core.c s/de/the

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or v4l2_dev->dev is
null - vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to keep track of entities instead of the old
entity->id
- Replace media_entity_init by media_entity_pads_init
---
 drivers/media/platform/Kconfig |   2 +
 drivers/media/platform/Makefile|   1 +
 drivers/media/platform/vimc/Kconfig|   7 +
 drivers/media/platform/vimc/Makefile   |   3 +
 drivers/media/platform/vimc/vimc-capture.c | 553 ++
 drivers/media/platform/vimc/vimc-capture.h |  28 ++
 drivers/media/platform/vimc/vimc-core.c| 600 ++
 drivers/media/platform/vimc/vimc-core.h|  57 +++
 drivers/media/platform/vimc/vimc-sensor.c  | 279 ++
 drivers/media/platform/vimc/vimc-sensor.h  |  28 ++
 10 files changed, 1558 insertions(+)
 create mode 100644 drivers/media/platform/vimc/Kconfig
 create mode 100644 drivers/media/platform/vimc/Makefile
 create mode 100644 drivers/media/platform/vimc/vimc-capture.c
 create mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-core.c
 create mode 100644 drivers/media/platform/vimc/vimc-core.h
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.c
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.h


[snip]


diff --git a/drivers/media/platform/vimc/vimc-capture.c
b/drivers/media/platform/vimc/vimc-capture.c new file mode 100644
index 000..b7636cf
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-capture.c


[snip]


+static void vimc_cap_return_all_buffers(struct vimc_cap_device *vcap,
+   enum vb2_buffer_state state)
+{
+   struct vimc_cap_buffer *vbuf, *node;
+
+   spin_lock(>qlock);
+
+   list_for_each_entry_safe(vbuf, node, >buf_list, list) {
+   vb2_buffer_done(>vb2.vb2_buf, state);
+   list_del(>list);


It shouldn't matter given that you protect this with a spinlock, but moving
the list_del() above makes the code flow follow a safer order.


+   }
+
+   spin_unlock(>qlock);
+}
+
+static int vimc_cap_pipeline_s_stream(struct vimc_cap_device *vcap, int
enable)
+{
+   int ret;
+   struct media_pad *pad;
+   struct media_entity *entity;
+   struct v4l2_subdev *sd;
+
+   /* Start the stream in the subdevice direct connected */
+   entity = >vdev.entity;
+   

[PATCH] [media] MAINTAINERS: add vimc entry

[Helen Koike]

Signed-off-by: Helen Koike <helen.ko...@collabora.com>
---
 MAINTAINERS | 8 
 1 file changed, 8 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 0a16a82..43e0eb4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -12540,6 +12540,14 @@ W: https://linuxtv.org
 S: Maintained
 F: drivers/media/platform/vivid/*
 
+VIMC VIRTUAL MEDIA CONTROLLER DRIVER
+M: Helen Koike <helen.ko...@collabora.com>
+L: linux-media@vger.kernel.org
+T: git git://linuxtv.org/media_tree.git
+W: https://linuxtv.org
+S: Maintained
+F: drivers/media/platform/vimc/*
+
 VLAN (802.1Q)
 M: Patrick McHardy <ka...@trash.net>
 L: net...@vger.kernel.org
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v5] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

Hi Hans,

Thank you for your review.

On 2016-08-22 07:57 AM, Hans Verkuil wrote:

Hi Helen,

A few small code comments are below.

Note that if I try to capture I see these two messages in the kernel log:

[588197.368145] vimc vimc.0: Entity type for entity Sensor A was not 
initialized!
[588197.368169] vimc vimc.0: Entity type for entity Sensor B was not 
initialized!



I correct this, I am sending it in v6.




I also can't capture anything: v4l2-ctl --stream-mmap just sits there, waiting 
for
frames, I guess.

I'm not sure if that has to do with the two warnings above.



This is weird, v4l2-ctl --stream-mmap works for me even with those 
messages above, could you try again with the v6 please?





I am assuming that the initial pipeline is correct and that you should be able
to start streaming. If not, then attempting to start streaming should return an
error.

On 08/18/2016 12:09 AM, Helen Koike wrote:

From: Helen Fornazier <helen.fornaz...@gmail.com>

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>





+static int vimc_cap_querycap(struct file *file, void *priv,
+struct v4l2_capability *cap)
+{
+   struct vimc_cap_device *vcap = video_drvdata(file);
+
+   strlcpy(cap->driver, KBUILD_MODNAME, sizeof(cap->driver));
+   strlcpy(cap->card, KBUILD_MODNAME, sizeof(cap->card));
+   snprintf(cap->bus_info, sizeof(cap->bus_info),
+"platform:%s", vcap->v4l2_dev->name);
+
+   cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;


This line should be moved to vimc_cap_create:

vdev->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;

This is new. The v4l2 core will fill in the querycap capabilities for you
based on vdev->device_caps.


+
+   return 0;
+}





+static int vimc_device_register(struct vimc_device *vimc)
+{
+   unsigned int i;
+   int ret = 0;
+
+   /* Allocate memory for the vimc_ent_devices pointers */
+   vimc->ved = devm_kcalloc(vimc->mdev.dev, vimc->pipe_cfg->num_ents,
+sizeof(*vimc->ved), GFP_KERNEL);
+   if (!vimc->ved)
+   return -ENOMEM;
+
+   /* Register the media device */
+   ret = media_device_register(>mdev);
+   if (ret) {
+   dev_err(vimc->mdev.dev,
+   "media device register failed (err=%d)\n", ret);
+   return ret;
+   }
+
+   /* Link the media device within the v4l2_device */
+   vimc->v4l2_dev.mdev = >mdev;
+
+   /* Register the v4l2 struct */
+   ret = v4l2_device_register(vimc->mdev.dev, >v4l2_dev);
+   if (ret) {
+   dev_err(vimc->mdev.dev,
+   "v4l2 device register failed (err=%d)\n", ret);
+   return ret;
+   }
+
+   /* Initialize entities */
+   for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
+   struct vimc_ent_device *(*create_func)(struct v4l2_device *,
+  const char *const,
+  u16,
+  const unsigned long *);
+
+   /* Register the specific node */
+   switch (vimc->pipe_cfg->ents[i].node) {
+   case VIMC_ENT_NODE_SENSOR:
+   create_func = vimc_sen_create;
+   break;
+
+   case VIMC_ENT_NODE_CAPTURE:
+   create_func = vimc_cap_create;
+   break;
+
+   /* TODO: Instantiate the specific topology node */
+   case VIMC_ENT_NODE_INPUT:
+   case VIMC_ENT_NODE_DEBAYER:
+   case VIMC_ENT_NODE_SCALER:
+   default:
+   /* TODO: remove this when all the entities specific
+* code are implemented
+*/
+   create_func = vimc_raw_create;
+   break;
+   }
+
+   vimc->ved[i] = create_func(>v4l2_dev,
+  vimc->pipe_cfg->ents[i].name,
+  vimc->pipe_cfg->ents[i].pads_qty,
+  vimc->pipe_cfg->ents[i].pads_flag);
+   if (IS_ERR(vimc->ved[i])) {
+   ret = PTR_ERR(vimc->ved[i]);
+   vimc->ved[i] = NULL;
+   goto err;
+   }
+
+   /* Set use_count to keep track of the ved structure */
+   vimc->ved[i]->ent->use_count = i;
+   }
+
+   

[PATCH v6] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

From: Helen Fornazier <helen.fornaz...@gmail.com>

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/vimc/devel/vpu

Changes since v5:
- Fix message "Entity type for entity Sensor A was not initialized!"
  by initializing the sensor entity.function after the calling
  v4l2_subded_init
- populate device_caps in vimc_cap_create instead of in
  vimc_cap_querycap
- Fix typo in vimc-core.c s/de/the

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or v4l2_dev->dev is null
- vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to keep track of entities instead of the old
entity->id
- Replace media_entity_init by media_entity_pads_init
---
 drivers/media/platform/Kconfig |   2 +
 drivers/media/platform/Makefile|   1 +
 drivers/media/platform/vimc/Kconfig|   7 +
 drivers/media/platform/vimc/Makefile   |   3 +
 drivers/media/platform/vimc/vimc-capture.c | 553 ++
 drivers/media/platform/vimc/vimc-capture.h |  28 ++
 drivers/media/platform/vimc/vimc-core.c| 600 +
 drivers/media/platform/vimc/vimc-core.h|  57 +++
 drivers/media/platform/vimc/vimc-sensor.c  | 279 ++
 drivers/media/platform/vimc/vimc-sensor.h  |  28 ++
 10 files changed, 1558 insertions(+)
 create mode 100644 drivers/media/platform/vimc/Kconfig
 create mode 100644 drivers/media/platform/vimc/Makefile
 create mode 100644 drivers/media/platform/vimc/vimc-capture.c
 create mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-core.c
 create mode 100644 drivers/media/platform/vimc/vimc-core.h
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.c
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 46f14dd..4a0577f 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -329,6 +329,8 @@ menuconfig V4L_TEST_DRIVERS
 
 if V4L_TEST_DRIVERS
 
+source "drivers/media/platform/vimc/Kconfig"
+
 source "drivers/media/platform/vivid/Kconfig"
 
 config VIDEO_VIM2M
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 536d1d8..dd4f658 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -12,6 +12,7 @@ obj-$(CONFIG_VIDEO_OMAP3) += omap3isp/
 
 obj-$(CONFIG_VIDEO_VIU) += fsl-viu.o
 
+obj-$(CONFIG_VIDEO_VIMC)   += vimc/
 obj-$(CONFIG_VIDEO_VIVID)  += vivid/
 obj-$(CONFIG_VIDEO_VIM2M)  += vim2m.o
 
diff --git a/drivers/media/platform/vimc/Kconfig 
b/drivers/media/platform/vimc/Kconfig
new file mode 100644
index 000..b48819c
--- /dev/null
+++ b/drivers/media/platform/vimc/Kconfig
@@ -0,0 +1,7 @@
+config VIDEO_VIMC
+   tristate "Virtual Media Controller Driver (VIMC)"
+   depends on VIDEO_DEV && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+   select VIDEOBUF2_VMALLOC
+   default n
+   ---help---
+ Skeleton driver for Virtual Media Controller
diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
new file mode 100644
index 000..c45195e
--- /dev/null
+++ b/drivers/media/platform/vimc/Makefile
@@ -0,0 +1,3 @@
+vimc-objs := vimc-core.o vi

[PATCH v5] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

From: Helen Fornazier <helen.fornaz...@gmail.com>

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Koike <helen.ko...@collabora.com>

---

Patch based in media/master tree, and available here:
https://github.com/helen-fornazier/opw-staging/tree/vimc/devel/vpu

Changes since v4:
- coding style fixes
- remove BUG_ON
- change copyright to 2016
- depens on VIDEO_V4L2_SUBDEV_API instead of select
- remove assignement of V4L2_CAP_DEVICE_CAPS
- s/vimc_cap_g_fmt_vid_cap/vimc_cap_fmt_vid_cap
- fix vimc_cap_queue_setup declaration type
- remove wrong buffer size check and add it in vimc_cap_buffer_prepare
- vimc_cap_create: remove unecessary check if v4l2_dev or v4l2_dev->dev is null
- vimc_cap_create: only allow a single pad
- vimc_sen_create: only allow source pads, remove unecessary source pads
checks in vimc_thread_sen

Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to keep track of entities instead of the old
entity->id
- Replace media_entity_init by media_entity_pads_init
---
 drivers/media/platform/Kconfig |   2 +
 drivers/media/platform/Makefile|   1 +
 drivers/media/platform/vimc/Kconfig|   7 +
 drivers/media/platform/vimc/Makefile   |   3 +
 drivers/media/platform/vimc/vimc-capture.c | 554 ++
 drivers/media/platform/vimc/vimc-capture.h |  28 ++
 drivers/media/platform/vimc/vimc-core.c| 600 +
 drivers/media/platform/vimc/vimc-core.h|  57 +++
 drivers/media/platform/vimc/vimc-sensor.c  | 279 ++
 drivers/media/platform/vimc/vimc-sensor.h  |  28 ++
 10 files changed, 1559 insertions(+)
 create mode 100644 drivers/media/platform/vimc/Kconfig
 create mode 100644 drivers/media/platform/vimc/Makefile
 create mode 100644 drivers/media/platform/vimc/vimc-capture.c
 create mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-core.c
 create mode 100644 drivers/media/platform/vimc/vimc-core.h
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.c
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index f25344b..ead66b2 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -329,6 +329,8 @@ menuconfig V4L_TEST_DRIVERS
 
 if V4L_TEST_DRIVERS
 
+source "drivers/media/platform/vimc/Kconfig"
+
 source "drivers/media/platform/vivid/Kconfig"
 
 config VIDEO_VIM2M
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 21771c1..fd0cd83 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -12,6 +12,7 @@ obj-$(CONFIG_VIDEO_OMAP3) += omap3isp/
 
 obj-$(CONFIG_VIDEO_VIU) += fsl-viu.o
 
+obj-$(CONFIG_VIDEO_VIMC)   += vimc/
 obj-$(CONFIG_VIDEO_VIVID)  += vivid/
 obj-$(CONFIG_VIDEO_VIM2M)  += vim2m.o
 
diff --git a/drivers/media/platform/vimc/Kconfig 
b/drivers/media/platform/vimc/Kconfig
new file mode 100644
index 000..b48819c
--- /dev/null
+++ b/drivers/media/platform/vimc/Kconfig
@@ -0,0 +1,7 @@
+config VIDEO_VIMC
+   tristate "Virtual Media Controller Driver (VIMC)"
+   depends on VIDEO_DEV && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
+   select VIDEOBUF2_VMALLOC
+   default n
+   ---help---
+ Skeleton driver for Virtual Media Controller
diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
new file mode 100644
index 000..c45195e
--- /dev/null
+++ b/drivers/media/platform/vimc/Makefile
@@ -0,0 +1,3 @@
+vimc-objs := vimc-core.o vimc-capture.o vimc-sensor.o
+
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o
diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
new file mode 100644
index 000..dfc031d
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -0,0 +1,554 @@

Re: [PATCH v4] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

Hi Hans

On 01-07-2016 09:39, Hans Verkuil wrote:

Hi Helen,

Better late than never, but I finally have time for a review, mostly with a eye 
for V4L2 issues.


Thank you for your review, I'll incorporate your suggestions in v5.
I am also preparing a patch series that integrates the tpg and have much 
more functionality, I want to add the optimization where I analyze the 
pipe and generate the final image directly from the video capture node 
instead of processing all the images in each processing unit. I hope 
that it will be useful.


Helen
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v4] [media] vimc: Virtual Media Controller core, capture and sensor

[Helen Koike]

From: Helen Fornazier 

First version of the Virtual Media Controller.
Add a simple version of the core of the driver, the capture and
sensor nodes in the topology, generating a grey image in a hardcoded
format.

Signed-off-by: Helen Fornazier 

---
Changes since v3: fix rmmod crash and built-in compile
- Re-order unregister calls in vimc_device_unregister function (remove
rmmod issue)
- Call media_device_unregister_entity in vimc_raw_destroy
- Add depends on VIDEO_DEV && VIDEO_V4L2 and select VIDEOBUF2_VMALLOC
- Check *nplanes in queue_setup (this remove v4l2-compliance fail)
- Include  in vimc-sensor.c
- Move include of  from vimc-core.c to vimc-core.h
- Generate 60 frames per sec instead of 1 in the sensor

Changes since v2: update with current media master tree
- Add struct media_pipeline in vimc_cap_device
- Use vb2_v4l2_buffer instead of vb2_buffer
- Typos
- Remove usage of entity->type and use entity->function instead
- Remove fmt argument from queue setup
- Use ktime_get_ns instead of v4l2_get_timestamp
- Iterate over link's list using list_for_each_entry
- Use media_device_{init, cleanup}
- Use entity->use_count to keep track of entities instead of the old
entity->id
- Replace media_entity_init by media_entity_pads_init

 drivers/media/platform/Kconfig |   2 +
 drivers/media/platform/Makefile|   1 +
 drivers/media/platform/vimc/Kconfig|   8 +
 drivers/media/platform/vimc/Makefile   |   3 +
 drivers/media/platform/vimc/vimc-capture.c | 536 ++
 drivers/media/platform/vimc/vimc-capture.h |  28 ++
 drivers/media/platform/vimc/vimc-core.c| 595 +
 drivers/media/platform/vimc/vimc-core.h|  56 +++
 drivers/media/platform/vimc/vimc-sensor.c  | 278 ++
 drivers/media/platform/vimc/vimc-sensor.h  |  28 ++
 10 files changed, 1535 insertions(+)
 create mode 100644 drivers/media/platform/vimc/Kconfig
 create mode 100644 drivers/media/platform/vimc/Makefile
 create mode 100644 drivers/media/platform/vimc/vimc-capture.c
 create mode 100644 drivers/media/platform/vimc/vimc-capture.h
 create mode 100644 drivers/media/platform/vimc/vimc-core.c
 create mode 100644 drivers/media/platform/vimc/vimc-core.h
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.c
 create mode 100644 drivers/media/platform/vimc/vimc-sensor.h

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index 84e041c..9e10d49 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -284,6 +284,8 @@ menuconfig V4L_TEST_DRIVERS
 
 if V4L_TEST_DRIVERS
 
+source "drivers/media/platform/vimc/Kconfig"
+
 source "drivers/media/platform/vivid/Kconfig"
 
 config VIDEO_VIM2M
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index bbb7bd1..e4508fe 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -12,6 +12,7 @@ obj-$(CONFIG_VIDEO_OMAP3) += omap3isp/
 
 obj-$(CONFIG_VIDEO_VIU) += fsl-viu.o
 
+obj-$(CONFIG_VIDEO_VIMC)   += vimc/
 obj-$(CONFIG_VIDEO_VIVID)  += vivid/
 obj-$(CONFIG_VIDEO_VIM2M)  += vim2m.o
 
diff --git a/drivers/media/platform/vimc/Kconfig 
b/drivers/media/platform/vimc/Kconfig
new file mode 100644
index 000..c99666a
--- /dev/null
+++ b/drivers/media/platform/vimc/Kconfig
@@ -0,0 +1,8 @@
+config VIDEO_VIMC
+   tristate "Virtual Media Controller Driver (VIMC)"
+   depends on VIDEO_DEV && VIDEO_V4L2
+   select VIDEO_V4L2_SUBDEV_API
+   select VIDEOBUF2_VMALLOC
+   default n
+   ---help---
+ Skeleton driver for Virtual Media Controller
diff --git a/drivers/media/platform/vimc/Makefile 
b/drivers/media/platform/vimc/Makefile
new file mode 100644
index 000..c45195e
--- /dev/null
+++ b/drivers/media/platform/vimc/Makefile
@@ -0,0 +1,3 @@
+vimc-objs := vimc-core.o vimc-capture.o vimc-sensor.o
+
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o
diff --git a/drivers/media/platform/vimc/vimc-capture.c 
b/drivers/media/platform/vimc/vimc-capture.c
new file mode 100644
index 000..82de8b89
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -0,0 +1,536 @@
+/*
+ * vimc-capture.c Virtual Media Controller Driver
+ *
+ * Copyright (C) 2015 Helen Fornazier 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include 
+#include 
+#include 
+
+#include "vimc-capture.h"
+
+struct vimc_cap_device {
+   struct