cron job: media_tree daily build: ERRORS

[Hans Verkuil]

This message is generated daily by a cron job that builds media_tree for
the kernels and architectures in the list below.

Results of the daily build of media_tree:

date:   Thu May 25 05:00:22 CEST 2017
media-tree git hash:36bcba973ad478042d1ffc6e89afd92e8bd17030
media_build git hash:   c8dfc17d6d049d79497c78737625f6ea3b08c456
v4l-utils git hash: d16a17abd1d8d7885ca2f44fb295035278baa89c
gcc version:i686-linux-gcc (GCC) 7.1.0
sparse version: v0.5.0-3553-g78b2ea6
smatch version: v0.5.0-3553-g78b2ea6
host hardware:  x86_64
host os:4.9.0-164

linux-git-arm-at91: WARNINGS
linux-git-arm-davinci: WARNINGS
linux-git-arm-multi: WARNINGS
linux-git-arm-pxa: OK
linux-git-blackfin-bf561: OK
linux-git-i686: OK
linux-git-m32r: OK
linux-git-mips: OK
linux-git-powerpc64: OK
linux-git-sh: OK
linux-git-x86_64: WARNINGS
linux-2.6.36.4-i686: ERRORS
linux-2.6.37.6-i686: ERRORS
linux-2.6.38.8-i686: ERRORS
linux-2.6.39.4-i686: ERRORS
linux-3.0.60-i686: ERRORS
linux-3.1.10-i686: ERRORS
linux-3.2.37-i686: ERRORS
linux-3.3.8-i686: ERRORS
linux-3.4.27-i686: ERRORS
linux-3.5.7-i686: ERRORS
linux-3.6.11-i686: ERRORS
linux-3.7.4-i686: ERRORS
linux-3.8-i686: ERRORS
linux-3.9.2-i686: ERRORS
linux-3.10.1-i686: ERRORS
linux-3.11.1-i686: ERRORS
linux-3.12.67-i686: ERRORS
linux-3.13.11-i686: ERRORS
linux-3.14.9-i686: ERRORS
linux-3.15.2-i686: ERRORS
linux-3.16.7-i686: ERRORS
linux-3.17.8-i686: ERRORS
linux-3.18.7-i686: ERRORS
linux-3.19-i686: ERRORS
linux-4.0.9-i686: ERRORS
linux-4.1.33-i686: ERRORS
linux-4.2.8-i686: ERRORS
linux-4.3.6-i686: ERRORS
linux-4.4.22-i686: ERRORS
linux-4.5.7-i686: ERRORS
linux-4.6.7-i686: ERRORS
linux-4.7.5-i686: ERRORS
linux-4.8-i686: ERRORS
linux-4.9.26-i686: ERRORS
linux-4.10.14-i686: ERRORS
linux-4.11-i686: ERRORS
linux-4.12-rc1-i686: OK
linux-2.6.36.4-x86_64: ERRORS
linux-2.6.37.6-x86_64: ERRORS
linux-2.6.38.8-x86_64: ERRORS
linux-2.6.39.4-x86_64: ERRORS
linux-3.0.60-x86_64: ERRORS
linux-3.1.10-x86_64: ERRORS
linux-3.2.37-x86_64: ERRORS
linux-3.3.8-x86_64: ERRORS
linux-3.4.27-x86_64: ERRORS
linux-3.5.7-x86_64: ERRORS
linux-3.6.11-x86_64: ERRORS
linux-3.7.4-x86_64: ERRORS
linux-3.8-x86_64: ERRORS
linux-3.9.2-x86_64: ERRORS
linux-3.10.1-x86_64: ERRORS
linux-3.11.1-x86_64: ERRORS
linux-3.12.67-x86_64: ERRORS
linux-3.13.11-x86_64: ERRORS
linux-3.14.9-x86_64: ERRORS
linux-3.15.2-x86_64: ERRORS
linux-3.16.7-x86_64: ERRORS
linux-3.17.8-x86_64: ERRORS
linux-3.18.7-x86_64: ERRORS
linux-3.19-x86_64: ERRORS
linux-4.0.9-x86_64: ERRORS
linux-4.1.33-x86_64: ERRORS
linux-4.2.8-x86_64: ERRORS
linux-4.3.6-x86_64: ERRORS
linux-4.4.22-x86_64: ERRORS
linux-4.5.7-x86_64: ERRORS
linux-4.6.7-x86_64: ERRORS
linux-4.7.5-x86_64: ERRORS
linux-4.8-x86_64: ERRORS
linux-4.9.26-x86_64: ERRORS
linux-4.10.14-x86_64: ERRORS
linux-4.11-x86_64: ERRORS
linux-4.12-rc1-x86_64: WARNINGS
apps: WARNINGS
spec-git: OK
sparse: WARNINGS

Detailed results are available here:

http://www.xs4all.nl/~hverkuil/logs/Thursday.log

Full logs are available here:

http://www.xs4all.nl/~hverkuil/logs/Thursday.tar.bz2

The Media Infrastructure API from this daily build is here:

http://www.xs4all.nl/~hverkuil/spec/index.html

[PATCH v7 01/34] dt-bindings: Add bindings for video-multiplexer device

[Steve Longerbeam]

From: Philipp Zabel 

Add bindings documentation for the video multiplexer device.

Signed-off-by: Sascha Hauer 
Signed-off-by: Philipp Zabel 
Signed-off-by: Steve Longerbeam 
Acked-by: Sakari Ailus 
Reviewed-by: Sebastian Reichel 
Acked-by: Rob Herring 
---
 .../devicetree/bindings/media/video-mux.txt| 60 ++
 1 file changed, 60 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/video-mux.txt

diff --git a/Documentation/devicetree/bindings/media/video-mux.txt 
b/Documentation/devicetree/bindings/media/video-mux.txt
new file mode 100644
index 000..63b9dc9
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/video-mux.txt
@@ -0,0 +1,60 @@
+Video Multiplexer
+=
+
+Video multiplexers allow to select between multiple input ports. Video received
+on the active input port is passed through to the output port. Muxes described
+by this binding are controlled by a multiplexer controller that is described by
+the bindings in Documentation/devicetree/bindings/mux/mux-controller.txt
+
+Required properties:
+- compatible : should be "video-mux"
+- mux-controls : mux controller node to use for operating the mux
+- #address-cells: should be <1>
+- #size-cells: should be <0>
+- port@*: at least three port nodes containing endpoints connecting to the
+  source and sink devices according to of_graph bindings. The last port is
+  the output port, all others are inputs.
+
+Optionally, #address-cells, #size-cells, and port nodes can be grouped under a
+ports node as described in Documentation/devicetree/bindings/graph.txt.
+
+Example:
+
+   mux: mux-controller {
+   compatible = "gpio-mux";
+   #mux-control-cells = <0>;
+
+   mux-gpios = < 15 GPIO_ACTIVE_HIGH>;
+   };
+
+   video-mux {
+   compatible = "video-mux";
+   mux-controls = <>;
+   #address-cells = <1>;
+   #size-cells = <0>;
+
+   port@0 {
+   reg = <0>;
+
+   mux_in0: endpoint {
+   remote-endpoint = <_source0_out>;
+   };
+   };
+
+   port@1 {
+   reg = <1>;
+
+   mux_in1: endpoint {
+   remote-endpoint = <_source1_out>;
+   };
+   };
+
+   port@2 {
+   reg = <2>;
+
+   mux_out: endpoint {
+   remote-endpoint = <_interface_in>;
+   };
+   };
+   };
+};
-- 
2.7.4

[PATCH v7 10/34] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors

[Steve Longerbeam]

Enables the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.

The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.

The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi.

Until the OV5652 sensor module compatible with the SabreSD becomes
available for testing, the ov5642 node is currently disabled.

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6dl-sabresd.dts   |   5 ++
 arch/arm/boot/dts/imx6q-sabresd.dts|   5 ++
 arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 -
 3 files changed, 123 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/imx6dl-sabresd.dts 
b/arch/arm/boot/dts/imx6dl-sabresd.dts
index 1e45f2f..9607afe 100644
--- a/arch/arm/boot/dts/imx6dl-sabresd.dts
+++ b/arch/arm/boot/dts/imx6dl-sabresd.dts
@@ -15,3 +15,8 @@
model = "Freescale i.MX6 DualLite SABRE Smart Device Board";
compatible = "fsl,imx6dl-sabresd", "fsl,imx6dl";
 };
+
+_csi1_from_ipu1_csi1_mux {
+   clock-lanes = <0>;
+   data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6q-sabresd.dts 
b/arch/arm/boot/dts/imx6q-sabresd.dts
index 9cbdfe7..527772b 100644
--- a/arch/arm/boot/dts/imx6q-sabresd.dts
+++ b/arch/arm/boot/dts/imx6q-sabresd.dts
@@ -23,3 +23,8 @@
  {
status = "okay";
 };
+
+_csi1_from_mipi_vc1 {
+   clock-lanes = <0>;
+   data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi 
b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
index 58055ce..342550d 100644
--- a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
@@ -10,6 +10,7 @@
  * http://www.gnu.org/copyleft/gpl.html
  */
 
+#include 
 #include 
 #include 
 
@@ -146,6 +147,36 @@
};
 };
 
+_csi0_from_ipu1_csi0_mux {
+   bus-width = <8>;
+   data-shift = <12>; /* Lines 19:12 used */
+   hsync-active = <1>;
+   vsync-active = <1>;
+};
+
+_csi0_mux_from_parallel_sensor {
+   remote-endpoint = <_to_ipu1_csi0_mux>;
+};
+
+_csi0 {
+   pinctrl-names = "default";
+   pinctrl-0 = <_ipu1_csi0>;
+};
+
+_csi {
+   status = "okay";
+
+   port@0 {
+   reg = <0>;
+
+   mipi_csi2_in: endpoint {
+   remote-endpoint = <_to_mipi_csi2>;
+   clock-lanes = <0>;
+   data-lanes = <1 2>;
+   };
+   };
+};
+
  {
pinctrl-names = "default";
pinctrl-0 = <_audmux>;
@@ -213,7 +244,32 @@
0x8014 /* 4:FN_DMICCDAT */
0x /* 5:Default */
>;
-   };
+   };
+
+   ov5642: camera@3c {
+   compatible = "ovti,ov5642";
+   pinctrl-names = "default";
+   pinctrl-0 = <_ov5642>;
+   clocks = < IMX6QDL_CLK_CKO>;
+   clock-names = "xclk";
+   reg = <0x3c>;
+   DOVDD-supply = <_reg>; /* 1.8v */
+   AVDD-supply = <_reg>;  /* 2.8v, rev C board is VGEN3
+   rev B board is VGEN5 */
+   DVDD-supply = <_reg>;  /* 1.5v*/
+   powerdown-gpios = < 16 GPIO_ACTIVE_HIGH>;
+   reset-gpios = < 17 GPIO_ACTIVE_LOW>;
+   status = "disabled";
+
+   port {
+   ov5642_to_ipu1_csi0_mux: endpoint {
+   remote-endpoint = 
<_csi0_mux_from_parallel_sensor>;
+   bus-width = <8>;
+   hsync-active = <1>;
+   vsync-active = <1>;
+   };
+   };
+   };
 };
 
  {
@@ -222,6 +278,32 @@
pinctrl-0 = <_i2c2>;
status = "okay";
 
+   ov5640: camera@3c {
+   compatible = "ovti,ov5640";
+   pinctrl-names = "default";
+   pinctrl-0 = <_ov5640>;
+   reg = <0x3c>;
+   clocks = < IMX6QDL_CLK_CKO>;
+   clock-names = "xclk";
+   DOVDD-supply = <_reg>; /* 1.8v */
+   AVDD-supply = <_reg>;  /* 2.8v, rev C board is VGEN3
+   rev B board is VGEN5 */
+   DVDD-supply = <_reg>;  /* 1.5v*/
+   powerdown-gpios = < 19 GPIO_ACTIVE_HIGH>;
+   reset-gpios = < 20 GPIO_ACTIVE_LOW>;
+
+   port {
+   #address-cells = <1>;
+   #size-cells = <0>;
+
+   ov5640_to_mipi_csi2: endpoint {
+   remote-endpoint = <_csi2_in>;
+   clock-lanes = <0>;
+   data-lanes = <1 2>;
+   };
+   };
+   };
+
pmic: pfuze100@08 {
compatible = "fsl,pfuze100";
reg = <0x08>;
@@ -425,6 +507,36 @@
>;
};
 
+   

[PATCH v7 11/34] ARM: dts: imx6-sabreauto: create i2cmux for i2c3

[Steve Longerbeam]

The sabreauto uses a steering pin to select between the SDA signal on
i2c3 bus, and a data-in pin for an SPI NOR chip. Use i2cmux to control
this steering pin. Idle state of the i2cmux selects SPI NOR. This is not
a classic way to use i2cmux, since one side of the mux selects something
other than an i2c bus, but it works and is probably the cleanest
solution. Note that if one thread is attempting to access SPI NOR while
another thread is accessing i2c3, the SPI NOR access will fail since the
i2cmux has selected the SDA pin rather than SPI NOR data-in. This couldn't
be avoided in any case, the board is not designed to allow concurrent
i2c3 and SPI NOR functions (and the default device-tree does not enable
SPI NOR anyway).

Devices hanging off i2c3 should now be defined under i2cmux, so
that the steering pin can be properly controlled to access those
devices. The port expanders (MAX7310) are thus moved into i2cmux.

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 65 +---
 1 file changed, 44 insertions(+), 21 deletions(-)

diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi 
b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index a2a714d..c8e35c4 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -108,6 +108,44 @@
default-brightness-level = <7>;
status = "okay";
};
+
+   i2cmux {
+   compatible = "i2c-mux-gpio";
+   #address-cells = <1>;
+   #size-cells = <0>;
+   pinctrl-names = "default";
+   pinctrl-0 = <_i2c3mux>;
+   mux-gpios = < 4 0>;
+   i2c-parent = <>;
+   idle-state = <0>;
+
+   i2c@1 {
+   #address-cells = <1>;
+   #size-cells = <0>;
+   reg = <1>;
+
+   max7310_a: gpio@30 {
+   compatible = "maxim,max7310";
+   reg = <0x30>;
+   gpio-controller;
+   #gpio-cells = <2>;
+   };
+
+   max7310_b: gpio@32 {
+   compatible = "maxim,max7310";
+   reg = <0x32>;
+   gpio-controller;
+   #gpio-cells = <2>;
+   };
+
+   max7310_c: gpio@34 {
+   compatible = "maxim,max7310";
+   reg = <0x34>;
+   gpio-controller;
+   #gpio-cells = <2>;
+   };
+   };
+   };
 };
 
  {
@@ -290,27 +328,6 @@
pinctrl-names = "default";
pinctrl-0 = <_i2c3>;
status = "okay";
-
-   max7310_a: gpio@30 {
-   compatible = "maxim,max7310";
-   reg = <0x30>;
-   gpio-controller;
-   #gpio-cells = <2>;
-   };
-
-   max7310_b: gpio@32 {
-   compatible = "maxim,max7310";
-   reg = <0x32>;
-   gpio-controller;
-   #gpio-cells = <2>;
-   };
-
-   max7310_c: gpio@34 {
-   compatible = "maxim,max7310";
-   reg = <0x34>;
-   gpio-controller;
-   #gpio-cells = <2>;
-   };
 };
 
  {
@@ -418,6 +435,12 @@
>;
};
 
+   pinctrl_i2c3mux: i2c3muxgrp {
+   fsl,pins = <
+   MX6QDL_PAD_EIM_A24__GPIO5_IO04 0x0b0b1
+   >;
+   };
+
pinctrl_pwm3: pwm1grp {
fsl,pins = <
MX6QDL_PAD_SD4_DAT1__PWM3_OUT   0x1b0b1
-- 
2.7.4

[PATCH v7 14/34] ARM: dts: imx6-sabreauto: add the ADV7180 video decoder

[Steve Longerbeam]

Enables the ADV7180 decoder sensor. The ADV7180 connects to the
parallel-bus mux input on ipu1_csi0_mux.

The ADV7180 power pin is via max7310_b port expander.

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 58 
 1 file changed, 58 insertions(+)

diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi 
b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index 1212f82..a712ff1 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -124,6 +124,21 @@
#size-cells = <0>;
reg = <1>;
 
+   adv7180: camera@21 {
+   compatible = "adi,adv7180";
+   reg = <0x21>;
+   powerdown-gpios = <_b 2 
GPIO_ACTIVE_LOW>;
+   interrupt-parent = <>;
+   interrupts = <27 0x8>;
+
+   port {
+   adv7180_to_ipu1_csi0_mux: endpoint {
+   remote-endpoint = 
<_csi0_mux_from_parallel_sensor>;
+   bus-width = <8>;
+   };
+   };
+   };
+
max7310_a: gpio@30 {
compatible = "maxim,max7310";
reg = <0x30>;
@@ -151,6 +166,25 @@
};
 };
 
+_csi0_from_ipu1_csi0_mux {
+   bus-width = <8>;
+};
+
+_csi0_mux_from_parallel_sensor {
+   remote-endpoint = <_to_ipu1_csi0_mux>;
+   bus-width = <8>;
+};
+
+_csi0 {
+   pinctrl-names = "default";
+   pinctrl-0 = <_ipu1_csi0>;
+
+   /* enable frame interval monitor on this port */
+   fim {
+   status = "okay";
+   };
+};
+
  {
assigned-clocks = < IMX6QDL_PLL4_BYPASS_SRC>,
  < IMX6QDL_PLL4_BYPASS>,
@@ -444,6 +478,30 @@
>;
};
 
+   pinctrl_ipu1_csi0: ipu1csi0grp {
+   fsl,pins = <
+   MX6QDL_PAD_CSI0_DAT4__IPU1_CSI0_DATA04   0x1b0b0
+   MX6QDL_PAD_CSI0_DAT5__IPU1_CSI0_DATA05   0x1b0b0
+   MX6QDL_PAD_CSI0_DAT6__IPU1_CSI0_DATA06   0x1b0b0
+   MX6QDL_PAD_CSI0_DAT7__IPU1_CSI0_DATA07   0x1b0b0
+   MX6QDL_PAD_CSI0_DAT8__IPU1_CSI0_DATA08   0x1b0b0
+   MX6QDL_PAD_CSI0_DAT9__IPU1_CSI0_DATA09   0x1b0b0
+   MX6QDL_PAD_CSI0_DAT10__IPU1_CSI0_DATA10  0x1b0b0
+   MX6QDL_PAD_CSI0_DAT11__IPU1_CSI0_DATA11  0x1b0b0
+   MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12  0x1b0b0
+   MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13  0x1b0b0
+   MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14  0x1b0b0
+   MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15  0x1b0b0
+   MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16  0x1b0b0
+   MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17  0x1b0b0
+   MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18  0x1b0b0
+   MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19  0x1b0b0
+   MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK 0x1b0b0
+   MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC0x1b0b0
+   MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC   0x1b0b0
+   >;
+   };
+
pinctrl_max7310: max7310grp {
fsl,pins = <
MX6QDL_PAD_SD2_DAT0__GPIO1_IO15 0x1b0b0
-- 
2.7.4

[PATCH v7 12/34] ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b

[Steve Longerbeam]

The reset pin to the port expander chip (MAX7310) is controlled by a gpio,
so define a reset-gpios property to control it. There are three MAX7310's
on the SabreAuto CPU card (max7310_[abc]), but all use the same pin for
their reset. Since all can't acquire the same pin, assign it to max7310_b,
that chip is needed by more functions (usb and adv7180).

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 9 +
 1 file changed, 9 insertions(+)

diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi 
b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index c8e35c4..21dea5f 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -136,6 +136,9 @@
reg = <0x32>;
gpio-controller;
#gpio-cells = <2>;
+   pinctrl-names = "default";
+   pinctrl-0 = <_max7310>;
+   reset-gpios = < 15 GPIO_ACTIVE_LOW>;
};
 
max7310_c: gpio@34 {
@@ -441,6 +444,12 @@
>;
};
 
+   pinctrl_max7310: max7310grp {
+   fsl,pins = <
+   MX6QDL_PAD_SD2_DAT0__GPIO1_IO15 0x1b0b0
+   >;
+   };
+
pinctrl_pwm3: pwm1grp {
fsl,pins = <
MX6QDL_PAD_SD4_DAT1__PWM3_OUT   0x1b0b1
-- 
2.7.4

[PATCH v7 04/34] ARM: dts: imx6qdl: add multiplexer controls

[Steve Longerbeam]

From: Philipp Zabel 

The IOMUXC General Purpose Register space contains various bitfields
that control video bus multiplexers. Describe them using a mmio-mux
node. The placement of the IPU CSI video mux controls differs between
i.MX6D/Q and i.MX6S/DL.

Signed-off-by: Philipp Zabel 
---
 arch/arm/boot/dts/imx6dl.dtsi  | 10 ++
 arch/arm/boot/dts/imx6q.dtsi   | 10 ++
 arch/arm/boot/dts/imx6qdl.dtsi |  7 ++-
 3 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index 7aa120f..10bc9d1 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -181,6 +181,16 @@
  "di0", "di1";
 };
 
+ {
+   mux-reg-masks = <0x34 0x0007>, /* IPU_CSI0_MUX */
+   <0x34 0x0038>, /* IPU_CSI1_MUX */
+   <0x0c 0x000c>, /* HDMI_MUX_CTL */
+   <0x0c 0x00c0>, /* LVDS0_MUX_CTL */
+   <0x0c 0x0300>, /* LVDS1_MUX_CTL */
+   <0x28 0x0003>, /* DCIC1_MUX_CTL */
+   <0x28 0x000c>; /* DCIC2_MUX_CTL */
+};
+
  {
compatible = "fsl,imx6dl-vpu", "cnm,coda960";
 };
diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index e9a5d0b..a6962be 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -332,6 +332,16 @@
};
 };
 
+ {
+   mux-reg-masks = <0x04 0x0008>, /* MIPI_IPU1_MUX */
+   <0x04 0x0010>, /* MIPI_IPU2_MUX */
+   <0x0c 0x000c>, /* HDMI_MUX_CTL */
+   <0x0c 0x00c0>, /* LVDS0_MUX_CTL */
+   <0x0c 0x0300>, /* LVDS1_MUX_CTL */
+   <0x28 0x0003>, /* DCIC1_MUX_CTL */
+   <0x28 0x000c>; /* DCIC2_MUX_CTL */
+};
+
  {
compatible = "fsl,imx6q-vpu", "cnm,coda960";
 };
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index e426faa..50534dd 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -808,8 +808,13 @@
};
 
gpr: iomuxc-gpr@020e {
-   compatible = "fsl,imx6q-iomuxc-gpr", "syscon";
+   compatible = "fsl,imx6q-iomuxc-gpr", "syscon", 
"simple-mfd";
reg = <0x020e 0x38>;
+
+   mux: mux-controller {
+   compatible = "mmio-mux";
+   #mux-control-cells = <1>;
+   };
};
 
iomuxc: iomuxc@020e {
-- 
2.7.4

[PATCH v7 06/34] ARM: dts: imx6qdl: Add video multiplexers, mipi_csi, and their connections

[Steve Longerbeam]

From: Philipp Zabel 

This patch adds the device tree graph connecting the input multiplexers
to the IPU CSIs and the MIPI-CSI2 gasket on i.MX6. The MIPI_IPU
multiplexers are added as children of the iomuxc-gpr syscon device node.
On i.MX6Q/D two two-input multiplexers in front of IPU1 CSI0 and IPU2
CSI1 allow to select between CSI0/1 parallel input pads and the MIPI
CSI-2 virtual channels 0/3.
On i.MX6DL/S two five-input multiplexers in front of IPU1 CSI0 and IPU1
CSI1 allow to select between CSI0/1 parallel input pads and any of the
four MIPI CSI-2 virtual channels.

Signed-off-by: Philipp Zabel 

- Removed some dangling/unused endpoints (ipu2_csi0_from_csi2ipu)
- Renamed the mipi virtual channel endpoint labels, from "mipi_csiX_..."
  to "mipi_vcX...".
- Added input endpoint anchors to the video muxes for the connections
  from parallel sensors.

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6dl.dtsi  | 174 +
 arch/arm/boot/dts/imx6q.dtsi   | 110 ++
 arch/arm/boot/dts/imx6qdl.dtsi |   6 ++
 3 files changed, 290 insertions(+)

diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index 10bc9d1..4049af7 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -164,6 +164,116 @@
  < 9 207 1>, < 10 206 1>, < 11 133 3>;
 };
 
+ {
+   ipu1_csi0_mux: ipu1_csi0_mux@34 {
+   compatible = "video-mux";
+   mux-controls = < 0>;
+   #address-cells = <1>;
+   #size-cells = <0>;
+
+   port@0 {
+   reg = <0>;
+
+   ipu1_csi0_mux_from_mipi_vc0: endpoint {
+   remote-endpoint = <_vc0_to_ipu1_csi0_mux>;
+   };
+   };
+
+   port@1 {
+   reg = <1>;
+
+   ipu1_csi0_mux_from_mipi_vc1: endpoint {
+   remote-endpoint = <_vc1_to_ipu1_csi0_mux>;
+   };
+   };
+
+   port@2 {
+   reg = <2>;
+
+   ipu1_csi0_mux_from_mipi_vc2: endpoint {
+   remote-endpoint = <_vc2_to_ipu1_csi0_mux>;
+   };
+   };
+
+   port@3 {
+   reg = <3>;
+
+   ipu1_csi0_mux_from_mipi_vc3: endpoint {
+   remote-endpoint = <_vc3_to_ipu1_csi0_mux>;
+   };
+   };
+
+   port@4 {
+   reg = <4>;
+
+   ipu1_csi0_mux_from_parallel_sensor: endpoint {
+   };
+   };
+
+   port@5 {
+   reg = <5>;
+
+   ipu1_csi0_mux_to_ipu1_csi0: endpoint {
+   remote-endpoint = 
<_csi0_from_ipu1_csi0_mux>;
+   };
+   };
+   };
+
+   ipu1_csi1_mux: ipu1_csi1_mux@34 {
+   compatible = "video-mux";
+   mux-controls = < 1>;
+   #address-cells = <1>;
+   #size-cells = <0>;
+
+   port@0 {
+   reg = <0>;
+
+   ipu1_csi1_mux_from_mipi_vc0: endpoint {
+   remote-endpoint = <_vc0_to_ipu1_csi1_mux>;
+   };
+   };
+
+   port@1 {
+   reg = <1>;
+
+   ipu1_csi1_mux_from_mipi_vc1: endpoint {
+   remote-endpoint = <_vc1_to_ipu1_csi1_mux>;
+   };
+   };
+
+   port@2 {
+   reg = <2>;
+
+   ipu1_csi1_mux_from_mipi_vc2: endpoint {
+   remote-endpoint = <_vc2_to_ipu1_csi1_mux>;
+   };
+   };
+
+   port@3 {
+   reg = <3>;
+
+   ipu1_csi1_mux_from_mipi_vc3: endpoint {
+   remote-endpoint = <_vc3_to_ipu1_csi1_mux>;
+   };
+   };
+
+   port@4 {
+   reg = <4>;
+
+   ipu1_csi1_mux_from_parallel_sensor: endpoint {
+   };
+   };
+
+   port@5 {
+   reg = <5>;
+
+   ipu1_csi1_mux_to_ipu1_csi1: endpoint {
+   remote-endpoint = 
<_csi1_from_ipu1_csi1_mux>;
+   };
+   };
+   };
+};
+
  {
compatible = "fsl,imx6dl-gpt";
 };
@@ -172,6 +282,12 @@
compatible = "fsl,imx6dl-hdmi";
 };
 
+_csi1 {
+   ipu1_csi1_from_ipu1_csi1_mux: endpoint {
+   remote-endpoint = <_csi1_mux_to_ipu1_csi1>;
+   };
+};
+
  {
  

[PATCH v7 13/34] ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture

[Steve Longerbeam]

Add pinctrl groups for both GPT input capture channels.

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 12 
 1 file changed, 12 insertions(+)

diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi 
b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index 21dea5f..1212f82 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -456,6 +456,18 @@
>;
};
 
+   pinctrl_gpt_input_capture0: gptinputcapture0grp {
+   fsl,pins = <
+   MX6QDL_PAD_SD1_DAT0__GPT_CAPTURE1   0x1b0b0
+   >;
+   };
+
+   pinctrl_gpt_input_capture1: gptinputcapture1grp {
+   fsl,pins = <
+   MX6QDL_PAD_SD1_DAT1__GPT_CAPTURE2   0x1b0b0
+   >;
+   };
+
pinctrl_spdif: spdifgrp {
fsl,pins = <
MX6QDL_PAD_KEY_COL3__SPDIF_IN 0x1b0b0
-- 
2.7.4

[PATCH v7 08/34] ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround

[Steve Longerbeam]

There is a pin conflict with GPIO_6. This pin functions as a power
input pin to the OV5642 camera sensor, but ENET uses it as the h/w
workaround for erratum ERR006687, to wake-up the ARM cores on normal
RX and TX packet done events. So we need to remove the h/w workaround
to support the OV5642. The result is that the CPUidle driver will no
longer allow entering the deep idle states on the sabrelite.

This is a partial revert of

commit 6261c4c8f13e ("ARM: dts: imx6qdl-sabrelite: use GPIO_6 for FEC
interrupt.")
commit a28eeb43ee57 ("ARM: dts: imx6: tag boards that have the HW workaround
for ERR006687")

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 4 
 1 file changed, 4 deletions(-)

diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi 
b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
index 8413179..89dce27 100644
--- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
@@ -270,9 +270,6 @@
txd1-skew-ps = <0>;
txd2-skew-ps = <0>;
txd3-skew-ps = <0>;
-   interrupts-extended = < 6 IRQ_TYPE_LEVEL_HIGH>,
- < 0 119 IRQ_TYPE_LEVEL_HIGH>;
-   fsl,err006687-workaround-present;
status = "okay";
 };
 
@@ -373,7 +370,6 @@
MX6QDL_PAD_RGMII_RX_CTL__RGMII_RX_CTL   0x1b030
/* Phy reset */
MX6QDL_PAD_EIM_D23__GPIO3_IO23  0x000b0
-   MX6QDL_PAD_GPIO_6__ENET_IRQ 0x000b1
>;
};
 
-- 
2.7.4

[PATCH v7 16/34] [media] add Omnivision OV5640 sensor driver

[Steve Longerbeam]

This driver is based on ov5640_mipi.c from Freescale imx_3.10.17_1.0.0_beta
branch, modified heavily to bring forward to latest interfaces and code
cleanup.

Signed-off-by: Steve Longerbeam 
---
 drivers/media/i2c/Kconfig  |9 +
 drivers/media/i2c/Makefile |1 +
 drivers/media/i2c/ov5640.c | 2224 
 3 files changed, 2234 insertions(+)
 create mode 100644 drivers/media/i2c/ov5640.c

diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
index fd181c9..ff082a7 100644
--- a/drivers/media/i2c/Kconfig
+++ b/drivers/media/i2c/Kconfig
@@ -539,6 +539,15 @@ config VIDEO_OV2659
  To compile this driver as a module, choose M here: the
  module will be called ov2659.
 
+config VIDEO_OV5640
+   tristate "OmniVision OV5640 sensor support"
+   depends on OF
+   depends on GPIOLIB && VIDEO_V4L2 && I2C && VIDEO_V4L2_SUBDEV_API
+   depends on MEDIA_CAMERA_SUPPORT
+   ---help---
+ This is a Video4Linux2 sensor-level driver for the Omnivision
+ OV5640 camera sensor with a MIPI CSI-2 interface.
+
 config VIDEO_OV5645
tristate "OmniVision OV5645 sensor support"
depends on OF
diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
index 62323ec..dc6b0c4 100644
--- a/drivers/media/i2c/Makefile
+++ b/drivers/media/i2c/Makefile
@@ -58,6 +58,7 @@ obj-$(CONFIG_VIDEO_SONY_BTF_MPX) += sony-btf-mpx.o
 obj-$(CONFIG_VIDEO_UPD64031A) += upd64031a.o
 obj-$(CONFIG_VIDEO_UPD64083) += upd64083.o
 obj-$(CONFIG_VIDEO_OV2640) += ov2640.o
+obj-$(CONFIG_VIDEO_OV5640) += ov5640.o
 obj-$(CONFIG_VIDEO_OV5645) += ov5645.o
 obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
 obj-$(CONFIG_VIDEO_OV7640) += ov7640.o
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
new file mode 100644
index 000..2a032bc
--- /dev/null
+++ b/drivers/media/i2c/ov5640.c
@@ -0,0 +1,2224 @@
+/*
+ * Copyright (C) 2011-2013 Freescale Semiconductor, Inc. All Rights Reserved.
+ * Copyright (C) 2014-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+/* min/typical/max system clock (xclk) frequencies */
+#define OV5640_XCLK_MIN  600
+#define OV5640_XCLK_MAX 2400
+
+/*
+ * FIXME: there is no subdev API to set the MIPI CSI-2
+ * virtual channel yet, so this is hardcoded for now.
+ */
+#define OV5640_MIPI_VC 1
+
+#define OV5640_DEFAULT_SLAVE_ID 0x3c
+
+#define OV5640_REG_CHIP_ID 0x300a
+#define OV5640_REG_PAD_OUTPUT000x3019
+#define OV5640_REG_SC_PLL_CTRL00x3034
+#define OV5640_REG_SC_PLL_CTRL10x3035
+#define OV5640_REG_SC_PLL_CTRL20x3036
+#define OV5640_REG_SC_PLL_CTRL30x3037
+#define OV5640_REG_SLAVE_ID0x3100
+#define OV5640_REG_SYS_ROOT_DIVIDER0x3108
+#define OV5640_REG_AWB_R_GAIN  0x3400
+#define OV5640_REG_AWB_G_GAIN  0x3402
+#define OV5640_REG_AWB_B_GAIN  0x3404
+#define OV5640_REG_AWB_MANUAL_CTRL 0x3406
+#define OV5640_REG_AEC_PK_EXPOSURE_HI  0x3500
+#define OV5640_REG_AEC_PK_EXPOSURE_MED 0x3501
+#define OV5640_REG_AEC_PK_EXPOSURE_LO  0x3502
+#define OV5640_REG_AEC_PK_MANUAL   0x3503
+#define OV5640_REG_AEC_PK_REAL_GAIN0x350a
+#define OV5640_REG_AEC_PK_VTS  0x350c
+#define OV5640_REG_TIMING_HTS  0x380c
+#define OV5640_REG_TIMING_VTS  0x380e
+#define OV5640_REG_TIMING_TC_REG21 0x3821
+#define OV5640_REG_AEC_CTRL00  0x3a00
+#define OV5640_REG_AEC_B50_STEP0x3a08
+#define OV5640_REG_AEC_B60_STEP0x3a0a
+#define OV5640_REG_AEC_CTRL0D  0x3a0d
+#define OV5640_REG_AEC_CTRL0E  0x3a0e
+#define OV5640_REG_AEC_CTRL0F  0x3a0f
+#define OV5640_REG_AEC_CTRL10  0x3a10
+#define OV5640_REG_AEC_CTRL11  0x3a11
+#define OV5640_REG_AEC_CTRL1B  0x3a1b
+#define OV5640_REG_AEC_CTRL1E  0x3a1e
+#define OV5640_REG_AEC_CTRL1F  0x3a1f
+#define OV5640_REG_HZ5060_CTRL00   0x3c00
+#define OV5640_REG_HZ5060_CTRL01   0x3c01
+#define OV5640_REG_SIGMADELTA_CTRL0C   0x3c0c
+#define OV5640_REG_FRAME_CTRL010x4202
+#define OV5640_REG_MIPI_CTRL00 0x4800
+#define OV5640_REG_DEBUG_MODE  0x4814
+#define OV5640_REG_PRE_ISP_TEST_SET1   0x503d
+#define OV5640_REG_SDE_CTRL0   0x5580
+#define OV5640_REG_SDE_CTRL1   0x5581
+#define OV5640_REG_SDE_CTRL3   0x5583
+#define OV5640_REG_SDE_CTRL4   0x5584
+#define OV5640_REG_SDE_CTRL5   0x5585
+#define 

[PATCH v7 15/34] add mux and video interface bridge entity functions

[Steve Longerbeam]

From: Philipp Zabel 

Signed-off-by: Philipp Zabel 

- renamed MEDIA_ENT_F_MUX to MEDIA_ENT_F_VID_MUX

Signed-off-by: Steve Longerbeam 
---
 Documentation/media/uapi/mediactl/media-types.rst | 22 ++
 include/uapi/linux/media.h|  6 ++
 2 files changed, 28 insertions(+)

diff --git a/Documentation/media/uapi/mediactl/media-types.rst 
b/Documentation/media/uapi/mediactl/media-types.rst
index 2a5164a..47ee003 100644
--- a/Documentation/media/uapi/mediactl/media-types.rst
+++ b/Documentation/media/uapi/mediactl/media-types.rst
@@ -299,6 +299,28 @@ Types and flags used to represent the media graph elements
  received on its sink pad and outputs the statistics data on
  its source pad.
 
+-  ..  row 29
+
+   ..  _MEDIA-ENT-F-VID-MUX:
+
+   -  ``MEDIA_ENT_F_VID_MUX``
+
+   - Video multiplexer. An entity capable of multiplexing must have at
+ least two sink pads and one source pad, and must pass the video
+ frame(s) received from the active sink pad to the source pad. Video
+ frame(s) from the inactive sink pads are discarded.
+
+-  ..  row 30
+
+   ..  _MEDIA-ENT-F-VID-IF-BRIDGE:
+
+   -  ``MEDIA_ENT_F_VID_IF_BRIDGE``
+
+   - Video interface bridge. A video interface bridge entity must have at
+ least one sink pad and one source pad. It receives video frame(s) on
+ its sink pad in one bus format (HDMI, eDP, MIPI CSI-2, ...) and
+ converts them and outputs them on its source pad in another bus format
+ (eDP, MIPI CSI-2, parallel, ...).
 
 ..  tabularcolumns:: |p{5.5cm}|p{12.0cm}|
 
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index 4890787..fac96c6 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -105,6 +105,12 @@ struct media_device_info {
 #define MEDIA_ENT_F_PROC_VIDEO_STATISTICS  (MEDIA_ENT_F_BASE + 0x4006)
 
 /*
+ * Switch and bridge entitites
+ */
+#define MEDIA_ENT_F_VID_MUX(MEDIA_ENT_F_BASE + 0x5001)
+#define MEDIA_ENT_F_VID_IF_BRIDGE  (MEDIA_ENT_F_BASE + 0x5002)
+
+/*
  * Connectors
  */
 /* It is a responsibility of the entity drivers to add connectors and links */
-- 
2.7.4

[PATCH v7 21/34] media: imx: Add Capture Device Interface

[Steve Longerbeam]

This is the capture device interface driver that provides the v4l2
user interface. Frames can be received from various sources:

- directly from CSI for capturing unconverted images directly from
  camera sensors.

- from the IC pre-process encode task.

- from the IC pre-process viewfinder task.

Signed-off-by: Steve Longerbeam 
---
 drivers/staging/media/imx/Makefile|   1 +
 drivers/staging/media/imx/imx-media-capture.c | 702 ++
 2 files changed, 703 insertions(+)
 create mode 100644 drivers/staging/media/imx/imx-media-capture.c

diff --git a/drivers/staging/media/imx/Makefile 
b/drivers/staging/media/imx/Makefile
index ddd7d94..4606a3a 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -3,3 +3,4 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o
 
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
+obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
diff --git a/drivers/staging/media/imx/imx-media-capture.c 
b/drivers/staging/media/imx/imx-media-capture.c
new file mode 100644
index 000..76c17d9
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -0,0 +1,702 @@
+/*
+ * Video Capture Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2012-2016 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include "imx-media.h"
+
+struct capture_priv {
+   struct imx_media_video_dev vdev;
+
+   struct v4l2_subdev*src_sd;
+   int   src_sd_pad;
+   struct device *dev;
+
+   struct imx_media_dev  *md;
+
+   struct media_pad  vdev_pad;
+
+   struct mutex  mutex;   /* capture device mutex */
+
+   /* the videobuf2 queue */
+   struct vb2_queue   q;
+   /* list of ready imx_media_buffer's from q */
+   struct list_head   ready_q;
+   /* protect ready_q */
+   spinlock_t q_lock;
+
+   /* controls inherited from subdevs */
+   struct v4l2_ctrl_handler ctrl_hdlr;
+
+   /* misc status */
+   bool  stop;  /* streaming is stopping */
+};
+
+#define to_capture_priv(v) container_of(v, struct capture_priv, vdev)
+
+/* In bytes, per queue */
+#define VID_MEM_LIMIT  SZ_64M
+
+static struct vb2_ops capture_qops;
+
+/*
+ * Video ioctls follow
+ */
+
+static int vidioc_querycap(struct file *file, void *fh,
+  struct v4l2_capability *cap)
+{
+   struct capture_priv *priv = video_drvdata(file);
+
+   strncpy(cap->driver, "imx-media-capture", sizeof(cap->driver) - 1);
+   strncpy(cap->card, "imx-media-capture", sizeof(cap->card) - 1);
+   snprintf(cap->bus_info, sizeof(cap->bus_info),
+"platform:%s", priv->src_sd->name);
+
+   return 0;
+}
+
+static int capture_enum_fmt_vid_cap(struct file *file, void *fh,
+   struct v4l2_fmtdesc *f)
+{
+   struct capture_priv *priv = video_drvdata(file);
+   const struct imx_media_pixfmt *cc_src;
+   struct v4l2_subdev_format fmt_src;
+   u32 fourcc;
+   int ret;
+
+   fmt_src.pad = priv->src_sd_pad;
+   fmt_src.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+   ret = v4l2_subdev_call(priv->src_sd, pad, get_fmt, NULL, _src);
+   if (ret) {
+   v4l2_err(priv->src_sd, "failed to get src_sd format\n");
+   return ret;
+   }
+
+   cc_src = imx_media_find_ipu_format(fmt_src.format.code, CS_SEL_ANY);
+   if (!cc_src)
+   cc_src = imx_media_find_mbus_format(fmt_src.format.code,
+   CS_SEL_ANY, true);
+   if (!cc_src)
+   return -EINVAL;
+
+   if (cc_src->bayer) {
+   if (f->index != 0)
+   return -EINVAL;
+   fourcc = cc_src->fourcc;
+   } else {
+   u32 cs_sel = (cc_src->cs == IPUV3_COLORSPACE_YUV) ?
+   CS_SEL_YUV : CS_SEL_RGB;
+
+   ret = imx_media_enum_format(, f->index, cs_sel);
+   if (ret)
+   return ret;
+   }
+
+   f->pixelformat = fourcc;
+
+   return 0;
+}
+
+static int capture_g_fmt_vid_cap(struct file *file, void *fh,
+struct v4l2_format *f)
+{
+   struct capture_priv *priv = video_drvdata(file);
+
+   *f = priv->vdev.fmt;
+
+   return 0;
+}
+
+static int capture_try_fmt_vid_cap(struct file *file, void *fh,

[PATCH v7 20/34] media: Add i.MX media core driver

[Steve Longerbeam]

Add the core media driver for i.MX SOC.

Signed-off-by: Steve Longerbeam 

Add the bayer formats to imx-media's list of supported pixel and bus
formats.

Signed-off-by: Russell King 
---
 Documentation/media/v4l-drivers/imx.rst   | 590 +++
 drivers/staging/media/Kconfig |   2 +
 drivers/staging/media/Makefile|   1 +
 drivers/staging/media/imx/Kconfig |   6 +
 drivers/staging/media/imx/Makefile|   5 +
 drivers/staging/media/imx/TODO|  15 +
 drivers/staging/media/imx/imx-media-dev.c | 665 +
 drivers/staging/media/imx/imx-media-fim.c | 463 
 drivers/staging/media/imx/imx-media-internal-sd.c | 349 +
 drivers/staging/media/imx/imx-media-of.c  | 268 +++
 drivers/staging/media/imx/imx-media-utils.c   | 834 ++
 drivers/staging/media/imx/imx-media.h | 324 +
 include/media/imx.h   |  15 +
 include/uapi/linux/v4l2-controls.h|   4 +
 14 files changed, 3541 insertions(+)
 create mode 100644 Documentation/media/v4l-drivers/imx.rst
 create mode 100644 drivers/staging/media/imx/Kconfig
 create mode 100644 drivers/staging/media/imx/Makefile
 create mode 100644 drivers/staging/media/imx/TODO
 create mode 100644 drivers/staging/media/imx/imx-media-dev.c
 create mode 100644 drivers/staging/media/imx/imx-media-fim.c
 create mode 100644 drivers/staging/media/imx/imx-media-internal-sd.c
 create mode 100644 drivers/staging/media/imx/imx-media-of.c
 create mode 100644 drivers/staging/media/imx/imx-media-utils.c
 create mode 100644 drivers/staging/media/imx/imx-media.h
 create mode 100644 include/media/imx.h

diff --git a/Documentation/media/v4l-drivers/imx.rst 
b/Documentation/media/v4l-drivers/imx.rst
new file mode 100644
index 000..b8796e4
--- /dev/null
+++ b/Documentation/media/v4l-drivers/imx.rst
@@ -0,0 +1,590 @@
+i.MX Video Capture Driver
+=
+
+Introduction
+
+
+The Freescale i.MX5/6 contains an Image Processing Unit (IPU), which
+handles the flow of image frames to and from capture devices and
+display devices.
+
+For image capture, the IPU contains the following internal subunits:
+
+- Image DMA Controller (IDMAC)
+- Camera Serial Interface (CSI)
+- Image Converter (IC)
+- Sensor Multi-FIFO Controller (SMFC)
+- Image Rotator (IRT)
+- Video De-Interlacing or Combining Block (VDIC)
+
+The IDMAC is the DMA controller for transfer of image frames to and from
+memory. Various dedicated DMA channels exist for both video capture and
+display paths. During transfer, the IDMAC is also capable of vertical
+image flip, 8x8 block transfer (see IRT description), pixel component
+re-ordering (for example UYVY to YUYV) within the same colorspace, and
+even packed <--> planar conversion. It can also perform a simple
+de-interlacing by interleaving even and odd lines during transfer
+(without motion compensation which requires the VDIC).
+
+The CSI is the backend capture unit that interfaces directly with
+camera sensors over Parallel, BT.656/1120, and MIPI CSI-2 busses.
+
+The IC handles color-space conversion, resizing (downscaling and
+upscaling), horizontal flip, and 90/270 degree rotation operations.
+
+There are three independent "tasks" within the IC that can carry out
+conversions concurrently: pre-process encoding, pre-process viewfinder,
+and post-processing. Within each task, conversions are split into three
+sections: downsizing section, main section (upsizing, flip, colorspace
+conversion, and graphics plane combining), and rotation section.
+
+The IPU time-shares the IC task operations. The time-slice granularity
+is one burst of eight pixels in the downsizing section, one image line
+in the main processing section, one image frame in the rotation section.
+
+The SMFC is composed of four independent FIFOs that each can transfer
+captured frames from sensors directly to memory concurrently via four
+IDMAC channels.
+
+The IRT carries out 90 and 270 degree image rotation operations. The
+rotation operation is carried out on 8x8 pixel blocks at a time. This
+operation is supported by the IDMAC which handles the 8x8 block transfer
+along with block reordering, in coordination with vertical flip.
+
+The VDIC handles the conversion of interlaced video to progressive, with
+support for different motion compensation modes (low, medium, and high
+motion). The deinterlaced output frames from the VDIC can be sent to the
+IC pre-process viewfinder task for further conversions. The VDIC also
+contains a Combiner that combines two image planes, with alpha blending
+and color keying.
+
+In addition to the IPU internal subunits, there are also two units
+outside the IPU that are also involved in video capture on i.MX:
+
+- MIPI CSI-2 Receiver for camera sensors with the MIPI CSI-2 bus
+ 

[PATCH v7 22/34] media: imx: Add CSI subdev driver

[Steve Longerbeam]

This is a media entity subdevice for the i.MX Camera
Sensor Interface module.

Signed-off-by: Steve Longerbeam 

- Added support for negotiation of frame intervals.

Signed-off-by: Russell King 

- Fixed cropping rectangle negotiation at input and output pads.
- Added support for /2 downscaling, if the output pad dimension(s)
  are 1/2 the crop dimension(s) at csi_setup() time.

Signed-off-by: Philipp Zabel 
---
 drivers/staging/media/imx/Kconfig |   14 +
 drivers/staging/media/imx/Makefile|2 +
 drivers/staging/media/imx/imx-media-csi.c | 1498 +
 3 files changed, 1514 insertions(+)
 create mode 100644 drivers/staging/media/imx/imx-media-csi.c

diff --git a/drivers/staging/media/imx/Kconfig 
b/drivers/staging/media/imx/Kconfig
index 62a3c34..e27ad6d 100644
--- a/drivers/staging/media/imx/Kconfig
+++ b/drivers/staging/media/imx/Kconfig
@@ -4,3 +4,17 @@ config VIDEO_IMX_MEDIA
---help---
  Say yes here to enable support for video4linux media controller
  driver for the i.MX5/6 SOC.
+
+if VIDEO_IMX_MEDIA
+menu "i.MX5/6 Media Sub devices"
+
+config VIDEO_IMX_CSI
+   tristate "i.MX5/6 Camera Sensor Interface driver"
+   depends on VIDEO_IMX_MEDIA && VIDEO_DEV && I2C
+   select VIDEOBUF2_DMA_CONTIG
+   default y
+   ---help---
+ A video4linux camera sensor interface driver for i.MX5/6.
+
+endmenu
+endif
diff --git a/drivers/staging/media/imx/Makefile 
b/drivers/staging/media/imx/Makefile
index 4606a3a..c054490 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -4,3 +4,5 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
+
+obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx-media-csi.c 
b/drivers/staging/media/imx/imx-media-csi.c
new file mode 100644
index 000..b9416ea6
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -0,0 +1,1498 @@
+/*
+ * V4L2 Capture CSI Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2014-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include "imx-media.h"
+
+/*
+ * Min/Max supported width and heights.
+ *
+ * We allow planar output, so we have to align width by 16 pixels
+ * to meet IDMAC alignment requirements.
+ *
+ * TODO: move this into pad format negotiation, if capture device
+ * has not requested planar formats, we should allow 8 pixel
+ * alignment.
+ */
+#define MIN_W   176
+#define MIN_H   144
+#define MAX_W  4096
+#define MAX_H  4096
+#define W_ALIGN4 /* multiple of 16 pixels */
+#define H_ALIGN1 /* multiple of 2 lines */
+#define S_ALIGN1 /* multiple of 2 */
+
+struct csi_priv {
+   struct device *dev;
+   struct ipu_soc *ipu;
+   struct imx_media_dev *md;
+   struct v4l2_subdev sd;
+   struct media_pad pad[CSI_NUM_PADS];
+   /* the video device at IDMAC output pad */
+   struct imx_media_video_dev *vdev;
+   struct imx_media_fim *fim;
+   int csi_id;
+   int smfc_id;
+
+   /* lock to protect all members below */
+   struct mutex lock;
+
+   int active_output_pad;
+
+   struct ipuv3_channel *idmac_ch;
+   struct ipu_smfc *smfc;
+   struct ipu_csi *csi;
+
+   struct v4l2_mbus_framefmt format_mbus[CSI_NUM_PADS];
+   const struct imx_media_pixfmt *cc[CSI_NUM_PADS];
+   struct v4l2_fract frame_interval;
+   struct v4l2_rect crop;
+
+   /* active vb2 buffers to send to video dev sink */
+   struct imx_media_buffer *active_vb2_buf[2];
+   struct imx_media_dma_buf underrun_buf;
+
+   int ipu_buf_num;  /* ipu double buffer index: 0-1 */
+
+   /* the sink for the captured frames */
+   struct media_entity *sink;
+   enum ipu_csi_dest dest;
+   /* the source subdev */
+   struct v4l2_subdev *src_sd;
+
+   /* the mipi virtual channel number at link validate */
+   int vc_num;
+
+   /* the attached sensor at stream on */
+   struct imx_media_subdev *sensor;
+
+   spinlock_t irqlock; /* protect eof_irq handler */
+   struct timer_list eof_timeout_timer;
+   int eof_irq;
+   int nfb4eof_irq;
+
+   struct v4l2_ctrl_handler ctrl_hdlr;
+
+   int power_count;  /* power counter */
+   int stream_count; /* streaming counter */
+   bool last_eof;   /* waiting for last EOF at stream off 

[PATCH v7 17/34] platform: add video-multiplexer subdevice driver

[Steve Longerbeam]

From: Philipp Zabel 

This driver can handle SoC internal and external video bus multiplexers,
controlled by mux controllers provided by the mux controller framework,
such as MMIO register bitfields or GPIOs. The subdevice passes through
the mbus configuration of the active input to the output side.

Signed-off-by: Sascha Hauer 
Signed-off-by: Philipp Zabel 
Signed-off-by: Steve Longerbeam 
Acked-by: Sakari Ailus 
Acked-by: Pavel Machek 
Reviewed-by: Sebastian Reichel 
---
 drivers/media/platform/Kconfig |   7 +
 drivers/media/platform/Makefile|   2 +
 drivers/media/platform/video-mux.c | 295 +
 3 files changed, 304 insertions(+)
 create mode 100644 drivers/media/platform/video-mux.c

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index ac026ee..b98f755 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -74,6 +74,13 @@ config VIDEO_M32R_AR_M64278
  To compile this driver as a module, choose M here: the
  module will be called arv.
 
+config VIDEO_MUX
+   tristate "Video Multiplexer"
+   depends on OF && VIDEO_V4L2_SUBDEV_API && MEDIA_CONTROLLER
+   depends on MULTIPLEXER
+   help
+ This driver provides support for N:1 video bus multiplexers.
+
 config VIDEO_OMAP3
tristate "OMAP 3 Camera support"
depends on VIDEO_V4L2 && I2C && VIDEO_V4L2_SUBDEV_API && ARCH_OMAP3
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index 63303d6..a636302 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -28,6 +28,8 @@ obj-$(CONFIG_VIDEO_SH_VEU)+= sh_veu.o
 
 obj-$(CONFIG_VIDEO_MEM2MEM_DEINTERLACE)+= m2m-deinterlace.o
 
+obj-$(CONFIG_VIDEO_MUX)+= video-mux.o
+
 obj-$(CONFIG_VIDEO_S3C_CAMIF)  += s3c-camif/
 obj-$(CONFIG_VIDEO_SAMSUNG_EXYNOS4_IS) += exynos4-is/
 obj-$(CONFIG_VIDEO_SAMSUNG_S5P_JPEG)   += s5p-jpeg/
diff --git a/drivers/media/platform/video-mux.c 
b/drivers/media/platform/video-mux.c
new file mode 100644
index 000..5ce2bd3
--- /dev/null
+++ b/drivers/media/platform/video-mux.c
@@ -0,0 +1,295 @@
+/*
+ * video stream multiplexer controlled via mux control
+ *
+ * Copyright (C) 2013 Pengutronix, Sascha Hauer 
+ * Copyright (C) 2016-2017 Pengutronix, Philipp Zabel 
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+struct video_mux {
+   struct v4l2_subdev subdev;
+   struct media_pad *pads;
+   struct v4l2_mbus_framefmt *format_mbus;
+   struct mux_control *mux;
+   struct mutex lock;
+   int active;
+};
+
+static inline struct video_mux *v4l2_subdev_to_video_mux(struct v4l2_subdev 
*sd)
+{
+   return container_of(sd, struct video_mux, subdev);
+}
+
+static int video_mux_link_setup(struct media_entity *entity,
+   const struct media_pad *local,
+   const struct media_pad *remote, u32 flags)
+{
+   struct v4l2_subdev *sd = media_entity_to_v4l2_subdev(entity);
+   struct video_mux *vmux = v4l2_subdev_to_video_mux(sd);
+   int ret = 0;
+
+   /*
+* The mux state is determined by the enabled sink pad link.
+* Enabling or disabling the source pad link has no effect.
+*/
+   if (local->flags & MEDIA_PAD_FL_SOURCE)
+   return 0;
+
+   dev_dbg(sd->dev, "link setup '%s':%d->'%s':%d[%d]",
+   remote->entity->name, remote->index, local->entity->name,
+   local->index, flags & MEDIA_LNK_FL_ENABLED);
+
+   mutex_lock(>lock);
+
+   if (flags & MEDIA_LNK_FL_ENABLED) {
+   if (vmux->active == local->index)
+   goto out;
+
+   if (vmux->active >= 0) {
+   ret = -EBUSY;
+   goto out;
+   }
+
+   dev_dbg(sd->dev, "setting %d active\n", local->index);
+   ret = mux_control_try_select(vmux->mux, local->index);
+   if (ret < 0)
+   goto out;
+   vmux->active = local->index;
+   } else {
+   if (vmux->active != 

[PATCH v7 23/34] media: imx: Add VDIC subdev driver

[Steve Longerbeam]

This is a media entity subdevice driver for the i.MX Video De-Interlacing
or Combining Block. So far this entity does not implement the Combining
function but only motion compensated deinterlacing. Video frames are
received from the CSI and are routed to the IC PRPVF entity.

Signed-off-by: Steve Longerbeam 
---
 drivers/staging/media/imx/Makefile |1 +
 drivers/staging/media/imx/imx-media-vdic.c | 1004 
 2 files changed, 1005 insertions(+)
 create mode 100644 drivers/staging/media/imx/imx-media-vdic.c

diff --git a/drivers/staging/media/imx/Makefile 
b/drivers/staging/media/imx/Makefile
index c054490..1f01520 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -4,5 +4,6 @@ imx-media-common-objs := imx-media-utils.o imx-media-fim.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
+obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
 
 obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx-media-vdic.c 
b/drivers/staging/media/imx/imx-media-vdic.c
new file mode 100644
index 000..3dedc47
--- /dev/null
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -0,0 +1,1004 @@
+/*
+ * V4L2 Deinterlacer Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include "imx-media.h"
+
+/*
+ * This subdev implements two different video pipelines:
+ *
+ * CSI -> VDIC
+ *
+ * In this pipeline, the CSI sends a single interlaced field F(n-1)
+ * directly to the VDIC (and optionally the following field F(n)
+ * can be sent to memory via IDMAC channel 13). This pipeline only works
+ * in VDIC's high motion mode, which only requires a single field for
+ * processing. The other motion modes (low and medium) require three
+ * fields, so this pipeline does not work in those modes. Also, it is
+ * not clear how this pipeline can deal with the various field orders
+ * (sequential BT/TB, interlaced BT/TB).
+ *
+ * MEM -> CH8,9,10 -> VDIC
+ *
+ * In this pipeline, previous field F(n-1), current field F(n), and next
+ * field F(n+1) are transferred to the VDIC via IDMAC channels 8,9,10.
+ * These memory buffers can come from a video output or mem2mem device.
+ * All motion modes are supported by this pipeline.
+ *
+ * The "direct" CSI->VDIC pipeline requires no DMA, but it can only be
+ * used in high motion mode.
+ */
+
+struct vdic_priv;
+
+struct vdic_pipeline_ops {
+   int (*setup)(struct vdic_priv *priv);
+   void (*start)(struct vdic_priv *priv);
+   void (*stop)(struct vdic_priv *priv);
+   void (*disable)(struct vdic_priv *priv);
+};
+
+/*
+ * Min/Max supported width and heights.
+ */
+#define MIN_W   176
+#define MIN_H   144
+#define MAX_W_VDIC  968
+#define MAX_H_VDIC 2048
+#define W_ALIGN4 /* multiple of 16 pixels */
+#define H_ALIGN1 /* multiple of 2 lines */
+#define S_ALIGN1 /* multiple of 2 */
+
+struct vdic_priv {
+   struct device*dev;
+   struct ipu_soc   *ipu;
+   struct imx_media_dev *md;
+   struct v4l2_subdev   sd;
+   struct media_pad pad[VDIC_NUM_PADS];
+   int ipu_id;
+
+   /* lock to protect all members below */
+   struct mutex lock;
+
+   /* IPU units we require */
+   struct ipu_vdi *vdi;
+
+   int active_input_pad;
+
+   struct ipuv3_channel *vdi_in_ch_p; /* F(n-1) transfer channel */
+   struct ipuv3_channel *vdi_in_ch;   /* F(n) transfer channel */
+   struct ipuv3_channel *vdi_in_ch_n; /* F(n+1) transfer channel */
+
+   /* pipeline operations */
+   struct vdic_pipeline_ops *ops;
+
+   /* current and previous input buffers indirect path */
+   struct imx_media_buffer *curr_in_buf;
+   struct imx_media_buffer *prev_in_buf;
+
+   /*
+* translated field type, input line stride, and field size
+* for indirect path
+*/
+   u32 fieldtype;
+   u32 in_stride;
+   u32 field_size;
+
+   /* the source (a video device or subdev) */
+   struct media_entity *src;
+   /* the sink that will receive the progressive out buffers */
+   struct v4l2_subdev *sink_sd;
+
+   struct v4l2_mbus_framefmt format_mbus[VDIC_NUM_PADS];
+   const struct imx_media_pixfmt *cc[VDIC_NUM_PADS];
+   struct v4l2_fract frame_interval[VDIC_NUM_PADS];
+
+   /* the video device at IDMAC input pad */
+   struct imx_media_video_dev *vdev;
+
+   bool csi_direct;  /* using 

[PATCH v7 19/34] media: Add userspace header file for i.MX

[Steve Longerbeam]

This adds a header file for use by userspace programs wanting to interact
with the i.MX media driver. It defines custom events and v4l2 controls for
the i.MX v4l2 subdevices.

Signed-off-by: Steve Longerbeam 
---
 include/linux/imx-media.h | 27 +++
 1 file changed, 27 insertions(+)
 create mode 100644 include/linux/imx-media.h

diff --git a/include/linux/imx-media.h b/include/linux/imx-media.h
new file mode 100644
index 000..26f
--- /dev/null
+++ b/include/linux/imx-media.h
@@ -0,0 +1,27 @@
+/*
+ * Copyright (c) 2014-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version
+ */
+
+#ifndef __LINUX_IMX_MEDIA_H__
+#define __LINUX_IMX_MEDIA_H__
+
+/*
+ * events from the subdevs
+ */
+#define V4L2_EVENT_IMX_CLASSV4L2_EVENT_PRIVATE_START
+#define V4L2_EVENT_IMX_FRAME_INTERVAL_ERROR (V4L2_EVENT_IMX_CLASS + 1)
+
+enum imx_ctrl_id {
+   V4L2_CID_IMX_FIM_ENABLE = (V4L2_CID_USER_IMX_BASE + 0),
+   V4L2_CID_IMX_FIM_NUM,
+   V4L2_CID_IMX_FIM_TOLERANCE_MIN,
+   V4L2_CID_IMX_FIM_TOLERANCE_MAX,
+   V4L2_CID_IMX_FIM_NUM_SKIP,
+};
+
+#endif
-- 
2.7.4

[PATCH v7 26/34] ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers

[Steve Longerbeam]

Enable i.MX v4l2 media staging driver. For video capture on i.MX, the
video multiplexer subdev is required. On the SabreAuto, the ADV7180
video decoder is required along with i2c-mux-gpio. The Sabrelite
and SabreSD require the OV5640 and the SabreLite requires PWM clocks
for the OV5640.

Increase max zoneorder to allow larger video buffer allocations.

Signed-off-by: Steve Longerbeam 
---
 arch/arm/configs/imx_v6_v7_defconfig | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/arch/arm/configs/imx_v6_v7_defconfig 
b/arch/arm/configs/imx_v6_v7_defconfig
index bb6fa56..63caf25 100644
--- a/arch/arm/configs/imx_v6_v7_defconfig
+++ b/arch/arm/configs/imx_v6_v7_defconfig
@@ -51,6 +51,7 @@ CONFIG_PREEMPT_VOLUNTARY=y
 CONFIG_AEABI=y
 CONFIG_HIGHMEM=y
 CONFIG_CMA=y
+CONFIG_FORCE_MAX_ZONEORDER=14
 CONFIG_CMDLINE="noinitrd console=ttymxc0,115200"
 CONFIG_KEXEC=y
 CONFIG_CPU_FREQ=y
@@ -183,6 +184,7 @@ CONFIG_SERIAL_FSL_LPUART=y
 CONFIG_SERIAL_FSL_LPUART_CONSOLE=y
 # CONFIG_I2C_COMPAT is not set
 CONFIG_I2C_CHARDEV=y
+CONFIG_I2C_MUX=y
 CONFIG_I2C_MUX_GPIO=y
 # CONFIG_I2C_HELPER_AUTO is not set
 CONFIG_I2C_ALGOPCF=m
@@ -223,14 +225,20 @@ CONFIG_REGULATOR_PFUZE100=y
 CONFIG_MEDIA_SUPPORT=y
 CONFIG_MEDIA_CAMERA_SUPPORT=y
 CONFIG_MEDIA_RC_SUPPORT=y
+CONFIG_MEDIA_CONTROLLER=y
+CONFIG_VIDEO_V4L2_SUBDEV_API=y
 CONFIG_RC_DEVICES=y
 CONFIG_IR_GPIO_CIR=y
 CONFIG_MEDIA_USB_SUPPORT=y
 CONFIG_USB_VIDEO_CLASS=m
 CONFIG_V4L_PLATFORM_DRIVERS=y
+CONFIG_VIDEO_MUX=y
 CONFIG_SOC_CAMERA=y
 CONFIG_V4L_MEM2MEM_DRIVERS=y
 CONFIG_VIDEO_CODA=y
+# CONFIG_MEDIA_SUBDRV_AUTOSELECT is not set
+CONFIG_VIDEO_ADV7180=m
+CONFIG_VIDEO_OV5640=m
 CONFIG_SOC_CAMERA_OV2640=y
 CONFIG_IMX_IPUV3_CORE=y
 CONFIG_DRM=y
@@ -340,6 +348,9 @@ CONFIG_FSL_EDMA=y
 CONFIG_IMX_SDMA=y
 CONFIG_MXS_DMA=y
 CONFIG_STAGING=y
+CONFIG_STAGING_MEDIA=y
+CONFIG_VIDEO_IMX_MEDIA=y
+CONFIG_COMMON_CLK_PWM=y
 CONFIG_IIO=y
 CONFIG_VF610_ADC=y
 CONFIG_MPL3115=y
-- 
2.7.4

[PATCH v7 18/34] platform: video-mux: include temporary mmio-mux support

[Steve Longerbeam]

From: Philipp Zabel 

As long as the mux framework is not merged, add temporary mmio-mux
support to the video-mux driver itself. This patch is to be reverted
once the "mux: minimal mux subsystem" and "mux: mmio-based syscon mux
controller" patches are merged.

Signed-off-by: Philipp Zabel 
---
 drivers/media/platform/Kconfig |  1 -
 drivers/media/platform/video-mux.c | 62 ++
 2 files changed, 62 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index b98f755..fea1dc0 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -77,7 +77,6 @@ config VIDEO_M32R_AR_M64278
 config VIDEO_MUX
tristate "Video Multiplexer"
depends on OF && VIDEO_V4L2_SUBDEV_API && MEDIA_CONTROLLER
-   depends on MULTIPLEXER
help
  This driver provides support for N:1 video bus multiplexers.
 
diff --git a/drivers/media/platform/video-mux.c 
b/drivers/media/platform/video-mux.c
index 5ce2bd3..630dd9a 100644
--- a/drivers/media/platform/video-mux.c
+++ b/drivers/media/platform/video-mux.c
@@ -17,7 +17,12 @@
 #include 
 #include 
 #include 
+#ifdef CONFIG_MULTIPLEXER
 #include 
+#else
+#include 
+#include 
+#endif
 #include 
 #include 
 #include 
@@ -29,7 +34,11 @@ struct video_mux {
struct v4l2_subdev subdev;
struct media_pad *pads;
struct v4l2_mbus_framefmt *format_mbus;
+#ifdef CONFIG_MULTIPLEXER
struct mux_control *mux;
+#else
+   struct regmap_field *field;
+#endif
struct mutex lock;
int active;
 };
@@ -70,7 +79,11 @@ static int video_mux_link_setup(struct media_entity *entity,
}
 
dev_dbg(sd->dev, "setting %d active\n", local->index);
+#ifdef CONFIG_MULTIPLEXER
ret = mux_control_try_select(vmux->mux, local->index);
+#else
+   ret = regmap_field_write(vmux->field, local->index);
+#endif
if (ret < 0)
goto out;
vmux->active = local->index;
@@ -79,7 +92,9 @@ static int video_mux_link_setup(struct media_entity *entity,
goto out;
 
dev_dbg(sd->dev, "going inactive\n");
+#ifdef CONFIG_MULTIPLEXER
mux_control_deselect(vmux->mux);
+#endif
vmux->active = -1;
}
 
@@ -193,6 +208,48 @@ static const struct v4l2_subdev_ops video_mux_subdev_ops = 
{
.video = _mux_subdev_video_ops,
 };
 
+#ifndef CONFIG_MULTIPLEXER
+static int video_mux_probe_mmio_mux(struct video_mux *vmux)
+{
+   struct device *dev = vmux->subdev.dev;
+   struct of_phandle_args args;
+   struct reg_field field;
+   struct regmap *regmap;
+   u32 reg, mask;
+   int ret;
+
+   ret = of_parse_phandle_with_args(dev->of_node, "mux-controls",
+"#mux-control-cells", 0, );
+   if (ret)
+   return ret;
+
+   if (!of_device_is_compatible(args.np, "mmio-mux"))
+   return -EINVAL;
+
+   regmap = syscon_node_to_regmap(args.np->parent);
+   if (IS_ERR(regmap))
+   return PTR_ERR(regmap);
+
+   ret = of_property_read_u32_index(args.np, "mux-reg-masks",
+2 * args.args[0], );
+   if (!ret)
+   ret = of_property_read_u32_index(args.np, "mux-reg-masks",
+2 * args.args[0] + 1, );
+   if (ret < 0)
+   return ret;
+
+   field.reg = reg;
+   field.msb = fls(mask) - 1;
+   field.lsb = ffs(mask) - 1;
+
+   vmux->field = devm_regmap_field_alloc(dev, regmap, field);
+   if (IS_ERR(vmux->field))
+   return PTR_ERR(vmux->field);
+
+   return 0;
+}
+#endif
+
 static int video_mux_probe(struct platform_device *pdev)
 {
struct device_node *np = pdev->dev.of_node;
@@ -230,9 +287,14 @@ static int video_mux_probe(struct platform_device *pdev)
return -EINVAL;
}
 
+#ifdef CONFIG_MULTIPLEXER
vmux->mux = devm_mux_control_get(dev, NULL);
if (IS_ERR(vmux->mux)) {
ret = PTR_ERR(vmux->mux);
+#else
+   ret = video_mux_probe_mmio_mux(vmux);
+   if (ret) {
+#endif
if (ret != -EPROBE_DEFER)
dev_err(dev, "Failed to get mux: %d\n", ret);
return ret;
-- 
2.7.4

[PATCH v7 24/34] media: imx: Add IC subdev drivers

[Steve Longerbeam]

This is a set of three media entity subdevice drivers for the i.MX
Image Converter:

- Pre-process Router: Takes input frames from CSI0, CSI1, or VDIC.
  Two output pads enable either or both of the preprocess tasks
  below. If the input is from one of the CSIs, both proprocess task
  links can be enabled to process frames from that CSI simultaneously.
  If the input is the VDIC, only the Pre-processing Viewfinder task
  link can be enabled.

- Pre-processing Encode task: video frames are routed directly from
  the CSI and can be scaled, color-space converted, and rotated.
  Scaled output is limited to 1024x1024 resolution. Output frames
  are routed to the prpenc capture device.

- Pre-processing Viewfinder task: this task can perform the same
  conversions as the pre-process encode task, but in addition can
  be used for hardware motion compensated deinterlacing. Frames can
  come either directly from the CSI or from the VDIC. Scaled output
  is limited to 1024x1024 resolution. Output frames are routed to
  the prpvf capture device.

Signed-off-by: Steve Longerbeam 
---
 drivers/staging/media/imx/Makefile  |2 +
 drivers/staging/media/imx/imx-ic-common.c   |  113 +++
 drivers/staging/media/imx/imx-ic-prp.c  |  509 +++
 drivers/staging/media/imx/imx-ic-prpencvf.c | 1299 +++
 drivers/staging/media/imx/imx-ic.h  |   38 +
 5 files changed, 1961 insertions(+)
 create mode 100644 drivers/staging/media/imx/imx-ic-common.c
 create mode 100644 drivers/staging/media/imx/imx-ic-prp.c
 create mode 100644 drivers/staging/media/imx/imx-ic-prpencvf.c
 create mode 100644 drivers/staging/media/imx/imx-ic.h

diff --git a/drivers/staging/media/imx/Makefile 
b/drivers/staging/media/imx/Makefile
index 1f01520..878a126 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -1,9 +1,11 @@
 imx-media-objs := imx-media-dev.o imx-media-internal-sd.o imx-media-of.o
 imx-media-common-objs := imx-media-utils.o imx-media-fim.o
+imx-media-ic-objs := imx-ic-common.o imx-ic-prp.o imx-ic-prpencvf.o
 
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-common.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-capture.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
+obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-ic.o
 
 obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
diff --git a/drivers/staging/media/imx/imx-ic-common.c 
b/drivers/staging/media/imx/imx-ic-common.c
new file mode 100644
index 000..cfdd4900
--- /dev/null
+++ b/drivers/staging/media/imx/imx-ic-common.c
@@ -0,0 +1,113 @@
+/*
+ * V4L2 Image Converter Subdev for Freescale i.MX5/6 SOC
+ *
+ * Copyright (c) 2014-2016 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include 
+#include 
+#include 
+#include 
+#include "imx-media.h"
+#include "imx-ic.h"
+
+#define IC_TASK_PRP IC_NUM_TASKS
+#define IC_NUM_OPS  (IC_NUM_TASKS + 1)
+
+static struct imx_ic_ops *ic_ops[IC_NUM_OPS] = {
+   [IC_TASK_PRP]= _ic_prp_ops,
+   [IC_TASK_ENCODER]= _ic_prpencvf_ops,
+   [IC_TASK_VIEWFINDER] = _ic_prpencvf_ops,
+};
+
+static int imx_ic_probe(struct platform_device *pdev)
+{
+   struct imx_media_internal_sd_platformdata *pdata;
+   struct imx_ic_priv *priv;
+   int ret;
+
+   priv = devm_kzalloc(>dev, sizeof(*priv), GFP_KERNEL);
+   if (!priv)
+   return -ENOMEM;
+
+   platform_set_drvdata(pdev, >sd);
+   priv->dev = >dev;
+
+   /* get our ipu_id, grp_id and IC task id */
+   pdata = priv->dev->platform_data;
+   priv->ipu_id = pdata->ipu_id;
+   switch (pdata->grp_id) {
+   case IMX_MEDIA_GRP_ID_IC_PRP:
+   priv->task_id = IC_TASK_PRP;
+   break;
+   case IMX_MEDIA_GRP_ID_IC_PRPENC:
+   priv->task_id = IC_TASK_ENCODER;
+   break;
+   case IMX_MEDIA_GRP_ID_IC_PRPVF:
+   priv->task_id = IC_TASK_VIEWFINDER;
+   break;
+   default:
+   return -EINVAL;
+   }
+
+   v4l2_subdev_init(>sd, ic_ops[priv->task_id]->subdev_ops);
+   v4l2_set_subdevdata(>sd, priv);
+   priv->sd.internal_ops = ic_ops[priv->task_id]->internal_ops;
+   priv->sd.entity.ops = ic_ops[priv->task_id]->entity_ops;
+   priv->sd.entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
+   priv->sd.dev = >dev;
+   priv->sd.owner = THIS_MODULE;
+   priv->sd.flags = V4L2_SUBDEV_FL_HAS_DEVNODE | V4L2_SUBDEV_FL_HAS_EVENTS;
+   priv->sd.grp_id = pdata->grp_id;
+   strncpy(priv->sd.name, pdata->sd_name, sizeof(priv->sd.name));
+
+   ret = ic_ops[priv->task_id]->init(priv);
+   if (ret)
+   return ret;
+
+   ret 

[PATCH v7 25/34] media: imx: Add MIPI CSI-2 Receiver subdev driver

[Steve Longerbeam]

Adds MIPI CSI-2 Receiver subdev driver. This subdev is required
for sensors with a MIPI CSI2 interface.

Signed-off-by: Steve Longerbeam 

- Add the function csi2ipu_gasket_init() to initialize the gasket at
  s_power(ON). The gasket needs to be programmed with the correct color
  component ordering to handle UYVY vs. YUYV ordered mbus formats from
  sensors. Note that the description of the CSI2IPU_GASKET register in
  the i.MX6 reference manual is wrong w.r.t bit 2 (the manual refers to
  this register as CSI2_SW_RST): setting bit 2 selects YUYV order, not UYVY.

Suggested-by: Marek Vasut 
---
 drivers/staging/media/imx/Makefile |   1 +
 drivers/staging/media/imx/imx6-mipi-csi2.c | 696 +
 2 files changed, 697 insertions(+)
 create mode 100644 drivers/staging/media/imx/imx6-mipi-csi2.c

diff --git a/drivers/staging/media/imx/Makefile 
b/drivers/staging/media/imx/Makefile
index 878a126..3569625 100644
--- a/drivers/staging/media/imx/Makefile
+++ b/drivers/staging/media/imx/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-vdic.o
 obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx-media-ic.o
 
 obj-$(CONFIG_VIDEO_IMX_CSI) += imx-media-csi.o
+obj-$(CONFIG_VIDEO_IMX_CSI) += imx6-mipi-csi2.o
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c 
b/drivers/staging/media/imx/imx6-mipi-csi2.c
new file mode 100644
index 000..49bc2c2
--- /dev/null
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -0,0 +1,696 @@
+/*
+ * MIPI CSI-2 Receiver Subdev for Freescale i.MX6 SOC.
+ *
+ * Copyright (c) 2012-2017 Mentor Graphics Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include "imx-media.h"
+
+/*
+ * there must be 5 pads: 1 input pad from sensor, and
+ * the 4 virtual channel output pads
+ */
+#define CSI2_SINK_PAD   0
+#define CSI2_NUM_SINK_PADS  1
+#define CSI2_NUM_SRC_PADS   4
+#define CSI2_NUM_PADS   5
+
+/*
+ * The default maximum bit-rate per lane in Mbps, if the
+ * source subdev does not provide V4L2_CID_LINK_FREQ.
+ */
+#define CSI2_DEFAULT_MAX_MBPS 849
+
+struct csi2_dev {
+   struct device  *dev;
+   struct v4l2_subdev  sd;
+   struct media_pad   pad[CSI2_NUM_PADS];
+   struct clk *dphy_clk;
+   struct clk *pllref_clk;
+   struct clk *pix_clk; /* what is this? */
+   void __iomem   *base;
+   struct v4l2_of_bus_mipi_csi2 bus;
+
+   /* lock to protect all members below */
+   struct mutex lock;
+
+   struct v4l2_mbus_framefmt format_mbus;
+
+   int stream_count;
+   struct v4l2_subdev  *src_sd;
+   boolsink_linked[CSI2_NUM_SRC_PADS];
+};
+
+#define DEVICE_NAME "imx6-mipi-csi2"
+
+/* Register offsets */
+#define CSI2_VERSION0x000
+#define CSI2_N_LANES0x004
+#define CSI2_PHY_SHUTDOWNZ  0x008
+#define CSI2_DPHY_RSTZ  0x00c
+#define CSI2_RESETN 0x010
+#define CSI2_PHY_STATE  0x014
+#define PHY_STOPSTATEDATA_BIT   4
+#define PHY_STOPSTATEDATA(n)BIT(PHY_STOPSTATEDATA_BIT + (n))
+#define PHY_RXCLKACTIVEHS   BIT(8)
+#define PHY_RXULPSCLKNOTBIT(9)
+#define PHY_STOPSTATECLKBIT(10)
+#define CSI2_DATA_IDS_1 0x018
+#define CSI2_DATA_IDS_2 0x01c
+#define CSI2_ERR1   0x020
+#define CSI2_ERR2   0x024
+#define CSI2_MSK1   0x028
+#define CSI2_MSK2   0x02c
+#define CSI2_PHY_TST_CTRL0  0x030
+#define PHY_TESTCLRBIT(0)
+#define PHY_TESTCLKBIT(1)
+#define CSI2_PHY_TST_CTRL1  0x034
+#define PHY_TESTEN BIT(16)
+/*
+ * i.MX CSI2IPU Gasket registers follow. The CSI2IPU gasket is
+ * not part of the MIPI CSI-2 core, but its registers fall in the
+ * same register map range.
+ */
+#define CSI2IPU_GASKET 0xf00
+#define CSI2IPU_YUV422_YUYVBIT(2)
+
+static inline struct csi2_dev *sd_to_dev(struct v4l2_subdev *sdev)
+{
+   return container_of(sdev, struct csi2_dev, sd);
+}
+
+/*
+ * The required sequence of MIPI CSI-2 startup as specified in the i.MX6
+ * reference manual is as follows:
+ *
+ * 1. Deassert presetn signal (global reset).
+ *It's not clear what this "global reset" signal is (maybe APB
+ *global reset), but in any case this step would be probably
+ *be carried out during driver load in csi2_probe().
+ *
+ * 2. Configure MIPI Camera Sensor to put all Tx lanes in LP-11 state.
+ *This must be carried out by the MIPI sensor's s_power(ON) subdev
+ *op.
+ *
+ * 3. D-PHY initialization.
+ * 4. CSI2 Controller 

Re: [RFC 00/10] V4L2 explicit synchronization support

[Gustavo Padovan]

Hi all,

I've been working on the v2 of this series, but I think I hit a blocker
when trying to cover the case where the driver asks to requeue the
buffer. It is related to the out-fence side.

In the current implementation we return on QBUF an out-fence fd that is not
tied to any buffer, because we don't know the queueing order until the
buffer is queued to the driver. Then when the buffer is queued we use
the BUF_QUEUED event to notify userspace of the index of the buffer,
so now userspace knows the buffer associated to the out-fence fd
received earlier.

Userspace goes ahead and send a DRM Atomic Request to the kernel to
display that buffer on the screen once the fence signals. If it is 
a nonblocking request the fence waiting is past the check phase, thus
it isn't allowed to fail anymore.

But now, what happens if the V4L2 driver calls buffer_done() asking
to requeue the buffer. That means the operation failed and can't
signal the fence, starving the DRM side.

We need to fix that. The only way I can see is to guarantee ordering of
buffers when out-fences are used. Ordering is something that HAL3 needs
to so maybe there is more than one reason to do it like this. I'm not
a V4L2 expert, so I don't know all the consequences of such a change.

Any other ideas?

The current patchset is at:

https://git.kernel.org/pub/scm/linux/kernel/git/padovan/linux.git/log/?h=v4l2-fences

Regards,

Gustavo

2017-03-13 Gustavo Padovan :

> From: Gustavo Padovan 
> 
> Hi,
> 
> This RFC adds support for Explicit Synchronization of shared buffers in V4L2.
> It uses the Sync File Framework[1] as vector to communicate the fences
> between kernel and userspace.
> 
> I'm sending this to start the discussion on the best approach to implement
> Explicit Synchronization, please check the TODO/OPEN section below.
> 
> Explicit Synchronization allows us to control the synchronization of
> shared buffers from userspace by passing fences to the kernel and/or 
> receiving them from the the kernel.
> 
> Fences passed to the kernel are named in-fences and the kernel should wait
> them to signal before using the buffer. On the other side, the kernel creates
> out-fences for every buffer it receives from userspace. This fence is sent 
> back
> to userspace and it will signal when the capture, for example, has finished.
> 
> Signalling an out-fence in V4L2 would mean that the job on the buffer is done
> and the buffer can be used by other drivers.
> 
> Current RFC implementation
> --
> 
> The current implementation is not intended to be more than a PoC to start
> the discussion on how Explicit Synchronization should be supported in V4L2.
> 
> The first patch proposes an userspace API for fences, then on patch 2
> we prepare to the addition of in-fences in patch 3, by introducing the
> infrastructure on vb2 to wait on an in-fence signal before queueing the buffer
> in the driver.
> 
> Patch 4 fix uvc v4l2 event handling and patch 5 configure q->dev for vivid
> drivers to enable to subscribe and dequeue events on it.
> 
> Patches 6-7 enables support to notify BUF_QUEUED events, i.e., let userspace
> know that particular buffer was enqueued in the driver. This is needed,
> because we return the out-fence fd as an out argument in QBUF, but at the time
> it returns we don't know to which buffer the fence will be attached thus
> the BUF_QUEUED event tells which buffer is associated to the fence received in
> QBUF by userspace.
> 
> Patches 8 and 9 add more fence infrastructure to support out-fences and 
> finally
> patch 10 adds support to out-fences.
> 
> TODO/OPEN:
> --
> 
> * For this first implementation we will keep the ordering of the buffers 
> queued
> in videobuf2, that means we will only enqueue buffer whose fence was signalled
> if that buffer is the first one in the queue. Otherwise it has to wait until 
> it
> is the first one. This is not implmented yet. Later we could create a flag to
> allow unordered queing in the drivers from vb2 if needed.
> 
> * Should we have out-fences per-buffer or per-plane? or both? In this RFC, for
> simplicity they are per-buffer, but Mauro and Javier raised the option of
> doing per-plane fences. That could benefit mem2mem and V4L2 <-> GPU operation
> at least on cases when we have Capture hw that releases the Y frame before the
> other frames for example. When using V4L2 per-plane out-fences to communicate
> with KMS they would need to be merged together as currently the DRM Plane
> interface only supports one fence per DRM Plane.
> 
> In-fences should be per-buffer as the DRM only has per-buffer fences, but
> in case of mem2mem operations per-plane fences might be useful?
> 
> So should we have both ways, per-plane and per-buffer, or just one of them
> for now?
> 
> * other open topics are how to deal with hw-fences and Request API.
> 
> Comments are welcome!
> 
> Regards,
> 
> Gustavo
> 
> ---
> Gustavo Padovan (9):

[PATCH v7 28/34] media: imx: csi: increase burst size for YUV formats

[Steve Longerbeam]

From: Philipp Zabel 

The IDMAC supports burst sizes of up to 32 pixels for interleaved YUV
formats and up to 64 pixels for planar YUV formats.

Signed-off-by: Philipp Zabel 
---
 drivers/staging/media/imx/imx-media-csi.c | 17 +
 1 file changed, 17 insertions(+)

diff --git a/drivers/staging/media/imx/imx-media-csi.c 
b/drivers/staging/media/imx/imx-media-csi.c
index 7defe53..e26c025 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -333,6 +333,23 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
passthrough = true;
passthrough_bits = 16;
break;
+   case V4L2_PIX_FMT_YUV420:
+   case V4L2_PIX_FMT_NV12:
+   burst_size = (image.pix.width & 0x3f) ?
+((image.pix.width & 0x1f) ?
+ ((image.pix.width & 0xf) ? 8 : 16) : 32) : 64;
+   passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
+  sensor_ep->bus.parallel.bus_width >= 16);
+   passthrough_bits = 16;
+   break;
+   case V4L2_PIX_FMT_YUYV:
+   case V4L2_PIX_FMT_UYVY:
+   burst_size = (image.pix.width & 0x1f) ?
+((image.pix.width & 0xf) ? 8 : 16) : 32;
+   passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
+  sensor_ep->bus.parallel.bus_width >= 16);
+   passthrough_bits = 16;
+   break;
default:
burst_size = (image.pix.width & 0xf) ? 8 : 16;
passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
-- 
2.7.4

[PATCH v7 32/34] media: imx: capture: add frame sizes/interval enumeration

[Steve Longerbeam]

From: Russell King 

Add support for enumerating frame sizes and frame intervals from the
first subdev via the V4L2 interfaces.

Signed-off-by: Russell King 
Signed-off-by: Steve Longerbeam 
---
 drivers/staging/media/imx/imx-media-capture.c | 73 +++
 1 file changed, 73 insertions(+)

diff --git a/drivers/staging/media/imx/imx-media-capture.c 
b/drivers/staging/media/imx/imx-media-capture.c
index 76c17d9..8b28dbc 100644
--- a/drivers/staging/media/imx/imx-media-capture.c
+++ b/drivers/staging/media/imx/imx-media-capture.c
@@ -81,6 +81,76 @@ static int vidioc_querycap(struct file *file, void *fh,
return 0;
 }
 
+static int capture_enum_framesizes(struct file *file, void *fh,
+  struct v4l2_frmsizeenum *fsize)
+{
+   struct capture_priv *priv = video_drvdata(file);
+   const struct imx_media_pixfmt *cc;
+   struct v4l2_subdev_frame_size_enum fse = {
+   .index = fsize->index,
+   .pad = priv->src_sd_pad,
+   .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+   };
+   int ret;
+
+   cc = imx_media_find_format(fsize->pixel_format, CS_SEL_ANY, true);
+   if (!cc)
+   return -EINVAL;
+
+   fse.code = cc->codes[0];
+
+   ret = v4l2_subdev_call(priv->src_sd, pad, enum_frame_size, NULL, );
+   if (ret)
+   return ret;
+
+   if (fse.min_width == fse.max_width &&
+   fse.min_height == fse.max_height) {
+   fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+   fsize->discrete.width = fse.min_width;
+   fsize->discrete.height = fse.min_height;
+   } else {
+   fsize->type = V4L2_FRMSIZE_TYPE_CONTINUOUS;
+   fsize->stepwise.min_width = fse.min_width;
+   fsize->stepwise.max_width = fse.max_width;
+   fsize->stepwise.min_height = fse.min_height;
+   fsize->stepwise.max_height = fse.max_height;
+   fsize->stepwise.step_width = 1;
+   fsize->stepwise.step_height = 1;
+   }
+
+   return 0;
+}
+
+static int capture_enum_frameintervals(struct file *file, void *fh,
+  struct v4l2_frmivalenum *fival)
+{
+   struct capture_priv *priv = video_drvdata(file);
+   const struct imx_media_pixfmt *cc;
+   struct v4l2_subdev_frame_interval_enum fie = {
+   .index = fival->index,
+   .pad = priv->src_sd_pad,
+   .width = fival->width,
+   .height = fival->height,
+   .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+   };
+   int ret;
+
+   cc = imx_media_find_format(fival->pixel_format, CS_SEL_ANY, true);
+   if (!cc)
+   return -EINVAL;
+
+   fie.code = cc->codes[0];
+
+   ret = v4l2_subdev_call(priv->src_sd, pad, enum_frame_interval, NULL, 
);
+   if (ret)
+   return ret;
+
+   fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
+   fival->discrete = fie.interval;
+
+   return 0;
+}
+
 static int capture_enum_fmt_vid_cap(struct file *file, void *fh,
struct v4l2_fmtdesc *f)
 {
@@ -269,6 +339,9 @@ static int capture_s_parm(struct file *file, void *fh,
 static const struct v4l2_ioctl_ops capture_ioctl_ops = {
.vidioc_querycap= vidioc_querycap,
 
+   .vidioc_enum_framesizes = capture_enum_framesizes,
+   .vidioc_enum_frameintervals = capture_enum_frameintervals,
+
.vidioc_enum_fmt_vid_cap= capture_enum_fmt_vid_cap,
.vidioc_g_fmt_vid_cap   = capture_g_fmt_vid_cap,
.vidioc_try_fmt_vid_cap = capture_try_fmt_vid_cap,
-- 
2.7.4

[PATCH v7 31/34] media: imx: csi: add frame size/interval enumeration

[Steve Longerbeam]

From: Russell King 

Add frame size and frame interval enumeration to CSI.

CSI can downscale the image independently horizontally and vertically by a
factor of two, which enumerates to four different frame sizes at the
output pads. The input pad supports a range of frame sizes.

CSI can also drop frames, resulting in frame rate reduction, so
enumerate the resulting possible output frame rates.

Signed-off-by: Russell King 
Signed-off-by: Steve Longerbeam 
---
 drivers/staging/media/imx/imx-media-csi.c | 70 +++
 1 file changed, 70 insertions(+)

diff --git a/drivers/staging/media/imx/imx-media-csi.c 
b/drivers/staging/media/imx/imx-media-csi.c
index 9766dee..b8b3630 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -1163,6 +1163,74 @@ static int csi_enum_mbus_code(struct v4l2_subdev *sd,
return ret;
 }
 
+static int csi_enum_frame_size(struct v4l2_subdev *sd,
+  struct v4l2_subdev_pad_config *cfg,
+  struct v4l2_subdev_frame_size_enum *fse)
+{
+   struct csi_priv *priv = v4l2_get_subdevdata(sd);
+   struct v4l2_rect *crop;
+   int ret = 0;
+
+   if (fse->pad >= CSI_NUM_PADS ||
+   fse->index > (fse->pad == CSI_SINK_PAD ? 0 : 3))
+   return -EINVAL;
+
+   mutex_lock(>lock);
+
+   if (fse->pad == CSI_SINK_PAD) {
+   fse->min_width = MIN_W;
+   fse->max_width = MAX_W;
+   fse->min_height = MIN_H;
+   fse->max_height = MAX_H;
+   } else {
+   crop = __csi_get_crop(priv, cfg, fse->which);
+
+   fse->min_width = fse->max_width = fse->index & 1 ?
+   crop->width / 2 : crop->width;
+   fse->min_height = fse->max_height = fse->index & 2 ?
+   crop->height / 2 : crop->height;
+   }
+
+   mutex_unlock(>lock);
+   return ret;
+}
+
+static int csi_enum_frame_interval(struct v4l2_subdev *sd,
+  struct v4l2_subdev_pad_config *cfg,
+  struct v4l2_subdev_frame_interval_enum *fie)
+{
+   struct csi_priv *priv = v4l2_get_subdevdata(sd);
+   struct v4l2_fract *input_fi;
+   struct v4l2_rect *crop;
+   int ret = 0;
+
+   if (fie->pad >= CSI_NUM_PADS ||
+   fie->index >= (fie->pad != CSI_SRC_PAD_IDMAC ?
+  1 : ARRAY_SIZE(csi_skip)))
+   return -EINVAL;
+
+   mutex_lock(>lock);
+
+   input_fi = >frame_interval[CSI_SINK_PAD];
+   crop = __csi_get_crop(priv, cfg, fie->which);
+
+   if ((fie->width != crop->width && fie->width != crop->width / 2) ||
+   (fie->height != crop->height && fie->height != crop->height / 2)) {
+   ret = -EINVAL;
+   goto out;
+   }
+
+   fie->interval = *input_fi;
+
+   if (fie->pad == CSI_SRC_PAD_IDMAC)
+   csi_apply_skip_interval(_skip[fie->index],
+   >interval);
+
+out:
+   mutex_unlock(>lock);
+   return ret;
+}
+
 static int csi_get_fmt(struct v4l2_subdev *sd,
   struct v4l2_subdev_pad_config *cfg,
   struct v4l2_subdev_format *sdformat)
@@ -1631,6 +1699,8 @@ static const struct v4l2_subdev_video_ops csi_video_ops = 
{
 
 static const struct v4l2_subdev_pad_ops csi_pad_ops = {
.enum_mbus_code = csi_enum_mbus_code,
+   .enum_frame_size = csi_enum_frame_size,
+   .enum_frame_interval = csi_enum_frame_interval,
.get_fmt = csi_get_fmt,
.set_fmt = csi_set_fmt,
.get_selection = csi_get_selection,
-- 
2.7.4

[PATCH v7 33/34] media: imx: set and propagate default field, colorimetry

[Steve Longerbeam]

This patch adds a call to imx_media_fill_default_mbus_fields() in the
*_try_fmt() functions at the sink pads, to set empty field order and
colorimetry parameters.

If the field order is set to ANY, choose the currently set field order
at the sink pad. If the colorspace is set to DEFAULT, choose the
current colorspace at the sink pad.  If any of xfer_func, ycbcr_enc
or quantization are set to DEFAULT, either choose the current sink pad
setting, or the default setting for the new colorspace, if non-DEFAULT
colorspace was given.

If a format is destined to be routed through the Image Converter,
fixed quantization and Y`CbCr encoding must be set.

Colorimetry is also propagated from sink to source pads anywhere
this has not already been done.

Signed-off-by: Steve Longerbeam 
Signed-off-by: Philipp Zabel 
---
 drivers/staging/media/imx/imx-ic-prp.c  | 10 +++--
 drivers/staging/media/imx/imx-ic-prpencvf.c | 17 ++--
 drivers/staging/media/imx/imx-media-csi.c   | 14 ++-
 drivers/staging/media/imx/imx-media-utils.c | 62 +
 drivers/staging/media/imx/imx-media-vdic.c  |  8 +++-
 drivers/staging/media/imx/imx-media.h   |  4 +-
 6 files changed, 103 insertions(+), 12 deletions(-)

diff --git a/drivers/staging/media/imx/imx-ic-prp.c 
b/drivers/staging/media/imx/imx-ic-prp.c
index 3fc2a4e..7bc293a 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -166,8 +166,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
   struct v4l2_subdev_format *sdformat)
 {
struct prp_priv *priv = sd_to_priv(sd);
+   struct v4l2_mbus_framefmt *fmt, *infmt;
const struct imx_media_pixfmt *cc;
-   struct v4l2_mbus_framefmt *fmt;
int ret = 0;
u32 code;
 
@@ -181,6 +181,8 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
goto out;
}
 
+   infmt = __prp_get_fmt(priv, cfg, PRP_SINK_PAD, sdformat->which);
+
switch (sdformat->pad) {
case PRP_SINK_PAD:
v4l_bound_align_image(>format.width, MIN_W, MAX_W,
@@ -194,12 +196,14 @@ static int prp_set_fmt(struct v4l2_subdev *sd,
cc = imx_media_find_ipu_format(code, CS_SEL_ANY);
sdformat->format.code = cc->codes[0];
}
+
+   imx_media_fill_default_mbus_fields(>format, infmt,
+  true);
break;
case PRP_SRC_PAD_PRPENC:
case PRP_SRC_PAD_PRPVF:
/* Output pads mirror input pad */
-   fmt = __prp_get_fmt(priv, cfg, PRP_SINK_PAD, sdformat->which);
-   sdformat->format = *fmt;
+   sdformat->format = *infmt;
break;
}
 
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c 
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index aef0387..5e9c817 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -805,6 +805,8 @@ static void prp_try_fmt(struct prp_priv *priv,
struct v4l2_subdev_format *sdformat,
const struct imx_media_pixfmt **cc)
 {
+   struct v4l2_mbus_framefmt *infmt;
+
*cc = imx_media_find_ipu_format(sdformat->format.code, CS_SEL_ANY);
if (!*cc) {
u32 code;
@@ -814,22 +816,29 @@ static void prp_try_fmt(struct prp_priv *priv,
sdformat->format.code = (*cc)->codes[0];
}
 
-   if (sdformat->pad == PRPENCVF_SRC_PAD) {
-   struct v4l2_mbus_framefmt *infmt =
-   __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD,
- sdformat->which);
+   infmt = __prp_get_fmt(priv, cfg, PRPENCVF_SINK_PAD, sdformat->which);
 
+   if (sdformat->pad == PRPENCVF_SRC_PAD) {
if (sdformat->format.field != V4L2_FIELD_NONE)
sdformat->format.field = infmt->field;
 
prp_bound_align_output(>format, infmt,
   priv->rot_mode);
+
+   /* propagate colorimetry from sink */
+   sdformat->format.colorspace = infmt->colorspace;
+   sdformat->format.xfer_func = infmt->xfer_func;
+   sdformat->format.quantization = infmt->quantization;
+   sdformat->format.ycbcr_enc = infmt->ycbcr_enc;
} else {
v4l_bound_align_image(>format.width,
  MIN_W_SINK, MAX_W_SINK, W_ALIGN_SINK,
  >format.height,
  MIN_H_SINK, MAX_H_SINK, H_ALIGN_SINK,
  S_ALIGN);
+
+   imx_media_fill_default_mbus_fields(>format, infmt,
+  true);
}
 }
 
diff --git 

[PATCH v7 29/34] media: imx: csi: add frame skipping support

[Steve Longerbeam]

From: Philipp Zabel 

The CSI can skip any out of up to 6 input frames, allowing to reduce the
frame rate at the output pads by small fractions.

Signed-off-by: Philipp Zabel 
Signed-off-by: Steve Longerbeam 
Signed-off-by: Russell King 
---
 drivers/staging/media/imx/imx-media-csi.c | 167 ++
 1 file changed, 146 insertions(+), 21 deletions(-)

diff --git a/drivers/staging/media/imx/imx-media-csi.c 
b/drivers/staging/media/imx/imx-media-csi.c
index e26c025..ed3748a 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -2,6 +2,7 @@
  * V4L2 Capture CSI Subdev for Freescale i.MX5/6 SOC
  *
  * Copyright (c) 2014-2017 Mentor Graphics Inc.
+ * Copyright (C) 2017 Pengutronix, Philipp Zabel 
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
@@ -9,6 +10,7 @@
  * (at your option) any later version.
  */
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -41,6 +43,18 @@
 #define H_ALIGN1 /* multiple of 2 lines */
 #define S_ALIGN1 /* multiple of 2 */
 
+/*
+ * struct csi_skip_desc - CSI frame skipping descriptor
+ * @keep - number of frames kept per max_ratio frames
+ * @max_ratio - width of skip_smfc, written to MAX_RATIO bitfield
+ * @skip_smfc - skip pattern written to the SKIP_SMFC bitfield
+ */
+struct csi_skip_desc {
+   u8 keep;
+   u8 max_ratio;
+   u8 skip_smfc;
+};
+
 struct csi_priv {
struct device *dev;
struct ipu_soc *ipu;
@@ -64,8 +78,9 @@ struct csi_priv {
 
struct v4l2_mbus_framefmt format_mbus[CSI_NUM_PADS];
const struct imx_media_pixfmt *cc[CSI_NUM_PADS];
-   struct v4l2_fract frame_interval;
+   struct v4l2_fract frame_interval[CSI_NUM_PADS];
struct v4l2_rect crop;
+   const struct csi_skip_desc *skip;
 
/* active vb2 buffers to send to video dev sink */
struct imx_media_buffer *active_vb2_buf[2];
@@ -581,6 +596,10 @@ static int csi_setup(struct csi_priv *priv)
 
ipu_csi_set_dest(priv->csi, priv->dest);
 
+   if (priv->dest == IPU_CSI_DEST_IDMAC)
+   ipu_csi_set_skip_smfc(priv->csi, priv->skip->skip_smfc,
+ priv->skip->max_ratio - 1, 0);
+
ipu_csi_dump(priv->csi);
 
return 0;
@@ -588,6 +607,7 @@ static int csi_setup(struct csi_priv *priv)
 
 static int csi_start(struct csi_priv *priv)
 {
+   struct v4l2_fract *output_fi, *input_fi;
u32 bad_frames = 0;
int ret;
 
@@ -596,10 +616,12 @@ static int csi_start(struct csi_priv *priv)
return -EINVAL;
}
 
+   output_fi = >frame_interval[priv->active_output_pad];
+   input_fi = >frame_interval[CSI_SINK_PAD];
+
ret = v4l2_subdev_call(priv->sensor->sd, sensor,
   g_skip_frames, _frames);
if (!ret && bad_frames) {
-   struct v4l2_fract *fi = >frame_interval;
u32 delay_usec;
 
/*
@@ -610,8 +632,8 @@ static int csi_start(struct csi_priv *priv)
 * to lose vert/horiz sync.
 */
delay_usec = DIV_ROUND_UP_ULL(
-   (u64)USEC_PER_SEC * fi->numerator * bad_frames,
-   fi->denominator);
+   (u64)USEC_PER_SEC * input_fi->numerator * bad_frames,
+   input_fi->denominator);
usleep_range(delay_usec, delay_usec + 1000);
}
 
@@ -627,8 +649,7 @@ static int csi_start(struct csi_priv *priv)
 
/* start the frame interval monitor */
if (priv->fim && priv->dest == IPU_CSI_DEST_IDMAC)
-   imx_media_fim_set_stream(priv->fim, >frame_interval,
-true);
+   imx_media_fim_set_stream(priv->fim, output_fi, true);
 
ret = ipu_csi_enable(priv->csi);
if (ret) {
@@ -640,8 +661,7 @@ static int csi_start(struct csi_priv *priv)
 
 fim_off:
if (priv->fim && priv->dest == IPU_CSI_DEST_IDMAC)
-   imx_media_fim_set_stream(priv->fim, >frame_interval,
-false);
+   imx_media_fim_set_stream(priv->fim, NULL, false);
 idmac_stop:
if (priv->dest == IPU_CSI_DEST_IDMAC)
csi_idmac_stop(priv);
@@ -655,14 +675,85 @@ static void csi_stop(struct csi_priv *priv)
 
/* stop the frame interval monitor */
if (priv->fim)
-   imx_media_fim_set_stream(priv->fim,
->frame_interval,
-false);
+   imx_media_fim_set_stream(priv->fim, NULL, false);
}
 
ipu_csi_disable(priv->csi);
 }
 
+static const 

[PATCH v7 30/34] media: imx: csi: add sink selection rectangles

[Steve Longerbeam]

From: Philipp Zabel 

Move the crop rectangle to the sink pad and add a sink compose rectangle
to configure scaling. Also propagate rectangles from sink pad to crop
rectangle, to compose rectangle, and to the source pads both in ACTIVE
and TRY variants of set_fmt/selection, and initialize the default crop
and compose rectangles.

Signed-off-by: Philipp Zabel 
---
 drivers/staging/media/imx/imx-media-csi.c | 153 +++---
 1 file changed, 117 insertions(+), 36 deletions(-)

diff --git a/drivers/staging/media/imx/imx-media-csi.c 
b/drivers/staging/media/imx/imx-media-csi.c
index ed3748a..9766dee 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -80,6 +80,7 @@ struct csi_priv {
const struct imx_media_pixfmt *cc[CSI_NUM_PADS];
struct v4l2_fract frame_interval[CSI_NUM_PADS];
struct v4l2_rect crop;
+   struct v4l2_rect compose;
const struct csi_skip_desc *skip;
 
/* active vb2 buffers to send to video dev sink */
@@ -589,8 +590,8 @@ static int csi_setup(struct csi_priv *priv)
ipu_csi_set_window(priv->csi, >crop);
 
ipu_csi_set_downsize(priv->csi,
-priv->crop.width == 2 * outfmt->width,
-priv->crop.height == 2 * outfmt->height);
+priv->crop.width == 2 * priv->compose.width,
+priv->crop.height == 2 * priv->compose.height);
 
ipu_csi_init_interface(priv->csi, _mbus_cfg, _fmt);
 
@@ -1072,6 +1073,17 @@ __csi_get_crop(struct csi_priv *priv, struct 
v4l2_subdev_pad_config *cfg,
return >crop;
 }
 
+static struct v4l2_rect *
+__csi_get_compose(struct csi_priv *priv, struct v4l2_subdev_pad_config *cfg,
+ enum v4l2_subdev_format_whence which)
+{
+   if (which == V4L2_SUBDEV_FORMAT_TRY)
+   return v4l2_subdev_get_try_compose(>sd, cfg,
+  CSI_SINK_PAD);
+   else
+   return >compose;
+}
+
 static void csi_try_crop(struct csi_priv *priv,
 struct v4l2_rect *crop,
 struct v4l2_subdev_pad_config *cfg,
@@ -1181,6 +1193,7 @@ static void csi_try_fmt(struct csi_priv *priv,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *sdformat,
struct v4l2_rect *crop,
+   struct v4l2_rect *compose,
const struct imx_media_pixfmt **cc)
 {
const struct imx_media_pixfmt *incc;
@@ -1195,15 +1208,8 @@ static void csi_try_fmt(struct csi_priv *priv,
incc = imx_media_find_mbus_format(infmt->code,
  CS_SEL_ANY, true);
 
-   if (sdformat->format.width < crop->width * 3 / 4)
-   sdformat->format.width = crop->width / 2;
-   else
-   sdformat->format.width = crop->width;
-
-   if (sdformat->format.height < crop->height * 3 / 4)
-   sdformat->format.height = crop->height / 2;
-   else
-   sdformat->format.height = crop->height;
+   sdformat->format.width = compose->width;
+   sdformat->format.height = compose->height;
 
if (incc->bayer) {
sdformat->format.code = infmt->code;
@@ -1239,11 +1245,17 @@ static void csi_try_fmt(struct csi_priv *priv,
v4l_bound_align_image(>format.width, MIN_W, MAX_W,
  W_ALIGN, >format.height,
  MIN_H, MAX_H, H_ALIGN, S_ALIGN);
+
+   /* Reset crop and compose rectangles */
crop->left = 0;
crop->top = 0;
crop->width = sdformat->format.width;
crop->height = sdformat->format.height;
csi_try_crop(priv, crop, cfg, >format, sensor);
+   compose->left = 0;
+   compose->top = 0;
+   compose->width = crop->width;
+   compose->height = crop->height;
 
*cc = imx_media_find_mbus_format(sdformat->format.code,
 CS_SEL_ANY, true);
@@ -1268,7 +1280,7 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
struct imx_media_subdev *sensor;
struct v4l2_pix_format vdev_fmt;
struct v4l2_mbus_framefmt *fmt;
-   struct v4l2_rect *crop;
+   struct v4l2_rect *crop, *compose;
int ret = 0;
 
if (sdformat->pad >= CSI_NUM_PADS)
@@ -1288,8 +1300,9 @@ static int csi_set_fmt(struct v4l2_subdev *sd,
}
 
crop = __csi_get_crop(priv, cfg, sdformat->which);
+   compose = __csi_get_compose(priv, cfg, sdformat->which);
 
-   csi_try_fmt(priv, sensor, cfg, sdformat, crop, );
+  

[PATCH v7 34/34] media: imx: Drop warning upon multiple S_STREAM disable calls

[Steve Longerbeam]

From: Marek Vasut 

Calling S_STREAM OFF multiple times on a video device is valid, although
dubious, practice. Instead of warning about it and setting stream count
lower than zero, just ignore the subsequent S_STREAM calls and correct
the stream count to zero.

Signed-off-by: Marek Vasut 
---
 drivers/staging/media/imx/imx-ic-prp.c  | 3 ++-
 drivers/staging/media/imx/imx-ic-prpencvf.c | 3 ++-
 drivers/staging/media/imx/imx-media-csi.c   | 3 ++-
 drivers/staging/media/imx/imx-media-vdic.c  | 3 ++-
 drivers/staging/media/imx/imx6-mipi-csi2.c  | 3 ++-
 5 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/media/imx/imx-ic-prp.c 
b/drivers/staging/media/imx/imx-ic-prp.c
index 7bc293a..c34367e 100644
--- a/drivers/staging/media/imx/imx-ic-prp.c
+++ b/drivers/staging/media/imx/imx-ic-prp.c
@@ -387,7 +387,8 @@ static int prp_s_stream(struct v4l2_subdev *sd, int enable)
 
 update_count:
priv->stream_count += enable ? 1 : -1;
-   WARN_ON(priv->stream_count < 0);
+   if (priv->stream_count < 0)
+   priv->stream_count = 0;
 out:
mutex_unlock(>lock);
return ret;
diff --git a/drivers/staging/media/imx/imx-ic-prpencvf.c 
b/drivers/staging/media/imx/imx-ic-prpencvf.c
index 5e9c817..ed363fe 100644
--- a/drivers/staging/media/imx/imx-ic-prpencvf.c
+++ b/drivers/staging/media/imx/imx-ic-prpencvf.c
@@ -1140,7 +1140,8 @@ static int prp_s_stream(struct v4l2_subdev *sd, int 
enable)
 
 update_count:
priv->stream_count += enable ? 1 : -1;
-   WARN_ON(priv->stream_count < 0);
+   if (priv->stream_count < 0)
+   priv->stream_count = 0;
 out:
mutex_unlock(>lock);
return ret;
diff --git a/drivers/staging/media/imx/imx-media-csi.c 
b/drivers/staging/media/imx/imx-media-csi.c
index 4ab401d..0ec3176 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -861,7 +861,8 @@ static int csi_s_stream(struct v4l2_subdev *sd, int enable)
 
 update_count:
priv->stream_count += enable ? 1 : -1;
-   WARN_ON(priv->stream_count < 0);
+   if (priv->stream_count < 0)
+   priv->stream_count = 0;
 out:
mutex_unlock(>lock);
return ret;
diff --git a/drivers/staging/media/imx/imx-media-vdic.c 
b/drivers/staging/media/imx/imx-media-vdic.c
index c0b6d7f..7eabdc4 100644
--- a/drivers/staging/media/imx/imx-media-vdic.c
+++ b/drivers/staging/media/imx/imx-media-vdic.c
@@ -536,7 +536,8 @@ static int vdic_s_stream(struct v4l2_subdev *sd, int enable)
 
 update_count:
priv->stream_count += enable ? 1 : -1;
-   WARN_ON(priv->stream_count < 0);
+   if (priv->stream_count < 0)
+   priv->stream_count = 0;
 out:
mutex_unlock(>lock);
return ret;
diff --git a/drivers/staging/media/imx/imx6-mipi-csi2.c 
b/drivers/staging/media/imx/imx6-mipi-csi2.c
index 49bc2c2..cfaf34a 100644
--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -396,7 +396,8 @@ static int csi2_s_stream(struct v4l2_subdev *sd, int enable)
 
 update_count:
csi2->stream_count += enable ? 1 : -1;
-   WARN_ON(csi2->stream_count < 0);
+   if (csi2->stream_count < 0)
+   csi2->stream_count = 0;
 out:
mutex_unlock(>lock);
return ret;
-- 
2.7.4

[PATCH v7 09/34] ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors

[Steve Longerbeam]

Adds the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.
Both hang off the same i2c2 bus, so they require different (and non-
default) i2c slave addresses.

The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.

The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi.

The OV5642 node is disabled temporarily while the subdev driver is
cleaned up and submitted later.

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6dl-sabrelite.dts   |   5 ++
 arch/arm/boot/dts/imx6q-sabrelite.dts|   5 ++
 arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 148 +++
 3 files changed, 158 insertions(+)

diff --git a/arch/arm/boot/dts/imx6dl-sabrelite.dts 
b/arch/arm/boot/dts/imx6dl-sabrelite.dts
index 2f90452..3304076 100644
--- a/arch/arm/boot/dts/imx6dl-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6dl-sabrelite.dts
@@ -48,3 +48,8 @@
model = "Freescale i.MX6 DualLite SABRE Lite Board";
compatible = "fsl,imx6dl-sabrelite", "fsl,imx6dl";
 };
+
+_csi1_from_ipu1_csi1_mux {
+   clock-lanes = <0>;
+   data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6q-sabrelite.dts 
b/arch/arm/boot/dts/imx6q-sabrelite.dts
index 02a7cdf..dc51262e 100644
--- a/arch/arm/boot/dts/imx6q-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6q-sabrelite.dts
@@ -52,3 +52,8 @@
  {
status = "okay";
 };
+
+_csi1_from_mipi_vc1 {
+   clock-lanes = <0>;
+   data-lanes = <1 2>;
+};
diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi 
b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
index 89dce27..afe7449 100644
--- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
@@ -39,6 +39,8 @@
  * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
  * OTHER DEALINGS IN THE SOFTWARE.
  */
+
+#include 
 #include 
 #include 
 
@@ -94,6 +96,42 @@
pinctrl-0 = <_can_xcvr>;
gpio = < 2 GPIO_ACTIVE_LOW>;
};
+
+   reg_1p5v: regulator@4 {
+   compatible = "regulator-fixed";
+   reg = <4>;
+   regulator-name = "1P5V";
+   regulator-min-microvolt = <150>;
+   regulator-max-microvolt = <150>;
+   regulator-always-on;
+   };
+
+   reg_1p8v: regulator@5 {
+   compatible = "regulator-fixed";
+   reg = <5>;
+   regulator-name = "1P8V";
+   regulator-min-microvolt = <180>;
+   regulator-max-microvolt = <180>;
+   regulator-always-on;
+   };
+
+   reg_2p8v: regulator@6 {
+   compatible = "regulator-fixed";
+   reg = <6>;
+   regulator-name = "2P8V";
+   regulator-min-microvolt = <280>;
+   regulator-max-microvolt = <280>;
+   regulator-always-on;
+   };
+   };
+
+   mipi_xclk: mipi_xclk {
+   compatible = "pwm-clock";
+   #clock-cells = <0>;
+   clock-frequency = <2200>;
+   clock-output-names = "mipi_pwm3";
+   pwms = < 0 45>; /* 1 / 45 ns = 22 MHz */
+   status = "okay";
};
 
gpio-keys {
@@ -220,6 +258,22 @@
};
 };
 
+_csi0_from_ipu1_csi0_mux {
+   bus-width = <8>;
+   data-shift = <12>; /* Lines 19:12 used */
+   hsync-active = <1>;
+   vync-active = <1>;
+};
+
+_csi0_mux_from_parallel_sensor {
+   remote-endpoint = <_to_ipu1_csi0_mux>;
+};
+
+_csi0 {
+   pinctrl-names = "default";
+   pinctrl-0 = <_ipu1_csi0>;
+};
+
  {
pinctrl-names = "default";
pinctrl-0 = <_audmux>;
@@ -298,6 +352,53 @@
pinctrl-names = "default";
pinctrl-0 = <_i2c2>;
status = "okay";
+
+   ov5640: camera@40 {
+   compatible = "ovti,ov5640";
+   pinctrl-names = "default";
+   pinctrl-0 = <_ov5640>;
+   reg = <0x40>;
+   clocks = <_xclk>;
+   clock-names = "xclk";
+   DOVDD-supply = <_1p8v>;
+   AVDD-supply = <_2p8v>;
+   DVDD-supply = <_1p5v>;
+   reset-gpios = < 5 GPIO_ACTIVE_LOW>; /* NANDF_D5 */
+   powerdown-gpios = < 9 GPIO_ACTIVE_HIGH>; /* NANDF_WP_B */
+
+   port {
+   #address-cells = <1>;
+   #size-cells = <0>;
+
+   ov5640_to_mipi_csi2: endpoint {
+   remote-endpoint = <_csi2_in>;
+   clock-lanes = <0>;
+   data-lanes = <1 2>;
+   };
+   };
+   };
+
+   ov5642: camera@42 {
+   compatible 

[PATCH v7 27/34] media: imx: csi: add support for bayer formats

[Steve Longerbeam]

From: Russell King 

Bayer formats must be treated as generic data and passthrough mode must
be used.  Add the correct setup for these formats.

Signed-off-by: Russell King 

- added check to csi_link_validate() to verify that destination is
  IDMAC output pad when passthrough conditions exist: bayer formats
  and 16-bit parallel buses.

Signed-off-by: Steve Longerbeam 
---
 drivers/staging/media/imx/imx-media-csi.c | 74 ---
 1 file changed, 57 insertions(+), 17 deletions(-)

diff --git a/drivers/staging/media/imx/imx-media-csi.c 
b/drivers/staging/media/imx/imx-media-csi.c
index b9416ea6..7defe53 100644
--- a/drivers/staging/media/imx/imx-media-csi.c
+++ b/drivers/staging/media/imx/imx-media-csi.c
@@ -288,10 +288,11 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
struct imx_media_video_dev *vdev = priv->vdev;
struct v4l2_of_endpoint *sensor_ep;
struct v4l2_mbus_framefmt *infmt;
-   unsigned int burst_size;
struct ipu_image image;
+   u32 passthrough_bits;
dma_addr_t phys[2];
bool passthrough;
+   u32 burst_size;
int ret;
 
infmt = >format_mbus[CSI_SINK_PAD];
@@ -309,24 +310,52 @@ static int csi_idmac_setup_channel(struct csi_priv *priv)
image.phys0 = phys[0];
image.phys1 = phys[1];
 
-   ret = ipu_cpmem_set_image(priv->idmac_ch, );
-   if (ret)
-   goto unsetup_vb2;
-
-   burst_size = (image.pix.width & 0xf) ? 8 : 16;
-
-   ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
-
/*
-* If the sensor uses 16-bit parallel CSI bus, we must handle
-* the data internally in the IPU as 16-bit generic, aka
-* passthrough mode.
+* Check for conditions that require the IPU to handle the
+* data internally as generic data, aka passthrough mode:
+* - raw bayer formats
+* - the sensor bus is 16-bit parallel
 */
-   passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
-  sensor_ep->bus.parallel.bus_width >= 16);
+   switch (image.pix.pixelformat) {
+   case V4L2_PIX_FMT_SBGGR8:
+   case V4L2_PIX_FMT_SGBRG8:
+   case V4L2_PIX_FMT_SGRBG8:
+   case V4L2_PIX_FMT_SRGGB8:
+   burst_size = 8;
+   passthrough = true;
+   passthrough_bits = 8;
+   break;
+   case V4L2_PIX_FMT_SBGGR16:
+   case V4L2_PIX_FMT_SGBRG16:
+   case V4L2_PIX_FMT_SGRBG16:
+   case V4L2_PIX_FMT_SRGGB16:
+   burst_size = 4;
+   passthrough = true;
+   passthrough_bits = 16;
+   break;
+   default:
+   burst_size = (image.pix.width & 0xf) ? 8 : 16;
+   passthrough = (sensor_ep->bus_type != V4L2_MBUS_CSI2 &&
+  sensor_ep->bus.parallel.bus_width >= 16);
+   passthrough_bits = 16;
+   break;
+   }
 
-   if (passthrough)
-   ipu_cpmem_set_format_passthrough(priv->idmac_ch, 16);
+   if (passthrough) {
+   ipu_cpmem_set_resolution(priv->idmac_ch, image.rect.width,
+image.rect.height);
+   ipu_cpmem_set_stride(priv->idmac_ch, image.pix.bytesperline);
+   ipu_cpmem_set_buffer(priv->idmac_ch, 0, image.phys0);
+   ipu_cpmem_set_buffer(priv->idmac_ch, 1, image.phys1);
+   ipu_cpmem_set_format_passthrough(priv->idmac_ch,
+passthrough_bits);
+   } else {
+   ret = ipu_cpmem_set_image(priv->idmac_ch, );
+   if (ret)
+   goto unsetup_vb2;
+   }
+
+   ipu_cpmem_set_burstsize(priv->idmac_ch, burst_size);
 
/*
 * Set the channel for the direct CSI-->memory via SMFC
@@ -823,6 +852,7 @@ static int csi_link_validate(struct v4l2_subdev *sd,
 struct v4l2_subdev_format *sink_fmt)
 {
struct csi_priv *priv = v4l2_get_subdevdata(sd);
+   const struct imx_media_pixfmt *incc;
struct v4l2_of_endpoint *sensor_ep;
struct imx_media_subdev *sensor;
bool is_csi2;
@@ -844,6 +874,16 @@ static int csi_link_validate(struct v4l2_subdev *sd,
priv->sensor = sensor;
sensor_ep = >sensor->sensor_ep;
is_csi2 = (sensor_ep->bus_type == V4L2_MBUS_CSI2);
+   incc = priv->cc[CSI_SINK_PAD];
+
+   if (priv->dest != IPU_CSI_DEST_IDMAC &&
+   (incc->bayer || (!is_csi2 &&
+sensor_ep->bus.parallel.bus_width >= 16))) {
+   v4l2_err(>sd,
+"bayer/16-bit parallel buses must go to IDMAC pad\n");
+   ret = -EINVAL;
+   goto out;
+   }
 
if (is_csi2) {
int vc_num = 0;
@@ -868,7 +908,7 @@ static int 

[PATCH v7 00/34] i.MX Media Driver

[Steve Longerbeam]

In version 7:

- video-mux: switched to Philipp's latest video-mux driver and updated
  bindings docs, that makes use of the mmio-mux framework.

- mmio-mux: includes Philipp's temporary patch that adds mmio-mux support
  to video-mux driver, until mux framework is merged.

- mmio-mux: updates to device tree from Philipp that define the i.MX6 mux
  devices and modifies the video-mux device to become a consumer of the
  video mmio-mux.

- minor updates to Documentation/media/v4l-drivers/imx.rst.

- ov5640: do nothing if entity stream count is greater than 1 in
  ov5640_s_stream().

- Previous versions of this driver had not tested the ability to enable
  multiple independent streams, for instance enabling multiple output
  pads from the imx6-mipi-csi2 subdevice, or enabling both prpenc and
  prpvf outputs. Marek Vasut tested this support and reported issues
  with it.

  v4l2_pipeline_inherit_controls() used the media graph walk APIs, but
  that walks both sink and source pads, so if there are multiple paths
  enabled to video capture devices, controls would be added to the wrong
  video capture device, and no controls added to the other enabled
  capture devices.

  These issues have been fixed. Control inheritance works correctly now
  even with multiple enabled capture paths, and (for example)
  simultaneous capture from prpenc and prpvf works also, and each with
  independent scaling, CSC, and controls. For example prpenc can be
  capturing with a 90 degree rotation, while prpvf is capturing with
  vertical flip.

  So the v4l2_pipeline_inherit_controls() patch has been dropped. The
  new version of control inheritance could be made generically available,
  but it would be more involved to incorporate it into v4l2-core.

- A new function imx_media_fill_default_mbus_fields() is added to setup
  colorimetry at sink pads, and these are propagated to source pads.

- Ensure that the current sink and source rectangles meet alignment
  restrictions before applying a new rotation control setting in
  prp-enc/vf subdevices.

- Chain the s_stream() subdev calls instead of implementing a custom
  stream on/off function that attempts to call a fixed set of subdevices
  in a pipeline in the correct order. This also simplifies imx6-mipi-csi2
  subdevice, since the correct MIPI CSI-2 startup sequence can be
  enforced completely in s_stream(), and s_power() is no longer
  required. This also paves the way for more arbitrary OF graphs
  external to the i.MX6.

- Converted the v4l2_subdev and media_entity ops structures to const.


Marek Vasut (1):
  media: imx: Drop warning upon multiple S_STREAM disable calls

Philipp Zabel (9):
  dt-bindings: Add bindings for video-multiplexer device
  ARM: dts: imx6qdl: add multiplexer controls
  ARM: dts: imx6qdl: Add video multiplexers, mipi_csi, and their
connections
  add mux and video interface bridge entity functions
  platform: add video-multiplexer subdevice driver
  platform: video-mux: include temporary mmio-mux support
  media: imx: csi: increase burst size for YUV formats
  media: imx: csi: add frame skipping support
  media: imx: csi: add sink selection rectangles

Russell King (3):
  media: imx: csi: add support for bayer formats
  media: imx: csi: add frame size/interval enumeration
  media: imx: capture: add frame sizes/interval enumeration

Steve Longerbeam (21):
  [media] dt-bindings: Add bindings for i.MX media driver
  [media] dt/bindings: Add bindings for OV5640
  ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
  ARM: dts: imx6qdl: add capture-subsystem device
  ARM: dts: imx6qdl-sabrelite: remove erratum ERR006687 workaround
  ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
  ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
  ARM: dts: imx6-sabreauto: create i2cmux for i2c3
  ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b
  ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
  ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
  [media] add Omnivision OV5640 sensor driver
  media: Add userspace header file for i.MX
  media: Add i.MX media core driver
  media: imx: Add Capture Device Interface
  media: imx: Add CSI subdev driver
  media: imx: Add VDIC subdev driver
  media: imx: Add IC subdev drivers
  media: imx: Add MIPI CSI-2 Receiver subdev driver
  ARM: imx_v6_v7_defconfig: Enable staging video4linux drivers
  media: imx: set and propagate default field, colorimetry

 .../devicetree/bindings/media/i2c/ov5640.txt   |   45 +
 Documentation/devicetree/bindings/media/imx.txt|   74 +
 .../devicetree/bindings/media/video-mux.txt|   60 +
 Documentation/media/uapi/mediactl/media-types.rst  |   22 +
 Documentation/media/v4l-drivers/imx.rst|  590 ++
 arch/arm/boot/dts/imx6dl-sabrelite.dts |5 +
 arch/arm/boot/dts/imx6dl-sabresd.dts   |5 +
 arch/arm/boot/dts/imx6dl.dtsi  |  189 ++
 

[PATCH v7 03/34] [media] dt/bindings: Add bindings for OV5640

[Steve Longerbeam]

Add device tree binding documentation for the OV5640 camera sensor.

Signed-off-by: Steve Longerbeam 
Acked-by: Rob Herring 
---
 .../devicetree/bindings/media/i2c/ov5640.txt   | 45 ++
 1 file changed, 45 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/i2c/ov5640.txt

diff --git a/Documentation/devicetree/bindings/media/i2c/ov5640.txt 
b/Documentation/devicetree/bindings/media/i2c/ov5640.txt
new file mode 100644
index 000..540b36c
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/i2c/ov5640.txt
@@ -0,0 +1,45 @@
+* Omnivision OV5640 MIPI CSI-2 sensor
+
+Required Properties:
+- compatible: should be "ovti,ov5640"
+- clocks: reference to the xclk input clock.
+- clock-names: should be "xclk".
+- DOVDD-supply: Digital I/O voltage supply, 1.8 volts
+- AVDD-supply: Analog voltage supply, 2.8 volts
+- DVDD-supply: Digital core voltage supply, 1.5 volts
+
+Optional Properties:
+- reset-gpios: reference to the GPIO connected to the reset pin, if any.
+  This is an active low signal to the OV5640.
+- powerdown-gpios: reference to the GPIO connected to the powerdown pin,
+  if any. This is an active high signal to the OV5640.
+
+The device node must contain one 'port' child node for its digital output
+video port, in accordance with the video interface bindings defined in
+Documentation/devicetree/bindings/media/video-interfaces.txt.
+
+Example:
+
+ {
+   ov5640: camera@3c {
+   compatible = "ovti,ov5640";
+   pinctrl-names = "default";
+   pinctrl-0 = <_ov5640>;
+   reg = <0x3c>;
+   clocks = < IMX6QDL_CLK_CKO>;
+   clock-names = "xclk";
+   DOVDD-supply = <_reg>; /* 1.8v */
+   AVDD-supply = <_reg>;  /* 2.8v */
+   DVDD-supply = <_reg>;  /* 1.5v */
+   powerdown-gpios = < 19 GPIO_ACTIVE_HIGH>;
+   reset-gpios = < 20 GPIO_ACTIVE_LOW>;
+
+   port {
+   ov5640_to_mipi_csi2: endpoint {
+   remote-endpoint = <_csi2_from_ov5640>;
+   clock-lanes = <0>;
+   data-lanes = <1 2>;
+   };
+   };
+   };
+};
-- 
2.7.4

[PATCH v7 07/34] ARM: dts: imx6qdl: add capture-subsystem device

[Steve Longerbeam]

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6dl.dtsi | 5 +
 arch/arm/boot/dts/imx6q.dtsi  | 5 +
 2 files changed, 10 insertions(+)

diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index 4049af7..8475e6c 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -100,6 +100,11 @@
};
};
 
+   capture-subsystem {
+   compatible = "fsl,imx-capture-subsystem";
+   ports = <_csi0>, <_csi1>;
+   };
+
display-subsystem {
compatible = "fsl,imx-display-subsystem";
ports = <_di0>, <_di1>;
diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index 214bbb3..89dab0f 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -206,6 +206,11 @@
};
};
 
+   capture-subsystem {
+   compatible = "fsl,imx-capture-subsystem";
+   ports = <_csi0>, <_csi1>, <_csi0>, <_csi1>;
+   };
+
display-subsystem {
compatible = "fsl,imx-display-subsystem";
ports = <_di0>, <_di1>, <_di0>, <_di1>;
-- 
2.7.4

[PATCH v7 02/34] [media] dt-bindings: Add bindings for i.MX media driver

[Steve Longerbeam]

Add bindings documentation for the i.MX media driver.

Signed-off-by: Steve Longerbeam 
---
 Documentation/devicetree/bindings/media/imx.txt | 74 +
 1 file changed, 74 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/imx.txt

diff --git a/Documentation/devicetree/bindings/media/imx.txt 
b/Documentation/devicetree/bindings/media/imx.txt
new file mode 100644
index 000..3059c06
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/imx.txt
@@ -0,0 +1,74 @@
+Freescale i.MX Media Video Device
+=
+
+Video Media Controller node
+---
+
+This is the media controller node for video capture support. It is a
+virtual device that lists the camera serial interface nodes that the
+media device will control.
+
+Required properties:
+- compatible : "fsl,imx-capture-subsystem";
+- ports  : Should contain a list of phandles pointing to camera
+   sensor interface ports of IPU devices
+
+example:
+
+capture-subsystem {
+   compatible = "fsl,imx-capture-subsystem";
+   ports = <_csi0>, <_csi1>;
+};
+
+fim child node
+--
+
+This is an optional child node of the ipu_csi port nodes. If present and
+available, it enables the Frame Interval Monitor. Its properties can be
+used to modify the method in which the FIM measures frame intervals.
+Refer to Documentation/media/v4l-drivers/imx.rst for more info on the
+Frame Interval Monitor.
+
+Optional properties:
+- fsl,input-capture-channel: an input capture channel and channel flags,
+specified as . The channel number
+must be 0 or 1. The flags can be
+IRQ_TYPE_EDGE_RISING, IRQ_TYPE_EDGE_FALLING, or
+IRQ_TYPE_EDGE_BOTH, and specify which input
+capture signal edge will trigger the input
+capture event. If an input capture channel is
+specified, the FIM will use this method to
+measure frame intervals instead of via the EOF
+interrupt. The input capture method is much
+preferred over EOF as it is not subject to
+interrupt latency errors. However it requires
+routing the VSYNC or FIELD output signals of
+the camera sensor to one of the i.MX input
+capture pads (SD1_DAT0, SD1_DAT1), which also
+gives up support for SD1.
+
+
+mipi_csi2 node
+--
+
+This is the device node for the MIPI CSI-2 Receiver, required for MIPI
+CSI-2 sensors.
+
+Required properties:
+- compatible   : "fsl,imx6-mipi-csi2", "snps,dw-mipi-csi2";
+- reg   : physical base address and length of the register set;
+- clocks   : the MIPI CSI-2 receiver requires three clocks: hsi_tx
+ (the D-PHY clock), video_27m (D-PHY PLL reference
+ clock), and eim_podf;
+- clock-names  : must contain "dphy", "ref", "pix";
+- port@*: five port nodes must exist, containing endpoints
+ connecting to the source and sink devices according to
+ of_graph bindings. The first port is an input port,
+ connecting with a MIPI CSI-2 source, and ports 1
+ through 4 are output ports connecting with parallel
+ bus sink endpoint nodes and correspond to the four
+ MIPI CSI-2 virtual channel outputs.
+
+Optional properties:
+- interrupts   : must contain two level-triggered interrupts,
+ in order: 100 and 101;
-- 
2.7.4

[PATCH v7 05/34] ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node

[Steve Longerbeam]

Add to the MIPI CSI2 receiver node: compatible strings,
interrupt sources, and clocks.

Signed-off-by: Steve Longerbeam 
---
 arch/arm/boot/dts/imx6qdl.dtsi | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index 50534dd..dd9917c 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -1140,7 +1140,14 @@
};
 
mipi_csi: mipi@021dc000 {
+   compatible = "fsl,imx6-mipi-csi2", 
"snps,dw-mipi-csi2";
reg = <0x021dc000 0x4000>;
+   interrupts = <0 100 0x04>, <0 101 0x04>;
+   clocks = < IMX6QDL_CLK_HSI_TX>,
+< IMX6QDL_CLK_VIDEO_27M>,
+< IMX6QDL_CLK_EIM_PODF>;
+   clock-names = "dphy", "ref", "pix";
+   status = "disabled";
};
 
mipi_dsi: mipi@021e {
-- 
2.7.4

RE: [PATCH 1/1] [media] i2c: add support for OV13858 sensor

[Yang, Hyungwoo]

Hi Sakari,

Thank you for the review, I've left my comments below. I'll submit v2 soon.

-Hyungwoo

-Original Message-
From: Sakari Ailus [mailto:sakari.ai...@iki.fi] 
Sent: Wednesday, May 24, 2017 5:51 AM
To: Yang, Hyungwoo 
Cc: linux-media@vger.kernel.org; sakari.ai...@linux.intel.com; Zheng, Jian Xu 

Subject: Re: [PATCH 1/1] [media] i2c: add support for OV13858 sensor
> Hi Hyungwoo,
> 
> Thanks for the patch. Please see my comments below.
> 
> On Tue, May 23, 2017 at 04:58:28PM -0700, Hyungwoo Yang wrote:
> > This patch adds driver for Omnivision's ov13858 sensor, the driver 
> > supports following features:
> > 
> > - manual exposure/analog gain
> > - two link frequencies
> > - media controller support
> > - runtime pm support
> > 
> > Signed-off-by: Hyungwoo Yang 
> > ---
> >  drivers/media/i2c/Kconfig   |8 +
> >  drivers/media/i2c/Makefile  |1 +
> >  drivers/media/i2c/ov13858.c | 1853 
> > +++
> >  3 files changed, 1862 insertions(+)
> >  create mode 100644 drivers/media/i2c/ov13858.c
> > 
> > diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig 
> > index fd181c9..f8c5cca 100644
> > --- a/drivers/media/i2c/Kconfig
> > +++ b/drivers/media/i2c/Kconfig
> > @@ -589,6 +589,14 @@ config VIDEO_OV9650
> >   This is a V4L2 sensor-level driver for the Omnivision
> >   OV9650 and OV9652 camera sensors.
> >  
> > +config VIDEO_OV13858
> > +   tristate "OmniVision OV13858 sensor support"
> > +   depends on I2C && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
> > +   depends on MEDIA_CAMERA_SUPPORT
> > +   ---help---
> > + This is a Video4Linux2 sensor-level driver for the OmniVision
> > + OV13858 camera.
> > +
> >  config VIDEO_VS6624
> > tristate "ST VS6624 sensor support"
> > depends on VIDEO_V4L2 && I2C
> > diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile 
> > index 62323ec..3f4dc02 100644
> > --- a/drivers/media/i2c/Makefile
> > +++ b/drivers/media/i2c/Makefile
> > @@ -63,6 +63,7 @@ obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
> >  obj-$(CONFIG_VIDEO_OV7640) += ov7640.o
> >  obj-$(CONFIG_VIDEO_OV7670) += ov7670.o
> >  obj-$(CONFIG_VIDEO_OV9650) += ov9650.o
> > +obj-$(CONFIG_VIDEO_OV13858) += ov13858.o
> >  obj-$(CONFIG_VIDEO_MT9M032) += mt9m032.o
> >  obj-$(CONFIG_VIDEO_MT9M111) += mt9m111.o
> >  obj-$(CONFIG_VIDEO_MT9P031) += mt9p031.o diff --git 
> > a/drivers/media/i2c/ov13858.c b/drivers/media/i2c/ov13858.c new file 
> > mode 100644 index 000..7fc735e
> > --- /dev/null
> > +++ b/drivers/media/i2c/ov13858.c
> > @@ -0,0 +1,1853 @@
> > +/*
> > + * Copyright (c) 2017 Intel Corporation.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License 
> > +version
> > + * 2 as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + */
> > +
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +
> > +#define OV13858_REG_VALUE_08BIT1
> > +#define OV13858_REG_VALUE_16BIT2
> > +#define OV13858_REG_VALUE_24BIT3
> > +
> > +#define OV13858_REG_MODE_SELECT0x0100
> > +#define OV13858_MODE_STANDBY   0x00
> > +#define OV13858_MODE_STREAMING 0x01
> > +
> > +#define OV13858_REG_SOFTWARE_RST   0x0103
> > +#define OV13858_SOFTWARE_RST   0x01
> > +
> > +/* PLL1 generates PCLK and MIPI_PHY_CLK */
> > +#define OV13858_REG_PLL1_CTRL_00x0300
> > +#define OV13858_REG_PLL1_CTRL_10x0301
> > +#define OV13858_REG_PLL1_CTRL_20x0302
> > +#define OV13858_REG_PLL1_CTRL_30x0303
> > +#define OV13858_REG_PLL1_CTRL_40x0304
> > +#define OV13858_REG_PLL1_CTRL_50x0305
> > +
> > +/* PLL2 generates DAC_CLK, SCLK and SRAM_CLK */
> > +#define OV13858_REG_PLL2_CTRL_B0x030b
> > +#define OV13858_REG_PLL2_CTRL_C0x030c
> > +#define OV13858_REG_PLL2_CTRL_D0x030d
> > +#define OV13858_REG_PLL2_CTRL_E0x030e
> > +#define OV13858_REG_PLL2_CTRL_F0x030f
> > +#define OV13858_REG_PLL2_CTRL_12   0x0312
> > +#define OV13858_REG_MIPI_SC_CTRL0  0x3016
> > +#define OV13858_REG_MIPI_SC_CTRL1  0x3022
> > +
> > +/* Chip ID */
> > +#define OV13858_REG_CHIP_ID0x300a
> > +#define OV13858_CHIP_ID0x00d855
> > +
> > +/* Group access */
> > +#define OV13858_REG_GROUP_ACCESS   0x3208
> > +#define OV13858_GROUP_HOLD_START(g_id) (0x00 | (g_id))
> > +#define OV13858_GROUP_HOLD_END(g_id)   (0x10 | (g_id))
> > +#define OV13858_GROUP_LAUNCH(g_id) (0xa0 | (g_id))
> > 

Composite MIPI camera devices

[Daniel Mack]

Hi everyone,

On an embedded board I'm working on, I need to interface a camera that
exposes three distinct I2C addresses: one for the imaging sensor
(OmniVision), one focus motor driver and one EEPROM. Usually such
cameras hide their complexity behind one common interface, but this one
just leaves that to the user.

I wonder what is the best way to support such a device under Linux. I'm
currently at the point where I have drivers for the imaging sensor and
for the focus motor, but I somehow need to link them, both in devicetree
and in the v4l userspace API. Is it possible to make a v4l2_subdev a
child of another v4l2_subdev or something? I guess an alternative would
be to offload certain functionality (V4L2_CID_FOCUS_ABSOLUTE) from the
image sensor driver to the motor driver, but I don't enough about the
v4l2 implementation details. Have similar devices been supported in the
past?

About the userspace side, is there any support for handling the focus
(setting focus to a certain point etc) in a convenient way?

I'd be grateful for any pointer.


Thank you,
Daniel

[PATCH] [media] rc-core: race condition before rc_register_device()

[Sean Young]

A rc device can call ir_raw_event_handle() after rc_allocate_device(),
but before rc_register_device() has completed. This is racey because
rcdev->raw is set before rcdev->raw->thread has a valid value.

Reported-by: kbuild test robot 
Signed-off-by: Sean Young 
---
 drivers/media/rc/rc-ir-raw.c | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/media/rc/rc-ir-raw.c b/drivers/media/rc/rc-ir-raw.c
index 90f66dc..a2fc1a1 100644
--- a/drivers/media/rc/rc-ir-raw.c
+++ b/drivers/media/rc/rc-ir-raw.c
@@ -211,7 +211,7 @@ EXPORT_SYMBOL_GPL(ir_raw_event_set_idle);
  */
 void ir_raw_event_handle(struct rc_dev *dev)
 {
-   if (!dev->raw)
+   if (!dev->raw || !dev->raw->thread)
return;
 
wake_up_process(dev->raw->thread);
@@ -490,6 +490,7 @@ int ir_raw_event_register(struct rc_dev *dev)
 {
int rc;
struct ir_raw_handler *handler;
+   struct task_struct *thread;
 
if (!dev)
return -EINVAL;
@@ -507,13 +508,15 @@ int ir_raw_event_register(struct rc_dev *dev)
 * because the event is coming from userspace
 */
if (dev->driver_type != RC_DRIVER_IR_RAW_TX) {
-   dev->raw->thread = kthread_run(ir_raw_event_thread, dev->raw,
-  "rc%u", dev->minor);
+   thread = kthread_run(ir_raw_event_thread, dev->raw, "rc%u",
+dev->minor);
 
-   if (IS_ERR(dev->raw->thread)) {
-   rc = PTR_ERR(dev->raw->thread);
+   if (IS_ERR(thread)) {
+   rc = PTR_ERR(thread);
goto out;
}
+
+   dev->raw->thread = thread;
}
 
mutex_lock(_raw_handler_lock);
-- 
2.9.4

Re: [PATCH v2 2/2] media: entity: Add media_entity_pad_from_fwnode() function

[Niklas Söderlund]

Hi Sakari,

Thanks for your feedback.

On 2017-05-24 16:27:42 +0300, Sakari Ailus wrote:
> Hi Niklas,
> 
> On Wed, May 24, 2017 at 02:09:07AM +0200, Niklas Söderlund wrote:
> > From: Niklas Söderlund 
> > 
> > This is a wrapper around the media entity pad_from_fwnode operation.
> > 
> > Signed-off-by: Niklas Söderlund 
> > ---
> >  drivers/media/media-entity.c | 39 +++
> >  include/media/media-entity.h | 22 ++
> >  2 files changed, 61 insertions(+)
> > 
> > diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> > index bc44193efa4798b4..c124754f739a8b94 100644
> > --- a/drivers/media/media-entity.c
> > +++ b/drivers/media/media-entity.c
> > @@ -18,6 +18,7 @@
> >  
> >  #include 
> >  #include 
> > +#include 
> >  #include 
> >  #include 
> >  #include 
> > @@ -386,6 +387,44 @@ struct media_entity *media_graph_walk_next(struct 
> > media_graph *graph)
> >  }
> >  EXPORT_SYMBOL_GPL(media_graph_walk_next);
> >  
> > +int media_entity_pad_from_fwnode(struct media_entity *entity,
> > +struct fwnode_handle *fwnode,
> > +int direction, unsigned int *pad)
> > +{
> > +   struct fwnode_endpoint endpoint;
> > +   int i, tmp, ret;
> > +
> > +   if (!entity->ops || !entity->ops->pad_from_fwnode) {
> > +   for (i = 0; i < entity->num_pads; i++) {
> > +   if (entity->pads[i].flags & direction) {
> > +   *pad = i;
> > +   return 0;
> > +   }
> > +   }
> > +
> > +   return -ENXIO;
> > +   }
> > +
> > +   ret = fwnode_graph_parse_endpoint(fwnode, );
> > +   if (ret)
> > +   return ret;
> > +
> > +   ret = entity->ops->pad_from_fwnode(, );
> > +   if (ret)
> > +   return ret;
> > +
> > +   if (tmp >= entity->num_pads)
> > +   return -ENXIO;
> > +
> > +   if (!(entity->pads[tmp].flags & direction))
> > +   return -ENXIO;
> > +
> > +   *pad = tmp;
> > +
> > +   return 0;
> 
> I'd just return the pad number to the caller.

Depending on if there is any comments about this in the previous patch I 
switch to returning an int from this function, where a negative value 
would signal an error.

> 
> > +}
> > +EXPORT_SYMBOL_GPL(media_entity_pad_from_fwnode);
> > +
> >  /* 
> > -
> >   * Pipeline management
> >   */
> > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > index 2aea22b0409d1070..7507181609bec43c 100644
> > --- a/include/media/media-entity.h
> > +++ b/include/media/media-entity.h
> > @@ -822,6 +822,28 @@ struct media_pad *media_entity_remote_pad(struct 
> > media_pad *pad);
> >  struct media_entity *media_entity_get(struct media_entity *entity);
> >  
> >  /**
> > + * media_entity_pad_from_fwnode - Get pad number from fwnode
> > + *
> > + * @entity: The entity
> > + * @fwnode: Pointer to fwnode_handle which should be used to find pad
> > + * @direction: Expected direction of the pad
> 
> You should document the possible values for this. What would you think about
> using a bool called e.g. is_sink? I don't have a strong opinion either way
> though.

Thanks, I think it is better to add documentation about possible values 
here. If no one else disagrees whit this I will do so in the next 
version.

> 
> > + * @pad: Pointer to pad which will should be filled in
> > + *
> > + * This function can be used to resolve the media pad number from
> > + * a fwnode. This is useful for devices which uses more complex
> > + * mappings of media pads.
> > + *
> > + * If the entity do not implement the pad_from_fwnode() operation
> > + * this function searches the entity for the first pad that matches
> > + * the @direction.
> > + *
> > + * Return: return 0 on success.
> > + */
> > +int media_entity_pad_from_fwnode(struct media_entity *entity,
> > +struct fwnode_handle *fwnode,
> > +int direction, unsigned int *pad);
> > +
> > +/**
> >   * media_graph_walk_init - Allocate resources used by graph walk.
> >   *
> >   * @graph: Media graph structure that will be used to walk the graph
> 
> -- 
> Regards,
> 
> Sakari Ailus
> e-mail: sakari.ai...@iki.fi   XMPP: sai...@retiisi.org.uk

-- 
Regards,
Niklas Söderlund

Re: [PATCH v2 1/2] media: entity: Add pad_from_fwnode entity operation

[Niklas Söderlund]

Hi Sakari,

Thanks for your feedback.

On 2017-05-24 16:21:37 +0300, Sakari Ailus wrote:
> Hi Niklas,
> 
> On Wed, May 24, 2017 at 02:09:06AM +0200, Niklas Söderlund wrote:
> > From: Niklas Söderlund 
> > 
> > The optional operation can be used by entities to report how it maps its
> > fwnode endpoints to media pad numbers. This is useful for devices which
> > require advanced mappings of pads.
> > 
> > Signed-off-by: Niklas Söderlund 
> > ---
> >  include/media/media-entity.h | 6 ++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> > index c7c254c5bca1761b..2aea22b0409d1070 100644
> > --- a/include/media/media-entity.h
> > +++ b/include/media/media-entity.h
> > @@ -21,6 +21,7 @@
> >  
> >  #include 
> >  #include 
> > +#include 
> >  #include 
> >  #include 
> >  #include 
> > @@ -171,6 +172,9 @@ struct media_pad {
> >  
> >  /**
> >   * struct media_entity_operations - Media entity operations
> > + * @pad_from_fwnode:   Return the pad number based on a fwnode 
> > endpoint.
> > + * This operation can be used to map a fwnode to a
> > + * media pad number. Optional.
> >   * @link_setup:Notify the entity of link changes. The 
> > operation can
> >   * return an error, in which case link setup will be
> >   * cancelled. Optional.
> > @@ -184,6 +188,8 @@ struct media_pad {
> >   *mutex held.
> >   */
> >  struct media_entity_operations {
> > +   int (*pad_from_fwnode)(struct fwnode_endpoint *endpoint,
> > +  unsigned int *pad);
> 
> Hmm. How about calling this get_fwnode_pad for instance? I wonder what
> others think.

I'm OK with this name change, will update for next version.

> 
> You could just return the pad number still, and a negative value on error. I
> think we won't have more than INT_MAX pads. :-)

I did that at first but then I remembered all the review comments I have 
gotten earlier about using int as the type for pads :-) If you and 
others agree in this case returning the pad as int or a negative value 
as error I have no problem chaining this for the next version.

> 
> > int (*link_setup)(struct media_entity *entity,
> >   const struct media_pad *local,
> >   const struct media_pad *remote, u32 flags);
> 
> -- 
> Regards,
> 
> Sakari Ailus
> e-mail: sakari.ai...@iki.fi   XMPP: sai...@retiisi.org.uk

-- 
Regards,
Niklas Söderlund

[PATCH] i2c: fix platform_no_drv_owner.cocci warnings

[kbuild test robot]

drivers/media/i2c/ov13858.c:1838:3-8: No need to set .owner here. The core will 
do it.

 Remove .owner field if calls are used which set it automatically

Generated by: scripts/coccinelle/api/platform_no_drv_owner.cocci

CC: Hyungwoo Yang 
Signed-off-by: Fengguang Wu 
---

 ov13858.c |1 -
 1 file changed, 1 deletion(-)

--- a/drivers/media/i2c/ov13858.c
+++ b/drivers/media/i2c/ov13858.c
@@ -1835,7 +1835,6 @@ MODULE_DEVICE_TABLE(acpi, ov13858_acpi_i
 static struct i2c_driver ov13858_i2c_driver = {
.driver = {
.name = "ov13858",
-   .owner = THIS_MODULE,
.pm = _pm_ops,
.acpi_match_table = ACPI_PTR(ov13858_acpi_ids),
},

[PATCH] i2c: fix semicolon.cocci warnings

[kbuild test robot]

drivers/media/i2c/ov13858.c:1319:2-3: Unneeded semicolon


 Remove unneeded semicolon.

Generated by: scripts/coccinelle/misc/semicolon.cocci

CC: Hyungwoo Yang 
Signed-off-by: Fengguang Wu 
---

 ov13858.c |2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- a/drivers/media/i2c/ov13858.c
+++ b/drivers/media/i2c/ov13858.c
@@ -1316,7 +1316,7 @@ static int ov13858_set_ctrl(struct v4l2_
 "ctrl(id:0x%x,val:0x%x) is not handled\n",
 ctrl->id, ctrl->val);
break;
-   };
+   }
 
pm_runtime_put(>dev);
 

Re: [PATCH 1/1] [media] i2c: add support for OV13858 sensor

[kbuild test robot]

Hi Hyungwoo,

[auto build test WARNING on linuxtv-media/master]
[also build test WARNING on v4.12-rc2 next-20170524]
[if your patch is applied to the wrong git tree, please drop us a note to help 
improve the system]

url:
https://github.com/0day-ci/linux/commits/Hyungwoo-Yang/i2c-add-support-for-OV13858-sensor/20170524-201204
base:   git://linuxtv.org/media_tree.git master


coccinelle warnings: (new ones prefixed by >>)

>> drivers/media/i2c/ov13858.c:1838:3-8: No need to set .owner here. The core 
>> will do it.
--
>> drivers/media/i2c/ov13858.c:1319:2-3: Unneeded semicolon

Please review and possibly fold the followup patch.

---
0-DAY kernel test infrastructureOpen Source Technology Center
https://lists.01.org/pipermail/kbuild-all   Intel Corporation

Re: [PATCH 1/1] [media] i2c: add support for OV13858 sensor

[kbuild test robot]

Hi Hyungwoo,

[auto build test ERROR on linuxtv-media/master]
[also build test ERROR on v4.12-rc2 next-20170524]
[if your patch is applied to the wrong git tree, please drop us a note to help 
improve the system]

url:
https://github.com/0day-ci/linux/commits/Hyungwoo-Yang/i2c-add-support-for-OV13858-sensor/20170524-201204
base:   git://linuxtv.org/media_tree.git master
config: tile-allmodconfig (attached as .config)
compiler: tilegx-linux-gcc (GCC) 4.6.2
reproduce:
wget 
https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O 
~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=tile 

All errors (new ones prefixed by >>):

   drivers/media/i2c/ov13858.c: In function 'ov13858_set_ctrl':
>> drivers/media/i2c/ov13858.c:1297:36: error: 'struct dev_pm_info' has no 
>> member named 'usage_count'
   drivers/media/i2c/ov13858.c: In function 'ov13858_prepare_streaming':
   drivers/media/i2c/ov13858.c:1568:2: error: implicit declaration of function 
'__v4l2_ctrl_handler_setup'
   cc1: some warnings being treated as errors

vim +1297 drivers/media/i2c/ov13858.c

  1291  struct ov13858 *ov13858 = container_of(ctrl->handler,
  1292 struct ov13858, 
ctrl_handler);
  1293  struct i2c_client *client = v4l2_get_subdevdata(>sd);
  1294  int ret = 0;
  1295  
  1296  /* Values of V4L2 controls will be applied only when power is 
up */
> 1297  if (atomic_read(>dev.power.usage_count) == 0)
  1298  return 0;
  1299  
  1300  ret = pm_runtime_get_sync(>dev);

---
0-DAY kernel test infrastructureOpen Source Technology Center
https://lists.01.org/pipermail/kbuild-all   Intel Corporation


.config.gz
Description: application/gzip

Re: [PATCH 1/1] [media] i2c: add support for OV13858 sensor

[kbuild test robot]

Hi Hyungwoo,

[auto build test ERROR on linuxtv-media/master]
[also build test ERROR on v4.12-rc2 next-20170524]
[if your patch is applied to the wrong git tree, please drop us a note to help 
improve the system]

url:
https://github.com/0day-ci/linux/commits/Hyungwoo-Yang/i2c-add-support-for-OV13858-sensor/20170524-201204
base:   git://linuxtv.org/media_tree.git master
config: i386-allmodconfig (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=i386 

All errors (new ones prefixed by >>):

   drivers/media/i2c/ov13858.c: In function 'ov13858_prepare_streaming':
>> drivers/media/i2c/ov13858.c:1568:2: error: implicit declaration of function 
>> '__v4l2_ctrl_handler_setup' [-Werror=implicit-function-declaration]
 __v4l2_ctrl_handler_setup(ov13858->sd.ctrl_handler);
 ^
   cc1: some warnings being treated as errors

vim +/__v4l2_ctrl_handler_setup +1568 drivers/media/i2c/ov13858.c

  1562  if (ret) {
  1563  dev_err(>dev, "%s failed to set mode\n", 
__func__);
  1564  return ret;
  1565  }
  1566  
  1567  /* Apply customized values from user */
> 1568  __v4l2_ctrl_handler_setup(ov13858->sd.ctrl_handler);
  1569  
  1570  /* Apply bayer order from user */
  1571  return ov13858_apply_bayer_order(ov13858);

---
0-DAY kernel test infrastructureOpen Source Technology Center
https://lists.01.org/pipermail/kbuild-all   Intel Corporation


.config.gz
Description: application/gzip

Re: [PATCH v2 1/2] v4l: async: check for v4l2_dev in v4l2_async_notifier_register()

[Sakari Ailus]

On Wed, May 24, 2017 at 02:07:26AM +0200, Niklas Söderlund wrote:
> From: Niklas Söderlund 
> 
> Add a check for v4l2_dev to v4l2_async_notifier_register() as to fail as
> early as possible since this will fail later in v4l2_async_test_notify().
> 
> Signed-off-by: Niklas Söderlund 

Acked-by: Sakari Ailus 

-- 
Sakari Ailus
e-mail: sakari.ai...@iki.fi XMPP: sai...@retiisi.org.uk

Re: [PATCH v2 2/2] media: entity: Add media_entity_pad_from_fwnode() function

[Sakari Ailus]

Hi Niklas,

On Wed, May 24, 2017 at 02:09:07AM +0200, Niklas Söderlund wrote:
> From: Niklas Söderlund 
> 
> This is a wrapper around the media entity pad_from_fwnode operation.
> 
> Signed-off-by: Niklas Söderlund 
> ---
>  drivers/media/media-entity.c | 39 +++
>  include/media/media-entity.h | 22 ++
>  2 files changed, 61 insertions(+)
> 
> diff --git a/drivers/media/media-entity.c b/drivers/media/media-entity.c
> index bc44193efa4798b4..c124754f739a8b94 100644
> --- a/drivers/media/media-entity.c
> +++ b/drivers/media/media-entity.c
> @@ -18,6 +18,7 @@
>  
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -386,6 +387,44 @@ struct media_entity *media_graph_walk_next(struct 
> media_graph *graph)
>  }
>  EXPORT_SYMBOL_GPL(media_graph_walk_next);
>  
> +int media_entity_pad_from_fwnode(struct media_entity *entity,
> +  struct fwnode_handle *fwnode,
> +  int direction, unsigned int *pad)
> +{
> + struct fwnode_endpoint endpoint;
> + int i, tmp, ret;
> +
> + if (!entity->ops || !entity->ops->pad_from_fwnode) {
> + for (i = 0; i < entity->num_pads; i++) {
> + if (entity->pads[i].flags & direction) {
> + *pad = i;
> + return 0;
> + }
> + }
> +
> + return -ENXIO;
> + }
> +
> + ret = fwnode_graph_parse_endpoint(fwnode, );
> + if (ret)
> + return ret;
> +
> + ret = entity->ops->pad_from_fwnode(, );
> + if (ret)
> + return ret;
> +
> + if (tmp >= entity->num_pads)
> + return -ENXIO;
> +
> + if (!(entity->pads[tmp].flags & direction))
> + return -ENXIO;
> +
> + *pad = tmp;
> +
> + return 0;

I'd just return the pad number to the caller.

> +}
> +EXPORT_SYMBOL_GPL(media_entity_pad_from_fwnode);
> +
>  /* 
> -
>   * Pipeline management
>   */
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index 2aea22b0409d1070..7507181609bec43c 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -822,6 +822,28 @@ struct media_pad *media_entity_remote_pad(struct 
> media_pad *pad);
>  struct media_entity *media_entity_get(struct media_entity *entity);
>  
>  /**
> + * media_entity_pad_from_fwnode - Get pad number from fwnode
> + *
> + * @entity: The entity
> + * @fwnode: Pointer to fwnode_handle which should be used to find pad
> + * @direction: Expected direction of the pad

You should document the possible values for this. What would you think about
using a bool called e.g. is_sink? I don't have a strong opinion either way
though.

> + * @pad: Pointer to pad which will should be filled in
> + *
> + * This function can be used to resolve the media pad number from
> + * a fwnode. This is useful for devices which uses more complex
> + * mappings of media pads.
> + *
> + * If the entity do not implement the pad_from_fwnode() operation
> + * this function searches the entity for the first pad that matches
> + * the @direction.
> + *
> + * Return: return 0 on success.
> + */
> +int media_entity_pad_from_fwnode(struct media_entity *entity,
> +  struct fwnode_handle *fwnode,
> +  int direction, unsigned int *pad);
> +
> +/**
>   * media_graph_walk_init - Allocate resources used by graph walk.
>   *
>   * @graph: Media graph structure that will be used to walk the graph

-- 
Regards,

Sakari Ailus
e-mail: sakari.ai...@iki.fi XMPP: sai...@retiisi.org.uk

Re: [PATCH v2 1/2] media: entity: Add pad_from_fwnode entity operation

[Sakari Ailus]

Hi Niklas,

On Wed, May 24, 2017 at 02:09:06AM +0200, Niklas Söderlund wrote:
> From: Niklas Söderlund 
> 
> The optional operation can be used by entities to report how it maps its
> fwnode endpoints to media pad numbers. This is useful for devices which
> require advanced mappings of pads.
> 
> Signed-off-by: Niklas Söderlund 
> ---
>  include/media/media-entity.h | 6 ++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/include/media/media-entity.h b/include/media/media-entity.h
> index c7c254c5bca1761b..2aea22b0409d1070 100644
> --- a/include/media/media-entity.h
> +++ b/include/media/media-entity.h
> @@ -21,6 +21,7 @@
>  
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -171,6 +172,9 @@ struct media_pad {
>  
>  /**
>   * struct media_entity_operations - Media entity operations
> + * @pad_from_fwnode: Return the pad number based on a fwnode endpoint.
> + *   This operation can be used to map a fwnode to a
> + *   media pad number. Optional.
>   * @link_setup:  Notify the entity of link changes. The 
> operation can
>   *   return an error, in which case link setup will be
>   *   cancelled. Optional.
> @@ -184,6 +188,8 @@ struct media_pad {
>   *mutex held.
>   */
>  struct media_entity_operations {
> + int (*pad_from_fwnode)(struct fwnode_endpoint *endpoint,
> +unsigned int *pad);

Hmm. How about calling this get_fwnode_pad for instance? I wonder what
others think.

You could just return the pad number still, and a negative value on error. I
think we won't have more than INT_MAX pads. :-)

>   int (*link_setup)(struct media_entity *entity,
> const struct media_pad *local,
> const struct media_pad *remote, u32 flags);

-- 
Regards,

Sakari Ailus
e-mail: sakari.ai...@iki.fi XMPP: sai...@retiisi.org.uk

Re: [PATCH 1/1] [media] i2c: add support for OV13858 sensor

[Sakari Ailus]

Hi Hyungwoo,

Thanks for the patch. Please see my comments below.

On Tue, May 23, 2017 at 04:58:28PM -0700, Hyungwoo Yang wrote:
> This patch adds driver for Omnivision's ov13858
> sensor, the driver supports following features:
> 
> - manual exposure/analog gain
> - two link frequencies
> - media controller support
> - runtime pm support
> 
> Signed-off-by: Hyungwoo Yang 
> ---
>  drivers/media/i2c/Kconfig   |8 +
>  drivers/media/i2c/Makefile  |1 +
>  drivers/media/i2c/ov13858.c | 1853 
> +++
>  3 files changed, 1862 insertions(+)
>  create mode 100644 drivers/media/i2c/ov13858.c
> 
> diff --git a/drivers/media/i2c/Kconfig b/drivers/media/i2c/Kconfig
> index fd181c9..f8c5cca 100644
> --- a/drivers/media/i2c/Kconfig
> +++ b/drivers/media/i2c/Kconfig
> @@ -589,6 +589,14 @@ config VIDEO_OV9650
> This is a V4L2 sensor-level driver for the Omnivision
> OV9650 and OV9652 camera sensors.
>  
> +config VIDEO_OV13858
> + tristate "OmniVision OV13858 sensor support"
> + depends on I2C && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
> + depends on MEDIA_CAMERA_SUPPORT
> + ---help---
> +   This is a Video4Linux2 sensor-level driver for the OmniVision
> +   OV13858 camera.
> +
>  config VIDEO_VS6624
>   tristate "ST VS6624 sensor support"
>   depends on VIDEO_V4L2 && I2C
> diff --git a/drivers/media/i2c/Makefile b/drivers/media/i2c/Makefile
> index 62323ec..3f4dc02 100644
> --- a/drivers/media/i2c/Makefile
> +++ b/drivers/media/i2c/Makefile
> @@ -63,6 +63,7 @@ obj-$(CONFIG_VIDEO_OV5647) += ov5647.o
>  obj-$(CONFIG_VIDEO_OV7640) += ov7640.o
>  obj-$(CONFIG_VIDEO_OV7670) += ov7670.o
>  obj-$(CONFIG_VIDEO_OV9650) += ov9650.o
> +obj-$(CONFIG_VIDEO_OV13858) += ov13858.o
>  obj-$(CONFIG_VIDEO_MT9M032) += mt9m032.o
>  obj-$(CONFIG_VIDEO_MT9M111) += mt9m111.o
>  obj-$(CONFIG_VIDEO_MT9P031) += mt9p031.o
> diff --git a/drivers/media/i2c/ov13858.c b/drivers/media/i2c/ov13858.c
> new file mode 100644
> index 000..7fc735e
> --- /dev/null
> +++ b/drivers/media/i2c/ov13858.c
> @@ -0,0 +1,1853 @@
> +/*
> + * Copyright (c) 2017 Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License version
> + * 2 as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + */
> +
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +#define OV13858_REG_VALUE_08BIT  1
> +#define OV13858_REG_VALUE_16BIT  2
> +#define OV13858_REG_VALUE_24BIT  3
> +
> +#define OV13858_REG_MODE_SELECT  0x0100
> +#define OV13858_MODE_STANDBY 0x00
> +#define OV13858_MODE_STREAMING   0x01
> +
> +#define OV13858_REG_SOFTWARE_RST 0x0103
> +#define OV13858_SOFTWARE_RST 0x01
> +
> +/* PLL1 generates PCLK and MIPI_PHY_CLK */
> +#define OV13858_REG_PLL1_CTRL_0  0x0300
> +#define OV13858_REG_PLL1_CTRL_1  0x0301
> +#define OV13858_REG_PLL1_CTRL_2  0x0302
> +#define OV13858_REG_PLL1_CTRL_3  0x0303
> +#define OV13858_REG_PLL1_CTRL_4  0x0304
> +#define OV13858_REG_PLL1_CTRL_5  0x0305
> +
> +/* PLL2 generates DAC_CLK, SCLK and SRAM_CLK */
> +#define OV13858_REG_PLL2_CTRL_B  0x030b
> +#define OV13858_REG_PLL2_CTRL_C  0x030c
> +#define OV13858_REG_PLL2_CTRL_D  0x030d
> +#define OV13858_REG_PLL2_CTRL_E  0x030e
> +#define OV13858_REG_PLL2_CTRL_F  0x030f
> +#define OV13858_REG_PLL2_CTRL_12 0x0312
> +#define OV13858_REG_MIPI_SC_CTRL00x3016
> +#define OV13858_REG_MIPI_SC_CTRL10x3022
> +
> +/* Chip ID */
> +#define OV13858_REG_CHIP_ID  0x300a
> +#define OV13858_CHIP_ID  0x00d855
> +
> +/* Group access */
> +#define OV13858_REG_GROUP_ACCESS 0x3208
> +#define OV13858_GROUP_HOLD_START(g_id)   (0x00 | (g_id))
> +#define OV13858_GROUP_HOLD_END(g_id) (0x10 | (g_id))
> +#define OV13858_GROUP_LAUNCH(g_id)   (0xa0 | (g_id))
> +#define OV13858_GROUP_FAST_LAUNCH(g_id)  (0xe0 | (g_id))
> +
> +/* H-/V-offset control for expected bayer order */
> +#define OV13858_REG_H_OFFSET 0x3810
> +#define OV13858_REG_V_OFFSET 0x3812
> +
> +/* V-timing internal */
> +#define OV13858_REG_VTS  0x380e
> +#define OV13858_VTS_30FPS0x0c8e /* 30 fps */
> +#define OV13858_VTS_60FPS0x0648 /* 60 fps */
> +#define OV13858_VTS_MAX  0x7fff
> +
> +/* Exposure control */
> +#define OV13858_REG_EXPOSURE 0x3500
> +#define OV13858_EXP_GAIN_MIN 4
> +#define OV13858_EXP_GAIN_MAX 

Re: [Linaro-mm-sig] [PATCH] dma-buf: avoid scheduling on fence status query v2

[Daniel Vetter]

On Wed, May 24, 2017 at 09:47:49AM +1000, Dave Airlie wrote:
> On 28 April 2017 at 07:27, Gustavo Padovan  wrote:
> > 2017-04-26 Christian König :
> >
> >> Am 26.04.2017 um 16:46 schrieb Andres Rodriguez:
> >> > When a timeout of zero is specified, the caller is only interested in
> >> > the fence status.
> >> >
> >> > In the current implementation, dma_fence_default_wait will always call
> >> > schedule_timeout() at least once for an unsignaled fence. This adds a
> >> > significant overhead to a fence status query.
> >> >
> >> > Avoid this overhead by returning early if a zero timeout is specified.
> >> >
> >> > v2: move early return after enable_signaling
> >> >
> >> > Signed-off-by: Andres Rodriguez 
> >>
> >> Reviewed-by: Christian König 
> >
> > pushed to drm-misc-next. Thanks all.
> 
> I don't see this patch in -rc2, where did it end up going?

Queued for 4.13. Makes imo sense since it's just a performance
improvement, not a clear bugfix. But it's in your drm-next, so if you want
to fast-track you can cherry-pick it over:

commit 03c0c5f6641533f5fc14bf4e76d2304197402552
Author: Andres Rodriguez 
Date:   Wed Apr 26 10:46:20 2017 -0400

dma-buf: avoid scheduling on fence status query v2

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v2 17/17] rcar-vin: fix bug in pixelformat selection

[Sergei Shtylyov]

Hello!

On 5/24/2017 3:15 AM, Niklas Söderlund wrote:


From: Niklas Söderlund 

If the requested pixelformat is not supported fallback to the default
format, do not revert the entire format.

Signed-off-by: Niklas Söderlund 
---
 drivers/media/platform/rcar-vin/rcar-v4l2.c | 17 +
 1 file changed, 5 insertions(+), 12 deletions(-)

diff --git a/drivers/media/platform/rcar-vin/rcar-v4l2.c 
b/drivers/media/platform/rcar-vin/rcar-v4l2.c
index de71e5fa8b10cb5e..81ff59c3b4744075 100644
--- a/drivers/media/platform/rcar-vin/rcar-v4l2.c
+++ b/drivers/media/platform/rcar-vin/rcar-v4l2.c

[...]

@@ -218,17 +217,11 @@ static int __rvin_try_format(struct rvin_dev *vin,
if (pix->field == V4L2_FIELD_ANY)
pix->field = vin->format.field;

-   /*
-* Retrieve format information and select the current format if the
-* requested format isn't supported.
-*/
-   info = rvin_format_from_pixel(pix->pixelformat);
-   if (!info) {
-   vin_dbg(vin, "Format %x not found, keeping %x\n",
-   pix->pixelformat, vin->format.pixelformat);
-   *pix = vin->format;
-   pix->width = rwidth;
-   pix->height = rheight;
+   /* If requested format is not supported fallback to the default */
+   if (!rvin_format_from_pixel(pix->pixelformat)) {
+   vin_dbg(vin, "Format 0x%x not found, using default 0x%x\n",


s/0x%x/%#x/, maybe?


+   pix->pixelformat, RVIN_DEFAULT_FORMAT);
+   pix->pixelformat = RVIN_DEFAULT_FORMAT;
}

/* Always recalculate */


MBR, Sergei

Re: [PATCH 1/3] [media] si2157: get chip id during probing

[Antti Palosaari]

On 05/15/2017 11:28 PM, Andreas Kemnade wrote:

Hi,

On Sun, 23 Apr 2017 15:19:21 +0300
Antti Palosaari  wrote:


On 03/16/2017 12:22 AM, Andreas Kemnade wrote:

If the si2157 is behind a e.g. si2168, the si2157 will
at least in some situations not be readable after the si268
got the command 0101. It still accepts commands but the answer
is just ff. So read the chip id before that so the
information is not lost.

The following line in kernel output is a symptome
of that problem:
si2157 7-0063: unknown chip version Si21255-\x\x\x

That is hackish solution :( Somehow I2C reads should be get working
rather than making this kind of work-around. Returning 0xff to i2c reads
means that signal strength also shows some wrong static value?


Also this is needed for the Terratec CinergyTC2.
I see the ff even on windows. So it cannot be solved by usb-sniffing of
a working system, so, again how should we proceed?

a) not support dvb sticks which do not work with your preferred
order of initialization.

b) change order of initialisation (maybe optionally add a flag like
INIT_TUNER_BEFORE_DEMOD to avoid risk of breaking other things)

c) something like the current patch.

d) while(!i2c_readable(tuner)) {
  write_random_data_to_demod();
  write_random_data_it9306_bridge();
}
remember_random_data();


There was not much feedback here.


If it is not possible to fix i2c communication then better to add some 
device specific logic to i2c adapter in order to meet demod/tuner 
requirements.





An excerpt from my windows sniff logs:
ep: 02 l:   15 GEN_I2C_WR 00 0603c61201
ep: 02 l:0
ep: 81 l:0
ep: 81 l:5 042300dcff
ep: 02 l:9 GEN_I2C_RD 00 0603c6
ep: 02 l:0
ep: 81 l:0
ep: 81 l:   11 0a240080ff5b02
ep: 02 l:   15 GEN_I2C_WR 00 0603c6140011070300
ep: 02 l:0
ep: 81 l:0
ep: 81 l:5 042500daff
ep: 02 l:9 GEN_I2C_RD 00 0403c6
ep: 02 l:0
ep: 81 l:0
ep: 81 l:9 08260080ff5901

here you see all the  from the device.



Regards,
Andreas



regards
Antti



--
http://palosaari.fi/