[PATCH v8 15/17] media: v4l: Add Intel IPU3 meta buffer formats
Add IPU3-specific meta formats for processing parameters and
3A statistics.
V4L2_META_FMT_IPU3_PARAMS
V4L2_META_FMT_IPU3_STAT_3A
Signed-off-by: Yong Zhi
Reviewed-by: Laurent Pinchart
---
Documentation/media/uapi/v4l/meta-formats.rst | 1 +
.../media/uapi/v4l/pixfmt-meta-intel-ipu3.rst | 178 +
drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
include/uapi/linux/videodev2.h | 4 +
4 files changed, 185 insertions(+)
create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
diff --git a/Documentation/media/uapi/v4l/meta-formats.rst
b/Documentation/media/uapi/v4l/meta-formats.rst
index 438bd244bd2f..5f956fa784b7 100644
--- a/Documentation/media/uapi/v4l/meta-formats.rst
+++ b/Documentation/media/uapi/v4l/meta-formats.rst
@@ -19,6 +19,7 @@ These formats are used for the :ref:`metadata` interface only.
.. toctree::
:maxdepth: 1
+pixfmt-meta-intel-ipu3
pixfmt-meta-d4xx
pixfmt-meta-uvc
pixfmt-meta-vsp1-hgo
diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
new file mode 100644
index ..8cd30ffbf8b8
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
@@ -0,0 +1,178 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _v4l2-meta-fmt-params:
+.. _v4l2-meta-fmt-stat-3a:
+
+**
+V4L2_META_FMT_IPU3_PARAMS ('ip3p'), V4L2_META_FMT_IPU3_3A ('ip3s')
+**
+
+.. c:type:: ipu3_uapi_stats_3a
+
+3A statistics
+=
+
+For IPU3 ImgU, the 3A statistics accelerators collect different statistics over
+an input bayer frame. Those statistics, defined in data struct
:c:type:`ipu3_uapi_stats_3a`,
+are obtained from "ipu3-imgu 3a stat" metadata capture video node, which are
then
+passed to user space for statistics analysis using :c:type:`v4l2_meta_format`
interface.
+
+The statistics collected are AWB (Auto-white balance) RGBS (Red, Green, Blue
and
+Saturation measure) cells, AWB filter response, AF (Auto-focus) filter
response,
+and AE (Auto-exposure) histogram.
+
+struct :c:type:`ipu3_uapi_4a_config` saves configurable parameters for all
above.
+
+.. code-block:: c
+
+ struct ipu3_uapi_stats_3a {
+ struct ipu3_uapi_awb_raw_buffer awb_raw_buffer;
+ struct ipu3_uapi_ae_raw_buffer_aligned
ae_raw_buffer[IPU3_UAPI_MAX_STRIPES];
+ struct ipu3_uapi_af_raw_buffer af_raw_buffer;
+ struct ipu3_uapi_awb_fr_raw_buffer awb_fr_raw_buffer;
+ struct ipu3_uapi_4a_config stats_4a_config;
+ __u32 ae_join_buffers;
+ __u8 padding[28];
+ struct ipu3_uapi_stats_3a_bubble_info_per_stripe
stats_3a_bubble_per_stripe;
+ struct ipu3_uapi_ff_status stats_3a_status;
+ };
+
+.. c:type:: ipu3_uapi_params
+
+Pipeline parameters
+===
+
+IPU3 pipeline has a number of image processing stages, each of which takes a
+set of parameters as input. The major stages of pipelines are shown here:
+
+Raw pixels -> Bayer Downscaling -> Optical Black Correction ->
+
+Linearization -> Lens Shading Correction -> White Balance / Exposure /
+
+Focus Apply -> Bayer Noise Reduction -> ANR -> Demosaicing -> Color
+
+Correction Matrix -> Gamma correction -> Color Space Conversion ->
+
+Chroma Down Scaling -> Chromatic Noise Reduction -> Total Color
+
+Correction -> XNR3 -> TNR -> DDR
+
+The table below presents a description of the above algorithms.
+
+
===
+NameDescription
+
===
+Optical Black Correction Optical Black Correction block subtracts a pre-defined
+value from the respective pixel values to obtain better
+image quality.
+Defined in :c:type:`ipu3_uapi_obgrid_param`.
+Linearization This algo block uses linearization parameters to
+address non-linearity sensor effects. The Lookup table
+table is defined in
+:c:type:`ipu3_uapi_isp_lin_vmem_params`.
+SHD Lens shading correction is used to correct spatial
+non-uniformity of the pixel response due to optical
+lens shading. This is done by applying a different gain
+for each pixel. The gain, black level etc are
+configured in :c:type:`ipu3_uapi_shd_config_static`.
+BNR Bayer noise reduction block removes image noise by
+applying a bilateral filter.
+
[PATCH v8 12/17] media: staging/intel-ipu3: Add imgu top level pci device driver
This patch adds support for the Intel IPU v3 as found on Skylake and Kaby Lake SoCs. The driver glues v4l2, css(camera sub system) and other pieces together to perform its functions, it also loads the IPU3 firmware binary as part of its initialization. Signed-off-by: Yong Zhi Signed-off-by: Tomasz Figa --- drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/ipu3/Kconfig | 14 + drivers/staging/media/ipu3/Makefile | 11 + drivers/staging/media/ipu3/TODO | 23 + drivers/staging/media/ipu3/ipu3.c | 844 drivers/staging/media/ipu3/ipu3.h | 152 +++ 7 files changed, 1047 insertions(+) create mode 100644 drivers/staging/media/ipu3/Kconfig create mode 100644 drivers/staging/media/ipu3/Makefile create mode 100644 drivers/staging/media/ipu3/TODO create mode 100644 drivers/staging/media/ipu3/ipu3.c create mode 100644 drivers/staging/media/ipu3/ipu3.h diff --git a/drivers/staging/media/Kconfig b/drivers/staging/media/Kconfig index c6f3404dea43..19cadd17e542 100644 --- a/drivers/staging/media/Kconfig +++ b/drivers/staging/media/Kconfig @@ -39,4 +39,6 @@ source "drivers/staging/media/tegra-vde/Kconfig" source "drivers/staging/media/zoran/Kconfig" +source "drivers/staging/media/ipu3/Kconfig" + endif diff --git a/drivers/staging/media/Makefile b/drivers/staging/media/Makefile index 43c7bee1fc8c..edde1960b030 100644 --- a/drivers/staging/media/Makefile +++ b/drivers/staging/media/Makefile @@ -9,3 +9,4 @@ obj-$(CONFIG_VIDEO_SUNXI) += sunxi/ obj-$(CONFIG_TEGRA_VDE)+= tegra-vde/ obj-$(CONFIG_VIDEO_ZORAN) += zoran/ obj-$(CONFIG_VIDEO_ROCKCHIP_VPU) += rockchip/vpu/ +obj-$(CONFIG_VIDEO_IPU3_IMGU) += ipu3/ diff --git a/drivers/staging/media/ipu3/Kconfig b/drivers/staging/media/ipu3/Kconfig new file mode 100644 index ..75cd889f18f7 --- /dev/null +++ b/drivers/staging/media/ipu3/Kconfig @@ -0,0 +1,14 @@ +config VIDEO_IPU3_IMGU + tristate "Intel ipu3-imgu driver" + depends on PCI && VIDEO_V4L2 + depends on MEDIA_CONTROLLER && VIDEO_V4L2_SUBDEV_API + depends on X86 + select IOMMU_IOVA + select VIDEOBUF2_DMA_SG + ---help--- + This is the Video4Linux2 driver for Intel IPU3 image processing unit, + found in Intel Skylake and Kaby Lake SoCs and used for processing + images and video. + + Say Y or M here if you have a Skylake/Kaby Lake SoC with a MIPI + camera. The module will be called ipu3-imgu. diff --git a/drivers/staging/media/ipu3/Makefile b/drivers/staging/media/ipu3/Makefile new file mode 100644 index ..fb146d178bd4 --- /dev/null +++ b/drivers/staging/media/ipu3/Makefile @@ -0,0 +1,11 @@ +# +# Makefile for the IPU3 ImgU drivers +# + +ipu3-imgu-objs += \ + ipu3-mmu.o ipu3-dmamap.o \ + ipu3-tables.o ipu3-css-pool.o \ + ipu3-css-fw.o ipu3-css-params.o \ + ipu3-css.o ipu3-v4l2.o ipu3.o + +obj-$(CONFIG_VIDEO_IPU3_IMGU) += ipu3-imgu.o diff --git a/drivers/staging/media/ipu3/TODO b/drivers/staging/media/ipu3/TODO new file mode 100644 index ..922b885f10a7 --- /dev/null +++ b/drivers/staging/media/ipu3/TODO @@ -0,0 +1,23 @@ +This is a list of things that need to be done to get this driver out of the +staging directory. + +- Request API conversion. Remove of the dual pipeline and associate buffers + as well as formats and the binary used to a request. Remove the + opportunistic buffer management. (Sakari) + +- Using ENABLED and IMMUTABLE link flags for the links where those are + relevant. (Sakari) + +- Prefix imgu for all public APIs, i.e. change ipu3_v4l2_register() to + imgu_v4l2_register(). (Sakari) + +- Use V4L2_CTRL_TYPE_MENU for dual-pipe mode control. (Sakari) + +- IPU3 driver documentation (Laurent) + Add diagram in driver rst to describe output capability. + Comments on configuring v4l2 subdevs for CIO2 and ImgU. + +- uAPI documentation: + Further clarification on some ambiguities such as data type conversion of + IEFD CU inputs. (Sakari) + Move acronyms to doc-rst file. (Mauro) diff --git a/drivers/staging/media/ipu3/ipu3.c b/drivers/staging/media/ipu3/ipu3.c new file mode 100644 index ..3d0a34b86ff4 --- /dev/null +++ b/drivers/staging/media/ipu3/ipu3.c @@ -0,0 +1,844 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2017 - 2018 Intel Corporation + * Copyright 2017 Google LLC + * + * Based on Intel IPU4 driver. + * + */ + +#include +#include +#include +#include + +#include "ipu3.h" +#include "ipu3-dmamap.h" +#include "ipu3-mmu.h" + +#define IMGU_PCI_ID0x1919 +#define IMGU_PCI_BAR 0 +#define IMGU_DMA_MASK DMA_BIT_MASK(39) +#define IMGU_MAX_QUEUE_DEPTH (2 + 2) + +/* + * pre-allocated buffer size for IMGU dummy buf
[PATCH v8 08/17] media: staging/intel-ipu3: css: Compute and program ccs
A collection of routines that are mainly used
to calculate the parameters for accelerator cluster.
Signed-off-by: Yong Zhi
---
drivers/staging/media/ipu3/ipu3-css-params.c | 2915 ++
drivers/staging/media/ipu3/ipu3-css-params.h | 25 +
2 files changed, 2940 insertions(+)
create mode 100644 drivers/staging/media/ipu3/ipu3-css-params.c
create mode 100644 drivers/staging/media/ipu3/ipu3-css-params.h
diff --git a/drivers/staging/media/ipu3/ipu3-css-params.c
b/drivers/staging/media/ipu3/ipu3-css-params.c
new file mode 100644
index ..747352c089dd
--- /dev/null
+++ b/drivers/staging/media/ipu3/ipu3-css-params.c
@@ -0,0 +1,2915 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+
+#include "ipu3-css.h"
+#include "ipu3-css-fw.h"
+#include "ipu3-tables.h"
+
+#define DIV_ROUND_CLOSEST_DOWN(a, b) (((a) + ((b) / 2) - 1) / (b))
+#define roundclosest_down(a, b)(DIV_ROUND_CLOSEST_DOWN(a, b) *
(b))
+
+#define IPU3_UAPI_ANR_MAX_RESET((1 << 12) - 1)
+#define IPU3_UAPI_ANR_MIN_RESET(((-1) << 12) + 1)
+
+struct ipu3_css_scaler_info {
+ unsigned int phase_step;/* Same for luma/chroma */
+ int exp_shift;
+
+ unsigned int phase_init;/* luma/chroma dependent */
+ int pad_left;
+ int pad_right;
+ int crop_left;
+ int crop_top;
+};
+
+static unsigned int ipu3_css_scaler_get_exp(unsigned int counter,
+ unsigned int divider)
+{
+ int i = fls(divider) - fls(counter);
+
+ if (i <= 0)
+ return 0;
+
+ if (divider >> i < counter)
+ i = i - 1;
+
+ return i;
+}
+
+/* Set up the CSS scaler look up table */
+static void
+ipu3_css_scaler_setup_lut(unsigned int taps, unsigned int input_width,
+ unsigned int output_width, int phase_step_correction,
+ const int *coeffs, unsigned int coeffs_size,
+ s8 coeff_lut[], struct ipu3_css_scaler_info *info)
+{
+ int tap, phase, phase_sum_left, phase_sum_right;
+ int exponent = ipu3_css_scaler_get_exp(output_width, input_width);
+ int mantissa = (1 << exponent) * output_width;
+ unsigned int phase_step;
+
+ if (input_width == output_width) {
+ for (phase = 0; phase < IMGU_SCALER_PHASES; phase++) {
+ for (tap = 0; tap < taps; tap++) {
+ coeff_lut[phase * IMGU_SCALER_FILTER_TAPS + tap]
+ = 0;
+ }
+ }
+
+ info->phase_step = IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF);
+ info->exp_shift = 0;
+ info->pad_left = 0;
+ info->pad_right = 0;
+ info->phase_init = 0;
+ info->crop_left = 0;
+ info->crop_top = 0;
+ return;
+ }
+
+ for (phase = 0; phase < IMGU_SCALER_PHASES; phase++) {
+ for (tap = 0; tap < taps; tap++) {
+ /* flip table to for convolution reverse indexing */
+ s64 coeff = coeffs[coeffs_size -
+ ((tap * (coeffs_size / taps)) + phase) - 1];
+ coeff *= mantissa;
+ coeff = div64_long(coeff, input_width);
+
+ /* Add +"0.5" */
+ coeff += 1 << (IMGU_SCALER_COEFF_BITS - 1);
+ coeff >>= IMGU_SCALER_COEFF_BITS;
+
+ coeff_lut[phase * IMGU_SCALER_FILTER_TAPS + tap] =
+ coeff;
+ }
+ }
+
+ phase_step = IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF) *
+ output_width / input_width;
+ phase_step += phase_step_correction;
+ phase_sum_left = (taps / 2 * IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF)) -
+ (1 << (IMGU_SCALER_PHASE_COUNTER_PREC_REF - 1));
+ phase_sum_right = (taps / 2 * IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF)) +
+ (1 << (IMGU_SCALER_PHASE_COUNTER_PREC_REF - 1));
+
+ info->exp_shift = IMGU_SCALER_MAX_EXPONENT_SHIFT - exponent;
+ info->pad_left = (phase_sum_left % phase_step == 0) ?
+ phase_sum_left / phase_step - 1 : phase_sum_left / phase_step;
+ info->pad_right = (phase_sum_right % phase_step == 0) ?
+ phase_sum_right / phase_step - 1 : phase_sum_right / phase_step;
+ info->phase_init = phase_sum_left - phase_step * info->pad_le
[PATCH v8 06/17] media: staging/intel-ipu3: css: Add support for firmware management
Introduce functions to load and install ImgU FW blobs.
Signed-off-by: Yong Zhi
---
drivers/staging/media/ipu3/ipu3-css-fw.c | 264 +++
drivers/staging/media/ipu3/ipu3-css-fw.h | 188 ++
2 files changed, 452 insertions(+)
create mode 100644 drivers/staging/media/ipu3/ipu3-css-fw.c
create mode 100644 drivers/staging/media/ipu3/ipu3-css-fw.h
diff --git a/drivers/staging/media/ipu3/ipu3-css-fw.c
b/drivers/staging/media/ipu3/ipu3-css-fw.c
new file mode 100644
index ..ba459e98d77d
--- /dev/null
+++ b/drivers/staging/media/ipu3/ipu3-css-fw.c
@@ -0,0 +1,264 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+#include
+#include
+#include
+
+#include "ipu3-css.h"
+#include "ipu3-css-fw.h"
+#include "ipu3-dmamap.h"
+
+static void ipu3_css_fw_show_binary(struct device *dev, struct imgu_fw_info
*bi,
+ const char *name)
+{
+ unsigned int i;
+
+ dev_dbg(dev, "found firmware binary type %i size %i name %s\n",
+ bi->type, bi->blob.size, name);
+ if (bi->type != IMGU_FW_ISP_FIRMWARE)
+ return;
+
+ dev_dbg(dev, "id %i mode %i bds 0x%x veceven %i/%i out_pins %i\n",
+ bi->info.isp.sp.id, bi->info.isp.sp.pipeline.mode,
+ bi->info.isp.sp.bds.supported_bds_factors,
+ bi->info.isp.sp.enable.vf_veceven,
+ bi->info.isp.sp.vf_dec.is_variable,
+ bi->info.isp.num_output_pins);
+
+ dev_dbg(dev, "input (%i,%i)-(%i,%i) formats %s%s%s\n",
+ bi->info.isp.sp.input.min_width,
+ bi->info.isp.sp.input.min_height,
+ bi->info.isp.sp.input.max_width,
+ bi->info.isp.sp.input.max_height,
+ bi->info.isp.sp.enable.input_yuv ? "yuv420 " : "",
+ bi->info.isp.sp.enable.input_feeder ||
+ bi->info.isp.sp.enable.input_raw ? "raw8 raw10 " : "",
+ bi->info.isp.sp.enable.input_raw ? "raw12" : "");
+
+ dev_dbg(dev, "internal (%i,%i)\n",
+ bi->info.isp.sp.internal.max_width,
+ bi->info.isp.sp.internal.max_height);
+
+ dev_dbg(dev, "output (%i,%i)-(%i,%i) formats",
+ bi->info.isp.sp.output.min_width,
+ bi->info.isp.sp.output.min_height,
+ bi->info.isp.sp.output.max_width,
+ bi->info.isp.sp.output.max_height);
+ for (i = 0; i < bi->info.isp.num_output_formats; i++)
+ dev_dbg(dev, " %i", bi->info.isp.output_formats[i]);
+ dev_dbg(dev, " vf");
+ for (i = 0; i < bi->info.isp.num_vf_formats; i++)
+ dev_dbg(dev, " %i", bi->info.isp.vf_formats[i]);
+ dev_dbg(dev, "\n");
+}
+
+unsigned int ipu3_css_fw_obgrid_size(const struct imgu_fw_info *bi)
+{
+ unsigned int width = DIV_ROUND_UP(bi->info.isp.sp.internal.max_width,
+ IMGU_OBGRID_TILE_SIZE * 2) + 1;
+ unsigned int height = DIV_ROUND_UP(bi->info.isp.sp.internal.max_height,
+ IMGU_OBGRID_TILE_SIZE * 2) + 1;
+ unsigned int obgrid_size;
+
+ width = ALIGN(width, IPU3_UAPI_ISP_VEC_ELEMS / 4);
+ obgrid_size = PAGE_ALIGN(width * height *
+sizeof(struct ipu3_uapi_obgrid_param)) *
+bi->info.isp.sp.iterator.num_stripes;
+ return obgrid_size;
+}
+
+void *ipu3_css_fw_pipeline_params(struct ipu3_css *css,
+ enum imgu_abi_param_class c,
+ enum imgu_abi_memories m,
+ struct imgu_fw_isp_parameter *par,
+ size_t par_size, void *binary_params)
+{
+ struct imgu_fw_info *bi = >fwp->binary_header[css->current_binary];
+
+ if (par->offset + par->size >
+ bi->info.isp.sp.mem_initializers.params[c][m].size)
+ return NULL;
+
+ if (par->size != par_size)
+ pr_warn("parameter size doesn't match defined size\n");
+
+ if (par->size < par_size)
+ return NULL;
+
+ return binary_params + par->offset;
+}
+
+void ipu3_css_fw_cleanup(struct ipu3_css *css)
+{
+ struct imgu_device *imgu = dev_get_drvdata(css->dev);
+
+ if (css->binary) {
+ unsigned int i;
+
+ for (i = 0; i < css->fwp->file_header.binary_nr; i++)
+ ipu3_dmamap_free(imgu, >binary[i]);
+ kfree(css->binary);
+ }
+ if (css->fw)
+
[PATCH v8 10/17] media: staging/intel-ipu3: Add css pipeline programming
This provides helper library to be used by v4l2 level to program
imaging pipelines and control the streaming.
Signed-off-by: Yong Zhi
---
drivers/staging/media/ipu3/ipu3-css.c | 1740 +
1 file changed, 1740 insertions(+)
diff --git a/drivers/staging/media/ipu3/ipu3-css.c
b/drivers/staging/media/ipu3/ipu3-css.c
index 164830fc91ad..3811ad752e8d 100644
--- a/drivers/staging/media/ipu3/ipu3-css.c
+++ b/drivers/staging/media/ipu3/ipu3-css.c
@@ -16,6 +16,173 @@
IMGU_IRQCTRL_IRQ_SW_PIN(0) | \
IMGU_IRQCTRL_IRQ_SW_PIN(1))
+#define IPU3_CSS_FORMAT_BPP_DEN50 /* Denominator */
+
+/* Some sane limits for resolutions */
+#define IPU3_CSS_MIN_RES 32
+#define IPU3_CSS_MAX_H 3136
+#define IPU3_CSS_MAX_W 4224
+
+/* filter size from graph settings is fixed as 4 */
+#define FILTER_SIZE 4
+#define MIN_ENVELOPE8
+
+/*
+ * pre-allocated buffer size for CSS ABI, auxiliary frames
+ * after BDS and before GDC. Those values should be tuned
+ * to big enough to avoid buffer re-allocation when
+ * streaming to lower streaming latency.
+ */
+#define CSS_ABI_SIZE136
+#define CSS_BDS_SIZE(4480 * 3200 * 3)
+#define CSS_GDC_SIZE(4224 * 3200 * 12 / 8)
+
+#define IPU3_CSS_QUEUE_TO_FLAGS(q) (1 << (q))
+#define IPU3_CSS_FORMAT_FL_IN \
+ IPU3_CSS_QUEUE_TO_FLAGS(IPU3_CSS_QUEUE_IN)
+#define IPU3_CSS_FORMAT_FL_OUT \
+ IPU3_CSS_QUEUE_TO_FLAGS(IPU3_CSS_QUEUE_OUT)
+#define IPU3_CSS_FORMAT_FL_VF \
+ IPU3_CSS_QUEUE_TO_FLAGS(IPU3_CSS_QUEUE_VF)
+
+/* Formats supported by IPU3 Camera Sub System */
+static const struct ipu3_css_format ipu3_css_formats[] = {
+ {
+ .pixelformat = V4L2_PIX_FMT_NV12,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_NV12,
+ .osys_format = IMGU_ABI_OSYS_FORMAT_NV12,
+ .osys_tiling = IMGU_ABI_OSYS_TILING_NONE,
+ .bytesperpixel_num = 1 * IPU3_CSS_FORMAT_BPP_DEN,
+ .chroma_decim = 4,
+ .width_align = IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_OUT | IPU3_CSS_FORMAT_FL_VF,
+ }, {
+ /* Each 32 bytes contains 25 10-bit pixels */
+ .pixelformat = V4L2_PIX_FMT_IPU3_SBGGR10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_BGGR,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ }, {
+ .pixelformat = V4L2_PIX_FMT_IPU3_SGBRG10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_GBRG,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ }, {
+ .pixelformat = V4L2_PIX_FMT_IPU3_SGRBG10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_GRBG,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ }, {
+ .pixelformat = V4L2_PIX_FMT_IPU3_SRGGB10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_RGGB,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ },
+};
+
+static const struct {
+ enum imgu_abi_queue_id qid;
+ size_t ptr_ofs;
+} ipu3_css_queues[IPU3_CSS_QUEUES] = {
+ [IPU3_CSS_QUEUE_IN] = {
+ IMGU_ABI_QUEUE_C_ID,
+ offsetof(struct imgu_abi_buffer, payload.frame.frame_data)
+ },
+ [IPU3_CSS_QUEUE_OUT] = {
+ IMGU_ABI_QUEUE_D_ID,
+ offsetof(struct imgu_abi_buffer, payload.frame.frame_data)
+ },
+ [IPU3_CSS_QUEUE_VF] = {
+ IMGU_ABI_QUEUE_E_ID,
+ offsetof(struct imgu_abi_buffer, payload.frame.frame_data)
+ },
+ [IPU3_CSS_QUEUE_STAT_3A] = {
+ IMGU_ABI_QUEUE_F_ID,
+ offsetof(struct imgu_abi_buffer, payload.s3a.data_ptr)
+ },
+};
+
+/* Initialize queue based on given format, adjust format as needed */
+static int ipu3_css_queue_init(struct ipu3_css_queue
[PATCH v8 01/17] media: staging/intel-ipu3: abi: Add register definitions and enum
Add macros and enums used for IPU3 firmware interface. Signed-off-by: Yong Zhi Signed-off-by: Rajmohan Mani Reviewed-by: Laurent Pinchart --- drivers/staging/media/ipu3/ipu3-abi.h | 661 ++ 1 file changed, 661 insertions(+) create mode 100644 drivers/staging/media/ipu3/ipu3-abi.h diff --git a/drivers/staging/media/ipu3/ipu3-abi.h b/drivers/staging/media/ipu3/ipu3-abi.h new file mode 100644 index ..e754ff5836c2 --- /dev/null +++ b/drivers/staging/media/ipu3/ipu3-abi.h @@ -0,0 +1,661 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2018 Intel Corporation */ + +#ifndef __IPU3_ABI_H +#define __IPU3_ABI_H + +#include "include/intel-ipu3.h" + +/*** IMGU Hardware information ***/ + +typedef u32 imgu_addr_t; + +#define IMGU_ISP_VMEM_ALIGN128 +#define IMGU_DVS_BLOCK_W 64 +#define IMGU_DVS_BLOCK_H 32 +#define IMGU_GDC_BUF_X (2 * IMGU_DVS_BLOCK_W) +#define IMGU_GDC_BUF_Y IMGU_DVS_BLOCK_H +/* n = 0..1 */ +#define IMGU_SP_PMEM_BASE(n) (0x2 + (n) * 0x4000) +#define IMGU_MAX_BQ_GRID_WIDTH 80 +#define IMGU_MAX_BQ_GRID_HEIGHT60 +#define IMGU_OBGRID_TILE_SIZE 16 +#define IMGU_PIXELS_PER_WORD 50 +#define IMGU_BYTES_PER_WORD64 +#define IMGU_STRIPE_FIXED_HALF_OVERLAP 2 +#define IMGU_SHD_SETS 3 +#define IMGU_BDS_MIN_CLIP_VAL 0 +#define IMGU_BDS_MAX_CLIP_VAL 2 + +#define IMGU_ABI_AWB_MAX_CELLS_PER_SET 160 +#define IMGU_ABI_AF_MAX_CELLS_PER_SET 32 +#define IMGU_ABI_AWB_FR_MAX_CELLS_PER_SET 32 + +#define IMGU_ABI_ACC_OP_IDLE 0 +#define IMGU_ABI_ACC_OP_END_OF_ACK 1 +#define IMGU_ABI_ACC_OP_END_OF_OPS 2 +#define IMGU_ABI_ACC_OP_NO_OPS 3 + +#define IMGU_ABI_ACC_OPTYPE_PROCESS_LINES 0 +#define IMGU_ABI_ACC_OPTYPE_TRANSFER_DATA 1 + +/* Register definitions */ + +/* PM_CTRL_0_5_0_IMGHMMADR */ +#define IMGU_REG_PM_CTRL 0x0 +#define IMGU_PM_CTRL_START BIT(0) +#define IMGU_PM_CTRL_CFG_DONE BIT(1) +#define IMGU_PM_CTRL_RACE_TO_HALT BIT(2) +#define IMGU_PM_CTRL_NACK_ALL BIT(3) +#define IMGU_PM_CTRL_CSS_PWRDN BIT(4) +#define IMGU_PM_CTRL_RST_AT_EOFBIT(5) +#define IMGU_PM_CTRL_FORCE_HALTBIT(6) +#define IMGU_PM_CTRL_FORCE_UNHALT BIT(7) +#define IMGU_PM_CTRL_FORCE_PWRDN BIT(8) +#define IMGU_PM_CTRL_FORCE_RESET BIT(9) + +/* SYSTEM_REQ_0_5_0_IMGHMMADR */ +#define IMGU_REG_SYSTEM_REQ0x18 +#define IMGU_SYSTEM_REQ_FREQ_MASK 0x3f +#define IMGU_SYSTEM_REQ_FREQ_DIVIDER 25 +#define IMGU_REG_INT_STATUS0x30 +#define IMGU_REG_INT_ENABLE0x34 +#define IMGU_REG_INT_CSS_IRQ BIT(31) +/* STATE_0_5_0_IMGHMMADR */ +#define IMGU_REG_STATE 0x130 +#define IMGU_STATE_HALT_STSBIT(0) +#define IMGU_STATE_IDLE_STSBIT(1) +#define IMGU_STATE_POWER_UPBIT(2) +#define IMGU_STATE_POWER_DOWN BIT(3) +#define IMGU_STATE_CSS_BUSY_MASK 0xc0 +#define IMGU_STATE_PM_FSM_MASK 0x180 +#define IMGU_STATE_PWRDNM_FSM_MASK 0x1E0 +/* PM_STS_0_5_0_IMGHMMADR */ +#define IMGU_REG_PM_STS0x140 + +#define IMGU_REG_BASE 0x4000 + +#define IMGU_REG_ISP_CTRL (IMGU_REG_BASE + 0x00) +#define IMGU_CTRL_RST BIT(0) +#define IMGU_CTRL_STARTBIT(1) +#define IMGU_CTRL_BREAKBIT(2) +#define IMGU_CTRL_RUN BIT(3) +#define IMGU_CTRL_BROKEN BIT(4) +#define IMGU_CTRL_IDLE BIT(5) +#define IMGU_CTRL_SLEEPING BIT(6) +#define IMGU_CTRL_STALLING BIT(7) +#define IMGU_CTRL_IRQ_CLEARBIT(8) +#define IMGU_CTRL_IRQ_READYBIT(10) +#define IMGU_CTRL_IRQ_SLEEPING BIT(11) +#define IMGU_CTRL_ICACHE_INV BIT(12) +#define IMGU_CTRL_IPREFETCH_EN BIT(13) +#define IMGU_REG_ISP_START_ADDR(IMGU_REG_BASE + 0x04) +#define IMGU_REG_ISP_ICACHE_ADDR (IMGU_REG_BASE + 0x10) +#define IMGU_REG_ISP_PC(IMGU_REG_BASE + 0x1c) + +/* SP Registers, sp = 0:SP0; 1:SP1 */ +#define IMGU_REG_SP_CTRL(sp) (IMGU_REG_BASE + (sp) * 0x100 + 0x100) + /* For bits in IMGU_REG_SP_CTRL, see IMGU_CTRL_* *
[PATCH v8 00/17] Intel IPU3 ImgU patchset
amap.c: Removed dma ops and dependencies on IOMMU_DMA lib. - ipu3-mmu.c: Restructured the driver. - intel-ipu3.h: Added __padding qualifier for uapi definitions. - Internal fix: power and performance related issues. - Fixed v4l2-compliance test. - Fixed build failure for x86 with 32bit config. version 3: - ipu3-mmu.c and ipu3-dmamap.c: Tomasz Figa reworked both drivers and updated related files. - ipu2-abi.h: update imgu_abi_binary_info ABI to support latest ipu3-fw.bin. use __packed qualifier on structs suggested by Sakari Ailus. - ipu3-css-fw.c/ipu3-css-fw.h: following fix were suggested by Tomasz Figa: remove pointer type in firmware blob structs. fix binary_header array in struct imgu_fw_header. fix calling ipu3_css_fw_show_binary() before proper checking. fix logic error for valid length checking of blob name. - ipu3-css-params.c/ipu3_css_scaler_get_exp(): use lib helper suggested by Andy Shevchenko. - ipu3-v4l2.c/ipu3_videoc_querycap(): fill device_caps fix suggested by Hans Verkuil. add VB2_DMABUF suggested by Tomasz Figa. - ipu3-css.c: increase IMGU freq from 300MHZ to 450MHZ (internal fix) - ipu3.c: use vb2_dma_sg_memop for the time being(internal fix). version 2: This version cherry-picked firmware ABI change and other fix in order to bring the code up-to-date with our internal release. I will go over the review comments in v1 and address them in v3 and future update. version 1: - Initial submission Cao,Bing Bu (2): media: staging/intel-ipu3: Add dual pipe support media: v4l2-ctrls: Reserve controls for IPU3 ImgU Rajmohan Mani (1): doc-rst: Add Intel IPU3 documentation Tomasz Figa (2): media: staging/intel-ipu3: mmu: Implement driver media: staging/intel-ipu3: Implement DMA mapping functions Yong Zhi (12): media: staging/intel-ipu3: abi: Add register definitions and enum media: staging/intel-ipu3: abi: Add structs media: staging/intel-ipu3: css: Add dma buff pool utility functions media: staging/intel-ipu3: css: Add support for firmware management media: staging/intel-ipu3: css: Add static settings for image pipeline media: staging/intel-ipu3: css: Compute and program ccs media: staging/intel-ipu3: css: Initialize css hardware media: staging/intel-ipu3: Add css pipeline programming media: staging/intel-ipu3: Add v4l2 driver based on media framework media: staging/intel-ipu3: Add imgu top level pci device driver media: staging/intel-ipu3: Add Intel IPU3 meta data uAPI media: v4l: Add Intel IPU3 meta buffer formats Documentation/media/uapi/v4l/meta-formats.rst |1 + .../media/uapi/v4l/pixfmt-meta-intel-ipu3.rst | 178 + Documentation/media/v4l-drivers/index.rst |1 + Documentation/media/v4l-drivers/ipu3.rst | 326 + drivers/media/v4l2-core/v4l2-ioctl.c |2 + drivers/staging/media/Kconfig |2 + drivers/staging/media/Makefile |1 + drivers/staging/media/ipu3/Kconfig | 14 + drivers/staging/media/ipu3/Makefile| 11 + drivers/staging/media/ipu3/TODO| 23 + drivers/staging/media/ipu3/include/intel-ipu3.h| 2785 ++ drivers/staging/media/ipu3/ipu3-abi.h | 2011 drivers/staging/media/ipu3/ipu3-css-fw.c | 265 + drivers/staging/media/ipu3/ipu3-css-fw.h | 188 + drivers/staging/media/ipu3/ipu3-css-params.c | 2943 ++ drivers/staging/media/ipu3/ipu3-css-params.h | 28 + drivers/staging/media/ipu3/ipu3-css-pool.c | 100 + drivers/staging/media/ipu3/ipu3-css-pool.h | 55 + drivers/staging/media/ipu3/ipu3-css.c | 2391 + drivers/staging/media/ipu3/ipu3-css.h | 213 + drivers/staging/media/ipu3/ipu3-dmamap.c | 270 + drivers/staging/media/ipu3/ipu3-dmamap.h | 22 + drivers/staging/media/ipu3/ipu3-mmu.c | 561 ++ drivers/staging/media/ipu3/ipu3-mmu.h | 35 + drivers/staging/media/ipu3/ipu3-tables.c | 9609 drivers/staging/media/ipu3/ipu3-tables.h | 66 + drivers/staging/media/ipu3/ipu3-v4l2.c | 1419 +++ drivers/staging/media/ipu3/ipu3.c | 830 ++ drivers/staging/media/ipu3/ipu3.h | 168 + include/uapi/linux/v4l2-controls.h |4 + include/uapi/linux/videodev2.h |4 + 31 files changed, 24526 insertions(+) create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst create mode 100644 Documentation/media/v4l-drivers/ipu3.rst create mode 100644 drivers/staging/media/ipu3/Kconfig create mode 100644 drivers/staging/media/ipu3/Makefile create mode 100644 drivers/staging/media/ipu3/TODO create mode 100644 drivers/staging/media/ipu3/include/intel-ipu3.h create mode 100644 drivers/staging/media/ipu3/ipu3-abi.h create mode 100644 drivers/staging/media/ipu3/ipu3-css-fw.c create mode 100
[PATCH v8 11/17] media: staging/intel-ipu3: Add v4l2 driver based on media framework
Implement video driver that utilizes v4l2, vb2 queue support
and media controller APIs. The driver exposes single subdevice and
six nodes.
Signed-off-by: Yong Zhi
---
drivers/staging/media/ipu3/ipu3-v4l2.c | 1086
1 file changed, 1086 insertions(+)
create mode 100644 drivers/staging/media/ipu3/ipu3-v4l2.c
diff --git a/drivers/staging/media/ipu3/ipu3-v4l2.c
b/drivers/staging/media/ipu3/ipu3-v4l2.c
new file mode 100644
index ..038ee749cb75
--- /dev/null
+++ b/drivers/staging/media/ipu3/ipu3-v4l2.c
@@ -0,0 +1,1086 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+#include
+
+#include
+
+#include "ipu3.h"
+#include "ipu3-dmamap.h"
+
+/ v4l2_subdev_ops /
+
+static int ipu3_subdev_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
+{
+ struct v4l2_rect try_crop = {
+ .top = 0,
+ .left = 0,
+ .width = 1920,
+ .height = 1080,
+ };
+ unsigned int i;
+
+ /* Initialize try_fmt */
+ for (i = 0; i < IMGU_NODE_NUM; i++) {
+ struct v4l2_mbus_framefmt *try_fmt =
+ v4l2_subdev_get_try_format(sd, fh->pad, i);
+
+ try_fmt->width = try_crop.width;
+ try_fmt->height = try_crop.height;
+ try_fmt->code = MEDIA_BUS_FMT_FIXED;
+ try_fmt->colorspace = V4L2_COLORSPACE_RAW;
+ try_fmt->field = V4L2_FIELD_NONE;
+ }
+
+ *v4l2_subdev_get_try_crop(sd, fh->pad, IMGU_NODE_IN) = try_crop;
+ *v4l2_subdev_get_try_compose(sd, fh->pad, IMGU_NODE_IN) = try_crop;
+
+ return 0;
+}
+
+static int ipu3_subdev_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct imgu_device *imgu = container_of(sd, struct imgu_device, subdev);
+ int r = 0;
+
+ r = imgu_s_stream(imgu, enable);
+ if (!r)
+ imgu->streaming = enable;
+
+ return r;
+}
+
+static int ipu3_subdev_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
+{
+ struct imgu_device *imgu = container_of(sd, struct imgu_device, subdev);
+ struct v4l2_mbus_framefmt *mf;
+ u32 pad = fmt->pad;
+
+ if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
+ fmt->format = imgu->nodes[pad].pad_fmt;
+ } else {
+ mf = v4l2_subdev_get_try_format(sd, cfg, pad);
+ fmt->format = *mf;
+ }
+
+ return 0;
+}
+
+static int ipu3_subdev_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
+{
+ struct imgu_device *imgu = container_of(sd, struct imgu_device, subdev);
+ struct v4l2_mbus_framefmt *mf;
+ u32 pad = fmt->pad;
+
+ if (fmt->which == V4L2_SUBDEV_FORMAT_TRY)
+ mf = v4l2_subdev_get_try_format(sd, cfg, pad);
+ else
+ mf = >nodes[pad].pad_fmt;
+
+ fmt->format.code = mf->code;
+ /* Clamp the w and h based on the hardware capabilities */
+ if (imgu->subdev_pads[pad].flags & MEDIA_PAD_FL_SOURCE) {
+ fmt->format.width = clamp(fmt->format.width,
+ IPU3_OUTPUT_MIN_WIDTH,
+ IPU3_OUTPUT_MAX_WIDTH);
+ fmt->format.height = clamp(fmt->format.height,
+ IPU3_OUTPUT_MIN_HEIGHT,
+ IPU3_OUTPUT_MAX_HEIGHT);
+ } else {
+ fmt->format.width = clamp(fmt->format.width,
+ IPU3_INPUT_MIN_WIDTH,
+ IPU3_INPUT_MAX_WIDTH);
+ fmt->format.height = clamp(fmt->format.height,
+ IPU3_INPUT_MIN_HEIGHT,
+ IPU3_INPUT_MAX_HEIGHT);
+ }
+
+ *mf = fmt->format;
+
+ return 0;
+}
+
+static int ipu3_subdev_get_selection(struct v4l2_subdev *sd,
+struct v4l2_subdev_pad_config *cfg,
+struct v4l2_subdev_selection *sel)
+{
+ struct imgu_device *imgu = container_of(sd, struct imgu_device, subdev);
+ struct v4l2_rect *try_sel, *r;
+
+ if (sel->pad != IMGU_NODE_IN)
+ return -EINVAL;
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_CROP:
+ try_sel = v4l2_subdev_get_try_crop(sd, cfg, sel->pad);
+ r = >rect.eff;
+ break;
+ case V4L2_SEL_TGT_COMPOSE:
+ try_sel = v4l2_subdev_get_try_compose(sd, cfg, sel->pad);
+
[PATCH v8 13/17] media: staging/intel-ipu3: Add Intel IPU3 meta data uAPI
These meta formats are used on Intel IPU3 ImgU video queues
to carry 3A statistics and ISP pipeline parameters.
V4L2_META_FMT_IPU3_3A
V4L2_META_FMT_IPU3_PARAMS
Signed-off-by: Yong Zhi
Signed-off-by: Chao C Li
Signed-off-by: Rajmohan Mani
---
drivers/staging/media/ipu3/include/intel-ipu3.h | 2775 +++
1 file changed, 2775 insertions(+)
create mode 100644 drivers/staging/media/ipu3/include/intel-ipu3.h
diff --git a/drivers/staging/media/ipu3/include/intel-ipu3.h
b/drivers/staging/media/ipu3/include/intel-ipu3.h
new file mode 100644
index ..07fd66817358
--- /dev/null
+++ b/drivers/staging/media/ipu3/include/intel-ipu3.h
@@ -0,0 +1,2775 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2017 - 2018 Intel Corporation */
+
+#ifndef __IPU3_UAPI_H
+#define __IPU3_UAPI_H
+
+#include
+
+/* from /drivers/staging/media/ipu3/include/videodev2.h */
+
+/* Vendor specific - used for IPU3 camera sub-system */
+#define V4L2_META_FMT_IPU3_PARAMS v4l2_fourcc('i', 'p', '3', 'p') /* IPU3
processing parameters */
+#define V4L2_META_FMT_IPU3_STAT_3A v4l2_fourcc('i', 'p', '3', 's') /* IPU3
3A statistics */
+
+/*** ipu3_uapi_stats_3a ***/
+
+#define IPU3_UAPI_MAX_STRIPES 2
+#define IPU3_UAPI_MAX_BUBBLE_SIZE 10
+
+#define IPU3_UAPI_GRID_START_MASK ((1 << 12) - 1)
+#define IPU3_UAPI_GRID_Y_START_EN (1 << 15)
+
+/* controls generation of meta_data (like FF enable/disable) */
+#define IPU3_UAPI_AWB_RGBS_THR_B_EN(1 << 14)
+#define IPU3_UAPI_AWB_RGBS_THR_B_INCL_SAT (1 << 15)
+
+/**
+ * struct ipu3_uapi_grid_config - Grid plane config
+ *
+ * @width: Grid horizontal dimensions, in number of grid blocks(cells).
+ * @height:Grid vertical dimensions, in number of grid cells.
+ * @block_width_log2: Log2 of the width of each cell in pixels.
+ * for (2^3, 2^4, 2^5, 2^6, 2^7), values [3, 7].
+ * @block_height_log2: Log2 of the height of each cell in pixels.
+ * for (2^3, 2^4, 2^5, 2^6, 2^7), values [3, 7].
+ * @height_per_slice: The number of blocks in vertical axis per slice.
+ * Default 2.
+ * @x_start: X value of top left corner of Region of Interest(ROI).
+ * @y_start: Y value of top left corner of ROI
+ * @x_end: X value of bottom right corner of ROI
+ * @y_end: Y value of bottom right corner of ROI
+ *
+ * Due to the size of total amount of collected data, most statistics
+ * create a grid-based output, and the data is then divided into "slices".
+ */
+struct ipu3_uapi_grid_config {
+ __u8 width;
+ __u8 height;
+ __u16 block_width_log2:3;
+ __u16 block_height_log2:3;
+ __u16 height_per_slice:8;
+ __u16 x_start;
+ __u16 y_start;
+ __u16 x_end;
+ __u16 y_end;
+} __packed;
+
+/*
+ * The grid based data is divided into "slices" called set, each slice of setX
+ * refers to ipu3_uapi_grid_config width * height_per_slice.
+ */
+#define IPU3_UAPI_AWB_MAX_SETS 60
+/* Based on grid size 80 * 60 and cell size 16 x 16 */
+#define IPU3_UAPI_AWB_SET_SIZE 1280
+#define IPU3_UAPI_AWB_MD_ITEM_SIZE 8
+#define IPU3_UAPI_AWB_SPARE_FOR_BUBBLES \
+ (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
+IPU3_UAPI_AWB_MD_ITEM_SIZE)
+#define IPU3_UAPI_AWB_MAX_BUFFER_SIZE \
+ (IPU3_UAPI_AWB_MAX_SETS * \
+(IPU3_UAPI_AWB_SET_SIZE + IPU3_UAPI_AWB_SPARE_FOR_BUBBLES))
+
+
+/**
+ * struct ipu3_uapi_awb_raw_buffer - AWB raw buffer
+ *
+ * @meta_data: buffer to hold auto white balance meta data which is
+ * the average values for each color channel.
+ */
+struct ipu3_uapi_awb_raw_buffer {
+ __u8 meta_data[IPU3_UAPI_AWB_MAX_BUFFER_SIZE]
+ __attribute__((aligned(32)));
+} __packed;
+
+/**
+ * struct ipu3_uapi_awb_config_s - AWB config
+ *
+ * @rgbs_thr_gr: gr threshold value.
+ * @rgbs_thr_r: Red threshold value.
+ * @rgbs_thr_gb: gb threshold value.
+ * @rgbs_thr_b: Blue threshold value.
+ * @grid: _uapi_grid_config, the default grid resolution is 16x16 cells.
+ *
+ * The threshold is a saturation measure range [0, 8191], 8191 is default.
+ * Values over threshold may be optionally rejected for averaging.
+ */
+struct ipu3_uapi_awb_config_s {
+ __u16 rgbs_thr_gr;
+ __u16 rgbs_thr_r;
+ __u16 rgbs_thr_gb;
+ __u16 rgbs_thr_b;
+ struct ipu3_uapi_grid_config grid;
+} __attribute__((aligned(32))) __packed;
+
+/**
+ * struct ipu3_uapi_awb_config - AWB config wrapper
+ *
+ * @config: config for auto white balance as defined by _uapi_awb_config_s
+ */
+struct ipu3_uapi_awb_config {
+ struct ipu3_uapi_awb_config_s config __attribute__((aligned(32)));
+} __packed;
+
+#define IPU3_UAPI_AE_COLORS4
[PATCH v8 05/17] media: staging/intel-ipu3: css: Add dma buff pool utility functions
The pools are used to store previous parameters set by
user with the parameter queue. Due to pipelining,
there needs to be multiple sets (up to four)
of parameters which are queued in a host-to-sp queue.
Signed-off-by: Yong Zhi
---
drivers/staging/media/ipu3/ipu3-css-pool.c | 100 +
drivers/staging/media/ipu3/ipu3-css-pool.h | 55
2 files changed, 155 insertions(+)
create mode 100644 drivers/staging/media/ipu3/ipu3-css-pool.c
create mode 100644 drivers/staging/media/ipu3/ipu3-css-pool.h
diff --git a/drivers/staging/media/ipu3/ipu3-css-pool.c
b/drivers/staging/media/ipu3/ipu3-css-pool.c
new file mode 100644
index ..6f271f81669b
--- /dev/null
+++ b/drivers/staging/media/ipu3/ipu3-css-pool.c
@@ -0,0 +1,100 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+
+#include "ipu3.h"
+#include "ipu3-css-pool.h"
+#include "ipu3-dmamap.h"
+
+int ipu3_css_dma_buffer_resize(struct imgu_device *imgu,
+ struct ipu3_css_map *map, size_t size)
+{
+ if (map->size < size && map->vaddr) {
+ dev_warn(>pci_dev->dev, "dma buf resized from %zu to %zu",
+map->size, size);
+
+ ipu3_dmamap_free(imgu, map);
+ if (!ipu3_dmamap_alloc(imgu, map, size))
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void ipu3_css_pool_cleanup(struct imgu_device *imgu, struct ipu3_css_pool
*pool)
+{
+ unsigned int i;
+
+ for (i = 0; i < IPU3_CSS_POOL_SIZE; i++)
+ ipu3_dmamap_free(imgu, >entry[i].param);
+}
+
+int ipu3_css_pool_init(struct imgu_device *imgu, struct ipu3_css_pool *pool,
+ size_t size)
+{
+ unsigned int i;
+
+ for (i = 0; i < IPU3_CSS_POOL_SIZE; i++) {
+ pool->entry[i].valid = false;
+ if (size == 0) {
+ pool->entry[i].param.vaddr = NULL;
+ continue;
+ }
+
+ if (!ipu3_dmamap_alloc(imgu, >entry[i].param, size))
+ goto fail;
+ }
+
+ pool->last = IPU3_CSS_POOL_SIZE;
+
+ return 0;
+
+fail:
+ ipu3_css_pool_cleanup(imgu, pool);
+ return -ENOMEM;
+}
+
+/*
+ * Allocate a new parameter via recycling the oldest entry in the pool.
+ */
+void ipu3_css_pool_get(struct ipu3_css_pool *pool)
+{
+ /* Get the oldest entry */
+ u32 n = (pool->last + 1) % IPU3_CSS_POOL_SIZE;
+
+ pool->entry[n].valid = true;
+ pool->last = n;
+}
+
+/*
+ * Undo, for all practical purposes, the effect of pool_get().
+ */
+void ipu3_css_pool_put(struct ipu3_css_pool *pool)
+{
+ pool->entry[pool->last].valid = false;
+ pool->last = (pool->last + IPU3_CSS_POOL_SIZE - 1) % IPU3_CSS_POOL_SIZE;
+}
+
+/**
+ * ipu3_css_pool_last - Retrieve the nth pool entry from last
+ *
+ * @pool: a pointer to ipu3_css_pool.
+ * @n: the distance to the last index.
+ *
+ * Returns:
+ * The nth entry from last or null map to indicate no frame stored.
+ */
+const struct ipu3_css_map *
+ipu3_css_pool_last(struct ipu3_css_pool *pool, unsigned int n)
+{
+ static const struct ipu3_css_map null_map = { 0 };
+ int i = (pool->last + IPU3_CSS_POOL_SIZE - n) % IPU3_CSS_POOL_SIZE;
+
+ WARN_ON(n >= IPU3_CSS_POOL_SIZE);
+
+ if (!pool->entry[i].valid)
+ return _map;
+
+ return >entry[i].param;
+}
diff --git a/drivers/staging/media/ipu3/ipu3-css-pool.h
b/drivers/staging/media/ipu3/ipu3-css-pool.h
new file mode 100644
index ..9c895efd2bfa
--- /dev/null
+++ b/drivers/staging/media/ipu3/ipu3-css-pool.h
@@ -0,0 +1,55 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2018 Intel Corporation */
+
+#ifndef __IPU3_UTIL_H
+#define __IPU3_UTIL_H
+
+struct device;
+struct imgu_device;
+
+#define IPU3_CSS_POOL_SIZE 4
+
+/**
+ * ipu3_css_map - store DMA mapping info for buffer
+ *
+ * @size: size of the buffer in bytes.
+ * @vaddr: kernel virtual address.
+ * @daddr: iova dma address to access IPU3.
+ * @vma: private, a pointer to vm_struct,
+ * used for ipu3_dmamap_free.
+ */
+struct ipu3_css_map {
+ size_t size;
+ void *vaddr;
+ dma_addr_t daddr;
+ struct vm_struct *vma;
+};
+
+/**
+ * ipu3_css_pool - circular buffer pool definition
+ *
+ * @entry: array with IPU3_CSS_POOL_SIZE elements.
+ * @entry.param: a ipu3_css_map for storing the mem mapping.
+ * @entry.valid: used to mark if the entry has vaid data.
+ * @last: write pointer, initialized to IPU3_CSS_POOL_SIZE.
+ */
+struct ipu3_css_pool {
+ struct {
+ struct ipu3_css_map param;
+ bool valid;
+ } entry[IPU3_CSS_POOL_SIZE];
[PATCH v8 04/17] media: staging/intel-ipu3: Implement DMA mapping functions
From: Tomasz Figa
This driver uses IOVA space for buffer mapping through IPU3 MMU
to transfer data between imaging pipelines and system DDR.
Signed-off-by: Tomasz Figa
Signed-off-by: Yong Zhi
---
drivers/staging/media/ipu3/ipu3-dmamap.c | 270 +++
drivers/staging/media/ipu3/ipu3-dmamap.h | 22 +++
2 files changed, 292 insertions(+)
create mode 100644 drivers/staging/media/ipu3/ipu3-dmamap.c
create mode 100644 drivers/staging/media/ipu3/ipu3-dmamap.h
diff --git a/drivers/staging/media/ipu3/ipu3-dmamap.c
b/drivers/staging/media/ipu3/ipu3-dmamap.c
new file mode 100644
index ..93a393d4e15e
--- /dev/null
+++ b/drivers/staging/media/ipu3/ipu3-dmamap.c
@@ -0,0 +1,270 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Intel Corporation
+ * Copyright 2018 Google LLC.
+ *
+ * Author: Tomasz Figa
+ * Author: Yong Zhi
+ */
+
+#include
+
+#include "ipu3.h"
+#include "ipu3-css-pool.h"
+#include "ipu3-mmu.h"
+
+/*
+ * Free a buffer allocated by ipu3_dmamap_alloc_buffer()
+ */
+static void ipu3_dmamap_free_buffer(struct page **pages,
+ size_t size)
+{
+ int count = size >> PAGE_SHIFT;
+
+ while (count--)
+ __free_page(pages[count]);
+ kvfree(pages);
+}
+
+/*
+ * Based on the implementation of __iommu_dma_alloc_pages()
+ * defined in drivers/iommu/dma-iommu.c
+ */
+static struct page **ipu3_dmamap_alloc_buffer(size_t size,
+ unsigned long order_mask,
+ gfp_t gfp)
+{
+ struct page **pages;
+ unsigned int i = 0, count = size >> PAGE_SHIFT;
+ const gfp_t high_order_gfp = __GFP_NOWARN | __GFP_NORETRY;
+
+ /* Allocate mem for array of page ptrs */
+ pages = kvmalloc_array(count, sizeof(*pages), GFP_KERNEL);
+
+ if (!pages)
+ return NULL;
+
+ order_mask &= (2U << MAX_ORDER) - 1;
+ if (!order_mask)
+ return NULL;
+
+ gfp |= __GFP_HIGHMEM | __GFP_ZERO;
+
+ while (count) {
+ struct page *page = NULL;
+ unsigned int order_size;
+
+ for (order_mask &= (2U << __fls(count)) - 1;
+order_mask; order_mask &= ~order_size) {
+ unsigned int order = __fls(order_mask);
+
+ order_size = 1U << order;
+ page = alloc_pages((order_mask - order_size) ?
+ gfp | high_order_gfp : gfp, order);
+ if (!page)
+ continue;
+ if (!order)
+ break;
+ if (!PageCompound(page)) {
+ split_page(page, order);
+ break;
+ }
+
+ __free_pages(page, order);
+ }
+ if (!page) {
+ ipu3_dmamap_free_buffer(pages, i << PAGE_SHIFT);
+ return NULL;
+ }
+ count -= order_size;
+ while (order_size--)
+ pages[i++] = page++;
+ }
+
+ return pages;
+}
+
+/**
+ * ipu3_dmamap_alloc - allocate and map a buffer into KVA
+ * @imgu: struct device pointer
+ * @map: struct to store mapping variables
+ * @len: size required
+ *
+ * Returns:
+ * KVA on success
+ * %NULL on failure
+ */
+void *ipu3_dmamap_alloc(struct imgu_device *imgu, struct ipu3_css_map *map,
+ size_t len)
+{
+ unsigned long shift = iova_shift(>iova_domain);
+ unsigned int alloc_sizes = imgu->mmu->pgsize_bitmap;
+ struct device *dev = >pci_dev->dev;
+ size_t size = PAGE_ALIGN(len);
+ struct page **pages;
+ dma_addr_t iovaddr;
+ struct iova *iova;
+ int i, rval;
+
+ dev_dbg(dev, "%s: allocating %zu\n", __func__, size);
+
+ iova = alloc_iova(>iova_domain, size >> shift,
+ imgu->mmu->aperture_end >> shift, 0);
+ if (!iova)
+ return NULL;
+
+ pages = ipu3_dmamap_alloc_buffer(size, alloc_sizes >> PAGE_SHIFT,
+GFP_KERNEL);
+ if (!pages)
+ goto out_free_iova;
+
+ /* Call IOMMU driver to setup pgt */
+ iovaddr = iova_dma_addr(>iova_domain, iova);
+ for (i = 0; i < size / PAGE_SIZE; ++i) {
+ rval = ipu3_mmu_map(imgu->mmu, iovaddr,
+ page_to_phys(pages[i]), PAGE_SIZE);
+ if (rval)
+ goto out_unmap;
+
+ iovaddr += PAGE_SIZE;
+ }
+
+ /* Now grab a virtual region */
+ map->vma = __get_vm_area(size, VM_USERMAP, VMALL
[PATCH v8 02/17] media: staging/intel-ipu3: abi: Add structs
This add all the structs of IPU3 firmware ABI.
Signed-off-by: Yong Zhi
Signed-off-by: Rajmohan Mani
Reviewed-by: Laurent Pinchart
---
drivers/staging/media/ipu3/ipu3-abi.h | 1350 +
1 file changed, 1350 insertions(+)
diff --git a/drivers/staging/media/ipu3/ipu3-abi.h
b/drivers/staging/media/ipu3/ipu3-abi.h
index e754ff5836c2..25be56ff01c8 100644
--- a/drivers/staging/media/ipu3/ipu3-abi.h
+++ b/drivers/staging/media/ipu3/ipu3-abi.h
@@ -658,4 +658,1354 @@ enum imgu_abi_stage_type {
IMGU_ABI_STAGE_TYPE_ISP,
};
+struct imgu_abi_acc_operation {
+ /*
+* zero means on init,
+* others mean upon receiving an ack signal from the BC acc.
+*/
+ u8 op_indicator;
+ u8 op_type;
+} __packed;
+
+struct imgu_abi_acc_process_lines_cmd_data {
+ u16 lines;
+ u8 cfg_set;
+ u8 reserved;/* Align to 4 bytes */
+} __packed;
+
+/* Bayer shading definitions */
+
+struct imgu_abi_shd_transfer_luts_set_data {
+ u8 set_number;
+ u8 padding[3];
+ imgu_addr_t rg_lut_ddr_addr;
+ imgu_addr_t bg_lut_ddr_addr;
+ u32 align_dummy;
+} __packed;
+
+struct imgu_abi_shd_grid_config {
+ /* reg 0 */
+ u32 grid_width:8;
+ u32 grid_height:8;
+ u32 block_width:3;
+ u32 reserved0:1;
+ u32 block_height:3;
+ u32 reserved1:1;
+ u32 grid_height_per_slice:8;
+ /* reg 1 */
+ s32 x_start:13;
+ s32 reserved2:3;
+ s32 y_start:13;
+ s32 reserved3:3;
+} __packed;
+
+struct imgu_abi_shd_general_config {
+ u32 init_set_vrt_offst_ul:8;
+ u32 shd_enable:1;
+ /* aka 'gf' */
+ u32 gain_factor:2;
+ u32 reserved:21;
+} __packed;
+
+struct imgu_abi_shd_black_level_config {
+ /* reg 0 */
+ s32 bl_r:12;
+ s32 reserved0:4;
+ s32 bl_gr:12;
+ u32 reserved1:1;
+ /* aka 'nf' */
+ u32 normalization_shift:3;
+ /* reg 1 */
+ s32 bl_gb:12;
+ s32 reserved2:4;
+ s32 bl_b:12;
+ s32 reserved3:4;
+} __packed;
+
+struct imgu_abi_shd_intra_frame_operations_data {
+ struct imgu_abi_acc_operation
+ operation_list[IMGU_ABI_SHD_MAX_OPERATIONS] __aligned(32);
+ struct imgu_abi_acc_process_lines_cmd_data
+ process_lines_data[IMGU_ABI_SHD_MAX_PROCESS_LINES]
__aligned(32);
+ struct imgu_abi_shd_transfer_luts_set_data
+ transfer_data[IMGU_ABI_SHD_MAX_TRANSFERS] __aligned(32);
+} __packed;
+
+struct imgu_abi_shd_config {
+ struct ipu3_uapi_shd_config_static shd __aligned(32);
+ struct imgu_abi_shd_intra_frame_operations_data shd_ops __aligned(32);
+ struct ipu3_uapi_shd_lut shd_lut __aligned(32);
+} __packed;
+
+struct imgu_abi_stripe_input_frame_resolution {
+ u16 width;
+ u16 height;
+ u32 bayer_order;/* enum ipu3_uapi_bayer_order */
+ u32 raw_bit_depth;
+} __packed;
+
+/* Stripe-based processing */
+
+struct imgu_abi_stripes {
+ /* offset from start of frame - measured in pixels */
+ u16 offset;
+ /* stripe width - measured in pixels */
+ u16 width;
+ /* stripe width - measured in pixels */
+ u16 height;
+} __packed;
+
+struct imgu_abi_stripe_data {
+ /*
+* number of stripes for current processing source
+* - VLIW binary parameter we currently support 1 or 2 stripes
+*/
+ u16 num_of_stripes;
+
+ u8 padding[2];
+
+ /*
+* the following data is derived from resolution-related
+* pipe config and from num_of_stripes
+*/
+
+ /*
+*'input-stripes' - before input cropping
+* used by input feeder
+*/
+ struct imgu_abi_stripe_input_frame_resolution input_frame;
+
+ /*'effective-stripes' - after input cropping used dpc, bds */
+ struct imgu_abi_stripes effective_stripes[IPU3_UAPI_MAX_STRIPES];
+
+ /* 'down-scaled-stripes' - after down-scaling ONLY. used by BDS */
+ struct imgu_abi_stripes down_scaled_stripes[IPU3_UAPI_MAX_STRIPES];
+
+ /*
+*'bds-out-stripes' - after bayer down-scaling and padding.
+* used by all algos starting with norm up to the ref-frame for GDC
+* (currently up to the output kernel)
+*/
+ struct imgu_abi_stripes bds_out_stripes[IPU3_UAPI_MAX_STRIPES];
+
+ /* 'bds-out-stripes (no overlap)' - used for ref kernel */
+ struct imgu_abi_stripes
+ bds_out_stripes_no_overlap[IPU3_UAPI_MAX_STRIPES];
+
+ /*
+* input resolution for output system (equal to bds_out - envelope)
+* output-system input frame width as configured by user
+*/
+ u16 output_system_in_frame_width;
+ /* output-system input frame height as configured by user */
+ u16 output_system_in_frame_height;
+
+ /*
+* 'output-stripes' - accounts for stiching on the output (no overlap
[PATCH v8 03/17] media: staging/intel-ipu3: mmu: Implement driver
From: Tomasz Figa
This driver translates IO virtual address to physical
address based on two levels page tables.
Signed-off-by: Tomasz Figa
Signed-off-by: Yong Zhi
---
drivers/staging/media/ipu3/ipu3-mmu.c | 561 ++
drivers/staging/media/ipu3/ipu3-mmu.h | 35 +++
2 files changed, 596 insertions(+)
create mode 100644 drivers/staging/media/ipu3/ipu3-mmu.c
create mode 100644 drivers/staging/media/ipu3/ipu3-mmu.h
diff --git a/drivers/staging/media/ipu3/ipu3-mmu.c
b/drivers/staging/media/ipu3/ipu3-mmu.c
new file mode 100644
index ..b9f209541f78
--- /dev/null
+++ b/drivers/staging/media/ipu3/ipu3-mmu.c
@@ -0,0 +1,561 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Intel Corporation.
+ * Copyright 2018 Google LLC.
+ *
+ * Author: Tuukka Toivonen
+ * Author: Sakari Ailus
+ * Author: Samu Onkalo
+ * Author: Tomasz Figa
+ *
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+
+#include "ipu3-mmu.h"
+
+#define IPU3_PAGE_SHIFT12
+#define IPU3_PAGE_SIZE (1UL << IPU3_PAGE_SHIFT)
+
+#define IPU3_PT_BITS 10
+#define IPU3_PT_PTES (1UL << IPU3_PT_BITS)
+#define IPU3_PT_SIZE (IPU3_PT_PTES << 2)
+#define IPU3_PT_ORDER (IPU3_PT_SIZE >> PAGE_SHIFT)
+
+#define IPU3_ADDR2PTE(addr)((addr) >> IPU3_PAGE_SHIFT)
+#define IPU3_PTE2ADDR(pte) ((phys_addr_t)(pte) << IPU3_PAGE_SHIFT)
+
+#define IPU3_L2PT_SHIFTIPU3_PT_BITS
+#define IPU3_L2PT_MASK ((1UL << IPU3_L2PT_SHIFT) - 1)
+
+#define IPU3_L1PT_SHIFTIPU3_PT_BITS
+#define IPU3_L1PT_MASK ((1UL << IPU3_L1PT_SHIFT) - 1)
+
+#define IPU3_MMU_ADDRESS_BITS (IPU3_PAGE_SHIFT + \
+IPU3_L2PT_SHIFT + \
+IPU3_L1PT_SHIFT)
+
+#define IMGU_REG_BASE 0x4000
+#define REG_TLB_INVALIDATE (IMGU_REG_BASE + 0x300)
+#define TLB_INVALIDATE 1
+#define REG_L1_PHYS(IMGU_REG_BASE + 0x304) /* 27-bit pfn */
+#define REG_GP_HALT(IMGU_REG_BASE + 0x5dc)
+#define REG_GP_HALTED (IMGU_REG_BASE + 0x5e0)
+
+struct ipu3_mmu {
+ struct device *dev;
+ void __iomem *base;
+ /* protect access to l2pts, l1pt */
+ spinlock_t lock;
+
+ void *dummy_page;
+ u32 dummy_page_pteval;
+
+ u32 *dummy_l2pt;
+ u32 dummy_l2pt_pteval;
+
+ u32 **l2pts;
+ u32 *l1pt;
+
+ struct ipu3_mmu_info geometry;
+};
+
+static inline struct ipu3_mmu *to_ipu3_mmu(struct ipu3_mmu_info *info)
+{
+ return container_of(info, struct ipu3_mmu, geometry);
+}
+
+/**
+ * ipu3_mmu_tlb_invalidate - invalidate translation look-aside buffer
+ * @mmu: MMU to perform the invalidate operation on
+ *
+ * This function invalidates the whole TLB. Must be called when the hardware
+ * is powered on.
+ */
+static void ipu3_mmu_tlb_invalidate(struct ipu3_mmu *mmu)
+{
+ writel(TLB_INVALIDATE, mmu->base + REG_TLB_INVALIDATE);
+}
+
+static void call_if_ipu3_is_powered(struct ipu3_mmu *mmu,
+ void (*func)(struct ipu3_mmu *mmu))
+{
+ if (!pm_runtime_get_if_in_use(mmu->dev))
+ return;
+
+ func(mmu);
+ pm_runtime_put(mmu->dev);
+}
+
+/**
+ * ipu3_mmu_set_halt - set CIO gate halt bit
+ * @mmu: MMU to set the CIO gate bit in.
+ * @halt: Desired state of the gate bit.
+ *
+ * This function sets the CIO gate bit that controls whether external memory
+ * accesses are allowed. Must be called when the hardware is powered on.
+ */
+static void ipu3_mmu_set_halt(struct ipu3_mmu *mmu, bool halt)
+{
+ int ret;
+ u32 val;
+
+ writel(halt, mmu->base + REG_GP_HALT);
+ ret = readl_poll_timeout(mmu->base + REG_GP_HALTED,
+val, (val & 1) == halt, 1000, 10);
+
+ if (ret)
+ dev_err(mmu->dev, "failed to %s CIO gate halt\n",
+ halt ? "set" : "clear");
+}
+
+/**
+ * ipu3_mmu_alloc_page_table - allocate a pre-filled page table
+ * @pteval: Value to initialize for page table entries with.
+ *
+ * Return: Pointer to allocated page table or NULL on failure.
+ */
+static u32 *ipu3_mmu_alloc_page_table(u32 pteval)
+{
+ u32 *pt;
+ int pte;
+
+ pt = (u32 *)__get_free_page(GFP_KERNEL);
+ if (!pt)
+ return NULL;
+
+ for (pte = 0; pte < IPU3_PT_PTES; pte++)
+ pt[pte] = pteval;
+
+ set_memory_uc((unsigned long int)pt, IPU3_PT_ORDER);
+
+ return pt;
+}
+
+/**
+ * ipu3_mmu_free_page_table - free page table
+ * @pt: Page table to free.
+ */
+static void ipu3_mmu_free_page_table(u32 *pt)
+{
+ set_memory_wb((unsigned long int)pt, IPU3_PT_ORDER);
+ free_page((unsigned long)pt);
+}
+
+/**
+ * address_to_pte_idx - split IOVA into L1 and L2 page table indices
+ * @io
[PATCH v8 17/17] doc-rst: Add Intel IPU3 documentation
From: Rajmohan Mani
This patch adds the details about the IPU3 Imaging Unit driver.
Change-Id: I560cecf673df2dcc3ec72767cf8077708d649656
Signed-off-by: Rajmohan Mani
---
Documentation/media/v4l-drivers/index.rst | 1 +
Documentation/media/v4l-drivers/ipu3.rst | 326 ++
2 files changed, 327 insertions(+)
create mode 100644 Documentation/media/v4l-drivers/ipu3.rst
diff --git a/Documentation/media/v4l-drivers/index.rst
b/Documentation/media/v4l-drivers/index.rst
index 6cdd3bc98202..f28570ec9e42 100644
--- a/Documentation/media/v4l-drivers/index.rst
+++ b/Documentation/media/v4l-drivers/index.rst
@@ -44,6 +44,7 @@ For more details see the file COPYING in the source
distribution of Linux.
davinci-vpbe
fimc
imx
+ ipu3
ivtv
max2175
meye
diff --git a/Documentation/media/v4l-drivers/ipu3.rst
b/Documentation/media/v4l-drivers/ipu3.rst
new file mode 100644
index ..045bf4222b1a
--- /dev/null
+++ b/Documentation/media/v4l-drivers/ipu3.rst
@@ -0,0 +1,326 @@
+.. include::
+
+===
+Intel Image Processing Unit 3 (IPU3) Imaging Unit (ImgU) driver
+===
+
+Copyright |copy| 2018 Intel Corporation
+
+Introduction
+
+
+This file documents Intel IPU3 (3rd generation Image Processing Unit) Imaging
+Unit driver located under drivers/media/pci/intel/ipu3.
+
+The Intel IPU3 found in certain Kaby Lake (as well as certain Sky Lake)
+platforms (U/Y processor lines) is made up of two parts namely Imaging Unit
+(ImgU) and CIO2 device (MIPI CSI2 receiver).
+
+The CIO2 device receives the raw bayer data from the sensors and outputs the
+frames in a format that is specific to IPU3 (for consumption by IPU3 ImgU).
+CIO2 driver is available as drivers/media/pci/intel/ipu3/ipu3-cio2* and is
+enabled through the CONFIG_VIDEO_IPU3_CIO2 config option.
+
+The Imaging Unit (ImgU) is responsible for processing images captured
+through IPU3 CIO2 device. The ImgU driver sources can be found under
+drivers/media/pci/intel/ipu3 directory. The driver is enabled through the
+CONFIG_VIDEO_IPU3_IMGU config option.
+
+The two driver modules are named ipu3-csi2 and ipu3-imgu, respectively.
+
+The driver has been tested on Kaby Lake platforms (U/Y processor lines).
+
+The driver implements V4L2, Media controller and V4L2 sub-device interfaces.
+Camera sensors that have CSI-2 bus, which are connected to the IPU3 CIO2
+device are supported. Support for lens and flash drivers depends on the
+above sensors.
+
+ImgU device nodes
+=
+
+The ImgU is represented as two V4L2 subdevs, each of which provides a V4L2
+subdev interface to the user space.
+
+Each V4L2 subdev represents a pipe, which can support a maximum of 2
+streams. A private ioctl can be used to configure the mode (video or still)
+of the pipe.
+
+This helps to support advanced camera features like Continuous View Finder
+(CVF) and Snapshot During Video(SDV).
+
+CIO2 device
+===
+
+The CIO2 is represented as a single V4L2 subdev, which provides a V4L2 subdev
+interface to the user space. There is a video node for each CSI-2 receiver,
+with a single media controller interface for the entire device.
+
+Media controller
+
+
+The media device interface allows to configure the ImgU links, which defines
+the behavior of the IPU3 firmware.
+
+Device operation
+
+
+With IPU3, once the input video node ("ipu3-imgu 0/1":0,
+in : format) is queued with buffer (in packed raw bayer
+format), IPU3 ISP starts processing the buffer and produces the video output
+in YUV format and statistics output on respective output nodes. The driver
+is expected to have buffers ready for all of parameter, output and
+statistics nodes, when input video node is queued with buffer.
+
+At a minimum, all of input, main output, 3A statistics and viewfinder
+video nodes should be enabled for IPU3 to start image processing.
+
+Each ImgU V4L2 subdev has the following set of video nodes.
+
+input, output and viewfinder video nodes
+
+
+The frames (in packed raw bayer format specific to IPU3) received by the
+input video node is processed by the IPU3 Imaging Unit and is output to 2
+video nodes, with each targeting different purpose (main output and viewfinder
+output).
+
+Details on raw bayer format specific to IPU3 can be found as below.
+Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
+
+The driver supports V4L2 Video Capture Interface as defined at :ref:`devices`.
+
+Only the multi-planar API is supported. More details can be found at
+:ref:`planar-apis`.
+
+
+parameters video node
+-
+
+The parameter video node receives the ISP algorithm parameters that are used
+to configure how the ISP algorithms process the image.
+
+Details on raw bayer format specific to IPU3 can be found as
[PATCH v8 16/17] media: v4l2-ctrls: Reserve controls for IPU3 ImgU
From: "Cao,Bing Bu"
Add a base to be used for allocation of all the IPU3 specific
controls in the ImgU driver.
Signed-off-by: Yong Zhi
Signed-off-by: Tian Shu Qiu
---
include/uapi/linux/v4l2-controls.h | 4
1 file changed, 4 insertions(+)
diff --git a/include/uapi/linux/v4l2-controls.h
b/include/uapi/linux/v4l2-controls.h
index 998983a6e6b7..2b52f44d0ba5 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -192,6 +192,10 @@ enum v4l2_colorfx {
* We reserve 16 controls for this driver. */
#define V4L2_CID_USER_IMX_BASE (V4L2_CID_USER_BASE + 0x10b0)
+/* The base for the ipu3 ImgU driver controls.
+ * We reserve 16 controls for this driver. */
+#define V4L2_CID_INTEL_IPU3_BASE (V4L2_CID_USER_BASE + 0x10c0)
+
/* MPEG-class control IDs */
/* The MPEG controls are applicable to all codec controls
* and the 'MPEG' part of the define is historical */
--
2.7.4
RE: [PATCH v7 01/16] v4l: Add Intel IPU3 meta buffer formats
Hi, Laurent,
Thanks for the review.
> -Original Message-
> From: Laurent Pinchart [mailto:laurent.pinch...@ideasonboard.com]
> Sent: Thursday, November 29, 2018 1:17 PM
> To: Zhi, Yong
> Cc: linux-media@vger.kernel.org; sakari.ai...@linux.intel.com;
> tf...@chromium.org; mche...@kernel.org; hans.verk...@cisco.com; Mani,
> Rajmohan ; Zheng, Jian Xu
> ; Hu, Jerry W ; Toivonen,
> Tuukka ; Qiu, Tian Shu
> ; Cao, Bingbu
> Subject: Re: [PATCH v7 01/16] v4l: Add Intel IPU3 meta buffer formats
>
> Hello Yong,
>
> Thank you for the patch.
>
> On Tuesday, 30 October 2018 00:22:55 EET Yong Zhi wrote:
> > Add IPU3-specific meta formats for parameter processing and 3A, DVS
> > statistics:
>
> Unless I'm mistaken DVS support has been removed. You can write this as
>
> Add IPU3-specific meta formats for processing parameters and 3A statistics.
>
Ack.
> >
> > V4L2_META_FMT_IPU3_PARAMS
> > V4L2_META_FMT_IPU3_STAT_3A
> >
> > Signed-off-by: Yong Zhi
> > ---
> > drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
> > include/uapi/linux/videodev2.h | 4
> > 2 files changed, 6 insertions(+)
>
> I would squash this with patch 03/16.
>
OK, will squash patch 01 and 03 into single patch for v8.
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
> > b/drivers/media/v4l2-core/v4l2-ioctl.c index 6489f25..abff64b 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1299,6 +1299,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
> *fmt)
> > case V4L2_META_FMT_VSP1_HGO:descr = "R-Car VSP1 1-D Histogram";
> break;
> > case V4L2_META_FMT_VSP1_HGT:descr = "R-Car VSP1 2-D Histogram";
> break;
> > case V4L2_META_FMT_UVC: descr = "UVC payload header
> metadata"; break;
> > + case V4L2_META_FMT_IPU3_PARAMS: descr = "IPU3 processing
> parameters";
> > break;
> > + case V4L2_META_FMT_IPU3_STAT_3A:descr = "IPU3 3A statistics";
> break;
> >
> > default:
> > /* Compressed formats */
> > diff --git a/include/uapi/linux/videodev2.h
> > b/include/uapi/linux/videodev2.h index f0a968a..bdccd7a 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -718,6 +718,10 @@ struct v4l2_pix_format {
> > #define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC
> > Payload Header metadata */ #define V4L2_META_FMT_D4XX
> > v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> >
> > +/* Vendor specific - used for IPU3 camera sub-system */
> > +#define V4L2_META_FMT_IPU3_PARAMS v4l2_fourcc('i', 'p', '3', 'p') /*
> IPU3
> > params */
>
> Maybe "IPU3 processing parameters" in full ?
>
Ack, thanks!!
> Apart from that the patch looks good to me.
>
> Reviewed-by: Laurent Pinchart
>
> > +#define V4L2_META_FMT_IPU3_STAT_3A v4l2_fourcc('i', 'p', '3', 's') /*
> IPU3
> > 3A statistics */
> > +
> > /* priv field value to indicates that subsequent fields are valid. */
> > #define V4L2_PIX_FMT_PRIV_MAGIC0xfeedcafe
>
>
> --
> Regards,
>
> Laurent Pinchart
>
>
RE: [PATCH v7 03/16] v4l: Add Intel IPU3 meta data uAPI
Hi, Sakari,
> -Original Message-
> From: Sakari Ailus [mailto:sakari.ai...@linux.intel.com]
> Sent: Thursday, November 29, 2018 4:46 PM
> To: Zhi, Yong
> Cc: linux-media@vger.kernel.org; tf...@chromium.org;
> mche...@kernel.org; hans.verk...@cisco.com;
> laurent.pinch...@ideasonboard.com; Mani, Rajmohan
> ; Zheng, Jian Xu ; Hu,
> Jerry W ; Toivonen, Tuukka
> ; Qiu, Tian Shu ; Cao,
> Bingbu ; Li, Chao C
> Subject: Re: [PATCH v7 03/16] v4l: Add Intel IPU3 meta data uAPI
>
> Hi Yong,
>
> On Fri, Nov 16, 2018 at 10:37:00PM +, Zhi, Yong wrote:
> ...
> > > > +/**
> > > > + * struct ipu3_uapi_shd_grid_config - Bayer shading(darkening)
> > > > +correction
> > > > + *
> > > > + * @width: Grid horizontal dimensions, u8, [8, 128], default 73
> > > > + * @height:Grid vertical dimensions, u8, [8, 128], default 56
> > > > + * @block_width_log2: Log2 of the width of the grid cell in pixel
> > > count
> > > > + * u4, [0, 15], default value 5.
> > > > + * @__reserved0: reserved
> > > > + * @block_height_log2: Log2 of the height of the grid cell in pixel
> > > count
> > > > + * u4, [0, 15], default value 6.
> > > > + * @__reserved1: reserved
> > > > + * @grid_height_per_slice: SHD_MAX_CELLS_PER_SET/width.
> > > > + * (with SHD_MAX_CELLS_PER_SET =
> 146).
> > > > + * @x_start: X value of top left corner of sensor relative to ROI
> > > > + * u12, [-4096, 0]. default 0, only negative values.
> > > > + * @y_start: Y value of top left corner of sensor relative to ROI
> > > > + * u12, [-4096, 0]. default 0, only negative values.
> > >
> > > I suppose u12 is incorrect here, if the value is signed --- and
> > > negative (sign bit) if not 0?
> > >
> >
> > The value will be written to 13 bit register, should use s12.0.
>
> If you have s12, that means the most significant bit is the sign bit. So if
> the
> smallest value is -4096, you'd need s13.
>
> But where is the sign bit, i.e. is this either s13 or s16?
>
The notation of s12.0 means 13 bit with fraction bit as 0 right?
> >
> > > > + */
> > > > +struct ipu3_uapi_shd_grid_config {
> > > > + /* reg 0 */
> > > > + __u8 width;
> > > > + __u8 height;
> > > > + __u8 block_width_log2:3;
> > > > + __u8 __reserved0:1;
> > > > + __u8 block_height_log2:3;
> > > > + __u8 __reserved1:1;
> > > > + __u8 grid_height_per_slice;
> > > > + /* reg 1 */
> > > > + __s16 x_start;
> > > > + __s16 y_start;
> > > > +} __packed;
>
> ...
>
> > > > +/**
> > > > + * struct ipu3_uapi_iefd_cux2_1 - Calculate power of non-directed
> denoise
> > > > + * element apply.
> > > > + * @x0: X0 point of Config Unit, u9.0, default 0.
> > > > + * @x1: X1 point of Config Unit, u9.0, default 0.
> > > > + * @a01: Slope A of Config Unit, s4.4, default 0.
> > >
> > > The field is marked unsigned below. Which one is correct?
> > >
> >
> > They are both correct, however, s4.4 is the internal representation
> > used by CU, the inputs are unsigned, I will add a note in v8, same
> > applies to the few other places as you commented.
>
> I still find this rather confusing. Is there a sign bit or is there not?
>
It's unsigned number from driver perspective, all CU inputs are unsigned,
however, they will be "converted" to signed for FW/HW to use. I have to consult
FW expert if more clarification is needed.
> >
> > > > + * @__reserved1: reserved
> > > > + * @b01: offset B0 of Config Unit, u7.0, default 0.
> > > > + * @__reserved2: reserved
> > > > + */
> > > > +struct ipu3_uapi_iefd_cux2_1 {
> > > > + __u32 x0:9;
> > > > + __u32 x1:9;
> > > > + __u32 a01:9;
> > > > + __u32 __reserved1:5;
> > > > +
> > > > + __u32 b01:8;
> > > > + __u32 __reserved2:24;
> > > > +} __packed;
> > > > +
> > > > +/**
> > > > + * struct ipu3_uapi_iefd_cux4 - Calculate power of non-directed
> > > sharpening
> > > > + * element.
> > > > + *
> > &g
RE: [PATCH v7 03/16] v4l: Add Intel IPU3 meta data uAPI
Hi, Sakari,
Thanks for the thorough review.
> -Original Message-
> From: Sakari Ailus [mailto:sakari.ai...@linux.intel.com]
> Sent: Friday, November 2, 2018 8:03 AM
> To: Zhi, Yong
> Cc: linux-media@vger.kernel.org; tf...@chromium.org;
> mche...@kernel.org; hans.verk...@cisco.com;
> laurent.pinch...@ideasonboard.com; Mani, Rajmohan
> ; Zheng, Jian Xu ; Hu,
> Jerry W ; Toivonen, Tuukka
> ; Qiu, Tian Shu ; Cao,
> Bingbu ; Li, Chao C
> Subject: Re: [PATCH v7 03/16] v4l: Add Intel IPU3 meta data uAPI
>
> Hi Yong,
>
> Thanks for the update! I went through this again... a few comments below
> but I'd say they're mostly pretty minor issues.
>
> On Mon, Oct 29, 2018 at 03:22:57PM -0700, Yong Zhi wrote:
> > These meta formats are used on Intel IPU3 ImgU video queues
> > to carry 3A statistics and ISP pipeline parameters.
> >
> > V4L2_META_FMT_IPU3_3A
> > V4L2_META_FMT_IPU3_PARAMS
> >
> > Signed-off-by: Yong Zhi
> > Signed-off-by: Chao C Li
> > Signed-off-by: Rajmohan Mani
> > ---
> > Documentation/media/uapi/v4l/meta-formats.rst |1 +
> > .../media/uapi/v4l/pixfmt-meta-intel-ipu3.rst | 181 ++
> > include/uapi/linux/intel-ipu3.h| 2819
> >
> > 3 files changed, 3001 insertions(+)
> > create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-intel-
> ipu3.rst
> > create mode 100644 include/uapi/linux/intel-ipu3.h
> >
> > diff --git a/Documentation/media/uapi/v4l/meta-formats.rst
> b/Documentation/media/uapi/v4l/meta-formats.rst
> > index cf971d5..eafc534 100644
> > --- a/Documentation/media/uapi/v4l/meta-formats.rst
> > +++ b/Documentation/media/uapi/v4l/meta-formats.rst
> > @@ -12,6 +12,7 @@ These formats are used for the :ref:`metadata`
> interface only.
> > .. toctree::
> > :maxdepth: 1
> >
> > +pixfmt-meta-intel-ipu3
> > pixfmt-meta-d4xx
> > pixfmt-meta-uvc
> > pixfmt-meta-vsp1-hgo
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
> b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
> > new file mode 100644
> > index 000..23b945b
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
> > @@ -0,0 +1,181 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _intel-ipu3:
>
> Instead, to avoid a warning from Sphinx, replace the line with these:
>
> .. _v4l2-meta-fmt-ipu3-params:
> .. _v4l2-meta-fmt-ipu3-stat-3a:
>
Ack.
> > +
> >
> +***
> ***
> > +V4L2_META_FMT_IPU3_PARAMS ('ip3p'), V4L2_META_FMT_IPU3_3A
> ('ip3s')
> >
> +***
> ***
> > +
> > +.. c:type:: ipu3_uapi_stats_3a
> > +
> > +3A statistics
> > +=
> > +
> > +For IPU3 ImgU, the 3A statistics accelerators collect different statistics
> > over
> > +an input bayer frame. Those statistics, defined in data struct
> > +:c:type:`ipu3_uapi_stats_3a`, are meta output obtained from "ipu3-imgu
> 3a stat"
> > +video node, which are then passed to user space for statistics analysis
> > +using :c:type:`v4l2_meta_format` interface.
> > +
> > +The statistics collected are AWB (Auto-white balance) RGBS (Red, Green,
> Blue and
>
> Extra whitespace at the end of the line.
>
Ack.
> > +Saturation measure) cells, AWB filter response, AF (Auto-focus) filter
> response,
> > +and AE (Auto-exposure) histogram.
> > +
> > +struct :c:type:`ipu3_uapi_4a_config` saves configurable parameters for all
> above.
> > +
> > +
> > +.. code-block:: c
> > +
> > +
> > + struct ipu3_uapi_stats_3a {
> > + struct ipu3_uapi_awb_raw_buffer awb_raw_buffer
> > +__attribute__((aligned(32)));
> > + struct ipu3_uapi_ae_raw_buffer_aligned
> > + ae_raw_buffer[IPU3_UAPI_MAX_STRIPES];
> > + struct ipu3_uapi_af_raw_buffer af_raw_buffer;
> > + struct ipu3_uapi_awb_fr_raw_buffer awb_fr_raw_buffer;
> > + struct ipu3_uapi_4a_config stats_4a_config;
> > + __u32 ae_join_buffers;
> > + __u8 padding[28];
> > + struct ipu3_uapi_stats_3a_bubble_info_per_stripe
> > + stats_3a_bubble_per_stripe;
>
> I think you could just unwrap these, even if it causes them to be over 80
> characters per line. They display better in a web browser that way. Or
> alternatively align the wrapped lines to the same column.
>
Ack.
> > + struct ipu3_u
RE: [PATCH v7 14/16] intel-ipu3: Add v4l2 driver based on media framework
Hi, Hans,
Thanks for the review.
> -Original Message-
> From: Hans Verkuil [mailto:hverk...@xs4all.nl]
> Sent: Thursday, November 15, 2018 6:51 AM
> To: Zhi, Yong ; linux-media@vger.kernel.org;
> sakari.ai...@linux.intel.com
> Cc: tf...@chromium.org; mche...@kernel.org; hans.verk...@cisco.com;
> laurent.pinch...@ideasonboard.com; Mani, Rajmohan
> ; Zheng, Jian Xu ; Hu,
> Jerry W ; Toivonen, Tuukka
> ; Qiu, Tian Shu ; Cao,
> Bingbu
> Subject: Re: [PATCH v7 14/16] intel-ipu3: Add v4l2 driver based on media
> framework
>
> On 10/29/18 23:23, Yong Zhi wrote:
> > Implement video driver that utilizes v4l2, vb2 queue support and media
> > controller APIs. The driver exposes single subdevice and six nodes.
> >
> > Signed-off-by: Yong Zhi
> > ---
> > drivers/media/pci/intel/ipu3/ipu3-v4l2.c | 1091
> > ++
> > 1 file changed, 1091 insertions(+)
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-v4l2.c
> >
> > diff --git a/drivers/media/pci/intel/ipu3/ipu3-v4l2.c
> > b/drivers/media/pci/intel/ipu3/ipu3-v4l2.c
>
>
>
> > +int ipu3_v4l2_register(struct imgu_device *imgu) {
>
>
>
> > + /* Initialize vbq */
> > + vbq->type = node->vdev_fmt.type;
> > + vbq->io_modes = VB2_USERPTR | VB2_MMAP |
> VB2_DMABUF;
>
> Are you sure USERPTR works? If you have alignment requirements that the
> buffer starts at a multiple of more than (I think) 8 bytes, then USERPTR won't
> work.
USRPTR was used at the beginning of project, we then switched to dma buffer
mainly for performance reason:
https://chromium-review.googlesource.com/c/chromiumos/platform/arc-camera/+/620252
>
> > + vbq->ops = _vb2_ops;
> > + vbq->mem_ops = _dma_sg_memops;
> > + if (imgu->buf_struct_size <= 0)
> > + imgu->buf_struct_size = sizeof(struct
> ipu3_vb2_buffer);
> > + vbq->buf_struct_size = imgu->buf_struct_size;
> > + vbq->timestamp_flags =
> V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> > + vbq->min_buffers_needed = 0;/* Can streamon w/o buffers
> */
> > + vbq->drv_priv = imgu;
> > + vbq->lock = >lock;
> > + r = vb2_queue_init(vbq);
> > + if (r) {
> > + dev_err(>pci_dev->dev,
> > + "failed to initialize video queue (%d)\n", r);
> > + goto fail_vdev;
> > + }
> > +
> > + /* Initialize vdev */
> > + snprintf(vdev->name, sizeof(vdev->name), "%s %s",
> > +IMGU_NAME, node->name);
> > + vdev->release = video_device_release_empty;
> > + vdev->fops = _v4l2_fops;
> > + vdev->lock = >lock;
> > + vdev->v4l2_dev = >v4l2_dev;
> > + vdev->queue = >vbq;
> > + vdev->vfl_dir = node->output ? VFL_DIR_TX : VFL_DIR_RX;
> > + video_set_drvdata(vdev, imgu);
> > + r = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
> > + if (r) {
> > + dev_err(>pci_dev->dev,
> > + "failed to register video device (%d)\n", r);
> > + goto fail_vdev;
> > + }
> > +
> > + /* Create link between video node and the subdev pad */
> > + flags = 0;
> > + if (node->enabled)
> > + flags |= MEDIA_LNK_FL_ENABLED;
> > + if (node->immutable)
> > + flags |= MEDIA_LNK_FL_IMMUTABLE;
> > + if (node->output) {
> > + r = media_create_pad_link(>entity, 0,
> > + >subdev.entity,
> > +i, flags);
> > + } else {
> > + r = media_create_pad_link(>subdev.entity,
> > + i, >entity, 0, flags);
> > + }
> > + if (r)
> > + goto fail_link;
> > + }
> > +
> > + r = media_device_register(>media_dev);
> > + if (r) {
> > + dev_err(>pci_dev->dev,
> > + "failed to register media device (%d)\n", r);
> > + i--;
> > + goto fail_link;
> > + }
> > +
> > + return 0;
> > +
> > + for (; i >
RE: [PATCH v7 15/16] intel-ipu3: Add imgu top level pci device driver
Hi, Sakari, Thanks again for the code review. > -Original Message- > From: Sakari Ailus [mailto:sakari.ai...@linux.intel.com] > Sent: Friday, November 9, 2018 6:54 AM > To: Zhi, Yong > Cc: linux-media@vger.kernel.org; tf...@chromium.org; > mche...@kernel.org; hans.verk...@cisco.com; > laurent.pinch...@ideasonboard.com; Mani, Rajmohan > ; Zheng, Jian Xu ; Hu, > Jerry W ; Toivonen, Tuukka > ; Qiu, Tian Shu ; Cao, > Bingbu > Subject: Re: [PATCH v7 15/16] intel-ipu3: Add imgu top level pci device driver > > Hi Yong, > > On Mon, Oct 29, 2018 at 03:23:09PM -0700, Yong Zhi wrote: > > This patch adds support for the Intel IPU v3 as found on Skylake and > > Kaby Lake SoCs. > > > > The driver glues v4l2, css(camera sub system) and other pieces > > together to perform its functions, it also loads the IPU3 firmware > > binary as part of its initialization. > > > > Signed-off-by: Yong Zhi > > Signed-off-by: Tomasz Figa > > --- > > drivers/media/pci/intel/ipu3/Kconfig | 16 + > > drivers/media/pci/intel/ipu3/Makefile | 12 + > > drivers/media/pci/intel/ipu3/ipu3.c | 844 > ++ > > drivers/media/pci/intel/ipu3/ipu3.h | 153 ++ > > 4 files changed, 1025 insertions(+) > > create mode 100644 drivers/media/pci/intel/ipu3/ipu3.c > > create mode 100644 drivers/media/pci/intel/ipu3/ipu3.h > > > > diff --git a/drivers/media/pci/intel/ipu3/Kconfig > > b/drivers/media/pci/intel/ipu3/Kconfig > > index 715f776..44ebcbb 100644 > > --- a/drivers/media/pci/intel/ipu3/Kconfig > > +++ b/drivers/media/pci/intel/ipu3/Kconfig > > @@ -15,3 +15,19 @@ config VIDEO_IPU3_CIO2 > > Say Y or M here if you have a Skylake/Kaby Lake SoC with MIPI CSI-2 > > connected camera. > > The module will be called ipu3-cio2. > > + > > +config VIDEO_IPU3_IMGU > > + tristate "Intel ipu3-imgu driver" > > + depends on PCI && VIDEO_V4L2 > > + depends on MEDIA_CONTROLLER && VIDEO_V4L2_SUBDEV_API > > + depends on X86 > > + select IOMMU_IOVA > > + select VIDEOBUF2_DMA_SG > > + > > Extra newline. > Ack. > > + ---help--- > > + This is the video4linux2 driver for Intel IPU3 image processing > > +unit, > > "Video4Linux2" > Ack. > > + found in Intel Skylake and Kaby Lake SoCs and used for processing > > + images and video. > > + > > + Say Y or M here if you have a Skylake/Kaby Lake SoC with a MIPI > > + camera. The module will be called ipu3-imgu. > > The latter tab should be a space only. > Ack. > > diff --git a/drivers/media/pci/intel/ipu3/Makefile > > b/drivers/media/pci/intel/ipu3/Makefile > > index 20186e3..60bd5db 100644 > > --- a/drivers/media/pci/intel/ipu3/Makefile > > +++ b/drivers/media/pci/intel/ipu3/Makefile > > @@ -1 +1,13 @@ > > +# > > +# Makefile for the IPU3 cio2 and ImgU drivers # > > + > > obj-$(CONFIG_VIDEO_IPU3_CIO2) += ipu3-cio2.o > > + > > +ipu3-imgu-objs += \ > > + ipu3-mmu.o ipu3-dmamap.o \ > > + ipu3-tables.o ipu3-css-pool.o \ > > + ipu3-css-fw.o ipu3-css-params.o \ > > + ipu3-css.o ipu3-v4l2.o ipu3.o > > + > > +obj-$(CONFIG_VIDEO_IPU3_IMGU) += ipu3-imgu.o > > diff --git a/drivers/media/pci/intel/ipu3/ipu3.c > > b/drivers/media/pci/intel/ipu3/ipu3.c > > new file mode 100644 > > index 000..eda7299 > > --- /dev/null > > +++ b/drivers/media/pci/intel/ipu3/ipu3.c > > @@ -0,0 +1,844 @@ > > +// SPDX-License-Identifier: GPL-2.0 > > +/* > > + * Copyright (C) 2017 Intel Corporation > > + * Copyright 2017 Google LLC > > + * > > + * Based on Intel IPU4 driver. > > + * > > + */ > > + > > +#include > > +#include > > +#include > > +#include > > + > > +#include "ipu3.h" > > +#include "ipu3-dmamap.h" > > +#include "ipu3-mmu.h" > > + > > +#define IMGU_PCI_ID0x1919 > > +#define IMGU_PCI_BAR 0 > > +#define IMGU_DMA_MASK DMA_BIT_MASK(39) > > +#define IMGU_MAX_QUEUE_DEPTH (2 + 2) > > + > > +/* > > + * pre-allocated buffer size for IMGU dummy buffers. Those > > + * values should be tuned to big enough to avoid buffer > > + * re-allocation when streaming to lower streaming latency. > > + */ > > +#define CSS_QUEUE_IN_BUF_SIZE 0 > > +#define CSS_QUEUE_PARAMS_BUF_SI
RE: [PATCH v7 14/16] intel-ipu3: Add v4l2 driver based on media framework
Hi, Sakari,
Thanks for the feedback.
> -Original Message-
> From: Sakari Ailus [mailto:sakari.ai...@linux.intel.com]
> Sent: Friday, November 9, 2018 6:37 AM
> To: Zhi, Yong
> Cc: linux-media@vger.kernel.org; tf...@chromium.org;
> mche...@kernel.org; hans.verk...@cisco.com;
> laurent.pinch...@ideasonboard.com; Mani, Rajmohan
> ; Zheng, Jian Xu ; Hu,
> Jerry W ; Toivonen, Tuukka
> ; Qiu, Tian Shu ; Cao,
> Bingbu
> Subject: Re: [PATCH v7 14/16] intel-ipu3: Add v4l2 driver based on media
> framework
>
> Hi Yong,
>
> On Mon, Oct 29, 2018 at 03:23:08PM -0700, Yong Zhi wrote:
> > Implement video driver that utilizes v4l2, vb2 queue support and media
> > controller APIs. The driver exposes single subdevice and six nodes.
> >
> > Signed-off-by: Yong Zhi
> > ---
> > drivers/media/pci/intel/ipu3/ipu3-v4l2.c | 1091
> > ++
> > 1 file changed, 1091 insertions(+)
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-v4l2.c
> >
> > diff --git a/drivers/media/pci/intel/ipu3/ipu3-v4l2.c
> > b/drivers/media/pci/intel/ipu3/ipu3-v4l2.c
> > new file mode 100644
> > index 000..31a3514
> > --- /dev/null
> > +++ b/drivers/media/pci/intel/ipu3/ipu3-v4l2.c
> > @@ -0,0 +1,1091 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +// Copyright (C) 2018 Intel Corporation
> > +
> > +#include
> > +#include
> > +
> > +#include
> > +
> > +#include "ipu3.h"
> > +#include "ipu3-dmamap.h"
> > +
> > +/ v4l2_subdev_ops /
> > +
> > +static int ipu3_subdev_open(struct v4l2_subdev *sd, struct
> > +v4l2_subdev_fh *fh) {
> > + struct imgu_device *imgu = container_of(sd, struct imgu_device,
> subdev);
> > + struct v4l2_rect try_crop = {
> > + .top = 0,
> > + .left = 0,
> > + .height = imgu-
> >nodes[IMGU_NODE_IN].vdev_fmt.fmt.pix_mp.height,
> > + .width = imgu-
> >nodes[IMGU_NODE_IN].vdev_fmt.fmt.pix_mp.width,
> > + };
> > + unsigned int i;
> > +
> > + /* Initialize try_fmt */
> > + for (i = 0; i < IMGU_NODE_NUM; i++)
> > + *v4l2_subdev_get_try_format(sd, fh->pad, i) =
> > + imgu->nodes[i].pad_fmt;
>
> The try formats should reflect the defaults, not the current device state.
>
Ack, will fix in next update.
> > +
> > + *v4l2_subdev_get_try_crop(sd, fh->pad, IMGU_NODE_IN) = try_crop;
>
> Same for the crop. How about the compose rectangle?
>
Ok, will check both crop and compose rectangle.
> > +
> > + return 0;
> > +}
>
> ...
>
> > +int ipu3_v4l2_register(struct imgu_device *imgu) {
> > + struct v4l2_mbus_framefmt def_bus_fmt = { 0 };
> > + struct v4l2_pix_format_mplane def_pix_fmt = { 0 };
> > +
>
> Extra newline.
Ack.
>
> > + int i, r;
> > +
> > + /* Initialize miscellaneous variables */
> > + imgu->streaming = false;
> > +
> > + /* Init media device */
> > + media_device_pci_init(>media_dev, imgu->pci_dev,
> IMGU_NAME);
> > +
> > + /* Set up v4l2 device */
> > + imgu->v4l2_dev.mdev = >media_dev;
> > + imgu->v4l2_dev.ctrl_handler = imgu->ctrl_handler;
> > + r = v4l2_device_register(>pci_dev->dev, >v4l2_dev);
> > + if (r) {
> > + dev_err(>pci_dev->dev,
> > + "failed to register V4L2 device (%d)\n", r);
> > + goto fail_v4l2_dev;
> > + }
> > +
> > + /* Initialize subdev media entity */
> > + imgu->subdev_pads = kzalloc(sizeof(*imgu->subdev_pads) *
> > + IMGU_NODE_NUM, GFP_KERNEL);
>
> As the number of pads is static, could you instead put the array directly to
> the struct, instead of using a pointer? Remember to remove to
> corresponding kfree, too.
Good point, kzalloc does not serve its purpose here.
>
> > + if (!imgu->subdev_pads) {
> > + r = -ENOMEM;
> > + goto fail_subdev_pads;
> > + }
> > + r = media_entity_pads_init(>subdev.entity,
> IMGU_NODE_NUM,
> > + imgu->subdev_pads);
> > + if (r) {
> > + dev_err(>pci_dev->dev,
> > + "failed initialize subdev media entity (%d)\n", r);
> > + goto fail_media_entity;
> > + }
> > + imgu->subdev.entity.ops = _media_ops;
> > + for (i = 0; i < IMGU_NODE_NUM; i++) {
>
RE: [PATCH v7 08/16] intel-ipu3: css: Add dma buff pool utility functions
Hi, Sakari,
> -Original Message-
> From: Sakari Ailus [mailto:sakari.ai...@linux.intel.com]
> Sent: Thursday, November 8, 2018 9:36 AM
> To: Zhi, Yong
> Cc: linux-media@vger.kernel.org; tf...@chromium.org;
> mche...@kernel.org; hans.verk...@cisco.com;
> laurent.pinch...@ideasonboard.com; Mani, Rajmohan
> ; Zheng, Jian Xu ; Hu,
> Jerry W ; Toivonen, Tuukka
> ; Qiu, Tian Shu ; Cao,
> Bingbu
> Subject: Re: [PATCH v7 08/16] intel-ipu3: css: Add dma buff pool utility
> functions
>
> Hi Yong,
>
> On Mon, Oct 29, 2018 at 03:23:02PM -0700, Yong Zhi wrote:
> > The pools are used to store previous parameters set by user with the
> > parameter queue. Due to pipelining, there needs to be multiple sets
> > (up to four) of parameters which are queued in a host-to-sp queue.
> >
> > Signed-off-by: Yong Zhi
> > ---
> > drivers/media/pci/intel/ipu3/ipu3-css-pool.c | 136
> > +++
> > drivers/media/pci/intel/ipu3/ipu3-css-pool.h | 56 +++
> > 2 files changed, 192 insertions(+)
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.c
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.h
> >
> > diff --git a/drivers/media/pci/intel/ipu3/ipu3-css-pool.c
> > b/drivers/media/pci/intel/ipu3/ipu3-css-pool.c
> > new file mode 100644
> > index 000..eab41c3
> > --- /dev/null
> > +++ b/drivers/media/pci/intel/ipu3/ipu3-css-pool.c
> > @@ -0,0 +1,136 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +// Copyright (C) 2018 Intel Corporation
> > +
> > +#include
> > +
> > +#include "ipu3.h"
> > +#include "ipu3-css-pool.h"
> > +#include "ipu3-dmamap.h"
> > +
> > +int ipu3_css_dma_buffer_resize(struct imgu_device *imgu,
> > + struct ipu3_css_map *map, size_t size) {
> > + if (map->size < size && map->vaddr) {
> > + dev_warn(>pci_dev->dev, "dma buf resized from %zu
> to %zu",
> > +map->size, size);
> > +
> > + ipu3_dmamap_free(imgu, map);
> > + if (!ipu3_dmamap_alloc(imgu, map, size))
> > + return -ENOMEM;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +void ipu3_css_pool_cleanup(struct imgu_device *imgu, struct
> > +ipu3_css_pool *pool) {
> > + unsigned int i;
> > +
> > + for (i = 0; i < IPU3_CSS_POOL_SIZE; i++)
> > + ipu3_dmamap_free(imgu, >entry[i].param); }
> > +
> > +int ipu3_css_pool_init(struct imgu_device *imgu, struct ipu3_css_pool
> *pool,
> > + size_t size)
> > +{
> > + unsigned int i;
> > +
> > + for (i = 0; i < IPU3_CSS_POOL_SIZE; i++) {
> > + /*
> > +* entry[i].framenum is initialized to INT_MIN so that
> > +* ipu3_css_pool_check() can treat it as usesable slot.
> > +*/
> > + pool->entry[i].framenum = INT_MIN;
> > +
> > + if (size == 0) {
> > + pool->entry[i].param.vaddr = NULL;
> > + continue;
> > + }
> > +
> > + if (!ipu3_dmamap_alloc(imgu, >entry[i].param, size))
> > + goto fail;
> > + }
> > +
> > + pool->last = IPU3_CSS_POOL_SIZE;
> > +
> > + return 0;
> > +
> > +fail:
> > + ipu3_css_pool_cleanup(imgu, pool);
> > + return -ENOMEM;
> > +}
> > +
> > +/*
> > + * Check that the following call to pool_get succeeds.
> > + * Return negative on error.
> > + */
> > +static int ipu3_css_pool_check(struct ipu3_css_pool *pool, long
> > +framenum) {
> > + /* Get the oldest entry */
> > + int n = (pool->last + 1) % IPU3_CSS_POOL_SIZE;
> > + long diff = framenum - pool->entry[n].framenum;
> > +
> > + /* if framenum wraps around and becomes smaller than entry n */
> > + if (diff < 0)
> > + diff += LONG_MAX;
>
> Have you tested the wrap-around? As a result, the value of the diff is
> between -1 and LONG_MAX - 1 (without considering more than just the two
> lines above). Is that intended?
>
Yes, I simulated wrap-around using a smaller limit in v5.
> You seem to be using different types for the frame number; sometimes int,
> sometimes long. Could you align that, preferrably to an unsigned type? u32
> would probably be a sound choice.
>
Will use u32 at places except entry.framenum, which is initialized to INT_MIN.
This is because the frame i
RE: [PATCH v7 00/16] Intel IPU3 ImgU patchset
Hi, Sakari, Thanks for your review and comments. Bingbu has replied to some of your questions, so I will continue with the rest. > -Original Message- > From: Bing Bu Cao [mailto:bingbu@linux.intel.com] > Sent: Tuesday, November 6, 2018 10:17 PM > To: Sakari Ailus ; Zhi, Yong > > Cc: linux-media@vger.kernel.org; tf...@chromium.org; > mche...@kernel.org; hans.verk...@cisco.com; > laurent.pinch...@ideasonboard.com; Mani, Rajmohan > ; Zheng, Jian Xu ; Hu, > Jerry W ; Toivonen, Tuukka > ; Qiu, Tian Shu ; Cao, > Bingbu > Subject: Re: [PATCH v7 00/16] Intel IPU3 ImgU patchset > > > On 11/01/2018 08:03 PM, Sakari Ailus wrote: > > Hi Yong, > > > > Thanks for the update! > > > > On Mon, Oct 29, 2018 at 03:22:54PM -0700, Yong Zhi wrote: > >> Hi, > >> > >> This series adds support for the Intel IPU3 (Image Processing Unit) > >> ImgU which is essentially a modern memory-to-memory ISP. It > >> implements raw Bayer to YUV image format conversion as well as a > >> large number of other pixel processing algorithms for improving the image > quality. > >> > >> Meta data formats are defined for image statistics (3A, i.e. > >> automatic white balance, exposure and focus, histogram and local area > >> contrast > >> enhancement) as well as for the pixel processing algorithm parameters. > >> The documentation for these formats is currently not included in the > >> patchset but will be added in a future version of this set. > >> > >> The algorithm parameters need to be considered specific to a given > >> frame and typically a large number of these parameters change on > >> frame to frame basis. Additionally, the parameters are highly > >> structured (and not a flat space of independent configuration > >> primitives). They also reflect the data structures used by the > >> firmware and the hardware. On top of that, the algorithms require > >> highly specialized user space to make meaningful use of them. For > >> these reasons it has been chosen video buffers to pass the parameters to > the device. > >> > >> On individual patches: > >> > >> The heart of ImgU is the CSS, or Camera Subsystem, which contains the > >> image processors and HW accelerators. > >> > >> The 3A statistics and other firmware parameter computation related > >> functions are implemented in patch 11. > >> > >> All IPU3 pipeline default settings can be found in patch 10. > >> > >> To access DDR via ImgU's own memory space, IPU3 is also equipped with > >> its own MMU unit, the driver is implemented in patch 6. > >> > >> Patch 7 uses above driver for DMA mapping operation. > >> > >> The communication between IPU3 firmware and driver is implemented > >> with circular queues in patch 8. > >> > >> Patch 9 provide some utility functions and manage IPU3 fw download > >> and install. > >> > >> The firmware which is called ipu3-fw.bin can be downloaded from: > >> > >> git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware > >> .git (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) > >> > >> Firmware ABI is defined in patches 4 and 5. > >> > >> Patches 12 and 13 are of the same file, the former contains all h/w > >> programming related code, the latter implements interface functions > >> for access fw & hw capabilities. > >> > >> Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT > work: > >> > >> https://patchwork.kernel.org/patch/9976295/> > > I've pushed the latest set here: > > > > https://git.linuxtv.org/sailus/media_tree.git/log/?h=meta-output> > > > > You can just say the entire set depends on those going forward; the > > documentation is needed, too. > > Ack. > >> Patch 15 represents the top level that glues all of the other > >> components together, passing arguments between the components. > >> > >> Patch 16 is a recent effort to extend v6 for advanced camera features > >> like Continuous View Finder (CVF) and Snapshot During Video(SDV) > support. > >> > >> Link to user space implementation: > >> > >> git clone > >> https://chromium.googlesource.com/chromiumos/platform/arc-camera > >> > >> ImgU media topology print: > >> > >> # media-ctl -d /dev/media0 -p > >> Media controller API version 4.19.0 > >> > >> Media d
RE: [PATCH v7 03/16] v4l: Add Intel IPU3 meta data uAPI
Hi, Mauro,
Thanks for your review.
> -Original Message-
> From: Mauro Carvalho Chehab [mailto:mchehab+sams...@kernel.org]
> Sent: Friday, November 2, 2018 6:49 AM
> To: Zhi, Yong
> Cc: linux-media@vger.kernel.org; sakari.ai...@linux.intel.com;
> tf...@chromium.org; hans.verk...@cisco.com;
> laurent.pinch...@ideasonboard.com; Mani, Rajmohan
> ; Zheng, Jian Xu ; Hu,
> Jerry W ; Toivonen, Tuukka
> ; Qiu, Tian Shu ; Cao,
> Bingbu ; Li, Chao C
> Subject: Re: [PATCH v7 03/16] v4l: Add Intel IPU3 meta data uAPI
>
> Em Mon, 29 Oct 2018 15:22:57 -0700
> Yong Zhi escreveu:
>
> > These meta formats are used on Intel IPU3 ImgU video queues
> > to carry 3A statistics and ISP pipeline parameters.
>
> Just minor things. See below.
>
> >
> > V4L2_META_FMT_IPU3_3A
> > V4L2_META_FMT_IPU3_PARAMS
> >
> > Signed-off-by: Yong Zhi
> > Signed-off-by: Chao C Li
> > Signed-off-by: Rajmohan Mani
> > ---
> > Documentation/media/uapi/v4l/meta-formats.rst |1 +
> > .../media/uapi/v4l/pixfmt-meta-intel-ipu3.rst | 181 ++
>
> I would actually prefer to have those two changes merged together with
> patch 1, as it makes easier for review.
>
> > include/uapi/linux/intel-ipu3.h| 2819
> >
>
> This one makes sense to have a separate patch.
>
Ack, will re-group the three files as suggested.
> > 3 files changed, 3001 insertions(+)
> > create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-intel-
> ipu3.rst
> > create mode 100644 include/uapi/linux/intel-ipu3.h
> >
> > diff --git a/Documentation/media/uapi/v4l/meta-formats.rst
> b/Documentation/media/uapi/v4l/meta-formats.rst
> > index cf971d5..eafc534 100644
> > --- a/Documentation/media/uapi/v4l/meta-formats.rst
> > +++ b/Documentation/media/uapi/v4l/meta-formats.rst
> > @@ -12,6 +12,7 @@ These formats are used for the :ref:`metadata`
> interface only.
> > .. toctree::
> > :maxdepth: 1
> >
> > +pixfmt-meta-intel-ipu3
> > pixfmt-meta-d4xx
> > pixfmt-meta-uvc
> > pixfmt-meta-vsp1-hgo
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
> b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
> > new file mode 100644
> > index 000..23b945b
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
> > @@ -0,0 +1,181 @@
> > +.. -*- coding: utf-8; mode: rst -*-
> > +
> > +.. _intel-ipu3:
> > +
> >
> +***
> ***
> > +V4L2_META_FMT_IPU3_PARAMS ('ip3p'), V4L2_META_FMT_IPU3_3A
> ('ip3s')
> >
> +***
> ***
> > +
> > +.. c:type:: ipu3_uapi_stats_3a
> > +
> > +3A statistics
> > +=
> > +
> > +For IPU3 ImgU, the 3A statistics accelerators collect different statistics
> > over
> > +an input bayer frame. Those statistics, defined in data struct
> > +:c:type:`ipu3_uapi_stats_3a`, are meta output obtained from "ipu3-imgu
> 3a stat"
> > +video node, which are then passed to user space for statistics analysis
> > +using :c:type:`v4l2_meta_format` interface.
> > +
> > +The statistics collected are AWB (Auto-white balance) RGBS (Red, Green,
> Blue and
> > +Saturation measure) cells, AWB filter response, AF (Auto-focus) filter
> response,
> > +and AE (Auto-exposure) histogram.
> > +
> > +struct :c:type:`ipu3_uapi_4a_config` saves configurable parameters for all
> above.
> > +
> > +
> > +.. code-block:: c
> > +
> > +
> > + struct ipu3_uapi_stats_3a {
> > + struct ipu3_uapi_awb_raw_buffer awb_raw_buffer
> > +__attribute__((aligned(32)));
> > + struct ipu3_uapi_ae_raw_buffer_aligned
> > + ae_raw_buffer[IPU3_UAPI_MAX_STRIPES];
> > + struct ipu3_uapi_af_raw_buffer af_raw_buffer;
> > + struct ipu3_uapi_awb_fr_raw_buffer awb_fr_raw_buffer;
> > + struct ipu3_uapi_4a_config stats_4a_config;
> > + __u32 ae_join_buffers;
> > + __u8 padding[28];
> > + struct ipu3_uapi_stats_3a_bubble_info_per_stripe
> > + stats_3a_bubble_per_stripe;
> > + struct ipu3_uapi_ff_status stats_3a_status;
> > + } __packed;
> > +
> > +
> > +.. c:type:: ipu3_uapi_params
> > +
> > +Pipeline parameters
> > +===
> > +
> > +IPU3 pip
[PATCH] [v4l-utils] libv4l2subdev: Add MEDIA_BUS_FMT_FIXED to mbus_formats[]
Also add V4L2_COLORSPACE_RAW to the colorspaces[].
Signed-off-by: Yong Zhi
---
utils/media-ctl/libv4l2subdev.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/utils/media-ctl/libv4l2subdev.c b/utils/media-ctl/libv4l2subdev.c
index a989efb..46668eb 100644
--- a/utils/media-ctl/libv4l2subdev.c
+++ b/utils/media-ctl/libv4l2subdev.c
@@ -855,6 +855,7 @@ static const struct {
enum v4l2_mbus_pixelcode code;
} mbus_formats[] = {
#include "media-bus-format-names.h"
+ { "FIXED", MEDIA_BUS_FMT_FIXED},
{ "Y8", MEDIA_BUS_FMT_Y8_1X8},
{ "Y10", MEDIA_BUS_FMT_Y10_1X10 },
{ "Y12", MEDIA_BUS_FMT_Y12_1X12 },
@@ -965,7 +966,9 @@ static struct {
{ "srgb", V4L2_COLORSPACE_SRGB },
{ "oprgb", V4L2_COLORSPACE_OPRGB },
{ "bt2020", V4L2_COLORSPACE_BT2020 },
+ { "raw", V4L2_COLORSPACE_RAW },
{ "dcip3", V4L2_COLORSPACE_DCI_P3 },
+
};
const char *v4l2_subdev_colorspace_to_string(enum v4l2_colorspace colorspace)
--
2.7.4
RE: [PATCH v7 06/16] intel-ipu3: mmu: Implement driver
Hi, Sakari,
Thanks for the feedback.
> -Original Message-
> From: Sakari Ailus [mailto:sakari.ai...@linux.intel.com]
> Sent: Monday, November 5, 2018 3:55 AM
> To: Zhi, Yong
> Cc: linux-media@vger.kernel.org; tf...@chromium.org;
> mche...@kernel.org; hans.verk...@cisco.com;
> laurent.pinch...@ideasonboard.com; Mani, Rajmohan
> ; Zheng, Jian Xu ; Hu,
> Jerry W ; Toivonen, Tuukka
> ; Qiu, Tian Shu ; Cao,
> Bingbu
> Subject: Re: [PATCH v7 06/16] intel-ipu3: mmu: Implement driver
>
> Hi Yong,
>
> On Mon, Oct 29, 2018 at 03:23:00PM -0700, Yong Zhi wrote:
> > From: Tomasz Figa
> >
> > This driver translates IO virtual address to physical address based on
> > two levels page tables.
> >
> > Signed-off-by: Tomasz Figa
> > Signed-off-by: Yong Zhi
> > ---
>
> ...
>
> > +static void call_if_ipu3_is_powered(struct ipu3_mmu *mmu,
> > + void (*func)(struct ipu3_mmu *mmu)) {
> > + pm_runtime_get_noresume(mmu->dev);
> > + if (pm_runtime_active(mmu->dev))
> > + func(mmu);
> > + pm_runtime_put(mmu->dev);
>
> How about:
>
> if (!pm_runtime_get_if_in_use(mmu->dev))
> return;
>
> func(mmu);
> pm_runtime_put(mmu->dev);
>
Ack, unless Tomasz has different opinion.
>
> > +}
>
> --
> Sakari Ailus
> sakari.ai...@linux.intel.com
[PATCH v7 09/16] intel-ipu3: css: Add support for firmware management
Introduce functions to load and install ImgU FW blobs.
Signed-off-by: Yong Zhi
---
drivers/media/pci/intel/ipu3/ipu3-css-fw.c | 264 +
drivers/media/pci/intel/ipu3/ipu3-css-fw.h | 188
2 files changed, 452 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.h
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css-fw.c
b/drivers/media/pci/intel/ipu3/ipu3-css-fw.c
new file mode 100644
index 000..ba459e9
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-css-fw.c
@@ -0,0 +1,264 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+#include
+#include
+#include
+
+#include "ipu3-css.h"
+#include "ipu3-css-fw.h"
+#include "ipu3-dmamap.h"
+
+static void ipu3_css_fw_show_binary(struct device *dev, struct imgu_fw_info
*bi,
+ const char *name)
+{
+ unsigned int i;
+
+ dev_dbg(dev, "found firmware binary type %i size %i name %s\n",
+ bi->type, bi->blob.size, name);
+ if (bi->type != IMGU_FW_ISP_FIRMWARE)
+ return;
+
+ dev_dbg(dev, "id %i mode %i bds 0x%x veceven %i/%i out_pins %i\n",
+ bi->info.isp.sp.id, bi->info.isp.sp.pipeline.mode,
+ bi->info.isp.sp.bds.supported_bds_factors,
+ bi->info.isp.sp.enable.vf_veceven,
+ bi->info.isp.sp.vf_dec.is_variable,
+ bi->info.isp.num_output_pins);
+
+ dev_dbg(dev, "input (%i,%i)-(%i,%i) formats %s%s%s\n",
+ bi->info.isp.sp.input.min_width,
+ bi->info.isp.sp.input.min_height,
+ bi->info.isp.sp.input.max_width,
+ bi->info.isp.sp.input.max_height,
+ bi->info.isp.sp.enable.input_yuv ? "yuv420 " : "",
+ bi->info.isp.sp.enable.input_feeder ||
+ bi->info.isp.sp.enable.input_raw ? "raw8 raw10 " : "",
+ bi->info.isp.sp.enable.input_raw ? "raw12" : "");
+
+ dev_dbg(dev, "internal (%i,%i)\n",
+ bi->info.isp.sp.internal.max_width,
+ bi->info.isp.sp.internal.max_height);
+
+ dev_dbg(dev, "output (%i,%i)-(%i,%i) formats",
+ bi->info.isp.sp.output.min_width,
+ bi->info.isp.sp.output.min_height,
+ bi->info.isp.sp.output.max_width,
+ bi->info.isp.sp.output.max_height);
+ for (i = 0; i < bi->info.isp.num_output_formats; i++)
+ dev_dbg(dev, " %i", bi->info.isp.output_formats[i]);
+ dev_dbg(dev, " vf");
+ for (i = 0; i < bi->info.isp.num_vf_formats; i++)
+ dev_dbg(dev, " %i", bi->info.isp.vf_formats[i]);
+ dev_dbg(dev, "\n");
+}
+
+unsigned int ipu3_css_fw_obgrid_size(const struct imgu_fw_info *bi)
+{
+ unsigned int width = DIV_ROUND_UP(bi->info.isp.sp.internal.max_width,
+ IMGU_OBGRID_TILE_SIZE * 2) + 1;
+ unsigned int height = DIV_ROUND_UP(bi->info.isp.sp.internal.max_height,
+ IMGU_OBGRID_TILE_SIZE * 2) + 1;
+ unsigned int obgrid_size;
+
+ width = ALIGN(width, IPU3_UAPI_ISP_VEC_ELEMS / 4);
+ obgrid_size = PAGE_ALIGN(width * height *
+sizeof(struct ipu3_uapi_obgrid_param)) *
+bi->info.isp.sp.iterator.num_stripes;
+ return obgrid_size;
+}
+
+void *ipu3_css_fw_pipeline_params(struct ipu3_css *css,
+ enum imgu_abi_param_class c,
+ enum imgu_abi_memories m,
+ struct imgu_fw_isp_parameter *par,
+ size_t par_size, void *binary_params)
+{
+ struct imgu_fw_info *bi = >fwp->binary_header[css->current_binary];
+
+ if (par->offset + par->size >
+ bi->info.isp.sp.mem_initializers.params[c][m].size)
+ return NULL;
+
+ if (par->size != par_size)
+ pr_warn("parameter size doesn't match defined size\n");
+
+ if (par->size < par_size)
+ return NULL;
+
+ return binary_params + par->offset;
+}
+
+void ipu3_css_fw_cleanup(struct ipu3_css *css)
+{
+ struct imgu_device *imgu = dev_get_drvdata(css->dev);
+
+ if (css->binary) {
+ unsigned int i;
+
+ for (i = 0; i < css->fwp->file_header.binary_nr; i++)
+ ipu3_dmamap_free(imgu, >binary[i]);
+ kfree(css->binary);
+ }
+ if (css->fw)
+
[PATCH v7 15/16] intel-ipu3: Add imgu top level pci device driver
This patch adds support for the Intel IPU v3 as found
on Skylake and Kaby Lake SoCs.
The driver glues v4l2, css(camera sub system) and other
pieces together to perform its functions, it also loads
the IPU3 firmware binary as part of its initialization.
Signed-off-by: Yong Zhi
Signed-off-by: Tomasz Figa
---
drivers/media/pci/intel/ipu3/Kconfig | 16 +
drivers/media/pci/intel/ipu3/Makefile | 12 +
drivers/media/pci/intel/ipu3/ipu3.c | 844 ++
drivers/media/pci/intel/ipu3/ipu3.h | 153 ++
4 files changed, 1025 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3.h
diff --git a/drivers/media/pci/intel/ipu3/Kconfig
b/drivers/media/pci/intel/ipu3/Kconfig
index 715f776..44ebcbb 100644
--- a/drivers/media/pci/intel/ipu3/Kconfig
+++ b/drivers/media/pci/intel/ipu3/Kconfig
@@ -15,3 +15,19 @@ config VIDEO_IPU3_CIO2
Say Y or M here if you have a Skylake/Kaby Lake SoC with MIPI CSI-2
connected camera.
The module will be called ipu3-cio2.
+
+config VIDEO_IPU3_IMGU
+ tristate "Intel ipu3-imgu driver"
+ depends on PCI && VIDEO_V4L2
+ depends on MEDIA_CONTROLLER && VIDEO_V4L2_SUBDEV_API
+ depends on X86
+ select IOMMU_IOVA
+ select VIDEOBUF2_DMA_SG
+
+ ---help---
+ This is the video4linux2 driver for Intel IPU3 image processing unit,
+ found in Intel Skylake and Kaby Lake SoCs and used for processing
+ images and video.
+
+ Say Y or M here if you have a Skylake/Kaby Lake SoC with a MIPI
+ camera. The module will be called ipu3-imgu.
diff --git a/drivers/media/pci/intel/ipu3/Makefile
b/drivers/media/pci/intel/ipu3/Makefile
index 20186e3..60bd5db 100644
--- a/drivers/media/pci/intel/ipu3/Makefile
+++ b/drivers/media/pci/intel/ipu3/Makefile
@@ -1 +1,13 @@
+#
+# Makefile for the IPU3 cio2 and ImgU drivers
+#
+
obj-$(CONFIG_VIDEO_IPU3_CIO2) += ipu3-cio2.o
+
+ipu3-imgu-objs += \
+ ipu3-mmu.o ipu3-dmamap.o \
+ ipu3-tables.o ipu3-css-pool.o \
+ ipu3-css-fw.o ipu3-css-params.o \
+ ipu3-css.o ipu3-v4l2.o ipu3.o
+
+obj-$(CONFIG_VIDEO_IPU3_IMGU) += ipu3-imgu.o
diff --git a/drivers/media/pci/intel/ipu3/ipu3.c
b/drivers/media/pci/intel/ipu3/ipu3.c
new file mode 100644
index 000..eda7299
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3.c
@@ -0,0 +1,844 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2017 Intel Corporation
+ * Copyright 2017 Google LLC
+ *
+ * Based on Intel IPU4 driver.
+ *
+ */
+
+#include
+#include
+#include
+#include
+
+#include "ipu3.h"
+#include "ipu3-dmamap.h"
+#include "ipu3-mmu.h"
+
+#define IMGU_PCI_ID0x1919
+#define IMGU_PCI_BAR 0
+#define IMGU_DMA_MASK DMA_BIT_MASK(39)
+#define IMGU_MAX_QUEUE_DEPTH (2 + 2)
+
+/*
+ * pre-allocated buffer size for IMGU dummy buffers. Those
+ * values should be tuned to big enough to avoid buffer
+ * re-allocation when streaming to lower streaming latency.
+ */
+#define CSS_QUEUE_IN_BUF_SIZE 0
+#define CSS_QUEUE_PARAMS_BUF_SIZE 0
+#define CSS_QUEUE_OUT_BUF_SIZE (4160 * 3120 * 12 / 8)
+#define CSS_QUEUE_VF_BUF_SIZE (1920 * 1080 * 12 / 8)
+#define CSS_QUEUE_STAT_3A_BUF_SIZE 125664
+
+static const size_t css_queue_buf_size_map[IPU3_CSS_QUEUES] = {
+ [IPU3_CSS_QUEUE_IN] = CSS_QUEUE_IN_BUF_SIZE,
+ [IPU3_CSS_QUEUE_PARAMS] = CSS_QUEUE_PARAMS_BUF_SIZE,
+ [IPU3_CSS_QUEUE_OUT] = CSS_QUEUE_OUT_BUF_SIZE,
+ [IPU3_CSS_QUEUE_VF] = CSS_QUEUE_VF_BUF_SIZE,
+ [IPU3_CSS_QUEUE_STAT_3A] = CSS_QUEUE_STAT_3A_BUF_SIZE,
+};
+
+static const struct imgu_node_mapping imgu_node_map[IMGU_NODE_NUM] = {
+ [IMGU_NODE_IN] = {IPU3_CSS_QUEUE_IN, "input"},
+ [IMGU_NODE_PARAMS] = {IPU3_CSS_QUEUE_PARAMS, "parameters"},
+ [IMGU_NODE_OUT] = {IPU3_CSS_QUEUE_OUT, "output"},
+ [IMGU_NODE_VF] = {IPU3_CSS_QUEUE_VF, "viewfinder"},
+ [IMGU_NODE_PV] = {IPU3_CSS_QUEUE_VF, "postview"},
+ [IMGU_NODE_STAT_3A] = {IPU3_CSS_QUEUE_STAT_3A, "3a stat"},
+};
+
+unsigned int imgu_node_to_queue(unsigned int node)
+{
+ return imgu_node_map[node].css_queue;
+}
+
+unsigned int imgu_map_node(struct imgu_device *imgu, unsigned int css_queue)
+{
+ unsigned int i;
+
+ if (css_queue == IPU3_CSS_QUEUE_VF)
+ return imgu->nodes[IMGU_NODE_VF].enabled ?
+ IMGU_NODE_VF : IMGU_NODE_PV;
+
+ for (i = 0; i < IMGU_NODE_NUM; i++)
+ if (imgu_node_map[i].css_queue == css_queue)
+ break;
+
+ return i;
+}
+
+/ Dummy buffers /
+
+static void imgu_dummybufs_cleanup(struct imgu_device *imgu)
+{
+ unsigned int i;
[PATCH v7 01/16] v4l: Add Intel IPU3 meta buffer formats
Add IPU3-specific meta formats for parameter
processing and 3A, DVS statistics:
V4L2_META_FMT_IPU3_PARAMS
V4L2_META_FMT_IPU3_STAT_3A
Signed-off-by: Yong Zhi
---
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
include/uapi/linux/videodev2.h | 4
2 files changed, 6 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index 6489f25..abff64b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1299,6 +1299,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_VSP1_HGO:descr = "R-Car VSP1 1-D Histogram";
break;
case V4L2_META_FMT_VSP1_HGT:descr = "R-Car VSP1 2-D Histogram";
break;
case V4L2_META_FMT_UVC: descr = "UVC payload header metadata";
break;
+ case V4L2_META_FMT_IPU3_PARAMS: descr = "IPU3 processing parameters";
break;
+ case V4L2_META_FMT_IPU3_STAT_3A:descr = "IPU3 3A statistics";
break;
default:
/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index f0a968a..bdccd7a 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -718,6 +718,10 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC
Payload Header metadata */
#define V4L2_META_FMT_D4XXv4l2_fourcc('D', '4', 'X', 'X') /* D4XX
Payload Header metadata */
+/* Vendor specific - used for IPU3 camera sub-system */
+#define V4L2_META_FMT_IPU3_PARAMS v4l2_fourcc('i', 'p', '3', 'p') /* IPU3
params */
+#define V4L2_META_FMT_IPU3_STAT_3A v4l2_fourcc('i', 'p', '3', 's') /* IPU3
3A statistics */
+
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC0xfeedcafe
--
2.7.4
[PATCH v7 05/16] intel-ipu3: abi: Add structs
This add all the structs of IPU3 firmware ABI.
Signed-off-by: Yong Zhi
Signed-off-by: Rajmohan Mani
---
drivers/media/pci/intel/ipu3/ipu3-abi.h | 1350 +++
1 file changed, 1350 insertions(+)
diff --git a/drivers/media/pci/intel/ipu3/ipu3-abi.h
b/drivers/media/pci/intel/ipu3/ipu3-abi.h
index ac08ad3..21703da 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-abi.h
+++ b/drivers/media/pci/intel/ipu3/ipu3-abi.h
@@ -658,4 +658,1354 @@ enum imgu_abi_stage_type {
IMGU_ABI_STAGE_TYPE_ISP,
};
+struct imgu_abi_acc_operation {
+ /*
+* zero means on init,
+* others mean upon receiving an ack signal from the BC acc.
+*/
+ u8 op_indicator;
+ u8 op_type;
+} __packed;
+
+struct imgu_abi_acc_process_lines_cmd_data {
+ u16 lines;
+ u8 cfg_set;
+ u8 __reserved; /* Align to 4 bytes */
+} __packed;
+
+/* Bayer shading definitions */
+
+struct imgu_abi_shd_transfer_luts_set_data {
+ u8 set_number;
+ u8 padding[3];
+ imgu_addr_t rg_lut_ddr_addr;
+ imgu_addr_t bg_lut_ddr_addr;
+ u32 align_dummy;
+} __packed;
+
+struct imgu_abi_shd_grid_config {
+ /* reg 0 */
+ u32 grid_width:8;
+ u32 grid_height:8;
+ u32 block_width:3;
+ u32 __reserved0:1;
+ u32 block_height:3;
+ u32 __reserved1:1;
+ u32 grid_height_per_slice:8;
+ /* reg 1 */
+ s32 x_start:13;
+ s32 __reserved2:3;
+ s32 y_start:13;
+ s32 __reserved3:3;
+} __packed;
+
+struct imgu_abi_shd_general_config {
+ u32 init_set_vrt_offst_ul:8;
+ u32 shd_enable:1;
+ /* aka 'gf' */
+ u32 gain_factor:2;
+ u32 __reserved:21;
+} __packed;
+
+struct imgu_abi_shd_black_level_config {
+ /* reg 0 */
+ s32 bl_r:12;
+ s32 __reserved0:4;
+ s32 bl_gr:12;
+ u32 __reserved1:1;
+ /* aka 'nf' */
+ u32 normalization_shift:3;
+ /* reg 1 */
+ s32 bl_gb:12;
+ s32 __reserved2:4;
+ s32 bl_b:12;
+ s32 __reserved3:4;
+} __packed;
+
+struct imgu_abi_shd_intra_frame_operations_data {
+ struct imgu_abi_acc_operation
+ operation_list[IMGU_ABI_SHD_MAX_OPERATIONS]
__attribute__((aligned(32)));
+ struct imgu_abi_acc_process_lines_cmd_data
+ process_lines_data[IMGU_ABI_SHD_MAX_PROCESS_LINES]
__attribute__((aligned(32)));
+ struct imgu_abi_shd_transfer_luts_set_data
+ transfer_data[IMGU_ABI_SHD_MAX_TRANSFERS]
__attribute__((aligned(32)));
+} __packed;
+
+struct imgu_abi_shd_config {
+ struct ipu3_uapi_shd_config_static shd __attribute__((aligned(32)));
+ struct imgu_abi_shd_intra_frame_operations_data shd_ops
__attribute__((aligned(32)));
+ struct ipu3_uapi_shd_lut shd_lut __attribute__((aligned(32)));
+} __packed;
+
+struct imgu_abi_stripe_input_frame_resolution {
+ u16 width;
+ u16 height;
+ u32 bayer_order;/* enum ipu3_uapi_bayer_order */
+ u32 raw_bit_depth;
+} __packed;
+
+/* Stripe-based processing */
+
+struct imgu_abi_stripes {
+ /* offset from start of frame - measured in pixels */
+ u16 offset;
+ /* stripe width - measured in pixels */
+ u16 width;
+ /* stripe width - measured in pixels */
+ u16 height;
+} __packed;
+
+struct imgu_abi_stripe_data {
+ /*
+* number of stripes for current processing source
+* - VLIW binary parameter we currently support 1 or 2 stripes
+*/
+ u16 num_of_stripes;
+
+ u8 padding[2];
+
+ /*
+* the following data is derived from resolution-related
+* pipe config and from num_of_stripes
+*/
+
+ /*
+*'input-stripes' - before input cropping
+* used by input feeder
+*/
+ struct imgu_abi_stripe_input_frame_resolution input_frame;
+
+ /*'effective-stripes' - after input cropping used dpc, bds */
+ struct imgu_abi_stripes effective_stripes[IPU3_UAPI_MAX_STRIPES];
+
+ /* 'down-scaled-stripes' - after down-scaling ONLY. used by BDS */
+ struct imgu_abi_stripes down_scaled_stripes[IPU3_UAPI_MAX_STRIPES];
+
+ /*
+*'bds-out-stripes' - after bayer down-scaling and padding.
+* used by all algos starting with norm up to the ref-frame for GDC
+* (currently up to the output kernel)
+*/
+ struct imgu_abi_stripes bds_out_stripes[IPU3_UAPI_MAX_STRIPES];
+
+ /* 'bds-out-stripes (no overlap)' - used for ref kernel */
+ struct imgu_abi_stripes
+ bds_out_stripes_no_overlap[IPU3_UAPI_MAX_STRIPES];
+
+ /*
+* input resolution for output system (equal to bds_out - envelope)
+* output-system input frame width as configured by user
+*/
+ u16 output_system_in_frame_width;
+ /* output-system input frame height as configured by user */
+ u16 output_system_in_frame_height
[PATCH v7 08/16] intel-ipu3: css: Add dma buff pool utility functions
The pools are used to store previous parameters set by
user with the parameter queue. Due to pipelining,
there needs to be multiple sets (up to four)
of parameters which are queued in a host-to-sp queue.
Signed-off-by: Yong Zhi
---
drivers/media/pci/intel/ipu3/ipu3-css-pool.c | 136 +++
drivers/media/pci/intel/ipu3/ipu3-css-pool.h | 56 +++
2 files changed, 192 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.h
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css-pool.c
b/drivers/media/pci/intel/ipu3/ipu3-css-pool.c
new file mode 100644
index 000..eab41c3
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-css-pool.c
@@ -0,0 +1,136 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+
+#include "ipu3.h"
+#include "ipu3-css-pool.h"
+#include "ipu3-dmamap.h"
+
+int ipu3_css_dma_buffer_resize(struct imgu_device *imgu,
+ struct ipu3_css_map *map, size_t size)
+{
+ if (map->size < size && map->vaddr) {
+ dev_warn(>pci_dev->dev, "dma buf resized from %zu to %zu",
+map->size, size);
+
+ ipu3_dmamap_free(imgu, map);
+ if (!ipu3_dmamap_alloc(imgu, map, size))
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void ipu3_css_pool_cleanup(struct imgu_device *imgu, struct ipu3_css_pool
*pool)
+{
+ unsigned int i;
+
+ for (i = 0; i < IPU3_CSS_POOL_SIZE; i++)
+ ipu3_dmamap_free(imgu, >entry[i].param);
+}
+
+int ipu3_css_pool_init(struct imgu_device *imgu, struct ipu3_css_pool *pool,
+ size_t size)
+{
+ unsigned int i;
+
+ for (i = 0; i < IPU3_CSS_POOL_SIZE; i++) {
+ /*
+* entry[i].framenum is initialized to INT_MIN so that
+* ipu3_css_pool_check() can treat it as usesable slot.
+*/
+ pool->entry[i].framenum = INT_MIN;
+
+ if (size == 0) {
+ pool->entry[i].param.vaddr = NULL;
+ continue;
+ }
+
+ if (!ipu3_dmamap_alloc(imgu, >entry[i].param, size))
+ goto fail;
+ }
+
+ pool->last = IPU3_CSS_POOL_SIZE;
+
+ return 0;
+
+fail:
+ ipu3_css_pool_cleanup(imgu, pool);
+ return -ENOMEM;
+}
+
+/*
+ * Check that the following call to pool_get succeeds.
+ * Return negative on error.
+ */
+static int ipu3_css_pool_check(struct ipu3_css_pool *pool, long framenum)
+{
+ /* Get the oldest entry */
+ int n = (pool->last + 1) % IPU3_CSS_POOL_SIZE;
+ long diff = framenum - pool->entry[n].framenum;
+
+ /* if framenum wraps around and becomes smaller than entry n */
+ if (diff < 0)
+ diff += LONG_MAX;
+
+ /*
+* pool->entry[n].framenum stores the frame number where that
+* entry was allocated. If that was allocated more than POOL_SIZE
+* frames back, it is old enough that we know it is no more in
+* use by firmware.
+*/
+ if (diff > IPU3_CSS_POOL_SIZE)
+ return n;
+
+ return -ENOSPC;
+}
+
+/*
+ * Allocate a new parameter from pool at frame number `framenum'.
+ * Release the oldest entry in the pool to make space for the new entry.
+ * Return negative on error.
+ */
+int ipu3_css_pool_get(struct ipu3_css_pool *pool, long framenum)
+{
+ int n = ipu3_css_pool_check(pool, framenum);
+
+ if (n < 0)
+ return n;
+
+ pool->entry[n].framenum = framenum;
+ pool->last = n;
+
+ return n;
+}
+
+/*
+ * Undo, for all practical purposes, the effect of pool_get().
+ */
+void ipu3_css_pool_put(struct ipu3_css_pool *pool)
+{
+ pool->entry[pool->last].framenum = INT_MIN;
+ pool->last = (pool->last + IPU3_CSS_POOL_SIZE - 1) % IPU3_CSS_POOL_SIZE;
+}
+
+/**
+ * ipu3_css_pool_last - Retrieve the nth pool entry from last
+ *
+ * @pool: a pointer to ipu3_css_pool.
+ * @n: the distance to the last index.
+ *
+ * Return: The nth entry from last or null map to indicate no frame stored.
+ */
+const struct ipu3_css_map *
+ipu3_css_pool_last(struct ipu3_css_pool *pool, unsigned int n)
+{
+ static const struct ipu3_css_map null_map = { 0 };
+ int i = (pool->last + IPU3_CSS_POOL_SIZE - n) % IPU3_CSS_POOL_SIZE;
+
+ WARN_ON(n >= IPU3_CSS_POOL_SIZE);
+
+ if (pool->entry[i].framenum < 0)
+ return _map;
+
+ return >entry[i].param;
+}
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css-pool.h
b/drivers/media/pci/intel/ipu3/ipu3-css-pool.h
new file mode 100644
index 000..71e48d1
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-css-pool.h
@@ -0,0 +1,56 @@
+/*
[PATCH v7 13/16] intel-ipu3: Add css pipeline programming
This provides helper library to be used by
v4l2 level to program imaging pipelines and
control the streaming.
Signed-off-by: Yong Zhi
---
drivers/media/pci/intel/ipu3/ipu3-css.c | 1760 +++
1 file changed, 1760 insertions(+)
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css.c
b/drivers/media/pci/intel/ipu3/ipu3-css.c
index 164830f..c63b387 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-css.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-css.c
@@ -16,6 +16,173 @@
IMGU_IRQCTRL_IRQ_SW_PIN(0) | \
IMGU_IRQCTRL_IRQ_SW_PIN(1))
+#define IPU3_CSS_FORMAT_BPP_DEN50 /* Denominator */
+
+/* Some sane limits for resolutions */
+#define IPU3_CSS_MIN_RES 32
+#define IPU3_CSS_MAX_H 3136
+#define IPU3_CSS_MAX_W 4224
+
+/* filter size from graph settings is fixed as 4 */
+#define FILTER_SIZE 4
+#define MIN_ENVELOPE8
+
+/*
+ * pre-allocated buffer size for CSS ABI, auxiliary frames
+ * after BDS and before GDC. Those values should be tuned
+ * to big enough to avoid buffer re-allocation when
+ * streaming to lower streaming latency.
+ */
+#define CSS_ABI_SIZE136
+#define CSS_BDS_SIZE(4480 * 3200 * 3)
+#define CSS_GDC_SIZE(4224 * 3200 * 12 / 8)
+
+#define IPU3_CSS_QUEUE_TO_FLAGS(q) (1 << (q))
+#define IPU3_CSS_FORMAT_FL_IN \
+ IPU3_CSS_QUEUE_TO_FLAGS(IPU3_CSS_QUEUE_IN)
+#define IPU3_CSS_FORMAT_FL_OUT \
+ IPU3_CSS_QUEUE_TO_FLAGS(IPU3_CSS_QUEUE_OUT)
+#define IPU3_CSS_FORMAT_FL_VF \
+ IPU3_CSS_QUEUE_TO_FLAGS(IPU3_CSS_QUEUE_VF)
+
+/* Formats supported by IPU3 Camera Sub System */
+static const struct ipu3_css_format ipu3_css_formats[] = {
+ {
+ .pixelformat = V4L2_PIX_FMT_NV12,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_NV12,
+ .osys_format = IMGU_ABI_OSYS_FORMAT_NV12,
+ .osys_tiling = IMGU_ABI_OSYS_TILING_NONE,
+ .bytesperpixel_num = 1 * IPU3_CSS_FORMAT_BPP_DEN,
+ .chroma_decim = 4,
+ .width_align = IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_OUT | IPU3_CSS_FORMAT_FL_VF,
+ }, {
+ /* Each 32 bytes contains 25 10-bit pixels */
+ .pixelformat = V4L2_PIX_FMT_IPU3_SBGGR10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_BGGR,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ }, {
+ .pixelformat = V4L2_PIX_FMT_IPU3_SGBRG10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_GBRG,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ }, {
+ .pixelformat = V4L2_PIX_FMT_IPU3_SGRBG10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_GRBG,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ }, {
+ .pixelformat = V4L2_PIX_FMT_IPU3_SRGGB10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_RGGB,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ },
+};
+
+static const struct {
+ enum imgu_abi_queue_id qid;
+ size_t ptr_ofs;
+} ipu3_css_queues[IPU3_CSS_QUEUES] = {
+ [IPU3_CSS_QUEUE_IN] = {
+ IMGU_ABI_QUEUE_C_ID,
+ offsetof(struct imgu_abi_buffer, payload.frame.frame_data)
+ },
+ [IPU3_CSS_QUEUE_OUT] = {
+ IMGU_ABI_QUEUE_D_ID,
+ offsetof(struct imgu_abi_buffer, payload.frame.frame_data)
+ },
+ [IPU3_CSS_QUEUE_VF] = {
+ IMGU_ABI_QUEUE_E_ID,
+ offsetof(struct imgu_abi_buffer, payload.frame.frame_data)
+ },
+ [IPU3_CSS_QUEUE_STAT_3A] = {
+ IMGU_ABI_QUEUE_F_ID,
+ offsetof(struct imgu_abi_buffer, payload.s3a.data_ptr)
+ },
+};
+
+/* Initialize queue based on given format, adjust format as needed */
+static int ipu3_css_queue_init(struct ipu3_css_queue
[PATCH v7 11/16] intel-ipu3: css: Compute and program ccs
A collection of routines that are mainly used
to calculate the parameters for accelerator cluster.
Signed-off-by: Yong Zhi
---
drivers/media/pci/intel/ipu3/ipu3-css-params.c | 2907
drivers/media/pci/intel/ipu3/ipu3-css-params.h | 25 +
2 files changed, 2932 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.h
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css-params.c
b/drivers/media/pci/intel/ipu3/ipu3-css-params.c
new file mode 100644
index 000..add2be4
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-css-params.c
@@ -0,0 +1,2907 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+
+#include "ipu3-css.h"
+#include "ipu3-css-fw.h"
+#include "ipu3-tables.h"
+
+#define DIV_ROUND_CLOSEST_DOWN(a, b) (((a) + ((b) / 2) - 1) / (b))
+#define roundclosest_down(a, b)(DIV_ROUND_CLOSEST_DOWN(a, b) *
(b))
+
+#define IPU3_UAPI_ANR_MAX_RESET((1 << 12) - 1)
+#define IPU3_UAPI_ANR_MIN_RESET(((-1) << 12) + 1)
+
+struct ipu3_css_scaler_info {
+ unsigned int phase_step;/* Same for luma/chroma */
+ int exp_shift;
+
+ unsigned int phase_init;/* luma/chroma dependent */
+ int pad_left;
+ int pad_right;
+ int crop_left;
+ int crop_top;
+};
+
+static unsigned int ipu3_css_scaler_get_exp(unsigned int counter,
+ unsigned int divider)
+{
+ int i = fls(divider) - fls(counter);
+
+ if (i <= 0)
+ return 0;
+
+ if (divider >> i < counter)
+ i = i - 1;
+
+ return i;
+}
+
+/* Set up the CSS scaler look up table */
+static void
+ipu3_css_scaler_setup_lut(unsigned int taps, unsigned int input_width,
+ unsigned int output_width, int phase_step_correction,
+ const int *coeffs, unsigned int coeffs_size,
+ s8 coeff_lut[], struct ipu3_css_scaler_info *info)
+{
+ int tap, phase, phase_sum_left, phase_sum_right;
+ int exponent = ipu3_css_scaler_get_exp(output_width, input_width);
+ int mantissa = (1 << exponent) * output_width;
+ unsigned int phase_step;
+
+ if (input_width == output_width) {
+ for (phase = 0; phase < IMGU_SCALER_PHASES; phase++) {
+ for (tap = 0; tap < taps; tap++) {
+ coeff_lut[phase * IMGU_SCALER_FILTER_TAPS + tap]
+ = 0;
+ }
+ }
+
+ info->phase_step = IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF);
+ info->exp_shift = 0;
+ info->pad_left = 0;
+ info->pad_right = 0;
+ info->phase_init = 0;
+ info->crop_left = 0;
+ info->crop_top = 0;
+ return;
+ }
+
+ for (phase = 0; phase < IMGU_SCALER_PHASES; phase++) {
+ for (tap = 0; tap < taps; tap++) {
+ /* flip table to for convolution reverse indexing */
+ s64 coeff = coeffs[coeffs_size -
+ ((tap * (coeffs_size / taps)) + phase) - 1];
+ coeff *= mantissa;
+ coeff = div64_long(coeff, input_width);
+
+ /* Add +"0.5" */
+ coeff += 1 << (IMGU_SCALER_COEFF_BITS - 1);
+ coeff >>= IMGU_SCALER_COEFF_BITS;
+
+ coeff_lut[phase * IMGU_SCALER_FILTER_TAPS + tap] =
+ coeff;
+ }
+ }
+
+ phase_step = IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF) *
+ output_width / input_width;
+ phase_step += phase_step_correction;
+ phase_sum_left = (taps / 2 * IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF)) -
+ (1 << (IMGU_SCALER_PHASE_COUNTER_PREC_REF - 1));
+ phase_sum_right = (taps / 2 * IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF)) +
+ (1 << (IMGU_SCALER_PHASE_COUNTER_PREC_REF - 1));
+
+ info->exp_shift = IMGU_SCALER_MAX_EXPONENT_SHIFT - exponent;
+ info->pad_left = (phase_sum_left % phase_step == 0) ?
+ phase_sum_left / phase_step - 1 : phase_sum_left / phase_step;
+ info->pad_right = (phase_sum_right % phase_step == 0) ?
+ phase_sum_right / phase_step - 1 : phase_sum_right / phase_step;
+ info->phase_init = phase_sum_left - phase_step * info->pad_le
[PATCH v7 03/16] v4l: Add Intel IPU3 meta data uAPI
These meta formats are used on Intel IPU3 ImgU video queues
to carry 3A statistics and ISP pipeline parameters.
V4L2_META_FMT_IPU3_3A
V4L2_META_FMT_IPU3_PARAMS
Signed-off-by: Yong Zhi
Signed-off-by: Chao C Li
Signed-off-by: Rajmohan Mani
---
Documentation/media/uapi/v4l/meta-formats.rst |1 +
.../media/uapi/v4l/pixfmt-meta-intel-ipu3.rst | 181 ++
include/uapi/linux/intel-ipu3.h| 2819
3 files changed, 3001 insertions(+)
create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
create mode 100644 include/uapi/linux/intel-ipu3.h
diff --git a/Documentation/media/uapi/v4l/meta-formats.rst
b/Documentation/media/uapi/v4l/meta-formats.rst
index cf971d5..eafc534 100644
--- a/Documentation/media/uapi/v4l/meta-formats.rst
+++ b/Documentation/media/uapi/v4l/meta-formats.rst
@@ -12,6 +12,7 @@ These formats are used for the :ref:`metadata` interface only.
.. toctree::
:maxdepth: 1
+pixfmt-meta-intel-ipu3
pixfmt-meta-d4xx
pixfmt-meta-uvc
pixfmt-meta-vsp1-hgo
diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
new file mode 100644
index 000..23b945b
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
@@ -0,0 +1,181 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _intel-ipu3:
+
+**
+V4L2_META_FMT_IPU3_PARAMS ('ip3p'), V4L2_META_FMT_IPU3_3A ('ip3s')
+**
+
+.. c:type:: ipu3_uapi_stats_3a
+
+3A statistics
+=
+
+For IPU3 ImgU, the 3A statistics accelerators collect different statistics over
+an input bayer frame. Those statistics, defined in data struct
+:c:type:`ipu3_uapi_stats_3a`, are meta output obtained from "ipu3-imgu 3a stat"
+video node, which are then passed to user space for statistics analysis
+using :c:type:`v4l2_meta_format` interface.
+
+The statistics collected are AWB (Auto-white balance) RGBS (Red, Green, Blue
and
+Saturation measure) cells, AWB filter response, AF (Auto-focus) filter
response,
+and AE (Auto-exposure) histogram.
+
+struct :c:type:`ipu3_uapi_4a_config` saves configurable parameters for all
above.
+
+
+.. code-block:: c
+
+
+ struct ipu3_uapi_stats_3a {
+ struct ipu3_uapi_awb_raw_buffer awb_raw_buffer
+__attribute__((aligned(32)));
+ struct ipu3_uapi_ae_raw_buffer_aligned
+ ae_raw_buffer[IPU3_UAPI_MAX_STRIPES];
+ struct ipu3_uapi_af_raw_buffer af_raw_buffer;
+ struct ipu3_uapi_awb_fr_raw_buffer awb_fr_raw_buffer;
+ struct ipu3_uapi_4a_config stats_4a_config;
+ __u32 ae_join_buffers;
+ __u8 padding[28];
+ struct ipu3_uapi_stats_3a_bubble_info_per_stripe
+ stats_3a_bubble_per_stripe;
+ struct ipu3_uapi_ff_status stats_3a_status;
+ } __packed;
+
+
+.. c:type:: ipu3_uapi_params
+
+Pipeline parameters
+===
+
+IPU3 pipeline has a number of image processing stages, each of which takes a
+set of parameters as input. The major stages of pipelines are shown here:
+
+Raw pixels -> Bayer Downscaling -> Optical Black Correction ->
+
+Linearization -> Lens Shading Correction -> White Balance / Exposure /
+
+Focus Apply -> Bayer Noise Reduction -> ANR -> Demosaicing -> Color
+
+Correction Matrix -> Gamma correction -> Color Space Conversion ->
+
+Chroma Down Scaling -> Chromatic Noise Reduction -> Total Color
+
+Correction -> XNR3 -> TNR -> DDR
+
+The table below presents a description of the above algorithms.
+
+
===
+NameDescription
+
===
+Optical Black Correction Optical Black Correction block subtracts a pre-defined
+value from the respective pixel values to obtain better
+image quality.
+Defined in :c:type:`ipu3_uapi_obgrid_param`.
+Linearization This algo block uses linearization parameters to
+address non-linearity sensor effects. The Lookup table
+table is defined in
+:c:type:`ipu3_uapi_isp_lin_vmem_params`.
+SHD Lens shading correction is used to correct spatial
+non-uniformity of the pixel response due to optical
+lens shading. This is done by applying a different gain
+for each pixel. The gain, black level etc are
+configured in :c:type:`ipu3_uapi_shd_config_static`.
+BNR Bayer noise reduction block removes image noise by
+applying a bilateral filter.
+
[PATCH v7 12/16] intel-ipu3: css: Initialize css hardware
This patch implements the functions to initialize
and configure IPU3 h/w such as clock, irq and power.
Signed-off-by: Yong Zhi
Signed-off-by: Tomasz Figa
---
drivers/media/pci/intel/ipu3/ipu3-css.c | 537
drivers/media/pci/intel/ipu3/ipu3-css.h | 203
2 files changed, 740 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.h
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css.c
b/drivers/media/pci/intel/ipu3/ipu3-css.c
new file mode 100644
index 000..164830f
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-css.c
@@ -0,0 +1,537 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+#include
+
+#include "ipu3-css.h"
+#include "ipu3-css-fw.h"
+#include "ipu3-css-params.h"
+#include "ipu3-dmamap.h"
+#include "ipu3-tables.h"
+
+/* IRQ configuration */
+#define IMGU_IRQCTRL_IRQ_MASK (IMGU_IRQCTRL_IRQ_SP1 | \
+IMGU_IRQCTRL_IRQ_SP2 | \
+IMGU_IRQCTRL_IRQ_SW_PIN(0) | \
+IMGU_IRQCTRL_IRQ_SW_PIN(1))
+
+/*** css hw ***/
+
+/* In the style of writesl() defined in include/asm-generic/io.h */
+static inline void writes(const void *mem, ssize_t count, void __iomem *addr)
+{
+ if (count >= 4) {
+ const u32 *buf = mem;
+
+ count /= 4;
+ do {
+ writel(*buf++, addr);
+ addr += 4;
+ } while (--count);
+ }
+}
+
+/* Wait until register `reg', masked with `mask', becomes `cmp' */
+static int ipu3_hw_wait(void __iomem *base, int reg, u32 mask, u32 cmp)
+{
+ u32 val;
+
+ return readl_poll_timeout(base + reg, val, (val & mask) == cmp,
+ 1000, 100 * 1000);
+}
+
+/* Initialize the IPU3 CSS hardware and associated h/w blocks */
+
+int ipu3_css_set_powerup(struct device *dev, void __iomem *base)
+{
+ static const unsigned int freq = 450;
+ u32 pm_ctrl, state, val;
+
+ dev_dbg(dev, "%s\n", __func__);
+ /* Clear the CSS busy signal */
+ readl(base + IMGU_REG_GP_BUSY);
+ writel(0, base + IMGU_REG_GP_BUSY);
+
+ /* Wait for idle signal */
+ if (ipu3_hw_wait(base, IMGU_REG_STATE, IMGU_STATE_IDLE_STS,
+IMGU_STATE_IDLE_STS)) {
+ dev_err(dev, "failed to set CSS idle\n");
+ goto fail;
+ }
+
+ /* Reset the css */
+ writel(readl(base + IMGU_REG_PM_CTRL) | IMGU_PM_CTRL_FORCE_RESET,
+ base + IMGU_REG_PM_CTRL);
+
+ usleep_range(200, 300);
+
+ /** Prepare CSS */
+
+ pm_ctrl = readl(base + IMGU_REG_PM_CTRL);
+ state = readl(base + IMGU_REG_STATE);
+
+ dev_dbg(dev, "CSS pm_ctrl 0x%x state 0x%x (power %s)\n",
+ pm_ctrl, state, state & IMGU_STATE_POWER_DOWN ? "down" : "up");
+
+ /* Power up CSS using wrapper */
+ if (state & IMGU_STATE_POWER_DOWN) {
+ writel(IMGU_PM_CTRL_RACE_TO_HALT | IMGU_PM_CTRL_START,
+ base + IMGU_REG_PM_CTRL);
+ if (ipu3_hw_wait(base, IMGU_REG_PM_CTRL,
+IMGU_PM_CTRL_START, 0)) {
+ dev_err(dev, "failed to power up CSS\n");
+ goto fail;
+ }
+ usleep_range(2000, 3000);
+ } else {
+ writel(IMGU_PM_CTRL_RACE_TO_HALT, base + IMGU_REG_PM_CTRL);
+ }
+
+ /* Set the busy bit */
+ writel(readl(base + IMGU_REG_GP_BUSY) | 1, base + IMGU_REG_GP_BUSY);
+
+ /* Set CSS clock frequency */
+ pm_ctrl = readl(base + IMGU_REG_PM_CTRL);
+ val = pm_ctrl & ~(IMGU_PM_CTRL_CSS_PWRDN | IMGU_PM_CTRL_RST_AT_EOF);
+ writel(val, base + IMGU_REG_PM_CTRL);
+ writel(0, base + IMGU_REG_GP_BUSY);
+ if (ipu3_hw_wait(base, IMGU_REG_STATE,
+IMGU_STATE_PWRDNM_FSM_MASK, 0)) {
+ dev_err(dev, "failed to pwrdn CSS\n");
+ goto fail;
+ }
+ val = (freq / IMGU_SYSTEM_REQ_FREQ_DIVIDER) & IMGU_SYSTEM_REQ_FREQ_MASK;
+ writel(val, base + IMGU_REG_SYSTEM_REQ);
+ writel(1, base + IMGU_REG_GP_BUSY);
+ writel(readl(base + IMGU_REG_PM_CTRL) | IMGU_PM_CTRL_FORCE_HALT,
+ base + IMGU_REG_PM_CTRL);
+ if (ipu3_hw_wait(base, IMGU_REG_STATE, IMGU_STATE_HALT_STS,
+IMGU_STATE_HALT_STS)) {
+ dev_err(dev, "failed to halt CSS\n");
+ goto fail;
+ }
+
+ writel(readl(base + IMGU_REG_PM_CTRL) | IMGU_PM_CTRL_START,
+ base + IMGU_REG_PM_CTRL);
+ if (ipu3_hw_wait(base, IMGU_REG_PM_CTRL, IMGU_PM_C
[PATCH v7 06/16] intel-ipu3: mmu: Implement driver
From: Tomasz Figa
This driver translates IO virtual address to physical
address based on two levels page tables.
Signed-off-by: Tomasz Figa
Signed-off-by: Yong Zhi
---
drivers/media/pci/intel/ipu3/ipu3-mmu.c | 560
drivers/media/pci/intel/ipu3/ipu3-mmu.h | 35 ++
2 files changed, 595 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.h
diff --git a/drivers/media/pci/intel/ipu3/ipu3-mmu.c
b/drivers/media/pci/intel/ipu3/ipu3-mmu.c
new file mode 100644
index 000..b66734a
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-mmu.c
@@ -0,0 +1,560 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Intel Corporation.
+ * Copyright 2018 Google LLC.
+ *
+ * Author: Tuukka Toivonen
+ * Author: Sakari Ailus
+ * Author: Samu Onkalo
+ * Author: Tomasz Figa
+ *
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+
+#include "ipu3-mmu.h"
+
+#define IPU3_PAGE_SHIFT12
+#define IPU3_PAGE_SIZE (1UL << IPU3_PAGE_SHIFT)
+
+#define IPU3_PT_BITS 10
+#define IPU3_PT_PTES (1UL << IPU3_PT_BITS)
+#define IPU3_PT_SIZE (IPU3_PT_PTES << 2)
+#define IPU3_PT_ORDER (IPU3_PT_SIZE >> PAGE_SHIFT)
+
+#define IPU3_ADDR2PTE(addr)((addr) >> IPU3_PAGE_SHIFT)
+#define IPU3_PTE2ADDR(pte) ((phys_addr_t)(pte) << IPU3_PAGE_SHIFT)
+
+#define IPU3_L2PT_SHIFTIPU3_PT_BITS
+#define IPU3_L2PT_MASK ((1UL << IPU3_L2PT_SHIFT) - 1)
+
+#define IPU3_L1PT_SHIFTIPU3_PT_BITS
+#define IPU3_L1PT_MASK ((1UL << IPU3_L1PT_SHIFT) - 1)
+
+#define IPU3_MMU_ADDRESS_BITS (IPU3_PAGE_SHIFT + \
+IPU3_L2PT_SHIFT + \
+IPU3_L1PT_SHIFT)
+
+#define IMGU_REG_BASE 0x4000
+#define REG_TLB_INVALIDATE (IMGU_REG_BASE + 0x300)
+#define TLB_INVALIDATE 1
+#define REG_L1_PHYS(IMGU_REG_BASE + 0x304) /* 27-bit pfn */
+#define REG_GP_HALT(IMGU_REG_BASE + 0x5dc)
+#define REG_GP_HALTED (IMGU_REG_BASE + 0x5e0)
+
+struct ipu3_mmu {
+ struct device *dev;
+ void __iomem *base;
+ /* protect access to l2pts, l1pt */
+ spinlock_t lock;
+
+ void *dummy_page;
+ u32 dummy_page_pteval;
+
+ u32 *dummy_l2pt;
+ u32 dummy_l2pt_pteval;
+
+ u32 **l2pts;
+ u32 *l1pt;
+
+ struct ipu3_mmu_info geometry;
+};
+
+static inline struct ipu3_mmu *to_ipu3_mmu(struct ipu3_mmu_info *info)
+{
+ return container_of(info, struct ipu3_mmu, geometry);
+}
+
+/**
+ * ipu3_mmu_tlb_invalidate - invalidate translation look-aside buffer
+ * @mmu: MMU to perform the invalidate operation on
+ *
+ * This function invalidates the whole TLB. Must be called when the hardware
+ * is powered on.
+ */
+static void ipu3_mmu_tlb_invalidate(struct ipu3_mmu *mmu)
+{
+ writel(TLB_INVALIDATE, mmu->base + REG_TLB_INVALIDATE);
+}
+
+static void call_if_ipu3_is_powered(struct ipu3_mmu *mmu,
+ void (*func)(struct ipu3_mmu *mmu))
+{
+ pm_runtime_get_noresume(mmu->dev);
+ if (pm_runtime_active(mmu->dev))
+ func(mmu);
+ pm_runtime_put(mmu->dev);
+}
+
+/**
+ * ipu3_mmu_set_halt - set CIO gate halt bit
+ * @mmu: MMU to set the CIO gate bit in.
+ * @halt: Desired state of the gate bit.
+ *
+ * This function sets the CIO gate bit that controls whether external memory
+ * accesses are allowed. Must be called when the hardware is powered on.
+ */
+static void ipu3_mmu_set_halt(struct ipu3_mmu *mmu, bool halt)
+{
+ int ret;
+ u32 val;
+
+ writel(halt, mmu->base + REG_GP_HALT);
+ ret = readl_poll_timeout(mmu->base + REG_GP_HALTED,
+val, (val & 1) == halt, 1000, 10);
+
+ if (ret)
+ dev_err(mmu->dev, "failed to %s CIO gate halt\n",
+ halt ? "set" : "clear");
+}
+
+/**
+ * ipu3_mmu_alloc_page_table - allocate a pre-filled page table
+ * @pteval: Value to initialize for page table entries with.
+ *
+ * Return: Pointer to allocated page table or NULL on failure.
+ */
+static u32 *ipu3_mmu_alloc_page_table(u32 pteval)
+{
+ u32 *pt;
+ int pte;
+
+ pt = (u32 *)__get_free_page(GFP_KERNEL);
+ if (!pt)
+ return NULL;
+
+ for (pte = 0; pte < IPU3_PT_PTES; pte++)
+ pt[pte] = pteval;
+
+ set_memory_uc((unsigned long int)pt, IPU3_PT_ORDER);
+
+ return pt;
+}
+
+/**
+ * ipu3_mmu_free_page_table - free page table
+ * @pt: Page table to free.
+ */
+static void ipu3_mmu_free_page_table(u32 *pt)
+{
+ set_memory_wb((unsigned long int)pt, IPU3_PT_ORDER);
+ free_page((unsigned long)pt);
+}
+
+/**
+ * address_to_pte_idx - split IOVA into L1 and L2 p
[PATCH v7 02/16] doc-rst: Add Intel IPU3 documentation
From: Rajmohan Mani
This patch adds the details about the IPU3 Imaging Unit driver.
Change-Id: I560cecf673df2dcc3ec72767cf8077708d649656
Signed-off-by: Rajmohan Mani
---
Documentation/media/v4l-drivers/index.rst | 1 +
Documentation/media/v4l-drivers/ipu3.rst | 326 ++
2 files changed, 327 insertions(+)
create mode 100644 Documentation/media/v4l-drivers/ipu3.rst
diff --git a/Documentation/media/v4l-drivers/index.rst
b/Documentation/media/v4l-drivers/index.rst
index 679238e..179a393 100644
--- a/Documentation/media/v4l-drivers/index.rst
+++ b/Documentation/media/v4l-drivers/index.rst
@@ -44,6 +44,7 @@ For more details see the file COPYING in the source
distribution of Linux.
davinci-vpbe
fimc
imx
+ ipu3
ivtv
max2175
meye
diff --git a/Documentation/media/v4l-drivers/ipu3.rst
b/Documentation/media/v4l-drivers/ipu3.rst
new file mode 100644
index 000..045bf42
--- /dev/null
+++ b/Documentation/media/v4l-drivers/ipu3.rst
@@ -0,0 +1,326 @@
+.. include::
+
+===
+Intel Image Processing Unit 3 (IPU3) Imaging Unit (ImgU) driver
+===
+
+Copyright |copy| 2018 Intel Corporation
+
+Introduction
+
+
+This file documents Intel IPU3 (3rd generation Image Processing Unit) Imaging
+Unit driver located under drivers/media/pci/intel/ipu3.
+
+The Intel IPU3 found in certain Kaby Lake (as well as certain Sky Lake)
+platforms (U/Y processor lines) is made up of two parts namely Imaging Unit
+(ImgU) and CIO2 device (MIPI CSI2 receiver).
+
+The CIO2 device receives the raw bayer data from the sensors and outputs the
+frames in a format that is specific to IPU3 (for consumption by IPU3 ImgU).
+CIO2 driver is available as drivers/media/pci/intel/ipu3/ipu3-cio2* and is
+enabled through the CONFIG_VIDEO_IPU3_CIO2 config option.
+
+The Imaging Unit (ImgU) is responsible for processing images captured
+through IPU3 CIO2 device. The ImgU driver sources can be found under
+drivers/media/pci/intel/ipu3 directory. The driver is enabled through the
+CONFIG_VIDEO_IPU3_IMGU config option.
+
+The two driver modules are named ipu3-csi2 and ipu3-imgu, respectively.
+
+The driver has been tested on Kaby Lake platforms (U/Y processor lines).
+
+The driver implements V4L2, Media controller and V4L2 sub-device interfaces.
+Camera sensors that have CSI-2 bus, which are connected to the IPU3 CIO2
+device are supported. Support for lens and flash drivers depends on the
+above sensors.
+
+ImgU device nodes
+=
+
+The ImgU is represented as two V4L2 subdevs, each of which provides a V4L2
+subdev interface to the user space.
+
+Each V4L2 subdev represents a pipe, which can support a maximum of 2
+streams. A private ioctl can be used to configure the mode (video or still)
+of the pipe.
+
+This helps to support advanced camera features like Continuous View Finder
+(CVF) and Snapshot During Video(SDV).
+
+CIO2 device
+===
+
+The CIO2 is represented as a single V4L2 subdev, which provides a V4L2 subdev
+interface to the user space. There is a video node for each CSI-2 receiver,
+with a single media controller interface for the entire device.
+
+Media controller
+
+
+The media device interface allows to configure the ImgU links, which defines
+the behavior of the IPU3 firmware.
+
+Device operation
+
+
+With IPU3, once the input video node ("ipu3-imgu 0/1":0,
+in : format) is queued with buffer (in packed raw bayer
+format), IPU3 ISP starts processing the buffer and produces the video output
+in YUV format and statistics output on respective output nodes. The driver
+is expected to have buffers ready for all of parameter, output and
+statistics nodes, when input video node is queued with buffer.
+
+At a minimum, all of input, main output, 3A statistics and viewfinder
+video nodes should be enabled for IPU3 to start image processing.
+
+Each ImgU V4L2 subdev has the following set of video nodes.
+
+input, output and viewfinder video nodes
+
+
+The frames (in packed raw bayer format specific to IPU3) received by the
+input video node is processed by the IPU3 Imaging Unit and is output to 2
+video nodes, with each targeting different purpose (main output and viewfinder
+output).
+
+Details on raw bayer format specific to IPU3 can be found as below.
+Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
+
+The driver supports V4L2 Video Capture Interface as defined at :ref:`devices`.
+
+Only the multi-planar API is supported. More details can be found at
+:ref:`planar-apis`.
+
+
+parameters video node
+-
+
+The parameter video node receives the ISP algorithm parameters that are used
+to configure how the ISP algorithms process the image.
+
+Details on raw bayer format specific to IPU3 can be found as below.
[PATCH v7 07/16] intel-ipu3: Implement DMA mapping functions
From: Tomasz Figa
This driver uses IOVA space for buffer mapping through IPU3 MMU
to transfer data between imaging pipelines and system DDR.
Signed-off-by: Tomasz Figa
Signed-off-by: Yong Zhi
---
drivers/media/pci/intel/ipu3/ipu3-dmamap.c | 270 +
drivers/media/pci/intel/ipu3/ipu3-dmamap.h | 22 +++
2 files changed, 292 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.h
diff --git a/drivers/media/pci/intel/ipu3/ipu3-dmamap.c
b/drivers/media/pci/intel/ipu3/ipu3-dmamap.c
new file mode 100644
index 000..93a393d
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-dmamap.c
@@ -0,0 +1,270 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Intel Corporation
+ * Copyright 2018 Google LLC.
+ *
+ * Author: Tomasz Figa
+ * Author: Yong Zhi
+ */
+
+#include
+
+#include "ipu3.h"
+#include "ipu3-css-pool.h"
+#include "ipu3-mmu.h"
+
+/*
+ * Free a buffer allocated by ipu3_dmamap_alloc_buffer()
+ */
+static void ipu3_dmamap_free_buffer(struct page **pages,
+ size_t size)
+{
+ int count = size >> PAGE_SHIFT;
+
+ while (count--)
+ __free_page(pages[count]);
+ kvfree(pages);
+}
+
+/*
+ * Based on the implementation of __iommu_dma_alloc_pages()
+ * defined in drivers/iommu/dma-iommu.c
+ */
+static struct page **ipu3_dmamap_alloc_buffer(size_t size,
+ unsigned long order_mask,
+ gfp_t gfp)
+{
+ struct page **pages;
+ unsigned int i = 0, count = size >> PAGE_SHIFT;
+ const gfp_t high_order_gfp = __GFP_NOWARN | __GFP_NORETRY;
+
+ /* Allocate mem for array of page ptrs */
+ pages = kvmalloc_array(count, sizeof(*pages), GFP_KERNEL);
+
+ if (!pages)
+ return NULL;
+
+ order_mask &= (2U << MAX_ORDER) - 1;
+ if (!order_mask)
+ return NULL;
+
+ gfp |= __GFP_HIGHMEM | __GFP_ZERO;
+
+ while (count) {
+ struct page *page = NULL;
+ unsigned int order_size;
+
+ for (order_mask &= (2U << __fls(count)) - 1;
+order_mask; order_mask &= ~order_size) {
+ unsigned int order = __fls(order_mask);
+
+ order_size = 1U << order;
+ page = alloc_pages((order_mask - order_size) ?
+ gfp | high_order_gfp : gfp, order);
+ if (!page)
+ continue;
+ if (!order)
+ break;
+ if (!PageCompound(page)) {
+ split_page(page, order);
+ break;
+ }
+
+ __free_pages(page, order);
+ }
+ if (!page) {
+ ipu3_dmamap_free_buffer(pages, i << PAGE_SHIFT);
+ return NULL;
+ }
+ count -= order_size;
+ while (order_size--)
+ pages[i++] = page++;
+ }
+
+ return pages;
+}
+
+/**
+ * ipu3_dmamap_alloc - allocate and map a buffer into KVA
+ * @imgu: struct device pointer
+ * @map: struct to store mapping variables
+ * @len: size required
+ *
+ * Returns:
+ * KVA on success
+ * %NULL on failure
+ */
+void *ipu3_dmamap_alloc(struct imgu_device *imgu, struct ipu3_css_map *map,
+ size_t len)
+{
+ unsigned long shift = iova_shift(>iova_domain);
+ unsigned int alloc_sizes = imgu->mmu->pgsize_bitmap;
+ struct device *dev = >pci_dev->dev;
+ size_t size = PAGE_ALIGN(len);
+ struct page **pages;
+ dma_addr_t iovaddr;
+ struct iova *iova;
+ int i, rval;
+
+ dev_dbg(dev, "%s: allocating %zu\n", __func__, size);
+
+ iova = alloc_iova(>iova_domain, size >> shift,
+ imgu->mmu->aperture_end >> shift, 0);
+ if (!iova)
+ return NULL;
+
+ pages = ipu3_dmamap_alloc_buffer(size, alloc_sizes >> PAGE_SHIFT,
+GFP_KERNEL);
+ if (!pages)
+ goto out_free_iova;
+
+ /* Call IOMMU driver to setup pgt */
+ iovaddr = iova_dma_addr(>iova_domain, iova);
+ for (i = 0; i < size / PAGE_SIZE; ++i) {
+ rval = ipu3_mmu_map(imgu->mmu, iovaddr,
+ page_to_phys(pages[i]), PAGE_SIZE);
+ if (rval)
+ goto out_unmap;
+
+ iovaddr += PAGE_SIZE;
+ }
+
+ /* Now grab a virtual region */
+ map->vma = __get_vm_area(size, VM_USERMAP, VMALL
[PATCH v7 04/16] intel-ipu3: abi: Add register definitions and enum
Add macros and enums used for IPU3 firmware interface. Signed-off-by: Yong Zhi Signed-off-by: Rajmohan Mani --- drivers/media/pci/intel/ipu3/ipu3-abi.h | 661 1 file changed, 661 insertions(+) create mode 100644 drivers/media/pci/intel/ipu3/ipu3-abi.h diff --git a/drivers/media/pci/intel/ipu3/ipu3-abi.h b/drivers/media/pci/intel/ipu3/ipu3-abi.h new file mode 100644 index 000..ac08ad3 --- /dev/null +++ b/drivers/media/pci/intel/ipu3/ipu3-abi.h @@ -0,0 +1,661 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2018 Intel Corporation */ + +#ifndef __IPU3_ABI_H +#define __IPU3_ABI_H + +#include + +/*** IMGU Hardware information ***/ + +typedef u32 imgu_addr_t; + +#define IMGU_ISP_VMEM_ALIGN128 +#define IMGU_DVS_BLOCK_W 64 +#define IMGU_DVS_BLOCK_H 32 +#define IMGU_GDC_BUF_X (2 * IMGU_DVS_BLOCK_W) +#define IMGU_GDC_BUF_Y IMGU_DVS_BLOCK_H +/* n = 0..1 */ +#define IMGU_SP_PMEM_BASE(n) (0x2 + (n) * 0x4000) +#define IMGU_MAX_BQ_GRID_WIDTH 80 +#define IMGU_MAX_BQ_GRID_HEIGHT60 +#define IMGU_OBGRID_TILE_SIZE 16 +#define IMGU_PIXELS_PER_WORD 50 +#define IMGU_BYTES_PER_WORD64 +#define IMGU_STRIPE_FIXED_HALF_OVERLAP 2 +#define IMGU_SHD_SETS 3 +#define IMGU_BDS_MIN_CLIP_VAL 0 +#define IMGU_BDS_MAX_CLIP_VAL 2 + +#define IMGU_ABI_AWB_MAX_CELLS_PER_SET 160 +#define IMGU_ABI_AF_MAX_CELLS_PER_SET 32 +#define IMGU_ABI_AWB_FR_MAX_CELLS_PER_SET 32 + +#define IMGU_ABI_ACC_OP_IDLE 0 +#define IMGU_ABI_ACC_OP_END_OF_ACK 1 +#define IMGU_ABI_ACC_OP_END_OF_OPS 2 +#define IMGU_ABI_ACC_OP_NO_OPS 3 + +#define IMGU_ABI_ACC_OPTYPE_PROCESS_LINES 0 +#define IMGU_ABI_ACC_OPTYPE_TRANSFER_DATA 1 + +/* Register definitions */ + +/* PM_CTRL_0_5_0_IMGHMMADR */ +#define IMGU_REG_PM_CTRL 0x0 +#define IMGU_PM_CTRL_START BIT(0) +#define IMGU_PM_CTRL_CFG_DONE BIT(1) +#define IMGU_PM_CTRL_RACE_TO_HALT BIT(2) +#define IMGU_PM_CTRL_NACK_ALL BIT(3) +#define IMGU_PM_CTRL_CSS_PWRDN BIT(4) +#define IMGU_PM_CTRL_RST_AT_EOFBIT(5) +#define IMGU_PM_CTRL_FORCE_HALTBIT(6) +#define IMGU_PM_CTRL_FORCE_UNHALT BIT(7) +#define IMGU_PM_CTRL_FORCE_PWRDN BIT(8) +#define IMGU_PM_CTRL_FORCE_RESET BIT(9) + +/* SYSTEM_REQ_0_5_0_IMGHMMADR */ +#define IMGU_REG_SYSTEM_REQ0x18 +#define IMGU_SYSTEM_REQ_FREQ_MASK 0x3f +#define IMGU_SYSTEM_REQ_FREQ_DIVIDER 25 +#define IMGU_REG_INT_STATUS0x30 +#define IMGU_REG_INT_ENABLE0x34 +#define IMGU_REG_INT_CSS_IRQ BIT(31) +/* STATE_0_5_0_IMGHMMADR */ +#define IMGU_REG_STATE 0x130 +#define IMGU_STATE_HALT_STSBIT(0) +#define IMGU_STATE_IDLE_STSBIT(1) +#define IMGU_STATE_POWER_UPBIT(2) +#define IMGU_STATE_POWER_DOWN BIT(3) +#define IMGU_STATE_CSS_BUSY_MASK 0xc0 +#define IMGU_STATE_PM_FSM_MASK 0x180 +#define IMGU_STATE_PWRDNM_FSM_MASK 0x1E0 +/* PM_STS_0_5_0_IMGHMMADR */ +#define IMGU_REG_PM_STS0x140 + +#define IMGU_REG_BASE 0x4000 + +#define IMGU_REG_ISP_CTRL (IMGU_REG_BASE + 0x00) +#define IMGU_CTRL_RST BIT(0) +#define IMGU_CTRL_STARTBIT(1) +#define IMGU_CTRL_BREAKBIT(2) +#define IMGU_CTRL_RUN BIT(3) +#define IMGU_CTRL_BROKEN BIT(4) +#define IMGU_CTRL_IDLE BIT(5) +#define IMGU_CTRL_SLEEPING BIT(6) +#define IMGU_CTRL_STALLING BIT(7) +#define IMGU_CTRL_IRQ_CLEARBIT(8) +#define IMGU_CTRL_IRQ_READYBIT(10) +#define IMGU_CTRL_IRQ_SLEEPING BIT(11) +#define IMGU_CTRL_ICACHE_INV BIT(12) +#define IMGU_CTRL_IPREFETCH_EN BIT(13) +#define IMGU_REG_ISP_START_ADDR(IMGU_REG_BASE + 0x04) +#define IMGU_REG_ISP_ICACHE_ADDR (IMGU_REG_BASE + 0x10) +#define IMGU_REG_ISP_PC(IMGU_REG_BASE + 0x1c) + +/* SP Registers, sp = 0:SP0; 1:SP1 */ +#define IMGU_REG_SP_CTRL(sp) (IMGU_REG_BASE + (sp) * 0x100 + 0x100) + /* For bits in IMGU_REG_SP_CTRL, see IMGU_CTRL_* */ +#define IMGU_REG_SP_START_ADDR(sp) (IMGU_REG_BASE + (sp) * 0x100
[PATCH v7 00/16] Intel IPU3 ImgU patchset
Hi, This series adds support for the Intel IPU3 (Image Processing Unit) ImgU which is essentially a modern memory-to-memory ISP. It implements raw Bayer to YUV image format conversion as well as a large number of other pixel processing algorithms for improving the image quality. Meta data formats are defined for image statistics (3A, i.e. automatic white balance, exposure and focus, histogram and local area contrast enhancement) as well as for the pixel processing algorithm parameters. The documentation for these formats is currently not included in the patchset but will be added in a future version of this set. The algorithm parameters need to be considered specific to a given frame and typically a large number of these parameters change on frame to frame basis. Additionally, the parameters are highly structured (and not a flat space of independent configuration primitives). They also reflect the data structures used by the firmware and the hardware. On top of that, the algorithms require highly specialized user space to make meaningful use of them. For these reasons it has been chosen video buffers to pass the parameters to the device. On individual patches: The heart of ImgU is the CSS, or Camera Subsystem, which contains the image processors and HW accelerators. The 3A statistics and other firmware parameter computation related functions are implemented in patch 11. All IPU3 pipeline default settings can be found in patch 10. To access DDR via ImgU's own memory space, IPU3 is also equipped with its own MMU unit, the driver is implemented in patch 6. Patch 7 uses above driver for DMA mapping operation. The communication between IPU3 firmware and driver is implemented with circular queues in patch 8. Patch 9 provide some utility functions and manage IPU3 fw download and install. The firmware which is called ipu3-fw.bin can be downloaded from: git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git (commit 2c27b0cb02f18c022d8378e0e1abaf8b7ae8188f) Firmware ABI is defined in patches 4 and 5. Patches 12 and 13 are of the same file, the former contains all h/w programming related code, the latter implements interface functions for access fw & hw capabilities. Patch 14 has a dependency on Sakari's V4L2_BUF_TYPE_META_OUTPUT work: https://patchwork.kernel.org/patch/9976295/> Patch 15 represents the top level that glues all of the other components together, passing arguments between the components. Patch 16 is a recent effort to extend v6 for advanced camera features like Continuous View Finder (CVF) and Snapshot During Video(SDV) support. Link to user space implementation: git clone https://chromium.googlesource.com/chromiumos/platform/arc-camera ImgU media topology print: # media-ctl -d /dev/media0 -p Media controller API version 4.19.0 Media device information driver ipu3-imgu model ipu3-imgu serial bus infoPCI::00:05.0 hw revision 0x80862015 driver version 4.19.0 Device topology - entity 1: ipu3-imgu 0 (5 pads, 5 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 pad0: Sink [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown crop:(0,0)/1920x1080 compose:(0,0)/1920x1080] <- "ipu3-imgu 0 input":0 [] pad1: Sink [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] <- "ipu3-imgu 0 parameters":0 [] pad2: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 0 output":0 [] pad3: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 0 viewfinder":0 [] pad4: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 0 3a stat":0 [] - entity 7: ipu3-imgu 1 (5 pads, 5 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev1 pad0: Sink [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown crop:(0,0)/1920x1080 compose:(0,0)/1920x1080] <- "ipu3-imgu 1 input":0 [] pad1: Sink [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] <- "ipu3-imgu 1 parameters":0 [] pad2: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 1 output":0 [] pad3: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 1 viewfinder":0 [] pad4: Source [fmt:UYVY8_2X8/1920x1080 field:none colorspace:unknown] -> "ipu3-imgu 1 3a stat":0 [] - entity 17: ipu3-imgu 0 input (1 pad, 1 link) type Node subtype V4L flags 0 device node name
RE: [PATCH v6 12/12] intel-ipu3: Add imgu top level pci device driver
Hi, Tomasz,
> -Original Message-
> From: linux-media-ow...@vger.kernel.org [mailto:linux-media-
> ow...@vger.kernel.org] On Behalf Of Tomasz Figa
> Sent: Tuesday, September 18, 2018 10:23 AM
> To: Zhi, Yong
> Cc: Linux Media Mailing List ; Sakari Ailus
> ; Mani, Rajmohan
> ; Toivonen, Tuukka
> ; Hu, Jerry W ; Zheng,
> Jian Xu
> Subject: Re: [PATCH v6 12/12] intel-ipu3: Add imgu top level pci device
> driver
>
> On Mon, Sep 17, 2018 at 5:20 AM Zhi, Yong wrote:
> >
> > Hi, Tomasz,
> >
> > Thanks for the code review.
> >
> > > -Original Message-
> > > From: linux-media-ow...@vger.kernel.org [mailto:linux-media-
> > > ow...@vger.kernel.org] On Behalf Of Tomasz Figa
> > > Sent: Monday, July 2, 2018 3:08 AM
> > > To: Zhi, Yong
> > > Cc: Linux Media Mailing List ; Sakari
> > > Ailus ; Mani, Rajmohan
> > > ; Toivonen, Tuukka
> > > ; Hu, Jerry W ;
> > > Zheng, Jian Xu
> > > Subject: Re: [PATCH v6 12/12] intel-ipu3: Add imgu top level pci
> > > device driver
> > >
> > > Hi Yong,
> > >
> > > On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi
> wrote:
> > > > +/*
> > > > + * Queue as many buffers to CSS as possible. If all buffers don't
> > > > +fit into
> > > > + * CSS buffer queues, they remain unqueued and will be queued later.
> > > > + */
> > > > +int imgu_queue_buffers(struct imgu_device *imgu, bool initial) {
> > > > + unsigned int node;
> > > > + int r = 0;
> > > > + struct imgu_buffer *ibuf;
> > > > +
> > > > + if (!ipu3_css_is_streaming(>css))
> > > > + return 0;
> > > > +
> > > > + mutex_lock(>lock);
> > > > +
> > > > + /* Buffer set is queued to FW only when input buffer is ready */
> > > > + if (!imgu_queue_getbuf(imgu, IMGU_NODE_IN)) {
> > > > + mutex_unlock(>lock);
> > > > + return 0;
> > > > + }
> > > > + for (node = IMGU_NODE_IN + 1; 1; node = (node + 1) %
> > > > + IMGU_NODE_NUM) {
> > >
> > > Shouldn't we make (node != IMGU_NODE_IN ||
> imgu_queue_getbuf(imgu,
> > > IMGU_NODE_IN)) the condition here, rather than 1?
> > >
> > > This would also let us remove the explicit call to
> > > imgu_queue_getbuf() above the loop.
> > >
> >
> > Ack, will make the suggested changes regarding the loop condition
> evaluation.
>
> Just to make sure, the suggestion also includes starting from
> IMGU_NODE_IN (not + 1), i.e.
>
> for (node = IMGU_NODE_IN;
> node != IMGU_NODE_IN || imgu_queue_getbuf(imgu, IMGU_NODE_IN);
> node = (node + 1) % IMGU_NODE_NUM) {
> // ...
> }
>
Thanks for the clarification.
> > > > +static int __maybe_unused imgu_suspend(struct device *dev) {
> > > > + struct pci_dev *pci_dev = to_pci_dev(dev);
> > > > + struct imgu_device *imgu = pci_get_drvdata(pci_dev);
> > > > + unsigned long expire;
> > > > +
> > > > + dev_dbg(dev, "enter %s\n", __func__);
> > > > + imgu->suspend_in_stream = ipu3_css_is_streaming(
> >css);
> > > > + if (!imgu->suspend_in_stream)
> > > > + goto out;
> > > > + /* Block new buffers to be queued to CSS. */
> > > > + atomic_set(>qbuf_barrier, 1);
> > > > + /*
> > > > +* Wait for currently running irq handler to be done so that
> > > > +* no new buffers will be queued to fw later.
> > > > +*/
> > > > + synchronize_irq(pci_dev->irq);
> > > > + /* Wait until all buffers in CSS are done. */
> > > > + expire = jiffies + msecs_to_jiffies(1000);
> > > > + while (!ipu3_css_queue_empty(>css)) {
> > > > + if (time_is_before_jiffies(expire)) {
> > > > + dev_err(dev, "wait buffer drain timeout.\n");
> > > > + break;
> > > > + }
> > > > + }
> > >
> > > Uhm. We struggle to save some power by suspending the device only to
> > > end up with an ugly busy wait that could take even a second here.
> > > This doesn't make any sense.
> > >
> > > We had a working solution using a wait queue in previous revision [1].
> > > What happened to it?
> > >
> > > [1] https://chromium-
> > >
> review.googlesource.com/c/chromiumos/third_party/kernel/+/1029594/2
> > > /drivers/media/pci/intel/ipu3/ipu3.c#b913
> > > (see the left side)
> > >
> >
> > The code here was based on an old version of patch "ipu3-imgu: Avoid
> might sleep operations in suspend callback" at submission, so it did have
> buf_drain_wq, sorry for the confusion.
> >
>
> I guess that means that v7 is going to have the workqueue back? :)
>
Yes, that's the plan.
> Best regards,
> Tomasz
RE: [PATCH v6 06/12] intel-ipu3: css: Add support for firmware management
Hi, Sakari,
> -Original Message-
> From: Sakari Ailus [mailto:sakari.ai...@linux.intel.com]
> Sent: Friday, September 21, 2018 6:52 AM
> To: Zhi, Yong
> Cc: Tomasz Figa ; Linux Media Mailing List me...@vger.kernel.org>; Mani, Rajmohan ;
> Toivonen, Tuukka ; Hu, Jerry W
> ; Zheng, Jian Xu
> Subject: Re: [PATCH v6 06/12] intel-ipu3: css: Add support for firmware
> management
>
> Hi Yong,
>
> On Wed, Sep 19, 2018 at 10:57:55PM +, Zhi, Yong wrote:
> ...
> > > > +struct imgu_abi_osys_frame_params {
> > > > + /* Output pins */
> > > > + __u32 enable;
> > > > + __u32 format; /* enum imgu_abi_osys_format */
> > > > + __u32 flip;
> > > > + __u32 mirror;
> > > > + __u32 tiling; /* enum imgu_abi_osys_tiling */
> > > > + __u32 width;
> > > > + __u32 height;
> > > > + __u32 stride;
> > > > + __u32 scaled;
> > > > +} __packed;
> > > [snip]
> > > > +/* Defect pixel correction */
> > > > +
> > > > +struct imgu_abi_dpc_config {
> > > > + __u8 __reserved[240832];
> > > > +} __packed;
> > >
> > > Do we need this structure? One could just add a reserved field in
> > > the parent structure. Also, just to confirm, is 240832 really the right
> value here?
> > > Where does it come from? Please create a macro for it, possibly
> > > further breaking it down into the values used to compute this number.
> > >
> >
> > We can add a reserved field in the parent structure, the size is based
> > on original definition of dpc config which was removed since it's not
> > enabled/used.
>
> What's your plan with the DPC? If you don't plan to add it now, you could
> as well drop the configuration for that block. If there's a need to add it
> later
> on, you can still do it by defining a new struct for the buffer. Or simply
> adding it at the end of the existing struct while allowing the use of the old
> size without the DPC configuration.
>
> There would be a little extra work to do there by that time when DPC
> support would be added, but OTOH it seems silly to have quarter of a
> megabyte of extra stuff to pass around in a struct that's never used.
>
> --
> Regards,
>
> Sakari Ailus
> sakari.ai...@linux.intel.com
It's a very good point, but as I was informed, there is no plan to update the
abi between the driver and firmware, so the size of imgu_abi_acc_param has not
changed since v1 to maintain the compatibility.
RE: [PATCH v6 06/12] intel-ipu3: css: Add support for firmware management
Hi, Tomasz,
> -Original Message-
> From: linux-media-ow...@vger.kernel.org [mailto:linux-media-
> ow...@vger.kernel.org] On Behalf Of Tomasz Figa
> Sent: Monday, July 2, 2018 2:05 AM
> To: Zhi, Yong
> Cc: Linux Media Mailing List ; Sakari Ailus
> ; Mani, Rajmohan
> ; Toivonen, Tuukka
> ; Hu, Jerry W ; Zheng,
> Jian Xu
> Subject: Re: [PATCH v6 06/12] intel-ipu3: css: Add support for firmware
> management
>
> Hi Yong,
>
> Continuing my review. Sorry for the delay.
>
> On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi wrote:
> >
> > Introduce functions to load and install ImgU FW blobs.
> >
> > Signed-off-by: Yong Zhi
> > ---
> > drivers/media/pci/intel/ipu3/ipu3-abi.h| 1888
>
> > drivers/media/pci/intel/ipu3/ipu3-css-fw.c | 261
> > drivers/media/pci/intel/ipu3/ipu3-css-fw.h | 198 +++
> > 3 files changed, 2347 insertions(+)
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-abi.h
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.c
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.h
> >
> > diff --git a/drivers/media/pci/intel/ipu3/ipu3-abi.h
> > b/drivers/media/pci/intel/ipu3/ipu3-abi.h
> > new file mode 100644
> > index ..24102647a89e
> > --- /dev/null
> > +++ b/drivers/media/pci/intel/ipu3/ipu3-abi.h
> [snip]
> > +/* SYSTEM_REQ_0_5_0_IMGHMMADR */
> > +#define IMGU_REG_SYSTEM_REQ0x18
> > +#define IMGU_SYSTEM_REQ_FREQ_MASK 0x3f
> > +#define IMGU_SYSTEM_REQ_FREQ_DIVIDER 25
> > +#define IMGU_REG_INT_STATUS0x30
> > +#define IMGU_REG_INT_ENABLE0x34
> > +#define IMGU_REG_INT_CSS_IRQ (1 << 31)
>
> BIT(31)
>
Ack.
> [snip]
> > + IMGU_ABI_FRAME_FORMAT_CSI_MIPI_LEGACY_YUV420_8, /*
> Legacy YUV420.
> > +* UY odd line;
> > +* VY even line
> > +*/
> > + IMGU_ABI_FRAME_FORMAT_CSI_MIPI_YUV420_10,/* 10 bit per
> Y/U/V. Y odd
> > + * line; UYVY interleaved
> > + * even line
> > + */
> > + IMGU_ABI_FRAME_FORMAT_YCgCo444_16, /* Internal format for
> > + ISP2.7,
>
> Macros and enums should be uppercase.
>
Ack.
> [snip]
> > +struct imgu_abi_shd_intra_frame_operations_data {
> > + struct imgu_abi_acc_operation
> > + operation_list[IMGU_ABI_SHD_MAX_OPERATIONS]
> IPU3_ALIGN;
> > + struct imgu_abi_acc_process_lines_cmd_data
> > + process_lines_data[IMGU_ABI_SHD_MAX_PROCESS_LINES]
> IPU3_ALIGN;
> > + struct imgu_abi_shd_transfer_luts_set_data
> > + transfer_data[IMGU_ABI_SHD_MAX_TRANSFERS] IPU3_ALIGN;
> > +} __packed;
> > +
> > +struct imgu_abi_shd_config {
> > + struct ipu3_uapi_shd_config_static shd IMGU_ABI_PAD;
> > + struct imgu_abi_shd_intra_frame_operations_data shd_ops
> IMGU_ABI_PAD;
> > + struct ipu3_uapi_shd_lut shd_lut IMGU_ABI_PAD;
>
> Definitions of both IPU3_ALIGN and IMGU_ABI_PAD seem to be equivalent.
> Could we remove one and use the other everywhere?
>
Agree, will remove IMGU_ABI_PAD.
> [snip]
> > +struct imgu_abi_osys_scaler_params {
> > + __u32 inp_buf_y_st_addr;
> > + __u32 inp_buf_y_line_stride;
> > + __u32 inp_buf_y_buffer_stride;
> > + __u32 inp_buf_u_st_addr;
> > + __u32 inp_buf_v_st_addr;
> > + __u32 inp_buf_uv_line_stride;
> > + __u32 inp_buf_uv_buffer_stride;
> > + __u32 inp_buf_chunk_width;
> > + __u32 inp_buf_nr_buffers;
> > + /* Output buffers */
> > + __u32 out_buf_y_st_addr;
> > + __u32 out_buf_y_line_stride;
> > + __u32 out_buf_y_buffer_stride;
> > + __u32 out_buf_u_st_addr;
> > + __u32 out_buf_v_st_addr;
> > + __u32 out_buf_uv_line_stride;
> > + __u32 out_buf_uv_buffer_stride;
> > + __u32 out_buf_nr_buffers;
> > + /* Intermediate buffers */
> > + __u32 int_buf_y_st_addr;
> > + __u32 int_buf_y_line_stride;
> > + __u32 int_buf_u_st_addr;
> > + __u32 int_buf_v_st_addr;
> > + __u32 int_buf_uv_line_stride;
> > + __u32 int_buf_height;
> > + __u32 int_buf_chunk_width;
> &
RE: [PATCH v6 12/12] intel-ipu3: Add imgu top level pci device driver
Hi, Tomasz,
Thanks for the code review.
> -Original Message-
> From: linux-media-ow...@vger.kernel.org [mailto:linux-media-
> ow...@vger.kernel.org] On Behalf Of Tomasz Figa
> Sent: Monday, July 2, 2018 3:08 AM
> To: Zhi, Yong
> Cc: Linux Media Mailing List ; Sakari Ailus
> ; Mani, Rajmohan
> ; Toivonen, Tuukka
> ; Hu, Jerry W ; Zheng,
> Jian Xu
> Subject: Re: [PATCH v6 12/12] intel-ipu3: Add imgu top level pci device
> driver
>
> Hi Yong,
>
> On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi wrote:
> > +/*
> > + * Queue as many buffers to CSS as possible. If all buffers don't fit
> > +into
> > + * CSS buffer queues, they remain unqueued and will be queued later.
> > + */
> > +int imgu_queue_buffers(struct imgu_device *imgu, bool initial) {
> > + unsigned int node;
> > + int r = 0;
> > + struct imgu_buffer *ibuf;
> > +
> > + if (!ipu3_css_is_streaming(>css))
> > + return 0;
> > +
> > + mutex_lock(>lock);
> > +
> > + /* Buffer set is queued to FW only when input buffer is ready */
> > + if (!imgu_queue_getbuf(imgu, IMGU_NODE_IN)) {
> > + mutex_unlock(>lock);
> > + return 0;
> > + }
> > + for (node = IMGU_NODE_IN + 1; 1; node = (node + 1) %
> > + IMGU_NODE_NUM) {
>
> Shouldn't we make (node != IMGU_NODE_IN || imgu_queue_getbuf(imgu,
> IMGU_NODE_IN)) the condition here, rather than 1?
>
> This would also let us remove the explicit call to imgu_queue_getbuf()
> above the loop.
>
Ack, will make the suggested changes regarding the loop condition evaluation.
> > + if (node == IMGU_NODE_VF &&
> > + (imgu->css.pipe_id == IPU3_CSS_PIPE_ID_CAPTURE ||
> > +!imgu->nodes[IMGU_NODE_VF].enabled)) {
> > + continue;
> > + } else if (node == IMGU_NODE_PV &&
> > + (imgu->css.pipe_id == IPU3_CSS_PIPE_ID_VIDEO ||
> > + !imgu->nodes[IMGU_NODE_PV].enabled)) {
> > + continue;
> > + } else if (imgu->queue_enabled[node]) {
> > + struct ipu3_css_buffer *buf =
> > + imgu_queue_getbuf(imgu, node);
> > + int dummy;
> > +
> > + if (!buf)
> > + break;
> > +
> > + r = ipu3_css_buf_queue(>css, buf);
> > + if (r)
> > + break;
> > + dummy = imgu_dummybufs_check(imgu, buf);
> > + if (!dummy)
> > + ibuf = container_of(buf, struct imgu_buffer,
> > + css_buf);
> > + dev_dbg(>pci_dev->dev,
> > + "queue %s %s buffer %d to css da: 0x%08x\n",
> > + dummy ? "dummy" : "user",
> > + imgu_node_map[node].name,
> > + dummy ? 0 : ibuf->vid_buf.vbb.vb2_buf.index,
> > + (u32)buf->daddr);
> > + }
> > + if (node == IMGU_NODE_IN &&
> > + !imgu_queue_getbuf(imgu, IMGU_NODE_IN))
> > + break;
>
> My suggestion to the for loop condition is based on this.
>
Got it.
> > + }
> > + mutex_unlock(>lock);
> > +
> > + if (r && r != -EBUSY)
> > + goto failed;
> > +
> > + return 0;
> > +
> > +failed:
> > + /*
> > +* On error, mark all buffers as failed which are not
> > +* yet queued to CSS
> > +*/
> > + dev_err(>pci_dev->dev,
> > + "failed to queue buffer to CSS on queue %i (%d)\n",
> > + node, r);
> > +
> > + if (initial)
> > + /* If we were called from streamon(), no need to finish
> > bufs */
> > + return r;
> > +
> > + for (node = 0; node < IMGU_NODE_NUM; node++) {
> > + struct imgu_buffer *buf, *buf0;
> > +
> > + if (!imgu->queue_enabled[node])
> > + continue; /* Skip disabled queues */
> > +
> > + mutex_lock(>
RE: [PATCH v6 11/12] intel-ipu3: Add v4l2 driver based on media framework
Hi, Tomasz,
Sorry for the delay in responding to your review.
> -Original Message-
> From: linux-media-ow...@vger.kernel.org [mailto:linux-media-
> ow...@vger.kernel.org] On Behalf Of Tomasz Figa
> Sent: Monday, July 2, 2018 2:50 AM
> To: Zhi, Yong
> Cc: Linux Media Mailing List ; Sakari Ailus
> ; Mani, Rajmohan
> ; Toivonen, Tuukka
> ; Hu, Jerry W ; Zheng,
> Jian Xu ; Vijaykumar, Ramya
>
> Subject: Re: [PATCH v6 11/12] intel-ipu3: Add v4l2 driver based on media
> framework
>
> Hi Yong,
>
> On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi wrote:
> [snip]
> > +static int ipu3_vidioc_enum_input(struct file *file, void *fh,
> > + struct v4l2_input *input) {
> > + if (input->index > 0)
> > + return -EINVAL;
> > + strlcpy(input->name, "camera", sizeof(input->name));
> > + input->type = V4L2_INPUT_TYPE_CAMERA;
> > +
> > + return 0;
> > +}
> > +
> > +static int ipu3_vidioc_g_input(struct file *file, void *fh, unsigned
> > +int *input) {
> > + *input = 0;
> > +
> > + return 0;
> > +}
> > +
> > +static int ipu3_vidioc_s_input(struct file *file, void *fh, unsigned
> > +int input) {
> > + return input == 0 ? 0 : -EINVAL; }
> > +
> > +static int ipu3_vidioc_enum_output(struct file *file, void *fh,
> > + struct v4l2_output *output) {
> > + if (output->index > 0)
> > + return -EINVAL;
> > + strlcpy(output->name, "camera", sizeof(output->name));
> > + output->type = V4L2_INPUT_TYPE_CAMERA;
> > +
> > + return 0;
> > +}
> > +
> > +static int ipu3_vidioc_g_output(struct file *file, void *fh,
> > + unsigned int *output) {
> > + *output = 0;
> > +
> > + return 0;
> > +}
> > +
> > +static int ipu3_vidioc_s_output(struct file *file, void *fh,
> > + unsigned int output) {
> > + return output == 0 ? 0 : -EINVAL; }
>
> Do we really need to implement the 6 functions above? They don't seem to
> be doing anything useful.
>
They are here to pass v4l2-compliance test. I can add a note in next update for
their purpose. We can remove them in the future when defaults callbacks are
available for those ops.
> [snip]
>
> > +int ipu3_v4l2_register(struct imgu_device *imgu) {
> > + struct v4l2_mbus_framefmt def_bus_fmt = { 0 };
> > + struct v4l2_pix_format_mplane def_pix_fmt = { 0 };
> > +
> > + int i, r;
> > +
> > + /* Initialize miscellaneous variables */
> > + imgu->streaming = false;
> > +
> > + /* Set up media device */
> > + imgu->media_dev.dev = >pci_dev->dev;
> > + strlcpy(imgu->media_dev.model, IMGU_NAME,
> > + sizeof(imgu->media_dev.model));
> > + snprintf(imgu->media_dev.bus_info, sizeof(imgu-
> >media_dev.bus_info),
> > +"%s", dev_name(>pci_dev->dev));
> > + imgu->media_dev.hw_revision = 0;
> > + media_device_init(>media_dev);
> > + r = media_device_register(>media_dev);
> > + if (r) {
> > + dev_err(>pci_dev->dev,
> > + "failed to register media device (%d)\n", r);
> > + return r;
> > + }
>
> Shouldn't we register the media device at the end, after all video nodes are
> registered below? Otherwise, since media_device_register() exposes the
> media node to userspace, we risk a race, when userspace opens the media
> device before all the entities are created and linked.
>
Make sense, will change the call order in v7.
> [snip]
>
> > +int ipu3_v4l2_unregister(struct imgu_device *imgu) {
> > + unsigned int i;
> > +
> > + for (i = 0; i < IMGU_NODE_NUM; i++) {
> > + video_unregister_device(>nodes[i].vdev);
> > + media_entity_cleanup(>nodes[i].vdev.entity);
> > + mutex_destroy(>nodes[i].lock);
> > + }
> > +
> > + v4l2_device_unregister_subdev(>subdev);
> > + media_entity_cleanup(>subdev.entity);
> > + kfree(imgu->subdev_pads);
> > + v4l2_device_unregister(>v4l2_dev);
> > + media_device_unregister(>media_dev);
>
> Should unregister media device at the beginning, so that it cannot be used
> when we continue to clean up the entities.
>
Agree, thanks for the review.
> > + media_device_cleanup(>media_dev);
> > +
> > + return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(ipu3_v4l2_unregister);
>
> Best regards,
> Tomasz
Re: [PATCH v10 2/2] media: V3s: Add support for Allwinner CSI.
Hi Sakari,
On Wed, 18 Jul 2018 12:55:14 +0300
Sakari Ailus wrote:
> Hi Yong,
>
> On Thu, Jul 05, 2018 at 03:48:02PM +0800, Yong wrote:
> > > > +
> > > > +/*
> > > > -
> > > > + * Media Operations
> > > > + */
> > > > +static int sun6i_video_formats_init(struct sun6i_video *video,
> > > > + const struct media_pad *remote)
> > > > +{
> > > > + struct v4l2_subdev_mbus_code_enum mbus_code = { 0 };
> > > > + struct sun6i_csi *csi = video->csi;
> > > > + struct v4l2_format format;
> > > > + struct v4l2_subdev *subdev;
> > > > + u32 pad;
> > > > + const u32 *pixformats;
> > > > + int pixformat_count = 0;
> > > > + u32 subdev_codes[32]; /* subdev format codes, 32 should be
> > > > enough */
> > > > + int codes_count = 0;
> > > > + int num_fmts = 0;
> > > > + int i, j;
> > > > +
> > > > + pad = remote->index;
> > > > + subdev = media_entity_to_v4l2_subdev(remote->entity);
> > > > + if (subdev == NULL)
> > > > + return -ENXIO;
> > > > +
> > > > + /* Get supported pixformats of CSI */
> > > > + pixformat_count = sun6i_csi_get_supported_pixformats(csi,
> > > > );
> > > > + if (pixformat_count <= 0)
> > > > + return -ENXIO;
> > > > +
> > > > + /* Get subdev formats codes */
> > > > + mbus_code.pad = pad;
> > > > + mbus_code.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> > > > + while (!v4l2_subdev_call(subdev, pad, enum_mbus_code, NULL,
> > > > +_code)) {
> > >
> > > The formats supported by the external sub-device may depend on horizontal
> > > and vertical flipping. You shouldn't assume any particular configuration
> > > here: instead, bridge drivers generally just need to make sure the formats
> > > match in link validation when streaming is started. At least the CSI-2
> > > receiver driver and the DMA engine driver (video device) should check the
> > > configuration is valid. See e.g. the IPU3 driver:
> > > drivers/media/pci/intel/ipu3/ipu3-cio2.c .
> >
> > Can mbus_code be added dynamically ?
> > The code here only enum the mbus code and get the possible supported
> > pairs of pixformat and mbus by SoC. Not try to check if the formats
> > (width height ...) is valid or not. The formats validation will be
> > in link validation when streaming is started as per your advise.
>
> The formats that can be enumerated from the sensor here are those settable
> using SUBDEV_S_FMT. The enumeration will change on raw sensors if you use
> the flipping controls. As the bridge driver implements MC as well as subdev
> APIs, generally the sensor configuration is out of scope of this driver
> since it's directly configured from the user space.
>
> Just check that the pipeline is valid before starting streaming in your
> driver.
Sorry. I am still confused.
As the CSI driver does not enum the formats supported by sensor.
How to get a valid format if I do not want to open the subdev?
Many applications still only open /dev/video*.
>
> --
> Kind regards,
>
> Sakari Ailus
> sakari.ai...@linux.intel.com
Thanks,
Yong
RE: [PATCH v6 02/12] intel-ipu3: Add user space API definitions
Hi, Tomasz,
Thank you for the time spent to review this long file.
> -Original Message-
> From: Tomasz Figa [mailto:tf...@chromium.org]
> Sent: Sunday, June 17, 2018 11:09 PM
> To: Zhi, Yong
> Cc: Linux Media Mailing List ; Sakari Ailus
> ; Mani, Rajmohan
> ; Toivonen, Tuukka
> ; Hu, Jerry W ; Zheng,
> Jian Xu
> Subject: Re: [PATCH v6 02/12] intel-ipu3: Add user space API definitions
>
> Hi Yong,
>
> On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi wrote:
> >
> > Define the structures and macros to be used by public.
> >
> > Signed-off-by: Yong Zhi
> > Signed-off-by: Rajmohan Mani
> > ---
> > include/uapi/linux/intel-ipu3.h | 1403
> > +++
> > 1 file changed, 1403 insertions(+)
> > create mode 100644 include/uapi/linux/intel-ipu3.h
> >
>
> Since we'll need 1 more resend with latest fixes from Chromium tree and
> recently posted documentation anyway, let me do some more nitpicking
> inline, so we can end up with slightly cleaner code. :)
>
> > diff --git a/include/uapi/linux/intel-ipu3.h
> > b/include/uapi/linux/intel-ipu3.h new file mode 100644 index
> > ..694ef0c8d7a7
> > --- /dev/null
> > +++ b/include/uapi/linux/intel-ipu3.h
> > @@ -0,0 +1,1403 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/* Copyright (C) 2018 Intel Corporation */
> > +
> > +#ifndef __IPU3_UAPI_H
> > +#define __IPU3_UAPI_H
> > +
> > +#include
> > +
> > +#define IPU3_UAPI_ISP_VEC_ELEMS64
> > +
> > +#define IMGU_ABI_PAD __aligned(IPU3_UAPI_ISP_WORD_BYTES)
>
> This seems unused.
Ack, will remove.
>
> > +#define IPU3_ALIGN
> __attribute__((aligned(IPU3_UAPI_ISP_WORD_BYTES)))
>
> Any reason to mix both __aligned() and __attribute__((aligned()))?
>
> > +
> > +#define IPU3_UAPI_ISP_WORD_BYTES 32
>
> It would make sense to define this above IPU3_ALIGN(), which references it.
>
Sure.
> > +#define IPU3_UAPI_MAX_STRIPES 2
> > +
> > +/*** ipu3_uapi_stats_3a ***/
> > +
> > +#define IPU3_UAPI_MAX_BUBBLE_SIZE 10
> > +
> > +#define IPU3_UAPI_AE_COLORS4
> > +#define IPU3_UAPI_AE_BINS 256
> > +
> > +#define IPU3_UAPI_AWB_MD_ITEM_SIZE 8
> > +#define IPU3_UAPI_AWB_MAX_SETS 60
> > +#define IPU3_UAPI_AWB_SET_SIZE 0x500
>
> Why not just decimal 1280?
Ok, will change above and similar places to decimal expression.
>
> > +#define IPU3_UAPI_AWB_SPARE_FOR_BUBBLES \
> > + (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
> > +IPU3_UAPI_AWB_MD_ITEM_SIZE)
> > +#define IPU3_UAPI_AWB_MAX_BUFFER_SIZE \
> > + (IPU3_UAPI_AWB_MAX_SETS * \
> > +(IPU3_UAPI_AWB_SET_SIZE +
> IPU3_UAPI_AWB_SPARE_FOR_BUBBLES))
> > +
> > +#define IPU3_UAPI_AF_MAX_SETS 24
> > +#define IPU3_UAPI_AF_MD_ITEM_SIZE 4
> > +#define IPU3_UAPI_AF_SPARE_FOR_BUBBLES \
> > + (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
> > +IPU3_UAPI_AF_MD_ITEM_SIZE)
> > +#define IPU3_UAPI_AF_Y_TABLE_SET_SIZE 0x80
>
> Why not just decimal 128?
Ack.
>
> > +#define IPU3_UAPI_AF_Y_TABLE_MAX_SIZE \
> > + (IPU3_UAPI_AF_MAX_SETS * \
> > +(IPU3_UAPI_AF_Y_TABLE_SET_SIZE +
> IPU3_UAPI_AF_SPARE_FOR_BUBBLES) * \
> > +IPU3_UAPI_MAX_STRIPES)
> > +
> > +#define IPU3_UAPI_AWB_FR_MAX_SETS 24
> > +#define IPU3_UAPI_AWB_FR_MD_ITEM_SIZE 8
> > +#define IPU3_UAPI_AWB_FR_BAYER_TBL_SIZE0x100
>
> Why not just decimal 256?
Ack.
>
> > +#define IPU3_UAPI_AWB_FR_SPARE_FOR_BUBBLES \
> > + (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
> > +IPU3_UAPI_AWB_FR_MD_ITEM_SIZE) #define
> > +IPU3_UAPI_AWB_FR_BAYER_TABLE_MAX_SIZE \
> > + (IPU3_UAPI_AWB_FR_MAX_SETS * \
> > + (IPU3_UAPI_AWB_FR_BAYER_TBL_SIZE + \
> > +IPU3_UAPI_AWB_FR_SPARE_FOR_BUBBLES) *
> IPU3_UAPI_MAX_STRIPES)
> [snip]
> > +struct ipu3_uapi_af_filter_config {
> > + struct {
> > + __u8 a1;
> > + __u8 a2;
> > + __u8 a3;
> > + __u8 a4;
> > + } y1_coeff_0;
> > + struct {
> > + __u8 a5;
> > + __u8 a6;
> &g
RE: [PATCH v6 04/12] intel-ipu3: Implement DMA mapping functions
Hi, Tomasz,
Thanks for the review.
> -Original Message-
> From: Tomasz Figa [mailto:tf...@chromium.org]
> Sent: Monday, June 18, 2018 12:09 AM
> To: Zhi, Yong
> Cc: Linux Media Mailing List ; Sakari Ailus
> ; Mani, Rajmohan
> ; Toivonen, Tuukka
> ; Hu, Jerry W ; Zheng,
> Jian Xu
> Subject: Re: [PATCH v6 04/12] intel-ipu3: Implement DMA mapping
> functions
>
> On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi wrote:
> >
> > From: Tomasz Figa
> >
> > This driver uses IOVA space for buffer mapping through IPU3 MMU to
> > transfer data between imaging pipelines and system DDR.
> >
> > Signed-off-by: Tomasz Figa
> > Signed-off-by: Yong Zhi
> > ---
> > drivers/media/pci/intel/ipu3/ipu3-css-pool.h | 36
> > drivers/media/pci/intel/ipu3/ipu3-dmamap.c | 280
> +++
> > drivers/media/pci/intel/ipu3/ipu3-dmamap.h | 22 +++
> > drivers/media/pci/intel/ipu3/ipu3.h | 151 +++
> > 4 files changed, 489 insertions(+)
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.h
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.c
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.h
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3.h
> >
> > diff --git a/drivers/media/pci/intel/ipu3/ipu3-css-pool.h
> > b/drivers/media/pci/intel/ipu3/ipu3-css-pool.h
> > new file mode 100644
> > index ..4b22e0856232
> > --- /dev/null
> > +++ b/drivers/media/pci/intel/ipu3/ipu3-css-pool.h
> > @@ -0,0 +1,36 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/* Copyright (C) 2018 Intel Corporation */
> > +
> > +#ifndef __IPU3_UTIL_H
> > +#define __IPU3_UTIL_H
> > +
> > +struct device;
> > +
> > +#define IPU3_CSS_POOL_SIZE 4
> > +
> > +struct ipu3_css_map {
> > + size_t size;
> > + void *vaddr;
> > + dma_addr_t daddr;
> > + struct vm_struct *vma;
> > +};
> > +
> > +struct ipu3_css_pool {
> > + struct {
> > + struct ipu3_css_map param;
> > + long framenum;
> > + } entry[IPU3_CSS_POOL_SIZE];
> > + unsigned int last; /* Latest entry */
>
> It's not clear what "Latest entry" means here. Since these structs are a part
> of the interface exposed by this header, could you write proper kerneldoc
> comments for all fields in both of them?
>
Sure.
> > +};
> > +
> > +int ipu3_css_dma_buffer_resize(struct device *dev, struct ipu3_css_map
> *map,
> > + size_t size); void
> > +ipu3_css_pool_cleanup(struct device *dev, struct ipu3_css_pool
> > +*pool); int ipu3_css_pool_init(struct device *dev, struct ipu3_css_pool
> *pool,
> > + size_t size);
> > +int ipu3_css_pool_get(struct ipu3_css_pool *pool, long framenum);
> > +void ipu3_css_pool_put(struct ipu3_css_pool *pool); const struct
> > +ipu3_css_map *ipu3_css_pool_last(struct ipu3_css_pool *pool,
> > + unsigned int last);
> > +
> > +#endif
> > diff --git a/drivers/media/pci/intel/ipu3/ipu3-dmamap.c
> > b/drivers/media/pci/intel/ipu3/ipu3-dmamap.c
> > new file mode 100644
> > index ..b2bc5d00debc
> > --- /dev/null
> > +++ b/drivers/media/pci/intel/ipu3/ipu3-dmamap.c
> > @@ -0,0 +1,280 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (C) 2018 Intel Corporation
> > + * Copyright (C) 2018 Google, Inc.
>
> Would you mind changing as below?
>
> Copyright 2018 Google LLC.
>
Ack.
> > + *
> > + * Author: Tomasz Figa
> > + * Author: Yong Zhi */
> > +
> > +#include
> > +
> > +#include "ipu3.h"
> > +#include "ipu3-css-pool.h"
> > +#include "ipu3-mmu.h"
> > +
> > +/*
> > + * Free a buffer allocated by ipu3_dmamap_alloc_buffer() */ static
> > +void ipu3_dmamap_free_buffer(struct page **pages,
> > + size_t size) {
> > + int count = size >> PAGE_SHIFT;
> > +
> > + while (count--)
> > + __free_page(pages[count]);
> > + kvfree(pages);
> > +}
> > +
> > +/*
> > + * Based on the implementation of __iommu_dma_alloc_pages()
> > + * defined in drivers/iommu/dma-iommu.c */ static struct page
> > +**ipu3_dmamap_alloc_buffer(size_t size,
> > + unsigned long order_mask,
> > +
RE: [PATCH v6 03/12] intel-ipu3: mmu: Implement driver
Hi, Tomasz,
Thanks for the code review.
> -Original Message-
> From: Tomasz Figa [mailto:tf...@chromium.org]
> Sent: Sunday, June 17, 2018 11:46 PM
> To: Zhi, Yong
> Cc: Linux Media Mailing List ; Sakari Ailus
> ; Mani, Rajmohan
> ; Toivonen, Tuukka
> ; Hu, Jerry W ; Zheng,
> Jian Xu
> Subject: Re: [PATCH v6 03/12] intel-ipu3: mmu: Implement driver
>
> On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi wrote:
> >
> > From: Tomasz Figa
> >
> > This driver translates IO virtual address to physical address based on
> > two levels page tables.
> >
> > Signed-off-by: Tomasz Figa
> > Signed-off-by: Yong Zhi
> > ---
> > drivers/media/pci/intel/ipu3/ipu3-mmu.c | 560
> >
> > drivers/media/pci/intel/ipu3/ipu3-mmu.h | 28 ++
> > 2 files changed, 588 insertions(+)
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.c
> > create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.h
> >
> > diff --git a/drivers/media/pci/intel/ipu3/ipu3-mmu.c
> > b/drivers/media/pci/intel/ipu3/ipu3-mmu.c
> > new file mode 100644
> > index ..a4b3e1680bbb
> > --- /dev/null
> > +++ b/drivers/media/pci/intel/ipu3/ipu3-mmu.c
> > @@ -0,0 +1,560 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Copyright (c) 2018 Intel Corporation.
> > + * Copyright (C) 2018 Google, Inc.
>
> I followed wrong guide when adding this one. Could you fix it up to the
> following?
>
> Copyright 2018 Google LLC.
>
Sure, will do.
> [snip]
> > +/**
> > + * ipu3_mmu_exit() - clean up IPU3 MMU block
> > + * @mmu: IPU3 MMU private data
> > + */
> > +void ipu3_mmu_exit(struct ipu3_mmu_info *info) {
> > + struct ipu3_mmu *mmu = to_ipu3_mmu(info);
> > +
> > + /* We are going to free our page tables, no more memory access. */
> > + ipu3_mmu_set_halt(mmu, true);
> > + ipu3_mmu_tlb_invalidate(mmu);
> > +
> > + ipu3_mmu_free_page_table(mmu->l1pt);
> > + vfree(mmu->l2pts);
> > + ipu3_mmu_free_page_table(mmu->dummy_l2pt);
> > + kfree(mmu->dummy_page);
>
> Should be free_page(). (Might be already included in your tree as per
> https://chromium-
> review.googlesource.com/c/chromiumos/third_party/kernel/+/1084522)
>
Yes, will add above fix to next upstream version.
> > + kfree(mmu);
> > +}
> > +
> > +void ipu3_mmu_suspend(struct ipu3_mmu_info *info) {
> > + struct ipu3_mmu *mmu = to_ipu3_mmu(info);
> > +
> > + ipu3_mmu_set_halt(mmu, true);
> > +}
> > +
> > +void ipu3_mmu_resume(struct ipu3_mmu_info *info) {
> > + struct ipu3_mmu *mmu = to_ipu3_mmu(info);
> > + u32 pteval;
> > +
> > + ipu3_mmu_set_halt(mmu, true);
> > +
> > + pteval = IPU3_ADDR2PTE(virt_to_phys(mmu->l1pt));
> > + writel(pteval, mmu->base + REG_L1_PHYS);
> > +
> > + ipu3_mmu_tlb_invalidate(mmu);
> > + ipu3_mmu_set_halt(mmu, false); }
> > diff --git a/drivers/media/pci/intel/ipu3/ipu3-mmu.h
> > b/drivers/media/pci/intel/ipu3/ipu3-mmu.h
> > new file mode 100644
> > index ..4976187c18f6
> > --- /dev/null
> > +++ b/drivers/media/pci/intel/ipu3/ipu3-mmu.h
> > @@ -0,0 +1,28 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/* Copyright (C) 2018 Intel Corporation */
> > +/* Copyright (C) 2018 Google, Inc. */
> > +
> > +#ifndef __IPU3_MMU_H
> > +#define __IPU3_MMU_H
> > +
> > +struct ipu3_mmu_info {
> > + dma_addr_t aperture_start; /* First address that can be mapped
> */
> > + dma_addr_t aperture_end; /* Last address that can be mapped
> */
> > + unsigned long pgsize_bitmap;/* Bitmap of page sizes in use */
>
> If documenting the fields, why not use a kerneldoc comment above the
> struct instead?
>
Ack.
> Best regards,
> Tomasz
[PATCH v1 0/2] Document Intel IPU3 ImgU driver and uAPI
Hi, All, This patch set adds documentation on Intel IPU3 ImgU driver and its uAPI, based on the feedback received for v1 RFC below and ImgU driver patch series: v1 RFC: https://patchwork.kernel.org/patch/10321897/> ImgU v6: https://patchwork.kernel.org/patch/10316739/> Few unused structs have being removed from intel-ipu3.h since v6. This patch set will be merged into the next ImgU driver update. Rajmohan Mani (1): doc-rst: Add Intel IPU3 documentation Yong Zhi (1): v4l: Document Intel IPU3 meta data uAPI Documentation/media/uapi/v4l/meta-formats.rst |1 + .../media/uapi/v4l/pixfmt-meta-intel-ipu3.rst | 174 ++ Documentation/media/v4l-drivers/index.rst |1 + Documentation/media/v4l-drivers/ipu3.rst | 304 +++ include/uapi/linux/intel-ipu3.h| 2816 5 files changed, 3296 insertions(+) create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst create mode 100644 Documentation/media/v4l-drivers/ipu3.rst create mode 100644 include/uapi/linux/intel-ipu3.h -- 2.7.4
[PATCH v1 1/2] doc-rst: Add Intel IPU3 documentation
From: Rajmohan Mani
This patch adds the details about the IPU3 Imaging Unit driver.
Signed-off-by: Rajmohan Mani
Signed-off-by: Tian Shu Qiu
---
Documentation/media/v4l-drivers/index.rst | 1 +
Documentation/media/v4l-drivers/ipu3.rst | 304 ++
2 files changed, 305 insertions(+)
create mode 100644 Documentation/media/v4l-drivers/ipu3.rst
diff --git a/Documentation/media/v4l-drivers/index.rst
b/Documentation/media/v4l-drivers/index.rst
index 679238e..179a393 100644
--- a/Documentation/media/v4l-drivers/index.rst
+++ b/Documentation/media/v4l-drivers/index.rst
@@ -44,6 +44,7 @@ For more details see the file COPYING in the source
distribution of Linux.
davinci-vpbe
fimc
imx
+ ipu3
ivtv
max2175
meye
diff --git a/Documentation/media/v4l-drivers/ipu3.rst
b/Documentation/media/v4l-drivers/ipu3.rst
new file mode 100644
index 000..a4550d8
--- /dev/null
+++ b/Documentation/media/v4l-drivers/ipu3.rst
@@ -0,0 +1,304 @@
+.. include::
+
+===
+Intel Image Processing Unit 3 (IPU3) Imaging Unit (ImgU) driver
+===
+
+Copyright |copy| 2018 Intel Corporation
+
+Introduction
+
+
+This file documents Intel IPU3 (3rd generation Image Processing Unit) Imaging
+Unit driver located under drivers/media/pci/intel/ipu3.
+
+The Intel IPU3 found in certain Kaby Lake (as well as certain Sky Lake)
+platforms (U/Y processor lines) is made up of two parts namely Imaging Unit
+(ImgU) and CIO2 device (MIPI CSI2 receiver).
+
+The CIO2 device receives the raw bayer data from the sensors and outputs the
+frames in a format that is specific to IPU3 (for consumption by IPU3 ImgU).
+CIO2 driver is available as drivers/media/pci/intel/ipu3/ipu3-cio2* and is
+enabled through the CONFIG_VIDEO_IPU3_CIO2 config option.
+
+The Imaging Unit (ImgU) is responsible for processing images captured
+through IPU3 CIO2 device. The ImgU driver sources can be found under
+drivers/media/pci/intel/ipu3 directory. The driver is enabled through the
+CONFIG_VIDEO_IPU3_IMGU config option.
+
+The two driver modules are named ipu3-csi2 and ipu3-imgu, respectively.
+
+The driver has been tested on Kaby Lake platforms (U/Y processor lines).
+
+The driver implements V4L2, Media controller and V4L2 sub-device interfaces.
+Camera sensors that have CSI-2 bus, which are connected to the IPU3 CIO2
+device are supported. Support for lens and flash drivers depends on the
+above sensors.
+
+ImgU device nodes
+=
+
+The ImgU is represented as a single V4L2 subdev, which provides a V4L2 subdev
+interface to the user space.
+
+CIO2 device
+===
+
+The CIO2 is represented as a single V4L2 subdev, which provides a V4L2 subdev
+interface to the user space. There is a video node for each CSI-2 receiver,
+with a single media controller interface for the entire device.
+
+Media controller
+
+
+The media device interface allows to configure the ImgU links, which defines
+the behavior of the IPU3 firmware. The link configuration tells the firmware
+whether viewfinder or postview ISP pipeline should be enabled.
+
+Device operation
+
+
+With IPU3, once the input video node ("ipu3-imgu":0, in :
+format) is queued with buffer (in packed raw bayer format), IPU3 ISP starts
+processing the buffer and produces the video output in YUV format and
+statistics output on respective output nodes. The driver is expected to have
+buffers ready for all of parameter, output and statistics nodes, when input
+video node is queued with buffer.
+
+At a minimum, all of input, main output, 3A statistics, and either of
+viewfinder or postview video nodes should be enabled for IPU3 to start image
+processing. viewfinder and postview video nodes are mutually exclusive.
+
+input, output, viewfinder and postview video nodes
+--
+
+The frames (in packed raw bayer format specific to IPU3) received by the
+input video node is processed by the IPU3 Imaging Unit and is output to 2
+video nodes, with each targeting different purpose (main output and viewfinder
+or postview output).
+
+Details on raw bayer format specific to IPU3 can be found as below.
+Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
+
+The driver supports V4L2 Video Capture Interface as defined at :ref:`devices`.
+
+Only the multi-planar API is supported. More details can be found at
+:ref:`planar-apis`.
+
+
+parameters video node
+-
+
+The parameter video node receives the ISP algorithm parameters that are used
+to configure how the ISP algorithms process the image.
+
+Details on raw bayer format specific to IPU3 can be found as below.
+Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
+
+3A statistics video node
+
+
+3A statistics video node is used by the ImgU driver to
[PATCH v1 2/2] v4l: Document Intel IPU3 meta data uAPI
These meta formats are used on Intel IPU3 ImgU video queues
to carry 3A statistics and ISP pipeline parameters.
V4L2_META_FMT_IPU3_3A
V4L2_META_FMT_IPU3_PARAMS
Signed-off-by: Yong Zhi
Signed-off-by: Chao C Li
Signed-off-by: Rajmohan Mani
---
Documentation/media/uapi/v4l/meta-formats.rst |1 +
.../media/uapi/v4l/pixfmt-meta-intel-ipu3.rst | 174 ++
include/uapi/linux/intel-ipu3.h| 2816
3 files changed, 2991 insertions(+)
create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
create mode 100644 include/uapi/linux/intel-ipu3.h
diff --git a/Documentation/media/uapi/v4l/meta-formats.rst
b/Documentation/media/uapi/v4l/meta-formats.rst
index 0c4e1ec..b887fca 100644
--- a/Documentation/media/uapi/v4l/meta-formats.rst
+++ b/Documentation/media/uapi/v4l/meta-formats.rst
@@ -12,6 +12,7 @@ These formats are used for the :ref:`metadata` interface only.
.. toctree::
:maxdepth: 1
+pixfmt-meta-intel-ipu3
pixfmt-meta-uvc
pixfmt-meta-vsp1-hgo
pixfmt-meta-vsp1-hgt
diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
new file mode 100644
index 000..5c050e6
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
@@ -0,0 +1,174 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _intel-ipu3:
+
+**
+V4L2_META_FMT_IPU3_PARAMS ('ip3p'), V4L2_META_FMT_IPU3_3A ('ip3s')
+**
+
+.. c:type:: ipu3_uapi_stats_3a
+
+3A statistics
+=
+
+For IPU3 ImgU, the 3A statistics accelerators collect different statistics over
+an input bayer frame. Those statistics, defined in data struct
+:c:type:`ipu3_uapi_stats_3a`, are meta output obtained from "ipu3-imgu 3a stat"
+video node, which are then passed to user space for statistics analysis
+using :c:type:`v4l2_meta_format` interface.
+
+The statistics collected are AWB (Auto-white balance) RGBS cells, AWB filter
+response, AF (Auto-focus) filter response, and AE (Auto-exposure) histogram.
+
+struct :c:type:`ipu3_uapi_4a_config` saves configurable parameters for all
above.
+
+
+.. code-block:: c
+
+
+ struct ipu3_uapi_stats_3a {
+ IPU3_ALIGN struct ipu3_uapi_awb_raw_buffer awb_raw_buffer;
+ struct ipu3_uapi_ae_raw_buffer_aligned
+ ae_raw_buffer[IPU3_UAPI_MAX_STRIPES];
+ struct ipu3_uapi_af_raw_buffer af_raw_buffer;
+ struct ipu3_uapi_awb_fr_raw_buffer awb_fr_raw_buffer;
+ struct ipu3_uapi_4a_config stats_4a_config;
+ __u32 ae_join_buffers;
+ __u8 padding[28];
+ struct ipu3_uapi_stats_3a_bubble_info_per_stripe
+ stats_3a_bubble_per_stripe;
+ struct ipu3_uapi_ff_status stats_3a_status;
+ } __packed;
+
+
+.. c:type:: ipu3_uapi_params
+
+Pipeline parameters
+===
+
+IPU3 pipeline has a number of image processing stages, each of which takes a
+set of parameters as input. The major stages of pipelines are shown here:
+
+Raw pixels -> Bayer Downscaling -> Optical Black Correction ->
+
+Linearization -> Lens Shading Correction -> White Balance / Exposure /
+
+Focus Apply -> Bayer Noise Reduction -> ANR -> Demosaicing -> Color
+
+Correction Matrix -> Gamma correction -> Color Space Conversion ->
+
+Chroma Down Scaling -> Chromatic Noise Reduction -> Total Color
+
+Correction -> XNR3 -> TNR -> DDR
+
+The table below presents a description of the above algorithms.
+
+
===
+NameDescription
+
===
+Optical Black Correction Optical Black Correction block subtracts a pre-defined
+value from the respective pixel values to obtain better
+image quality.
+Defined in :c:type:`ipu3_uapi_obgrid_param`.
+Linearization This algo block uses linearization parameters to
+address non-linearity sensor effects. The Lookup table
+table is defined in
+:c:type:`ipu3_uapi_isp_lin_vmem_params`.
+SHD Lens shading correction is used to correct spatial
+non-uniformity of the pixel response due to optical
+lens shading. This is done by applying a different gain
+for each pixel. The gain, black level etc are
+configured in :c:type:`ipu3_uapi_shd_config_static`.
+BNR Bayer noise reduction block removes image noise by
+applying a bilateral filter.
+See :c:type:`ipu3_uapi_bnr_static_config` for details.
+ANR
[PATCH v3] [media] MAINTAINERS: Update entry for Intel IPU3 cio2 driver
This patch adds Bingbu as additional maintainer, and both Tian Shu and Jian Xu as reviewers for IPU3 CIO2 driver. Signed-off-by: Yong Zhi --- Third time's a charm :) MAINTAINERS | 3 +++ 1 file changed, 3 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index a38e24a..3dd530e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -7157,6 +7157,9 @@ F:drivers/dma/iop-adma.c INTEL IPU3 CSI-2 CIO2 DRIVER M: Yong Zhi M: Sakari Ailus +M: Bingbu Cao +R: Tian Shu Qiu +R: Jian Xu Zheng L: linux-media@vger.kernel.org S: Maintained F: drivers/media/pci/intel/ipu3/ -- 2.7.4
[PATCH v2] [media] MAINTAINERS: Update entry for Intel IPU3 cio2 driver
This patch adds three more maintainers to the IPU3 CIO2 driver. Signed-off-by: Yong Zhi --- MAINTAINERS | 3 +++ 1 file changed, 3 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index a38e24a..3dd530e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -7157,6 +7157,9 @@ F:drivers/dma/iop-adma.c INTEL IPU3 CSI-2 CIO2 DRIVER M: Yong Zhi M: Sakari Ailus +M: Bingbu Cao +R: Tian Shu Qiu +R: Jian Xu Zheng L: linux-media@vger.kernel.org S: Maintained F: drivers/media/pci/intel/ipu3/ -- 2.7.4
[PATCH] [media] MAINTAINERS: Update entry for Intel IPU3 cio2 driver
This patch adds three more maintainers to the IPU3 CIO2 driver. Signed-off-by: Yong Zhi <yong@intel.com> --- MAINTAINERS | 3 +++ 1 file changed, 3 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 49003f77cedd..309d49a54db8 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -7150,6 +7150,9 @@ F:drivers/dma/iop-adma.c INTEL IPU3 CSI-2 CIO2 DRIVER M: Yong Zhi <yong@intel.com> M: Sakari Ailus <sakari.ai...@linux.intel.com> +M: Tian Shu Qiu <tian.shu@intel.com> +M: Bingbu Cao <bingbu@intel.com> +M: Jian Xu Zheng <jian.xu.zh...@intel.com> L: linux-media@vger.kernel.org S: Maintained F: drivers/media/pci/intel/ipu3/ -- 2.7.4
[PATCH v10 2/2] media: V3s: Add support for Allwinner CSI.
Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2 interface and CSI1 is used for parallel interface. This is not documented in datasheet but by test and guess. This patch implement a v4l2 framework driver for it. Currently, the driver only support the parallel interface. MIPI-CSI2, ISP's support are not included in this patch. Reviewed-by: Hans Verkuil <hans.verk...@cisco.com> Reviewed-by: Maxime Ripard <maxime.rip...@bootlin.com> Tested-by: Maxime Ripard <maxime.rip...@bootlin.com> Signed-off-by: Yong Deng <yong.d...@magewell.com> --- MAINTAINERS| 8 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile| 2 + drivers/media/platform/sunxi/sun6i-csi/Kconfig | 9 + drivers/media/platform/sunxi/sun6i-csi/Makefile| 3 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 932 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 145 .../media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h | 196 + .../media/platform/sunxi/sun6i-csi/sun6i_video.c | 767 + .../media/platform/sunxi/sun6i-csi/sun6i_video.h | 53 ++ 10 files changed, 2116 insertions(+) create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Kconfig create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Makefile create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h diff --git a/MAINTAINERS b/MAINTAINERS index 0a1410d5a621..a8f9fe91dd14 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3771,6 +3771,14 @@ M: Jaya Kumar <jayakumar.a...@gmail.com> S: Maintained F: sound/pci/cs5535audio/ +CSI DRIVERS FOR ALLWINNER V3s +M: Yong Deng <yong.d...@magewell.com> +L: linux-media@vger.kernel.org +T: git git://linuxtv.org/media_tree.git +S: Maintained +F: drivers/media/platform/sunxi/sun6i-csi/ +F: Documentation/devicetree/bindings/media/sun6i-csi.txt + CW1200 WLAN driver M: Solomon Peachy <pi...@shaftnet.org> S: Maintained diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index e3229f7baed1..43f25c9c05e1 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -138,6 +138,7 @@ source "drivers/media/platform/am437x/Kconfig" source "drivers/media/platform/xilinx/Kconfig" source "drivers/media/platform/rcar-vin/Kconfig" source "drivers/media/platform/atmel/Kconfig" +source "drivers/media/platform/sunxi/sun6i-csi/Kconfig" config VIDEO_TI_CAL tristate "TI CAL (Camera Adaptation Layer) driver" diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile index 932515df4477..69472cbfa360 100644 --- a/drivers/media/platform/Makefile +++ b/drivers/media/platform/Makefile @@ -92,3 +92,5 @@ obj-$(CONFIG_VIDEO_QCOM_CAMSS)+= qcom/camss-8x16/ obj-$(CONFIG_VIDEO_QCOM_VENUS) += qcom/venus/ obj-y += meson/ + +obj-$(CONFIG_VIDEO_SUN6I_CSI) += sunxi/sun6i-csi/ diff --git a/drivers/media/platform/sunxi/sun6i-csi/Kconfig b/drivers/media/platform/sunxi/sun6i-csi/Kconfig new file mode 100644 index ..018e3ec788c0 --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/Kconfig @@ -0,0 +1,9 @@ +config VIDEO_SUN6I_CSI + tristate "Allwinner V3s Camera Sensor Interface driver" + depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API && HAS_DMA + depends on ARCH_SUNXI || COMPILE_TEST + select VIDEOBUF2_DMA_CONTIG + select REGMAP_MMIO + select V4L2_FWNODE + help + Support for the Allwinner Camera Sensor Interface Controller on V3s. diff --git a/drivers/media/platform/sunxi/sun6i-csi/Makefile b/drivers/media/platform/sunxi/sun6i-csi/Makefile new file mode 100644 index ..213cb6be9e9c --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/Makefile @@ -0,0 +1,3 @@ +sun6i-csi-y += sun6i_video.o sun6i_csi.o + +obj-$(CONFIG_VIDEO_SUN6I_CSI) += sun6i-csi.o diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c new file mode 100644 index ..d03a367dad44 --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c @@ -0,0 +1,932 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) + * All rights reserved. + * Author: Yong Deng <yong.d...@magewell.com> + */ + +#include +#include +#include +#include +#include +#include +#i
[PATCH v10 1/2] dt-bindings: media: Add Allwinner V3s Camera Sensor Interface (CSI)
Add binding documentation for Allwinner V3s CSI.
Acked-by: Maxime Ripard <maxime.rip...@bootlin.com>
Acked-by: Sakari Ailus <sakari.ai...@linux.intel.com>
Reviewed-by: Rob Herring <r...@kernel.org>
Signed-off-by: Yong Deng <yong.d...@magewell.com>
---
.../devicetree/bindings/media/sun6i-csi.txt| 59 ++
1 file changed, 59 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/sun6i-csi.txt
diff --git a/Documentation/devicetree/bindings/media/sun6i-csi.txt
b/Documentation/devicetree/bindings/media/sun6i-csi.txt
new file mode 100644
index ..2ff47a9507a6
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/sun6i-csi.txt
@@ -0,0 +1,59 @@
+Allwinner V3s Camera Sensor Interface
+-
+
+Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2
+interface and CSI1 is used for parallel interface.
+
+Required properties:
+ - compatible: value must be "allwinner,sun8i-v3s-csi"
+ - reg: base address and size of the memory-mapped region.
+ - interrupts: interrupt associated to this IP
+ - clocks: phandles to the clocks feeding the CSI
+* bus: the CSI interface clock
+* mod: the CSI module clock
+* ram: the CSI DRAM clock
+ - clock-names: the clock names mentioned above
+ - resets: phandles to the reset line driving the CSI
+
+Each CSI node should contain one 'port' child node with one child 'endpoint'
+node, according to the bindings defined in
+Documentation/devicetree/bindings/media/video-interfaces.txt. As mentioned
+above, the endpoint's bus type should be MIPI CSI-2 for CSI0 and parallel or
+Bt656 for CSI1.
+
+Endpoint node properties for CSI1
+-
+
+- remote-endpoint : (required) a phandle to the bus receiver's endpoint
+ node
+- bus-width: : (required) must be 8, 10, 12 or 16
+- pclk-sample : (optional) (default: sample on falling edge)
+- hsync-active : (only required for parallel)
+- vsync-active : (only required for parallel)
+
+Example:
+
+csi1: csi@1cb4000 {
+ compatible = "allwinner,sun8i-v3s-csi";
+ reg = <0x01cb4000 0x1000>;
+ interrupts = ;
+ clocks = < CLK_BUS_CSI>,
+< CLK_CSI1_SCLK>,
+< CLK_DRAM_CSI>;
+ clock-names = "bus", "mod", "ram";
+ resets = < RST_BUS_CSI>;
+
+ port {
+ /* Parallel bus endpoint */
+ csi1_ep: endpoint {
+ remote-endpoint = <_ep>;
+ bus-width = <16>;
+
+ /* If hsync-active/vsync-active are missing,
+ embedded BT.656 sync is used */
+ hsync-active = <0>; /* Active low */
+ vsync-active = <0>; /* Active low */
+ pclk-sample = <1>; /* Rising */
+ };
+ };
+};
--
1.8.3.1
[PATCH v10 0/2] Initial Allwinner V3s CSI Support
: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Test input 0: Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK Test input 0: Streaming ioctls: test read/write: OK (Not Supported) test MMAP: OK test USERPTR: OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device Stream using all formats: test MMAP for Format HM12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV21, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format YU12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format YV12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV16, Frame Size 1280x720: Stride 2560, Field None: OK test MMAP for Format NV61, Frame Size 1280x720: Stride 2560, Field None: OK test MMAP for Format 422P, Frame Size 1280x720: Stride 2560, Field None: OK Total: 54, Succeeded: 54, Failed: 0, Warnings: 0 Yong Deng (2): dt-bindings: media: Add Allwinner V3s Camera Sensor Interface (CSI) media: V3s: Add support for Allwinner CSI. .../devicetree/bindings/media/sun6i-csi.txt| 59 ++ MAINTAINERS| 8 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile| 2 + drivers/media/platform/sunxi/sun6i-csi/Kconfig | 9 + drivers/media/platform/sunxi/sun6i-csi/Makefile| 3 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 931 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 145 .../media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h | 196 + .../media/platform/sunxi/sun6i-csi/sun6i_video.c | 767 + .../media/platform/sunxi/sun6i-csi/sun6i_video.h | 53 ++ 11 files changed, 2174 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/sun6i-csi.txt create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Kconfig create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Makefile create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h -- 1.8.3.1
Re: [PATCH v9 0/2] Initial Allwinner V3s CSI Support
Hi Maxime, On Thu, 3 May 2018 19:14:10 +0200 Maxime Ripard <maxime.rip...@bootlin.com> wrote: > Hi Yong, > > On Tue, Mar 06, 2018 at 09:51:10AM +0800, Yong Deng wrote: > > This patchset add initial support for Allwinner V3s CSI. > > > > Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2 > > interface and CSI1 is used for parallel interface. This is not > > documented in datasheet but by test and guess. > > > > This patchset implement a v4l2 framework driver and add a binding > > documentation for it. > > > > Currently, the driver only support the parallel interface. And has been > > tested with a BT1120 signal which generating from FPGA. The following > > fetures are not support with this patchset: > > - ISP > > - MIPI-CSI2 > > - Master clock for camera sensor > > - Power regulator for the front end IC > > Do you plan on sending another version some time soon? It would be > awesome to have this in 4.18. I was waiting for Sakari Ailus's feedback. But ... I will send a new version soon. But not all suggestion from Sakari Ailus would be accepted. > > Thanks! > Maxime > > -- > Maxime Ripard, Bootlin (formerly Free Electrons) > Embedded Linux and Kernel engineering > https://bootlin.com Thanks, Yong
[PATCH] media: intel-ipu3: cio2: Handle IRQs until INT_STS is cleared
From: Bingbu Cao <bingbu@intel.com>
Interrupt behavior shows that some time the frame end and frame start
of next frame is unstable and can range from several to hundreds of micro-sec.
In the case of ~10us, isr may not clear next sof interrupt status in
single handling, which prevents new interrupts from coming.
Fix this by handling all pending IRQs before exiting isr, so any abnormal
behavior results from very short interrupt status changes is protected.
Signed-off-by: Bingbu Cao <bingbu@intel.com>
Signed-off-by: Andy Yeh <andy@intel.com>
Signed-off-by: Yong Zhi <yong@intel.com>
---
Hi, Sakari,
Re-send with correct signed-off-by order.
drivers/media/pci/intel/ipu3/ipu3-cio2.c | 32 ++--
1 file changed, 22 insertions(+), 10 deletions(-)
diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c
b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
index 7d768ec0f824..29027159eced 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
@@ -640,18 +640,10 @@ static const char *const cio2_port_errs[] = {
"PKT2LONG",
};
-static irqreturn_t cio2_irq(int irq, void *cio2_ptr)
+static void cio2_irq_handle_once(struct cio2_device *cio2, u32 int_status)
{
- struct cio2_device *cio2 = cio2_ptr;
void __iomem *const base = cio2->base;
struct device *dev = >pci_dev->dev;
- u32 int_status, int_clear;
-
- int_status = readl(base + CIO2_REG_INT_STS);
- int_clear = int_status;
-
- if (!int_status)
- return IRQ_NONE;
if (int_status & CIO2_INT_IOOE) {
/*
@@ -770,9 +762,29 @@ static irqreturn_t cio2_irq(int irq, void *cio2_ptr)
int_status &= ~(CIO2_INT_IOIE | CIO2_INT_IOIRQ);
}
- writel(int_clear, base + CIO2_REG_INT_STS);
if (int_status)
dev_warn(dev, "unknown interrupt 0x%x on INT\n", int_status);
+}
+
+static irqreturn_t cio2_irq(int irq, void *cio2_ptr)
+{
+ struct cio2_device *cio2 = cio2_ptr;
+ void __iomem *const base = cio2->base;
+ struct device *dev = >pci_dev->dev;
+ u32 int_status;
+
+ int_status = readl(base + CIO2_REG_INT_STS);
+ dev_dbg(dev, "isr enter - interrupt status 0x%x\n", int_status);
+ if (!int_status)
+ return IRQ_NONE;
+
+ do {
+ writel(int_status, base + CIO2_REG_INT_STS);
+ cio2_irq_handle_once(cio2, int_status);
+ int_status = readl(base + CIO2_REG_INT_STS);
+ if (int_status)
+ dev_dbg(dev, "pending status 0x%x\n", int_status);
+ } while (int_status);
return IRQ_HANDLED;
}
--
2.7.4
RE: [PATCH v6 12/12] intel-ipu3: Add imgu top level pci device driver
Hi, Tomasz,
Thanks for the review again.
> -Original Message-
> From: Tomasz Figa [mailto:tf...@chromium.org]
> Sent: Thursday, April 26, 2018 12:15 AM
> To: Zhi, Yong <yong@intel.com>
> Cc: Linux Media Mailing List <linux-media@vger.kernel.org>; Sakari Ailus
> <sakari.ai...@linux.intel.com>; Mani, Rajmohan
> <rajmohan.m...@intel.com>; Toivonen, Tuukka
> <tuukka.toivo...@intel.com>; Hu, Jerry W <jerry.w...@intel.com>; Zheng,
> Jian Xu <jian.xu.zh...@intel.com>
> Subject: Re: [PATCH v6 12/12] intel-ipu3: Add imgu top level pci device
> driver
>
> Hi Yong,
>
> On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi <yong@intel.com> wrote:
> [snip]
> > +static int imgu_video_nodes_init(struct imgu_device *imgu) {
> > + struct v4l2_pix_format_mplane *fmts[IPU3_CSS_QUEUES] = { NULL };
> > + struct v4l2_rect *rects[IPU3_CSS_RECTS] = { NULL };
> > + unsigned int i;
> > + int r;
> > +
> > + imgu->buf_struct_size = sizeof(struct imgu_buffer);
> > +
> > + for (i = 0; i < IMGU_NODE_NUM; i++) {
> > + imgu->nodes[i].name = imgu_node_map[i].name;
> > + imgu->nodes[i].output = i < IMGU_QUEUE_FIRST_INPUT;
> > + imgu->nodes[i].immutable = false;
> > + imgu->nodes[i].enabled = false;
> > +
> > + if (i != IMGU_NODE_PARAMS && i != IMGU_NODE_STAT_3A)
> > + fmts[imgu_node_map[i].css_queue] =
> > + >nodes[i].vdev_fmt.fmt.pix_mp;
> > + atomic_set(>nodes[i].sequence, 0);
> > + }
> > +
> > + /* Master queue is always enabled */
> > + imgu->nodes[IMGU_QUEUE_MASTER].immutable = true;
> > + imgu->nodes[IMGU_QUEUE_MASTER].enabled = true;
> > +
> > + r = ipu3_v4l2_register(imgu);
> > + if (r)
> > + return r;
> > +
> > + /* Set initial formats and initialize formats of video nodes */
> > + rects[IPU3_CSS_RECT_EFFECTIVE] = >rect.eff;
> > + rects[IPU3_CSS_RECT_BDS] = >rect.bds;
> > + ipu3_css_fmt_set(>css, fmts, rects);
> > +
> > + /* Pre-allocate dummy buffers */
> > + r = imgu_dummybufs_preallocate(imgu);
> > + if (r) {
> > + dev_err(>pci_dev->dev,
> > + "failed to pre-allocate dummy buffers (%d)", r);
> > + imgu_dummybufs_cleanup(imgu);
>
> No need to call ipu3_v4l2_unregister() here?
>
> (That's why I keep suggesting use of single return error path with labels
> named after the first cleanup step that needs to be done, as it makes it
> easier to spot such mistakes.)
>
Good catch, suggestion taken :)
Maybe I should move the imgu_dummybufs_preallocate() out from
imgu_video_nodes_init().
> > + return r;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static void imgu_video_nodes_exit(struct imgu_device *imgu) {
> > + imgu_dummybufs_cleanup(imgu);
> > + ipu3_v4l2_unregister(imgu);
> > +}
>
> Best regards,
> Tomasz
RE: [PATCH v6 10/12] intel-ipu3: Add css pipeline programming
Hi, Tomasz,
Thanks for the code review.
> -Original Message-
> From: Tomasz Figa [mailto:tf...@chromium.org]
> Sent: Thursday, April 26, 2018 12:12 AM
> To: Zhi, Yong <yong@intel.com>
> Cc: Linux Media Mailing List <linux-media@vger.kernel.org>; Sakari Ailus
> <sakari.ai...@linux.intel.com>; Mani, Rajmohan
> <rajmohan.m...@intel.com>; Toivonen, Tuukka
> <tuukka.toivo...@intel.com>; Hu, Jerry W <jerry.w...@intel.com>; Zheng,
> Jian Xu <jian.xu.zh...@intel.com>
> Subject: Re: [PATCH v6 10/12] intel-ipu3: Add css pipeline programming
>
> Hi Yong,
>
> On Fri, Mar 30, 2018 at 11:15 AM Yong Zhi <yong@intel.com> wrote:
> [snip]
> > +int ipu3_css_init(struct device *dev, struct ipu3_css *css,
> > + void __iomem *base, int length) {
> > + int r, p, q, i;
> > +
> > + /* Initialize main data structure */
> > + css->dev = dev;
> > + css->base = base;
> > + css->iomem_length = length;
> > + css->current_binary = IPU3_CSS_DEFAULT_BINARY;
> > + css->pipe_id = IPU3_CSS_PIPE_ID_NUM;
> > + css->vf_output_en = IPU3_NODE_VF_DISABLED;
> > + spin_lock_init(>qlock);
> > +
> > + for (q = 0; q < IPU3_CSS_QUEUES; q++) {
> > + r = ipu3_css_queue_init(>queue[q], NULL, 0);
> > + if (r)
> > + return r;
> > + }
> > +
> > + r = ipu3_css_fw_init(css);
> > + if (r)
> > + return r;
> > +
> > + /* Allocate and map common structures with imgu hardware */
> > +
> > + for (p = 0; p < IPU3_CSS_PIPE_ID_NUM; p++)
> > + for (i = 0; i < IMGU_ABI_MAX_STAGES; i++) {
> > + if (!ipu3_dmamap_alloc(dev,
> > +
> >xmem_sp_stage_ptrs[p][i],
> > + sizeof(struct
> imgu_abi_sp_stage)))
>
> checkpatch reports line over 80 characters here.
Ack, I opted for alignment over line limit here , will fix in v7.
>
> > + goto error_no_memory;
> > + if (!ipu3_dmamap_alloc(dev,
> > +
> >xmem_isp_stage_ptrs[p][i],
> > + sizeof(struct
> imgu_abi_isp_stage)))
>
> Ditto.
Sure, thanks!!
>
> > + goto error_no_memory;
> > + }
>
> Best regards,
> Tomasz
[PATCH] media: intel-ipu3: cio2: Handle IRQs until INT_STS is cleared
From: Bingbu Cao <bingbu@intel.com>
Interrupt behavior shows that some time the frame end and frame start
of next frame is unstable and can range from several to hundreds
of micro-sec. In the case of ~10us, isr may not clear next sof interrupt
status in single handling, which prevents new interrupts from coming.
Fix this by handling all pending IRQs before exiting isr, so
any abnormal behavior results from very short interrupt status
changes is protected.
Signed-off-by: Andy Yeh <andy@intel.com>
Signed-off-by: Bingbu Cao <bingbu@intel.com>
Signed-off-by: Yong Zhi <yong@intel.com>
---
drivers/media/pci/intel/ipu3/ipu3-cio2.c | 32 ++--
1 file changed, 22 insertions(+), 10 deletions(-)
diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c
b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
index 7d768ec..2902715 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c
@@ -640,18 +640,10 @@ static const char *const cio2_port_errs[] = {
"PKT2LONG",
};
-static irqreturn_t cio2_irq(int irq, void *cio2_ptr)
+static void cio2_irq_handle_once(struct cio2_device *cio2, u32 int_status)
{
- struct cio2_device *cio2 = cio2_ptr;
void __iomem *const base = cio2->base;
struct device *dev = >pci_dev->dev;
- u32 int_status, int_clear;
-
- int_status = readl(base + CIO2_REG_INT_STS);
- int_clear = int_status;
-
- if (!int_status)
- return IRQ_NONE;
if (int_status & CIO2_INT_IOOE) {
/*
@@ -770,9 +762,29 @@ static irqreturn_t cio2_irq(int irq, void *cio2_ptr)
int_status &= ~(CIO2_INT_IOIE | CIO2_INT_IOIRQ);
}
- writel(int_clear, base + CIO2_REG_INT_STS);
if (int_status)
dev_warn(dev, "unknown interrupt 0x%x on INT\n", int_status);
+}
+
+static irqreturn_t cio2_irq(int irq, void *cio2_ptr)
+{
+ struct cio2_device *cio2 = cio2_ptr;
+ void __iomem *const base = cio2->base;
+ struct device *dev = >pci_dev->dev;
+ u32 int_status;
+
+ int_status = readl(base + CIO2_REG_INT_STS);
+ dev_dbg(dev, "isr enter - interrupt status 0x%x\n", int_status);
+ if (!int_status)
+ return IRQ_NONE;
+
+ do {
+ writel(int_status, base + CIO2_REG_INT_STS);
+ cio2_irq_handle_once(cio2, int_status);
+ int_status = readl(base + CIO2_REG_INT_STS);
+ if (int_status)
+ dev_dbg(dev, "pending status 0x%x\n", int_status);
+ } while (int_status);
return IRQ_HANDLED;
}
--
2.7.4
[RFC PATCH]: intel-ipu3: Add uAPI documentation
This is a preliminary effort to add documentation for the
following BNR(bayer noise reduction) structs:
ipu3_uapi_bnr_static_config_wb_gains_config
ipu3_uapi_bnr_static_config_wb_gains_thr_config
ipu3_uapi_bnr_static_config_thr_coeffs_config
ipu3_uapi_bnr_static_config_thr_ctrl_shd_config
ipu3_uapi_bnr_static_config_opt_center_sqr_config
ipu3_uapi_bnr_static_config_bp_ctrl_config
ipu3_uapi_bnr_static_config_dn_detect_ctrl_config
ipu3_uapi_bnr_static_config_opt_center_sqr_config
ipu3_uapi_bnr_static_config_green_disparity
The feedback on this patch will be used towards the
documentation of reset of the uAPI in intel-ipu3.h.
Link to v6 IPU3 ImgU patchset:
<URL:https://patchwork.kernel.org/patch/10316725/>
Signed-off-by: Yong Zhi <yong@intel.com>
Signed-off-by: Chao C Li <chao.c...@intel.com>
---
Documentation/media/media_uapi.rst |1 +
Documentation/media/uapi/intel-ipu3.rst |8 +
include/uapi/linux/intel-ipu3.h | 1520 +++
3 files changed, 1529 insertions(+)
create mode 100644 Documentation/media/uapi/intel-ipu3.rst
create mode 100644 include/uapi/linux/intel-ipu3.h
diff --git a/Documentation/media/media_uapi.rst
b/Documentation/media/media_uapi.rst
index 28eb35a1f965..edfa674b49c3 100644
--- a/Documentation/media/media_uapi.rst
+++ b/Documentation/media/media_uapi.rst
@@ -31,3 +31,4 @@ License".
uapi/cec/cec-api
uapi/gen-errors
uapi/fdl-appendix
+uapi/intel-ipu3
diff --git a/Documentation/media/uapi/intel-ipu3.rst
b/Documentation/media/uapi/intel-ipu3.rst
new file mode 100644
index ..d4d9b2806fe9
--- /dev/null
+++ b/Documentation/media/uapi/intel-ipu3.rst
@@ -0,0 +1,8 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _intel-ipu3:
+
+Intel IPU3 ImgU uAPI headers
+
+
+.. kernel-doc:: include/uapi/linux/intel-ipu3.h
diff --git a/include/uapi/linux/intel-ipu3.h b/include/uapi/linux/intel-ipu3.h
new file mode 100644
index ..34b071524beb
--- /dev/null
+++ b/include/uapi/linux/intel-ipu3.h
@@ -0,0 +1,1520 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2018 Intel Corporation */
+
+#ifndef __IPU3_UAPI_H
+#define __IPU3_UAPI_H
+
+#include
+
+#define IPU3_UAPI_ISP_VEC_ELEMS64
+
+#define IMGU_ABI_PAD __aligned(IPU3_UAPI_ISP_WORD_BYTES)
+#define IPU3_ALIGN __attribute__((aligned(IPU3_UAPI_ISP_WORD_BYTES)))
+
+#define IPU3_UAPI_ISP_WORD_BYTES 32
+#define IPU3_UAPI_MAX_STRIPES 2
+
+/*** ipu3_uapi_stats_3a ***/
+
+#define IPU3_UAPI_MAX_BUBBLE_SIZE 10
+
+#define IPU3_UAPI_AE_COLORS4
+#define IPU3_UAPI_AE_BINS 256
+
+#define IPU3_UAPI_AWB_MD_ITEM_SIZE 8
+#define IPU3_UAPI_AWB_MAX_SETS 60
+#define IPU3_UAPI_AWB_SET_SIZE 0x500
+#define IPU3_UAPI_AWB_SPARE_FOR_BUBBLES \
+ (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
+IPU3_UAPI_AWB_MD_ITEM_SIZE)
+#define IPU3_UAPI_AWB_MAX_BUFFER_SIZE \
+ (IPU3_UAPI_AWB_MAX_SETS * \
+(IPU3_UAPI_AWB_SET_SIZE + IPU3_UAPI_AWB_SPARE_FOR_BUBBLES))
+
+#define IPU3_UAPI_AF_MAX_SETS 24
+#define IPU3_UAPI_AF_MD_ITEM_SIZE 4
+#define IPU3_UAPI_AF_SPARE_FOR_BUBBLES \
+ (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
+IPU3_UAPI_AF_MD_ITEM_SIZE)
+#define IPU3_UAPI_AF_Y_TABLE_SET_SIZE 0x80
+#define IPU3_UAPI_AF_Y_TABLE_MAX_SIZE \
+ (IPU3_UAPI_AF_MAX_SETS * \
+(IPU3_UAPI_AF_Y_TABLE_SET_SIZE + IPU3_UAPI_AF_SPARE_FOR_BUBBLES) * \
+IPU3_UAPI_MAX_STRIPES)
+
+#define IPU3_UAPI_AWB_FR_MAX_SETS 24
+#define IPU3_UAPI_AWB_FR_MD_ITEM_SIZE 8
+#define IPU3_UAPI_AWB_FR_BAYER_TBL_SIZE0x100
+#define IPU3_UAPI_AWB_FR_SPARE_FOR_BUBBLES \
+ (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
+IPU3_UAPI_AWB_FR_MD_ITEM_SIZE)
+#define IPU3_UAPI_AWB_FR_BAYER_TABLE_MAX_SIZE \
+ (IPU3_UAPI_AWB_FR_MAX_SETS * \
+ (IPU3_UAPI_AWB_FR_BAYER_TBL_SIZE + \
+IPU3_UAPI_AWB_FR_SPARE_FOR_BUBBLES) * IPU3_UAPI_MAX_STRIPES)
+
+struct ipu3_uapi_grid_config {
+ __u8 width; /* 6 or 7 (rgbs_grd_cfg) bits */
+ __u8 height;
+ __u16 block_width_log2:3;
+ __u16 block_height_log2:3;
+ __u16 height_per_slice:8; /* default value 1 */
+ __u16 x_start; /* 12 bits */
+ __u16 y_start;
+#define IPU3_UAPI_GRID_START_MASK ((1 << 12) - 1)
+#define IPU3_UAPI_GRID_Y_START_EN (1 << 15)
+ __u16 x_end;/* 12 bits */
+ __u16 y_end;
+} __packed;
+
+struct ipu3_uapi_awb_meta_data {
+ __u8 meta_data_b
[PATCH v6 11/12] intel-ipu3: Add v4l2 driver based on media framework
Implement video driver that utilizes v4l2, vb2 queue support
and media controller APIs. The driver exposes single
subdevice and six nodes.
Signed-off-by: Yong Zhi <yong@intel.com>
Signed-off-by: Ramya Vijaykumar <ramya.vijayku...@intel.com>
---
drivers/media/pci/intel/ipu3/ipu3-v4l2.c | 1089 ++
1 file changed, 1089 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-v4l2.c
diff --git a/drivers/media/pci/intel/ipu3/ipu3-v4l2.c
b/drivers/media/pci/intel/ipu3/ipu3-v4l2.c
new file mode 100644
index ..1095ef8a29a5
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-v4l2.c
@@ -0,0 +1,1089 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+#include
+
+#include
+
+#include "ipu3.h"
+#include "ipu3-dmamap.h"
+
+/ v4l2_subdev_ops /
+
+static int ipu3_subdev_open(struct v4l2_subdev *sd, struct v4l2_subdev_fh *fh)
+{
+ struct imgu_device *imgu = container_of(sd, struct imgu_device, subdev);
+ struct v4l2_rect try_crop = {
+ .top = 0,
+ .left = 0,
+ .height = imgu->nodes[IMGU_NODE_IN].vdev_fmt.fmt.pix_mp.height,
+ .width = imgu->nodes[IMGU_NODE_IN].vdev_fmt.fmt.pix_mp.width,
+ };
+ unsigned int i;
+
+ /* Initialize try_fmt */
+ for (i = 0; i < IMGU_NODE_NUM; i++)
+ *v4l2_subdev_get_try_format(sd, fh->pad, i) =
+ imgu->nodes[i].pad_fmt;
+
+ *v4l2_subdev_get_try_crop(sd, fh->pad, IMGU_NODE_IN) = try_crop;
+
+ return 0;
+}
+
+static int ipu3_subdev_s_stream(struct v4l2_subdev *sd, int enable)
+{
+ struct imgu_device *imgu = container_of(sd, struct imgu_device, subdev);
+ int r = 0;
+
+ r = imgu_s_stream(imgu, enable);
+ if (!r)
+ imgu->streaming = enable;
+
+ return r;
+}
+
+static int ipu3_subdev_get_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
+{
+ struct imgu_device *imgu = container_of(sd, struct imgu_device, subdev);
+ struct v4l2_mbus_framefmt *mf;
+ u32 pad = fmt->pad;
+
+ if (fmt->which == V4L2_SUBDEV_FORMAT_ACTIVE) {
+ fmt->format = imgu->nodes[pad].pad_fmt;
+ } else {
+ mf = v4l2_subdev_get_try_format(sd, cfg, pad);
+ fmt->format = *mf;
+ }
+
+ return 0;
+}
+
+static int ipu3_subdev_set_fmt(struct v4l2_subdev *sd,
+ struct v4l2_subdev_pad_config *cfg,
+ struct v4l2_subdev_format *fmt)
+{
+ struct imgu_device *imgu = container_of(sd, struct imgu_device, subdev);
+ struct v4l2_mbus_framefmt *mf;
+ u32 pad = fmt->pad;
+
+ if (fmt->which == V4L2_SUBDEV_FORMAT_TRY)
+ mf = v4l2_subdev_get_try_format(sd, cfg, pad);
+ else
+ mf = >nodes[pad].pad_fmt;
+
+ fmt->format.code = mf->code;
+ /* Clamp the w and h based on the hardware capabilities */
+ if (imgu->subdev_pads[pad].flags & MEDIA_PAD_FL_SOURCE) {
+ fmt->format.width = clamp(fmt->format.width,
+ IPU3_OUTPUT_MIN_WIDTH,
+ IPU3_OUTPUT_MAX_WIDTH);
+ fmt->format.height = clamp(fmt->format.height,
+ IPU3_OUTPUT_MIN_HEIGHT,
+ IPU3_OUTPUT_MAX_HEIGHT);
+ } else {
+ fmt->format.width = clamp(fmt->format.width,
+ IPU3_INPUT_MIN_WIDTH,
+ IPU3_INPUT_MAX_WIDTH);
+ fmt->format.height = clamp(fmt->format.height,
+ IPU3_INPUT_MIN_HEIGHT,
+ IPU3_INPUT_MAX_HEIGHT);
+ }
+
+ *mf = fmt->format;
+
+ return 0;
+}
+
+static int ipu3_subdev_get_selection(struct v4l2_subdev *sd,
+struct v4l2_subdev_pad_config *cfg,
+struct v4l2_subdev_selection *sel)
+{
+ struct imgu_device *imgu = container_of(sd, struct imgu_device, subdev);
+ struct v4l2_rect *try_sel, *r;
+
+ if (sel->pad != IMGU_NODE_IN)
+ return -EINVAL;
+
+ switch (sel->target) {
+ case V4L2_SEL_TGT_CROP:
+ try_sel = v4l2_subdev_get_try_crop(sd, cfg, sel->pad);
+ r = >rect.eff;
+ break;
+ case V4L2_SEL_TGT_COMPOSE:
+ try_sel = v4l2_subdev_get_try_compose(sd, cfg, sel->pad);
+ r = >rect.bds;
+ break;
+ default:
+
[PATCH v6 05/12] intel-ipu3: css: Add dma buff pool utility functions
The pools are used to store previous parameters set by
user with the parameter queue. Due to pipelining,
there needs to be multiple sets (up to four)
of parameters which are queued in a host-to-sp queue.
Signed-off-by: Yong Zhi <yong@intel.com>
---
drivers/media/pci/intel/ipu3/ipu3-css-pool.c | 131 +++
1 file changed, 131 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.c
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css-pool.c
b/drivers/media/pci/intel/ipu3/ipu3-css-pool.c
new file mode 100644
index ..84f6a7801a01
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-css-pool.c
@@ -0,0 +1,131 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+
+#include "ipu3-css-pool.h"
+#include "ipu3-dmamap.h"
+
+int ipu3_css_dma_buffer_resize(struct device *dev, struct ipu3_css_map *map,
+ size_t size)
+{
+ if (map->size < size && map->vaddr) {
+ dev_warn(dev, "dma buffer is resized from %zu to %zu",
+map->size, size);
+
+ ipu3_dmamap_free(dev, map);
+ if (!ipu3_dmamap_alloc(dev, map, size))
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+void ipu3_css_pool_cleanup(struct device *dev, struct ipu3_css_pool *pool)
+{
+ unsigned int i;
+
+ for (i = 0; i < IPU3_CSS_POOL_SIZE; i++)
+ ipu3_dmamap_free(dev, >entry[i].param);
+}
+
+int ipu3_css_pool_init(struct device *dev, struct ipu3_css_pool *pool,
+ size_t size)
+{
+ unsigned int i;
+
+ for (i = 0; i < IPU3_CSS_POOL_SIZE; i++) {
+ /*
+* entry[i].framenum is initialized to INT_MIN so that
+* ipu3_css_pool_check() can treat it as usesable slot.
+*/
+ pool->entry[i].framenum = INT_MIN;
+
+ if (size == 0) {
+ pool->entry[i].param.vaddr = NULL;
+ continue;
+ }
+
+ if (!ipu3_dmamap_alloc(dev, >entry[i].param, size))
+ goto fail;
+ }
+
+ pool->last = IPU3_CSS_POOL_SIZE;
+
+ return 0;
+
+fail:
+ ipu3_css_pool_cleanup(dev, pool);
+ return -ENOMEM;
+}
+
+/*
+ * Check that the following call to pool_get succeeds.
+ * Return negative on error.
+ */
+static int ipu3_css_pool_check(struct ipu3_css_pool *pool, long framenum)
+{
+ /* Get the oldest entry */
+ int n = (pool->last + 1) % IPU3_CSS_POOL_SIZE;
+ long diff = framenum - pool->entry[n].framenum;
+
+ /* if framenum wraps around and becomes smaller than entry n */
+ if (diff < 0)
+ diff += LONG_MAX;
+
+ /*
+* pool->entry[n].framenum stores the frame number where that
+* entry was allocated. If that was allocated more than POOL_SIZE
+* frames back, it is old enough that we know it is no more in
+* use by firmware.
+*/
+ if (diff > IPU3_CSS_POOL_SIZE)
+ return n;
+
+ return -ENOSPC;
+}
+
+/*
+ * Allocate a new parameter from pool at frame number `framenum'.
+ * Release the oldest entry in the pool to make space for the new entry.
+ * Return negative on error.
+ */
+int ipu3_css_pool_get(struct ipu3_css_pool *pool, long framenum)
+{
+ int n = ipu3_css_pool_check(pool, framenum);
+
+ if (n < 0)
+ return n;
+
+ pool->entry[n].framenum = framenum;
+ pool->last = n;
+
+ return n;
+}
+
+/*
+ * Undo, for all practical purposes, the effect of pool_get().
+ */
+void ipu3_css_pool_put(struct ipu3_css_pool *pool)
+{
+ pool->entry[pool->last].framenum = INT_MIN;
+ pool->last = (pool->last + IPU3_CSS_POOL_SIZE - 1) % IPU3_CSS_POOL_SIZE;
+}
+
+/*
+ * Return the nth entry from last, if that entry has no frame stored,
+ * return a null map instead to indicate frame not available for the entry.
+ */
+const struct ipu3_css_map *
+ipu3_css_pool_last(struct ipu3_css_pool *pool, unsigned int n)
+{
+ static const struct ipu3_css_map null_map = { 0 };
+ int i = (pool->last + IPU3_CSS_POOL_SIZE - n) % IPU3_CSS_POOL_SIZE;
+
+ WARN_ON(n >= IPU3_CSS_POOL_SIZE);
+
+ if (pool->entry[i].framenum < 0)
+ return _map;
+
+ return >entry[i].param;
+}
--
2.7.4
[PATCH v6 02/12] intel-ipu3: Add user space API definitions
Define the structures and macros to be used by public.
Signed-off-by: Yong Zhi <yong@intel.com>
Signed-off-by: Rajmohan Mani <rajmohan.m...@intel.com>
---
include/uapi/linux/intel-ipu3.h | 1403 +++
1 file changed, 1403 insertions(+)
create mode 100644 include/uapi/linux/intel-ipu3.h
diff --git a/include/uapi/linux/intel-ipu3.h b/include/uapi/linux/intel-ipu3.h
new file mode 100644
index ..694ef0c8d7a7
--- /dev/null
+++ b/include/uapi/linux/intel-ipu3.h
@@ -0,0 +1,1403 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2018 Intel Corporation */
+
+#ifndef __IPU3_UAPI_H
+#define __IPU3_UAPI_H
+
+#include
+
+#define IPU3_UAPI_ISP_VEC_ELEMS64
+
+#define IMGU_ABI_PAD __aligned(IPU3_UAPI_ISP_WORD_BYTES)
+#define IPU3_ALIGN __attribute__((aligned(IPU3_UAPI_ISP_WORD_BYTES)))
+
+#define IPU3_UAPI_ISP_WORD_BYTES 32
+#define IPU3_UAPI_MAX_STRIPES 2
+
+/*** ipu3_uapi_stats_3a ***/
+
+#define IPU3_UAPI_MAX_BUBBLE_SIZE 10
+
+#define IPU3_UAPI_AE_COLORS4
+#define IPU3_UAPI_AE_BINS 256
+
+#define IPU3_UAPI_AWB_MD_ITEM_SIZE 8
+#define IPU3_UAPI_AWB_MAX_SETS 60
+#define IPU3_UAPI_AWB_SET_SIZE 0x500
+#define IPU3_UAPI_AWB_SPARE_FOR_BUBBLES \
+ (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
+IPU3_UAPI_AWB_MD_ITEM_SIZE)
+#define IPU3_UAPI_AWB_MAX_BUFFER_SIZE \
+ (IPU3_UAPI_AWB_MAX_SETS * \
+(IPU3_UAPI_AWB_SET_SIZE + IPU3_UAPI_AWB_SPARE_FOR_BUBBLES))
+
+#define IPU3_UAPI_AF_MAX_SETS 24
+#define IPU3_UAPI_AF_MD_ITEM_SIZE 4
+#define IPU3_UAPI_AF_SPARE_FOR_BUBBLES \
+ (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
+IPU3_UAPI_AF_MD_ITEM_SIZE)
+#define IPU3_UAPI_AF_Y_TABLE_SET_SIZE 0x80
+#define IPU3_UAPI_AF_Y_TABLE_MAX_SIZE \
+ (IPU3_UAPI_AF_MAX_SETS * \
+(IPU3_UAPI_AF_Y_TABLE_SET_SIZE + IPU3_UAPI_AF_SPARE_FOR_BUBBLES) * \
+IPU3_UAPI_MAX_STRIPES)
+
+#define IPU3_UAPI_AWB_FR_MAX_SETS 24
+#define IPU3_UAPI_AWB_FR_MD_ITEM_SIZE 8
+#define IPU3_UAPI_AWB_FR_BAYER_TBL_SIZE0x100
+#define IPU3_UAPI_AWB_FR_SPARE_FOR_BUBBLES \
+ (IPU3_UAPI_MAX_BUBBLE_SIZE * IPU3_UAPI_MAX_STRIPES * \
+IPU3_UAPI_AWB_FR_MD_ITEM_SIZE)
+#define IPU3_UAPI_AWB_FR_BAYER_TABLE_MAX_SIZE \
+ (IPU3_UAPI_AWB_FR_MAX_SETS * \
+ (IPU3_UAPI_AWB_FR_BAYER_TBL_SIZE + \
+IPU3_UAPI_AWB_FR_SPARE_FOR_BUBBLES) * IPU3_UAPI_MAX_STRIPES)
+
+struct ipu3_uapi_grid_config {
+ __u8 width; /* 6 or 7 (rgbs_grd_cfg) bits */
+ __u8 height;
+ __u16 block_width_log2:3;
+ __u16 block_height_log2:3;
+ __u16 height_per_slice:8; /* default value 1 */
+ __u16 x_start; /* 12 bits */
+ __u16 y_start;
+#define IPU3_UAPI_GRID_START_MASK ((1 << 12) - 1)
+#define IPU3_UAPI_GRID_Y_START_EN (1 << 15)
+ __u16 x_end;/* 12 bits */
+ __u16 y_end;
+} __packed;
+
+struct ipu3_uapi_awb_meta_data {
+ __u8 meta_data_buffer[IPU3_UAPI_AWB_MAX_BUFFER_SIZE];
+} __packed;
+
+struct ipu3_uapi_awb_raw_buffer {
+ struct ipu3_uapi_awb_meta_data meta_data;
+} __packed;
+
+struct IPU3_ALIGN ipu3_uapi_awb_config_s {
+ __u16 rgbs_thr_gr;
+ __u16 rgbs_thr_r;
+ __u16 rgbs_thr_gb;
+ __u16 rgbs_thr_b;
+/* controls generation of meta_data (like FF enable/disable) */
+#define IPU3_UAPI_AWB_RGBS_THR_B_EN(1 << 14)
+#define IPU3_UAPI_AWB_RGBS_THR_B_INCL_SAT (1 << 15)
+
+ struct ipu3_uapi_grid_config grid;
+} __packed;
+
+struct ipu3_uapi_ae_raw_buffer {
+ __u32 vals[IPU3_UAPI_AE_BINS * IPU3_UAPI_AE_COLORS];
+} __packed;
+
+struct ipu3_uapi_ae_raw_buffer_aligned {
+ struct ipu3_uapi_ae_raw_buffer buff IPU3_ALIGN;
+} __packed;
+
+struct ipu3_uapi_ae_grid_config {
+ __u8 width;
+ __u8 height;
+ __u8 block_width_log2:4;
+ __u8 block_height_log2:4;
+ __u8 __reserved0:5;
+ __u8 ae_en:1;
+ __u8 rst_hist_array:1;
+ __u8 done_rst_hist_array:1;
+ __u16 x_start; /* 12 bits */
+ __u16 y_start;
+ __u16 x_end;
+ __u16 y_end;
+} __packed;
+
+struct ipu3_uapi_af_filter_config {
+ struct {
+ __u8 a1;
+ __u8 a2;
+ __u8 a3;
+ __u8 a4;
+ } y1_coeff_0;
+ struct {
+ __u8 a5;
+ __u8 a6;
+ __u8 a7;
+ __u8 a8;
+ } y1_coeff_1;
+ struct {
+ __u8 a9;
+ __u8 a10;
+
[PATCH v6 04/12] intel-ipu3: Implement DMA mapping functions
From: Tomasz Figa <tf...@chromium.org>
This driver uses IOVA space for buffer mapping through IPU3 MMU
to transfer data between imaging pipelines and system DDR.
Signed-off-by: Tomasz Figa <tf...@chromium.org>
Signed-off-by: Yong Zhi <yong@intel.com>
---
drivers/media/pci/intel/ipu3/ipu3-css-pool.h | 36
drivers/media/pci/intel/ipu3/ipu3-dmamap.c | 280 +++
drivers/media/pci/intel/ipu3/ipu3-dmamap.h | 22 +++
drivers/media/pci/intel/ipu3/ipu3.h | 151 +++
4 files changed, 489 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.h
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.h
create mode 100644 drivers/media/pci/intel/ipu3/ipu3.h
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css-pool.h
b/drivers/media/pci/intel/ipu3/ipu3-css-pool.h
new file mode 100644
index ..4b22e0856232
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-css-pool.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2018 Intel Corporation */
+
+#ifndef __IPU3_UTIL_H
+#define __IPU3_UTIL_H
+
+struct device;
+
+#define IPU3_CSS_POOL_SIZE 4
+
+struct ipu3_css_map {
+ size_t size;
+ void *vaddr;
+ dma_addr_t daddr;
+ struct vm_struct *vma;
+};
+
+struct ipu3_css_pool {
+ struct {
+ struct ipu3_css_map param;
+ long framenum;
+ } entry[IPU3_CSS_POOL_SIZE];
+ unsigned int last; /* Latest entry */
+};
+
+int ipu3_css_dma_buffer_resize(struct device *dev, struct ipu3_css_map *map,
+ size_t size);
+void ipu3_css_pool_cleanup(struct device *dev, struct ipu3_css_pool *pool);
+int ipu3_css_pool_init(struct device *dev, struct ipu3_css_pool *pool,
+ size_t size);
+int ipu3_css_pool_get(struct ipu3_css_pool *pool, long framenum);
+void ipu3_css_pool_put(struct ipu3_css_pool *pool);
+const struct ipu3_css_map *ipu3_css_pool_last(struct ipu3_css_pool *pool,
+ unsigned int last);
+
+#endif
diff --git a/drivers/media/pci/intel/ipu3/ipu3-dmamap.c
b/drivers/media/pci/intel/ipu3/ipu3-dmamap.c
new file mode 100644
index ..b2bc5d00debc
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-dmamap.c
@@ -0,0 +1,280 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2018 Intel Corporation
+ * Copyright (C) 2018 Google, Inc.
+ *
+ * Author: Tomasz Figa <tf...@chromium.org>
+ * Author: Yong Zhi <yong@intel.com>
+ */
+
+#include
+
+#include "ipu3.h"
+#include "ipu3-css-pool.h"
+#include "ipu3-mmu.h"
+
+/*
+ * Free a buffer allocated by ipu3_dmamap_alloc_buffer()
+ */
+static void ipu3_dmamap_free_buffer(struct page **pages,
+ size_t size)
+{
+ int count = size >> PAGE_SHIFT;
+
+ while (count--)
+ __free_page(pages[count]);
+ kvfree(pages);
+}
+
+/*
+ * Based on the implementation of __iommu_dma_alloc_pages()
+ * defined in drivers/iommu/dma-iommu.c
+ */
+static struct page **ipu3_dmamap_alloc_buffer(size_t size,
+ unsigned long order_mask,
+ gfp_t gfp)
+{
+ struct page **pages;
+ unsigned int i = 0, count = size >> PAGE_SHIFT;
+ const gfp_t high_order_gfp = __GFP_NOWARN | __GFP_NORETRY;
+
+ /* Allocate mem for array of page ptrs */
+ pages = kvmalloc_array(count, sizeof(struct page *), GFP_KERNEL);
+
+ if (!pages)
+ return NULL;
+
+ order_mask &= (2U << MAX_ORDER) - 1;
+ if (!order_mask)
+ return NULL;
+
+ gfp |= __GFP_HIGHMEM | __GFP_ZERO;
+
+ while (count) {
+ struct page *page = NULL;
+ unsigned int order_size;
+
+ for (order_mask &= (2U << __fls(count)) - 1;
+order_mask; order_mask &= ~order_size) {
+ unsigned int order = __fls(order_mask);
+
+ order_size = 1U << order;
+ page = alloc_pages((order_mask - order_size) ?
+ gfp | high_order_gfp : gfp, order);
+ if (!page)
+ continue;
+ if (!order)
+ break;
+ if (!PageCompound(page)) {
+ split_page(page, order);
+ break;
+ }
+
+ __free_pages(page, order);
+ }
+ if (!page) {
+ ipu3_dmamap_free_buffer(pages, i << PAGE_SHIFT);
+ return NULL;
+ }
+ count -= order_size;
+
[PATCH v6 01/12] v4l: Add Intel IPU3 meta buffer formats
Add IPU3-specific meta formats for parameter
processing and 3A statistics:
V4L2_META_FMT_IPU3_PARAMS
V4L2_META_FMT_IPU3_STAT_3A
Signed-off-by: Yong Zhi <yong@intel.com>
---
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
include/uapi/linux/videodev2.h | 4
2 files changed, 6 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
index ec98b93b2a55..5360b3915af7 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1257,6 +1257,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_VSP1_HGO:descr = "R-Car VSP1 1-D Histogram";
break;
case V4L2_META_FMT_VSP1_HGT:descr = "R-Car VSP1 2-D Histogram";
break;
case V4L2_META_FMT_UVC: descr = "UVC payload header metadata";
break;
+ case V4L2_META_FMT_IPU3_PARAMS: descr = "IPU3 processing parameters";
break;
+ case V4L2_META_FMT_IPU3_STAT_3A:descr = "IPU3 3A statistics";
break;
default:
/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index e20d10df75c1..83ef5a0fbdc2 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -698,6 +698,10 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_VSP1_HGTv4l2_fourcc('V', 'S', 'P', 'T') /* R-Car
VSP1 2-D Histogram */
#define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC
Payload Header metadata */
+/* Vendor specific - used for IPU3 camera sub-system */
+#define V4L2_META_FMT_IPU3_PARAMS v4l2_fourcc('i', 'p', '3', 'p') /* IPU3
params */
+#define V4L2_META_FMT_IPU3_STAT_3A v4l2_fourcc('i', 'p', '3', 's') /* IPU3
3A statistics */
+
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC0xfeedcafe
--
2.7.4
[PATCH v6 03/12] intel-ipu3: mmu: Implement driver
From: Tomasz Figa <tf...@chromium.org>
This driver translates IO virtual address to physical
address based on two levels page tables.
Signed-off-by: Tomasz Figa <tf...@chromium.org>
Signed-off-by: Yong Zhi <yong@intel.com>
---
drivers/media/pci/intel/ipu3/ipu3-mmu.c | 560
drivers/media/pci/intel/ipu3/ipu3-mmu.h | 28 ++
2 files changed, 588 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.h
diff --git a/drivers/media/pci/intel/ipu3/ipu3-mmu.c
b/drivers/media/pci/intel/ipu3/ipu3-mmu.c
new file mode 100644
index ..a4b3e1680bbb
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-mmu.c
@@ -0,0 +1,560 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2018 Intel Corporation.
+ * Copyright (C) 2018 Google, Inc.
+ *
+ * Author: Tuukka Toivonen <tuukka.toivo...@intel.com>
+ * Author: Sakari Ailus <sakari.ai...@linux.intel.com>
+ * Author: Samu Onkalo <samu.onk...@intel.com>
+ * Author: Tomasz Figa <tf...@chromium.org>
+ *
+ */
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+
+#include "ipu3-mmu.h"
+
+#define IPU3_PAGE_SHIFT12
+#define IPU3_PAGE_SIZE (1UL << IPU3_PAGE_SHIFT)
+
+#define IPU3_PT_BITS 10
+#define IPU3_PT_PTES (1UL << IPU3_PT_BITS)
+#define IPU3_PT_SIZE (IPU3_PT_PTES << 2)
+#define IPU3_PT_ORDER (IPU3_PT_SIZE >> PAGE_SHIFT)
+
+#define IPU3_ADDR2PTE(addr)((addr) >> IPU3_PAGE_SHIFT)
+#define IPU3_PTE2ADDR(pte) ((phys_addr_t)(pte) << IPU3_PAGE_SHIFT)
+
+#define IPU3_L2PT_SHIFTIPU3_PT_BITS
+#define IPU3_L2PT_MASK ((1UL << IPU3_L2PT_SHIFT) - 1)
+
+#define IPU3_L1PT_SHIFTIPU3_PT_BITS
+#define IPU3_L1PT_MASK ((1UL << IPU3_L1PT_SHIFT) - 1)
+
+#define IPU3_MMU_ADDRESS_BITS (IPU3_PAGE_SHIFT + \
+IPU3_L2PT_SHIFT + \
+IPU3_L1PT_SHIFT)
+
+#define IMGU_REG_BASE 0x4000
+#define REG_TLB_INVALIDATE (IMGU_REG_BASE + 0x300)
+#define TLB_INVALIDATE 1
+#define REG_L1_PHYS(IMGU_REG_BASE + 0x304) /* 27-bit pfn */
+#define REG_GP_HALT(IMGU_REG_BASE + 0x5dc)
+#define REG_GP_HALTED (IMGU_REG_BASE + 0x5e0)
+
+struct ipu3_mmu {
+ struct device *dev;
+ void __iomem *base;
+ /* protect access to l2pts, l1pt */
+ spinlock_t lock;
+
+ void *dummy_page;
+ u32 dummy_page_pteval;
+
+ u32 *dummy_l2pt;
+ u32 dummy_l2pt_pteval;
+
+ u32 **l2pts;
+ u32 *l1pt;
+
+ struct ipu3_mmu_info geometry;
+};
+
+static inline struct ipu3_mmu *to_ipu3_mmu(struct ipu3_mmu_info *info)
+{
+ return container_of(info, struct ipu3_mmu, geometry);
+}
+
+/**
+ * ipu3_mmu_tlb_invalidate - invalidate translation look-aside buffer
+ * @mmu: MMU to perform the invalidate operation on
+ *
+ * This function invalidates the whole TLB. Must be called when the hardware
+ * is powered on.
+ */
+static void ipu3_mmu_tlb_invalidate(struct ipu3_mmu *mmu)
+{
+ writel(TLB_INVALIDATE, mmu->base + REG_TLB_INVALIDATE);
+}
+
+static void call_if_ipu3_is_powered(struct ipu3_mmu *mmu,
+ void (*func)(struct ipu3_mmu *mmu))
+{
+ pm_runtime_get_noresume(mmu->dev);
+ if (pm_runtime_active(mmu->dev))
+ func(mmu);
+ pm_runtime_put(mmu->dev);
+}
+
+/**
+ * ipu3_mmu_set_halt - set CIO gate halt bit
+ * @mmu: MMU to set the CIO gate bit in.
+ * @halt: Desired state of the gate bit.
+ *
+ * This function sets the CIO gate bit that controls whether external memory
+ * accesses are allowed. Must be called when the hardware is powered on.
+ */
+static void ipu3_mmu_set_halt(struct ipu3_mmu *mmu, bool halt)
+{
+ int ret;
+ u32 val;
+
+ writel(halt, mmu->base + REG_GP_HALT);
+ ret = readl_poll_timeout(mmu->base + REG_GP_HALTED,
+val, (val & 1) == halt, 1000, 10);
+
+ if (ret)
+ dev_err(mmu->dev, "failed to %s CIO gate halt\n",
+ halt ? "set" : "clear");
+}
+
+/**
+ * ipu3_mmu_alloc_page_table - allocate a pre-filled page table
+ * @pteval: Value to initialize for page table entries with.
+ *
+ * Return: Pointer to allocated page table or NULL on failure.
+ */
+static u32 *ipu3_mmu_alloc_page_table(u32 pteval)
+{
+ u32 *pt;
+ int pte;
+
+ pt = (u32 *)__get_free_page(GFP_KERNEL);
+ if (!pt)
+ return NULL;
+
+ for (pte = 0; pte < IPU3_PT_PTES; pte++)
+ pt[pte] = pteval;
+
+ set_memory_uc((unsigned long int)pt, IPU3_PT_ORDER);
+
+ return pt;
+}
+
+/**
+ * ipu3_mmu_free_page_table - free page table
+ * @pt: Page table to
[PATCH v6 06/12] intel-ipu3: css: Add support for firmware management
Introduce functions to load and install ImgU FW blobs. Signed-off-by: Yong Zhi <yong@intel.com> --- drivers/media/pci/intel/ipu3/ipu3-abi.h| 1888 drivers/media/pci/intel/ipu3/ipu3-css-fw.c | 261 drivers/media/pci/intel/ipu3/ipu3-css-fw.h | 198 +++ 3 files changed, 2347 insertions(+) create mode 100644 drivers/media/pci/intel/ipu3/ipu3-abi.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.h diff --git a/drivers/media/pci/intel/ipu3/ipu3-abi.h b/drivers/media/pci/intel/ipu3/ipu3-abi.h new file mode 100644 index ..24102647a89e --- /dev/null +++ b/drivers/media/pci/intel/ipu3/ipu3-abi.h @@ -0,0 +1,1888 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2018 Intel Corporation */ + +#ifndef __IPU3_ABI_H +#define __IPU3_ABI_H + +#include + +/*** IMGU Hardware information ***/ + +typedef __u32 imgu_addr_t; + +#define IMGU_ISP_VMEM_ALIGN128 +#define IMGU_DVS_BLOCK_W 64 +#define IMGU_DVS_BLOCK_H 32 +#define IMGU_GDC_BUF_X (2 * IMGU_DVS_BLOCK_W) +#define IMGU_GDC_BUF_Y IMGU_DVS_BLOCK_H +/* n = 0..1 */ +#define IMGU_SP_PMEM_BASE(n) (0x2 + (n) * 0x4000) +#define IMGU_MAX_BQ_GRID_WIDTH 80 +#define IMGU_MAX_BQ_GRID_HEIGHT60 +#define IMGU_OBGRID_TILE_SIZE 16 +#define IMGU_PIXELS_PER_WORD 50 +#define IMGU_BYTES_PER_WORD64 +#define IMGU_STRIPE_FIXED_HALF_OVERLAP 2 +#define IMGU_SHD_SETS 3 +#define IMGU_BDS_MIN_CLIP_VAL 0 +#define IMGU_BDS_MAX_CLIP_VAL 2 + +#define IMGU_ABI_AWB_MAX_CELLS_PER_SET 160 +#define IMGU_ABI_AF_MAX_CELLS_PER_SET 32 +#define IMGU_ABI_AWB_FR_MAX_CELLS_PER_SET 32 + +#define IMGU_ABI_ACC_OP_IDLE 0 +#define IMGU_ABI_ACC_OP_END_OF_ACK 1 +#define IMGU_ABI_ACC_OP_END_OF_OPS 2 +#define IMGU_ABI_ACC_OP_NO_OPS 3 + +#define IMGU_ABI_ACC_OPTYPE_PROCESS_LINES 0 +#define IMGU_ABI_ACC_OPTYPE_TRANSFER_DATA 1 + +/* Register definitions */ + +/* PM_CTRL_0_5_0_IMGHMMADR */ +#define IMGU_REG_PM_CTRL 0x0 +#define IMGU_PM_CTRL_START BIT(0) +#define IMGU_PM_CTRL_CFG_DONE BIT(1) +#define IMGU_PM_CTRL_RACE_TO_HALT BIT(2) +#define IMGU_PM_CTRL_NACK_ALL BIT(3) +#define IMGU_PM_CTRL_CSS_PWRDN BIT(4) +#define IMGU_PM_CTRL_RST_AT_EOFBIT(5) +#define IMGU_PM_CTRL_FORCE_HALTBIT(6) +#define IMGU_PM_CTRL_FORCE_UNHALT BIT(7) +#define IMGU_PM_CTRL_FORCE_PWRDN BIT(8) +#define IMGU_PM_CTRL_FORCE_RESET BIT(9) + +/* SYSTEM_REQ_0_5_0_IMGHMMADR */ +#define IMGU_REG_SYSTEM_REQ0x18 +#define IMGU_SYSTEM_REQ_FREQ_MASK 0x3f +#define IMGU_SYSTEM_REQ_FREQ_DIVIDER 25 +#define IMGU_REG_INT_STATUS0x30 +#define IMGU_REG_INT_ENABLE0x34 +#define IMGU_REG_INT_CSS_IRQ (1 << 31) +/* STATE_0_5_0_IMGHMMADR */ +#define IMGU_REG_STATE 0x130 +#define IMGU_STATE_HALT_STSBIT(0) +#define IMGU_STATE_IDLE_STSBIT(1) +#define IMGU_STATE_POWER_UPBIT(2) +#define IMGU_STATE_POWER_DOWN BIT(3) +#define IMGU_STATE_CSS_BUSY_MASK 0xc0 +#define IMGU_STATE_PM_FSM_MASK 0x180 +#define IMGU_STATE_PWRDNM_FSM_MASK 0x1E0 +/* PM_STS_0_5_0_IMGHMMADR */ +#define IMGU_REG_PM_STS0x140 + +#define IMGU_REG_BASE 0x4000 + +#define IMGU_REG_ISP_CTRL (IMGU_REG_BASE + 0x00) +#define IMGU_CTRL_RST BIT(0) +#define IMGU_CTRL_STARTBIT(1) +#define IMGU_CTRL_BREAKBIT(2) +#define IMGU_CTRL_RUN BIT(3) +#define IMGU_CTRL_BROKEN BIT(4) +#define IMGU_CTRL_IDLE BIT(5) +#define IMGU_CTRL_SLEEPING BIT(6) +#define IMGU_CTRL_STALLING BIT(7) +#define IMGU_CTRL_IRQ_CLEARBIT(8) +#define IMGU_CTRL_IRQ_READYBIT(10) +#define IMGU_CTRL_IRQ_SLEEPING BIT(11) +#define IMGU_CTRL_ICACHE_INV BIT(12) +#define IMGU_CTRL_IPREFETCH_EN BIT(13) +#define IMGU_REG_ISP_START_ADDR(IMGU_REG_BASE + 0x04) +#define IMGU_REG_ISP_ICACHE_ADDR (IMGU_REG_BASE + 0x10) +#define IMGU_REG_ISP_PC(IMGU_REG_BASE + 0x
[PATCH v6 00/12] Intel IPU3 ImgU patchset
Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK Total: 353, Succeeded: 353, Failed: 0, Warnings: 0 Note: v4l2-compliance stream test was not performed as some manual steps are needed to pre-configure the media pipleline. Tomasz Figa (2): intel-ipu3: mmu: Implement driver intel-ipu3: Implement DMA mapping functions Yong Zhi (10): v4l: Add Intel IPU3 meta buffer formats intel-ipu3: Add user space API definitions intel-ipu3: css: Add dma buff pool utility functions intel-ipu3: css: Add support for firmware management intel-ipu3: css: Add static settings for image pipeline intel-ipu3: css: Compute and program ccs intel-ipu3: css: Initialize css hardware intel-ipu3: Add css pipeline programming intel-ipu3: Add v4l2 driver based on media framework intel-ipu3: Add imgu top level pci device driver drivers/media/pci/intel/ipu3/Kconfig | 14 + drivers/media/pci/intel/ipu3/Makefile | 11 + drivers/media/pci/intel/ipu3/ipu3-abi.h| 1888 + drivers/media/pci/intel/ipu3/ipu3-css-fw.c | 261 + drivers/media/pci/intel/ipu3/ipu3-css-fw.h | 198 + drivers/media/pci/intel/ipu3/ipu3-css-params.c | 2890 +++ drivers/media/pci/intel/ipu3/ipu3-css-params.h | 25 + drivers/media/pci/intel/ipu3/ipu3-css-pool.c | 131 + drivers/media/pci/intel/ipu3/ipu3-css-pool.h | 36 + drivers/media/pci/intel/ipu3/ipu3-css.c| 2289 ++ drivers/media/pci/intel/ipu3/ipu3-css.h| 205 + drivers/media/pci/intel/ipu3/ipu3-dmamap.c | 280 + drivers/media/pci/intel/ipu3/ipu3-dmamap.h | 22 + drivers/media/pci/intel/ipu3/ipu3-mmu.c| 560 ++ drivers/media/pci/intel/ipu3/ipu3-mmu.h| 28 + drivers/media/pci/intel/ipu3/ipu3-tables.c | 9609 drivers/media/pci/intel/ipu3/ipu3-tables.h | 66 + drivers/media/pci/intel/ipu3/ipu3-v4l2.c | 1089 +++ drivers/media/pci/intel/ipu3/ipu3.c| 849 +++ drivers/media/pci/intel/ipu3/ipu3.h| 151 + drivers/media/v4l2-core/v4l2-ioctl.c |2 + include/uapi/linux/intel-ipu3.h| 1403 include/uapi/linux/videodev2.h |4 + 23 files changed, 22011 insertions(+) create mode 100644 drivers/media/pci/intel/ipu3/ipu3-abi.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-fw.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-pool.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-dmamap.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-mmu.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-tables.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3-tables.h create mode 100644 drivers/media/pci/intel/ipu3/ipu3-v4l2.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3.c create mode 100644 drivers/media/pci/intel/ipu3/ipu3.h create mode 100644 include/uapi/linux/intel-ipu3.h -- 2.7.4
[PATCH 08/12] intel-ipu3: css: Compute and program ccs
A collection of routines that are mainly used
to calculate the parameters for accelerator cluster.
Signed-off-by: Yong Zhi <yong@intel.com>
---
drivers/media/pci/intel/ipu3/ipu3-css-params.c | 2890
drivers/media/pci/intel/ipu3/ipu3-css-params.h | 25 +
drivers/media/pci/intel/ipu3/ipu3-css.h| 205 ++
3 files changed, 3120 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.c
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css-params.h
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.h
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css-params.c
b/drivers/media/pci/intel/ipu3/ipu3-css-params.c
new file mode 100644
index ..0492a87868ec
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-css-params.c
@@ -0,0 +1,2890 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+
+#include "ipu3-css.h"
+#include "ipu3-css-fw.h"
+#include "ipu3-tables.h"
+
+#define sqr(x) ((x) * (x))
+#define DIV_ROUND_CLOSEST_DOWN(a, b) (((a) + ((b) / 2) - 1) / (b))
+#define roundclosest_down(a, b)(DIV_ROUND_CLOSEST_DOWN(a, b) *
(b))
+
+struct ipu3_css_scaler_info {
+ unsigned int phase_step;/* Same for luma/chroma */
+ int exp_shift;
+
+ unsigned int phase_init;/* luma/chroma dependent */
+ int pad_left;
+ int pad_right;
+ int crop_left;
+ int crop_top;
+};
+
+static unsigned int ipu3_css_scaler_get_exp(unsigned int counter,
+ unsigned int divider)
+{
+ int i = fls(divider) - fls(counter);
+
+ if (i <= 0)
+ return 0;
+
+ if (divider >> i < counter)
+ i = i - 1;
+
+ return i;
+}
+
+/* Set up the CSS scaler look up table */
+static void
+ipu3_css_scaler_setup_lut(unsigned int taps, unsigned int input_width,
+ unsigned int output_width, int phase_step_correction,
+ const int *coeffs, unsigned int coeffs_size,
+ s8 coeff_lut[], struct ipu3_css_scaler_info *info)
+{
+ int tap, phase, phase_sum_left, phase_sum_right;
+ int exponent = ipu3_css_scaler_get_exp(output_width, input_width);
+ int mantissa = (1 << exponent) * output_width;
+ unsigned int phase_step;
+
+ if (input_width == output_width) {
+ for (phase = 0; phase < IMGU_SCALER_PHASES; phase++) {
+ for (tap = 0; tap < taps; tap++) {
+ coeff_lut[phase * IMGU_SCALER_FILTER_TAPS + tap]
+ = 0;
+ }
+ }
+
+ info->phase_step = IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF);
+ info->exp_shift = 0;
+ info->pad_left = 0;
+ info->pad_right = 0;
+ info->phase_init = 0;
+ info->crop_left = 0;
+ info->crop_top = 0;
+ return;
+ }
+
+ for (phase = 0; phase < IMGU_SCALER_PHASES; phase++) {
+ for (tap = 0; tap < taps; tap++) {
+ /* flip table to for convolution reverse indexing */
+ s64 coeff = coeffs[coeffs_size -
+ ((tap * (coeffs_size / taps)) + phase) - 1];
+ coeff *= mantissa;
+ coeff = div64_long(coeff, input_width);
+
+ /* Add +"0.5" */
+ coeff += 1 << (IMGU_SCALER_COEFF_BITS - 1);
+ coeff >>= IMGU_SCALER_COEFF_BITS;
+
+ coeff_lut[phase * IMGU_SCALER_FILTER_TAPS + tap] =
+ coeff;
+ }
+ }
+
+ phase_step = IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF) *
+ output_width / input_width;
+ phase_step += phase_step_correction;
+ phase_sum_left = (taps / 2 * IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF)) -
+ (1 << (IMGU_SCALER_PHASE_COUNTER_PREC_REF - 1));
+ phase_sum_right = (taps / 2 * IMGU_SCALER_PHASES *
+ (1 << IMGU_SCALER_PHASE_COUNTER_PREC_REF)) +
+ (1 << (IMGU_SCALER_PHASE_COUNTER_PREC_REF - 1));
+
+ info->exp_shift = IMGU_SCALER_MAX_EXPONENT_SHIFT - exponent;
+ info->pad_left = (phase_sum_left % phase_step == 0) ?
+ phase_sum_left / phase_step - 1 : phase_sum_left / phase_step;
+ info->pad_right = (phase_sum_right % phase_step == 0) ?
+ phase_sum_right / phase_step - 1 : phase_sum_right / phase_step;
+
[PATCH v6 10/12] intel-ipu3: Add css pipeline programming
This provides helper library to be used by
v4l2 level to program imaging pipelines and
control the streaming.
Signed-off-by: Yong Zhi <yong@intel.com>
---
drivers/media/pci/intel/ipu3/ipu3-css.c | 1752 +++
1 file changed, 1752 insertions(+)
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css.c
b/drivers/media/pci/intel/ipu3/ipu3-css.c
index 164830fc91ad..48fbc037c4a3 100644
--- a/drivers/media/pci/intel/ipu3/ipu3-css.c
+++ b/drivers/media/pci/intel/ipu3/ipu3-css.c
@@ -16,6 +16,173 @@
IMGU_IRQCTRL_IRQ_SW_PIN(0) | \
IMGU_IRQCTRL_IRQ_SW_PIN(1))
+#define IPU3_CSS_FORMAT_BPP_DEN50 /* Denominator */
+
+/* Some sane limits for resolutions */
+#define IPU3_CSS_MIN_RES 32
+#define IPU3_CSS_MAX_H 3136
+#define IPU3_CSS_MAX_W 4224
+
+/* filter size from graph settings is fixed as 4 */
+#define FILTER_SIZE 4
+#define MIN_ENVELOPE8
+
+/*
+ * pre-allocated buffer size for CSS ABI, auxiliary frames
+ * after BDS and before GDC. Those values should be tuned
+ * to big enough to avoid buffer re-allocation when
+ * streaming to lower streaming latency.
+ */
+#define CSS_ABI_SIZE136
+#define CSS_BDS_SIZE(4480 * 3200 * 3)
+#define CSS_GDC_SIZE(4224 * 3200 * 12 / 8)
+
+#define IPU3_CSS_QUEUE_TO_FLAGS(q) (1 << (q))
+#define IPU3_CSS_FORMAT_FL_IN \
+ IPU3_CSS_QUEUE_TO_FLAGS(IPU3_CSS_QUEUE_IN)
+#define IPU3_CSS_FORMAT_FL_OUT \
+ IPU3_CSS_QUEUE_TO_FLAGS(IPU3_CSS_QUEUE_OUT)
+#define IPU3_CSS_FORMAT_FL_VF \
+ IPU3_CSS_QUEUE_TO_FLAGS(IPU3_CSS_QUEUE_VF)
+
+/* Formats supported by IPU3 Camera Sub System */
+static const struct ipu3_css_format ipu3_css_formats[] = {
+ {
+ .pixelformat = V4L2_PIX_FMT_NV12,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_NV12,
+ .osys_format = IMGU_ABI_OSYS_FORMAT_NV12,
+ .osys_tiling = IMGU_ABI_OSYS_TILING_NONE,
+ .bytesperpixel_num = 1 * IPU3_CSS_FORMAT_BPP_DEN,
+ .chroma_decim = 4,
+ .width_align = IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_OUT | IPU3_CSS_FORMAT_FL_VF,
+ }, {
+ /* Each 32 bytes contains 25 10-bit pixels */
+ .pixelformat = V4L2_PIX_FMT_IPU3_SBGGR10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_BGGR,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ }, {
+ .pixelformat = V4L2_PIX_FMT_IPU3_SGBRG10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_GBRG,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ }, {
+ .pixelformat = V4L2_PIX_FMT_IPU3_SGRBG10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_GRBG,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ }, {
+ .pixelformat = V4L2_PIX_FMT_IPU3_SRGGB10,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .frame_format = IMGU_ABI_FRAME_FORMAT_RAW_PACKED,
+ .bayer_order = IMGU_ABI_BAYER_ORDER_RGGB,
+ .bit_depth = 10,
+ .bytesperpixel_num = 64,
+ .width_align = 2 * IPU3_UAPI_ISP_VEC_ELEMS,
+ .flags = IPU3_CSS_FORMAT_FL_IN,
+ },
+};
+
+static const struct {
+ enum imgu_abi_queue_id qid;
+ size_t ptr_ofs;
+} ipu3_css_queues[IPU3_CSS_QUEUES] = {
+ [IPU3_CSS_QUEUE_IN] = {
+ IMGU_ABI_QUEUE_C_ID,
+ offsetof(struct imgu_abi_buffer, payload.frame.frame_data)
+ },
+ [IPU3_CSS_QUEUE_OUT] = {
+ IMGU_ABI_QUEUE_D_ID,
+ offsetof(struct imgu_abi_buffer, payload.frame.frame_data)
+ },
+ [IPU3_CSS_QUEUE_VF] = {
+ IMGU_ABI_QUEUE_E_ID,
+ offsetof(struct imgu_abi_buffer, payload.frame.frame_data)
+ },
+ [IPU3_CSS_QUEUE_STAT_3A] = {
+ IMGU_ABI_QUEUE_F_ID,
+ offsetof(struct imgu_abi_buffer, payload.s3a.data_ptr)
+ },
+};
+
+/* Initialize queue based on given format, adjust format as needed */
+static int ipu3_cs
[PATCH v6 09/12] intel-ipu3: css: Initialize css hardware
This patch implements the functions to initialize
and configure IPU3 h/w such as clock, irq and power.
Signed-off-by: Yong Zhi <yong@intel.com>
Signed-off-by: Tomasz Figa <tf...@chromium.org>
---
drivers/media/pci/intel/ipu3/ipu3-css.c | 537
1 file changed, 537 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3-css.c
diff --git a/drivers/media/pci/intel/ipu3/ipu3-css.c
b/drivers/media/pci/intel/ipu3/ipu3-css.c
new file mode 100644
index ..164830fc91ad
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3-css.c
@@ -0,0 +1,537 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright (C) 2018 Intel Corporation
+
+#include
+#include
+
+#include "ipu3-css.h"
+#include "ipu3-css-fw.h"
+#include "ipu3-css-params.h"
+#include "ipu3-dmamap.h"
+#include "ipu3-tables.h"
+
+/* IRQ configuration */
+#define IMGU_IRQCTRL_IRQ_MASK (IMGU_IRQCTRL_IRQ_SP1 | \
+IMGU_IRQCTRL_IRQ_SP2 | \
+IMGU_IRQCTRL_IRQ_SW_PIN(0) | \
+IMGU_IRQCTRL_IRQ_SW_PIN(1))
+
+/*** css hw ***/
+
+/* In the style of writesl() defined in include/asm-generic/io.h */
+static inline void writes(const void *mem, ssize_t count, void __iomem *addr)
+{
+ if (count >= 4) {
+ const u32 *buf = mem;
+
+ count /= 4;
+ do {
+ writel(*buf++, addr);
+ addr += 4;
+ } while (--count);
+ }
+}
+
+/* Wait until register `reg', masked with `mask', becomes `cmp' */
+static int ipu3_hw_wait(void __iomem *base, int reg, u32 mask, u32 cmp)
+{
+ u32 val;
+
+ return readl_poll_timeout(base + reg, val, (val & mask) == cmp,
+ 1000, 100 * 1000);
+}
+
+/* Initialize the IPU3 CSS hardware and associated h/w blocks */
+
+int ipu3_css_set_powerup(struct device *dev, void __iomem *base)
+{
+ static const unsigned int freq = 450;
+ u32 pm_ctrl, state, val;
+
+ dev_dbg(dev, "%s\n", __func__);
+ /* Clear the CSS busy signal */
+ readl(base + IMGU_REG_GP_BUSY);
+ writel(0, base + IMGU_REG_GP_BUSY);
+
+ /* Wait for idle signal */
+ if (ipu3_hw_wait(base, IMGU_REG_STATE, IMGU_STATE_IDLE_STS,
+IMGU_STATE_IDLE_STS)) {
+ dev_err(dev, "failed to set CSS idle\n");
+ goto fail;
+ }
+
+ /* Reset the css */
+ writel(readl(base + IMGU_REG_PM_CTRL) | IMGU_PM_CTRL_FORCE_RESET,
+ base + IMGU_REG_PM_CTRL);
+
+ usleep_range(200, 300);
+
+ /** Prepare CSS */
+
+ pm_ctrl = readl(base + IMGU_REG_PM_CTRL);
+ state = readl(base + IMGU_REG_STATE);
+
+ dev_dbg(dev, "CSS pm_ctrl 0x%x state 0x%x (power %s)\n",
+ pm_ctrl, state, state & IMGU_STATE_POWER_DOWN ? "down" : "up");
+
+ /* Power up CSS using wrapper */
+ if (state & IMGU_STATE_POWER_DOWN) {
+ writel(IMGU_PM_CTRL_RACE_TO_HALT | IMGU_PM_CTRL_START,
+ base + IMGU_REG_PM_CTRL);
+ if (ipu3_hw_wait(base, IMGU_REG_PM_CTRL,
+IMGU_PM_CTRL_START, 0)) {
+ dev_err(dev, "failed to power up CSS\n");
+ goto fail;
+ }
+ usleep_range(2000, 3000);
+ } else {
+ writel(IMGU_PM_CTRL_RACE_TO_HALT, base + IMGU_REG_PM_CTRL);
+ }
+
+ /* Set the busy bit */
+ writel(readl(base + IMGU_REG_GP_BUSY) | 1, base + IMGU_REG_GP_BUSY);
+
+ /* Set CSS clock frequency */
+ pm_ctrl = readl(base + IMGU_REG_PM_CTRL);
+ val = pm_ctrl & ~(IMGU_PM_CTRL_CSS_PWRDN | IMGU_PM_CTRL_RST_AT_EOF);
+ writel(val, base + IMGU_REG_PM_CTRL);
+ writel(0, base + IMGU_REG_GP_BUSY);
+ if (ipu3_hw_wait(base, IMGU_REG_STATE,
+IMGU_STATE_PWRDNM_FSM_MASK, 0)) {
+ dev_err(dev, "failed to pwrdn CSS\n");
+ goto fail;
+ }
+ val = (freq / IMGU_SYSTEM_REQ_FREQ_DIVIDER) & IMGU_SYSTEM_REQ_FREQ_MASK;
+ writel(val, base + IMGU_REG_SYSTEM_REQ);
+ writel(1, base + IMGU_REG_GP_BUSY);
+ writel(readl(base + IMGU_REG_PM_CTRL) | IMGU_PM_CTRL_FORCE_HALT,
+ base + IMGU_REG_PM_CTRL);
+ if (ipu3_hw_wait(base, IMGU_REG_STATE, IMGU_STATE_HALT_STS,
+IMGU_STATE_HALT_STS)) {
+ dev_err(dev, "failed to halt CSS\n");
+ goto fail;
+ }
+
+ writel(readl(base + IMGU_REG_PM_CTRL) | IMGU_PM_CTRL_START,
+ base + IMGU_REG_PM_CTRL);
+ if (ipu3_hw_wait(base, IMGU_REG_PM_CTRL, IMGU_PM_CTRL_START, 0)) {
+ dev_err(dev, "failed to start CSS\n
[PATCH v6 12/12] intel-ipu3: Add imgu top level pci device driver
This patch adds support for the Intel IPU v3 as found
on Skylake and Kaby Lake SoCs.
The driver glues v4l2, css(camera sub system) and other
pieces together to perform its functions, it also loads
the IPU3 firmware binary as part of its initialization.
Signed-off-by: Yong Zhi <yong@intel.com>
Signed-off-by: Tomasz Figa <tf...@chromium.org>
---
drivers/media/pci/intel/ipu3/Kconfig | 14 +
drivers/media/pci/intel/ipu3/Makefile | 11 +
drivers/media/pci/intel/ipu3/ipu3.c | 849 ++
3 files changed, 874 insertions(+)
create mode 100644 drivers/media/pci/intel/ipu3/ipu3.c
diff --git a/drivers/media/pci/intel/ipu3/Kconfig
b/drivers/media/pci/intel/ipu3/Kconfig
index a82d3fe277d2..a844735cab2e 100644
--- a/drivers/media/pci/intel/ipu3/Kconfig
+++ b/drivers/media/pci/intel/ipu3/Kconfig
@@ -17,3 +17,17 @@ config VIDEO_IPU3_CIO2
Say Y or M here if you have a Skylake/Kaby Lake SoC with MIPI CSI-2
connected camera.
The module will be called ipu3-cio2.
+
+config VIDEO_IPU3_IMGU
+ tristate "Intel ipu3-imgu driver"
+ depends on PCI && VIDEO_V4L2
+ depends on MEDIA_CONTROLLER && VIDEO_V4L2_SUBDEV_API
+ select IOMMU_IOVA
+ select VIDEOBUF2_DMA_SG
+ ---help---
+ This is the video4linux2 driver for Intel IPU3 image processing unit,
+ found in Intel Skylake and Kaby Lake SoCs and used for processing
+ images and video.
+
+ Say Y or M here if you have a Skylake/Kaby Lake SoC with a MIPI
+ camera. The module will be called ipu3-imgu.
diff --git a/drivers/media/pci/intel/ipu3/Makefile
b/drivers/media/pci/intel/ipu3/Makefile
index 20186e3ff2ae..dbfe37db3412 100644
--- a/drivers/media/pci/intel/ipu3/Makefile
+++ b/drivers/media/pci/intel/ipu3/Makefile
@@ -1 +1,12 @@
+#
+# Makefile for the IPU3 cio2 and ImgU drivers
+#
+
obj-$(CONFIG_VIDEO_IPU3_CIO2) += ipu3-cio2.o
+
+ipu3-imgu-objs += ipu3-mmu.o ipu3-dmamap.o \
+ ipu3-tables.o ipu3-css-pool.o \
+ ipu3-css-fw.o ipu3-css-params.o \
+ ipu3-css.o ipu3-v4l2.o ipu3.o
+
+obj-$(CONFIG_VIDEO_IPU3_IMGU) += ipu3-imgu.o
diff --git a/drivers/media/pci/intel/ipu3/ipu3.c
b/drivers/media/pci/intel/ipu3/ipu3.c
new file mode 100644
index ..c2fb0ab3bd6d
--- /dev/null
+++ b/drivers/media/pci/intel/ipu3/ipu3.c
@@ -0,0 +1,849 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2017 Intel Corporation
+ * Copyright (C) 2017 Google, Inc.
+ *
+ * Based on Intel IPU4 driver.
+ *
+ */
+
+#include
+#include
+#include
+#include
+
+#include "ipu3.h"
+#include "ipu3-dmamap.h"
+#include "ipu3-mmu.h"
+
+#define IMGU_PCI_ID0x1919
+#define IMGU_PCI_BAR 0
+#define IMGU_DMA_MASK DMA_BIT_MASK(39)
+#define IMGU_MAX_QUEUE_DEPTH (2 + 2)
+
+/*
+ * pre-allocated buffer size for IMGU dummy buffers. Those
+ * values should be tuned to big enough to avoid buffer
+ * re-allocation when streaming to lower streaming latency.
+ */
+#define CSS_QUEUE_IN_BUF_SIZE 0
+#define CSS_QUEUE_PARAMS_BUF_SIZE 0
+#define CSS_QUEUE_OUT_BUF_SIZE (4160 * 3120 * 12 / 8)
+#define CSS_QUEUE_VF_BUF_SIZE (1920 * 1080 * 12 / 8)
+#define CSS_QUEUE_STAT_3A_BUF_SIZE 125664
+
+static const size_t css_queue_buf_size_map[IPU3_CSS_QUEUES] = {
+ [IPU3_CSS_QUEUE_IN] = CSS_QUEUE_IN_BUF_SIZE,
+ [IPU3_CSS_QUEUE_PARAMS] = CSS_QUEUE_PARAMS_BUF_SIZE,
+ [IPU3_CSS_QUEUE_OUT] = CSS_QUEUE_OUT_BUF_SIZE,
+ [IPU3_CSS_QUEUE_VF] = CSS_QUEUE_VF_BUF_SIZE,
+ [IPU3_CSS_QUEUE_STAT_3A] = CSS_QUEUE_STAT_3A_BUF_SIZE,
+};
+
+static const struct imgu_node_mapping imgu_node_map[IMGU_NODE_NUM] = {
+ [IMGU_NODE_IN] = {IPU3_CSS_QUEUE_IN, "input"},
+ [IMGU_NODE_PARAMS] = {IPU3_CSS_QUEUE_PARAMS, "parameters"},
+ [IMGU_NODE_OUT] = {IPU3_CSS_QUEUE_OUT, "output"},
+ [IMGU_NODE_VF] = {IPU3_CSS_QUEUE_VF, "viewfinder"},
+ [IMGU_NODE_PV] = {IPU3_CSS_QUEUE_VF, "postview"},
+ [IMGU_NODE_STAT_3A] = {IPU3_CSS_QUEUE_STAT_3A, "3a stat"},
+};
+
+unsigned int imgu_node_to_queue(unsigned int node)
+{
+ return imgu_node_map[node].css_queue;
+}
+
+unsigned int imgu_map_node(struct imgu_device *imgu, unsigned int css_queue)
+{
+ unsigned int i;
+
+ if (css_queue == IPU3_CSS_QUEUE_VF)
+ return imgu->nodes[IMGU_NODE_VF].enabled ?
+ IMGU_NODE_VF : IMGU_NODE_PV;
+
+ for (i = 0; i < IMGU_NODE_NUM; i++)
+ if (imgu_node_map[i].css_queue == css_queue)
+ break;
+
+ return i;
+}
+
+/ Dummy buffers /
+
+static void imgu_dummybufs_cleanup(struct imgu_device *imgu)
+{
+ unsigned int i;
+
+ for (i = 0; i < IPU3_CSS_QUEUES; i++)
+ ipu3_dmamap_free(>pci_dev-
Re: [linux-sunxi] [PATCH v9 0/2] Initial Allwinner V3s CSI Support
Hi, On Wed, 28 Mar 2018 16:29:47 -0700 Martin Kelly <mke...@xevo.com> wrote: > On 03/05/2018 05:51 PM, Yong Deng wrote: > > This patchset add initial support for Allwinner V3s CSI. > > > > Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2 > > interface and CSI1 is used for parallel interface. This is not > > documented in datasheet but by test and guess. > > > > This patchset implement a v4l2 framework driver and add a binding > > documentation for it. > > > > Currently, the driver only support the parallel interface. And has been > > tested with a BT1120 signal which generating from FPGA. The following > > fetures are not support with this patchset: > >- ISP > >- MIPI-CSI2 > > - Master clock for camera sensor > >- Power regulator for the front end IC > > > > Hi Yong, > > Thanks so much, this driver is a great contribution! > > Unfortunately the board I'm working with (nanopi neo air) uses the MIPI > CSI-2 CSI0 interface rather than CSI1. Do you have any plans to support > the MIPI CSI-2 interface at some point? If not, do you know the scope of > what would be involved? AFAIK, there is no document about MIPI CSI-2. You can take a look at the source code in BSP: https://github.com/friendlyarm/h3_lichee/tree/master/linux-3.4/drivers/media/video/sunxi-vfe/mipi_csi And try to port it to mainline. Thanks, Yong
Re: [PATCH v9 2/2] media: V3s: Add support for Allwinner CSI.
Hi,
On Tue, 6 Mar 2018 17:14:18 +0200
Sakari Ailus <sakari.ai...@linux.intel.com> wrote:
> Hi Yong,
>
> Thanks for the patchset; please see my comments below.
>
> On Tue, Mar 06, 2018 at 10:16:02AM +0800, Yong Deng wrote:
> > Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2
> > interface and CSI1 is used for parallel interface. This is not
> > documented in datasheet but by test and guess.
...
> > +
> > +static const u32 supported_pixformats[] = {
> > + V4L2_PIX_FMT_SBGGR8,
> > + V4L2_PIX_FMT_SGBRG8,
> > + V4L2_PIX_FMT_SGRBG8,
> > + V4L2_PIX_FMT_SRGGB8,
> > + V4L2_PIX_FMT_SBGGR10,
> > + V4L2_PIX_FMT_SGBRG10,
> > + V4L2_PIX_FMT_SGRBG10,
> > + V4L2_PIX_FMT_SRGGB10,
> > + V4L2_PIX_FMT_SBGGR12,
> > + V4L2_PIX_FMT_SGBRG12,
> > + V4L2_PIX_FMT_SGRBG12,
> > + V4L2_PIX_FMT_SRGGB12,
> > + V4L2_PIX_FMT_YUYV,
> > + V4L2_PIX_FMT_YVYU,
> > + V4L2_PIX_FMT_UYVY,
> > + V4L2_PIX_FMT_VYUY,
> > + V4L2_PIX_FMT_HM12,
> > + V4L2_PIX_FMT_NV12,
> > + V4L2_PIX_FMT_NV21,
> > + V4L2_PIX_FMT_YUV420,
> > + V4L2_PIX_FMT_YVU420,
> > + V4L2_PIX_FMT_NV16,
> > + V4L2_PIX_FMT_NV61,
> > + V4L2_PIX_FMT_YUV422P,
> > +};
>
> How about moving this where it's actually used? You'd also get rid of the
> function to obtain this list.
I think which formats are supported is determined by hardware (CSI).
And different SoCs may support different formats. The distinction will
be made in sun6i-csi.c.
>
> > +
> > +static inline struct sun6i_csi_dev *sun6i_csi_to_dev(struct sun6i_csi *csi)
> > +{
> > + return container_of(csi, struct sun6i_csi_dev, csi);
> > +}
> > +
> > +int sun6i_csi_get_supported_pixformats(struct sun6i_csi *csi,
> > + const u32 **pixformats)
> > +{
> > + if (pixformats != NULL)
> > + *pixformats = supported_pixformats;
> > +
> > + return ARRAY_SIZE(supported_pixformats);
> > +}
> > +
> > +/* TODO add 10&12 bit YUV, RGB support */
> > +bool sun6i_csi_is_format_support(struct sun6i_csi *csi,
>
> s/support/supported/
OK.
>
> > +u32 pixformat, u32 mbus_code)
> > +{
> > + struct sun6i_csi_dev *sdev = sun6i_csi_to_dev(csi);
> > +
> > + /*
> > +* Some video receivers have the ability to be compatible with
> > +* 8bit and 16bit bus width.
> > +* Identify the media bus format from device tree.
> > +*/
> > + if ((sdev->csi.v4l2_ep.bus_type == V4L2_MBUS_PARALLEL
> > + || sdev->csi.v4l2_ep.bus_type == V4L2_MBUS_BT656)
> > +&& sdev->csi.v4l2_ep.bus.parallel.bus_width == 16) {
> > + switch (pixformat) {
> > + case V4L2_PIX_FMT_HM12:
> > + case V4L2_PIX_FMT_NV12:
> > + case V4L2_PIX_FMT_NV21:
> > + case V4L2_PIX_FMT_NV16:
> > + case V4L2_PIX_FMT_NV61:
> > + case V4L2_PIX_FMT_YUV420:
> > + case V4L2_PIX_FMT_YVU420:
> > + case V4L2_PIX_FMT_YUV422P:
> > + switch (mbus_code) {
> > + case MEDIA_BUS_FMT_UYVY8_1X16:
> > + case MEDIA_BUS_FMT_VYUY8_1X16:
> > + case MEDIA_BUS_FMT_YUYV8_1X16:
> > + case MEDIA_BUS_FMT_YVYU8_1X16:
> > + return true;
> > + default:
> > + dev_dbg(sdev->dev, "Unsupported mbus code:
> > 0x%x\n",
> > + mbus_code);
> > + break;
> > + }
> > + break;
> > + default:
> > + dev_dbg(sdev->dev, "Unsupported pixformat: 0x%x\n",
> > + pixformat);
> > + break;
> > + }
> > + return false;
> > + }
> > +
> > + switch (pixformat) {
> > + case V4L2_PIX_FMT_SBGGR8:
> > + return (mbus_code == MEDIA_BUS_FMT_SBGGR8_1X8);
> > + case V4L2_PIX_FMT_SGBRG8:
> > + return (mbus_code == MEDIA_BUS_FMT_SGBRG8_1X8);
> > + case V4L2_PIX_FMT_SGRBG8:
> > + return (mbus_code == MEDIA_BUS_FMT_SGRBG8_1X8);
> > + case V4L2_PIX_FMT_SRGGB8:
> > + return (mbus_code == MEDIA_BUS_FMT_SRGGB8_1X8);
> > + case V4L2_PIX_FMT_SBGGR10:
> > + return (mbus_code == MEDIA_BUS_FMT_SBGGR10_1X10);
> > + c
[PATCH v9 2/2] media: V3s: Add support for Allwinner CSI.
Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2 interface and CSI1 is used for parallel interface. This is not documented in datasheet but by test and guess. This patch implement a v4l2 framework driver for it. Currently, the driver only support the parallel interface. MIPI-CSI2, ISP's support are not included in this patch. Reviewed-by: Maxime Ripard <maxime.rip...@bootlin.com> Tested-by: Maxime Ripard <maxime.rip...@bootlin.com> Signed-off-by: Yong Deng <yong.d...@magewell.com> --- MAINTAINERS| 8 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile| 2 + drivers/media/platform/sunxi/sun6i-csi/Kconfig | 9 + drivers/media/platform/sunxi/sun6i-csi/Makefile| 3 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 936 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 145 .../media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h | 196 + .../media/platform/sunxi/sun6i-csi/sun6i_video.c | 759 + .../media/platform/sunxi/sun6i-csi/sun6i_video.h | 53 ++ 10 files changed, 2112 insertions(+) create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Kconfig create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Makefile create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h diff --git a/MAINTAINERS b/MAINTAINERS index 91ed6adfa4a6..b4a331ad35b5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3793,6 +3793,14 @@ M: Jaya Kumar <jayakumar.a...@gmail.com> S: Maintained F: sound/pci/cs5535audio/ +CSI DRIVERS FOR ALLWINNER V3s +M: Yong Deng <yong.d...@magewell.com> +L: linux-media@vger.kernel.org +T: git git://linuxtv.org/media_tree.git +S: Maintained +F: drivers/media/platform/sunxi/sun6i-csi/ +F: Documentation/devicetree/bindings/media/sun6i-csi.txt + CW1200 WLAN driver M: Solomon Peachy <pi...@shaftnet.org> S: Maintained diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index f9cc0582c8a9..7f1ee46c3258 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -159,6 +159,7 @@ source "drivers/media/platform/am437x/Kconfig" source "drivers/media/platform/xilinx/Kconfig" source "drivers/media/platform/rcar-vin/Kconfig" source "drivers/media/platform/atmel/Kconfig" +source "drivers/media/platform/sunxi/sun6i-csi/Kconfig" config VIDEO_TI_CAL tristate "TI CAL (Camera Adaptation Layer) driver" diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile index 85e112122f32..143d8a473b0a 100644 --- a/drivers/media/platform/Makefile +++ b/drivers/media/platform/Makefile @@ -96,3 +96,5 @@ obj-$(CONFIG_VIDEO_QCOM_CAMSS)+= qcom/camss-8x16/ obj-$(CONFIG_VIDEO_QCOM_VENUS) += qcom/venus/ obj-y += meson/ + +obj-$(CONFIG_VIDEO_SUN6I_CSI) += sunxi/sun6i-csi/ diff --git a/drivers/media/platform/sunxi/sun6i-csi/Kconfig b/drivers/media/platform/sunxi/sun6i-csi/Kconfig new file mode 100644 index ..314188aae2c2 --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/Kconfig @@ -0,0 +1,9 @@ +config VIDEO_SUN6I_CSI + tristate "Allwinner V3s Camera Sensor Interface driver" + depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API && HAS_DMA + depends on ARCH_SUNXI || COMPILE_TEST + select VIDEOBUF2_DMA_CONTIG + select REGMAP_MMIO + select V4L2_FWNODE + ---help--- + Support for the Allwinner Camera Sensor Interface Controller on V3s. diff --git a/drivers/media/platform/sunxi/sun6i-csi/Makefile b/drivers/media/platform/sunxi/sun6i-csi/Makefile new file mode 100644 index ..213cb6be9e9c --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/Makefile @@ -0,0 +1,3 @@ +sun6i-csi-y += sun6i_video.o sun6i_csi.o + +obj-$(CONFIG_VIDEO_SUN6I_CSI) += sun6i-csi.o diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c new file mode 100644 index ..26d57e6053df --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c @@ -0,0 +1,936 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) + * All rights reserved. + * Author: Yong Deng <yong.d...@magewell.com> + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#incl
[PATCH v9 1/2] dt-bindings: media: Add Allwinner V3s Camera Sensor Interface (CSI)
Add binding documentation for Allwinner V3s CSI.
Acked-by: Maxime Ripard <maxime.rip...@bootlin.com>
Acked-by: Sakari Ailus <sakari.ai...@linux.intel.com>
Reviewed-by: Rob Herring <r...@kernel.org>
Signed-off-by: Yong Deng <yong.d...@magewell.com>
---
.../devicetree/bindings/media/sun6i-csi.txt| 59 ++
1 file changed, 59 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/sun6i-csi.txt
diff --git a/Documentation/devicetree/bindings/media/sun6i-csi.txt
b/Documentation/devicetree/bindings/media/sun6i-csi.txt
new file mode 100644
index ..2ff47a9507a6
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/sun6i-csi.txt
@@ -0,0 +1,59 @@
+Allwinner V3s Camera Sensor Interface
+-
+
+Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2
+interface and CSI1 is used for parallel interface.
+
+Required properties:
+ - compatible: value must be "allwinner,sun8i-v3s-csi"
+ - reg: base address and size of the memory-mapped region.
+ - interrupts: interrupt associated to this IP
+ - clocks: phandles to the clocks feeding the CSI
+* bus: the CSI interface clock
+* mod: the CSI module clock
+* ram: the CSI DRAM clock
+ - clock-names: the clock names mentioned above
+ - resets: phandles to the reset line driving the CSI
+
+Each CSI node should contain one 'port' child node with one child 'endpoint'
+node, according to the bindings defined in
+Documentation/devicetree/bindings/media/video-interfaces.txt. As mentioned
+above, the endpoint's bus type should be MIPI CSI-2 for CSI0 and parallel or
+Bt656 for CSI1.
+
+Endpoint node properties for CSI1
+-
+
+- remote-endpoint : (required) a phandle to the bus receiver's endpoint
+ node
+- bus-width: : (required) must be 8, 10, 12 or 16
+- pclk-sample : (optional) (default: sample on falling edge)
+- hsync-active : (only required for parallel)
+- vsync-active : (only required for parallel)
+
+Example:
+
+csi1: csi@1cb4000 {
+ compatible = "allwinner,sun8i-v3s-csi";
+ reg = <0x01cb4000 0x1000>;
+ interrupts = ;
+ clocks = < CLK_BUS_CSI>,
+< CLK_CSI1_SCLK>,
+< CLK_DRAM_CSI>;
+ clock-names = "bus", "mod", "ram";
+ resets = < RST_BUS_CSI>;
+
+ port {
+ /* Parallel bus endpoint */
+ csi1_ep: endpoint {
+ remote-endpoint = <_ep>;
+ bus-width = <16>;
+
+ /* If hsync-active/vsync-active are missing,
+ embedded BT.656 sync is used */
+ hsync-active = <0>; /* Active low */
+ vsync-active = <0>; /* Active low */
+ pclk-sample = <1>; /* Rising */
+ };
+ };
+};
--
1.8.3.1
[PATCH v9 0/2] Initial Allwinner V3s CSI Support
VIDIOC_G/S_EDID: OK (Not Supported) Test input 0: Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported) test VIDIOC_QUERYCTRL: OK (Not Supported) test VIDIOC_G/S_CTRL: OK (Not Supported) test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported) test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK Test input 0: Streaming ioctls: test read/write: OK (Not Supported) test MMAP: OK test USERPTR: OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device Stream using all formats: test MMAP for Format HM12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV21, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format YU12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format YV12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV16, Frame Size 1280x720: Stride 2560, Field None: OK test MMAP for Format NV61, Frame Size 1280x720: Stride 2560, Field None: OK test MMAP for Format 422P, Frame Size 1280x720: Stride 2560, Field None: OK Total: 54, Succeeded: 54, Failed: 0, Warnings: 0 Yong Deng (2): dt-bindings: media: Add Allwinner V3s Camera Sensor Interface (CSI) media: V3s: Add support for Allwinner CSI. .../devicetree/bindings/media/sun6i-csi.txt| 59 ++ MAINTAINERS| 8 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile| 2 + drivers/media/platform/sunxi/sun6i-csi/Kconfig | 9 + drivers/media/platform/sunxi/sun6i-csi/Makefile| 3 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 936 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 145 .../media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h | 196 + .../media/platform/sunxi/sun6i-csi/sun6i_video.c | 759 + .../media/platform/sunxi/sun6i-csi/sun6i_video.h | 53 ++ 11 files changed, 2171 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/sun6i-csi.txt create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Kconfig create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Makefile create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h -- 1.8.3.1
Re: [PATCH 0/7] media: sun6i: Various fixes and improvements
Hi Maxime, On Mon, 5 Mar 2018 10:35:27 +0100 Maxime Ripard <maxime.rip...@bootlin.com> wrote: > Hi Yong, > > Here are a bunch of patches I came up with after testing your last > (v8) version of the CSI patches. > > There's some improvements (patches 1 and 7) and fixes for > regressions found in the v8 compared to the v7 (patches 2, 3, 4 and > 5), and one fix that we discussed for the signals polarity for the > parallel interface (patch 6). > > Feel free to squash them in your serie for the v9. OK. Thank you! I notice that your responses have not been listed in google group since February. > Thanks! > Maxime > > Maxime Ripard (7): > media: sun6i: Fill dma_pfn_offset to accomodate for the RAM offset > media: sun6i: Reduce the error level > media: sun6i: Pass the sun6i_csi_dev pointer to our helpers > media: sun6i: Don't emit a warning when the configured format isn't > found > media: sun6i: Support the YUYV format properly > media: sun6i: Invert the polarities > media: sun6i: Expose controls on the v4l2_device > > drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 85 > ++ > drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 2 + > .../media/platform/sunxi/sun6i-csi/sun6i_video.c | 6 ++ > 3 files changed, 63 insertions(+), 30 deletions(-) > > -- > 2.14.3 Thanks, Yong
Re: [PATCH v8 1/2] dt-bindings: media: Add Allwinner V3s Camera Sensor Interface (CSI)
On Tue, 27 Feb 2018 09:20:39 +0100 Philipp Zabel <p.za...@pengutronix.de> wrote: > On Tue, 2018-02-27 at 10:07 +0800, Yong Deng wrote: > > Add binding documentation for Allwinner V3s CSI. > > > > Acked-by: Maxime Ripard <maxime.rip...@free-electrons.com> > > Acked-by: Sakari Ailus <sakari.ai...@linux.intel.com> > > Reviewed-by: Rob Herring <r...@kernel.org> > > Signed-off-by: Yong Deng <yong.d...@magewell.com> > > --- > > .../devicetree/bindings/media/sun6i-csi.txt| 59 > > ++ > > 1 file changed, 59 insertions(+) > > create mode 100644 Documentation/devicetree/bindings/media/sun6i-csi.txt > > > > diff --git a/Documentation/devicetree/bindings/media/sun6i-csi.txt > > b/Documentation/devicetree/bindings/media/sun6i-csi.txt > > new file mode 100644 > > index ..2ff47a9507a6 > > --- /dev/null > > +++ b/Documentation/devicetree/bindings/media/sun6i-csi.txt > > @@ -0,0 +1,59 @@ > > +Allwinner V3s Camera Sensor Interface > > +- > > + > > +Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2 > > +interface and CSI1 is used for parallel interface. > > + > > +Required properties: > > + - compatible: value must be "allwinner,sun8i-v3s-csi" > > + - reg: base address and size of the memory-mapped region. > > + - interrupts: interrupt associated to this IP > > + - clocks: phandles to the clocks feeding the CSI > > +* bus: the CSI interface clock > > +* mod: the CSI module clock > > +* ram: the CSI DRAM clock > > + - clock-names: the clock names mentioned above > > + - resets: phandles to the reset line driving the CSI > > + > > +Each CSI node should contain one 'port' child node with one child > > 'endpoint' > > +node, according to the bindings defined in > > +Documentation/devicetree/bindings/media/video-interfaces.txt. As mentioned > > +above, the endpoint's bus type should be MIPI CSI-2 for CSI0 and parallel > > or > > +Bt656 for CSI1. > > + > > +Endpoint node properties for CSI1 > > +- > > + > > +- remote-endpoint : (required) a phandle to the bus receiver's endpoint > > + node > > +- bus-width: : (required) must be 8, 10, 12 or 16 > > +- pclk-sample : (optional) (default: sample on falling edge) > > It would be helpful to state that 1 is rising edge and 0 is falling > edge, see for example ov5640.txt I think this is already documented at Documentation/devicetree/bindings/media/video-interfaces.txt. > > regards > Philipp Thanks, Yong
[PATCH v8 2/2] media: V3s: Add support for Allwinner CSI.
Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2 interface and CSI1 is used for parallel interface. This is not documented in datasheet but by test and guess. This patch implement a v4l2 framework driver for it. Currently, the driver only support the parallel interface. MIPI-CSI2, ISP's support are not included in this patch. Tested-by: Maxime Ripard <maxime.rip...@free-electrons.com> Signed-off-by: Yong Deng <yong.d...@magewell.com> --- MAINTAINERS| 8 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile| 2 + drivers/media/platform/sunxi/sun6i-csi/Kconfig | 9 + drivers/media/platform/sunxi/sun6i-csi/Makefile| 3 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 911 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 143 .../media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h | 196 + .../media/platform/sunxi/sun6i-csi/sun6i_video.c | 753 + .../media/platform/sunxi/sun6i-csi/sun6i_video.h | 53 ++ 10 files changed, 2079 insertions(+) create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Kconfig create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Makefile create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h diff --git a/MAINTAINERS b/MAINTAINERS index 91ed6adfa4a6..b4a331ad35b5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3793,6 +3793,14 @@ M: Jaya Kumar <jayakumar.a...@gmail.com> S: Maintained F: sound/pci/cs5535audio/ +CSI DRIVERS FOR ALLWINNER V3s +M: Yong Deng <yong.d...@magewell.com> +L: linux-media@vger.kernel.org +T: git git://linuxtv.org/media_tree.git +S: Maintained +F: drivers/media/platform/sunxi/sun6i-csi/ +F: Documentation/devicetree/bindings/media/sun6i-csi.txt + CW1200 WLAN driver M: Solomon Peachy <pi...@shaftnet.org> S: Maintained diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index f9cc0582c8a9..7f1ee46c3258 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -159,6 +159,7 @@ source "drivers/media/platform/am437x/Kconfig" source "drivers/media/platform/xilinx/Kconfig" source "drivers/media/platform/rcar-vin/Kconfig" source "drivers/media/platform/atmel/Kconfig" +source "drivers/media/platform/sunxi/sun6i-csi/Kconfig" config VIDEO_TI_CAL tristate "TI CAL (Camera Adaptation Layer) driver" diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile index 85e112122f32..143d8a473b0a 100644 --- a/drivers/media/platform/Makefile +++ b/drivers/media/platform/Makefile @@ -96,3 +96,5 @@ obj-$(CONFIG_VIDEO_QCOM_CAMSS)+= qcom/camss-8x16/ obj-$(CONFIG_VIDEO_QCOM_VENUS) += qcom/venus/ obj-y += meson/ + +obj-$(CONFIG_VIDEO_SUN6I_CSI) += sunxi/sun6i-csi/ diff --git a/drivers/media/platform/sunxi/sun6i-csi/Kconfig b/drivers/media/platform/sunxi/sun6i-csi/Kconfig new file mode 100644 index ..314188aae2c2 --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/Kconfig @@ -0,0 +1,9 @@ +config VIDEO_SUN6I_CSI + tristate "Allwinner V3s Camera Sensor Interface driver" + depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API && HAS_DMA + depends on ARCH_SUNXI || COMPILE_TEST + select VIDEOBUF2_DMA_CONTIG + select REGMAP_MMIO + select V4L2_FWNODE + ---help--- + Support for the Allwinner Camera Sensor Interface Controller on V3s. diff --git a/drivers/media/platform/sunxi/sun6i-csi/Makefile b/drivers/media/platform/sunxi/sun6i-csi/Makefile new file mode 100644 index ..213cb6be9e9c --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/Makefile @@ -0,0 +1,3 @@ +sun6i-csi-y += sun6i_video.o sun6i_csi.o + +obj-$(CONFIG_VIDEO_SUN6I_CSI) += sun6i-csi.o diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c new file mode 100644 index ..1e238d57 --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c @@ -0,0 +1,911 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) + * All rights reserved. + * Author: Yong Deng <yong.d...@magewell.com> + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#inclu
[PATCH v8 1/2] dt-bindings: media: Add Allwinner V3s Camera Sensor Interface (CSI)
Add binding documentation for Allwinner V3s CSI.
Acked-by: Maxime Ripard <maxime.rip...@free-electrons.com>
Acked-by: Sakari Ailus <sakari.ai...@linux.intel.com>
Reviewed-by: Rob Herring <r...@kernel.org>
Signed-off-by: Yong Deng <yong.d...@magewell.com>
---
.../devicetree/bindings/media/sun6i-csi.txt| 59 ++
1 file changed, 59 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/sun6i-csi.txt
diff --git a/Documentation/devicetree/bindings/media/sun6i-csi.txt
b/Documentation/devicetree/bindings/media/sun6i-csi.txt
new file mode 100644
index ..2ff47a9507a6
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/sun6i-csi.txt
@@ -0,0 +1,59 @@
+Allwinner V3s Camera Sensor Interface
+-
+
+Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2
+interface and CSI1 is used for parallel interface.
+
+Required properties:
+ - compatible: value must be "allwinner,sun8i-v3s-csi"
+ - reg: base address and size of the memory-mapped region.
+ - interrupts: interrupt associated to this IP
+ - clocks: phandles to the clocks feeding the CSI
+* bus: the CSI interface clock
+* mod: the CSI module clock
+* ram: the CSI DRAM clock
+ - clock-names: the clock names mentioned above
+ - resets: phandles to the reset line driving the CSI
+
+Each CSI node should contain one 'port' child node with one child 'endpoint'
+node, according to the bindings defined in
+Documentation/devicetree/bindings/media/video-interfaces.txt. As mentioned
+above, the endpoint's bus type should be MIPI CSI-2 for CSI0 and parallel or
+Bt656 for CSI1.
+
+Endpoint node properties for CSI1
+-
+
+- remote-endpoint : (required) a phandle to the bus receiver's endpoint
+ node
+- bus-width: : (required) must be 8, 10, 12 or 16
+- pclk-sample : (optional) (default: sample on falling edge)
+- hsync-active : (only required for parallel)
+- vsync-active : (only required for parallel)
+
+Example:
+
+csi1: csi@1cb4000 {
+ compatible = "allwinner,sun8i-v3s-csi";
+ reg = <0x01cb4000 0x1000>;
+ interrupts = ;
+ clocks = < CLK_BUS_CSI>,
+< CLK_CSI1_SCLK>,
+< CLK_DRAM_CSI>;
+ clock-names = "bus", "mod", "ram";
+ resets = < RST_BUS_CSI>;
+
+ port {
+ /* Parallel bus endpoint */
+ csi1_ep: endpoint {
+ remote-endpoint = <_ep>;
+ bus-width = <16>;
+
+ /* If hsync-active/vsync-active are missing,
+ embedded BT.656 sync is used */
+ hsync-active = <0>; /* Active low */
+ vsync-active = <0>; /* Active low */
+ pclk-sample = <1>; /* Rising */
+ };
+ };
+};
--
1.8.3.1
[PATCH v8 0/2] Initial Allwinner V3s CSI Support
: OK (Not Supported) test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK Test input 0: Streaming ioctls: test read/write: OK (Not Supported) test MMAP: OK test USERPTR: OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device Stream using all formats: test MMAP for Format HM12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV21, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format YU12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format YV12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV16, Frame Size 1280x720: Stride 2560, Field None: OK test MMAP for Format NV61, Frame Size 1280x720: Stride 2560, Field None: OK test MMAP for Format 422P, Frame Size 1280x720: Stride 2560, Field None: OK Total: 54, Succeeded: 54, Failed: 0, Warnings: 0 Yong Deng (2): dt-bindings: media: Add Allwinner V3s Camera Sensor Interface (CSI) media: V3s: Add support for Allwinner CSI. .../devicetree/bindings/media/sun6i-csi.txt| 59 ++ MAINTAINERS| 8 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile| 2 + drivers/media/platform/sunxi/sun6i-csi/Kconfig | 9 + drivers/media/platform/sunxi/sun6i-csi/Makefile| 3 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 911 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 143 .../media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h | 196 + .../media/platform/sunxi/sun6i-csi/sun6i_video.c | 753 + .../media/platform/sunxi/sun6i-csi/sun6i_video.h | 53 ++ 11 files changed, 2138 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/sun6i-csi.txt create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Kconfig create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Makefile create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h -- 1.8.3.1
Re: [PATCH v7 2/2] media: V3s: Add support for Allwinner CSI.
Hi, On Mon, 26 Feb 2018 12:06:37 +0100 Hans Verkuil <hverk...@xs4all.nl> wrote: > Hi all, > > On 01/30/2018 03:48 AM, Yong wrote: > > Hi, > > > > On Mon, 29 Jan 2018 13:49:14 -0800 > > Randy Dunlap <rdun...@infradead.org> wrote: > > > >> On 01/29/2018 01:21 AM, Yong Deng wrote: > >>> Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2 > >>> interface and CSI1 is used for parallel interface. This is not > >>> documented in datasheet but by test and guess. > >>> > >>> This patch implement a v4l2 framework driver for it. > >>> > >>> Currently, the driver only support the parallel interface. MIPI-CSI2, > >>> ISP's support are not included in this patch. > >>> > >>> Tested-by: Maxime Ripard <maxime.rip...@free-electrons.com> > >>> Signed-off-by: Yong Deng <yong.d...@magewell.com> > >>> --- > >> > >> > >> A previous version (I think v6) had a build error with the use of > >> PHYS_OFFSET, so Kconfig was modified to depend on ARM and ARCH_SUNXI > >> (one of which seems to be overkill). As is here, the COMPILE_TEST piece is > >> meaningless for all arches except ARM. If you care enough for COMPILE_TEST > >> (and I would), then you could make COMPILE_TEST useful on any arch by > >> removing the "depends on ARM" (the ARCH_SUNXI takes care of that) and by > >> having an alternate value for PHYS_OFFSET, like so: > >> > >> +#if defined(CONFIG_COMPILE_TEST) && !defined(PHYS_OFFSET) > >> +#define PHYS_OFFSET 0 > >> +#endif > >> > >> With those 2 changes, the driver builds for me on x86_64. > > > > I have considered this method. > > But it's so sick to put these code in dirver (for my own). I mean > > this is meaningless for the driver itself and make people confused. > > > > I grepped the driver/ code and I found many drivers writing Kconfig > > like this. For example: > > ARM && COMPILE_TEST > > MIPS && COMPILE_TEST > > PPC64 && COMPILE_TEST > > > > BTW, for my own, I do not care about COMPILE_TEST. > > There was a discussion about this in the v6 patch, but it petered out. > > I want to merge this driver, but I would very much prefer that this > compiles with COMPILE_TEST. So unless someone has a better solution, then > adding 'hack' that defines PHYS_OFFSET to 0 for COMPILE_TEST would be > required. If so, I will take the advice of Randy. > > Otherwise this driver looks good, so it is just this issue blocking it. > > Regards, > > Hans > > > > >> > >>> diff --git a/drivers/media/platform/sunxi/sun6i-csi/Kconfig > >>> b/drivers/media/platform/sunxi/sun6i-csi/Kconfig > >>> new file mode 100644 > >>> index 000..f80c965 > >>> --- /dev/null > >>> +++ b/drivers/media/platform/sunxi/sun6i-csi/Kconfig > >>> @@ -0,0 +1,10 @@ > >>> +config VIDEO_SUN6I_CSI > >>> + tristate "Allwinner V3s Camera Sensor Interface driver" > >>> + depends on ARM > >>> + depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API && HAS_DMA > >>> + depends on ARCH_SUNXI || COMPILE_TEST > >>> + select VIDEOBUF2_DMA_CONTIG > >>> + select REGMAP_MMIO > >>> + select V4L2_FWNODE > >>> + ---help--- > >>> +Support for the Allwinner Camera Sensor Interface Controller on V3s. > >> > >> thanks, > >> -- > >> ~Randy > > > > > > Thanks, > > Yong > > Thanks, Yong
[PATCH] media: intel-ipu3: cio2: Use SPDX license headers
Adopt SPDX license headers for ipu3 cio2 driver. Signed-off-by: Yong Zhi <yong@intel.com> --- drivers/media/pci/intel/ipu3/ipu3-cio2.c | 12 ++-- drivers/media/pci/intel/ipu3/ipu3-cio2.h | 14 ++ 2 files changed, 4 insertions(+), 22 deletions(-) diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c b/drivers/media/pci/intel/ipu3/ipu3-cio2.c index 6cb31f4b..af2da03a160f 100644 --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c @@ -1,14 +1,6 @@ +// SPDX-License-Identifier: GPL-2.0 /* - * Copyright (c) 2017 Intel Corporation. - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License version - * 2 as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. + * Copyright (C) 2017 Intel Corporation * * Based partially on Intel IPU4 driver written by * Sakari Ailus <sakari.ai...@linux.intel.com> diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.h b/drivers/media/pci/intel/ipu3/ipu3-cio2.h index 78a5799f08e7..240635be7a31 100644 --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.h +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.h @@ -1,15 +1,5 @@ -/* - * Copyright (c) 2017 Intel Corporation. - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License version - * 2 as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - */ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2017 Intel Corporation */ #ifndef __IPU3_CIO2_H #define __IPU3_CIO2_H -- 2.7.4
RE: [PATCH] media: intel-ipu3: cio2: Use SPDX license headers
Hi, Sakari, Sorry for the late response, somehow, I missed this email in my Inbox. > -Original Message- > From: linux-media-ow...@vger.kernel.org [mailto:linux-media- > ow...@vger.kernel.org] On Behalf Of Sakari Ailus > Sent: Wednesday, February 7, 2018 2:36 PM > To: Zhi, Yong <yong@intel.com> > Cc: linux-media@vger.kernel.org; Zheng, Jian Xu <jian.xu.zh...@intel.com>; > Mani, Rajmohan <rajmohan.m...@intel.com> > Subject: Re: [PATCH] media: intel-ipu3: cio2: Use SPDX license headers > > Hi Yong, > > Thanks for the patch. > > On Mon, Feb 05, 2018 at 08:19:53PM -0800, Yong Zhi wrote: > > Adopt SPDX license headers and update year to 2018. > > > > Signed-off-by: Yong Zhi <yong@intel.com> > > --- > > drivers/media/pci/intel/ipu3/ipu3-cio2.c | 12 ++-- > > drivers/media/pci/intel/ipu3/ipu3-cio2.h | 14 ++ > > 2 files changed, 4 insertions(+), 22 deletions(-) > > > > diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c > > b/drivers/media/pci/intel/ipu3/ipu3-cio2.c > > index 6cb..725973f 100644 > > --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c > > +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c > > @@ -1,14 +1,6 @@ > > +// SPDX-License-Identifier: GPL-2.0 > > /* > > - * Copyright (c) 2017 Intel Corporation. > > - * > > - * This program is free software; you can redistribute it and/or > > - * modify it under the terms of the GNU General Public License > > version > > - * 2 as published by the Free Software Foundation. > > - * > > - * This program is distributed in the hope that it will be useful, > > - * but WITHOUT ANY WARRANTY; without even the implied warranty of > > - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > - * GNU General Public License for more details. > > + * Copyright (C) 2018 Intel Corporation > > * > > * Based partially on Intel IPU4 driver written by > > * Sakari Ailus <sakari.ai...@linux.intel.com> diff --git > > a/drivers/media/pci/intel/ipu3/ipu3-cio2.h > > b/drivers/media/pci/intel/ipu3/ipu3-cio2.h > > index 78a5799..6a11051 100644 > > --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.h > > +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.h > > @@ -1,15 +1,5 @@ > > -/* > > - * Copyright (c) 2017 Intel Corporation. > > - * > > - * This program is free software; you can redistribute it and/or > > - * modify it under the terms of the GNU General Public License > > version > > - * 2 as published by the Free Software Foundation. > > - * > > - * This program is distributed in the hope that it will be useful, > > - * but WITHOUT ANY WARRANTY; without even the implied warranty of > > - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > > - * GNU General Public License for more details. > > - */ > > +/* SPDX-License-Identifier: GPL-2.0 */ > > +/* Copyright (C) 2018 Intel Corporation */ > > Should this be: > > /* Copyright (C) 2017 -- 2018 Intel Corporation */ > > ? > > Same for the one above. > Sure, will send an update. Thanks!! > > > > #ifndef __IPU3_CIO2_H > > #define __IPU3_CIO2_H > > -- > > 1.9.1 > > > > -- > Sakari Ailus > sakari.ai...@linux.intel.com
[PATCH] media: intel-ipu3: cio2: Disable and sync irq before stream off
This is to avoid pending interrupts to be handled during stream off, in which case, the ready buffer will be removed from buffer list, thus not all buffers can be returned to VB2 as expected. Disable CIO2 irq at cio2_hw_exit() so no new interrupts are generated. Signed-off-by: Yong Zhi <yong@intel.com> Signed-off-by: Tianshu Qiu <tian.shu@intel.com> --- drivers/media/pci/intel/ipu3/ipu3-cio2.c | 4 1 file changed, 4 insertions(+) diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c b/drivers/media/pci/intel/ipu3/ipu3-cio2.c index 6cb..b6b0cfe 100644 --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c @@ -526,6 +526,8 @@ static void cio2_hw_exit(struct cio2_device *cio2, struct cio2_queue *q) unsigned int i, maxloops = 1000; /* Disable CSI receiver and MIPI backend devices */ + writel(0, q->csi_rx_base + CIO2_REG_IRQCTRL_MASK); + writel(0, q->csi_rx_base + CIO2_REG_IRQCTRL_ENABLE); writel(0, q->csi_rx_base + CIO2_REG_CSIRX_ENABLE); writel(0, q->csi_rx_base + CIO2_REG_MIPIBE_ENABLE); @@ -1035,6 +1037,7 @@ static void cio2_vb2_stop_streaming(struct vb2_queue *vq) "failed to stop sensor streaming\n"); cio2_hw_exit(cio2, q); + synchronize_irq(cio2->pci_dev->irq); cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_ERROR); media_pipeline_stop(>vdev.entity); pm_runtime_put(>pci_dev->dev); @@ -1976,6 +1979,7 @@ static int __maybe_unused cio2_suspend(struct device *dev) /* Stop stream */ cio2_hw_exit(cio2, q); + synchronize_irq(pci_dev->irq); pm_runtime_force_suspend(dev); -- 1.9.1
RE: [PATCH] media: intel-ipu3: cio2: Synchronize irqs at stop_streaming
Hi, Sakari, > -Original Message- > From: Sakari Ailus [mailto:sakari.ai...@iki.fi] > Sent: Wednesday, February 7, 2018 11:38 PM > To: Zhi, Yong <yong@intel.com> > Cc: linux-media@vger.kernel.org; sakari.ai...@linux.intel.com; > tf...@chromium.org; Qiu, Tian Shu <tian.shu@intel.com>; Zheng, Jian > Xu <jian.xu.zh...@intel.com>; Mani, Rajmohan > <rajmohan.m...@intel.com> > Subject: Re: [PATCH] media: intel-ipu3: cio2: Synchronize irqs at > stop_streaming > > Hi Yong, > > On Wed, Feb 07, 2018 at 02:47:50PM -0800, Yong Zhi wrote: > > This is to avoid pending interrupts to be handled during stream off, > > in which case, the ready buffer will be removed from buffer list, thus > > not all buffers can be returned to VB2 as expected. Disable CIO2 irq > > at cio2_hw_exit() so no new interrupts are generated. > > > > Signed-off-by: Yong Zhi <yong@intel.com> > > Signed-off-by: Tianshu Qiu <tian.shu@intel.com> > > --- > > drivers/media/pci/intel/ipu3/ipu3-cio2.c | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c > > b/drivers/media/pci/intel/ipu3/ipu3-cio2.c > > index 725973f..8d75146 100644 > > --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c > > +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c > > @@ -518,6 +518,8 @@ static void cio2_hw_exit(struct cio2_device *cio2, > struct cio2_queue *q) > > unsigned int i, maxloops = 1000; > > > > /* Disable CSI receiver and MIPI backend devices */ > > + writel(0, q->csi_rx_base + CIO2_REG_IRQCTRL_MASK); > > + writel(0, q->csi_rx_base + CIO2_REG_IRQCTRL_ENABLE); > > writel(0, q->csi_rx_base + CIO2_REG_CSIRX_ENABLE); > > writel(0, q->csi_rx_base + CIO2_REG_MIPIBE_ENABLE); > > > > @@ -1027,6 +1029,7 @@ static void cio2_vb2_stop_streaming(struct > vb2_queue *vq) > > "failed to stop sensor streaming\n"); > > > > cio2_hw_exit(cio2, q); > > + synchronize_irq(cio2->pci_dev->irq); > > Shouldn't this be put in cio2_hw_exit(), which is called from multiple > locations? Presumably the same issue exists there, too. > Thanks for catching this, cio2_hw_exit() is used at two other places, and only one of them is subject to racing, so I will add synchronize_irq there in next update if it's OK. > > cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_ERROR); > > media_pipeline_stop(>vdev.entity); > > pm_runtime_put(>pci_dev->dev); > > -- > Regards, > > Sakari Ailus > e-mail: sakari.ai...@iki.fi
[PATCH] media: intel-ipu3: cio2: Synchronize irqs at stop_streaming
This is to avoid pending interrupts to be handled during stream off, in which case, the ready buffer will be removed from buffer list, thus not all buffers can be returned to VB2 as expected. Disable CIO2 irq at cio2_hw_exit() so no new interrupts are generated. Signed-off-by: Yong Zhi <yong@intel.com> Signed-off-by: Tianshu Qiu <tian.shu@intel.com> --- drivers/media/pci/intel/ipu3/ipu3-cio2.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c b/drivers/media/pci/intel/ipu3/ipu3-cio2.c index 725973f..8d75146 100644 --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c @@ -518,6 +518,8 @@ static void cio2_hw_exit(struct cio2_device *cio2, struct cio2_queue *q) unsigned int i, maxloops = 1000; /* Disable CSI receiver and MIPI backend devices */ + writel(0, q->csi_rx_base + CIO2_REG_IRQCTRL_MASK); + writel(0, q->csi_rx_base + CIO2_REG_IRQCTRL_ENABLE); writel(0, q->csi_rx_base + CIO2_REG_CSIRX_ENABLE); writel(0, q->csi_rx_base + CIO2_REG_MIPIBE_ENABLE); @@ -1027,6 +1029,7 @@ static void cio2_vb2_stop_streaming(struct vb2_queue *vq) "failed to stop sensor streaming\n"); cio2_hw_exit(cio2, q); + synchronize_irq(cio2->pci_dev->irq); cio2_vb2_return_all_buffers(q, VB2_BUF_STATE_ERROR); media_pipeline_stop(>vdev.entity); pm_runtime_put(>pci_dev->dev); -- 1.9.1
[PATCH] media: intel-ipu3: cio2: Use SPDX license headers
Adopt SPDX license headers and update year to 2018. Signed-off-by: Yong Zhi <yong@intel.com> --- drivers/media/pci/intel/ipu3/ipu3-cio2.c | 12 ++-- drivers/media/pci/intel/ipu3/ipu3-cio2.h | 14 ++ 2 files changed, 4 insertions(+), 22 deletions(-) diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.c b/drivers/media/pci/intel/ipu3/ipu3-cio2.c index 6cb..725973f 100644 --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.c +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.c @@ -1,14 +1,6 @@ +// SPDX-License-Identifier: GPL-2.0 /* - * Copyright (c) 2017 Intel Corporation. - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License version - * 2 as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. + * Copyright (C) 2018 Intel Corporation * * Based partially on Intel IPU4 driver written by * Sakari Ailus <sakari.ai...@linux.intel.com> diff --git a/drivers/media/pci/intel/ipu3/ipu3-cio2.h b/drivers/media/pci/intel/ipu3/ipu3-cio2.h index 78a5799..6a11051 100644 --- a/drivers/media/pci/intel/ipu3/ipu3-cio2.h +++ b/drivers/media/pci/intel/ipu3/ipu3-cio2.h @@ -1,15 +1,5 @@ -/* - * Copyright (c) 2017 Intel Corporation. - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License version - * 2 as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - */ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2018 Intel Corporation */ #ifndef __IPU3_CIO2_H #define __IPU3_CIO2_H -- 1.9.1
Re: [PATCH v7 2/2] media: V3s: Add support for Allwinner CSI.
Hi, On Mon, 29 Jan 2018 13:49:14 -0800 Randy Dunlap <rdun...@infradead.org> wrote: > On 01/29/2018 01:21 AM, Yong Deng wrote: > > Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2 > > interface and CSI1 is used for parallel interface. This is not > > documented in datasheet but by test and guess. > > > > This patch implement a v4l2 framework driver for it. > > > > Currently, the driver only support the parallel interface. MIPI-CSI2, > > ISP's support are not included in this patch. > > > > Tested-by: Maxime Ripard <maxime.rip...@free-electrons.com> > > Signed-off-by: Yong Deng <yong.d...@magewell.com> > > --- > > > A previous version (I think v6) had a build error with the use of > PHYS_OFFSET, so Kconfig was modified to depend on ARM and ARCH_SUNXI > (one of which seems to be overkill). As is here, the COMPILE_TEST piece is > meaningless for all arches except ARM. If you care enough for COMPILE_TEST > (and I would), then you could make COMPILE_TEST useful on any arch by > removing the "depends on ARM" (the ARCH_SUNXI takes care of that) and by > having an alternate value for PHYS_OFFSET, like so: > > +#if defined(CONFIG_COMPILE_TEST) && !defined(PHYS_OFFSET) > +#define PHYS_OFFSET 0 > +#endif > > With those 2 changes, the driver builds for me on x86_64. I have considered this method. But it's so sick to put these code in dirver (for my own). I mean this is meaningless for the driver itself and make people confused. I grepped the driver/ code and I found many drivers writing Kconfig like this. For example: ARM && COMPILE_TEST MIPS && COMPILE_TEST PPC64 && COMPILE_TEST BTW, for my own, I do not care about COMPILE_TEST. > > > diff --git a/drivers/media/platform/sunxi/sun6i-csi/Kconfig > > b/drivers/media/platform/sunxi/sun6i-csi/Kconfig > > new file mode 100644 > > index 000..f80c965 > > --- /dev/null > > +++ b/drivers/media/platform/sunxi/sun6i-csi/Kconfig > > @@ -0,0 +1,10 @@ > > +config VIDEO_SUN6I_CSI > > + tristate "Allwinner V3s Camera Sensor Interface driver" > > + depends on ARM > > + depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API && HAS_DMA > > + depends on ARCH_SUNXI || COMPILE_TEST > > + select VIDEOBUF2_DMA_CONTIG > > + select REGMAP_MMIO > > + select V4L2_FWNODE > > + ---help--- > > + Support for the Allwinner Camera Sensor Interface Controller on V3s. > > thanks, > -- > ~Randy Thanks, Yong
[PATCH v7 2/2] media: V3s: Add support for Allwinner CSI.
Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2 interface and CSI1 is used for parallel interface. This is not documented in datasheet but by test and guess. This patch implement a v4l2 framework driver for it. Currently, the driver only support the parallel interface. MIPI-CSI2, ISP's support are not included in this patch. Tested-by: Maxime Ripard <maxime.rip...@free-electrons.com> Signed-off-by: Yong Deng <yong.d...@magewell.com> --- MAINTAINERS| 8 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile| 2 + drivers/media/platform/sunxi/sun6i-csi/Kconfig | 10 + drivers/media/platform/sunxi/sun6i-csi/Makefile| 3 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 908 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 143 .../media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h | 196 + .../media/platform/sunxi/sun6i-csi/sun6i_video.c | 753 + .../media/platform/sunxi/sun6i-csi/sun6i_video.h | 53 ++ 10 files changed, 2077 insertions(+) create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Kconfig create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Makefile create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h diff --git a/MAINTAINERS b/MAINTAINERS index aee793b..1c9f296 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3774,6 +3774,14 @@ M: Jaya Kumar <jayakumar.a...@gmail.com> S: Maintained F: sound/pci/cs5535audio/ +CSI DRIVERS FOR ALLWINNER V3s +M: Yong Deng <yong.d...@magewell.com> +L: linux-media@vger.kernel.org +T: git git://linuxtv.org/media_tree.git +S: Maintained +F: drivers/media/platform/sunxi/sun6i-csi/ +F: Documentation/devicetree/bindings/media/sun6i-csi.txt + CW1200 WLAN driver M: Solomon Peachy <pi...@shaftnet.org> S: Maintained diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index 614fbef..82c9e00 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -150,6 +150,7 @@ source "drivers/media/platform/am437x/Kconfig" source "drivers/media/platform/xilinx/Kconfig" source "drivers/media/platform/rcar-vin/Kconfig" source "drivers/media/platform/atmel/Kconfig" +source "drivers/media/platform/sunxi/sun6i-csi/Kconfig" config VIDEO_TI_CAL tristate "TI CAL (Camera Adaptation Layer) driver" diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile index 7f30804..ace8628 100644 --- a/drivers/media/platform/Makefile +++ b/drivers/media/platform/Makefile @@ -95,3 +95,5 @@ obj-$(CONFIG_VIDEO_QCOM_CAMSS)+= qcom/camss-8x16/ obj-$(CONFIG_VIDEO_QCOM_VENUS) += qcom/venus/ obj-y += meson/ + +obj-$(CONFIG_VIDEO_SUN6I_CSI) += sunxi/sun6i-csi/ diff --git a/drivers/media/platform/sunxi/sun6i-csi/Kconfig b/drivers/media/platform/sunxi/sun6i-csi/Kconfig new file mode 100644 index 000..f80c965 --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/Kconfig @@ -0,0 +1,10 @@ +config VIDEO_SUN6I_CSI + tristate "Allwinner V3s Camera Sensor Interface driver" + depends on ARM + depends on VIDEO_V4L2 && COMMON_CLK && VIDEO_V4L2_SUBDEV_API && HAS_DMA + depends on ARCH_SUNXI || COMPILE_TEST + select VIDEOBUF2_DMA_CONTIG + select REGMAP_MMIO + select V4L2_FWNODE + ---help--- + Support for the Allwinner Camera Sensor Interface Controller on V3s. diff --git a/drivers/media/platform/sunxi/sun6i-csi/Makefile b/drivers/media/platform/sunxi/sun6i-csi/Makefile new file mode 100644 index 000..213cb6b --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/Makefile @@ -0,0 +1,3 @@ +sun6i-csi-y += sun6i_video.o sun6i_csi.o + +obj-$(CONFIG_VIDEO_SUN6I_CSI) += sun6i-csi.o diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c new file mode 100644 index 000..9c341f0 --- /dev/null +++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c @@ -0,0 +1,908 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2011-2018 Magewell Electronics Co., Ltd. (Nanjing) + * All rights reserved. + * Author: Yong Deng <yong.d...@magewell.com> + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "sun6i_csi.h&q
[PATCH v7 1/2] dt-bindings: media: Add Allwinner V3s Camera Sensor Interface (CSI)
Add binding documentation for Allwinner V3s CSI.
Reviewed-by: Rob Herring <r...@kernel.org>
Signed-off-by: Yong Deng <yong.d...@magewell.com>
---
.../devicetree/bindings/media/sun6i-csi.txt| 59 ++
1 file changed, 59 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/sun6i-csi.txt
diff --git a/Documentation/devicetree/bindings/media/sun6i-csi.txt
b/Documentation/devicetree/bindings/media/sun6i-csi.txt
new file mode 100644
index 000..2ff47a9
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/sun6i-csi.txt
@@ -0,0 +1,59 @@
+Allwinner V3s Camera Sensor Interface
+-
+
+Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2
+interface and CSI1 is used for parallel interface.
+
+Required properties:
+ - compatible: value must be "allwinner,sun8i-v3s-csi"
+ - reg: base address and size of the memory-mapped region.
+ - interrupts: interrupt associated to this IP
+ - clocks: phandles to the clocks feeding the CSI
+* bus: the CSI interface clock
+* mod: the CSI module clock
+* ram: the CSI DRAM clock
+ - clock-names: the clock names mentioned above
+ - resets: phandles to the reset line driving the CSI
+
+Each CSI node should contain one 'port' child node with one child 'endpoint'
+node, according to the bindings defined in
+Documentation/devicetree/bindings/media/video-interfaces.txt. As mentioned
+above, the endpoint's bus type should be MIPI CSI-2 for CSI0 and parallel or
+Bt656 for CSI1.
+
+Endpoint node properties for CSI1
+-
+
+- remote-endpoint : (required) a phandle to the bus receiver's endpoint
+ node
+- bus-width: : (required) must be 8, 10, 12 or 16
+- pclk-sample : (optional) (default: sample on falling edge)
+- hsync-active : (only required for parallel)
+- vsync-active : (only required for parallel)
+
+Example:
+
+csi1: csi@1cb4000 {
+ compatible = "allwinner,sun8i-v3s-csi";
+ reg = <0x01cb4000 0x1000>;
+ interrupts = ;
+ clocks = < CLK_BUS_CSI>,
+< CLK_CSI1_SCLK>,
+< CLK_DRAM_CSI>;
+ clock-names = "bus", "mod", "ram";
+ resets = < RST_BUS_CSI>;
+
+ port {
+ /* Parallel bus endpoint */
+ csi1_ep: endpoint {
+ remote-endpoint = <_ep>;
+ bus-width = <16>;
+
+ /* If hsync-active/vsync-active are missing,
+ embedded BT.656 sync is used */
+ hsync-active = <0>; /* Active low */
+ vsync-active = <0>; /* Active low */
+ pclk-sample = <1>; /* Rising */
+ };
+ };
+};
--
1.8.3.1
[PATCH v7 0/2] Initial Allwinner V3s CSI Support
(Not Supported) Standard Controls: 0 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported) test Composing: OK (Not Supported) test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK Test input 0: Streaming ioctls: test read/write: OK (Not Supported) test MMAP: OK test USERPTR: OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device Stream using all formats: test MMAP for Format HM12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV21, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format YU12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format YV12, Frame Size 1280x720: Stride 1920, Field None: OK test MMAP for Format NV16, Frame Size 1280x720: Stride 2560, Field None: OK test MMAP for Format NV61, Frame Size 1280x720: Stride 2560, Field None: OK test MMAP for Format 422P, Frame Size 1280x720: Stride 2560, Field None: OK Total: 54, Succeeded: 54, Failed: 0, Warnings: 0 Yong Deng (2): dt-bindings: media: Add Allwinner V3s Camera Sensor Interface (CSI) media: V3s: Add support for Allwinner CSI. .../devicetree/bindings/media/sun6i-csi.txt| 59 ++ MAINTAINERS| 8 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile| 2 + drivers/media/platform/sunxi/sun6i-csi/Kconfig | 10 + drivers/media/platform/sunxi/sun6i-csi/Makefile| 3 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c | 908 + drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h | 143 .../media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h | 196 + .../media/platform/sunxi/sun6i-csi/sun6i_video.c | 753 + .../media/platform/sunxi/sun6i-csi/sun6i_video.h | 53 ++ 11 files changed, 2136 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/sun6i-csi.txt create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Kconfig create mode 100644 drivers/media/platform/sunxi/sun6i-csi/Makefile create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_reg.h create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c create mode 100644 drivers/media/platform/sunxi/sun6i-csi/sun6i_video.h -- 1.8.3.1
Re: [PATCH v6 2/2] media: V3s: Add support for Allwinner CSI.
Hi,
On Sat, 27 Jan 2018 17:14:26 +0100
Linus Walleij <linus.wall...@linaro.org> wrote:
> On Tue, Jan 23, 2018 at 9:18 AM, Yong Deng <yong.d...@magewell.com> wrote:
>
> > Allwinner V3s SoC features two CSI module. CSI0 is used for MIPI CSI-2
> > interface and CSI1 is used for parallel interface. This is not
> > documented in datasheet but by test and guess.
> >
> > This patch implement a v4l2 framework driver for it.
> >
> > Currently, the driver only support the parallel interface. MIPI-CSI2,
> > ISP's support are not included in this patch.
> >
> > Tested-by: Maxime Ripard <maxime.rip...@free-electrons.com>
> > Signed-off-by: Yong Deng <yong.d...@magewell.com>
>
> This is cool stuff :)
>
> > +void sun6i_csi_update_buf_addr(struct sun6i_csi *csi, dma_addr_t addr)
> > +{
> > + struct sun6i_csi_dev *sdev = sun6i_csi_to_dev(csi);
> > + /* transform physical address to bus address */
> > + dma_addr_t bus_addr = addr - PHYS_OFFSET;
>
> I am sorry if this is an unjustified drive-by comment. Maybe you
> have already investigate other ways to do this.
>
> Accessing PHYS_OFFSET directly seems unintuitive
> and not good practice.
>
> But normally an dma_addr_t only comes from some
> function inside such as:
> dma_alloc_coherent() for a contigous buffer which is coherent
> in physical memory, or from some buffer <= 64KB that
> is switching ownership between device and CPU explicitly
> with dma_map* or so. Did you check with
> Documentation/DMA-API.txt?
The dma_addr_t here comes from v4l2 vb2 and it's already 'mapped'.
Maybe the dma-mapping code of sunXi does not do conversion between
device and CPU. I'm not familiar with this.
Maxime, do you have any idea about this?
Can we get bus address directly from dma_alloc_coherent or dma_map*
at the system layer but not doing the conversion per driver?
Yong
Re: [linux-sunxi] Re: [PATCH v6 2/2] media: V3s: Add support for Allwinner CSI.
Hi Maxime, On Fri, 26 Jan 2018 09:10:00 +0100 Maxime Ripard <maxime.rip...@free-electrons.com> wrote: > On Fri, Jan 26, 2018 at 11:00:41AM +0800, Yong wrote: > > Hi Maxime, > > > > On Fri, 26 Jan 2018 09:46:58 +0800 > > Yong <yong.d...@magewell.com> wrote: > > > > > Hi Maxime, > > > > > > Do you have any experience in solving this problem? > > > It seems the PHYS_OFFSET maybe undeclared when the ARCH is not arm. > > > > Got it. > > Should I add 'depends on ARM' in Kconfig? > > Yes, or even better a depends on MACH_SUNXI :) Do you mean ARCH_SUNXI? ARCH_SUNXI is alreay there. In the early version, my Kconfig is like this: depends on ARCH_SUNXI But Hans suggest me to change this to: depends on ARCH_SUNXI || COMPILE_TEST to allow this driver to be compiled on e.g. Intel for compile testing. Should we get rid of COMPILE_TEST? Yong
Re: [PATCH v6 2/2] media: V3s: Add support for Allwinner CSI.
Hi Maxime,
On Fri, 26 Jan 2018 09:46:58 +0800
Yong <yong.d...@magewell.com> wrote:
> Hi Maxime,
>
> Do you have any experience in solving this problem?
> It seems the PHYS_OFFSET maybe undeclared when the ARCH is not arm.
Got it.
Should I add 'depends on ARM' in Kconfig?
>
> On Fri, 26 Jan 2018 08:04:18 +0800
> kbuild test robot <l...@intel.com> wrote:
>
> > Hi Yong,
> >
> > I love your patch! Yet something to improve:
> >
> > [auto build test ERROR on linuxtv-media/master]
> > [also build test ERROR on v4.15-rc9 next-20180119]
> > [if your patch is applied to the wrong git tree, please drop us a note to
> > help improve the system]
> >
> > url:
> > https://github.com/0day-ci/linux/commits/Yong-Deng/dt-bindings-media-Add-Allwinner-V3s-Camera-Sensor-Interface-CSI/20180126-054511
> > base: git://linuxtv.org/media_tree.git master
> > config: i386-allmodconfig (attached as .config)
> > compiler: gcc-7 (Debian 7.2.0-12) 7.2.1 20171025
> > reproduce:
> > # save the attached .config to linux build tree
> > make ARCH=i386
> >
> > All errors (new ones prefixed by >>):
> >
> >drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c: In function
> > 'sun6i_csi_update_buf_addr':
> > >> drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c:567:31: error:
> > >> 'PHYS_OFFSET' undeclared (first use in this function); did you mean
> > >> 'PAGE_OFFSET'?
> > dma_addr_t bus_addr = addr - PHYS_OFFSET;
> > ^~~
> > PAGE_OFFSET
> >drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c:567:31: note: each
> > undeclared identifier is reported only once for each function it appears in
> >
> > vim +567 drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
> >
> >562
> >563 void sun6i_csi_update_buf_addr(struct sun6i_csi *csi,
> > dma_addr_t addr)
> >564 {
> >565 struct sun6i_csi_dev *sdev = sun6i_csi_to_dev(csi);
> >566 /* transform physical address to bus address */
> > > 567 dma_addr_t bus_addr = addr - PHYS_OFFSET;
> >568
> >569 regmap_write(sdev->regmap, CSI_CH_F0_BUFA_REG,
> >570 (bus_addr + sdev->planar_offset[0]) >> 2);
> >571 if (sdev->planar_offset[1] != -1)
> >572 regmap_write(sdev->regmap, CSI_CH_F1_BUFA_REG,
> >573 (bus_addr +
> > sdev->planar_offset[1]) >> 2);
> >574 if (sdev->planar_offset[2] != -1)
> >575 regmap_write(sdev->regmap, CSI_CH_F2_BUFA_REG,
> >576 (bus_addr +
> > sdev->planar_offset[2]) >> 2);
> >577 }
> >578
> >
> > ---
> > 0-DAY kernel test infrastructureOpen Source Technology
> > Center
> > https://lists.01.org/pipermail/kbuild-all Intel
> > Corporation
>
>
> Thanks,
> Yong
Thanks,
Yong
