This is the architecture overview we put together for Qualcomm MSM camera
support in linux-media tree. Your comments are very much appreciated!


This is the video4linux driver for Qualcomm MSM (Mobile Station Modem)

The driver supports video and camera functionality on Qualcomm MSM SoC.
It supports camera sensors provided by OEMs with parallel/MIPI
interfaces, and operates in continuous streaming, snapshot, or video
recording modes.

Hardware description

MSM camera hardware contains the following components:
1. One or more camera sensors controlled via I2C bus, and data outputs
on AXI or MIPI bus.
2. Video Front End (VFE) core is an image signal processing hardware
pipeline that takes the sensor output, does the necessary processing,
and outputs to a buffer for V4L2 application. VFE is clocked by PCLK
(pixel clock) from the sensor.

Software description

The driver encapsulates the low-level hardware management and aligns
the interface to V4L2 driver framework. There is also a user space
camera service daemon which handles events from driver and changes
settings accordingly.

During boot-up, the sensor is probed for and if the probe succeeds
/dev/video0 and /dev/msm_camera/config0 device nodes are created. The
/dev/video0 node is used for video buffer allocation in the kernel and for
receiving V4L2 ioctls for controlling the camera hardware (VFE, sensors).
The /dev/msm_camera/config0 node is used for sending commands and other
statistics available from the hardware to the camera service daemon. Note
that if more than one camera sensor is detected, there will be /dev/video1
and /dev/msm_camera/config1 device nodes created as well.


For MSM camera IC, significant portion of image processing and optimization
codes are proprietary, so they cannot sit in kernel space. This plays an
important role when making design decisions.

Our design is to have a light-weighted kernel driver, and put the
proprietary code in a user space daemon. The daemon polls on events
from /dev/msm_camera/config0 device in the form of v4l2_event. The events
could either be asynchronous (generated by the hardware), or synchronous
(control command from the application). Based on the events it receives,
the daemon will send appropriate control commands to the hardware.

   +-------------+        +----------------+
   | Application |        | Service Daemon |
   +-------------+        +----------------+     User Space
   +--------------------------------------+      Kernel Space
   |                V4L2                  |
   +---------+     +--------+     +-------+
   | VFE/ISP |     | Sensor |     | Flash |
   +---------+     +--------+     +-------+

Power Management

None at this point.


Mutex is used for synchronization of threads accessing the driver
simultaneously. Between hardware interrupt handler and threads,
we use spinlock to make sure locking is done properly.




The driver uses V4L2 API for user kernel interaction. Refer to

Between camera service daemon and the driver, we have customized IOCTL
commands associated with /dev/msm_camera/config0 node, that controls MSM
camera hardware. The list of IOCTLs are (declarations can be found in

        Get the basic information such as name, flash support for the
        detected sensor.
        Get or set sensor configurations: fps, line_pf, pixels_pl,
        exposure and gain, etc. The setting is stored in sensor_cfg_data
        Change settings of different components of VFE hardware.
        Notify the driver that the ctrl command is finished.
        Notify the driver that the service daemon has done processing the
        stats buffer, and is returning it to the driver.
        Configure AXI bus parameters (frame buffer addresses, offsets) to
        the VFE hardware.

Driver parameters


Config options

MSM_CAMERA in drivers/media/video/msm/Kconfig file


PMIC, I2C, Clock, GPIO

User space utilities

A daemon process in the user space called mm-qcamera-daemon is polling
on events from the driver. This daemon is required in order for the V4L2
client application to run, and it's started by the init script.


To do

1. Eliminate creation of /dev/msm_camera/config0 by routing private
ioctls through /dev/video0.
2. Create sub devices for sensor and VFE.

Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.

To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reply via email to