blues-fun opened a new pull request, #22004:
URL: https://github.com/apache/nuttx/pull/22004

   ## Summary
   
   * Why change is necessary: the sim platform has no JPEG codec, so video 
pipeline code that expects a V4L2 mem2mem JPEG decoder cannot be exercised 
without hardware. A simulated JPEG decoder -- the JPEG counterpart of the 
existing sim H.264 decoder -- makes that possible and is usable from the 
upstream `apps/system/nxcodec` tool.
   * What functional part of the code is being changed: `arch/sim` only: new 
`arch/sim/src/sim/sim_jpeg_decoder.c` (V4L2 m2m codec ops), `sim_jpegdec.c` 
(decode core), `sim_jpegdec.h`, the `sim_initialize.c` / `sim_internal.h` 
hooks, `arch/sim/Kconfig`, both build systems, and the sim board documentation 
page.
   * How the change exactly work:
     * `CONFIG_SIM_JPEG_DECODER` registers a V4L2 memory-to-memory codec at 
`CONFIG_SIM_JPEG_DECODER_DEV_PATH` (default `/dev/video3`): the output queue 
takes JPEG/MJPEG frames, the capture queue yields decoded frames in the 
requested format.
     * The decode core uses the standard libjpeg API (`jpeglib.h`) and links 
the host libjpeg with `-ljpeg`, the same way the sim links its other host 
libraries (`-lasound`, `-lv4l2`, `-lX11`), so no additional package is required 
on the host beyond libjpeg itself.
     * YUV420P and NV12 output goes through the raw data interface 
(`jpeg_read_raw_data`) and accepts 4:2:0 and grayscale input; grayscale chroma 
is filled with 0x80. Any other input sampling is rejected with `-EINVAL` 
instead of overrunning the capture buffer.
     * RGB24, BGR24, XRGB32 and XBGR32 output goes through 
`jpeg_read_scanlines` and accepts any input sampling; the padding byte of the 
32-bit formats is 0xff.
     * libjpeg errors recover through `setjmp()` / `longjmp()` and return 
`-EIO`; the decoder handle stays usable afterwards.
   
   Why this decoder is worth having: JPEG/MJPEG is what most camera sensors and 
UVC devices actually deliver, so a mem2mem JPEG decoder is the workhorse of the 
capture path. With one on sim, the whole video pipeline (capture -> m2m decode 
-> output), the v4l2m2m framework itself and everything built on it (nxcodec, 
nxcamera snapshot and record flows) can be developed, regression-tested and run 
in CI without hardware on the desk, against a deterministic decoder, while the 
same userspace code runs unchanged on real devices.
   
   Commits (2):
   
   * arch/sim: add JPEG M2M decoder driver using libjpeg
   * Documentation/sim: describe the JPEG m2m decoder
   
   ## Impact
   
   * Is new feature added? Is existing feature changed? YES -- a new opt-in sim 
driver; nothing else changes.
   * Impact on user (will user need to adapt to change)? NO.
   * Impact on build (will build process change)? NO -- 
`CONFIG_SIM_JPEG_DECODER` is default off; Make and CMake are both wired.
   * Impact on hardware (will arch(s) / board(s) / driver(s) change)? NO -- 
arch/sim only.
   * Impact on documentation (is update required / provided)? YES -- provided: 
the sim board page documents the new device next to the H.264 m2m decoder.
   * Impact on security (any sort of implications)? NO.
   * Impact on compatibility (backward/forward/interoperability)? NO.
   
   ## Testing
   
   I confirm that changes are verified on local setup and works as intended:
   
   * Build Host(s): Linux x86_64, gcc 13.4.
   * Target(s): sim (flat, 64-bit) with `CONFIG_SIM_JPEG_DECODER`, 
`CONFIG_VIDEO_STREAM`, `CONFIG_SYSTEM_NXCODEC` and hostfs, built on master 
8a282a51d42 with this branch. Build and link clean; checkpatch passes.
   
   Decode of a 320x240 JPEG through nxcodec on `/dev/video3`:
   
       nsh> nxcodec -d /dev/video3 -s 320x240 -f JPEG -i /host/color420.jpg -f 
YU12 -o /host/out_c420_yuv420.yuv
       nxcodec device name: /dev/video3
       nxcodec size: 320x240
       nxcodec input  file: /host/color420.jpg, format JPEG - 1195724874
       nxcodec output file: /host/out_c420_yuv420.yuv, format YU12 - 842093913
       nxcodec is single-planar
       nxcodec output VIDIOC_S_FMT
       nxcodec set output format DONE
       nxcodec capture VIDIOC_S_FMT
       nxcodec set capture format DONE
       nxcodec init DONE.
       nxcodec started.
       nxcodec stop DONE.
   
   The output file sizes match the expected frame sizes exactly: YUV420 and
   NV12 115200 bytes, RGB24 and BGR24 230400, XRGB32 and XBGR32 307200, and
   an odd-sized 333x245 YUV420 frame 122667 = 333*245 + 2*167*123 bytes.
   
   All six capture formats were decoded for 4:2:0 color, grayscale, 4:4:4
   color and odd-sized (333x245) input JPEGs and cross-checked on the host
   against the same libjpeg and against an independent decoder (Pillow):
   
   * RGB24/BGR24/XRGB32/XBGR32 are bit-identical to Pillow's decode of the
     same JPEG (max absolute difference 0), for every input tested.
   * The NV12 output is bit-identical to the YUV420 output of the same
     frame on all three planes.
   * The YUV420 Y plane matches the pre-encode image within JPEG
     quantization at quality 95 (max absolute difference 8, mean 0.26).
   * Grayscale input produces chroma planes filled with 0x80; 4:4:4 input
     decodes on the RGB paths and is rejected with -EINVAL on the packed
     YUV paths.
   * Corrupt input returns -EIO through the error path without crashing,
     and the decoder handle can be reused afterwards.
   
   The v4l2m2m JPEG decode flow was additionally verified on BES platform
   (Cortex-M55) real hardware.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to