Re: [PATCH v2 0/4] media: meson: Add support for the Amlogic GE2D Accelerator Unit
Hi Linus, On 05/11/2020 08:53, Linus Walleij wrote: > Hi Neil, > > this is just a drive-by question and I'm looping in Todd in the hopes for > a discussion or clarification. > > On Fri, Oct 30, 2020 at 3:37 PM Neil Armstrong > wrote: > >> The GE2D is a 2D accelerator with various features like configurable blitter >> with alpha blending, frame rotation, scaling, format conversion and >> colorspace >> conversion. >> >> The driver implements a Memory2Memory VB2 V4L2 streaming device permitting: >> - 0, 90, 180, 270deg rotation >> - horizontal/vertical flipping >> - source cropping >> - destination compositing >> - 32bit/24bit/16bit format conversion >> >> This adds the support for the GE2D version found in the AXG SoCs Family. > > We are starting to see a bunch of these really nicely abstracted blitters > and other 2D-accelerators now. The actual blitting functionality is limited to non-alpha blitting since no standard CID are available for this, but we could totally try to find common CIDs to describe the possible Alpha Blending properties for these 2D accelerators. > > Is stuff like Android going to pick up and use this to blit and blend > generic buffers? I'm not sure this is Google's plan right now, but maybe it should be doable. > > Or is this in essence a camera and/or video out accelerator thing? No it's really a blitter & scaler, rotate & format converter, like the samsung and rockhip drivers, and somehow the allwinner rotate driver. Amlogic mainly uses it to copy frames beeing displayed for encoding, or to convert frames from the HDMI RX on their TV SoCs. > > The placement of this driver in drivers/media makes me think that > it is for cameras or video output, but the functionality is actually > quite generic. It's really a memory-2-memory driver, like the video codecs, it's a separate class than the camera & video output drivers. > > I've been half-guessing that userspace like Android actually mostly > use GPUs to composit their graphics, but IIUC this can sometimes be > used for 2D compositing, and when used will often be quicker and/or > more energy efficient than using a GPU for the same task. Well drm-hwcomposer can already use the DRM universal planes and the virtual writeback connector when available for compositing. But this kind of driver can be really useful for display rotation for example when the DRM driver doesn't support it. Honestly I don't understand the Android graphics stack enough to formally answer this question, but if it can be used, this kind of driver is much faster and much simpler than a GPU for simple blitting and rotation. And since they support DMA-BUF, they can totally be used in a modern graphics pipeline. Maybe someone could answer ? Maybe drm-hwcomposer could be extended for that ? I know there is an issue opened in GloDroid for that: https://github.com/GloDroid/glodroid_manifest/issues/66 For the record, I use this driver to accelerate the LVGL flush to display on the AXG SoCs lacking a GPU, this by using DMA-BUF and DRM atomic modesetting with the DRM LVGL display driver I submitted (and tweaked for V4L2 M2M): https://github.com/lvgl/lv_drivers/blob/master/display/drm.c Neil > > Yours, > Linus Walleij >
Re: [PATCH v2 0/4] media: meson: Add support for the Amlogic GE2D Accelerator Unit
Hi Neil, this is just a drive-by question and I'm looping in Todd in the hopes for a discussion or clarification. On Fri, Oct 30, 2020 at 3:37 PM Neil Armstrong wrote: > The GE2D is a 2D accelerator with various features like configurable blitter > with alpha blending, frame rotation, scaling, format conversion and colorspace > conversion. > > The driver implements a Memory2Memory VB2 V4L2 streaming device permitting: > - 0, 90, 180, 270deg rotation > - horizontal/vertical flipping > - source cropping > - destination compositing > - 32bit/24bit/16bit format conversion > > This adds the support for the GE2D version found in the AXG SoCs Family. We are starting to see a bunch of these really nicely abstracted blitters and other 2D-accelerators now. Is stuff like Android going to pick up and use this to blit and blend generic buffers? Or is this in essence a camera and/or video out accelerator thing? The placement of this driver in drivers/media makes me think that it is for cameras or video output, but the functionality is actually quite generic. I've been half-guessing that userspace like Android actually mostly use GPUs to composit their graphics, but IIUC this can sometimes be used for 2D compositing, and when used will often be quicker and/or more energy efficient than using a GPU for the same task. Yours, Linus Walleij
[PATCH v2 0/4] media: meson: Add support for the Amlogic GE2D Accelerator Unit
The GE2D is a 2D accelerator with various features like configurable blitter with alpha blending, frame rotation, scaling, format conversion and colorspace conversion. The driver implements a Memory2Memory VB2 V4L2 streaming device permitting: - 0, 90, 180, 270deg rotation - horizontal/vertical flipping - source cropping - destination compositing - 32bit/24bit/16bit format conversion This adds the support for the GE2D version found in the AXG SoCs Family. The missing features are: - Source scaling - Colorspace conversion - Advanced alpha blending & blitting options Dependencies: - Patches 1-3: None - Patch 4: https://lkml.kernel.org/r/20200915124553.8056-1-narmstr...@baylibre.com (applied for 5.11) Changes since v1: - Rebased on v5.10-rc1 / # v4l2-compliance -s v4l2-compliance SHA: ea16a7ef13a902793a5c2626b0cefc4d956147f3, 64 bits, 64-bit time_t Compliance test for meson-ge2d device /dev/video0: Driver Info: Driver name : meson-ge2d Card type: meson-ge2d Bus info : platform:meson-ge2d Driver version : 5.9.0 Capabilities : 0x84208000 Video Memory-to-Memory Streaming Extended Pix Format Device Capabilities Device Caps : 0x04208000 Video Memory-to-Memory Streaming Extended Pix Format Required ioctls: test VIDIOC_QUERYCAP: OK Allow for multiple opens: test second /dev/video0 open: OK test VIDIOC_QUERYCAP: OK test VIDIOC_G/S_PRIORITY: OK test for unlimited opens: OK test invalid ioctls: OK Debug ioctls: test VIDIOC_DBG_G/S_REGISTER: OK test VIDIOC_LOG_STATUS: OK (Not Supported) Input ioctls: test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported) test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported) Inputs: 0 Audio Inputs: 0 Tuners: 0 Output ioctls: test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported) Outputs: 0 Audio Outputs: 0 Modulators: 0 Input/Output configuration ioctls: test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported) Control ioctls: test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK test VIDIOC_G/S_CTRL: OK test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 4 Private Controls: 0 Format ioctls: test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK test VIDIOC_TRY_FMT: OK test VIDIOC_S_FMT: OK test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK test Composing: OK test Scaling: OK (Not Supported) Codec ioctls: test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported) Buffer ioctls: test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK test Requests: OK (Not Supported) Test input 0: Streaming ioctls: test read/write: OK (Not Supported) test blocking wait: OK Video Capture: Captured 58 buffers test MMAP (no poll): OK Video Capture: Captured 58 buffers test MMAP (select): OK Video Capture: Captured 58 buffers test MMAP (epoll): OK test USERPTR (no poll): OK (Not Supported) test USERPTR (select): OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device Total for meson-ge2d device /dev/video0: 52, Succeeded: 52, Failed: 0, Warnings: 0 Neil Armstrong (4): dt-bindings: media: Add bindings for the Amlogic GE2D Accelerator Unit media: meson: Add M2M driver for the Amlogic GE2D Accelerator Unit MAINTAINERS: Add myself as maintainer of the Amlogic GE2D driver arm64: dts: meson-axg: add GE2D node .../bindings/media/amlogic,axg-ge2d.yaml | 47 + MAINTAINERS |9 + arch/arm64/boot/dts/amlogic/meson-axg.dtsi|9 + drivers/media/platform/Kconfig| 13 + drivers/media/platform/Makefile |2 +
