Hi everyone,

In Collabora, and as part of our kernelci work, we are doing
research on kernel functional testing with kernelci.

For those new to kernelci, see 
https://github.com/kernelci/kernelci-doc/wiki/KernelCI 
and https://kernelci.org/.

The goal is to lay down the infrastructure required to make
automated test coverage an integral part of our feature
and bugfix development process.

So, as a first attempt, we've decided to extend kernelci test
v4l2 plan support, leading the way to extending
other subsystems' test plans.

Currently, kernelci looks for a list of branches every hour and
see if anything changed. For any branch that has changed, it triggers
builds, boots, tests and reports for each branch that had some changes
since last time it ran.

For this pilot, we've decided to target just a few devices:
qemu with vivid, rk3399-gru-kevin and rk3288-veyron-jaq
with uvc.

We'd like to get some early feedback on this, so we are sending
an example of how a kernelci report would look like, to trigger
some discussion around
the direction this should take.

Thanks!

===
Test results for:
  Tree:    gtucker
  Branch:  kernelci-media
  Kernel:  gtucker-kernelci-media-002-2-gaa27eb0392c7
  URL:     https://gitlab.collabora.com/gtucker/linux.git
  Commit:  aa27eb0392c70adec713e911a9b5267a1d853624
  Test plans: v4l2

Summary
-------
3 test groups results

1  | v4l2       | rk3399-gru-kevin       | arm64 |  49 total:  17 PASS   4 FAIL 
 28 SKIP
2  | v4l2       | rk3288-veyron-jaq      | arm   |  49 total:  17 PASS   4 FAIL 
 28 SKIP
3  | v4l2       | qemu                   | arm64 | 168 total: 102 PASS   0 FAIL 
 66 SKIP


Tests
-----

1  | v4l2       | rk3399-gru-kevin       | arm64 |  49 total:  17 PASS   4 FAIL 
 28 SKIP

  Config:      defconfig
  Lab Name:    lab-collabora-dev
  Date:        2018-10-18 18:48:52.426000
  TXT log:     
http://staging-storage.kernelci.org/gtucker/kernelci-media/gtucker-kernelci-media-002-2-gaa27eb0392c7/arm64/defconfig/lab-collabora-dev
/v4l2-rk3399-gru-kevin.txt
  HTML log:    
http://staging-storage.kernelci.org/gtucker/kernelci-media/gtucker-kernelci-media-002-2-gaa27eb0392c7/arm64/defconfig/lab-collabora-dev
/v4l2-rk3399-gru-kevin.html
  Rootfs:      
http://staging-storage.kernelci.org/images/rootfs/debian/stretchv4l2/20181017.1//arm64/rootfs.cpio.gz
  Test Git:    git://linuxtv.org/v4l-utils.git
  Test Commit: e889b0d56757b9a74eb8c4c8cc2ebd5b18e228b2


  Test cases:

    * MMAP: FAIL
    * blocking-wait: PASS
    * read/write: SKIP
    * read/write: SKIP
    * read/write: SKIP
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * VIDIOC_G_SLICED_VBI_CAP: SKIP
    * VIDIOC_S_FMT: PASS
    * VIDIOC_TRY_FMT: PASS
    * VIDIOC_G_FMT: PASS
    * VIDIOC_G_FBUF: SKIP
    * VIDIOC_G/S_PARM: FAIL
    * VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: PASS
    * VIDIOC_G/S_JPEGCOMP: SKIP
    * VIDIOC_UNSUBSCRIBE_EVENT/DQEVENT: PASS
    * VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
    * VIDIOC_G/S_CTRL: PASS
    * VIDIOC_QUERYCTRL: PASS
    * VIDIOC_QUERY_EXT_CTRL/QUERYMENU: FAIL
    * VIDIOC_G/S_EDID: SKIP
    * VIDIOC_DV_TIMINGS_CAP: SKIP
    * VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: SKIP
    * VIDIOC_ENUM/G/S/QUERY_STD: SKIP
    * VIDIOC_G/S_AUDOUT: SKIP
    * VIDIOC_G/S/ENUMOUTPUT: SKIP
    * VIDIOC_ENUMAUDOUT: SKIP
    * VIDIOC_G/S_FREQUENCY: SKIP
    * VIDIOC_G/S_MODULATOR: SKIP
    * VIDIOC_G/S_AUDIO: SKIP
    * VIDIOC_G/S/ENUMINPUT: PASS
    * VIDIOC_ENUMAUDIO: SKIP
    * VIDIOC_S_HW_FREQ_SEEK: SKIP
    * VIDIOC_G/S_FREQUENCY: SKIP
    * VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: SKIP
    * VIDIOC_LOG_STATUS: SKIP
    * VIDIOC_DBG_G/S_REGISTER: SKIP
    * for-unlimited-opens: PASS
    * VIDIOC_G/S_PRIORITY: PASS
    * VIDIOC_QUERYCAP: PASS
    * second-/dev/video0-open: PASS
    * VIDIOC_QUERYCAP: PASS
    * MC-information-see-Media-Driver-Info-above: PASS



2  | v4l2       | rk3288-veyron-jaq      | arm   |  49 total:  17 PASS   4 FAIL 
 28 SKIP

  Config:      multi_v7_defconfig
  Lab Name:    lab-collabora-dev
  Date:        2018-10-18 17:00:41.724000
  TXT log:     
http://staging-storage.kernelci.org/gtucker/kernelci-media/gtucker-kernelci-media-002-2-gaa27eb0392c7/arm/multi_v7_defconfig/lab-collab
ora-dev/v4l2-rk3288-veyron-jaq.txt
  HTML log:    
http://staging-storage.kernelci.org/gtucker/kernelci-media/gtucker-kernelci-media-002-2-gaa27eb0392c7/arm/multi_v7_defconfig/lab-collab
ora-dev/v4l2-rk3288-veyron-jaq.html
  Rootfs:      
http://staging-storage.kernelci.org/images/rootfs/debian/stretchv4l2/20181017.1//armhf/rootfs.cpio.gz
  Test Git:    git://linuxtv.org/v4l-utils.git
  Test Commit: e889b0d56757b9a74eb8c4c8cc2ebd5b18e228b2


  Test cases:

    * MMAP: FAIL
    * blocking-wait: PASS
    * read/write: SKIP
    * read/write: SKIP
    * read/write: SKIP
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * VIDIOC_G_SLICED_VBI_CAP: SKIP
    * VIDIOC_S_FMT: PASS
    * VIDIOC_TRY_FMT: PASS
    * VIDIOC_G_FMT: PASS
    * VIDIOC_G_FBUF: SKIP
    * VIDIOC_G/S_PARM: FAIL
    * VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: PASS
    * VIDIOC_G/S_JPEGCOMP: SKIP
    * VIDIOC_UNSUBSCRIBE_EVENT/DQEVENT: PASS
    * VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
    * VIDIOC_G/S_CTRL: PASS
    * VIDIOC_QUERYCTRL: PASS
    * VIDIOC_QUERY_EXT_CTRL/QUERYMENU: FAIL
    * VIDIOC_G/S_EDID: SKIP
    * VIDIOC_DV_TIMINGS_CAP: SKIP
    * VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: SKIP
    * VIDIOC_ENUM/G/S/QUERY_STD: SKIP
    * VIDIOC_G/S_AUDOUT: SKIP
    * VIDIOC_G/S/ENUMOUTPUT: SKIP
    * VIDIOC_ENUMAUDOUT: SKIP
    * VIDIOC_G/S_FREQUENCY: SKIP
    * VIDIOC_G/S_MODULATOR: SKIP
    * VIDIOC_G/S_AUDIO: SKIP
    * VIDIOC_G/S/ENUMINPUT: PASS
    * VIDIOC_ENUMAUDIO: SKIP
    * VIDIOC_S_HW_FREQ_SEEK: SKIP
    * VIDIOC_G/S_FREQUENCY: SKIP
    * VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: SKIP
    * VIDIOC_LOG_STATUS: SKIP
    * VIDIOC_DBG_G/S_REGISTER: SKIP
    * for-unlimited-opens: PASS
    * VIDIOC_G/S_PRIORITY: PASS
    * VIDIOC_QUERYCAP: PASS
    * second-/dev/video0-open: PASS
    * VIDIOC_QUERYCAP: PASS
    * MC-information-see-Media-Driver-Info-above: PASS



3  | v4l2       | qemu                   | arm64 | 168 total: 102 PASS   0 FAIL 
 66 SKIP

  Config:      defconfig+virtualvideo
  Lab Name:    lab-collabora-dev
  Date:        2018-10-18 18:51:19.917000
  TXT log:     
http://staging-storage.kernelci.org/gtucker/kernelci-media/gtucker-kernelci-media-002-2-gaa27eb0392c7/arm64/defconfig+virtualvideo/lab-
collabora-dev/v4l2-qemu.txt
  HTML log:    
http://staging-storage.kernelci.org/gtucker/kernelci-media/gtucker-kernelci-media-002-2-gaa27eb0392c7/arm64/defconfig+virtualvideo/lab-
collabora-dev/v4l2-qemu.html
  Rootfs:      
http://staging-storage.kernelci.org/images/rootfs/debian/stretchv4l2/20181017.1//arm64/rootfs.cpio.gz
  Test Git:    git://linuxtv.org/v4l-utils.git
  Test Commit: e889b0d56757b9a74eb8c4c8cc2ebd5b18e228b2


  Test cases:

    * blocking-wait: PASS
    * read/write: PASS
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * VIDIOC_G_SLICED_VBI_CAP: SKIP
    * VIDIOC_S_FMT: PASS
    * VIDIOC_TRY_FMT: PASS
    * VIDIOC_G_FMT: PASS
    * VIDIOC_G_FBUF: PASS
    * VIDIOC_G/S_PARM: PASS
    * VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: PASS
    * VIDIOC_G/S_JPEGCOMP: SKIP
    * VIDIOC_UNSUBSCRIBE_EVENT/DQEVENT: PASS
    * VIDIOC_G/S/TRY_EXT_CTRLS: PASS
    * VIDIOC_G/S_CTRL: PASS
    * VIDIOC_QUERYCTRL: PASS
    * VIDIOC_QUERY_EXT_CTRL/QUERYMENU: PASS
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * read/write: PASS
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * VIDIOC_G_SLICED_VBI_CAP: SKIP
    * VIDIOC_S_FMT: PASS
    * VIDIOC_TRY_FMT: PASS
    * VIDIOC_G_FMT: PASS
    * VIDIOC_G_FBUF: PASS
    * VIDIOC_G/S_PARM: PASS
    * VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: PASS
    * VIDIOC_G/S_JPEGCOMP: SKIP
    * VIDIOC_UNSUBSCRIBE_EVENT/DQEVENT: PASS
    * VIDIOC_G/S/TRY_EXT_CTRLS: PASS
    * VIDIOC_G/S_CTRL: PASS
    * VIDIOC_QUERYCTRL: PASS
    * VIDIOC_QUERY_EXT_CTRL/QUERYMENU: PASS
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * read/write: PASS
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * VIDIOC_G_SLICED_VBI_CAP: SKIP
    * VIDIOC_S_FMT: PASS
    * VIDIOC_TRY_FMT: PASS
    * VIDIOC_G_FMT: PASS
    * VIDIOC_G_FBUF: PASS
    * VIDIOC_G/S_PARM: PASS
    * VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: PASS
    * VIDIOC_G/S_JPEGCOMP: SKIP
    * VIDIOC_UNSUBSCRIBE_EVENT/DQEVENT: PASS
    * VIDIOC_G/S/TRY_EXT_CTRLS: PASS
    * VIDIOC_G/S_CTRL: PASS
    * VIDIOC_QUERYCTRL: PASS
    * VIDIOC_QUERY_EXT_CTRL/QUERYMENU: PASS
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * VIDIOC_G_SLICED_VBI_CAP: SKIP
    * VIDIOC_S_FMT: PASS
    * VIDIOC_TRY_FMT: PASS
    * VIDIOC_G_FMT: PASS
    * VIDIOC_G_FBUF: PASS
    * VIDIOC_G/S_PARM: PASS
    * VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: PASS
    * VIDIOC_G/S_JPEGCOMP: SKIP
    * VIDIOC_UNSUBSCRIBE_EVENT/DQEVENT: PASS
    * VIDIOC_G/S/TRY_EXT_CTRLS: PASS
    * VIDIOC_G/S_CTRL: PASS
    * VIDIOC_QUERYCTRL: PASS
    * VIDIOC_QUERY_EXT_CTRL/QUERYMENU: PASS
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * VIDIOC_G_SLICED_VBI_CAP: SKIP
    * VIDIOC_S_FMT: PASS
    * VIDIOC_TRY_FMT: PASS
    * VIDIOC_G_FMT: PASS
    * VIDIOC_G_FBUF: PASS
    * VIDIOC_G/S_PARM: PASS
    * VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: PASS
    * VIDIOC_G/S_JPEGCOMP: SKIP
    * VIDIOC_UNSUBSCRIBE_EVENT/DQEVENT: PASS
    * VIDIOC_G/S/TRY_EXT_CTRLS: PASS
    * VIDIOC_G/S_CTRL: PASS
    * VIDIOC_QUERYCTRL: PASS
    * VIDIOC_QUERY_EXT_CTRL/QUERYMENU: PASS
    * VIDIOC_EXPBUF: PASS
    * VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: PASS
    * VIDIOC_TRY_DECODER_CMD: SKIP
    * VIDIOC_G_ENC_INDEX: SKIP
    * VIDIOC_TRY_ENCODER_CMD: SKIP
    * Scaling: SKIP
    * Composing: SKIP
    * Cropping: SKIP
    * VIDIOC_G_SLICED_VBI_CAP: SKIP
    * VIDIOC_S_FMT: PASS
    * VIDIOC_TRY_FMT: PASS
    * VIDIOC_G_FMT: PASS
    * VIDIOC_G_FBUF: PASS
    * VIDIOC_G/S_PARM: PASS
    * VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: PASS
    * VIDIOC_G/S_JPEGCOMP: SKIP
    * VIDIOC_UNSUBSCRIBE_EVENT/DQEVENT: PASS
    * VIDIOC_G/S/TRY_EXT_CTRLS: PASS
    * VIDIOC_G/S_CTRL: PASS
    * VIDIOC_QUERYCTRL: PASS
    * VIDIOC_QUERY_EXT_CTRL/QUERYMENU: PASS
    * VIDIOC_G/S_EDID: PASS
    * VIDIOC_DV_TIMINGS_CAP: PASS
    * VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: PASS
    * VIDIOC_ENUM/G/S/QUERY_STD: PASS
    * VIDIOC_G/S_AUDOUT: SKIP
    * VIDIOC_G/S/ENUMOUTPUT: SKIP
    * VIDIOC_ENUMAUDOUT: SKIP
    * VIDIOC_G/S_FREQUENCY: PASS
    * VIDIOC_G/S_MODULATOR: SKIP
    * VIDIOC_G/S_AUDIO: PASS
    * VIDIOC_G/S/ENUMINPUT: PASS
    * VIDIOC_ENUMAUDIO: PASS
    * VIDIOC_S_HW_FREQ_SEEK: SKIP
    * VIDIOC_G/S_FREQUENCY: PASS
    * VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: PASS
    * VIDIOC_LOG_STATUS: PASS
    * VIDIOC_DBG_G/S_REGISTER: SKIP
    * for-unlimited-opens: PASS
    * VIDIOC_G/S_PRIORITY: PASS
    * VIDIOC_QUERYCAP: PASS
    * second-/dev/video0-open: PASS
    * VIDIOC_QUERYCAP: PASS


Reply via email to