Title: [236165] trunk
Revision
236165
Author
[email protected]
Date
2018-09-19 01:53:50 -0700 (Wed, 19 Sep 2018)

Log Message

[GStreamer] Add support for AV1 decoding
https://bugs.webkit.org/show_bug.cgi?id=189647

Source/WebCore:

Tweaked by Xabier Rodriguez Calvar <[email protected]>.
Patch by Philippe Normand <[email protected]> on 2018-09-19
Reviewed by Žan Doberšek.

AV1 can be muxed in MP4 and WebM containers. The test is an adaptation from Chromium's unittest:
https://chromium.googlesource.com/chromium/src/+/master/content/browser/media/media_canplaytype_browsertest.cc

Test: media/media-can-play-av1.html

* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::supportsType): Add AV1
support. The av01 codec has to be explicitely checked, along with
the presence of a compatible AV1 decoder.

Tools:

Patch by Philippe Normand <[email protected]> on 2018-09-19
Reviewed by Žan Doberšek.

Add patches required for AV1 decoding support. They're all
upstream already and will be shipped in GStreamer 1.16. The aom
GStreamer plugin depends on the aom library for which there's no
official release yet.

* gstreamer/jhbuild.modules:
* gstreamer/patches/gst-plugins-bad-0001-aomenc-Add-support-for-10-12bit-decoding.patch: Added.
* gstreamer/patches/gst-plugins-bad-0002-aomenc-Handle-8-bit_depth-images-with-AOM_IMG_FMT_HI.patch: Added.
* gstreamer/patches/gst-plugins-good-0001-qtdemux-Detect-and-expose-CEA-608-708-Closed-Caption.patch: Added. This one is unrelated but an implicit dependency of the next one.
* gstreamer/patches/gst-plugins-good-0004-qtdemux-Add-initial-support-for-AV1-demuxing.patch: Added.
* gstreamer/patches/gst-plugins-good-0005-qtdemux-Extract-AV1-codec_data-and-put-it-in-the-cap.patch: Added.
* gstreamer/patches/gst-plugins-good-0006-qtdemux-Recognize-more-AV1-atoms.patch: Added.

LayoutTests:

Patch by Philippe Normand <[email protected]> on 2018-09-19
Reviewed by Žan Doberšek.

Layout test for AV1 canPlay() compatiblity checking. Right now
GStreamer-based ports are the only ones expecting to support AV1.

* media/media-can-play-av1-expected.txt: Added.
* media/media-can-play-av1.html: Added.
* platform/ios/media/media-can-play-av1-expected.txt: Added.
* platform/mac/media/media-can-play-av1-expected.txt: Added.

Modified Paths

Added Paths

Diff

Modified: trunk/LayoutTests/ChangeLog (236164 => 236165)


--- trunk/LayoutTests/ChangeLog	2018-09-19 08:16:23 UTC (rev 236164)
+++ trunk/LayoutTests/ChangeLog	2018-09-19 08:53:50 UTC (rev 236165)
@@ -1,3 +1,18 @@
+2018-09-19  Philippe Normand <[email protected]>
+
+        [GStreamer] Add support for AV1 decoding
+        https://bugs.webkit.org/show_bug.cgi?id=189647
+
+        Reviewed by Žan Doberšek.
+
+        Layout test for AV1 canPlay() compatiblity checking. Right now
+        GStreamer-based ports are the only ones expecting to support AV1.
+
+        * media/media-can-play-av1-expected.txt: Added.
+        * media/media-can-play-av1.html: Added.
+        * platform/ios/media/media-can-play-av1-expected.txt: Added.
+        * platform/mac/media/media-can-play-av1-expected.txt: Added.
+
 2018-09-19  Chris Dumez  <[email protected]>
 
         http/tests/navigation/window-open-cross-origin-then-navigated-back-same-origin.html is flaky

Added: trunk/LayoutTests/media/media-can-play-av1-expected.txt (0 => 236165)


--- trunk/LayoutTests/media/media-can-play-av1-expected.txt	                        (rev 0)
+++ trunk/LayoutTests/media/media-can-play-av1-expected.txt	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,11 @@
+
+Test HTMLMediaElement canPlayType() method for the AV1 codec.
+
+These tests may be expected to fail if the WebKit port does not support the format.
+
+EXPECTED (video.canPlayType('video/mp4; codecs=av1') == '') OK
+EXPECTED (video.canPlayType('video/webm; codecs=av1') == '') OK
+EXPECTED (video.canPlayType('video/mp4; codecs=av01.0.04M.08') == 'probably') OK
+EXPECTED (video.canPlayType('video/webm; codecs=av01.0.04M.08') == 'probably') OK
+END OF TEST
+

Added: trunk/LayoutTests/media/media-can-play-av1.html (0 => 236165)


--- trunk/LayoutTests/media/media-can-play-av1.html	                        (rev 0)
+++ trunk/LayoutTests/media/media-can-play-av1.html	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,24 @@
+<!DOCTYPE html>
+<html>
+    <head>
+        <script src=""
+        <script>
+         function start() {
+             video = mediaElement = document.getElementsByTagName('video')[0];
+
+             testExpected("video.canPlayType('video/mp4; codecs=av1')", "");
+             testExpected("video.canPlayType('video/webm; codecs=av1')", "");
+
+             testExpected("video.canPlayType('video/mp4; codecs=av01.0.04M.08')", "probably");
+             testExpected("video.canPlayType('video/webm; codecs=av01.0.04M.08')", "probably");
+
+             endTest();
+         }
+        </script>
+    </head>
+    <body _onload_="start()">
+        <video controls></video>
+        <p>Test HTMLMediaElement <em>canPlayType()</em> method for the AV1 codec.</p>
+        <p>These tests may be expected to fail if the WebKit port does not support the format.</p>
+    </body>
+</html>

Added: trunk/LayoutTests/platform/ios/media/media-can-play-av1-expected.txt (0 => 236165)


--- trunk/LayoutTests/platform/ios/media/media-can-play-av1-expected.txt	                        (rev 0)
+++ trunk/LayoutTests/platform/ios/media/media-can-play-av1-expected.txt	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,11 @@
+
+Test HTMLMediaElement canPlayType() method for the AV1 codec.
+
+These tests may be expected to fail if the WebKit port does not support the format.
+
+EXPECTED (video.canPlayType('video/mp4; codecs=av1') == ''), OBSERVED 'maybe' FAIL
+EXPECTED (video.canPlayType('video/webm; codecs=av1') == '') OK
+EXPECTED (video.canPlayType('video/mp4; codecs=av01.0.04M.08') == 'probably'), OBSERVED 'maybe' FAIL
+EXPECTED (video.canPlayType('video/webm; codecs=av01.0.04M.08') == 'probably'), OBSERVED '' FAIL
+END OF TEST
+

Added: trunk/LayoutTests/platform/mac/media/media-can-play-av1-expected.txt (0 => 236165)


--- trunk/LayoutTests/platform/mac/media/media-can-play-av1-expected.txt	                        (rev 0)
+++ trunk/LayoutTests/platform/mac/media/media-can-play-av1-expected.txt	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,11 @@
+
+Test HTMLMediaElement canPlayType() method for the AV1 codec.
+
+These tests may be expected to fail if the WebKit port does not support the format.
+
+EXPECTED (video.canPlayType('video/mp4; codecs=av1') == ''), OBSERVED 'maybe' FAIL
+EXPECTED (video.canPlayType('video/webm; codecs=av1') == '') OK
+EXPECTED (video.canPlayType('video/mp4; codecs=av01.0.04M.08') == 'probably'), OBSERVED 'maybe' FAIL
+EXPECTED (video.canPlayType('video/webm; codecs=av01.0.04M.08') == 'probably'), OBSERVED '' FAIL
+END OF TEST
+

Modified: trunk/Source/WebCore/ChangeLog (236164 => 236165)


--- trunk/Source/WebCore/ChangeLog	2018-09-19 08:16:23 UTC (rev 236164)
+++ trunk/Source/WebCore/ChangeLog	2018-09-19 08:53:50 UTC (rev 236165)
@@ -1,3 +1,21 @@
+2018-09-19  Philippe Normand <[email protected]>
+
+        [GStreamer] Add support for AV1 decoding
+        https://bugs.webkit.org/show_bug.cgi?id=189647
+
+        Tweaked by Xabier Rodriguez Calvar <[email protected]>.
+        Reviewed by Žan Doberšek.
+
+        AV1 can be muxed in MP4 and WebM containers. The test is an adaptation from Chromium's unittest:
+        https://chromium.googlesource.com/chromium/src/+/master/content/browser/media/media_canplaytype_browsertest.cc
+
+        Test: media/media-can-play-av1.html
+
+        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
+        (WebCore::MediaPlayerPrivateGStreamer::supportsType): Add AV1
+        support. The av01 codec has to be explicitely checked, along with
+        the presence of a compatible AV1 decoder.
+
 2018-09-18  Basuke Suzuki  <[email protected]>
 
         [Curl] Limit capturing extra metrics for Web Inspector when not required.

Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp (236164 => 236165)


--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp	2018-09-19 08:16:23 UTC (rev 236164)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp	2018-09-19 08:53:50 UTC (rev 236165)
@@ -2352,10 +2352,29 @@
     if (parameters.type.isEmpty())
         return result;
 
-    // spec says we should not return "probably" if the codecs string is empty
+    // Spec says we should not return "probably" if the codecs string is empty.
     if (mimeTypeSet().contains(parameters.type.containerType()))
         result = parameters.type.codecs().isEmpty() ? MediaPlayer::MayBeSupported : MediaPlayer::IsSupported;
 
+    auto containerType = parameters.type.containerType();
+    if (containerType == "video/mp4"_s || containerType == "video/webm"_s) {
+        if (mimeTypeSet().contains(containerType)) {
+            GList* videoDecoderFactories = gst_element_factory_list_get_elements(GST_ELEMENT_FACTORY_TYPE_DECODER | GST_ELEMENT_FACTORY_TYPE_MEDIA_VIDEO, GST_RANK_MARGINAL);
+            bool av1DecoderFound = gstRegistryHasElementForMediaType(videoDecoderFactories, "video/x-av1"_s);
+            gst_plugin_feature_list_free(videoDecoderFactories);
+            for (auto& codec : parameters.type.codecs()) {
+                if (codec.startsWith("av01"_s)) {
+                    result = av1DecoderFound ? MediaPlayer::IsSupported : MediaPlayer::IsNotSupported;
+                    break;
+                }
+                if (codec.startsWith("av1"_s)) {
+                    result = MediaPlayer::IsNotSupported;
+                    break;
+                }
+            }
+        }
+    }
+
     return extendedSupportsType(parameters, result);
 }
 

Modified: trunk/Tools/ChangeLog (236164 => 236165)


--- trunk/Tools/ChangeLog	2018-09-19 08:16:23 UTC (rev 236164)
+++ trunk/Tools/ChangeLog	2018-09-19 08:53:50 UTC (rev 236165)
@@ -1,3 +1,23 @@
+2018-09-19  Philippe Normand  <[email protected]>
+
+        [GStreamer] Add support for AV1 decoding
+        https://bugs.webkit.org/show_bug.cgi?id=189647
+
+        Reviewed by Žan Doberšek.
+
+        Add patches required for AV1 decoding support. They're all
+        upstream already and will be shipped in GStreamer 1.16. The aom
+        GStreamer plugin depends on the aom library for which there's no
+        official release yet.
+
+        * gstreamer/jhbuild.modules:
+        * gstreamer/patches/gst-plugins-bad-0001-aomenc-Add-support-for-10-12bit-decoding.patch: Added.
+        * gstreamer/patches/gst-plugins-bad-0002-aomenc-Handle-8-bit_depth-images-with-AOM_IMG_FMT_HI.patch: Added.
+        * gstreamer/patches/gst-plugins-good-0001-qtdemux-Detect-and-expose-CEA-608-708-Closed-Caption.patch: Added. This one is unrelated but an implicit dependency of the next one.
+        * gstreamer/patches/gst-plugins-good-0004-qtdemux-Add-initial-support-for-AV1-demuxing.patch: Added.
+        * gstreamer/patches/gst-plugins-good-0005-qtdemux-Extract-AV1-codec_data-and-put-it-in-the-cap.patch: Added.
+        * gstreamer/patches/gst-plugins-good-0006-qtdemux-Recognize-more-AV1-atoms.patch: Added.
+
 2018-09-18  Jonathan Bedard  <[email protected]>
 
         webkitpy: Clobbering and building occurs multiple times for iOS Simulator ports

Modified: trunk/Tools/gstreamer/jhbuild.modules (236164 => 236165)


--- trunk/Tools/gstreamer/jhbuild.modules	2018-09-19 08:16:23 UTC (rev 236164)
+++ trunk/Tools/gstreamer/jhbuild.modules	2018-09-19 08:53:50 UTC (rev 236165)
@@ -21,6 +21,8 @@
       href=""
   <repository type="git" name="chromium.googlesource.com"
       href=""
+  <repository type="git" name="aomedia.googlesource.com"
+      href=""
 
   <autotools id="orc" autogenargs="--disable-gtk-doc" autogen-sh="configure">
     <branch module="orc/orc-${version}.tar.xz" version="0.4.28"
@@ -40,6 +42,11 @@
     <branch repo="chromium.googlesource.com" checkoutdir="libvpx" tag="v1.7.0"/>
   </autotools>
 
+  <cmake id="aom" cmakeargs="-DBUILD_SHARED_LIBS=1">
+    <branch repo="aomedia.googlesource.com" module="aom" checkoutdir="aom" branch="master"
+            revision="master" tag="cdb083e84a71d6d31163a4f64f18dc1328fd782c" />
+  </cmake>
+
   <autotools id="gstreamer" autogen-sh="configure" autogenargs="--disable-gtk-doc --disable-introspection --disable-examples --disable-tests">
     <dependencies>
       <dep package="orc"/>
@@ -74,6 +81,10 @@
       <patch file="gst-plugins-good-0001-qtdemux-Do-not-run-the-preferred-decryptor-context-q.patch" strip="1" /> <!-- Merged as 3e063703b3a51b8aaa7f75f36c4660c583a60e93 -->
       <patch file="gst-plugins-good-0002-qtdemux-Do-not-unref-a-NULL-stream_tags.patch" strip="1" /> <!-- Merged as 43a540b1cd9f162d3dae5d50e36703dfaf558a3e -->
       <patch file="gst-plugins-good-0003-qtdemux-Clarify-field-name-about-stream-encryption-s.patch" strip="1" /> <!-- Merged as 0fbde2a07928ba79b7a4ddb4585f6b1de36a74bb -->
+      <patch file="gst-plugins-good-0001-qtdemux-Detect-and-expose-CEA-608-708-Closed-Caption.patch" strip="1" /> <!-- Merged as 2869edeea29a022007a99db2df41b5a7e1a3025c (to be shipped in 1.16) -->
+      <patch file="gst-plugins-good-0004-qtdemux-Add-initial-support-for-AV1-demuxing.patch" strip="1" /> <!-- Merged as b6e8c6323e01249bb3ac4f121c3adc3ea8d924bf (to be shipped in 1.16) -->
+      <patch file="gst-plugins-good-0005-qtdemux-Extract-AV1-codec_data-and-put-it-in-the-cap.patch" strip="1" /> <!-- Merged as 43ee4055c3d03a9f99acdf3c05f45e154899edef (to be shipped in 1.16) -->
+      <patch file="gst-plugins-good-0006-qtdemux-Recognize-more-AV1-atoms.patch" strip="1" /> <!-- Merged as e07ce9afb4edb2ea898763c7ea11f0af2704104e (to be shipped in 1.16) -->
     </branch>
   </autotools>
 
@@ -82,8 +93,12 @@
       <dep package="graphene"/>
       <dep package="gst-plugins-base"/>
       <dep package="openh264"/>
+      <dep package="aom"/>
     </dependencies>
-    <branch hash="sha256:b2224e5d9c1b85ad51233f6135524bb9e16a9172d395edc79c73b89094659fd5" module="gst-plugins-bad/gst-plugins-bad-${version}.tar.xz" repo="gstreamer" version="1.14.3" />
+    <branch hash="sha256:b2224e5d9c1b85ad51233f6135524bb9e16a9172d395edc79c73b89094659fd5" module="gst-plugins-bad/gst-plugins-bad-${version}.tar.xz" repo="gstreamer" version="1.14.3">
+      <patch file="gst-plugins-bad-0001-aomenc-Add-support-for-10-12bit-decoding.patch" strip="1" /> <!-- Merged as 1d96d9e842dd71882f54ddffbf6c1ccecdb03fcd (to be shipped in 1.16) -->
+      <patch file="gst-plugins-bad-0002-aomenc-Handle-8-bit_depth-images-with-AOM_IMG_FMT_HI.patch" strip="1" /> <!-- Merged as 10a37e0c3528a721084441c37af37baef0908ddc (to be shipped in 1.16) -->
+    </branch>
   </autotools>
 
   <autotools id="gst-libav" autogen-sh="configure" autogenargs="--with-libav-extra-configure='--disable-yasm' --disable-gtk-doc">

Added: trunk/Tools/gstreamer/patches/gst-plugins-bad-0001-aomenc-Add-support-for-10-12bit-decoding.patch (0 => 236165)


--- trunk/Tools/gstreamer/patches/gst-plugins-bad-0001-aomenc-Add-support-for-10-12bit-decoding.patch	                        (rev 0)
+++ trunk/Tools/gstreamer/patches/gst-plugins-bad-0001-aomenc-Add-support-for-10-12bit-decoding.patch	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,165 @@
+From 1d96d9e842dd71882f54ddffbf6c1ccecdb03fcd Mon Sep 17 00:00:00 2001
+From: Sean-Der <[email protected]>
+Date: Wed, 27 Jun 2018 09:44:00 +0000
+Subject: [PATCH] aomenc: Add support for 10/12bit decoding
+
+https://bugzilla.gnome.org/show_bug.cgi?id=791674
+---
+ ext/aom/gstav1dec.c | 108 ++++++++++++++++++++++++++++++++++++++++++--
+ 1 file changed, 103 insertions(+), 5 deletions(-)
+
+diff --git a/ext/aom/gstav1dec.c b/ext/aom/gstav1dec.c
+index d33118e64..23b18f44c 100644
+--- a/ext/aom/gstav1dec.c
++++ b/ext/aom/gstav1dec.c
+@@ -52,15 +52,26 @@ static GstStaticPadTemplate gst_av1_dec_src_pad_template =
+ GST_STATIC_PAD_TEMPLATE ("src",
+     GST_PAD_SRC,
+     GST_PAD_ALWAYS,
+-    GST_STATIC_CAPS ("video/x-raw, "
+-        "format = (string) \"I420\", "
+-        "framerate = (fraction) [0, MAX], "
+-        "width = (int) [ 4, MAX ], " "height = (int) [ 4, MAX ]")
++    GST_STATIC_CAPS (GST_VIDEO_CAPS_MAKE ("{ I420, YV12, Y42B, Y444"
++#if G_BYTE_ORDER == G_LITTLE_ENDIAN
++            ", I420_10LE, I420_12LE, I422_10LE, I422_12LE, Y444_10LE, Y444_12LE"
++#else
++            ", I420_10BE, I420_12BE, I422_10BE, I422_12BE, Y444_10BE, Y444_12BE"
++#endif
++            " }"))
+     );
+ 
+ GST_DEBUG_CATEGORY_STATIC (av1_dec_debug);
+ #define GST_CAT_DEFAULT av1_dec_debug
+ 
++#define GST_VIDEO_FORMAT_WITH_ENDIAN(fmt,endian) GST_VIDEO_FORMAT_##fmt##endian
++
++#if G_BYTE_ORDER == G_LITTLE_ENDIAN
++#define AOM_FMT_TO_GST(fmt) GST_VIDEO_FORMAT_WITH_ENDIAN(fmt,LE)
++#else
++#define AOM_FMT_TO_GST(fmt) GST_VIDEO_FORMAT_WITH_ENDIAN(fmt,BE)
++#endif
++
+ static void gst_av1_dec_set_property (GObject * object, guint prop_id,
+     const GValue * value, GParamSpec * pspec);
+ static void gst_av1_dec_get_property (GObject * object, guint prop_id,
+@@ -79,6 +90,8 @@ static void gst_av1_dec_image_to_buffer (GstAV1Dec * dec,
+     const aom_image_t * img, GstBuffer * buffer);
+ static GstFlowReturn gst_av1_dec_open_codec (GstAV1Dec * av1dec,
+     GstVideoCodecFrame * frame);
++static gboolean gst_av1_dec_get_valid_format (GstAV1Dec * dec,
++    const aom_image_t * img, GstVideoFormat * fmt);
+ 
+ #define gst_av1_dec_parent_class parent_class
+ G_DEFINE_TYPE (GstAV1Dec, gst_av1_dec, GST_TYPE_VIDEO_DECODER);
+@@ -309,6 +322,81 @@ gst_av1_dec_image_to_buffer (GstAV1Dec * dec, const aom_image_t * img,
+   gst_video_frame_unmap (&frame);
+ }
+ 
++gboolean
++gst_av1_dec_get_valid_format (GstAV1Dec * dec, const aom_image_t * img,
++    GstVideoFormat * fmt)
++{
++  switch (img->fmt) {
++    case AOM_IMG_FMT_I420:
++    case AOM_IMG_FMT_I42016:
++      if (img->bit_depth == 8) {
++        *fmt = img->monochrome ? GST_VIDEO_FORMAT_GRAY8 : GST_VIDEO_FORMAT_I420;
++        return TRUE;
++      } else if (img->bit_depth == 10) {
++        *fmt = AOM_FMT_TO_GST (I420_10);
++        return TRUE;
++      } else if (img->bit_depth == 12) {
++        *fmt = AOM_FMT_TO_GST (I420_12);
++        return TRUE;
++      }
++
++      GST_FIXME_OBJECT (dec,
++          "Please add a 4:2:0 planar %u bit depth frame format",
++          img->bit_depth);
++      GST_ELEMENT_WARNING (dec, STREAM, NOT_IMPLEMENTED, (NULL),
++          ("Unsupported frame format - 4:2:0 planar %u bit depth",
++              img->bit_depth));
++      return FALSE;
++
++    case AOM_IMG_FMT_I422:
++    case AOM_IMG_FMT_I42216:
++      if (img->bit_depth == 8) {
++        *fmt = GST_VIDEO_FORMAT_Y42B;
++        return TRUE;
++      } else if (img->bit_depth == 10) {
++        *fmt = AOM_FMT_TO_GST (I422_10);
++        return TRUE;
++      } else if (img->bit_depth == 12) {
++        *fmt = AOM_FMT_TO_GST (I422_12);
++        return TRUE;
++      }
++      GST_FIXME_OBJECT (dec,
++          "Please add a 4:2:2 planar %u bit depth frame format",
++          img->bit_depth);
++      GST_ELEMENT_WARNING (dec, STREAM, NOT_IMPLEMENTED, (NULL),
++          ("Unsupported frame format - 4:2:2 planar %u bit depth",
++              img->bit_depth));
++      return FALSE;
++
++    case AOM_IMG_FMT_I444:
++    case AOM_IMG_FMT_I44416:
++      if (img->bit_depth == 8) {
++        *fmt = GST_VIDEO_FORMAT_Y444;
++        return TRUE;
++      } else if (img->bit_depth == 10) {
++        *fmt = AOM_FMT_TO_GST (Y444_10);
++        return TRUE;
++      } else if (img->bit_depth == 12) {
++        *fmt = AOM_FMT_TO_GST (Y444_12);
++        return TRUE;
++      }
++      GST_FIXME_OBJECT (dec,
++          "Please add a 4:4:4 planar %u bit depth frame format",
++          img->bit_depth);
++      GST_ELEMENT_WARNING (dec, STREAM, NOT_IMPLEMENTED, (NULL),
++          ("Unsupported frame format - 4:4:4 planar %u bit depth",
++              img->bit_depth));
++      return FALSE;
++
++    case AOM_IMG_FMT_YV12:
++      *fmt = GST_VIDEO_FORMAT_YV12;
++      return TRUE;
++
++    default:
++      return FALSE;
++  }
++}
++
+ static GstFlowReturn
+ gst_av1_dec_handle_frame (GstVideoDecoder * dec, GstVideoCodecFrame * frame)
+ {
+@@ -318,6 +406,7 @@ gst_av1_dec_handle_frame (GstVideoDecoder * dec, GstVideoCodecFrame * frame)
+   aom_codec_err_t status;
+   aom_image_t *img;
+   aom_codec_iter_t iter = NULL;
++  GstVideoFormat fmt;
+ 
+   if (!av1dec->decoder_inited) {
+     ret = gst_av1_dec_open_codec (av1dec, frame);
+@@ -349,7 +438,16 @@ gst_av1_dec_handle_frame (GstVideoDecoder * dec, GstVideoCodecFrame * frame)
+ 
+   img = aom_codec_get_frame (&av1dec->decoder, &iter);
+   if (img) {
+-    gst_av1_dec_handle_resolution_change (av1dec, img, GST_VIDEO_FORMAT_I420);
++    if (gst_av1_dec_get_valid_format (av1dec, img, &fmt) == FALSE) {
++      aom_img_free (img);
++      GST_ELEMENT_ERROR (dec, LIBRARY, ENCODE,
++          ("Failed to decode frame"), ("Unsupported color format %d",
++              img->fmt));
++      gst_video_codec_frame_unref (frame);
++      return GST_FLOW_ERROR;
++    }
++
++    gst_av1_dec_handle_resolution_change (av1dec, img, fmt);
+ 
+     ret = gst_video_decoder_allocate_output_frame (dec, frame);
+     if (ret == GST_FLOW_OK) {
+-- 
+2.18.0
+

Added: trunk/Tools/gstreamer/patches/gst-plugins-bad-0002-aomenc-Handle-8-bit_depth-images-with-AOM_IMG_FMT_HI.patch (0 => 236165)


--- trunk/Tools/gstreamer/patches/gst-plugins-bad-0002-aomenc-Handle-8-bit_depth-images-with-AOM_IMG_FMT_HI.patch	                        (rev 0)
+++ trunk/Tools/gstreamer/patches/gst-plugins-bad-0002-aomenc-Handle-8-bit_depth-images-with-AOM_IMG_FMT_HI.patch	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,56 @@
+From 10a37e0c3528a721084441c37af37baef0908ddc Mon Sep 17 00:00:00 2001
+From: Sean-Der <[email protected]>
+Date: Wed, 27 Jun 2018 09:48:00 +0000
+Subject: [PATCH] aomenc: Handle 8 bit_depth images with
+ AOM_IMG_FMT_HIGHBITDEPTH enabled
+
+https://bugzilla.gnome.org/show_bug.cgi?id=791674
+---
+ ext/aom/gstav1dec.c | 20 ++++++++++++++++----
+ 1 file changed, 16 insertions(+), 4 deletions(-)
+
+diff --git a/ext/aom/gstav1dec.c b/ext/aom/gstav1dec.c
+index 23b18f44c..6e0a9bd5d 100644
+--- a/ext/aom/gstav1dec.c
++++ b/ext/aom/gstav1dec.c
+@@ -285,7 +285,7 @@ static void
+ gst_av1_dec_image_to_buffer (GstAV1Dec * dec, const aom_image_t * img,
+     GstBuffer * buffer)
+ {
+-  int deststride, srcstride, height, width, line, comp;
++  int deststride, srcstride, height, width, line, comp, y;
+   guint8 *dest, *src;
+   GstVideoFrame frame;
+   GstVideoInfo *info = &dec->output_state->info;
+@@ -298,13 +298,25 @@ gst_av1_dec_image_to_buffer (GstAV1Dec * dec, const aom_image_t * img,
+   for (comp = 0; comp < 3; comp++) {
+     dest = GST_VIDEO_FRAME_COMP_DATA (&frame, comp);
+     src = ""
+-    width = GST_VIDEO_FRAME_COMP_WIDTH (&frame, comp)
+-        * GST_VIDEO_FRAME_COMP_PSTRIDE (&frame, comp);
++    width =
++        GST_VIDEO_FRAME_COMP_WIDTH (&frame,
++        comp) * GST_VIDEO_FRAME_COMP_PSTRIDE (&frame, comp);
+     height = GST_VIDEO_FRAME_COMP_HEIGHT (&frame, comp);
+     deststride = GST_VIDEO_FRAME_COMP_STRIDE (&frame, comp);
+     srcstride = img->stride[comp];
+ 
+-    if (srcstride == deststride) {
++    if ((img->fmt & AOM_IMG_FMT_HIGHBITDEPTH) && img->bit_depth == 8) {
++      GST_TRACE_OBJECT (dec,
++          "HIGHBITDEPTH image with 8 bit_depth. Comp %d: %d != %d, copying "
++          "line by line.", comp, srcstride, deststride);
++      for (line = 0; line < height; line++) {
++        for (y = 0; y < width; y++) {
++          dest[y] = src[y * 2];
++        }
++        dest += deststride;
++        src += srcstride;
++      }
++    } else if (srcstride == deststride) {
+       GST_TRACE_OBJECT (dec, "Stride matches. Comp %d: %d, copying full plane",
+           comp, srcstride);
+       memcpy (dest, src, srcstride * height);
+-- 
+2.18.0
+

Added: trunk/Tools/gstreamer/patches/gst-plugins-good-0001-qtdemux-Detect-and-expose-CEA-608-708-Closed-Caption.patch (0 => 236165)


--- trunk/Tools/gstreamer/patches/gst-plugins-good-0001-qtdemux-Detect-and-expose-CEA-608-708-Closed-Caption.patch	                        (rev 0)
+++ trunk/Tools/gstreamer/patches/gst-plugins-good-0001-qtdemux-Detect-and-expose-CEA-608-708-Closed-Caption.patch	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,234 @@
+From 2869edeea29a022007a99db2df41b5a7e1a3025c Mon Sep 17 00:00:00 2001
+From: Edward Hervey <[email protected]>
+Date: Wed, 31 Jan 2018 15:10:03 +0100
+Subject: [PATCH] qtdemux: Detect and expose CEA 608/708 Closed Caption tracks
+
+https://bugzilla.gnome.org/show_bug.cgi?id=606643
+---
+ gst/isomp4/fourcc.h        |   6 ++
+ gst/isomp4/qtdemux.c       | 151 ++++++++++++++++++++++++++++++++++++-
+ gst/isomp4/qtdemux_types.c |   1 +
+ 3 files changed, 156 insertions(+), 2 deletions(-)
+
+diff --git a/gst/isomp4/fourcc.h b/gst/isomp4/fourcc.h
+index be5c7c04e..6f467ffbb 100644
+--- a/gst/isomp4/fourcc.h
++++ b/gst/isomp4/fourcc.h
+@@ -95,6 +95,12 @@ G_BEGIN_DECLS
+ #define FOURCC_avc1     GST_MAKE_FOURCC('a','v','c','1')
+ #define FOURCC_avc3     GST_MAKE_FOURCC('a','v','c','3')
+ #define FOURCC_avcC     GST_MAKE_FOURCC('a','v','c','C')
++#define FOURCC_c608     GST_MAKE_FOURCC('c','6','0','8')
++#define FOURCC_c708     GST_MAKE_FOURCC('c','7','0','8')
++#define FOURCC_ccdp     GST_MAKE_FOURCC('c','c','d','p')
++#define FOURCC_cdat     GST_MAKE_FOURCC('c','d','a','t')
++#define FOURCC_cdt2     GST_MAKE_FOURCC('c','d','t','2')
++#define FOURCC_clcp     GST_MAKE_FOURCC('c','l','c','p')
+ #define FOURCC_clip     GST_MAKE_FOURCC('c','l','i','p')
+ #define FOURCC_cmov     GST_MAKE_FOURCC('c','m','o','v')
+ #define FOURCC_cmvd     GST_MAKE_FOURCC('c','m','v','d')
+diff --git a/gst/isomp4/qtdemux.c b/gst/isomp4/qtdemux.c
+index 79ef5f55b..f732e92b3 100644
+--- a/gst/isomp4/qtdemux.c
++++ b/gst/isomp4/qtdemux.c
+@@ -5370,6 +5370,120 @@ gst_qtdemux_align_buffer (GstQTDemux * demux,
+   return buffer;
+ }
+ 
++static guint8 *
++convert_to_ccdata (const guint8 * ccpair, guint8 ccpair_size, guint field,
++    gsize * res)
++{
++  guint8 *storage;
++  gsize i;
++
++  /* We are converting from pairs to triplets */
++  *res = ccpair_size / 2 * 3;
++  storage = g_malloc (*res);
++  for (i = 0; i * 2 < ccpair_size; i += 1) {
++    if (field == 1)
++      storage[i * 3] = 0xfc;
++    else
++      storage[i * 3] = 0xfd;
++    storage[i * 3 + 1] = ccpair[i * 2];
++    storage[i * 3 + 2] = ccpair[i * 2 + 1];
++  }
++
++  return storage;
++}
++
++static guint8 *
++extract_cc_from_data (QtDemuxStream * stream, const guint8 * data, gsize size,
++    gsize * cclen)
++{
++  guint8 *res = NULL;
++  guint32 atom_length, fourcc;
++  QtDemuxStreamStsdEntry *stsd_entry;
++
++  GST_MEMDUMP ("caption atom", data, size);
++
++  /* There might be multiple atoms */
++
++  *cclen = 0;
++  if (size < 8)
++    goto invalid_cdat;
++  atom_length = QT_UINT32 (data);
++  fourcc = QT_FOURCC (data + 4);
++  if (G_UNLIKELY (atom_length > size || atom_length == 8))
++    goto invalid_cdat;
++
++  GST_DEBUG_OBJECT (stream->pad, "here");
++
++  /* Check if we have somethig compatible */
++  stsd_entry = CUR_STREAM (stream);
++  switch (stsd_entry->fourcc) {
++    case FOURCC_c608:{
++      guint8 *cdat = NULL, *cdt2 = NULL;
++      gsize cdat_size = 0, cdt2_size = 0;
++      /* Should be cdat or cdt2 */
++      if (fourcc != FOURCC_cdat && fourcc != FOURCC_cdt2) {
++        GST_WARNING_OBJECT (stream->pad,
++            "Unknown data atom (%" GST_FOURCC_FORMAT ") for CEA608",
++            GST_FOURCC_ARGS (fourcc));
++        goto invalid_cdat;
++      }
++      /* Convert to cc_data triplet */
++      if (fourcc == FOURCC_cdat)
++        cdat = convert_to_ccdata (data + 8, atom_length - 8, 1, &cdat_size);
++      else
++        cdt2 = convert_to_ccdata (data + 8, atom_length - 8, 2, &cdt2_size);
++      GST_DEBUG_OBJECT (stream->pad, "size:%" G_GSIZE_FORMAT " atom_length:%u",
++          size, atom_length);
++      /* Check for another atom ? */
++      if (size > atom_length + 8) {
++        guint32 new_atom_length = QT_UINT32 (data + atom_length);
++        if (size <= atom_length + new_atom_length) {
++          fourcc = QT_FOURCC (data + atom_length + 4);
++          if (fourcc == FOURCC_cdat)
++            cdat =
++                convert_to_ccdata (data + atom_length + 8, new_atom_length - 8,
++                1, &cdat_size);
++          else
++            cdt2 =
++                convert_to_ccdata (data + atom_length + 8, new_atom_length - 8,
++                2, &cdt2_size);
++        }
++      }
++      *cclen = cdat_size + cdt2_size;
++      res = g_malloc (*cclen);
++      if (cdat_size)
++        memcpy (res, cdat, cdat_size);
++      if (cdt2_size)
++        memcpy (res + cdat_size, cdt2, cdt2_size);
++      g_free (cdat);
++      g_free (cdt2);
++    }
++      break;
++    case FOURCC_c708:
++      if (fourcc != FOURCC_ccdp) {
++        GST_WARNING_OBJECT (stream->pad,
++            "Unknown data atom (%" GST_FOURCC_FORMAT ") for CEA708",
++            GST_FOURCC_ARGS (fourcc));
++        goto invalid_cdat;
++      }
++      *cclen = atom_length - 8;
++      res = g_memdup (data + 8, *cclen);
++      break;
++    default:
++      /* Keep this here in case other closed caption formats are added */
++      g_assert_not_reached ();
++      break;
++  }
++
++  GST_MEMDUMP ("Output", res, *cclen);
++  return res;
++
++  /* Errors */
++invalid_cdat:
++  GST_WARNING ("[cdat] atom is too small or invalid");
++  return NULL;
++}
++
+ /* the input buffer metadata must be writable,
+  * but time/duration etc not yet set and need not be preserved */
+ static GstBuffer *
+@@ -5390,7 +5504,7 @@ gst_qtdemux_process_buffer (GstQTDemux * qtdemux, QtDemuxStream * stream,
+ 
+   if (G_UNLIKELY (stream->subtype != FOURCC_text
+           && stream->subtype != FOURCC_sbtl &&
+-          stream->subtype != FOURCC_subp)) {
++          stream->subtype != FOURCC_subp && stream->subtype != FOURCC_clcp)) {
+     return buf;
+   }
+ 
+@@ -5408,6 +5522,23 @@ gst_qtdemux_process_buffer (GstQTDemux * qtdemux, QtDemuxStream * stream,
+     return buf;
+   }
+ 
++  if (stream->subtype == FOURCC_clcp) {
++    guint8 *cc;
++    gsize cclen = 0;
++    /* For closed caption, we need to extract the information from the
++     * [cdat],[cdt2] or [ccdp] atom */
++    cc = extract_cc_from_data (stream, map.data, map.size, &cclen);
++    gst_buffer_unmap (buf, &map);
++    gst_buffer_unref (buf);
++    if (cc) {
++      buf = _gst_buffer_new_wrapped (cc, cclen, g_free);
++    } else {
++      /* Conversion failed or there's nothing */
++      buf = NULL;
++    }
++    return buf;
++  }
++
+   nsize = GST_READ_UINT16_BE (map.data);
+   nsize = MIN (nsize, map.size - 2);
+ 
+@@ -11557,7 +11688,8 @@ qtdemux_parse_trak (GstQTDemux * qtdemux, GNode * trak)
+       }
+       entry->sampled = TRUE;
+     } else if (stream->subtype == FOURCC_subp || stream->subtype == FOURCC_text
+-        || stream->subtype == FOURCC_sbtl || stream->subtype == FOURCC_subt) {
++        || stream->subtype == FOURCC_sbtl || stream->subtype == FOURCC_subt
++        || stream->subtype == FOURCC_clcp) {
+ 
+       entry->sampled = TRUE;
+       entry->sparse = TRUE;
+@@ -14548,6 +14680,21 @@ qtdemux_sub_caps (GstQTDemux * qtdemux, QtDemuxStream * stream,
+       _codec ("XML subtitles");
+       caps = gst_caps_new_empty_simple ("application/ttml+xml");
+       break;
++    case FOURCC_c608:
++      _codec ("CEA 608 Closed Caption");
++      caps =
++          gst_caps_new_simple ("closedcaption/x-cea-608", "format",
++          G_TYPE_STRING, "cc_data", NULL);
++      stream->need_process = TRUE;
++      break;
++    case FOURCC_c708:
++      _codec ("CEA 708 Closed Caption");
++      caps =
++          gst_caps_new_simple ("closedcaption/x-cea-708", "format",
++          G_TYPE_STRING, "cdp", NULL);
++      stream->need_process = TRUE;
++      break;
++
+     default:
+     {
+       caps = _get_unknown_codec_name ("text", fourcc);
+diff --git a/gst/isomp4/qtdemux_types.c b/gst/isomp4/qtdemux_types.c
+index 88db8c2ce..1d5840394 100644
+--- a/gst/isomp4/qtdemux_types.c
++++ b/gst/isomp4/qtdemux_types.c
+@@ -213,6 +213,7 @@ static const QtNodeType qt_node_types[] = {
+   {FOURCC_pssh, "protection system specific header", 0},
+   {FOURCC_tenc, "track encryption", 0},
+   {FOURCC_stpp, "XML subtitle sample entry", 0},
++  {FOURCC_clcp, "Closed Caption", 0},
+   {0, "unknown", 0,},
+ };
+ 
+-- 
+2.18.0
+

Added: trunk/Tools/gstreamer/patches/gst-plugins-good-0004-qtdemux-Add-initial-support-for-AV1-demuxing.patch (0 => 236165)


--- trunk/Tools/gstreamer/patches/gst-plugins-good-0004-qtdemux-Add-initial-support-for-AV1-demuxing.patch	                        (rev 0)
+++ trunk/Tools/gstreamer/patches/gst-plugins-good-0004-qtdemux-Add-initial-support-for-AV1-demuxing.patch	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,123 @@
+From b6e8c6323e01249bb3ac4f121c3adc3ea8d924bf Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Olivier=20Cr=C3=AAte?= <[email protected]>
+Date: Wed, 25 Jul 2018 10:43:11 -0400
+Subject: [PATCH] qtdemux: Add initial support for AV1 demuxing
+
+Following the spec at
+https://aomediacodec.github.io/av1-isobmff/
+---
+ gst/isomp4/fourcc.h        |  1 +
+ gst/isomp4/qtdemux.c       |  4 ++++
+ gst/isomp4/qtdemux_dump.c  | 49 ++++++++++++++++++++++++++++++++++++++
+ gst/isomp4/qtdemux_types.c |  1 +
+ 4 files changed, 55 insertions(+)
+
+diff --git a/gst/isomp4/fourcc.h b/gst/isomp4/fourcc.h
+index 6f467ffbb..c06b69418 100644
+--- a/gst/isomp4/fourcc.h
++++ b/gst/isomp4/fourcc.h
+@@ -264,6 +264,7 @@ G_BEGIN_DECLS
+ #define FOURCC_wide     GST_MAKE_FOURCC('w','i','d','e')
+ #define FOURCC_zlib     GST_MAKE_FOURCC('z','l','i','b')
+ #define FOURCC_lpcm     GST_MAKE_FOURCC('l','p','c','m')
++#define FOURCC_av01     GST_MAKE_FOURCC('a','v','0','1')
+ 
+ #define FOURCC_cfhd     GST_MAKE_FOURCC('C','F','H','D')
+ #define FOURCC_ap4x     GST_MAKE_FOURCC('a','p','4','x')
+diff --git a/gst/isomp4/qtdemux.c b/gst/isomp4/qtdemux.c
+index 3efb5ba9d..7a682833f 100644
+--- a/gst/isomp4/qtdemux.c
++++ b/gst/isomp4/qtdemux.c
+@@ -14651,6 +14651,10 @@ qtdemux_video_caps (GstQTDemux * qtdemux, QtDemuxStream * stream,
+       caps = gst_caps_new_simple ("video/x-wmv",
+           "wmvversion", G_TYPE_INT, 3, "format", G_TYPE_STRING, "WVC1", NULL);
+       break;
++    case FOURCC_av01:
++      _codec ("AV1");
++      caps = gst_caps_new_empty_simple ("video/x-av1");
++      break;
+     case GST_MAKE_FOURCC ('k', 'p', 'c', 'd'):
+     default:
+     {
+diff --git a/gst/isomp4/qtdemux_dump.c b/gst/isomp4/qtdemux_dump.c
+index 5ff2e9288..25921dfe2 100644
+--- a/gst/isomp4/qtdemux_dump.c
++++ b/gst/isomp4/qtdemux_dump.c
+@@ -303,6 +303,51 @@ qtdemux_dump_stsd_avc1 (GstQTDemux * qtdemux, GstByteReader * data, guint size,
+   return TRUE;
+ }
+ 
++
++static gboolean
++qtdemux_dump_stsd_av01 (GstQTDemux * qtdemux, GstByteReader * data, guint size,
++    int depth)
++{
++  guint compressor_len;
++  char compressor_name[32];
++
++  /* Size of av01 = 78 bytes */
++  if (size < (6 + 2 + 4 + 12 + 2 + 2 + 4 + 4 + 4 + 2 + 1 + 31 + 2 + 2))
++    return FALSE;
++
++  gst_byte_reader_skip_unchecked (data, 6);
++  GST_LOG_OBJECT (qtdemux, "%*s    data reference:%d", depth, "",
++      GET_UINT16 (data));
++  GST_LOG_OBJECT (qtdemux, "%*s    version/rev.:  %08x", depth, "",
++      GET_UINT32 (data));
++  gst_byte_reader_skip_unchecked (data, 12);    /* pre-defined & reserved */
++  GST_LOG_OBJECT (qtdemux, "%*s    width:         %u", depth, "",
++      GET_UINT16 (data));
++  GST_LOG_OBJECT (qtdemux, "%*s    height:        %u", depth, "",
++      GET_UINT16 (data));
++  GST_LOG_OBJECT (qtdemux, "%*s    horiz. resol:  %g", depth, "",
++      GET_FP32 (data));
++  GST_LOG_OBJECT (qtdemux, "%*s    vert. resol.:  %g", depth, "",
++      GET_FP32 (data));
++  GST_LOG_OBJECT (qtdemux, "%*s    data size:     %u", depth, "",
++      GET_UINT32 (data));
++  GST_LOG_OBJECT (qtdemux, "%*s    frame count:   %u", depth, "",
++      GET_UINT16 (data));
++  /* something is not right with this, it's supposed to be a string but it's
++   * not apparently, so just skip this for now */
++  compressor_len = MAX (GET_UINT8 (data), 31);
++  memcpy (compressor_name, gst_byte_reader_get_data_unchecked (data, 31), 31);
++  compressor_name[compressor_len] = 0;
++  GST_LOG_OBJECT (qtdemux, "%*s    compressor:    %s", depth, "",
++      compressor_name);
++  GST_LOG_OBJECT (qtdemux, "%*s    depth:         %u", depth, "",
++      GET_UINT16 (data));
++  GST_LOG_OBJECT (qtdemux, "%*s    color table ID:%u", depth, "",
++      GET_UINT16 (data));
++
++  return TRUE;
++}
++
+ gboolean
+ qtdemux_dump_stsd (GstQTDemux * qtdemux, GstByteReader * data, int depth)
+ {
+@@ -353,6 +398,10 @@ qtdemux_dump_stsd (GstQTDemux * qtdemux, GstByteReader * data, int depth)
+         if (!qtdemux_dump_unknown (qtdemux, &sub, depth + 1))
+           return FALSE;
+         break;
++      case FOURCC_av01:
++        if (!qtdemux_dump_stsd_av01 (qtdemux, &sub, size, depth + 1))
++          return FALSE;
++        break;
+       default:
+         /* Unknown stsd data, dump the bytes */
+         if (!qtdemux_dump_unknown (qtdemux, &sub, depth + 1))
+diff --git a/gst/isomp4/qtdemux_types.c b/gst/isomp4/qtdemux_types.c
+index 1d5840394..68f98bd45 100644
+--- a/gst/isomp4/qtdemux_types.c
++++ b/gst/isomp4/qtdemux_types.c
+@@ -214,6 +214,7 @@ static const QtNodeType qt_node_types[] = {
+   {FOURCC_tenc, "track encryption", 0},
+   {FOURCC_stpp, "XML subtitle sample entry", 0},
+   {FOURCC_clcp, "Closed Caption", 0},
++  {FOURCC_av01, "AV1 Sample Entry", 0},
+   {0, "unknown", 0,},
+ };
+ 
+-- 
+2.18.0
+

Added: trunk/Tools/gstreamer/patches/gst-plugins-good-0005-qtdemux-Extract-AV1-codec_data-and-put-it-in-the-cap.patch (0 => 236165)


--- trunk/Tools/gstreamer/patches/gst-plugins-good-0005-qtdemux-Extract-AV1-codec_data-and-put-it-in-the-cap.patch	                        (rev 0)
+++ trunk/Tools/gstreamer/patches/gst-plugins-good-0005-qtdemux-Extract-AV1-codec_data-and-put-it-in-the-cap.patch	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,112 @@
+From 43ee4055c3d03a9f99acdf3c05f45e154899edef Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Olivier=20Cr=C3=AAte?= <[email protected]>
+Date: Wed, 25 Jul 2018 14:31:39 -0400
+Subject: [PATCH] qtdemux: Extract AV1 codec_data and put it in the caps
+
+Also extract the presentation-delay and put it in the caps.
+---
+ gst/isomp4/fourcc.h  |  1 +
+ gst/isomp4/qtdemux.c | 75 ++++++++++++++++++++++++++++++++++++++++++++
+ 2 files changed, 76 insertions(+)
+
+diff --git a/gst/isomp4/fourcc.h b/gst/isomp4/fourcc.h
+index c06b69418..e18e6c2c9 100644
+--- a/gst/isomp4/fourcc.h
++++ b/gst/isomp4/fourcc.h
+@@ -265,6 +265,7 @@ G_BEGIN_DECLS
+ #define FOURCC_zlib     GST_MAKE_FOURCC('z','l','i','b')
+ #define FOURCC_lpcm     GST_MAKE_FOURCC('l','p','c','m')
+ #define FOURCC_av01     GST_MAKE_FOURCC('a','v','0','1')
++#define FOURCC_av1C     GST_MAKE_FOURCC('a','v','1','C')
+ 
+ #define FOURCC_cfhd     GST_MAKE_FOURCC('C','F','H','D')
+ #define FOURCC_ap4x     GST_MAKE_FOURCC('a','p','4','x')
+diff --git a/gst/isomp4/qtdemux.c b/gst/isomp4/qtdemux.c
+index 7a682833f..d1ce320b8 100644
+--- a/gst/isomp4/qtdemux.c
++++ b/gst/isomp4/qtdemux.c
+@@ -11187,6 +11187,81 @@ qtdemux_parse_trak (GstQTDemux * qtdemux, GNode * trak)
+             }
+             break;
+           }
++          case FOURCC_av01:
++          {
++            gint len = QT_UINT32 (stsd_entry_data) - 0x56;
++            const guint8 *av1_data = stsd_entry_data + 0x56;
++
++            /* find av1C */
++            while (len >= 0x8) {
++              gint size;
++
++              if (QT_UINT32 (av1_data) <= len)
++                size = QT_UINT32 (av1_data) - 0x8;
++              else
++                size = len - 0x8;
++
++              if (size < 1)
++                /* No real data, so break out */
++                break;
++
++              switch (QT_FOURCC (av1_data + 0x4)) {
++                case FOURCC_av1C:
++                {
++                  /* parse, if found */
++                  GstBuffer *buf;
++                  guint8 pres_delay_field;
++
++                  GST_DEBUG_OBJECT (qtdemux,
++                      "found av1C codec_data in stsd of size %d", size);
++
++                  /* not enough data, just ignore and hope for the best */
++                  if (size < 5)
++                    break;
++
++                  /* Content is:
++                   * 4 bytes: atom length
++                   * 4 bytes: fourcc
++                   * 1 byte: version
++                   * 3 bytes: flags
++                   * 3 bits: reserved
++                   * 1 bits:  initial_presentation_delay_present
++                   * 4 bits: initial_presentation_delay (if present else reserved
++                   * rest: OBUs.
++                   */
++
++                  if (av1_data[9] != 0) {
++                    GST_WARNING ("Unknown version %d of av1C box", av1_data[9]);
++                    break;
++                  }
++
++                  /* We skip initial_presentation_delay* for now */
++                  pres_delay_field = *(av1_data + 12);
++                  if (pres_delay_field & (1 << 5)) {
++                    gst_caps_set_simple (entry->caps,
++                        "presentation-delay", G_TYPE_INT,
++                        (gint) (pres_delay_field & 0x0F) + 1, NULL);
++                  }
++                  if (size > 5) {
++                    buf = gst_buffer_new_and_alloc (size - 5);
++                    GST_BUFFER_FLAG_SET (buf, GST_BUFFER_FLAG_HEADER);
++                    gst_buffer_fill (buf, 0, av1_data + 13, size - 5);
++                    gst_caps_set_simple (entry->caps,
++                        "codec_data", GST_TYPE_BUFFER, buf, NULL);
++                    gst_buffer_unref (buf);
++                  }
++                  break;
++                }
++                default:
++                  break;
++              }
++
++              len -= size + 8;
++              av1_data += size + 8;
++            }
++
++            break;
++          }
+           default:
+             break;
+         }
+-- 
+2.18.0
+

Added: trunk/Tools/gstreamer/patches/gst-plugins-good-0006-qtdemux-Recognize-more-AV1-atoms.patch (0 => 236165)


--- trunk/Tools/gstreamer/patches/gst-plugins-good-0006-qtdemux-Recognize-more-AV1-atoms.patch	                        (rev 0)
+++ trunk/Tools/gstreamer/patches/gst-plugins-good-0006-qtdemux-Recognize-more-AV1-atoms.patch	2018-09-19 08:53:50 UTC (rev 236165)
@@ -0,0 +1,44 @@
+From e07ce9afb4edb2ea898763c7ea11f0af2704104e Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Olivier=20Cr=C3=AAte?= <[email protected]>
+Date: Wed, 25 Jul 2018 16:51:38 -0400
+Subject: [PATCH] qtdemux: Recognize more AV1 atoms
+
+---
+ gst/isomp4/fourcc.h        | 4 ++++
+ gst/isomp4/qtdemux_types.c | 5 +++++
+ 2 files changed, 9 insertions(+)
+
+diff --git a/gst/isomp4/fourcc.h b/gst/isomp4/fourcc.h
+index e18e6c2c9..6c425e6f9 100644
+--- a/gst/isomp4/fourcc.h
++++ b/gst/isomp4/fourcc.h
+@@ -266,6 +266,10 @@ G_BEGIN_DECLS
+ #define FOURCC_lpcm     GST_MAKE_FOURCC('l','p','c','m')
+ #define FOURCC_av01     GST_MAKE_FOURCC('a','v','0','1')
+ #define FOURCC_av1C     GST_MAKE_FOURCC('a','v','1','C')
++#define FOURCC_av1f     GST_MAKE_FOURCC('a','v','1','f')
++#define FOURCC_av1m     GST_MAKE_FOURCC('a','v','1','m')
++#define FOURCC_av1s     GST_MAKE_FOURCC('a','v','1','s')
++#define FOURCC_av1M     GST_MAKE_FOURCC('a','v','1','M')
+ 
+ #define FOURCC_cfhd     GST_MAKE_FOURCC('C','F','H','D')
+ #define FOURCC_ap4x     GST_MAKE_FOURCC('a','p','4','x')
+diff --git a/gst/isomp4/qtdemux_types.c b/gst/isomp4/qtdemux_types.c
+index 68f98bd45..9ab2e7e3e 100644
+--- a/gst/isomp4/qtdemux_types.c
++++ b/gst/isomp4/qtdemux_types.c
+@@ -215,6 +215,11 @@ static const QtNodeType qt_node_types[] = {
+   {FOURCC_stpp, "XML subtitle sample entry", 0},
+   {FOURCC_clcp, "Closed Caption", 0},
+   {FOURCC_av01, "AV1 Sample Entry", 0},
++  {FOURCC_av1C, "AV1 Codec Configuration", 0},
++  {FOURCC_av1f, "AV1 Forward Key Frame sample group entry", 0},
++  {FOURCC_av1m, "AV1 Multi-Frame sample group entry", 0},
++  {FOURCC_av1s, "AV1 S-Frame sample group entry", 0},
++  {FOURCC_av1M, "AV1 Metadata sample group entry", 0},
+   {0, "unknown", 0,},
+ };
+ 
+-- 
+2.18.0
+
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to