Title: [270294] trunk
Revision
270294
Author
[email protected]
Date
2020-12-01 04:34:23 -0800 (Tue, 01 Dec 2020)

Log Message

Allow RTCRtpScriptTransformer to request key frames
https://bugs.webkit.org/show_bug.cgi?id=219199
Source/ThirdParty/libwebrtc:

Reviewed by Eric Carlson.

Add GenerateKeyFrame to sender and receiver interface.
Pipe the request to the necessary underlying objects implementing this as follows:
- Ask encoder to generate a key frame for sender transforms
- Send a FIR network request from receiver to sender for receiver transforms

* Source/webrtc/api/rtp_receiver_interface.h:
* Source/webrtc/api/rtp_sender_interface.h:
* Source/webrtc/call/video_send_stream.h:
* Source/webrtc/media/base/media_channel.h:
(cricket::MediaChannel::GenerateKeyFrame):
* Source/webrtc/media/engine/webrtc_video_engine.cc:
* Source/webrtc/media/engine/webrtc_video_engine.h:
* Source/webrtc/pc/rtp_sender.cc:
* Source/webrtc/pc/rtp_sender.h:
* Source/webrtc/pc/video_rtp_receiver.cc:
* Source/webrtc/pc/video_rtp_receiver.h:
* Source/webrtc/video/video_send_stream.cc:
* Source/webrtc/video/video_send_stream.h:

Source/WebCore:

<rdar://problem/71763010>

Reviewed by Eric Carlson.

Expose RTCRtpScriptTransformerContext to RTCRtpScriptTransformer as part of third parameter of the 'start' callback.
This context exposes some states (sender or receiver transform, audio or video transform).
It also exposes the ability to request key frames for video transforms.

Make RTCRtpTransformableFrame ref counted so that we can keep it even though the frame was piped to the writable stream.
Make LibWebRTCRtpTransformableFrame able to cope with a nullptr internal rtc frame.

Add support for RTCEncodedVideoFrame.type.

Tests: http/wpt/webrtc/audio-script-transform.html
       http/wpt/webrtc/video-script-transform.html

* CMakeLists.txt:
* DerivedSources-input.xcfilelist:
* DerivedSources-output.xcfilelist:
* DerivedSources.make:
* Modules/mediastream/RTCEncodedVideoFrame.cpp:
(WebCore::RTCEncodedVideoFrame::RTCEncodedVideoFrame):
* Modules/mediastream/RTCEncodedVideoFrame.h:
(WebCore::RTCEncodedVideoFrame::type const):
* Modules/mediastream/RTCEncodedVideoFrame.idl:
* Modules/mediastream/RTCRtpScriptTransformerContext.h: Added.
(WebCore::RTCRtpScriptTransformerContext::side const):
(WebCore::RTCRtpScriptTransformerContext::mediaType const):
(WebCore::RTCRtpScriptTransformerContext::create):
(WebCore::RTCRtpScriptTransformerContext::RTCRtpScriptTransformerContext):
(WebCore::RTCRtpScriptTransformerContext::requestKeyFrame):
* Modules/mediastream/RTCRtpScriptTransformerContext.idl: Added.
* Modules/mediastream/RTCRtpTransformBackend.h:
* Modules/mediastream/RTCRtpTransformableFrame.h:
* Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.cpp:
(WebCore::LibWebRTCRtpReceiverTransformBackend::requestKeyFrame):
* Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.h:
* Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.cpp:
(WebCore::LibWebRTCRtpSenderTransformBackend::requestKeyFrame):
* Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.h:
* Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.cpp:
(WebCore::LibWebRTCRtpTransformableFrame::isKeyFrame const):
* Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.h:
* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:

LayoutTests:

Reviewed by Eric Carlson.

* http/wpt/webrtc/audio-script-transform-expected.txt: Added.
* http/wpt/webrtc/audio-script-transform.html: Added.
* http/wpt/webrtc/video-script-transform-expected.txt: Added.
* http/wpt/webrtc/video-script-transform.html: Added.
* http/wpt/webrtc/context-transform.js: Added.

Modified Paths

Added Paths

Diff

Modified: trunk/LayoutTests/ChangeLog (270293 => 270294)


--- trunk/LayoutTests/ChangeLog	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/LayoutTests/ChangeLog	2020-12-01 12:34:23 UTC (rev 270294)
@@ -1,5 +1,18 @@
 2020-12-01  Youenn Fablet  <[email protected]>
 
+        Allow RTCRtpScriptTransformer to request key frames
+        https://bugs.webkit.org/show_bug.cgi?id=219199
+
+        Reviewed by Eric Carlson.
+
+        * http/wpt/webrtc/audio-script-transform-expected.txt: Added.
+        * http/wpt/webrtc/audio-script-transform.html: Added.
+        * http/wpt/webrtc/video-script-transform-expected.txt: Added.
+        * http/wpt/webrtc/video-script-transform.html: Added.
+        * http/wpt/webrtc/context-transform.js: Added.
+
+2020-12-01  Youenn Fablet  <[email protected]>
+
         Add support for readable/writable to RTCRtpSFrameTransform
         https://bugs.webkit.org/show_bug.cgi?id=219298
 

Added: trunk/LayoutTests/http/wpt/webrtc/audio-script-transform-expected.txt (0 => 270294)


--- trunk/LayoutTests/http/wpt/webrtc/audio-script-transform-expected.txt	                        (rev 0)
+++ trunk/LayoutTests/http/wpt/webrtc/audio-script-transform-expected.txt	2020-12-01 12:34:23 UTC (rev 270294)
@@ -0,0 +1,5 @@
+
+
+PASS setup and check context
+PASS audio is playing
+

Added: trunk/LayoutTests/http/wpt/webrtc/audio-script-transform.html (0 => 270294)


--- trunk/LayoutTests/http/wpt/webrtc/audio-script-transform.html	                        (rev 0)
+++ trunk/LayoutTests/http/wpt/webrtc/audio-script-transform.html	2020-12-01 12:34:23 UTC (rev 270294)
@@ -0,0 +1,66 @@
+<!doctype html>
+<html>
+    <head>
+        <meta charset="utf-8">
+        <script src=""
+        <script src=""
+    </head>
+    <body>
+        <video id="video" autoplay playsInline></video>
+        <script src=""
+        <script>
+function waitForMessage(port, data)
+{
+    let gotMessage;
+    const promise = new Promise((resolve, reject) => {
+        gotMessage = resolve;
+        setTimeout(() => { reject("did not get " + data) }, 5000);
+    });
+    port._onmessage_ = event => {
+       if (event.data ="" data)
+           gotMessage();
+    };
+    return promise;
+}
+
+let senderTransform, receiverTransform;
+let stream;
+promise_test(async (test) => {
+    worker = new Worker("context-transform.js");
+    const data = "" new Promise(resolve => worker._onmessage_ = (event) => resolve(event.data));
+    assert_equals(data, "registered");
+
+    const localStream = await navigator.mediaDevices.getUserMedia({audio: true});
+
+    let sender, receiver;
+    senderTransform = new RTCRtpScriptTransform(worker, 'MockRTCRtpTransform');
+    receiverTransform = new RTCRtpScriptTransform(worker, 'MockRTCRtpTransform');
+
+    promise1 = waitForMessage(senderTransform.port, "started audio sender");
+    promise2 = waitForMessage(receiverTransform.port, "started audio receiver");
+
+    stream = await new Promise((resolve, reject) => {
+        createConnections((firstConnection) => {
+            sender = firstConnection.addTrack(localStream.getAudioTracks()[0], localStream);
+            sender.transform = senderTransform;
+        }, (secondConnection) => {
+            secondConnection._ontrack_ = (trackEvent) => {
+                receiver = trackEvent.receiver;
+                receiver.transform = receiverTransform;
+                resolve(trackEvent.streams[0]);
+            };
+        });
+        setTimeout(() => reject("Test timed out"), 5000);
+    });
+
+    await promise1;
+    await promise2;
+}, "setup and check context");
+
+promise_test((test) => {
+    video.srcObject = stream;
+    return video.play();
+}, "audio is playing");
+        </script>
+    </body>
+</html>

Added: trunk/LayoutTests/http/wpt/webrtc/context-transform.js (0 => 270294)


--- trunk/LayoutTests/http/wpt/webrtc/context-transform.js	                        (rev 0)
+++ trunk/LayoutTests/http/wpt/webrtc/context-transform.js	2020-12-01 12:34:23 UTC (rev 270294)
@@ -0,0 +1,43 @@
+class MockRTCRtpTransformer extends RTCRtpScriptTransformer {
+    constructor() {
+        super();
+        this.askKeyFrame = false;
+        this.port._onmessage_ = (event) => {
+            if (event.data ="" "startKeyFrames")
+                this.askKeyFrame = true;
+            else if (event.data ="" "endKeyFrames")
+                this.askKeyFrame = false;
+        };
+    }
+    start(readableStream, writableStream, context)
+    {
+        this.reader = readableStream.getReader();
+        this.writer = writableStream.getWriter();
+        this.process();
+        this.context = context;
+        this.port.postMessage("started " + context.mediaType + " " + context.side);
+    }
+
+    process()
+    {
+        this.reader.read().then(chunk => {
+            if (chunk.done)
+                return;
+
+            this.writer.write(chunk.value);
+
+            if (this.context.mediaType === "video") {
+                if (chunk.value instanceof RTCEncodedVideoFrame)
+                    this.port.postMessage("video frame " + chunk.value.type);
+
+                if (this.askKeyFrame)
+                    this.context.requestKeyFrame();
+            }
+
+            this.process();
+        });
+    }
+};
+
+registerRTCRtpScriptTransformer("MockRTCRtpTransform", MockRTCRtpTransformer);
+self.postMessage("registered");

Added: trunk/LayoutTests/http/wpt/webrtc/video-script-transform-expected.txt (0 => 270294)


--- trunk/LayoutTests/http/wpt/webrtc/video-script-transform-expected.txt	                        (rev 0)
+++ trunk/LayoutTests/http/wpt/webrtc/video-script-transform-expected.txt	2020-12-01 12:34:23 UTC (rev 270294)
@@ -0,0 +1,8 @@
+
+
+PASS setup
+PASS video is playing
+PASS video frame type
+PASS key frame requests from sender
+PASS key frame requests from receiver
+

Added: trunk/LayoutTests/http/wpt/webrtc/video-script-transform.html (0 => 270294)


--- trunk/LayoutTests/http/wpt/webrtc/video-script-transform.html	                        (rev 0)
+++ trunk/LayoutTests/http/wpt/webrtc/video-script-transform.html	2020-12-01 12:34:23 UTC (rev 270294)
@@ -0,0 +1,87 @@
+<!doctype html>
+<html>
+    <head>
+        <meta charset="utf-8">
+        <script src=""
+        <script src=""
+    </head>
+    <body>
+        <video id="video" autoplay playsInline></video>
+        <script src=""
+        <script>
+function waitForMessage(port, data)
+{
+    let gotMessage;
+    const promise = new Promise((resolve, reject) => {
+        gotMessage = resolve;
+        setTimeout(() => { reject("did not get " + data) }, 5000);
+    });
+    port._onmessage_ = event => {
+       if (event.data ="" data)
+           gotMessage();
+    };
+    return promise;
+}
+
+let senderTransform, receiverTransform;
+let stream;
+promise_test(async (test) => {
+    worker = new Worker('context-transform.js');
+    const data = "" new Promise(resolve => worker._onmessage_ = (event) => resolve(event.data));
+    assert_equals(data, "registered");
+
+    const localStream = await navigator.mediaDevices.getUserMedia({video: true});
+
+    let sender, receiver;
+    senderTransform = new RTCRtpScriptTransform(worker, 'MockRTCRtpTransform');
+    receiverTransform = new RTCRtpScriptTransform(worker, 'MockRTCRtpTransform');
+
+    promise1 = waitForMessage(senderTransform.port, "started video sender");
+    promise2 = waitForMessage(receiverTransform.port, "started video receiver");
+
+    stream = await new Promise((resolve, reject) => {
+        createConnections((firstConnection) => {
+            sender = firstConnection.addTrack(localStream.getVideoTracks()[0], localStream);
+            sender.transform = senderTransform;
+        }, (secondConnection) => {
+            secondConnection._ontrack_ = (trackEvent) => {
+                receiver = trackEvent.receiver;
+                receiver.transform = receiverTransform;
+                resolve(trackEvent.streams[0]);
+            };
+        });
+        setTimeout(() => reject("Test timed out"), 5000);
+    });
+
+    await promise1;
+    await promise2;
+}, "setup");
+
+promise_test((test) => {
+    video.srcObject = stream;
+    return video.play();
+}, "video is playing");
+
+promise_test(async (test) => {
+    await waitForMessage(receiverTransform.port, "video frame delta");
+}, "video frame type");
+
+promise_test(async (test) => {
+    senderTransform.port.postMessage("startKeyFrames");
+    await waitForMessage(senderTransform.port, "video frame key");
+    await waitForMessage(receiverTransform.port, "video frame key");
+    senderTransform.port.postMessage("endKeyFrames");
+
+    await waitForMessage(receiverTransform.port, "video frame delta");
+}, "key frame requests from sender");
+
+promise_test(async (test) => {
+    receiverTransform.port.postMessage("startKeyFrames");
+    await waitForMessage(senderTransform.port, "video frame key");
+    await waitForMessage(receiverTransform.port, "video frame key");
+    receiverTransform.port.postMessage("endKeyFrames");
+
+}, "key frame requests from receiver");
+        </script>
+    </body>
+</html>

Modified: trunk/Source/ThirdParty/libwebrtc/ChangeLog (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/ChangeLog	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/ChangeLog	2020-12-01 12:34:23 UTC (rev 270294)
@@ -1,3 +1,29 @@
+2020-12-01  Youenn Fablet  <[email protected]>
+
+        Allow RTCRtpScriptTransformer to request key frames
+        https://bugs.webkit.org/show_bug.cgi?id=219199
+
+        Reviewed by Eric Carlson.
+
+        Add GenerateKeyFrame to sender and receiver interface.
+        Pipe the request to the necessary underlying objects implementing this as follows:
+        - Ask encoder to generate a key frame for sender transforms
+        - Send a FIR network request from receiver to sender for receiver transforms
+
+        * Source/webrtc/api/rtp_receiver_interface.h:
+        * Source/webrtc/api/rtp_sender_interface.h:
+        * Source/webrtc/call/video_send_stream.h:
+        * Source/webrtc/media/base/media_channel.h:
+        (cricket::MediaChannel::GenerateKeyFrame):
+        * Source/webrtc/media/engine/webrtc_video_engine.cc:
+        * Source/webrtc/media/engine/webrtc_video_engine.h:
+        * Source/webrtc/pc/rtp_sender.cc:
+        * Source/webrtc/pc/rtp_sender.h:
+        * Source/webrtc/pc/video_rtp_receiver.cc:
+        * Source/webrtc/pc/video_rtp_receiver.h:
+        * Source/webrtc/video/video_send_stream.cc:
+        * Source/webrtc/video/video_send_stream.h:
+
 2020-11-30  Youenn Fablet  <[email protected]>
 
         Introduce an experimental flag specific to VP9 profile 2

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/api/rtp_receiver_interface.h (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/api/rtp_receiver_interface.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/api/rtp_receiver_interface.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -113,6 +113,9 @@
   // logic.
   virtual void SetDepacketizerToDecoderFrameTransformer(
       rtc::scoped_refptr<FrameTransformerInterface> frame_transformer);
+#if defined(WEBRTC_WEBKIT_BUILD)
+  virtual void GenerateKeyFrame() { }
+#endif
 
  protected:
   ~RtpReceiverInterface() override = default;
@@ -142,6 +145,9 @@
 PROXY_METHOD1(void,
               SetDepacketizerToDecoderFrameTransformer,
               rtc::scoped_refptr<FrameTransformerInterface>)
+#if defined(WEBRTC_WEBKIT_BUILD)
+PROXY_METHOD0(void, GenerateKeyFrame)
+#endif
 END_PROXY_MAP()
 
 }  // namespace webrtc

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/api/rtp_sender_interface.h (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/api/rtp_sender_interface.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/api/rtp_sender_interface.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -96,6 +96,9 @@
 
   virtual void SetEncoderToPacketizerFrameTransformer(
       rtc::scoped_refptr<FrameTransformerInterface> frame_transformer);
+#if defined(WEBRTC_WEBKIT_BUILD)
+  virtual void GenerateKeyFrame() { }
+#endif
 
  protected:
   ~RtpSenderInterface() override = default;
@@ -126,6 +129,9 @@
 PROXY_METHOD1(void,
               SetEncoderToPacketizerFrameTransformer,
               rtc::scoped_refptr<FrameTransformerInterface>)
+#if defined(WEBRTC_WEBKIT_BUILD)
+PROXY_METHOD0(void, GenerateKeyFrame)
+#endif
 END_PROXY_MAP()
 
 }  // namespace webrtc

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/call/video_send_stream.h (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/call/video_send_stream.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/call/video_send_stream.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -238,6 +238,10 @@
 
   virtual Stats GetStats() = 0;
 
+#if defined(WEBRTC_WEBKIT_BUILD)
+  virtual void GenerateKeyFrame() = 0;
+#endif
+
  protected:
   virtual ~VideoSendStream() {}
 };

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/webrtc_video_engine.cc (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/webrtc_video_engine.cc	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/webrtc_video_engine.cc	2020-12-01 12:34:23 UTC (rev 270294)
@@ -2644,6 +2644,14 @@
     RecreateWebRtcStream();
 }
 
+#if defined(WEBRTC_WEBKIT_BUILD)
+void WebRtcVideoChannel::WebRtcVideoSendStream::GenerateKeyFrame() {
+  RTC_DCHECK_RUN_ON(&thread_checker_);
+  if (stream_ != NULL)
+    stream_->GenerateKeyFrame();
+}
+#endif
+
 void WebRtcVideoChannel::WebRtcVideoSendStream::RecreateWebRtcStream() {
   RTC_DCHECK_RUN_ON(&thread_checker_);
   if (stream_ != NULL) {
@@ -3316,11 +3324,20 @@
   WebRtcVideoReceiveStream* stream = FindReceiveStream(ssrc);
   if (stream) {
     stream->GenerateKeyFrame();
-  } else {
-    RTC_LOG(LS_ERROR)
-        << "Absent receive stream; ignoring key frame generation for ssrc "
-        << ssrc;
+    return;
   }
+#if defined(WEBRTC_WEBKIT_BUILD)
+  if (ssrc != 0) {
+    auto matching_stream = send_streams_.find(ssrc);
+    if (matching_stream != send_streams_.end()) {
+      matching_stream->second->GenerateKeyFrame();
+      return;
+    }
+  }
+#endif
+  RTC_LOG(LS_ERROR)
+      << "Absent receive stream; ignoring key frame generation for ssrc "
+      << ssrc;
 }
 
 void WebRtcVideoChannel::SetEncoderToPacketizerFrameTransformer(

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/webrtc_video_engine.h (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/webrtc_video_engine.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/media/engine/webrtc_video_engine.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -359,6 +359,9 @@
     void SetEncoderToPacketizerFrameTransformer(
         rtc::scoped_refptr<webrtc::FrameTransformerInterface>
             frame_transformer);
+#if defined(WEBRTC_WEBKIT_BUILD)
+    void GenerateKeyFrame();
+#endif
 
    private:
     // Parameters needed to reconstruct the underlying stream.

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/rtp_sender.cc (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/rtp_sender.cc	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/rtp_sender.cc	2020-12-01 12:34:23 UTC (rev 270294)
@@ -647,4 +647,15 @@
   });
 }
 
+#if defined(WEBRTC_WEBKIT_BUILD)
+void VideoRtpSender::GenerateKeyFrame()
+{
+  if (video_media_channel() && ssrc_ && !stopped_) {
+    worker_thread_->Invoke<void>(RTC_FROM_HERE, [&] {
+        video_media_channel()->GenerateKeyFrame(ssrc_);
+    });
+  }
+}
+#endif
+
 }  // namespace webrtc

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/rtp_sender.h (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/rtp_sender.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/rtp_sender.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -348,6 +348,10 @@
   // Hook to allow custom logic when tracks are attached.
   void AttachTrack() override;
 
+#if defined(WEBRTC_WEBKIT_BUILD)
+  void GenerateKeyFrame() override;
+#endif
+
  private:
   cricket::VideoMediaChannel* video_media_channel() {
     return static_cast<cricket::VideoMediaChannel*>(media_channel_);

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/video_rtp_receiver.cc (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/video_rtp_receiver.cc	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/video_rtp_receiver.cc	2020-12-01 12:34:23 UTC (rev 270294)
@@ -115,6 +115,18 @@
   });
 }
 
+#if defined(WEBRTC_WEBKIT_BUILD)
+void VideoRtpReceiver::GenerateKeyFrame()
+{
+  worker_thread_->Invoke<void>(RTC_FROM_HERE, [&] {
+    RTC_DCHECK_RUN_ON(worker_thread_);
+    if (media_channel_ && !stopped_) {
+      media_channel_->GenerateKeyFrame(ssrc_.value_or(0));
+    }
+  });
+}
+#endif
+
 void VideoRtpReceiver::Stop() {
   // TODO(deadbeef): Need to do more here to fully stop receiving packets.
   if (stopped_) {

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/video_rtp_receiver.h (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/video_rtp_receiver.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/pc/video_rtp_receiver.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -88,6 +88,9 @@
 
   void SetDepacketizerToDecoderFrameTransformer(
       rtc::scoped_refptr<FrameTransformerInterface> frame_transformer) override;
+#if defined(WEBRTC_WEBKIT_BUILD)
+  void GenerateKeyFrame() override;
+#endif
 
   // RtpReceiverInternal implementation.
   void Stop() override;

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/video/video_send_stream.cc (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/video/video_send_stream.cc	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/video/video_send_stream.cc	2020-12-01 12:34:23 UTC (rev 270294)
@@ -240,5 +240,14 @@
   send_stream_->DeliverRtcp(packet, length);
 }
 
+#if defined(WEBRTC_WEBKIT_BUILD)
+void VideoSendStream::GenerateKeyFrame()
+{
+  RTC_DCHECK_RUN_ON(&thread_checker_);
+  if (video_stream_encoder_)
+      video_stream_encoder_->SendKeyFrame();
+}
+#endif
+
 }  // namespace internal
 }  // namespace webrtc

Modified: trunk/Source/ThirdParty/libwebrtc/Source/webrtc/video/video_send_stream.h (270293 => 270294)


--- trunk/Source/ThirdParty/libwebrtc/Source/webrtc/video/video_send_stream.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/ThirdParty/libwebrtc/Source/webrtc/video/video_send_stream.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -86,6 +86,9 @@
 
   void ReconfigureVideoEncoder(VideoEncoderConfig) override;
   Stats GetStats() override;
+#if defined(WEBRTC_WEBKIT_BUILD)
+  void GenerateKeyFrame() override;
+#endif
 
   void StopPermanentlyAndGetRtpStates(RtpStateMap* rtp_state_map,
                                       RtpPayloadStateMap* payload_state_map);

Modified: trunk/Source/WebCore/CMakeLists.txt (270293 => 270294)


--- trunk/Source/WebCore/CMakeLists.txt	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/CMakeLists.txt	2020-12-01 12:34:23 UTC (rev 270294)
@@ -410,6 +410,7 @@
     Modules/mediastream/RTCRtpScriptTransformProvider.idl
     Modules/mediastream/RTCRtpScriptTransformer.idl
     Modules/mediastream/RTCRtpScriptTransformerConstructor.idl
+    Modules/mediastream/RTCRtpScriptTransformerContext.idl
     Modules/mediastream/RTCRtpSendParameters.idl
     Modules/mediastream/RTCRtpSender+Transform.idl
     Modules/mediastream/RTCRtpSender.idl

Modified: trunk/Source/WebCore/ChangeLog (270293 => 270294)


--- trunk/Source/WebCore/ChangeLog	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/ChangeLog	2020-12-01 12:34:23 UTC (rev 270294)
@@ -1,3 +1,53 @@
+2020-12-01  Youenn Fablet  <[email protected]>
+
+        Allow RTCRtpScriptTransformer to request key frames
+        https://bugs.webkit.org/show_bug.cgi?id=219199
+        <rdar://problem/71763010>
+
+        Reviewed by Eric Carlson.
+
+        Expose RTCRtpScriptTransformerContext to RTCRtpScriptTransformer as part of third parameter of the 'start' callback.
+        This context exposes some states (sender or receiver transform, audio or video transform).
+        It also exposes the ability to request key frames for video transforms.
+
+        Make RTCRtpTransformableFrame ref counted so that we can keep it even though the frame was piped to the writable stream.
+        Make LibWebRTCRtpTransformableFrame able to cope with a nullptr internal rtc frame.
+
+        Add support for RTCEncodedVideoFrame.type.
+
+        Tests: http/wpt/webrtc/audio-script-transform.html
+               http/wpt/webrtc/video-script-transform.html
+
+        * CMakeLists.txt:
+        * DerivedSources-input.xcfilelist:
+        * DerivedSources-output.xcfilelist:
+        * DerivedSources.make:
+        * Modules/mediastream/RTCEncodedVideoFrame.cpp:
+        (WebCore::RTCEncodedVideoFrame::RTCEncodedVideoFrame):
+        * Modules/mediastream/RTCEncodedVideoFrame.h:
+        (WebCore::RTCEncodedVideoFrame::type const):
+        * Modules/mediastream/RTCEncodedVideoFrame.idl:
+        * Modules/mediastream/RTCRtpScriptTransformerContext.h: Added.
+        (WebCore::RTCRtpScriptTransformerContext::side const):
+        (WebCore::RTCRtpScriptTransformerContext::mediaType const):
+        (WebCore::RTCRtpScriptTransformerContext::create):
+        (WebCore::RTCRtpScriptTransformerContext::RTCRtpScriptTransformerContext):
+        (WebCore::RTCRtpScriptTransformerContext::requestKeyFrame):
+        * Modules/mediastream/RTCRtpScriptTransformerContext.idl: Added.
+        * Modules/mediastream/RTCRtpTransformBackend.h:
+        * Modules/mediastream/RTCRtpTransformableFrame.h:
+        * Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.cpp:
+        (WebCore::LibWebRTCRtpReceiverTransformBackend::requestKeyFrame):
+        * Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.h:
+        * Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.cpp:
+        (WebCore::LibWebRTCRtpSenderTransformBackend::requestKeyFrame):
+        * Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.h:
+        * Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.cpp:
+        (WebCore::LibWebRTCRtpTransformableFrame::isKeyFrame const):
+        * Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.h:
+        * Sources.txt:
+        * WebCore.xcodeproj/project.pbxproj:
+
 2020-12-01  Rob Buis  <[email protected]>
 
         Remove m_gridItemsIndexesMap

Modified: trunk/Source/WebCore/DerivedSources-input.xcfilelist (270293 => 270294)


--- trunk/Source/WebCore/DerivedSources-input.xcfilelist	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/DerivedSources-input.xcfilelist	2020-12-01 12:34:23 UTC (rev 270294)
@@ -244,6 +244,7 @@
 $(PROJECT_DIR)/Modules/mediastream/RTCRtpScriptTransformProvider.idl
 $(PROJECT_DIR)/Modules/mediastream/RTCRtpScriptTransformer.idl
 $(PROJECT_DIR)/Modules/mediastream/RTCRtpScriptTransformerConstructor.idl
+$(PROJECT_DIR)/Modules/mediastream/RTCRtpScriptTransformerContext.idl
 $(PROJECT_DIR)/Modules/mediastream/RTCRtpSendParameters.idl
 $(PROJECT_DIR)/Modules/mediastream/RTCRtpSender+Transform.idl
 $(PROJECT_DIR)/Modules/mediastream/RTCRtpSender.idl

Modified: trunk/Source/WebCore/DerivedSources-output.xcfilelist (270293 => 270294)


--- trunk/Source/WebCore/DerivedSources-output.xcfilelist	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/DerivedSources-output.xcfilelist	2020-12-01 12:34:23 UTC (rev 270294)
@@ -1717,6 +1717,8 @@
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpScriptTransformer.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpScriptTransformerConstructor.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpScriptTransformerConstructor.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpScriptTransformerContext.cpp
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpScriptTransformerContext.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpSendParameters.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpSendParameters.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebCore/JSRTCRtpSender+Transform.cpp

Modified: trunk/Source/WebCore/DerivedSources.make (270293 => 270294)


--- trunk/Source/WebCore/DerivedSources.make	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/DerivedSources.make	2020-12-01 12:34:23 UTC (rev 270294)
@@ -271,6 +271,7 @@
     $(WebCore)/Modules/mediastream/RTCRtpScriptTransformProvider.idl \
     $(WebCore)/Modules/mediastream/RTCRtpScriptTransformer.idl \
     $(WebCore)/Modules/mediastream/RTCRtpScriptTransformerConstructor.idl \
+    $(WebCore)/Modules/mediastream/RTCRtpScriptTransformerContext.idl \
     $(WebCore)/Modules/mediastream/RTCRtpSynchronizationSource.idl \
     $(WebCore)/Modules/mediastream/RTCRtpTransceiver.idl \
     $(WebCore)/Modules/mediastream/RTCRtpTransceiverDirection.idl \

Modified: trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.cpp (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.cpp	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.cpp	2020-12-01 12:34:23 UTC (rev 270294)
@@ -32,6 +32,7 @@
 
 RTCEncodedVideoFrame::RTCEncodedVideoFrame(Ref<RTCRtpTransformableFrame>&& frame)
     : RTCEncodedFrame(WTFMove(frame))
+    , m_type(m_frame->isKeyFrame() ? Type::Key : Type::Delta)
 {
 }
 

Modified: trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.h (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -36,8 +36,13 @@
     static Ref<RTCEncodedVideoFrame> create(Ref<RTCRtpTransformableFrame>&& frame) { return adoptRef(*new RTCEncodedVideoFrame(WTFMove(frame))); }
     ~RTCEncodedVideoFrame();
 
+    enum class Type { Empty, Key, Delta };
+    Type type() const { return m_type; }
+
 private:
     explicit RTCEncodedVideoFrame(Ref<RTCRtpTransformableFrame>&&);
+
+    Type m_type;
 };
 
 } // namespace WebCore

Modified: trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.idl (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.idl	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.idl	2020-12-01 12:34:23 UTC (rev 270294)
@@ -23,12 +23,14 @@
  * THE POSSIBILITY OF SUCH DAMAGE.
  */
 
+enum RTCEncodedVideoFrameType { "empty", "key", "delta" };
+
 [
     Conditional=WEB_RTC,
     Exposed=DedicatedWorker,
     ImplementationLacksVTable,
 ] interface RTCEncodedVideoFrame {
-    // readonly attribute RTCEncodedVideoFrameType type;
+    readonly attribute RTCEncodedVideoFrameType type;
     // readonly attribute unsigned long long timestamp;
     attribute ArrayBuffer data;
     // RTCVideoFrameMetadata getMetadata();

Modified: trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformer.cpp (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformer.cpp	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformer.cpp	2020-12-01 12:34:23 UTC (rev 270294)
@@ -32,6 +32,7 @@
 #include "JSCallbackData.h"
 #include "JSRTCEncodedAudioFrame.h"
 #include "JSRTCEncodedVideoFrame.h"
+#include "JSRTCRtpScriptTransformerContext.h"
 #include "RTCRtpTransformableFrame.h"
 #include "ReadableStream.h"
 #include "ReadableStreamSource.h"
@@ -102,6 +103,7 @@
     JSC::MarkedArgumentBuffer args;
     args.append(toJSNewlyCreated(&globalObject, &globalObject, readableStream.releaseReturnValue()));
     args.append(toJSNewlyCreated(&globalObject, &globalObject, writableStream.releaseReturnValue()));
+    args.append(toJSNewlyCreated(&globalObject, &globalObject, RTCRtpScriptTransformerContext::create(makeRef(backend))));
 
     NakedPtr<JSC::Exception> returnedException;
     callback->invokeCallback(JSC::jsUndefined(), args, JSCallbackData::CallbackType::Object, JSC::Identifier::fromString(vm, "start"), returnedException);

Copied: trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformerContext.h (from rev 270293, trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.h) (0 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformerContext.h	                        (rev 0)
+++ trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformerContext.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -0,0 +1,80 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(WEB_RTC)
+
+#include "RTCRtpTransformBackend.h"
+#include <wtf/Ref.h>
+
+namespace WebCore {
+
+class RTCRtpTransformBackend;
+
+class RTCRtpScriptTransformerContext
+    : public RefCounted<RTCRtpScriptTransformerContext> {
+public:
+    static Ref<RTCRtpScriptTransformerContext> create(Ref<RTCRtpTransformBackend>&&);
+    ~RTCRtpScriptTransformerContext() = default;
+
+    using Side = RTCRtpTransformBackend::Side;
+    Side side() const { return m_backend->side(); }
+
+    using MediaType = RTCRtpTransformBackend::MediaType;
+    MediaType mediaType() const { return m_backend->mediaType(); }
+
+    ExceptionOr<void> requestKeyFrame();
+
+private:
+    explicit RTCRtpScriptTransformerContext(Ref<RTCRtpTransformBackend>&&);
+
+    Ref<RTCRtpTransformBackend> m_backend;
+};
+
+inline Ref<RTCRtpScriptTransformerContext> RTCRtpScriptTransformerContext::create(Ref<RTCRtpTransformBackend>&& backend)
+{
+    return adoptRef(*new RTCRtpScriptTransformerContext(WTFMove(backend)));
+}
+
+inline RTCRtpScriptTransformerContext::RTCRtpScriptTransformerContext(Ref<RTCRtpTransformBackend>&& backend)
+    : m_backend(WTFMove(backend))
+{
+}
+
+inline ExceptionOr<void> RTCRtpScriptTransformerContext::requestKeyFrame()
+{
+    switch (mediaType()) {
+    case MediaType::Audio:
+        return Exception { InvalidStateError, "Cannot request key frame on audio sender"_s };
+    case MediaType::Video:
+        m_backend->requestKeyFrame();
+        return { };
+    }
+}
+
+} // namespace WebCore
+
+#endif // ENABLE(WEB_RTC)

Copied: trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformerContext.idl (from rev 270293, trunk/Source/WebCore/Modules/mediastream/RTCEncodedVideoFrame.idl) (0 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformerContext.idl	                        (rev 0)
+++ trunk/Source/WebCore/Modules/mediastream/RTCRtpScriptTransformerContext.idl	2020-12-01 12:34:23 UTC (rev 270294)
@@ -0,0 +1,38 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+enum RTCRtpScriptTransformerContextSide { "receiver", "sender" };
+enum RTCRtpScriptTransformerContextMediaType { "audio", "video" };
+
+[
+    Conditional=WEB_RTC,
+    Exposed=DedicatedWorker,
+    ImplementationLacksVTable,
+] interface RTCRtpScriptTransformerContext {
+    readonly attribute RTCRtpScriptTransformerContextSide side;
+    readonly attribute RTCRtpScriptTransformerContextMediaType mediaType;
+
+    [MayThrowException] undefined requestKeyFrame();
+};

Modified: trunk/Source/WebCore/Modules/mediastream/RTCRtpTransformBackend.h (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/RTCRtpTransformBackend.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/RTCRtpTransformBackend.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -46,6 +46,8 @@
 
     enum class Side { Receiver, Sender };
     virtual Side side() const = 0;
+
+    virtual void requestKeyFrame() = 0;
 };
 
 } // namespace WebCore

Modified: trunk/Source/WebCore/Modules/mediastream/RTCRtpTransformableFrame.h (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/RTCRtpTransformableFrame.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/RTCRtpTransformableFrame.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -40,6 +40,8 @@
     };
     virtual Data data() const = 0;
     virtual void setData(Data) = 0;
+
+    virtual bool isKeyFrame() const = 0;
 };
 
 } // namespace WebCore

Modified: trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.cpp (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.cpp	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.cpp	2020-12-01 12:34:23 UTC (rev 270294)
@@ -52,6 +52,12 @@
     m_rtcReceiver->SetDepacketizerToDecoderFrameTransformer(this);
 }
 
+void LibWebRTCRtpReceiverTransformBackend::requestKeyFrame()
+{
+    ASSERT(mediaType() == MediaType::Video);
+    m_rtcReceiver->GenerateKeyFrame();
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(WEB_RTC) && USE(LIBWEBRTC)

Modified: trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.h (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpReceiverTransformBackend.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -53,6 +53,7 @@
 
     // RTCRtpTransformBackend
     void setTransformableFrameCallback(Callback&&) final;
+    void requestKeyFrame() final;
 
     rtc::scoped_refptr<webrtc::RtpReceiverInterface> m_rtcReceiver;
 };

Modified: trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.cpp (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.cpp	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.cpp	2020-12-01 12:34:23 UTC (rev 270294)
@@ -52,6 +52,12 @@
     m_rtcSender->SetEncoderToPacketizerFrameTransformer(this);
 }
 
+void LibWebRTCRtpSenderTransformBackend::requestKeyFrame()
+{
+    ASSERT(mediaType() == MediaType::Video);
+    m_rtcSender->GenerateKeyFrame();
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(WEB_RTC) && USE(LIBWEBRTC)

Modified: trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.h (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpSenderTransformBackend.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -56,6 +56,7 @@
 
     // RTCRtpTransformBackend
     void setTransformableFrameCallback(Callback&&) final;
+    void requestKeyFrame() final;
 
     rtc::scoped_refptr<webrtc::RtpSenderInterface> m_rtcSender;
 };

Modified: trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.cpp (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.cpp	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.cpp	2020-12-01 12:34:23 UTC (rev 270294)
@@ -65,6 +65,13 @@
         m_rtcFrame->SetData({ data.data, data.size });
 }
 
+bool LibWebRTCRtpTransformableFrame::isKeyFrame() const
+{
+    ASSERT(m_rtcFrame);
+    auto* videoFrame = static_cast<webrtc::TransformableVideoFrameInterface*>(m_rtcFrame.get());
+    return videoFrame && videoFrame->IsKeyFrame();
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(WEB_RTC) && USE(LIBWEBRTC)

Modified: trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.h (270293 => 270294)


--- trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.h	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Modules/mediastream/libwebrtc/LibWebRTCRtpTransformableFrame.h	2020-12-01 12:34:23 UTC (rev 270294)
@@ -46,8 +46,10 @@
 private:
     explicit LibWebRTCRtpTransformableFrame(std::unique_ptr<webrtc::TransformableFrameInterface>&&);
 
+    // RTCRtpTransformableFrame
     Data data() const final;
     void setData(Data) final;
+    bool isKeyFrame() const final;
 
     std::unique_ptr<webrtc::TransformableFrameInterface> m_rtcFrame;
 };

Modified: trunk/Source/WebCore/Sources.txt (270293 => 270294)


--- trunk/Source/WebCore/Sources.txt	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/Sources.txt	2020-12-01 12:34:23 UTC (rev 270294)
@@ -3347,6 +3347,7 @@
 JSRTCRtpScriptTransform.cpp
 JSRTCRtpScriptTransformer.cpp
 JSRTCRtpScriptTransformerConstructor.cpp
+JSRTCRtpScriptTransformerContext.cpp
 JSRTCRtpSendParameters.cpp
 JSRTCRtpSender.cpp
 JSRTCRtpSynchronizationSource.cpp

Modified: trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj (270293 => 270294)


--- trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj	2020-12-01 11:31:23 UTC (rev 270293)
+++ trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj	2020-12-01 12:34:23 UTC (rev 270294)
@@ -7795,6 +7795,9 @@
 		419E6EC42566D211002B5010 /* RTCPeerConnectionIceErrorEvent.idl */ = {isa = PBXFileReference; lastKnownFileType = text; path = RTCPeerConnectionIceErrorEvent.idl; sourceTree = "<group>"; };
 		419E6EC62566D213002B5010 /* RTCPeerConnectionIceErrorEvent.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = RTCPeerConnectionIceErrorEvent.cpp; sourceTree = "<group>"; };
 		419E6EC72566D214002B5010 /* RTCPeerConnectionIceErrorEvent.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = RTCPeerConnectionIceErrorEvent.h; sourceTree = "<group>"; };
+		419E6ED22567BF69002B5010 /* RTCRtpScriptTransformerContext.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = RTCRtpScriptTransformerContext.idl; sourceTree = "<group>"; };
+		419E6ED52567BF6A002B5010 /* RTCRtpScriptTransformerContext.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RTCRtpScriptTransformerContext.h; sourceTree = "<group>"; };
+		419E6ED82567D8D9002B5010 /* LibWebRTCRtpTransformableFrame.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = LibWebRTCRtpTransformableFrame.cpp; path = libwebrtc/LibWebRTCRtpTransformableFrame.cpp; sourceTree = "<group>"; };
 		419FAFAD1ABABCD5005B3572 /* ReadableStreamBYOBReader.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = ReadableStreamBYOBReader.idl; sourceTree = "<group>"; };
 		419FAFAD1ABABCD5005B828B /* ReadableStreamDefaultReader.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = ReadableStreamDefaultReader.idl; sourceTree = "<group>"; };
 		41A023ED1A39DB7900F722CF /* ReadableStream.idl */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = ReadableStream.idl; sourceTree = "<group>"; };
@@ -17186,6 +17189,8 @@
 				414189F12562DF5600363417 /* RTCRtpScriptTransformer.idl */,
 				414189F62562E67A00363417 /* RTCRtpScriptTransformerConstructor.h */,
 				414189F72562E67A00363417 /* RTCRtpScriptTransformerConstructor.idl */,
+				419E6ED52567BF6A002B5010 /* RTCRtpScriptTransformerContext.h */,
+				419E6ED22567BF69002B5010 /* RTCRtpScriptTransformerContext.idl */,
 				414189EF2562DF0C00363417 /* RTCRtpScriptTransformProvider.idl */,
 				413C8B262552EEA900E65055 /* RTCRtpSender+Transform.idl */,
 				5E2C43591BCEE30D0001E2BC /* RTCRtpSender.cpp */,
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to