Title: [268136] trunk
Revision
268136
Author
you...@apple.com
Date
2020-10-07 11:07:22 -0700 (Wed, 07 Oct 2020)

Log Message

Add support for BlobEvent.timecode
https://bugs.webkit.org/show_bug.cgi?id=217379

Reviewed by Eric Carlson.

LayoutTests/imported/w3c:

* web-platform-tests/mediacapture-record/idlharness.window-expected.txt:

Source/WebCore:

Expose timecode getter for BlobEvent.
Make MediaRecorder backend return the timecode value for each fetch data.
For the real backend, compute the timecode as the current audio timestamp for the next fetch data request,
or the current video timestamp if only video is captured.

Covered by rebased test.

* Modules/mediarecorder/BlobEvent.cpp:
(WebCore::BlobEvent::BlobEvent):
* Modules/mediarecorder/BlobEvent.h:
* Modules/mediarecorder/BlobEvent.idl:
* Modules/mediarecorder/MediaRecorder.cpp:
(WebCore::createDataAvailableEvent):
(WebCore::MediaRecorder::stopRecording):
(WebCore::MediaRecorder::requestData):
(WebCore::MediaRecorder::fetchData):
(WebCore::MediaRecorder::handleTrackChange):
(WebCore::MediaRecorder::trackEnded):
* Modules/mediarecorder/MediaRecorder.h:
* platform/mediarecorder/MediaRecorderPrivate.h:
* platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp:
(WebCore::MediaRecorderPrivateAVFImpl::fetchData):
* platform/mediarecorder/MediaRecorderPrivateMock.cpp:
(WebCore::MediaRecorderPrivateMock::fetchData):
* platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
* platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:
(WebCore::MediaRecorderPrivateWriter::clear):
(WebCore::MediaRecorderPrivateWriter::stopRecording):
(WebCore::MediaRecorderPrivateWriter::fetchData):
(WebCore::MediaRecorderPrivateWriter::updateTimeCode):

Source/WebKit:

Exchange timecode through IPC.

* GPUProcess/webrtc/RemoteMediaRecorder.cpp:
(WebKit::RemoteMediaRecorder::fetchData):
* GPUProcess/webrtc/RemoteMediaRecorder.h:
* GPUProcess/webrtc/RemoteMediaRecorder.messages.in:
* WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp:
(WebKit::MediaRecorderPrivate::fetchData):
* WebProcess/GPU/webrtc/MediaRecorderPrivate.h:

Modified Paths

Diff

Modified: trunk/LayoutTests/imported/w3c/ChangeLog (268135 => 268136)


--- trunk/LayoutTests/imported/w3c/ChangeLog	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/LayoutTests/imported/w3c/ChangeLog	2020-10-07 18:07:22 UTC (rev 268136)
@@ -1,5 +1,14 @@
 2020-10-07  Youenn Fablet  <you...@apple.com>
 
+        Add support for BlobEvent.timecode
+        https://bugs.webkit.org/show_bug.cgi?id=217379
+
+        Reviewed by Eric Carlson.
+
+        * web-platform-tests/mediacapture-record/idlharness.window-expected.txt:
+
+2020-10-07  Youenn Fablet  <you...@apple.com>
+
         Add support for MediaRecorder pause/resume
         https://bugs.webkit.org/show_bug.cgi?id=217375
 

Modified: trunk/LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/idlharness.window-expected.txt (268135 => 268136)


--- trunk/LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/idlharness.window-expected.txt	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/idlharness.window-expected.txt	2020-10-07 18:07:22 UTC (rev 268136)
@@ -54,11 +54,11 @@
 PASS BlobEvent interface: existence and properties of interface prototype object's "constructor" property
 PASS BlobEvent interface: existence and properties of interface prototype object's @@unscopables property
 PASS BlobEvent interface: attribute data
-FAIL BlobEvent interface: attribute timecode assert_true: The prototype object must have a property "timecode" expected true got false
+PASS BlobEvent interface: attribute timecode
 PASS BlobEvent must be primary interface of [object BlobEvent]
 PASS Stringification of [object BlobEvent]
 PASS BlobEvent interface: [object BlobEvent] must inherit property "data" with the proper type
-FAIL BlobEvent interface: [object BlobEvent] must inherit property "timecode" with the proper type assert_inherits: property "timecode" not found in prototype chain
+PASS BlobEvent interface: [object BlobEvent] must inherit property "timecode" with the proper type
 PASS MediaRecorderErrorEvent interface: existence and properties of interface object
 PASS MediaRecorderErrorEvent interface object length
 PASS MediaRecorderErrorEvent interface object name

Modified: trunk/Source/WebCore/ChangeLog (268135 => 268136)


--- trunk/Source/WebCore/ChangeLog	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/ChangeLog	2020-10-07 18:07:22 UTC (rev 268136)
@@ -1,3 +1,41 @@
+2020-10-07  Youenn Fablet  <you...@apple.com>
+
+        Add support for BlobEvent.timecode
+        https://bugs.webkit.org/show_bug.cgi?id=217379
+
+        Reviewed by Eric Carlson.
+
+        Expose timecode getter for BlobEvent.
+        Make MediaRecorder backend return the timecode value for each fetch data.
+        For the real backend, compute the timecode as the current audio timestamp for the next fetch data request,
+        or the current video timestamp if only video is captured.
+
+        Covered by rebased test.
+
+        * Modules/mediarecorder/BlobEvent.cpp:
+        (WebCore::BlobEvent::BlobEvent):
+        * Modules/mediarecorder/BlobEvent.h:
+        * Modules/mediarecorder/BlobEvent.idl:
+        * Modules/mediarecorder/MediaRecorder.cpp:
+        (WebCore::createDataAvailableEvent):
+        (WebCore::MediaRecorder::stopRecording):
+        (WebCore::MediaRecorder::requestData):
+        (WebCore::MediaRecorder::fetchData):
+        (WebCore::MediaRecorder::handleTrackChange):
+        (WebCore::MediaRecorder::trackEnded):
+        * Modules/mediarecorder/MediaRecorder.h:
+        * platform/mediarecorder/MediaRecorderPrivate.h:
+        * platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp:
+        (WebCore::MediaRecorderPrivateAVFImpl::fetchData):
+        * platform/mediarecorder/MediaRecorderPrivateMock.cpp:
+        (WebCore::MediaRecorderPrivateMock::fetchData):
+        * platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
+        * platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:
+        (WebCore::MediaRecorderPrivateWriter::clear):
+        (WebCore::MediaRecorderPrivateWriter::stopRecording):
+        (WebCore::MediaRecorderPrivateWriter::fetchData):
+        (WebCore::MediaRecorderPrivateWriter::updateTimeCode):
+
 2020-10-07  Zalan Bujtas  <za...@apple.com>
 
         [LFC][IFC] Adjust the inline formatting context root's content height when the last line has float clear

Modified: trunk/Source/WebCore/Modules/mediarecorder/BlobEvent.cpp (268135 => 268136)


--- trunk/Source/WebCore/Modules/mediarecorder/BlobEvent.cpp	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/Modules/mediarecorder/BlobEvent.cpp	2020-10-07 18:07:22 UTC (rev 268136)
@@ -40,23 +40,13 @@
     return adoptRef(*new BlobEvent(type, WTFMove(init), isTrusted));
 }
 
-Ref<BlobEvent> BlobEvent::create(const AtomString& type, CanBubble canBubble, IsCancelable isCancelable, Ref<Blob>&& data)
-{
-    return adoptRef(*new BlobEvent(type, canBubble, isCancelable, WTFMove(data)));
-}
-
 BlobEvent::BlobEvent(const AtomString& type, Init&& init, IsTrusted isTrusted)
     : Event(type, init, isTrusted)
     , m_blob(init.data.releaseNonNull())
+    , m_timecode(init.timecode)
 {
 }
 
-BlobEvent::BlobEvent(const AtomString& type, CanBubble canBubble, IsCancelable isCancelable, Ref<Blob>&& data)
-    : Event(type, canBubble, isCancelable)
-    , m_blob(WTFMove(data))
-{
-}
-    
 EventInterface BlobEvent::eventInterface() const
 {
     return BlobEventInterfaceType;

Modified: trunk/Source/WebCore/Modules/mediarecorder/BlobEvent.h (268135 => 268136)


--- trunk/Source/WebCore/Modules/mediarecorder/BlobEvent.h	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/Modules/mediarecorder/BlobEvent.h	2020-10-07 18:07:22 UTC (rev 268136)
@@ -41,10 +41,10 @@
     };
     
     static Ref<BlobEvent> create(const AtomString&, Init&&, IsTrusted = IsTrusted::No);
-    static Ref<BlobEvent> create(const AtomString&, CanBubble, IsCancelable, Ref<Blob>&&);
 
     Blob& data() const { return m_blob.get(); }
-    
+    double timecode() const { return m_timecode; }
+
 private:
     BlobEvent(const AtomString&, Init&&, IsTrusted);
     BlobEvent(const AtomString&, CanBubble, IsCancelable, Ref<Blob>&&);
@@ -53,6 +53,7 @@
     EventInterface eventInterface() const final;
     
     Ref<Blob> m_blob;
+    double m_timecode { 0 };
 };
     
 } // namespace WebCore

Modified: trunk/Source/WebCore/Modules/mediarecorder/BlobEvent.idl (268135 => 268136)


--- trunk/Source/WebCore/Modules/mediarecorder/BlobEvent.idl	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/Modules/mediarecorder/BlobEvent.idl	2020-10-07 18:07:22 UTC (rev 268136)
@@ -33,7 +33,7 @@
 
     // FIXME: Implement these:
     [SameObject] readonly attribute Blob data;
-    // readonly attribute DOMHighResTimeStamp timecode;
+    readonly attribute DOMHighResTimeStamp timecode;
 };
 
 dictionary BlobEventInit : EventInit {

Modified: trunk/Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp (268135 => 268136)


--- trunk/Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp	2020-10-07 18:07:22 UTC (rev 268136)
@@ -190,6 +190,12 @@
     return { };
 }
 
+static inline Ref<BlobEvent> createDataAvailableEvent(ScriptExecutionContext* context, RefPtr<SharedBuffer>&& buffer, const String& mimeType, double timeCode)
+{
+    auto blob = buffer ? Blob::create(context, buffer.releaseNonNull(), mimeType) : Blob::create(context);
+    return BlobEvent::create(eventNames().dataavailableEvent, BlobEvent::Init { { false, false, false }, WTFMove(blob), timeCode }, BlobEvent::IsTrusted::Yes);
+}
+
 ExceptionOr<void> MediaRecorder::stopRecording()
 {
     if (state() == RecordingState::Inactive)
@@ -196,10 +202,12 @@
         return Exception { InvalidStateError, "The MediaRecorder's state cannot be inactive"_s };
 
     stopRecordingInternal();
-    fetchData([this](auto&& buffer, auto& mimeType) {
+    fetchData([this](auto&& buffer, auto& mimeType, auto timeCode) {
         if (!m_isActive)
             return;
-        dispatchEvent(BlobEvent::create(eventNames().dataavailableEvent, Event::CanBubble::No, Event::IsCancelable::No, buffer ? Blob::create(scriptExecutionContext(), buffer.releaseNonNull(), mimeType) : Blob::create(scriptExecutionContext())));
+
+        dispatchEvent(createDataAvailableEvent(scriptExecutionContext(), WTFMove(buffer), mimeType, timeCode));
+
         if (!m_isActive)
             return;
         dispatchEvent(Event::create(eventNames().stopEvent, Event::CanBubble::No, Event::IsCancelable::No));
@@ -215,11 +223,11 @@
     if (m_timeSliceTimer.isActive())
         m_timeSliceTimer.stop();
 
-    fetchData([this](auto&& buffer, auto& mimeType) {
+    fetchData([this](auto&& buffer, auto& mimeType, auto timeCode) {
         if (!m_isActive)
             return;
 
-        dispatchEvent(BlobEvent::create(eventNames().dataavailableEvent, Event::CanBubble::No, Event::IsCancelable::No, buffer ? Blob::create(scriptExecutionContext(), buffer.releaseNonNull(), mimeType) : Blob::create(scriptExecutionContext())));
+        dispatchEvent(createDataAvailableEvent(scriptExecutionContext(), WTFMove(buffer), mimeType, timeCode));
 
         if (m_isActive && m_timeSlice)
             m_timeSliceTimer.startOneShot(Seconds::fromMilliseconds(*m_timeSlice));
@@ -277,9 +285,9 @@
     if (takeRecorder == TakePrivateRecorder::Yes)
         takenPrivateRecorder = WTFMove(m_private);
 
-    auto fetchDataCallback = [this, privateRecorder = WTFMove(takenPrivateRecorder), callback = WTFMove(callback)](auto&& buffer, auto& mimeType) mutable {
-        queueTaskKeepingObjectAlive(*this, TaskSource::Networking, [buffer = WTFMove(buffer), mimeType, callback = WTFMove(callback)]() mutable {
-            callback(WTFMove(buffer), mimeType);
+    auto fetchDataCallback = [this, privateRecorder = WTFMove(takenPrivateRecorder), callback = WTFMove(callback)](auto&& buffer, auto& mimeType, auto timeCode) mutable {
+        queueTaskKeepingObjectAlive(*this, TaskSource::Networking, [buffer = WTFMove(buffer), mimeType, timeCode, callback = WTFMove(callback)]() mutable {
+            callback(WTFMove(buffer), mimeType, timeCode);
         });
     };
 
@@ -289,11 +297,11 @@
     }
 
     m_isFetchingData = true;
-    privateRecorder.fetchData([this, pendingActivity = makePendingActivity(*this), callback = WTFMove(fetchDataCallback)](auto&& buffer, auto& mimeType) mutable {
+    privateRecorder.fetchData([this, pendingActivity = makePendingActivity(*this), callback = WTFMove(fetchDataCallback)](auto&& buffer, auto& mimeType, auto timeCode) mutable {
         m_isFetchingData = false;
-        callback(WTFMove(buffer), mimeType);
+        callback(WTFMove(buffer), mimeType, timeCode);
         for (auto& task : std::exchange(m_pendingFetchDataTasks, { }))
-            task({ }, mimeType);
+            task({ }, mimeType, timeCode);
     });
 }
 
@@ -318,7 +326,9 @@
         dispatchError(Exception { InvalidModificationError, "Track cannot be added to or removed from the MediaStream while recording"_s });
         if (!m_isActive)
             return;
-        dispatchEvent(BlobEvent::create(eventNames().dataavailableEvent, Event::CanBubble::No, Event::IsCancelable::No, Blob::create(scriptExecutionContext())));
+
+        dispatchEvent(createDataAvailableEvent(scriptExecutionContext(), { }, { }, 0));
+
         if (!m_isActive)
             return;
         dispatchEvent(Event::create(eventNames().stopEvent, Event::CanBubble::No, Event::IsCancelable::No));
@@ -345,7 +355,7 @@
             return;
 
         stopRecordingInternal();
-        dispatchEvent(BlobEvent::create(eventNames().dataavailableEvent, Event::CanBubble::No, Event::IsCancelable::No, Blob::create(scriptExecutionContext())));
+        dispatchEvent(createDataAvailableEvent(scriptExecutionContext(), { }, { }, 0));
         if (!m_isActive)
             return;
         dispatchEvent(Event::create(eventNames().stopEvent, Event::CanBubble::No, Event::IsCancelable::No));

Modified: trunk/Source/WebCore/Modules/mediarecorder/MediaRecorder.h (268135 => 268136)


--- trunk/Source/WebCore/Modules/mediarecorder/MediaRecorder.h	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/Modules/mediarecorder/MediaRecorder.h	2020-10-07 18:07:22 UTC (rev 268136)
@@ -99,7 +99,7 @@
     void dispatchError(Exception&&);
 
     enum class TakePrivateRecorder { No, Yes };
-    using FetchDataCallback = Function<void(RefPtr<SharedBuffer>&&, const String& mimeType)>;
+    using FetchDataCallback = Function<void(RefPtr<SharedBuffer>&&, const String& mimeType, double)>;
     void fetchData(FetchDataCallback&&, TakePrivateRecorder);
 
     // MediaStream::Observer

Modified: trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivate.h (268135 => 268136)


--- trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivate.h	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivate.h	2020-10-07 18:07:22 UTC (rev 268136)
@@ -57,7 +57,7 @@
     };
     WEBCORE_EXPORT static AudioVideoSelectedTracks selectTracks(MediaStreamPrivate&);
 
-    using FetchDataCallback = CompletionHandler<void(RefPtr<SharedBuffer>&&, const String& mimeType)>;
+    using FetchDataCallback = CompletionHandler<void(RefPtr<SharedBuffer>&&, const String& mimeType, double)>;
     virtual void fetchData(FetchDataCallback&&) = 0;
     virtual const String& mimeType() const = 0;
 

Modified: trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp (268135 => 268136)


--- trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp	2020-10-07 18:07:22 UTC (rev 268136)
@@ -151,8 +151,8 @@
 
 void MediaRecorderPrivateAVFImpl::fetchData(FetchDataCallback&& completionHandler)
 {
-    m_writer->fetchData([completionHandler = WTFMove(completionHandler), mimeType = mimeType()](auto&& buffer) mutable {
-        completionHandler(WTFMove(buffer), mimeType);
+    m_writer->fetchData([completionHandler = WTFMove(completionHandler), mimeType = mimeType()](auto&& buffer, auto timeCode) mutable {
+        completionHandler(WTFMove(buffer), mimeType, timeCode);
     });
 }
 

Modified: trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivateMock.cpp (268135 => 268136)


--- trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivateMock.cpp	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/platform/mediarecorder/MediaRecorderPrivateMock.cpp	2020-10-07 18:07:22 UTC (rev 268136)
@@ -31,6 +31,7 @@
 #include "MediaStreamTrackPrivate.h"
 #include "SharedBuffer.h"
 #include "Timer.h"
+#include <wtf/MonotonicTime.h>
 
 namespace WebCore {
 
@@ -100,8 +101,8 @@
     }
 
     // Delay calling the completion handler a bit to mimick real writer behavior.
-    m_delayCompletingTimer.doTask([completionHandler = WTFMove(completionHandler), buffer = WTFMove(buffer), mimeType = mimeType()]() mutable {
-        completionHandler(WTFMove(buffer), mimeType);
+    m_delayCompletingTimer.doTask([completionHandler = WTFMove(completionHandler), buffer = WTFMove(buffer), mimeType = mimeType(), timeCode = MonotonicTime::now().secondsSinceEpoch().value()]() mutable {
+        completionHandler(WTFMove(buffer), mimeType, timeCode);
     }, 50_ms);
 }
 

Modified: trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h (268135 => 268136)


--- trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h	2020-10-07 18:07:22 UTC (rev 268136)
@@ -70,7 +70,7 @@
     void appendVideoSampleBuffer(MediaSample&);
     void appendAudioSampleBuffer(const PlatformAudioData&, const AudioStreamDescription&, const WTF::MediaTime&, size_t);
     void stopRecording();
-    void fetchData(CompletionHandler<void(RefPtr<SharedBuffer>&&)>&&);
+    void fetchData(CompletionHandler<void(RefPtr<SharedBuffer>&&, double)>&&);
 
     void pause();
     void resume();
@@ -105,6 +105,7 @@
     void flushCompressedSampleBuffers(CompletionHandler<void()>&&);
 
     void finishedFlushingSamples();
+    void updateTimeCode();
 
     bool m_hasAudio { false };
     bool m_hasVideo { false };
@@ -116,7 +117,7 @@
     RetainPtr<AVAssetWriter> m_writer;
 
     RefPtr<SharedBuffer> m_data;
-    CompletionHandler<void(RefPtr<SharedBuffer>&&)> m_fetchDataCompletionHandler;
+    CompletionHandler<void(RefPtr<SharedBuffer>&&, double)> m_fetchDataCompletionHandler;
 
     RetainPtr<CMFormatDescriptionRef> m_audioFormatDescription;
     std::unique_ptr<AudioSampleBufferCompressor> m_audioCompressor;
@@ -140,6 +141,7 @@
     CMTime m_resumedVideoTime { kCMTimeZero };
     CMTime m_currentVideoDuration { kCMTimeZero };
     CMTime m_currentAudioSampleTime { kCMTimeZero };
+    double m_timeCode { 0 };
 };
 
 } // namespace WebCore

Modified: trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm (268135 => 268136)


--- trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm	2020-10-07 18:07:22 UTC (rev 268136)
@@ -399,7 +399,7 @@
 
     m_data = nullptr;
     if (auto completionHandler = WTFMove(m_fetchDataCompletionHandler))
-        completionHandler(nullptr);
+        completionHandler(nullptr, 0);
 }
 
 
@@ -523,7 +523,7 @@
             if (m_writer)
                 m_writer.clear();
             if (m_fetchDataCompletionHandler)
-                m_fetchDataCompletionHandler(std::exchange(m_data, nullptr));
+                m_fetchDataCompletionHandler(std::exchange(m_data, nullptr), 0);
         };
 
         if (!m_hasStartedWriting) {
@@ -547,7 +547,7 @@
     });
 }
 
-void MediaRecorderPrivateWriter::fetchData(CompletionHandler<void(RefPtr<SharedBuffer>&&)>&& completionHandler)
+void MediaRecorderPrivateWriter::fetchData(CompletionHandler<void(RefPtr<SharedBuffer>&&, double)>&& completionHandler)
 {
     if (m_isStopping) {
         m_fetchDataCompletionHandler = WTFMove(completionHandler);
@@ -557,7 +557,7 @@
     if (m_hasStartedWriting) {
         flushCompressedSampleBuffers([this, weakThis = makeWeakPtr(this), completionHandler = WTFMove(completionHandler)]() mutable {
             if (!weakThis) {
-                completionHandler(nullptr);
+                completionHandler(nullptr, 0);
                 return;
             }
 
@@ -567,19 +567,31 @@
 
             callOnMainThread([this, weakThis = WTFMove(weakThis), completionHandler = WTFMove(completionHandler)]() mutable {
                 if (!weakThis) {
-                    completionHandler(nullptr);
+                    completionHandler(nullptr, 0);
                     return;
                 }
 
-                completionHandler(std::exchange(m_data, nullptr));
+                completionHandler(std::exchange(m_data, nullptr), m_timeCode);
+                updateTimeCode();
             });
         });
         return;
     }
 
-    completionHandler(std::exchange(m_data, nullptr));
+    completionHandler(std::exchange(m_data, nullptr), m_timeCode);
+    updateTimeCode();
 }
 
+void MediaRecorderPrivateWriter::updateTimeCode()
+{
+    if (m_hasAudio) {
+        m_timeCode = CMTimeGetSeconds(m_currentAudioSampleTime);
+        return;
+    }
+    auto sampleTime = CMTimeSubtract(CMClockGetTime(CMClockGetHostTimeClock()), m_resumedVideoTime);
+    m_timeCode = CMTimeGetSeconds(CMTimeAdd(sampleTime, m_currentVideoDuration));
+}
+
 void MediaRecorderPrivateWriter::appendData(const char* data, size_t size)
 {
     if (!m_data) {

Modified: trunk/Source/WebKit/ChangeLog (268135 => 268136)


--- trunk/Source/WebKit/ChangeLog	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebKit/ChangeLog	2020-10-07 18:07:22 UTC (rev 268136)
@@ -1,5 +1,22 @@
 2020-10-07  Youenn Fablet  <you...@apple.com>
 
+        Add support for BlobEvent.timecode
+        https://bugs.webkit.org/show_bug.cgi?id=217379
+
+        Reviewed by Eric Carlson.
+
+        Exchange timecode through IPC.
+
+        * GPUProcess/webrtc/RemoteMediaRecorder.cpp:
+        (WebKit::RemoteMediaRecorder::fetchData):
+        * GPUProcess/webrtc/RemoteMediaRecorder.h:
+        * GPUProcess/webrtc/RemoteMediaRecorder.messages.in:
+        * WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp:
+        (WebKit::MediaRecorderPrivate::fetchData):
+        * WebProcess/GPU/webrtc/MediaRecorderPrivate.h:
+
+2020-10-07  Youenn Fablet  <you...@apple.com>
+
         Add support for MediaRecorder pause/resume
         https://bugs.webkit.org/show_bug.cgi?id=217375
 

Modified: trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp (268135 => 268136)


--- trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp	2020-10-07 18:07:22 UTC (rev 268136)
@@ -121,11 +121,11 @@
     m_writer->appendVideoSampleBuffer(*sampleBuffer);
 }
 
-void RemoteMediaRecorder::fetchData(CompletionHandler<void(IPC::DataReference&&)>&& completionHandler)
+void RemoteMediaRecorder::fetchData(CompletionHandler<void(IPC::DataReference&&, double)>&& completionHandler)
 {
-    m_writer->fetchData([completionHandler = WTFMove(completionHandler)](auto&& data) mutable {
+    m_writer->fetchData([completionHandler = WTFMove(completionHandler)](auto&& data, auto timeCode) mutable {
         auto* pointer = reinterpret_cast<const uint8_t*>(data ? data->data() : nullptr);
-        completionHandler(IPC::DataReference { pointer, data ? data->size() : 0 });
+        completionHandler(IPC::DataReference { pointer, data ? data->size() : 0 }, timeCode);
     });
 }
 

Modified: trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h (268135 => 268136)


--- trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h	2020-10-07 18:07:22 UTC (rev 268136)
@@ -69,7 +69,7 @@
     void audioSamplesStorageChanged(const SharedMemory::IPCHandle&, const WebCore::CAAudioStreamDescription&, uint64_t numberOfFrames);
     void audioSamplesAvailable(MediaTime, uint64_t numberOfFrames, uint64_t startFrame, uint64_t endFrame);
     void videoSampleAvailable(WebCore::RemoteVideoSample&&);
-    void fetchData(CompletionHandler<void(IPC::DataReference&&)>&&);
+    void fetchData(CompletionHandler<void(IPC::DataReference&&, double)>&&);
     void stopRecording();
     void pause(CompletionHandler<void()>&&);
     void resume(CompletionHandler<void()>&&);

Modified: trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in (268135 => 268136)


--- trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in	2020-10-07 18:07:22 UTC (rev 268136)
@@ -27,7 +27,7 @@
     AudioSamplesStorageChanged(WebKit::SharedMemory::IPCHandle storageHandle, WebCore::CAAudioStreamDescription description, uint64_t numberOfFrames)
     AudioSamplesAvailable(MediaTime time, uint64_t numberOfFrames, uint64_t startFrame, uint64_t endFrame)
     VideoSampleAvailable(WebCore::RemoteVideoSample sample)
-    FetchData() -> (IPC::DataReference buffer) Async
+    FetchData() -> (IPC::DataReference buffer, double timeCode) Async
     StopRecording()
     Pause() -> () Async
     Resume() -> () Async

Modified: trunk/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp (268135 => 268136)


--- trunk/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp	2020-10-07 18:07:22 UTC (rev 268136)
@@ -125,13 +125,13 @@
     m_connection->send(Messages::RemoteMediaRecorder::AudioSamplesStorageChanged { SharedMemory::IPCHandle { WTFMove(handle), dataSize }, m_description, static_cast<uint64_t>(m_numberOfFrames) }, m_identifier);
 }
 
-void MediaRecorderPrivate::fetchData(CompletionHandler<void(RefPtr<WebCore::SharedBuffer>&&, const String& mimeType)>&& completionHandler)
+void MediaRecorderPrivate::fetchData(CompletionHandler<void(RefPtr<WebCore::SharedBuffer>&&, const String& mimeType, double)>&& completionHandler)
 {
-    m_connection->sendWithAsyncReply(Messages::RemoteMediaRecorder::FetchData { }, [completionHandler = WTFMove(completionHandler), mimeType = mimeType()](auto&& data) mutable {
+    m_connection->sendWithAsyncReply(Messages::RemoteMediaRecorder::FetchData { }, [completionHandler = WTFMove(completionHandler), mimeType = mimeType()](auto&& data, double timeCode) mutable {
         RefPtr<SharedBuffer> buffer;
         if (data.size())
             buffer = SharedBuffer::create(data.data(), data.size());
-        completionHandler(WTFMove(buffer), mimeType);
+        completionHandler(WTFMove(buffer), mimeType, timeCode);
     }, m_identifier);
 }
 

Modified: trunk/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h (268135 => 268136)


--- trunk/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h	2020-10-07 18:05:09 UTC (rev 268135)
+++ trunk/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h	2020-10-07 18:07:22 UTC (rev 268136)
@@ -56,7 +56,7 @@
 private:
     // WebCore::MediaRecorderPrivate
     void videoSampleAvailable(WebCore::MediaSample&) final;
-    void fetchData(CompletionHandler<void(RefPtr<WebCore::SharedBuffer>&&, const String& mimeType)>&&) final;
+    void fetchData(CompletionHandler<void(RefPtr<WebCore::SharedBuffer>&&, const String& mimeType, double)>&&) final;
     void stopRecording() final;
     void startRecording(StartRecordingCallback&&) final;
     void audioSamplesAvailable(const WTF::MediaTime&, const WebCore::PlatformAudioData&, const WebCore::AudioStreamDescription&, size_t) final;
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to