Title: [271024] trunk
Revision
271024
Author
[email protected]
Date
2020-12-21 10:14:08 -0800 (Mon, 21 Dec 2020)

Log Message

[MSE] Fix allSamplesInTrackEnqueued() handling
https://bugs.webkit.org/show_bug.cgi?id=220055

Reviewed by Philippe Normand.

Source/WebCore:

Within the changes introduced by r270612 a spurious call to
allSamplesInTrackEnqueued() was added to provideMediaData().

provideMediaData() is called every time there may be new samples that
need to be enqueued into the playback pipeline.

allSamplesInTrackEnqueued() is supposed to be called when
MediaSource.endOfStream() has been called and all pending samples have
been enqueued, therefore signaling the playback pipeline that no more
samples will be added.

Some decoders need to be notified of this condition in order to move
the last samples in their queue downstream. This is true at least of
the avdec (ffmpeg) decoders that are commonly used in desktop versions
of the GStreamer port.

Calling allSamplesInTrackEnqueued() prematurely will mess with the
playback as the code will not accept any more samples, a serious
problem. This patch fixes it by removing the spurious call and
restoring its original purpose when it was introduced in r230909.

* Modules/mediasource/MediaSource.cpp:
(WebCore::MediaSource::streamEndedWithError):
(WebCore::MediaSource::openIfInEndedState):
* Modules/mediasource/SourceBuffer.cpp:
(WebCore::SourceBuffer::setMediaSourceEnded):
(WebCore::SourceBuffer::trySignalAllSamplesEnqueued): Deleted.
* Modules/mediasource/SourceBuffer.h:
* platform/graphics/SourceBufferPrivate.cpp:
(WebCore::SourceBufferPrivate::setMediaSourceEnded):
(WebCore::SourceBufferPrivate::trySignalAllSamplesInTrackEnqueued):
(WebCore::SourceBufferPrivate::provideMediaData):
* platform/graphics/SourceBufferPrivate.h:

Source/WebKit:

trySignalAllSamplesInTrackEnqueued() does not need to be exposed
outside of SourceBufferPrivate.

* GPUProcess/media/RemoteSourceBufferProxy.cpp:
(WebKit::RemoteSourceBufferProxy::trySignalAllSamplesInTrackEnqueued): Deleted.
* GPUProcess/media/RemoteSourceBufferProxy.h:
* GPUProcess/media/RemoteSourceBufferProxy.messages.in:
* WebProcess/GPU/media/SourceBufferPrivateRemote.cpp:
(WebKit::SourceBufferPrivateRemote::trySignalAllSamplesInTrackEnqueued): Deleted.
* WebProcess/GPU/media/SourceBufferPrivateRemote.h:

LayoutTests:

Adjusted test expectations.

* platform/glib/TestExpectations:

Modified Paths

Diff

Modified: trunk/LayoutTests/ChangeLog (271023 => 271024)


--- trunk/LayoutTests/ChangeLog	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/LayoutTests/ChangeLog	2020-12-21 18:14:08 UTC (rev 271024)
@@ -1,3 +1,14 @@
+2020-12-21  Alicia Boya García  <[email protected]>
+
+        [MSE] Fix allSamplesInTrackEnqueued() handling
+        https://bugs.webkit.org/show_bug.cgi?id=220055
+
+        Reviewed by Philippe Normand.
+
+        Adjusted test expectations.
+
+        * platform/glib/TestExpectations:
+
 2020-12-21  Ryan Haddad  <[email protected]>
 
         Unreviewed test gardening, rebaseline tests for Windows.

Modified: trunk/LayoutTests/platform/glib/TestExpectations (271023 => 271024)


--- trunk/LayoutTests/platform/glib/TestExpectations	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/LayoutTests/platform/glib/TestExpectations	2020-12-21 18:14:08 UTC (rev 271024)
@@ -463,7 +463,6 @@
 webkit.org/b/219822 imported/w3c/web-platform-tests/media-source/mediasource-buffered-seek.html [ Failure ]
 webkit.org/b/219822 imported/w3c/web-platform-tests/media-source/mediasource-changetype-play-negative.html [ Failure ]
 webkit.org/b/219822 imported/w3c/web-platform-tests/media-source/mediasource-config-change-webm-av-video-bitrate.html [ Pass Failure ]
-webkit.org/b/219822 imported/w3c/web-platform-tests/media-source/mediasource-h264-play-starved.html [ Failure ]
 webkit.org/b/219822 imported/w3c/web-platform-tests/media-source/mediasource-liveseekable.html [ Failure ]
 webkit.org/b/219822 imported/w3c/web-platform-tests/media-source/mediasource-play-then-seek-back.html [ Failure ]
 webkit.org/b/219822 imported/w3c/web-platform-tests/media-source/mediasource-redundant-seek.html [ Failure ]
@@ -472,8 +471,6 @@
 webkit.org/b/219822 imported/w3c/web-platform-tests/media-source/mediasource-sequencemode-append-buffer.html [ Failure ]
 webkit.org/b/219822 media/media-source/media-source-canplaythrough-event.html [ Failure ]
 webkit.org/b/219822 media/media-source/media-source-stalled-holds-sleep-assertion.html [ Failure ]
-webkit.org/b/219822 media/media-source/media-source-unnecessary-seek-seeked.html [ Failure ]
-webkit.org/b/219822 imported/w3c/web-platform-tests/media-source/mediasource-video-is-visible.html [ Timeout ]
 webkit.org/b/219822 media/encrypted-media/clearKey/clearKey-cenc-audio-playback-mse.html [ Timeout ]
 webkit.org/b/219822 media/encrypted-media/clearKey/clearKey-cenc-video-playback-mse.html [ Timeout ]
 webkit.org/b/219822 media/media-source/media-source-paint-to-canvas.html [ Timeout ]

Modified: trunk/Source/WebCore/ChangeLog (271023 => 271024)


--- trunk/Source/WebCore/ChangeLog	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebCore/ChangeLog	2020-12-21 18:14:08 UTC (rev 271024)
@@ -1,3 +1,44 @@
+2020-12-21  Alicia Boya García  <[email protected]>
+
+        [MSE] Fix allSamplesInTrackEnqueued() handling
+        https://bugs.webkit.org/show_bug.cgi?id=220055
+
+        Reviewed by Philippe Normand.
+
+        Within the changes introduced by r270612 a spurious call to
+        allSamplesInTrackEnqueued() was added to provideMediaData().
+
+        provideMediaData() is called every time there may be new samples that
+        need to be enqueued into the playback pipeline.
+
+        allSamplesInTrackEnqueued() is supposed to be called when
+        MediaSource.endOfStream() has been called and all pending samples have
+        been enqueued, therefore signaling the playback pipeline that no more
+        samples will be added.
+
+        Some decoders need to be notified of this condition in order to move
+        the last samples in their queue downstream. This is true at least of
+        the avdec (ffmpeg) decoders that are commonly used in desktop versions
+        of the GStreamer port.
+
+        Calling allSamplesInTrackEnqueued() prematurely will mess with the
+        playback as the code will not accept any more samples, a serious
+        problem. This patch fixes it by removing the spurious call and
+        restoring its original purpose when it was introduced in r230909.
+
+        * Modules/mediasource/MediaSource.cpp:
+        (WebCore::MediaSource::streamEndedWithError):
+        (WebCore::MediaSource::openIfInEndedState):
+        * Modules/mediasource/SourceBuffer.cpp:
+        (WebCore::SourceBuffer::setMediaSourceEnded):
+        (WebCore::SourceBuffer::trySignalAllSamplesEnqueued): Deleted.
+        * Modules/mediasource/SourceBuffer.h:
+        * platform/graphics/SourceBufferPrivate.cpp:
+        (WebCore::SourceBufferPrivate::setMediaSourceEnded):
+        (WebCore::SourceBufferPrivate::trySignalAllSamplesInTrackEnqueued):
+        (WebCore::SourceBufferPrivate::provideMediaData):
+        * platform/graphics/SourceBufferPrivate.h:
+
 2020-12-21  Aditya Keerthi  <[email protected]>
 
         REGRESSION (r262729): Poor contrast for specific color/background combinations on disabled input fields

Modified: trunk/Source/WebCore/Modules/mediasource/MediaSource.cpp (271023 => 271024)


--- trunk/Source/WebCore/Modules/mediasource/MediaSource.cpp	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebCore/Modules/mediasource/MediaSource.cpp	2020-12-21 18:14:08 UTC (rev 271024)
@@ -627,7 +627,7 @@
 
         // 2. Notify the media element that it now has all of the media data.
         for (auto& sourceBuffer : *m_sourceBuffers)
-            sourceBuffer->trySignalAllSamplesEnqueued();
+            sourceBuffer->setMediaSourceEnded(true);
         m_private->markEndOfStream(MediaSourcePrivate::EosNoError);
     } else if (error == EndOfStreamError::Network) {
         m_private->markEndOfStream(MediaSourcePrivate::EosNetworkError);
@@ -1029,6 +1029,8 @@
 
     setReadyState(ReadyState::Open);
     m_private->unmarkEndOfStream();
+    for (auto& sourceBuffer : *m_sourceBuffers)
+        sourceBuffer->setMediaSourceEnded(false);
 }
 
 bool MediaSource::virtualHasPendingActivity() const

Modified: trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp (271023 => 271024)


--- trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp	2020-12-21 18:14:08 UTC (rev 271024)
@@ -1125,15 +1125,6 @@
         m_source->mediaElement()->textTrackKindChanged(track);
 }
 
-void SourceBuffer::trySignalAllSamplesEnqueued()
-{
-    if (isRemoved() || !m_source->isEnded())
-        return;
-
-    DEBUG_LOG(LOGIDENTIFIER, "Try to enqueue all samples");
-    m_private->trySignalAllSamplesInTrackEnqueued();
-}
-
 void SourceBuffer::sourceBufferPrivateDidParseSample(double frameDuration)
 {
     m_bufferedSinceLastMonitor += frameDuration;
@@ -1313,6 +1304,11 @@
     m_bufferedDirty = flag;
 }
 
+void SourceBuffer::setMediaSourceEnded(bool isEnded)
+{
+    m_private->setMediaSourceEnded(isEnded);
+}
+
 #if !RELEASE_LOG_DISABLED
 WTFLogChannel& SourceBuffer::logChannel() const
 {

Modified: trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h (271023 => 271024)


--- trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h	2020-12-21 18:14:08 UTC (rev 271024)
@@ -127,7 +127,7 @@
     MediaTime highestPresentationTimestamp() const;
     void readyStateChanged();
 
-    void trySignalAllSamplesEnqueued();
+    void setMediaSourceEnded(bool isEnded);
 
 #if !RELEASE_LOG_DISABLED
     const Logger& logger() const final { return m_logger.get(); }

Modified: trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.cpp (271023 => 271024)


--- trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.cpp	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.cpp	2020-12-21 18:14:08 UTC (rev 271024)
@@ -277,14 +277,31 @@
     setMinimumUpcomingPresentationTime(trackID, trackBuffer.minimumEnqueuedPresentationTime);
 }
 
-void SourceBufferPrivate::trySignalAllSamplesInTrackEnqueued()
+void SourceBufferPrivate::setMediaSourceEnded(bool isEnded)
 {
-    for (const AtomString& trackID : m_trackBufferMap.keys()) {
-        if (m_trackBufferMap.get(trackID).decodeQueue.empty())
-            allSamplesInTrackEnqueued(trackID);
+    if (m_isMediaSourceEnded == isEnded)
+        return;
+
+    m_isMediaSourceEnded = isEnded;
+
+    if (m_isMediaSourceEnded) {
+        for (auto& trackBufferPair : m_trackBufferMap) {
+            TrackBuffer& trackBuffer = trackBufferPair.value;
+            const AtomString& trackID = trackBufferPair.key;
+
+            trySignalAllSamplesInTrackEnqueued(trackBuffer, trackID);
+        }
     }
 }
 
+void SourceBufferPrivate::trySignalAllSamplesInTrackEnqueued(TrackBuffer& trackBuffer, const AtomString& trackID)
+{
+    if (m_isMediaSourceEnded && trackBuffer.decodeQueue.empty()) {
+        DEBUG_LOG(LOGIDENTIFIER, "All samples in track \"", trackID, "\" enqueued.");
+        allSamplesInTrackEnqueued(trackID);
+    }
+}
+
 void SourceBufferPrivate::provideMediaData(const AtomString& trackID)
 {
     auto it = m_trackBufferMap.find(trackID);
@@ -347,7 +364,7 @@
     DEBUG_LOG(LOGIDENTIFIER, "enqueued ", enqueuedSamples, " samples, ", static_cast<uint64_t>(trackBuffer.decodeQueue.size()), " remaining");
 #endif
 
-    allSamplesInTrackEnqueued(trackID);
+    trySignalAllSamplesInTrackEnqueued(trackBuffer, trackID);
 }
 
 void SourceBufferPrivate::reenqueueMediaForTime(TrackBuffer& trackBuffer, const AtomString& trackID, const MediaTime& time)

Modified: trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h (271023 => 271024)


--- trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h	2020-12-21 18:14:08 UTC (rev 271024)
@@ -79,7 +79,6 @@
     virtual void addTrackBuffer(const AtomString& trackId, RefPtr<MediaDescription>&&);
     virtual void resetTrackBuffers();
     virtual void clearTrackBuffers();
-    virtual void trySignalAllSamplesInTrackEnqueued();
     WEBCORE_EXPORT virtual void updateBufferedFromTrackBuffers(bool sourceIsEnded);
     WEBCORE_EXPORT virtual void evictCodedFrames(uint64_t newDataSize, uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded);
     virtual void resetTimestampOffsetInTrackBuffers();
@@ -106,6 +105,7 @@
     void setGroupStartTimestamp(const MediaTime& mediaTime) { m_groupStartTimestamp = mediaTime; }
     void setGroupStartTimestampToEndTimestamp() { m_groupStartTimestamp = m_groupEndTimestamp; }
     uint64_t totalTrackBufferSizeInBytes() const;
+    void setMediaSourceEnded(bool isEnded);
 
     struct TrackBuffer {
         MediaTime lastDecodeTimestamp;
@@ -173,6 +173,7 @@
     void provideMediaData(TrackBuffer&, const AtomString& trackID);
     void setBufferedRanges(const PlatformTimeRanges&);
     void setBufferedDirty(bool);
+    void trySignalAllSamplesInTrackEnqueued(TrackBuffer&, const AtomString& trackID);
 
     bool m_isAttached { false };
     bool m_hasAudio { false };
@@ -194,6 +195,7 @@
     MediaTime m_groupEndTimestamp { MediaTime::zeroTime() };
 
     bool m_bufferFull { false };
+    bool m_isMediaSourceEnded { false };
     RefPtr<TimeRanges> m_buffered;
 };
 

Modified: trunk/Source/WebKit/ChangeLog (271023 => 271024)


--- trunk/Source/WebKit/ChangeLog	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebKit/ChangeLog	2020-12-21 18:14:08 UTC (rev 271024)
@@ -1,3 +1,21 @@
+2020-12-21  Alicia Boya García  <[email protected]>
+
+        [MSE] Fix allSamplesInTrackEnqueued() handling
+        https://bugs.webkit.org/show_bug.cgi?id=220055
+
+        Reviewed by Philippe Normand.
+
+        trySignalAllSamplesInTrackEnqueued() does not need to be exposed
+        outside of SourceBufferPrivate.
+
+        * GPUProcess/media/RemoteSourceBufferProxy.cpp:
+        (WebKit::RemoteSourceBufferProxy::trySignalAllSamplesInTrackEnqueued): Deleted.
+        * GPUProcess/media/RemoteSourceBufferProxy.h:
+        * GPUProcess/media/RemoteSourceBufferProxy.messages.in:
+        * WebProcess/GPU/media/SourceBufferPrivateRemote.cpp:
+        (WebKit::SourceBufferPrivateRemote::trySignalAllSamplesInTrackEnqueued): Deleted.
+        * WebProcess/GPU/media/SourceBufferPrivateRemote.h:
+
 2020-12-20  Chris Dumez  <[email protected]>
 
         Unreviewed, reverting r270969.

Modified: trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp (271023 => 271024)


--- trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.cpp	2020-12-21 18:14:08 UTC (rev 271024)
@@ -217,11 +217,6 @@
     m_sourceBufferPrivate->reenqueueMediaIfNeeded(currentMediaTime, pendingAppendDataCapacity, maximumBufferSize);
 }
 
-void RemoteSourceBufferProxy::trySignalAllSamplesInTrackEnqueued()
-{
-    m_sourceBufferPrivate->trySignalAllSamplesInTrackEnqueued();
-}
-
 void RemoteSourceBufferProxy::resetTimestampOffsetInTrackBuffers()
 {
     m_sourceBufferPrivate->resetTimestampOffsetInTrackBuffers();

Modified: trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h (271023 => 271024)


--- trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.h	2020-12-21 18:14:08 UTC (rev 271024)
@@ -94,7 +94,6 @@
     void resetTrackBuffers();
     void clearTrackBuffers();
     void reenqueueMediaIfNeeded(const MediaTime& currentMediaTime, uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize);
-    void trySignalAllSamplesInTrackEnqueued();
     void resetTimestampOffsetInTrackBuffers();
     void setTimestampOffset(const MediaTime&);
     void setAppendWindowStart(const MediaTime&);

Modified: trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in (271023 => 271024)


--- trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebKit/GPUProcess/media/RemoteSourceBufferProxy.messages.in	2020-12-21 18:14:08 UTC (rev 271024)
@@ -38,7 +38,6 @@
     ResetTrackBuffers()
     ClearTrackBuffers()
     ReenqueueMediaIfNeeded(MediaTime currentMediaTime, uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize)
-    TrySignalAllSamplesInTrackEnqueued()
     ResetTimestampOffsetInTrackBuffers()
     SetTimestampOffset(MediaTime timestampOffset)
     SetAppendWindowStart(MediaTime appendWindowStart)

Modified: trunk/Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp (271023 => 271024)


--- trunk/Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp	2020-12-21 18:14:08 UTC (rev 271024)
@@ -158,11 +158,6 @@
     m_gpuProcessConnection.connection().send(Messages::RemoteSourceBufferProxy::ReenqueueMediaIfNeeded(currentMediaTime, pendingAppendDataCapacity, maximumBufferSize), m_remoteSourceBufferIdentifier);
 }
 
-void SourceBufferPrivateRemote::trySignalAllSamplesInTrackEnqueued()
-{
-    m_gpuProcessConnection.connection().send(Messages::RemoteSourceBufferProxy::TrySignalAllSamplesInTrackEnqueued(), m_remoteSourceBufferIdentifier);
-}
-
 void SourceBufferPrivateRemote::resetTimestampOffsetInTrackBuffers()
 {
     m_gpuProcessConnection.connection().send(Messages::RemoteSourceBufferProxy::ResetTimestampOffsetInTrackBuffers(), m_remoteSourceBufferIdentifier);

Modified: trunk/Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h (271023 => 271024)


--- trunk/Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h	2020-12-21 18:11:44 UTC (rev 271023)
+++ trunk/Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h	2020-12-21 18:14:08 UTC (rev 271024)
@@ -84,7 +84,6 @@
     void addTrackBuffer(const AtomString& trackId, RefPtr<WebCore::MediaDescription>&&) final;
     void resetTrackBuffers() final;
     void clearTrackBuffers() final;
-    void trySignalAllSamplesInTrackEnqueued() final;
     void updateBufferedFromTrackBuffers(bool sourceIsEnded) final;
     void evictCodedFrames(uint64_t newDataSize, uint64_t pendingAppendDataCapacity, uint64_t maximumBufferSize, const MediaTime& currentTime, const MediaTime& duration, bool isEnded) final;
     void resetTimestampOffsetInTrackBuffers() final;
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to