Diff
Modified: trunk/LayoutTests/ChangeLog (206000 => 206001)
--- trunk/LayoutTests/ChangeLog 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/LayoutTests/ChangeLog 2016-09-15 23:13:56 UTC (rev 206001)
@@ -1,3 +1,12 @@
+2016-09-13 Jer Noble <jer.no...@apple.com>
+
+ [media-source] web-platform-test/media-source/mediasource-remove.html test failing
+ https://bugs.webkit.org/show_bug.cgi?id=161950
+
+ Reviewed by Eric Carlson.
+
+ * platform/mac/TestExpectations:
+
2016-09-15 Zalan Bujtas <za...@apple.com>
ASSERTION FAILED: willBeComposited == needsToBeComposited(layer) in WebCore::RenderLayerCompositor::computeCompositingRequirements
Modified: trunk/LayoutTests/platform/mac/TestExpectations (206000 => 206001)
--- trunk/LayoutTests/platform/mac/TestExpectations 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/LayoutTests/platform/mac/TestExpectations 2016-09-15 23:13:56 UTC (rev 206001)
@@ -1052,6 +1052,7 @@
[ Yosemite+ ] imported/w3c/web-platform-tests/media-source/mediasource-endofstream-invaliderror.html [ Pass ]
[ Yosemite+ ] imported/w3c/web-platform-tests/media-source/mediasource-multiple-attach.html [ Pass ]
[ Yosemite+ ] imported/w3c/web-platform-tests/media-source/mediasource-play.html [ Pass ]
+[ Yosemite+ ] imported/w3c/web-platform-tests/media-source/mediasource-remove.html [ Pass ]
[ Yosemite+ ] imported/w3c/web-platform-tests/media-source/mediasource-sourcebufferlist.html [ Pass ]
# Flaky Media Source tests
@@ -1061,7 +1062,6 @@
# Newly failing Media Source tests
webkit.org/b/161725 [ Yosemite+ ] imported/w3c/web-platform-tests/media-source/mediasource-append-buffer.html [ Failure ]
webkit.org/b/161725 [ Yosemite+ ] imported/w3c/web-platform-tests/media-source/mediasource-is-type-supported.html [ Failure ]
-webkit.org/b/161725 [ Yosemite+ ] imported/w3c/web-platform-tests/media-source/mediasource-remove.html [ Failure ]
webkit.org/b/161725 [ Yosemite+ ] imported/w3c/web-platform-tests/media-source/mediasource-sourcebuffer-mode.html [ Failure ]
# These two tests have "InvalidStateError (DOM Exception 11): The object is in an invalid state." in output.
Modified: trunk/Source/WebCore/ChangeLog (206000 => 206001)
--- trunk/Source/WebCore/ChangeLog 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/ChangeLog 2016-09-15 23:13:56 UTC (rev 206001)
@@ -1,3 +1,89 @@
+2016-09-13 Jer Noble <jer.no...@apple.com>
+
+ [media-source] web-platform-test/media-source/mediasource-remove.html test failing
+ https://bugs.webkit.org/show_bug.cgi?id=161950
+
+ Reviewed by Eric Carlson.
+
+ Fixes test: web-platform-test/media-source/mediasource-remove.html
+
+ The mediasource-remove.html test was failing in a number of ways:
+
+ - Tests with invalid start or end times were not throwing the correct exception
+ code, or not throwing exception codes at all
+
+ - Tests were showing an incorrect start buffered range at the beginning of each test.
+
+ - Tests which removed samples were not getting the expected buffered values at the end
+ each test.
+
+ For the exception failures, update the implementation of abort() and remove() to throw
+ the correct exceptions at the correct times.
+
+ For the incorrect initial buffered range, update our buffered calculations to store
+ individual PlatformTimeRanges on each TrackBuffer, and coalesce them into a single
+ value when an append operation completes, a remove operation completes, or when the
+ MediaSource's ready state changes.
+
+ For the incorrect buffered ranges after removal, this is caused because the "samples"
+ that make up an audio track are actually a collection of a large number of individual
+ samples. So when we are asked to remove media data in a given range, break these audio
+ meta-samples into two pieces at the removal points. This allows the removal algorithm
+ to operate on a individual audio sample basis. (We should look into using this technique
+ when audio samples are evicted during an append operation.) This requires adding some
+ methods to MediaSample and it's concrete subclasses to "divide" a sample into two at
+ a given presentation time.
+
+ Fixing these behaviors, however, breaks the media-source-end-of-stream-buffered.html
+ test, which expects the buffered range for the entire element to expand to the maximum
+ buffered time of any of the element's MediaSource's active sourceBuffers. To fix this,
+ update the MediaSource's monitorSourceBuffer() implementation to match the current
+ specification. The new spec no longer queries the individual SourceBuffers, but rather
+ queries the already coalesced buffered ranges. So move the helper methods hasCurrentTime()
+ hasFutureTime(), and canPlayThrough() up into MediaSource from SourceBuffer. Also, update
+ seekToTime, since it has the same problem as monitorSourceBuffer().
+
+ However, this breaks the media-source-monitor-source-buffers.html test, which appends
+ 10s of movie data instantaneously, then never appends again. The SourceBuffer's
+ monitorBufferingRate() method only re-evaluates the rate after data has been appended,
+ so the SourceBuffer thinks it's buffered data at a prodigious rate forever. Instead,
+ allow the SourceBuffer to continuously re-evalute it's buffering rate by modifying the
+ exponential moving average so that the co-efficient scales based on how frequently the
+ method is called. Call the method more often, and the moving average changes less quickly,
+ and it means that when media is stalled out at the end of a buffered range, the readyState
+ of a video element will eventually drop to HAVE_CURRENT_DATA when the average buffering
+ rate falls below the level where playback would continue uninterrupted.
+
+ * Modules/mediasource/MediaSource.cpp:
+ (WebCore::MediaSource::seekToTime):
+ (WebCore::MediaSource::currentTimeFudgeFactor):
+ (WebCore::MediaSource::hasBufferedTime):
+ (WebCore::MediaSource::hasCurrentTime):
+ (WebCore::MediaSource::hasFutureTime):
+ (WebCore::MediaSource::monitorSourceBuffers):
+ * Modules/mediasource/MediaSource.h:
+ * Modules/mediasource/SourceBuffer.cpp:
+ (WebCore::SourceBuffer::abort):
+ (WebCore::SourceBuffer::remove):
+ (WebCore::SourceBuffer::abortIfUpdating):
+ (WebCore::SourceBuffer::removeCodedFrames):
+ (WebCore::SourceBuffer::sourceBufferPrivateDidReceiveSample):
+ (WebCore::SourceBuffer::updateBufferedFromTrackBuffers):
+ (WebCore::SourceBuffer::canPlayThroughRange):
+ (WebCore::SourceBuffer::monitorBufferingRate):
+ (WebCore::currentTimeFudgeFactor): Deleted.
+ (WebCore::SourceBuffer::hasCurrentTime): Deleted.
+ (WebCore::SourceBuffer::hasFutureTime): Deleted.
+ (WebCore::SourceBuffer::canPlayThrough): Deleted.
+ * platform/MediaSample.h:
+ * platform/cf/CoreMediaSoftLink.cpp:
+ * platform/cf/CoreMediaSoftLink.h:
+ * platform/graphics/avfoundation/MediaSampleAVFObjC.h:
+ * platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm:
+ (WebCore::MediaSampleAVFObjC::isDivisable):
+ (WebCore::MediaSampleAVFObjC::divide):
+ * platform/mock/mediasource/MockSourceBufferPrivate.cpp:
+
2016-09-15 Zalan Bujtas <za...@apple.com>
ASSERTION FAILED: willBeComposited == needsToBeComposited(layer) in WebCore::RenderLayerCompositor::computeCompositingRequirements
Modified: trunk/Source/WebCore/Modules/mediasource/MediaSource.cpp (206000 => 206001)
--- trunk/Source/WebCore/Modules/mediasource/MediaSource.cpp 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/Modules/mediasource/MediaSource.cpp 2016-09-15 23:13:56 UTC (rev 206001)
@@ -214,7 +214,7 @@
void MediaSource::seekToTime(const MediaTime& time)
{
// 2.4.3 Seeking
- // https://dvcs.w3.org/hg/html-media/raw-file/tip/media-source/media-source.html#mediasource-seeking
+ // https://rawgit.com/w3c/media-source/45627646344eea0170dd1cbc5a3d508ca751abb8/media-source-respec.html#mediasource-seeking
m_pendingSeekTime = time;
@@ -221,25 +221,21 @@
// Run the following steps as part of the "Wait until the user agent has established whether or not the
// media data for the new playback position is available, and, if it is, until it has decoded enough data
// to play back that position" step of the seek algorithm:
- // 1. The media element looks for media segments containing the new playback position in each SourceBuffer
- // object in activeSourceBuffers.
- for (auto& sourceBuffer : *m_activeSourceBuffers) {
- // ↳ If one or more of the objects in activeSourceBuffers is missing media segments for the new
- // playback position
- if (!sourceBuffer->buffered()->ranges().contain(time)) {
- // 1.1 Set the HTMLMediaElement.readyState attribute to HAVE_METADATA.
- m_private->setReadyState(MediaPlayer::HaveMetadata);
+ // ↳ If new playback position is not in any TimeRange of HTMLMediaElement.buffered
+ if (!hasBufferedTime(time)) {
+ // 1. If the HTMLMediaElement.readyState attribute is greater than HAVE_METADATA,
+ // then set the HTMLMediaElement.readyState attribute to HAVE_METADATA.
+ m_private->setReadyState(MediaPlayer::HaveMetadata);
- // 1.2 The media element waits until an appendBuffer() or an appendStream() call causes the coded
- // frame processing algorithm to set the HTMLMediaElement.readyState attribute to a value greater
- // than HAVE_METADATA.
- LOG(MediaSource, "MediaSource::seekToTime(%p) - waitForSeekCompleted()", this);
- m_private->waitForSeekCompleted();
- return;
- }
- // ↳ Otherwise
- // Continue
+ // 2. The media element waits until an appendBuffer() or an appendStream() call causes the coded
+ // frame processing algorithm to set the HTMLMediaElement.readyState attribute to a value greater
+ // than HAVE_METADATA.
+ LOG(MediaSource, "MediaSource::seekToTime(%p) - waitForSeekCompleted()", this);
+ m_private->waitForSeekCompleted();
+ return;
}
+ // ↳ Otherwise
+ // Continue
completeSeek();
}
@@ -265,10 +261,58 @@
monitorSourceBuffers();
}
+const MediaTime& MediaSource::currentTimeFudgeFactor()
+{
+ // Allow hasCurrentTime() to be off by as much as the length of two 24fps video frames
+ static NeverDestroyed<MediaTime> fudgeFactor(2002, 24000);
+ return fudgeFactor;
+}
+
+bool MediaSource::hasBufferedTime(const MediaTime& time)
+{
+ if (time >= duration())
+ return false;
+
+ auto ranges = buffered();
+ if (!ranges->length())
+ return false;
+
+ return abs(ranges->nearest(time) - time) <= currentTimeFudgeFactor();
+}
+
+bool MediaSource::hasCurrentTime()
+{
+ return hasBufferedTime(currentTime());
+}
+
+bool MediaSource::hasFutureTime()
+{
+ MediaTime currentTime = this->currentTime();
+ MediaTime duration = this->duration();
+
+ if (currentTime >= duration)
+ return true;
+
+ auto ranges = buffered();
+ MediaTime nearest = ranges->nearest(currentTime);
+ if (abs(nearest - currentTime) > currentTimeFudgeFactor())
+ return false;
+
+ size_t found = ranges->find(nearest);
+ if (found == notFound)
+ return false;
+
+ MediaTime localEnd = ranges->end(found);
+ if (localEnd == duration)
+ return true;
+
+ return localEnd - currentTime > currentTimeFudgeFactor();
+}
+
void MediaSource::monitorSourceBuffers()
{
// 2.4.4 SourceBuffer Monitoring
- // https://dvcs.w3.org/hg/html-media/raw-file/default/media-source/media-source.html#buffer-monitoring
+ // https://rawgit.com/w3c/media-source/45627646344eea0170dd1cbc5a3d508ca751abb8/media-source-respec.html#buffer-monitoring
// Note, the behavior if activeSourceBuffers is empty is undefined.
if (!m_activeSourceBuffers) {
@@ -276,7 +320,6 @@
return;
}
- // http://w3c.github.io/media-source/#buffer-monitoring, change from 11 December 2014
// ↳ If the the HTMLMediaElement.readyState attribute equals HAVE_NOTHING:
if (mediaElement()->readyState() == HTMLMediaElement::HAVE_NOTHING) {
// 1. Abort these steps.
@@ -283,13 +326,8 @@
return;
}
- // ↳ If buffered for all objects in activeSourceBuffers do not contain TimeRanges for the current
- // playback position:
- auto begin = m_activeSourceBuffers->begin();
- auto end = m_activeSourceBuffers->end();
- if (std::all_of(begin, end, [](auto& sourceBuffer) {
- return !sourceBuffer->hasCurrentTime();
- })) {
+ // ↳ If HTMLMediaElement.buffered does not contain a TimeRange for the current playback position:
+ if (!hasCurrentTime()) {
// 1. Set the HTMLMediaElement.readyState attribute to HAVE_METADATA.
// 2. If this is the first transition to HAVE_METADATA, then queue a task to fire a simple event
// named loadedmetadata at the media element.
@@ -299,10 +337,11 @@
return;
}
- // ↳ If buffered for all objects in activeSourceBuffers contain TimeRanges that include the current
- // playback position and enough data to ensure uninterrupted playback:
- if (std::all_of(begin, end, [](auto& sourceBuffer) {
- return sourceBuffer->hasFutureTime() && sourceBuffer->canPlayThrough();
+ // ↳ If HTMLMediaElement.buffered contains a TimeRange that includes the current
+ // playback position and enough data to ensure uninterrupted playback:
+ auto ranges = buffered();
+ if (std::all_of(m_activeSourceBuffers->begin(), m_activeSourceBuffers->end(), [&](auto& sourceBuffer) {
+ return sourceBuffer->canPlayThroughRange(*ranges);
})) {
// 1. Set the HTMLMediaElement.readyState attribute to HAVE_ENOUGH_DATA.
// 2. Queue a task to fire a simple event named canplaythrough at the media element.
@@ -316,11 +355,9 @@
return;
}
- // ↳ If buffered for all objects in activeSourceBuffers contain a TimeRange that includes
- // the current playback position and some time beyond the current playback position, then run the following steps:
- if (std::all_of(begin, end, [](auto& sourceBuffer) {
- return sourceBuffer->hasFutureTime();
- })) {
+ // ↳ If HTMLMediaElement.buffered contains a TimeRange that includes the current playback
+ // position and some time beyond the current playback position, then run the following steps:
+ if (hasFutureTime()) {
// 1. Set the HTMLMediaElement.readyState attribute to HAVE_FUTURE_DATA.
// 2. If the previous value of HTMLMediaElement.readyState was less than HAVE_FUTURE_DATA, then queue a task to fire a simple event named canplay at the media element.
// 3. Playback may resume at this point if it was previously suspended by a transition to HAVE_CURRENT_DATA.
@@ -333,9 +370,7 @@
return;
}
- // ↳ If buffered for at least one object in activeSourceBuffers contains a TimeRange that ends
- // at the current playback position and does not have a range covering the time immediately
- // after the current position:
+ // ↳ If HTMLMediaElement.buffered contains a TimeRange that ends at the current playback position and does not have a range covering the time immediately after the current position:
// NOTE: Logically, !(all objects do not contain currentTime) == (some objects contain current time)
// 1. Set the HTMLMediaElement.readyState attribute to HAVE_CURRENT_DATA.
@@ -840,6 +875,9 @@
void MediaSource::onReadyStateChange(const AtomicString& oldState, const AtomicString& newState)
{
+ for (auto& buffer : *m_sourceBuffers)
+ buffer->readyStateChanged();
+
if (isOpen()) {
scheduleEvent(eventNames().sourceopenEvent);
return;
Modified: trunk/Source/WebCore/Modules/mediasource/MediaSource.h (206000 => 206001)
--- trunk/Source/WebCore/Modules/mediasource/MediaSource.h 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/Modules/mediasource/MediaSource.h 2016-09-15 23:13:56 UTC (rev 206001)
@@ -114,6 +114,8 @@
// ActiveDOMObject API.
bool hasPendingActivity() const override;
+ static const MediaTime& currentTimeFudgeFactor();
+
protected:
explicit MediaSource(ScriptExecutionContext&);
@@ -129,6 +131,10 @@
void scheduleEvent(const AtomicString& eventName);
GenericEventQueue& asyncEventQueue() { return m_asyncEventQueue; }
+ bool hasBufferedTime(const MediaTime&);
+ bool hasCurrentTime();
+ bool hasFutureTime();
+
void regenerateActiveSourceBuffers();
static URLRegistry* s_registry;
Modified: trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp (206000 => 206001)
--- trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp 2016-09-15 23:13:56 UTC (rev 206001)
@@ -65,13 +65,6 @@
static const double ExponentialMovingAverageCoefficient = 0.1;
-// Allow hasCurrentTime() to be off by as much as the length of two 24fps video frames
-static const MediaTime& currentTimeFudgeFactor()
-{
- static NeverDestroyed<MediaTime> fudgeFactor(2002, 24000);
- return fudgeFactor;
-}
-
struct SourceBuffer::TrackBuffer {
MediaTime lastDecodeTimestamp;
MediaTime lastFrameDuration;
@@ -84,6 +77,7 @@
SampleMap samples;
DecodeOrderSampleMap::MapType decodeQueue;
RefPtr<MediaDescription> description;
+ PlatformTimeRanges buffered;
TrackBuffer()
: lastDecodeTimestamp(MediaTime::invalidTime())
@@ -288,7 +282,7 @@
void SourceBuffer::abort(ExceptionCode& ec)
{
// Section 3.2 abort() method steps.
- // https://dvcs.w3.org/hg/html-media/raw-file/default/media-source/media-source.html#widl-SourceBuffer-abort-void
+ // https://rawgit.com/w3c/media-source/45627646344eea0170dd1cbc5a3d508ca751abb8/media-source-respec.html#dom-sourcebuffer-abort
// 1. If this object has been removed from the sourceBuffers attribute of the parent media source
// then throw an INVALID_STATE_ERR exception and abort these steps.
// 2. If the readyState attribute of the parent media source is not in the "open" state
@@ -298,16 +292,22 @@
return;
}
- // 3. If the sourceBuffer.updating attribute equals true, then run the following steps: ...
+ // 3. If the range removal algorithm is running, then throw an InvalidStateError exception and abort these steps.
+ if (m_removeTimer.isActive()) {
+ ec = INVALID_STATE_ERR;
+ return;
+ }
+
+ // 4. If the sourceBuffer.updating attribute equals true, then run the following steps: ...
abortIfUpdating();
- // 4. Run the reset parser state algorithm.
+ // 5. Run the reset parser state algorithm.
resetParserState();
- // 5. Set appendWindowStart to the presentation start time.
+ // 6. Set appendWindowStart to the presentation start time.
m_appendWindowStart = MediaTime::zeroTime();
- // 6. Set appendWindowEnd to positive Infinity.
+ // 7. Set appendWindowEnd to positive Infinity.
m_appendWindowEnd = MediaTime::positiveInfiniteTime();
}
@@ -320,25 +320,26 @@
{
LOG(MediaSource, "SourceBuffer::remove(%p) - start(%lf), end(%lf)", this, start.toDouble(), end.toDouble());
+ // https://rawgit.com/w3c/media-source/45627646344eea0170dd1cbc5a3d508ca751abb8/media-source-respec.html#dom-sourcebuffer-remove
// Section 3.2 remove() method steps.
- // 1. If duration equals NaN, then throw an InvalidAccessError exception and abort these steps.
- // 2. If start is negative or greater than duration, then throw an InvalidAccessError exception and abort these steps.
- // 3. If end is less than or equal to start, then throw an InvalidAccessError exception and abort these steps.
-
- // FIXME: reorder/revisit this section once <https://www.w3.org/Bugs/Public/show_bug.cgi?id=27857> got resolved
- // as it seems wrong to check mediaSource duration before checking isRemoved().
- if ((m_source && m_source->duration().isInvalid())
- || start < MediaTime::zeroTime() || (m_source && start > m_source->duration())
- || end <= start) {
- ec = INVALID_ACCESS_ERR;
+ // 1. If this object has been removed from the sourceBuffers attribute of the parent media source then throw
+ // an InvalidStateError exception and abort these steps.
+ // 2. If the updating attribute equals true, then throw an InvalidStateError exception and abort these steps.
+ if (isRemoved() || m_updating) {
+ ec = INVALID_STATE_ERR;
return;
}
- // 4. If this object has been removed from the sourceBuffers attribute of the parent media source then throw an
- // InvalidStateError exception and abort these steps.
- // 5. If the updating attribute equals true, then throw an InvalidStateError exception and abort these steps.
- if (isRemoved() || m_updating) {
- ec = INVALID_STATE_ERR;
+ // 3. If duration equals NaN, then throw a TypeError exception and abort these steps.
+ // 4. If start is negative or greater than duration, then throw a TypeError exception and abort these steps.
+ // 5. If end is less than or equal to start or end equals NaN, then throw a TypeError exception and abort these steps.
+ if (m_source->duration().isInvalid()
+ || end.isInvalid()
+ || start.isInvalid()
+ || start < MediaTime::zeroTime()
+ || start > m_source->duration()
+ || end <= start) {
+ ec = TypeError;
return;
}
@@ -371,27 +372,23 @@
void SourceBuffer::abortIfUpdating()
{
- // Section 3.2 abort() method step 3 substeps.
- // https://dvcs.w3.org/hg/html-media/raw-file/default/media-source/media-source.html#widl-SourceBuffer-abort-void
+ // Section 3.2 abort() method step 4 substeps.
+ // https://rawgit.com/w3c/media-source/45627646344eea0170dd1cbc5a3d508ca751abb8/media-source-respec.html#dom-sourcebuffer-abort
if (!m_updating)
return;
- // 3.1. Abort the buffer append and stream append loop algorithms if they are running.
+ // 4.1. Abort the buffer append algorithm if it is running.
m_appendBufferTimer.stop();
m_pendingAppendData.clear();
- m_removeTimer.stop();
- m_pendingRemoveStart = MediaTime::invalidTime();
- m_pendingRemoveEnd = MediaTime::invalidTime();
-
- // 3.2. Set the updating attribute to false.
+ // 4.2. Set the updating attribute to false.
m_updating = false;
- // 3.3. Queue a task to fire a simple event named abort at this SourceBuffer object.
+ // 4.3. Queue a task to fire a simple event named abort at this SourceBuffer object.
scheduleEvent(eventNames().abortEvent);
- // 3.4. Queue a task to fire a simple event named updateend at this SourceBuffer object.
+ // 4.4. Queue a task to fire a simple event named updateend at this SourceBuffer object.
scheduleEvent(eventNames().updateendEvent);
}
@@ -407,6 +404,11 @@
return highestTime;
}
+void SourceBuffer::readyStateChanged()
+{
+ updateBufferedFromTrackBuffers();
+}
+
void SourceBuffer::removedFromMediaSource()
{
if (isRemoved())
@@ -595,6 +597,10 @@
if (isRemoved())
return;
+ // Resolve the changes it TrackBuffers' buffered ranges
+ // into the SourceBuffer's buffered ranges
+ updateBufferedFromTrackBuffers();
+
// Section 3.5.5 Buffer Append Algorithm, ctd.
// https://dvcs.w3.org/hg/html-media/raw-file/default/media-source/media-source.html#sourcebuffer-buffer-append
@@ -757,6 +763,29 @@
// 3.1. Let remove end timestamp be the current value of duration
// 3.2 If this track buffer has a random access point timestamp that is greater than or equal to end, then update
// remove end timestamp to that random access point timestamp.
+
+ // NOTE: To handle MediaSamples which may be an amalgamation of multiple shorter samples, find samples whose presentation
+ // interval straddles the start and end times, and divide them if possible:
+ auto divideSampleIfPossibleAtPresentationTime = [&] (const MediaTime& time) {
+ auto sampleIterator = trackBuffer.samples.presentationOrder().findSampleContainingPresentationTime(time);
+ if (sampleIterator == trackBuffer.samples.presentationOrder().end())
+ return;
+ if (!sampleIterator->second->isDivisable())
+ return;
+ std::pair<RefPtr<MediaSample>, RefPtr<MediaSample>> replacementSamples = sampleIterator->second->divide(time);
+ if (!replacementSamples.first || !replacementSamples.second)
+ return;
+ LOG(MediaSource, "SourceBuffer::removeCodedFrames(%p) - splitting sample (%s) into\n\t(%s)\n\t(%s)", this,
+ toString(sampleIterator->second).utf8().data(),
+ toString(replacementSamples.first).utf8().data(),
+ toString(replacementSamples.second).utf8().data());
+ trackBuffer.samples.removeSample(sampleIterator->second.get());
+ trackBuffer.samples.addSample(*replacementSamples.first);
+ trackBuffer.samples.addSample(*replacementSamples.second);
+ };
+ divideSampleIfPossibleAtPresentationTime(start);
+ divideSampleIfPossibleAtPresentationTime(end);
+
// NOTE: findSyncSampleAfterPresentationTime will return the next sync sample on or after the presentation time
// or decodeOrder().end() if no sync sample exists after that presentation time.
DecodeOrderSampleMap::iterator removeDecodeEnd = trackBuffer.samples.decodeOrder().findSyncSampleAfterPresentationTime(end);
@@ -792,7 +821,7 @@
}
erasedRanges.invert();
- m_buffered->ranges().intersectWith(erasedRanges);
+ trackBuffer.buffered.intersectWith(erasedRanges);
setBufferedDirty(true);
// 3.4 If this object is in activeSourceBuffers, the current playback position is greater than or equal to start
@@ -801,6 +830,8 @@
if (m_active && currentMediaTime >= start && currentMediaTime < end && m_private->readyState() > MediaPlayer::HaveMetadata)
m_private->setReadyState(MediaPlayer::HaveMetadata);
}
+
+ updateBufferedFromTrackBuffers();
// 4. If buffer full flag equals true and this object is ready to accept more bytes, then set the buffer full flag to false.
// No-op
@@ -1535,14 +1566,11 @@
// NOTE: Searching from the end of the trackBuffer will be vastly more efficient if the search range is
// near the end of the buffered range. Use a linear-backwards search if the search range is within one
// frame duration of the end:
- if (!m_buffered)
- break;
-
- unsigned bufferedLength = m_buffered->ranges().length();
+ unsigned bufferedLength = trackBuffer.buffered.length();
if (!bufferedLength)
break;
- MediaTime highestBufferedTime = m_buffered->ranges().maximumBufferedTime();
+ MediaTime highestBufferedTime = trackBuffer.buffered.maximumBufferedTime();
PresentationOrderSampleMap::iterator_range range;
if (highestBufferedTime - trackBuffer.highestPresentationTimestamp < trackBuffer.lastFrameDuration)
@@ -1581,7 +1609,7 @@
}
erasedRanges.invert();
- m_buffered->ranges().intersectWith(erasedRanges);
+ trackBuffer.buffered.intersectWith(erasedRanges);
setBufferedDirty(true);
}
@@ -1624,15 +1652,15 @@
// Eliminate small gaps between buffered ranges by coalescing
// disjoint ranges separated by less than a "fudge factor".
auto presentationEndTime = presentationTimestamp + frameDuration;
- auto nearestToPresentationStartTime = m_buffered->ranges().nearest(presentationTimestamp);
- if ((presentationTimestamp - nearestToPresentationStartTime).isBetween(MediaTime::zeroTime(), currentTimeFudgeFactor()))
+ auto nearestToPresentationStartTime = trackBuffer.buffered.nearest(presentationTimestamp);
+ if (nearestToPresentationStartTime.isValid() && (presentationTimestamp - nearestToPresentationStartTime).isBetween(MediaTime::zeroTime(), MediaSource::currentTimeFudgeFactor()))
presentationTimestamp = nearestToPresentationStartTime;
- auto nearestToPresentationEndTime = m_buffered->ranges().nearest(presentationEndTime);
- if ((nearestToPresentationEndTime - presentationEndTime).isBetween(MediaTime::zeroTime(), currentTimeFudgeFactor()))
+ auto nearestToPresentationEndTime = trackBuffer.buffered.nearest(presentationEndTime);
+ if (nearestToPresentationStartTime.isValid() && (nearestToPresentationEndTime - presentationEndTime).isBetween(MediaTime::zeroTime(), MediaSource::currentTimeFudgeFactor()))
presentationEndTime = nearestToPresentationEndTime;
- m_buffered->ranges().add(presentationTimestamp, presentationEndTime);
+ trackBuffer.buffered.add(presentationTimestamp, presentationEndTime);
m_bufferedSinceLastMonitor += frameDuration.toDouble();
setBufferedDirty(true);
@@ -1878,9 +1906,6 @@
void SourceBuffer::monitorBufferingRate()
{
- if (!m_bufferedSinceLastMonitor)
- return;
-
double now = monotonicallyIncreasingTime();
double interval = now - m_timeOfBufferingMonitor;
double rateSinceLastMonitor = m_bufferedSinceLastMonitor / interval;
@@ -1888,68 +1913,54 @@
m_timeOfBufferingMonitor = now;
m_bufferedSinceLastMonitor = 0;
- m_averageBufferRate = m_averageBufferRate * (1 - ExponentialMovingAverageCoefficient) + rateSinceLastMonitor * ExponentialMovingAverageCoefficient;
+ m_averageBufferRate += (interval * ExponentialMovingAverageCoefficient) * (rateSinceLastMonitor - m_averageBufferRate);
LOG(MediaSource, "SourceBuffer::monitorBufferingRate(%p) - m_avegareBufferRate: %lf", this, m_averageBufferRate);
}
-std::unique_ptr<PlatformTimeRanges> SourceBuffer::bufferedAccountingForEndOfStream() const
+void SourceBuffer::updateBufferedFromTrackBuffers()
{
- // FIXME: Revisit this method once the spec bug <https://www.w3.org/Bugs/Public/show_bug.cgi?id=26436> is resolved.
- auto virtualRanges = std::make_unique<PlatformTimeRanges>(m_buffered->ranges());
- if (m_source->isEnded()) {
- MediaTime start = virtualRanges->maximumBufferedTime();
- MediaTime end = m_source->duration();
- if (start <= end)
- virtualRanges->add(start, end);
+ // 3.1 Attributes, buffered
+ // https://rawgit.com/w3c/media-source/45627646344eea0170dd1cbc5a3d508ca751abb8/media-source-respec.html#dom-sourcebuffer-buffered
+
+ // 2. Let highest end time be the largest track buffer ranges end time across all the track buffers managed by this SourceBuffer object.
+ MediaTime highestEndTime = MediaTime::negativeInfiniteTime();
+ for (auto& trackBuffer : m_trackBufferMap.values()) {
+ if (!trackBuffer.buffered.length())
+ continue;
+ highestEndTime = std::max(highestEndTime, trackBuffer.buffered.maximumBufferedTime());
}
- return virtualRanges;
-}
-bool SourceBuffer::hasCurrentTime() const
-{
- if (isRemoved() || !m_buffered->length())
- return false;
+ // NOTE: Short circuit the following if none of the TrackBuffers have buffered ranges to avoid generating
+ // a single range of {0, 0}.
+ if (highestEndTime.isNegativeInfinite()) {
+ m_buffered->ranges() = PlatformTimeRanges();
+ return;
+ }
- MediaTime currentTime = m_source->currentTime();
- MediaTime duration = m_source->duration();
- if (currentTime >= duration)
- return true;
+ // 3. Let intersection ranges equal a TimeRange object containing a single range from 0 to highest end time.
+ PlatformTimeRanges intersectionRanges { MediaTime::zeroTime(), highestEndTime };
- std::unique_ptr<PlatformTimeRanges> ranges = bufferedAccountingForEndOfStream();
- return abs(ranges->nearest(currentTime) - currentTime) <= currentTimeFudgeFactor();
-}
+ // 4. For each audio and video track buffer managed by this SourceBuffer, run the following steps:
+ for (auto& trackBuffer : m_trackBufferMap.values()) {
+ // 4.1 Let track ranges equal the track buffer ranges for the current track buffer.
+ PlatformTimeRanges trackRanges = trackBuffer.buffered;
+ // 4.2 If readyState is "ended", then set the end time on the last range in track ranges to highest end time.
+ if (m_source->isEnded())
+ trackRanges.add(trackRanges.maximumBufferedTime(), highestEndTime);
-bool SourceBuffer::hasFutureTime() const
-{
- if (isRemoved())
- return false;
+ // 4.3 Let new intersection ranges equal the intersection between the intersection ranges and the track ranges.
+ // 4.4 Replace the ranges in intersection ranges with the new intersection ranges.
+ intersectionRanges.intersectWith(trackRanges);
+ }
- std::unique_ptr<PlatformTimeRanges> ranges = bufferedAccountingForEndOfStream();
- if (!ranges->length())
- return false;
-
- MediaTime currentTime = m_source->currentTime();
- MediaTime duration = m_source->duration();
- if (currentTime >= duration)
- return true;
-
- MediaTime nearest = ranges->nearest(currentTime);
- if (abs(nearest - currentTime) > currentTimeFudgeFactor())
- return false;
-
- size_t found = ranges->find(nearest);
- if (found == notFound)
- return false;
-
- MediaTime localEnd = ranges->end(found);
- if (localEnd == duration)
- return true;
-
- return localEnd - currentTime > currentTimeFudgeFactor();
+ // 5. If intersection ranges does not contain the exact same range information as the current value of this attribute,
+ // then update the current value of this attribute to intersection ranges.
+ m_buffered->ranges() = intersectionRanges;
+ setBufferedDirty(true);
}
-bool SourceBuffer::canPlayThrough()
+bool SourceBuffer::canPlayThroughRange(PlatformTimeRanges& ranges)
{
if (isRemoved())
return false;
@@ -1965,10 +1976,10 @@
MediaTime currentTime = m_source->currentTime();
MediaTime duration = m_source->duration();
- std::unique_ptr<PlatformTimeRanges> unbufferedRanges = bufferedAccountingForEndOfStream();
- unbufferedRanges->invert();
- unbufferedRanges->intersectWith(PlatformTimeRanges(currentTime, std::max(currentTime, duration)));
- MediaTime unbufferedTime = unbufferedRanges->totalDuration();
+ PlatformTimeRanges unbufferedRanges = ranges;
+ unbufferedRanges.invert();
+ unbufferedRanges.intersectWith(PlatformTimeRanges(currentTime, std::max(currentTime, duration)));
+ MediaTime unbufferedTime = unbufferedRanges.totalDuration();
if (!unbufferedTime.isValid())
return true;
Modified: trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h (206000 => 206001)
--- trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h 2016-09-15 23:13:56 UTC (rev 206001)
@@ -92,9 +92,7 @@
void removedFromMediaSource();
void seekToTime(const MediaTime&);
- bool hasCurrentTime() const;
- bool hasFutureTime() const;
- bool canPlayThrough();
+ bool canPlayThroughRange(PlatformTimeRanges&);
bool hasVideo() const;
bool hasAudio() const;
@@ -125,6 +123,7 @@
void setBufferedDirty(bool flag) { m_bufferedDirty = flag; }
MediaTime highestPresentationTimestamp() const;
+ void readyStateChanged();
// ActiveDOMObject API.
bool hasPendingActivity() const override;
@@ -191,7 +190,7 @@
size_t extraMemoryCost() const;
void reportExtraMemoryAllocated();
- std::unique_ptr<PlatformTimeRanges> bufferedAccountingForEndOfStream() const;
+ void updateBufferedFromTrackBuffers();
// Internals
friend class Internals;
Modified: trunk/Source/WebCore/platform/MediaSample.h (206000 => 206001)
--- trunk/Source/WebCore/platform/MediaSample.h 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/platform/MediaSample.h 2016-09-15 23:13:56 UTC (rev 206001)
@@ -62,6 +62,9 @@
virtual FloatSize presentationSize() const = 0;
virtual void offsetTimestampsBy(const MediaTime&) = 0;
virtual void setTimestamps(const MediaTime&, const MediaTime&) = 0;
+ virtual bool isDivisable() const = 0;
+ enum DivideFlags { BeforePresentationTime, AfterPresentationTime };
+ virtual std::pair<RefPtr<MediaSample>, RefPtr<MediaSample>> divide(const MediaTime& presentationTime) = 0;
enum SampleFlags {
None = 0,
Modified: trunk/Source/WebCore/platform/cf/CoreMediaSoftLink.cpp (206000 => 206001)
--- trunk/Source/WebCore/platform/cf/CoreMediaSoftLink.cpp 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/platform/cf/CoreMediaSoftLink.cpp 2016-09-15 23:13:56 UTC (rev 206001)
@@ -109,6 +109,9 @@
SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMAudioFormatDescriptionGetStreamBasicDescription, const AudioStreamBasicDescription *, (CMAudioFormatDescriptionRef desc), (desc))
SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer, OSStatus, (CMSampleBufferRef sbuf, size_t *bufferListSizeNeededOut, AudioBufferList *bufferListOut, size_t bufferListSize, CFAllocatorRef bbufStructAllocator, CFAllocatorRef bbufMemoryAllocator, uint32_t flags, CMBlockBufferRef *blockBufferOut), (sbuf, bufferListSizeNeededOut, bufferListOut, bufferListSize, bbufStructAllocator, bbufMemoryAllocator, flags, blockBufferOut))
SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMSampleBufferGetNumSamples, CMItemCount, (CMSampleBufferRef sbuf), (sbuf))
+SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMSampleBufferCallBlockForEachSample, OSStatus, (CMSampleBufferRef sbuf, OSStatus (^handler)(CMSampleBufferRef, CMItemCount)), (sbuf, handler))
+SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMSampleBufferCopySampleBufferForRange, OSStatus, (CFAllocatorRef allocator, CMSampleBufferRef sbuf, CFRange sampleRange, CMSampleBufferRef* sBufOut), (allocator, sbuf, sampleRange, sBufOut))
+SOFT_LINK_FUNCTION_FOR_SOURCE(WebCore, CoreMedia, CMSampleBufferGetSampleSizeArray, OSStatus, (CMSampleBufferRef sbuf, CMItemCount sizeArrayEntries, size_t* sizeArrayOut, CMItemCount* sizeArrayEntriesNeededOut), (sbuf, sizeArrayEntries, sizeArrayOut, sizeArrayEntriesNeededOut))
#endif // PLATFORM(COCOA)
#if PLATFORM(IOS)
Modified: trunk/Source/WebCore/platform/cf/CoreMediaSoftLink.h (206000 => 206001)
--- trunk/Source/WebCore/platform/cf/CoreMediaSoftLink.h 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/platform/cf/CoreMediaSoftLink.h 2016-09-15 23:13:56 UTC (rev 206001)
@@ -184,6 +184,12 @@
#define CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer softLink_CoreMedia_CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer
SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMSampleBufferGetNumSamples, CMItemCount, (CMSampleBufferRef sbuf), (sbuf))
#define CMSampleBufferGetNumSamples softLink_CoreMedia_CMSampleBufferGetNumSamples
+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMSampleBufferCallBlockForEachSample, OSStatus, (CMSampleBufferRef sbuf, OSStatus (^handler)(CMSampleBufferRef, CMItemCount)), (sbuf, handler))
+#define CMSampleBufferCallBlockForEachSample softLink_CoreMedia_CMSampleBufferCallBlockForEachSample
+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMSampleBufferCopySampleBufferForRange, OSStatus, (CFAllocatorRef allocator, CMSampleBufferRef sbuf, CFRange sampleRange, CMSampleBufferRef* sBufOut), (allocator, sbuf, sampleRange, sBufOut))
+#define CMSampleBufferCopySampleBufferForRange softLink_CoreMedia_CMSampleBufferCopySampleBufferForRange
+SOFT_LINK_FUNCTION_FOR_HEADER(WebCore, CoreMedia, CMSampleBufferGetSampleSizeArray, OSStatus, (CMSampleBufferRef sbuf, CMItemCount sizeArrayEntries, size_t* sizeArrayOut, CMItemCount* sizeArrayEntriesNeededOut), (sbuf, sizeArrayEntries, sizeArrayOut, sizeArrayEntriesNeededOut))
+#define CMSampleBufferGetSampleSizeArray softLink_CoreMedia_CMSampleBufferGetSampleSizeArray
#endif // PLATFORM(COCOA)
Modified: trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.cpp (206000 => 206001)
--- trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.cpp 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.cpp 2016-09-15 23:13:56 UTC (rev 206001)
@@ -193,6 +193,9 @@
MediaTime closestDelta = MediaTime::positiveInfiniteTime();
MediaTime closestTime = MediaTime::zeroTime();
unsigned count = length();
+ if (!count)
+ return MediaTime::invalidTime();
+
bool ignoreInvalid;
for (unsigned ndx = 0; ndx < count; ndx++) {
Modified: trunk/Source/WebCore/platform/graphics/avfoundation/MediaSampleAVFObjC.h (206000 => 206001)
--- trunk/Source/WebCore/platform/graphics/avfoundation/MediaSampleAVFObjC.h 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/MediaSampleAVFObjC.h 2016-09-15 23:13:56 UTC (rev 206001)
@@ -33,6 +33,7 @@
class MediaSampleAVFObjC final : public MediaSample {
public:
static Ref<MediaSampleAVFObjC> create(CMSampleBufferRef sample, int trackID) { return adoptRef(*new MediaSampleAVFObjC(sample, trackID)); }
+ static Ref<MediaSampleAVFObjC> create(CMSampleBufferRef sample, AtomicString trackID) { return adoptRef(*new MediaSampleAVFObjC(sample, trackID)); }
static Ref<MediaSampleAVFObjC> create(CMSampleBufferRef sample) { return adoptRef(*new MediaSampleAVFObjC(sample)); }
private:
@@ -40,6 +41,11 @@
: m_sample(sample)
{
}
+ MediaSampleAVFObjC(CMSampleBufferRef sample, AtomicString trackID)
+ : m_sample(sample)
+ , m_id(trackID)
+ {
+ }
MediaSampleAVFObjC(CMSampleBufferRef sample, int trackID)
: m_sample(sample)
, m_id(String::format("%d", trackID))
@@ -62,6 +68,8 @@
void dump(PrintStream&) const override;
void offsetTimestampsBy(const MediaTime&) override;
void setTimestamps(const MediaTime&, const MediaTime&) override;
+ bool isDivisable() const override;
+ std::pair<RefPtr<MediaSample>, RefPtr<MediaSample>> divide(const MediaTime& presentationTime) override;
RetainPtr<CMSampleBufferRef> m_sample;
AtomicString m_id;
Modified: trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm (206000 => 206001)
--- trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm 2016-09-15 23:13:56 UTC (rev 206001)
@@ -142,4 +142,51 @@
m_sample = adoptCF(newSample);
}
+bool MediaSampleAVFObjC::isDivisable() const
+{
+ if (CMSampleBufferGetNumSamples(m_sample.get()) == 1)
+ return false;
+
+ if (CMSampleBufferGetSampleSizeArray(m_sample.get(), 0, nullptr, nullptr) == kCMSampleBufferError_BufferHasNoSampleSizes)
+ return false;
+
+ return true;
}
+
+std::pair<RefPtr<MediaSample>, RefPtr<MediaSample>> MediaSampleAVFObjC::divide(const MediaTime& presentationTime)
+{
+ if (!isDivisable())
+ return { nullptr, nullptr };
+
+ CFIndex samplesBeforePresentationTime = 0;
+
+ CMSampleBufferCallBlockForEachSample(m_sample.get(), [&] (CMSampleBufferRef sampleBuffer, CMItemCount) -> OSStatus {
+ if (toMediaTime(CMSampleBufferGetPresentationTimeStamp(sampleBuffer)) >= presentationTime)
+ return 1;
+ ++samplesBeforePresentationTime;
+ return noErr;
+ });
+
+ if (!samplesBeforePresentationTime)
+ return { nullptr, this };
+
+ CMItemCount sampleCount = CMSampleBufferGetNumSamples(m_sample.get());
+ if (samplesBeforePresentationTime >= sampleCount)
+ return { this, nullptr };
+
+ CMSampleBufferRef rawSampleBefore = nullptr;
+ CFRange rangeBefore = CFRangeMake(0, samplesBeforePresentationTime);
+ if (CMSampleBufferCopySampleBufferForRange(kCFAllocatorDefault, m_sample.get(), rangeBefore, &rawSampleBefore) != noErr)
+ return { nullptr, nullptr };
+ RetainPtr<CMSampleBufferRef> sampleBefore = adoptCF(rawSampleBefore);
+
+ CMSampleBufferRef rawSampleAfter = nullptr;
+ CFRange rangeAfter = CFRangeMake(samplesBeforePresentationTime, sampleCount - samplesBeforePresentationTime);
+ if (CMSampleBufferCopySampleBufferForRange(kCFAllocatorDefault, m_sample.get(), rangeAfter, &rawSampleAfter) != noErr)
+ return { nullptr, nullptr };
+ RetainPtr<CMSampleBufferRef> sampleAfter = adoptCF(rawSampleAfter);
+
+ return { MediaSampleAVFObjC::create(sampleBefore.get(), m_id), MediaSampleAVFObjC::create(sampleAfter.get(), m_id) };
+}
+
+}
Modified: trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp (206000 => 206001)
--- trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp 2016-09-15 22:38:17 UTC (rev 206000)
+++ trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp 2016-09-15 23:13:56 UTC (rev 206001)
@@ -66,7 +66,10 @@
void dump(PrintStream&) const override;
void offsetTimestampsBy(const MediaTime& offset) override { m_box.offsetTimestampsBy(offset); }
void setTimestamps(const MediaTime& presentationTimestamp, const MediaTime& decodeTimestamp) override { m_box.setTimestamps(presentationTimestamp, decodeTimestamp); }
+ bool isDivisable() const override { return false; }
+ std::pair<RefPtr<MediaSample>, RefPtr<MediaSample>> divide(const MediaTime&) override { return {nullptr, nullptr}; }
+
unsigned generation() const { return m_box.generation(); }
MockSampleBox m_box;
