Diff
Modified: trunk/Source/WebCore/ChangeLog (172656 => 172657)
--- trunk/Source/WebCore/ChangeLog 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/ChangeLog 2014-08-15 23:09:19 UTC (rev 172657)
@@ -1,3 +1,89 @@
+2014-08-15 Eric Carlson <[email protected]>
+
+ [MSE] Implement a maximum buffer size for SourceBuffer
+ https://bugs.webkit.org/show_bug.cgi?id=135614
+
+ Reviewed by Jer Noble.
+
+ Implement the MSE coded frame eviction algorithm: when new buffers are appended attempt
+ to keep the amount of data buffered below a maximum size (which is determined by the media
+ session manager) by freeing coded frames as follows:
+ 1 - Free frames in 30 second chunks from 0 to 30 seconds before the current time.
+ 2 - If there are time ranges after the range with the current time, free frames in
+ 30 second chunks from duration back to the beginning of the time range after
+ current time.
+
+ For now we DO NOT throw a QUOTA_EXCEEDED_ERR when we are not able to free up enough
+ space to remain below the prescribed quota, because some big name, widely deployed,
+ code bases ignore the error and continue appending data as though the failed append
+ succeeded, leading to a corrupted bitstream and failure to play.
+
+ * Modules/mediasource/SampleMap.cpp:
+ (WebCore::SampleMap::addSample): Drive-by performance optimization: sample->presentationTime()
+ is used more than once, stash it in a local variable.
+ (WebCore::SampleMap::removeSample): Ditto.
+
+ * Modules/mediasource/SourceBuffer.cpp:
+ (WebCore::logRanges): Debug-only function to log TimeRanges.
+ (WebCore::SourceBuffer::SourceBuffer):
+ (WebCore::SourceBuffer::remove): Optimize logging. Buffer full and coded frame eviction logic
+ is in SourceBuffer instead of SourceBufferPrivate.
+ (WebCore::SourceBuffer::appendBufferInternal): Ditto.
+ (WebCore::SourceBuffer::sourceBufferPrivateAppendComplete): Set m_bufferFull when more data
+ has been buffered than allowed.
+ (WebCore::removeSamplesFromTrackBuffer): Add logging.
+ (WebCore::SourceBuffer::removeCodedFrames): Improve logging. Avoid debug build assert when
+ current time is after last enqueued presentation time.
+ (WebCore::SourceBuffer::evictCodedFrames): The coded frame eviction algorithm.
+ (WebCore::SourceBuffer::maximumBufferSize): Return the maximum amount of data that can
+ be buffered.
+ (WebCore::SourceBuffer::sourceBufferPrivateDidReceiveSample): Improve logging. Don't attempt
+ to create a PlatformTimeRanges with inverted start and end to avoid an assert in debug.
+ (WebCore::SourceBuffer::hasAudio): New, return true if there are any audio tracks.
+ (WebCore::SourceBuffer::hasVideo): New, return true if there are any video tracks.
+ (WebCore::SourceBuffer::sourceBufferPrivateHasAudio): Call hasAudio.
+ (WebCore::SourceBuffer::sourceBufferPrivateHasVideo): Call hasVideo.
+ (WebCore::SourceBuffer::hasCurrentTime): Return true if currentTime is greater than duration.
+ (WebCore::SourceBuffer::hasFutureTime): Ditto.
+ (WebCore::SourceBuffer::extraMemoryCost): Return the amount of data buffered: the size of
+ the input buffer plus the size of all track samples.
+ (WebCore::SourceBuffer::reportExtraMemoryCost): Move buffered size calculation to extraMemoryCost.
+ (WebCore::SourceBuffer::document): Document accessor.
+ * Modules/mediasource/SourceBuffer.h: Drive-by size optimization by moving all bool member
+ variables to the end of class.
+
+ * html/HTMLMediaElement.cpp:
+ (WebCore::HTMLMediaElement::refreshCachedTime): Drive-by removal of overly chatty logging.
+ (WebCore::HTMLMediaElement::maximumSourceBufferSize): New, maximum source buffer size.
+ * html/HTMLMediaElement.h:
+
+ * html/HTMLMediaSession.cpp:
+ (WebCore::HTMLMediaSession::maximumMediaSourceBufferSize): New, get maximum source buffer
+ size from settings, return full amount for an SourceBuffer with video and audio, return 5%
+ of the maximum for an audio-only SourceBuffer.
+ * html/HTMLMediaSession.h:
+
+ * page/Settings.in: Add maximumSourceBufferSize. Default value is enough for approximately
+ five minutes of 1080p video and stereo audio.
+
+ * platform/graphics/PlatformTimeRanges.cpp:
+ (WebCore::PlatformTimeRanges::totalDuration): Drive-by optimization.
+ (WebCore::PlatformTimeRanges::dump): New, allow a PlatformTimeRanges to be printed.
+ * platform/graphics/PlatformTimeRanges.h:
+
+ * platform/graphics/SourceBufferPrivate.h: Delete evictCodedFrames and isFull, that logic
+ is not in SourceBuffer.
+ * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
+ * platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
+ (WebCore::SourceBufferPrivateAVFObjC::processCodedFrame): Drive-by logging fix.
+ (WebCore::SourceBufferPrivateAVFObjC::evictCodedFrames): Deleted.
+ (WebCore::SourceBufferPrivateAVFObjC::isFull): Deleted.
+
+ * platform/mock/mediasource/MockSourceBufferPrivate.cpp:
+ (WebCore::MockSourceBufferPrivate::evictCodedFrames): Deleted.
+ (WebCore::MockSourceBufferPrivate::isFull): Deleted.
+ * platform/mock/mediasource/MockSourceBufferPrivate.h:
+
2014-08-15 Zalan Bujtas <[email protected]>
REGRESSION: Parts of the route/route options windows are invisible at maps.google.com
Modified: trunk/Source/WebCore/Modules/mediasource/SampleMap.cpp (172656 => 172657)
--- trunk/Source/WebCore/Modules/mediasource/SampleMap.cpp 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/Modules/mediasource/SampleMap.cpp 2014-08-15 23:09:19 UTC (rev 172657)
@@ -113,9 +113,11 @@
RefPtr<MediaSample> sample = prpSample;
ASSERT(sample);
- presentationOrder().m_samples.insert(PresentationOrderSampleMap::MapType::value_type(sample->presentationTime(), sample));
+ MediaTime presentationTime = sample->presentationTime();
- auto decodeKey = DecodeOrderSampleMap::KeyType(sample->decodeTime(), sample->presentationTime());
+ presentationOrder().m_samples.insert(PresentationOrderSampleMap::MapType::value_type(presentationTime, sample));
+
+ auto decodeKey = DecodeOrderSampleMap::KeyType(sample->decodeTime(), presentationTime);
decodeOrder().m_samples.insert(DecodeOrderSampleMap::MapType::value_type(decodeKey, sample));
m_totalSize += sample->sizeInBytes();
@@ -124,9 +126,11 @@
void SampleMap::removeSample(MediaSample* sample)
{
ASSERT(sample);
- presentationOrder().m_samples.erase(sample->presentationTime());
+ MediaTime presentationTime = sample->presentationTime();
- auto decodeKey = DecodeOrderSampleMap::KeyType(sample->decodeTime(), sample->presentationTime());
+ presentationOrder().m_samples.erase(presentationTime);
+
+ auto decodeKey = DecodeOrderSampleMap::KeyType(sample->decodeTime(), presentationTime);
decodeOrder().m_samples.erase(decodeKey);
m_totalSize -= sample->sizeInBytes();
Modified: trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp (172656 => 172657)
--- trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/Modules/mediasource/SourceBuffer.cpp 2014-08-15 23:09:19 UTC (rev 172657)
@@ -1,5 +1,6 @@
/*
* Copyright (C) 2013 Google Inc. All rights reserved.
+ * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
@@ -48,12 +49,16 @@
#include "TextTrackList.h"
#include "TimeRanges.h"
#include "VideoTrackList.h"
+#include <limits>
#include <map>
#include <runtime/JSCInlines.h>
#include <runtime/JSLock.h>
#include <runtime/VM.h>
#include <wtf/CurrentTime.h>
#include <wtf/NeverDestroyed.h>
+#if !LOG_DISABLED
+#include <wtf/text/StringBuilder.h>
+#endif
namespace WebCore {
@@ -104,12 +109,9 @@
, m_private(WTF::move(sourceBufferPrivate))
, m_source(source)
, m_asyncEventQueue(*this)
- , m_updating(false)
, m_appendBufferTimer(this, &SourceBuffer::appendBufferTimerFired)
, m_highestPresentationEndTimestamp(MediaTime::invalidTime())
- , m_receivedFirstInitializationSegment(false)
, m_buffered(TimeRanges::create())
- , m_active(false)
, m_appendState(WaitingForSegment)
, m_timeOfBufferingMonitor(monotonicallyIncreasingTime())
, m_bufferedSinceLastMonitor(0)
@@ -118,6 +120,10 @@
, m_pendingRemoveStart(MediaTime::invalidTime())
, m_pendingRemoveEnd(MediaTime::invalidTime())
, m_removeTimer(this, &SourceBuffer::removeTimerFired)
+ , m_updating(false)
+ , m_receivedFirstInitializationSegment(false)
+ , m_active(false)
+ , m_bufferFull(false)
{
ASSERT(m_source);
@@ -235,7 +241,8 @@
void SourceBuffer::remove(double start, double end, ExceptionCode& ec)
{
- LOG(MediaSource, "SourceBuffer::remove(%p) - start(%s), end(%s)", this, toString(start).utf8().data(), toString(end).utf8().data());
+ LOG(MediaSource, "SourceBuffer::remove(%p) - start(%lf), end(%lf)", this, start, end);
+
// Section 3.2 remove() method steps.
// 1. If start is negative or greater than duration, then throw an InvalidAccessError exception and abort these steps.
// 2. If end is less than or equal to start, then throw an InvalidAccessError exception and abort these steps.
@@ -407,13 +414,17 @@
m_source->openIfInEndedState();
// 4. Run the coded frame eviction algorithm.
- m_private->evictCodedFrames();
+ evictCodedFrames(size);
+ // FIXME: enable this code when MSE libraries have been updated to support it.
+#if 0
// 5. If the buffer full flag equals true, then throw a QUOTA_EXCEEDED_ERR exception and abort these step.
- if (m_private->isFull()) {
+ if (m_bufferFull) {
+ LOG(MediaSource, "SourceBuffer::appendBufferInternal(%p) - buffer full, failing with QUOTA_EXCEEDED_ERR error", this);
ec = QUOTA_EXCEEDED_ERR;
return;
}
+#endif
// NOTE: Return to 3.2 appendBuffer()
// 3. Add data to the end of the input buffer.
@@ -514,6 +525,10 @@
}
reportExtraMemoryCost();
+ if (extraMemoryCost() > this->maximumBufferSize())
+ m_bufferFull = true;
+
+ LOG(Media, "SourceBuffer::sourceBufferPrivateAppendComplete(%p) - buffered = %s", this, toString(m_buffered->ranges()).utf8().data());
}
void SourceBuffer::sourceBufferPrivateDidReceiveRenderingError(SourceBufferPrivate*, int)
@@ -527,13 +542,27 @@
return a.second->decodeTime() < b.second->decodeTime();
}
-static PassRefPtr<TimeRanges> removeSamplesFromTrackBuffer(const DecodeOrderSampleMap::MapType& samples, SourceBuffer::TrackBuffer& trackBuffer)
+static PassRefPtr<TimeRanges> removeSamplesFromTrackBuffer(const DecodeOrderSampleMap::MapType& samples, SourceBuffer::TrackBuffer& trackBuffer, const SourceBuffer* buffer, const char* logPrefix)
{
+#if !LOG_DISABLED
+ double earliestSample = std::numeric_limits<double>::infinity();
+ double latestSample = 0;
+ size_t bytesRemoved = 0;
+#else
+ UNUSED_PARAM(logPrefix);
+ UNUSED_PARAM(buffer);
+#endif
+
RefPtr<TimeRanges> erasedRanges = TimeRanges::create();
MediaTime microsecond(1, 1000000);
for (auto sampleIt : samples) {
const DecodeOrderSampleMap::KeyType& decodeKey = sampleIt.first;
+#if !LOG_DISABLED
+ size_t startBufferSize = trackBuffer.samples.sizeInBytes();
+#endif
+
RefPtr<MediaSample>& sample = sampleIt.second;
+ LOG(MediaSource, "SourceBuffer::%s(%p) - removing sample(%s)", logPrefix, buffer, toString(*sampleIt.second).utf8().data());
// Remove the erased samples from the TrackBuffer sample map.
trackBuffer.samples.removeSample(sample.get());
@@ -544,7 +573,21 @@
double startTime = sample->presentationTime().toDouble();
double endTime = startTime + (sample->duration() + microsecond).toDouble();
erasedRanges->add(startTime, endTime);
+
+#if !LOG_DISABLED
+ bytesRemoved += startBufferSize - trackBuffer.samples.sizeInBytes();
+ if (startTime < earliestSample)
+ earliestSample = startTime;
+ if (endTime > latestSample)
+ latestSample = endTime;
+#endif
}
+
+#if !LOG_DISABLED
+ if (bytesRemoved)
+ LOG(MediaSource, "SourceBuffer::%s(%p) removed %zu bytes, start(%lf), end(%lf)", logPrefix, buffer, bytesRemoved, earliestSample, latestSample);
+#endif
+
return erasedRanges.release();
}
@@ -590,14 +633,16 @@
DecodeOrderSampleMap::iterator removeDecodeStart = trackBuffer.samples.decodeOrder().findSampleWithDecodeKey(decodeKey);
DecodeOrderSampleMap::MapType erasedSamples(removeDecodeStart, removeDecodeEnd);
- RefPtr<TimeRanges> erasedRanges = removeSamplesFromTrackBuffer(erasedSamples, trackBuffer);
+ RefPtr<TimeRanges> erasedRanges = removeSamplesFromTrackBuffer(erasedSamples, trackBuffer, this, "removeCodedFrames");
// Only force the TrackBuffer to re-enqueue if the removed ranges overlap with enqueued and possibly
// not yet displayed samples.
- PlatformTimeRanges possiblyEnqueuedRanges(currentMediaTime, trackBuffer.lastEnqueuedPresentationTime);
- possiblyEnqueuedRanges.intersectWith(erasedRanges->ranges());
- if (possiblyEnqueuedRanges.length())
- trackBuffer.needsReenqueueing = true;
+ if (currentMediaTime < trackBuffer.lastEnqueuedPresentationTime) {
+ PlatformTimeRanges possiblyEnqueuedRanges(currentMediaTime, trackBuffer.lastEnqueuedPresentationTime);
+ possiblyEnqueuedRanges.intersectWith(erasedRanges->ranges());
+ if (possiblyEnqueuedRanges.length())
+ trackBuffer.needsReenqueueing = true;
+ }
erasedRanges->invert();
m_buffered->intersectWith(*erasedRanges);
@@ -611,6 +656,8 @@
// 4. If buffer full flag equals true and this object is ready to accept more bytes, then set the buffer full flag to false.
// No-op
+
+ LOG(Media, "SourceBuffer::removeCodedFrames(%p) - buffered = %s", this, toString(m_buffered->ranges()).utf8().data());
}
void SourceBuffer::removeTimerFired(Timer<SourceBuffer>*)
@@ -637,6 +684,109 @@
scheduleEvent(eventNames().updateendEvent);
}
+void SourceBuffer::evictCodedFrames(size_t newDataSize)
+{
+ // 3.5.13 Coded Frame Eviction Algorithm
+ // http://www.w3.org/TR/media-source/#sourcebuffer-coded-frame-eviction
+
+ if (isRemoved())
+ return;
+
+ // This algorithm is run to free up space in this source buffer when new data is appended.
+ // 1. Let new data equal the data that is about to be appended to this SourceBuffer.
+ // 2. If the buffer full flag equals false, then abort these steps.
+ if (!m_bufferFull)
+ return;
+
+ size_t maximumBufferSize = this->maximumBufferSize();
+
+ // 3. Let removal ranges equal a list of presentation time ranges that can be evicted from
+ // the presentation to make room for the new data.
+
+ // NOTE: begin by removing data from the beginning of the buffered ranges, 30 seconds at
+ // a time, up to 30 seconds before currentTime.
+ MediaTime thirtySeconds = MediaTime(30, 1);
+ MediaTime currentTime = MediaTime::createWithDouble(m_source->currentTime());
+ MediaTime maximumRangeEnd = currentTime - thirtySeconds;
+
+#if !LOG_DISABLED
+ LOG(MediaSource, "SourceBuffer::evictCodedFrames(%p) - currentTime = %lf, require %zu bytes, maximum buffer size is %zu", this, m_source->currentTime(), extraMemoryCost() + newDataSize, maximumBufferSize);
+ size_t initialBufferedSize = extraMemoryCost();
+#endif
+
+ MediaTime rangeStart = MediaTime::zeroTime();
+ MediaTime rangeEnd = rangeStart + thirtySeconds;
+ while (rangeStart < maximumRangeEnd) {
+ // 4. For each range in removal ranges, run the coded frame removal algorithm with start and
+ // end equal to the removal range start and end timestamp respectively.
+ removeCodedFrames(rangeStart, std::min(rangeEnd, maximumRangeEnd));
+ if (extraMemoryCost() + newDataSize < maximumBufferSize) {
+ m_bufferFull = false;
+ break;
+ }
+
+ rangeStart += thirtySeconds;
+ rangeEnd += thirtySeconds;
+ }
+
+ if (!m_bufferFull) {
+ LOG(MediaSource, "SourceBuffer::evictCodedFrames(%p) - evicted %zu bytes", this, initialBufferedSize - extraMemoryCost());
+ return;
+ }
+
+ // If there still isn't enough free space and there buffers in time ranges after the current range (ie. there is a gap after
+ // the current buffered range), delete 30 seconds at a time from duration back to the current time range or 30 seconds after
+ // currenTime whichever we hit first.
+ auto buffered = m_buffered->ranges();
+ size_t currentTimeRange = buffered.find(currentTime);
+ if (currentTimeRange == notFound || currentTimeRange == buffered.length() - 1) {
+ LOG(MediaSource, "SourceBuffer::evictCodedFrames(%p) - evicted %zu bytes but FAILED to free enough", this, initialBufferedSize - extraMemoryCost());
+ return;
+ }
+
+ MediaTime minimumRangeStart = currentTime + thirtySeconds;
+
+ rangeEnd = MediaTime::createWithDouble(m_source->duration());
+ rangeStart = rangeEnd - thirtySeconds;
+ while (rangeStart > minimumRangeStart) {
+
+ // Do not evict data from the time range that contains currentTime.
+ size_t startTimeRange = buffered.find(rangeStart);
+ if (startTimeRange == currentTimeRange) {
+ size_t endTimeRange = buffered.find(rangeEnd);
+ if (endTimeRange == currentTimeRange)
+ break;
+
+ rangeEnd = buffered.start(endTimeRange);
+ }
+
+ // 4. For each range in removal ranges, run the coded frame removal algorithm with start and
+ // end equal to the removal range start and end timestamp respectively.
+ removeCodedFrames(std::max(minimumRangeStart, rangeStart), rangeEnd);
+ if (extraMemoryCost() + newDataSize < maximumBufferSize) {
+ m_bufferFull = false;
+ break;
+ }
+
+ rangeStart -= thirtySeconds;
+ rangeEnd -= thirtySeconds;
+ }
+
+ LOG(MediaSource, "SourceBuffer::evictCodedFrames(%p) - evicted %zu bytes%s", this, initialBufferedSize - extraMemoryCost(), m_bufferFull ? "" : " but FAILED to free enough");
+}
+
+size_t SourceBuffer::maximumBufferSize() const
+{
+ if (isRemoved())
+ return 0;
+
+ HTMLMediaElement* element = m_source->mediaElement();
+ if (!element)
+ return 0;
+
+ return element->maximumSourceBufferSize(*this);
+}
+
const AtomicString& SourceBuffer::decodeError()
{
static NeverDestroyed<AtomicString> decode("decode", AtomicString::ConstructFromLiteral);
@@ -1198,19 +1348,17 @@
auto nextSyncIter = trackBuffer.samples.decodeOrder().findSyncSampleAfterDecodeIterator(lastDecodeIter);
dependentSamples.insert(firstDecodeIter, nextSyncIter);
- RefPtr<TimeRanges> erasedRanges = removeSamplesFromTrackBuffer(dependentSamples, trackBuffer);
-#if !LOG_DISABLED
- for (auto& samplePair : dependentSamples)
- LOG(MediaSource, "SourceBuffer::sourceBufferPrivateDidReceiveSample(%p) - removing sample(%s)", this, toString(*samplePair.second).utf8().data());
-#endif
+ RefPtr<TimeRanges> erasedRanges = removeSamplesFromTrackBuffer(dependentSamples, trackBuffer, this, "sourceBufferPrivateDidReceiveSample");
// Only force the TrackBuffer to re-enqueue if the removed ranges overlap with enqueued and possibly
// not yet displayed samples.
MediaTime currentMediaTime = MediaTime::createWithDouble(m_source->currentTime());
- PlatformTimeRanges possiblyEnqueuedRanges(currentMediaTime, trackBuffer.lastEnqueuedPresentationTime);
- possiblyEnqueuedRanges.intersectWith(erasedRanges->ranges());
- if (possiblyEnqueuedRanges.length())
- trackBuffer.needsReenqueueing = true;
+ if (currentMediaTime < trackBuffer.lastEnqueuedPresentationTime) {
+ PlatformTimeRanges possiblyEnqueuedRanges(currentMediaTime, trackBuffer.lastEnqueuedPresentationTime);
+ possiblyEnqueuedRanges.intersectWith(erasedRanges->ranges());
+ if (possiblyEnqueuedRanges.length())
+ trackBuffer.needsReenqueueing = true;
+ }
erasedRanges->invert();
m_buffered->intersectWith(*erasedRanges.get());
@@ -1262,16 +1410,26 @@
m_source->setDurationInternal(highestPresentationEndTimestamp().toDouble());
}
-bool SourceBuffer::sourceBufferPrivateHasAudio(const SourceBufferPrivate*) const
+bool SourceBuffer::hasAudio() const
{
return m_audioTracks && m_audioTracks->length();
}
-bool SourceBuffer::sourceBufferPrivateHasVideo(const SourceBufferPrivate*) const
+bool SourceBuffer::hasVideo() const
{
return m_videoTracks && m_videoTracks->length();
}
+bool SourceBuffer::sourceBufferPrivateHasAudio(const SourceBufferPrivate*) const
+{
+ return hasAudio();
+}
+
+bool SourceBuffer::sourceBufferPrivateHasVideo(const SourceBufferPrivate*) const
+{
+ return hasVideo();
+}
+
void SourceBuffer::videoTrackSelectedChanged(VideoTrack* track)
{
// 2.4.5 Changes to selected/enabled track state
@@ -1515,6 +1673,10 @@
return false;
MediaTime currentTime = MediaTime::createWithDouble(m_source->currentTime());
+ MediaTime duration = MediaTime::createWithDouble(m_source->duration());
+ if (currentTime >= duration)
+ return true;
+
std::unique_ptr<PlatformTimeRanges> ranges = bufferedAccountingForEndOfStream();
return abs(ranges->nearest(currentTime) - currentTime) <= currentTimeFudgeFactor();
}
@@ -1529,6 +1691,10 @@
return false;
MediaTime currentTime = MediaTime::createWithDouble(m_source->currentTime());
+ MediaTime duration = MediaTime::createWithDouble(m_source->duration());
+ if (currentTime >= duration)
+ return true;
+
MediaTime nearest = ranges->nearest(currentTime);
if (abs(nearest - currentTime) > currentTimeFudgeFactor())
return false;
@@ -1538,7 +1704,6 @@
return false;
MediaTime localEnd = ranges->end(found);
- MediaTime duration = MediaTime::createWithDouble(m_source->duration());
if (localEnd == duration)
return true;
@@ -1572,12 +1737,18 @@
return unbufferedTime.toDouble() / m_averageBufferRate < timeRemaining.toDouble();
}
-void SourceBuffer::reportExtraMemoryCost()
+size_t SourceBuffer::extraMemoryCost() const
{
size_t extraMemoryCost = m_pendingAppendData.capacity();
for (auto& trackBuffer : m_trackBufferMap.values())
extraMemoryCost += trackBuffer.samples.sizeInBytes();
+ return extraMemoryCost;
+}
+
+void SourceBuffer::reportExtraMemoryCost()
+{
+ size_t extraMemoryCost = this->extraMemoryCost();
if (extraMemoryCost < m_reportedExtraMemoryCost)
return;
@@ -1603,6 +1774,12 @@
return sampleDescriptions;
}
+Document& SourceBuffer::document() const
+{
+ ASSERT(scriptExecutionContext()->isDocument());
+ return *static_cast<Document*>(scriptExecutionContext());
+}
+
} // namespace WebCore
#endif
Modified: trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h (172656 => 172657)
--- trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/Modules/mediasource/SourceBuffer.h 2014-08-15 23:09:19 UTC (rev 172657)
@@ -1,5 +1,6 @@
/*
* Copyright (C) 2013 Google Inc. All rights reserved.
+ * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
@@ -90,6 +91,9 @@
bool hasFutureTime() const;
bool canPlayThrough();
+ bool hasVideo() const;
+ bool hasAudio() const;
+
bool active() const { return m_active; }
// ActiveDOMObject interface
@@ -105,6 +109,8 @@
struct TrackBuffer;
+ Document& document() const;
+
protected:
// EventTarget interface
virtual void refEventTarget() override { ref(); }
@@ -154,12 +160,15 @@
void reenqueueMediaForTime(TrackBuffer&, AtomicString trackID, const MediaTime&);
void provideMediaData(TrackBuffer&, AtomicString trackID);
void didDropSample();
+ void evictCodedFrames(size_t newDataSize);
+ size_t maximumBufferSize() const;
void monitorBufferingRate();
void removeTimerFired(Timer<SourceBuffer>*);
void removeCodedFrames(const MediaTime& start, const MediaTime& end);
+ size_t extraMemoryCost() const;
void reportExtraMemoryCost();
std::unique_ptr<PlatformTimeRanges> bufferedAccountingForEndOfStream() const;
@@ -172,8 +181,6 @@
MediaSource* m_source;
GenericEventQueue m_asyncEventQueue;
- bool m_updating;
-
Vector<unsigned char> m_pendingAppendData;
Timer<SourceBuffer> m_appendBufferTimer;
@@ -189,9 +196,7 @@
MediaTime m_highestPresentationEndTimestamp;
HashMap<AtomicString, TrackBuffer> m_trackBufferMap;
- bool m_receivedFirstInitializationSegment;
RefPtr<TimeRanges> m_buffered;
- bool m_active;
enum AppendStateType { WaitingForSegment, ParsingInitSegment, ParsingMediaSegment };
AppendStateType m_appendState;
@@ -205,6 +210,11 @@
MediaTime m_pendingRemoveStart;
MediaTime m_pendingRemoveEnd;
Timer<SourceBuffer> m_removeTimer;
+
+ bool m_updating;
+ bool m_receivedFirstInitializationSegment;
+ bool m_active;
+ bool m_bufferFull;
};
} // namespace WebCore
Modified: trunk/Source/WebCore/html/HTMLMediaElement.cpp (172656 => 172657)
--- trunk/Source/WebCore/html/HTMLMediaElement.cpp 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/html/HTMLMediaElement.cpp 2014-08-15 23:09:19 UTC (rev 172657)
@@ -2518,7 +2518,6 @@
return;
}
- LOG(Media, "HTMLMediaElement::refreshCachedTime - caching time %f", m_cachedTime);
m_clockTimeAtLastCachedTimeUpdate = monotonicallyIncreasingTime();
}
@@ -5972,6 +5971,13 @@
return MediaSession::Audio;
}
+#if ENABLE(MEDIA_SOURCE)
+size_t HTMLMediaElement::maximumSourceBufferSize(const SourceBuffer& buffer) const
+{
+ return m_mediaSession->maximumMediaSourceBufferSize(buffer);
+}
+#endif
+
void HTMLMediaElement::pausePlayback()
{
LOG(Media, "HTMLMediaElement::pausePlayback - paused = %s", boolString(paused()));
Modified: trunk/Source/WebCore/html/HTMLMediaElement.h (172656 => 172657)
--- trunk/Source/WebCore/html/HTMLMediaElement.h 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/html/HTMLMediaElement.h 2014-08-15 23:09:19 UTC (rev 172657)
@@ -72,6 +72,7 @@
#endif
#if ENABLE(MEDIA_SOURCE)
class MediaSource;
+class SourceBuffer;
class VideoPlaybackQuality;
#endif
@@ -208,6 +209,7 @@
// Media Source.
void closeMediaSource();
void incrementDroppedFrameCount() { ++m_droppedVideoFrames; }
+ size_t maximumSourceBufferSize(const SourceBuffer&) const;
#endif
#if ENABLE(ENCRYPTED_MEDIA)
Modified: trunk/Source/WebCore/html/HTMLMediaSession.cpp (172656 => 172657)
--- trunk/Source/WebCore/html/HTMLMediaSession.cpp 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/html/HTMLMediaSession.cpp 2014-08-15 23:09:19 UTC (rev 172657)
@@ -38,6 +38,7 @@
#include "MediaSessionManager.h"
#include "Page.h"
#include "ScriptController.h"
+#include "SourceBuffer.h"
#if PLATFORM(IOS)
#include "AudioSession.h"
@@ -301,6 +302,38 @@
}
+#if ENABLE(MEDIA_SOURCE)
+const unsigned fiveMinutesOf1080PVideo = 290 * 1024 * 1024; // 290 MB is approximately 5 minutes of 8Mbps (1080p) content.
+const unsigned fiveMinutesStereoAudio = 14 * 1024 * 1024; // 14 MB is approximately 5 minutes of 384kbps content.
+
+size_t HTMLMediaSession::maximumMediaSourceBufferSize(const SourceBuffer& buffer) const
+{
+ // A good quality 1080p video uses 8,000 kbps and stereo audio uses 384 kbps, so assume 95% for video and 5% for audio.
+ const float bufferBudgetPercentageForVideo = .95;
+ const float bufferBudgetPercentageForAudio = .05;
+
+ size_t maximum;
+ Settings* settings = buffer.document().settings();
+ if (settings)
+ maximum = settings->maximumSourceBufferSize();
+ else
+ maximum = fiveMinutesOf1080PVideo + fiveMinutesStereoAudio;
+
+ // Allow a SourceBuffer to buffer as though it is audio-only even if it doesn't have any active tracks (yet).
+ size_t bufferSize = static_cast<size_t>(maximum * bufferBudgetPercentageForAudio);
+ if (buffer.hasVideo())
+ bufferSize += static_cast<size_t>(maximum * bufferBudgetPercentageForVideo);
+
+ // FIXME: we might want to modify this algorithm to:
+ // - decrease the maximum size for background tabs
+ // - decrease the maximum size allowed for inactive elements when a process has more than one
+ // element, eg. so a page with many elements which are played one at a time doesn't keep
+ // everything buffered after an element has finished playing.
+
+ return bufferSize;
}
+#endif
+}
+
#endif // ENABLE(VIDEO)
Modified: trunk/Source/WebCore/html/HTMLMediaSession.h (172656 => 172657)
--- trunk/Source/WebCore/html/HTMLMediaSession.h 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/html/HTMLMediaSession.h 2014-08-15 23:09:19 UTC (rev 172657)
@@ -34,6 +34,7 @@
namespace WebCore {
class HTMLMediaElement;
+class SourceBuffer;
class HTMLMediaSession : public MediaSession {
public:
@@ -82,6 +83,10 @@
void addBehaviorRestriction(BehaviorRestrictions);
void removeBehaviorRestriction(BehaviorRestrictions);
+#if ENABLE(MEDIA_SOURCE)
+ size_t maximumMediaSourceBufferSize(const SourceBuffer&) const;
+#endif
+
private:
BehaviorRestrictions m_restrictions;
};
Modified: trunk/Source/WebCore/page/Settings.in (172656 => 172657)
--- trunk/Source/WebCore/page/Settings.in 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/page/Settings.in 2014-08-15 23:09:19 UTC (rev 172657)
@@ -230,4 +230,8 @@
# as a means to pass data from the web-portion of their app to the native portion.
allowNavigationToInvalidURL initial=false
+# Allow SourceBuffers to store up to 304MB each, enough for approximately five minutes
+# of 1080p video and stereo audio.
+maximumSourceBufferSize type=int, initial=318767104, conditional=MEDIA_SOURCE
+
longMousePressEnabled initial=false
Modified: trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.cpp (172656 => 172657)
--- trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.cpp 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.cpp 2014-08-15 23:09:19 UTC (rev 172657)
@@ -27,6 +27,7 @@
#include "PlatformTimeRanges.h"
#include <math.h>
+#include <wtf/PrintStream.h>
namespace WebCore {
@@ -252,11 +253,19 @@
MediaTime PlatformTimeRanges::totalDuration() const
{
MediaTime total = MediaTime::zeroTime();
- bool ignoreInvalid;
for (unsigned n = 0; n < length(); n++)
- total += abs(end(n, ignoreInvalid) - start(n, ignoreInvalid));
+ total += abs(end(n) - start(n));
return total;
}
+void PlatformTimeRanges::dump(PrintStream& out) const
+{
+ if (!length())
+ return;
+
+ for (size_t i = 0; i < length(); ++i)
+ out.print("[", start(i), "..", end(i), "] ");
}
+
+}
Modified: trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.h (172656 => 172657)
--- trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.h 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/platform/graphics/PlatformTimeRanges.h 2014-08-15 23:09:19 UTC (rev 172657)
@@ -32,6 +32,10 @@
#include <wtf/RefCounted.h>
#include <wtf/Vector.h>
+namespace WTF {
+class PrintStream;
+}
+
namespace WebCore {
class PlatformTimeRanges {
@@ -69,6 +73,8 @@
MediaTime totalDuration() const;
+ void dump(WTF::PrintStream&) const;
+
private:
PlatformTimeRanges& copy(const PlatformTimeRanges&);
Modified: trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h (172656 => 172657)
--- trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/platform/graphics/SourceBufferPrivate.h 2014-08-15 23:09:19 UTC (rev 172657)
@@ -1,5 +1,6 @@
/*
* Copyright (C) 2013 Google Inc. All rights reserved.
+ * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are
@@ -55,8 +56,6 @@
virtual MediaPlayer::ReadyState readyState() const = 0;
virtual void setReadyState(MediaPlayer::ReadyState) = 0;
- virtual void evictCodedFrames() = 0;
- virtual bool isFull() = 0;
virtual void flushAndEnqueueNonDisplayingSamples(Vector<RefPtr<MediaSample>>, AtomicString) { }
virtual void enqueueSample(PassRefPtr<MediaSample>, AtomicString) { }
Modified: trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h (172656 => 172657)
--- trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h 2014-08-15 23:09:19 UTC (rev 172657)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -112,8 +112,6 @@
virtual void removedFromMediaSource() override;
virtual MediaPlayer::ReadyState readyState() const override;
virtual void setReadyState(MediaPlayer::ReadyState) override;
- virtual void evictCodedFrames() override;
- virtual bool isFull() override;
virtual void flushAndEnqueueNonDisplayingSamples(Vector<RefPtr<MediaSample>>, AtomicString trackID) override;
virtual void enqueueSample(PassRefPtr<MediaSample>, AtomicString trackID) override;
virtual bool isReadyForMoreSamples(AtomicString trackID) override;
Modified: trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm (172656 => 172657)
--- trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm 2014-08-15 23:09:19 UTC (rev 172657)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -632,7 +632,7 @@
CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer);
FloatSize formatSize = FloatSize(CMVideoFormatDescriptionGetPresentationDimensions(formatDescription, true, true));
if (formatSize != m_cachedSize) {
- LOG(MediaSource, "SourceBufferPrivateAVFObjC::processCodedFrame(%p) - size change detected: {width=%lf, height=%lf", formatSize.width(), formatSize.height());
+ LOG(MediaSource, "SourceBufferPrivateAVFObjC::processCodedFrame(%p) - size change detected: {width=%lf, height=%lf}", formatSize.width(), formatSize.height());
m_cachedSize = formatSize;
if (m_mediaSource)
m_mediaSource->player()->sizeChanged();
@@ -779,18 +779,6 @@
m_mediaSource->player()->setReadyState(readyState);
}
-void SourceBufferPrivateAVFObjC::evictCodedFrames()
-{
- notImplemented();
-}
-
-bool SourceBufferPrivateAVFObjC::isFull()
-{
- notImplemented();
- return false;
-}
-
-
bool SourceBufferPrivateAVFObjC::hasVideo() const
{
if (!m_client)
Modified: trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp (172656 => 172657)
--- trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp 2014-08-15 23:09:19 UTC (rev 172657)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -213,16 +213,6 @@
m_mediaSource->player()->setReadyState(readyState);
}
-void MockSourceBufferPrivate::evictCodedFrames()
-{
- // No-op.
-}
-
-bool MockSourceBufferPrivate::isFull()
-{
- return false;
-}
-
void MockSourceBufferPrivate::setActive(bool isActive)
{
if (m_mediaSource)
Modified: trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.h (172656 => 172657)
--- trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.h 2014-08-15 22:43:50 UTC (rev 172656)
+++ trunk/Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.h 2014-08-15 23:09:19 UTC (rev 172657)
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2013 Apple Inc. All rights reserved.
+ * Copyright (C) 2013-2014 Apple Inc. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
@@ -68,8 +68,6 @@
virtual void removedFromMediaSource() override;
virtual MediaPlayer::ReadyState readyState() const override;
virtual void setReadyState(MediaPlayer::ReadyState) override;
- virtual void evictCodedFrames() override;
- virtual bool isFull() override;
virtual void flushAndEnqueueNonDisplayingSamples(Vector<RefPtr<MediaSample>>, AtomicString) override { }
virtual void enqueueSample(PassRefPtr<MediaSample>, AtomicString) override;
