- Revision
- 261525
- Author
- [email protected]
- Date
- 2020-05-11 17:22:12 -0700 (Mon, 11 May 2020)
Log Message
Cherry-pick r258977. rdar://problem/62978907
Audio fails to capture stream in WebRTC if AudioSession gets interrupted
https://bugs.webkit.org/show_bug.cgi?id=208516
<rdar://problem/60020467>
Reviewed by Eric Carlson.
In case of page going to hidden, continue calling each capture factory to mute the corresponding sources if needed.
In case of page being visible again, reset all tracks according page muted state. This allows restarting tracks that have been
muted while page was hidden (video tracks or suspended audio tracks).
Since tracks can go to muted when visibility changes, we no longer return early when setting the muted state of a page to the same value.
Instead we apply it which ensures we comply with what UIProcess wants.
We start removing the concept of a RealtimeMediaSource be interrupted. Instead we use muting of sources.
This allows UIProcess or the page to override any muted state, for instance if page goes in foreground again.
We update the AudioSharedUnit to allow restarting capture even if suspended.
This ensures that we are able to restart capturing even if we do not receive the audio session end of interruption.
Also, this notification sometimes takes a long time to happen and we do not want to wait for it when user is interacting with the page.
A future refactoring will further remove RealtimeMediaSource interrupted-related code.
Manually tested.
* dom/Document.cpp:
(WebCore::Document::visibilityStateChanged):
* page/Page.cpp:
(WebCore::Page::setMuted):
* platform/audio/PlatformMediaSessionManager.h:
(WebCore::PlatformMediaSessionManager::isInterrupted const):
* platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::setInterrupted):
(WebCore::RealtimeMediaSource::setMuted):
* platform/mediastream/mac/BaseAudioSharedUnit.cpp:
(WebCore::BaseAudioSharedUnit::startProducingData):
(WebCore::BaseAudioSharedUnit::resume):
(WebCore::BaseAudioSharedUnit::suspend):
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@258977 268f45cc-cd09-0410-ab3c-d52691b4dbfc
Modified Paths
Diff
Modified: branches/safari-609-branch/Source/WebCore/ChangeLog (261524 => 261525)
--- branches/safari-609-branch/Source/WebCore/ChangeLog 2020-05-12 00:22:08 UTC (rev 261524)
+++ branches/safari-609-branch/Source/WebCore/ChangeLog 2020-05-12 00:22:12 UTC (rev 261525)
@@ -1,3 +1,85 @@
+2020-05-11 Alan Coon <[email protected]>
+
+ Cherry-pick r258977. rdar://problem/62978907
+
+ Audio fails to capture stream in WebRTC if AudioSession gets interrupted
+ https://bugs.webkit.org/show_bug.cgi?id=208516
+ <rdar://problem/60020467>
+
+ Reviewed by Eric Carlson.
+
+ In case of page going to hidden, continue calling each capture factory to mute the corresponding sources if needed.
+ In case of page being visible again, reset all tracks according page muted state. This allows restarting tracks that have been
+ muted while page was hidden (video tracks or suspended audio tracks).
+
+ Since tracks can go to muted when visibility changes, we no longer return early when setting the muted state of a page to the same value.
+ Instead we apply it which ensures we comply with what UIProcess wants.
+
+ We start removing the concept of a RealtimeMediaSource be interrupted. Instead we use muting of sources.
+ This allows UIProcess or the page to override any muted state, for instance if page goes in foreground again.
+
+ We update the AudioSharedUnit to allow restarting capture even if suspended.
+ This ensures that we are able to restart capturing even if we do not receive the audio session end of interruption.
+ Also, this notification sometimes takes a long time to happen and we do not want to wait for it when user is interacting with the page.
+ A future refactoring will further remove RealtimeMediaSource interrupted-related code.
+
+ Manually tested.
+
+ * dom/Document.cpp:
+ (WebCore::Document::visibilityStateChanged):
+ * page/Page.cpp:
+ (WebCore::Page::setMuted):
+ * platform/audio/PlatformMediaSessionManager.h:
+ (WebCore::PlatformMediaSessionManager::isInterrupted const):
+ * platform/mediastream/RealtimeMediaSource.cpp:
+ (WebCore::RealtimeMediaSource::setInterrupted):
+ (WebCore::RealtimeMediaSource::setMuted):
+ * platform/mediastream/mac/BaseAudioSharedUnit.cpp:
+ (WebCore::BaseAudioSharedUnit::startProducingData):
+ (WebCore::BaseAudioSharedUnit::resume):
+ (WebCore::BaseAudioSharedUnit::suspend):
+
+ git-svn-id: https://svn.webkit.org/repository/webkit/trunk@258977 268f45cc-cd09-0410-ab3c-d52691b4dbfc
+
+ 2020-03-25 Youenn Fablet <[email protected]>
+
+ Audio fails to capture stream in WebRTC if AudioSession gets interrupted
+ https://bugs.webkit.org/show_bug.cgi?id=208516
+ <rdar://problem/60020467>
+
+ Reviewed by Eric Carlson.
+
+ In case of page going to hidden, continue calling each capture factory to mute the corresponding sources if needed.
+ In case of page being visible again, reset all tracks according page muted state. This allows restarting tracks that have been
+ muted while page was hidden (video tracks or suspended audio tracks).
+
+ Since tracks can go to muted when visibility changes, we no longer return early when setting the muted state of a page to the same value.
+ Instead we apply it which ensures we comply with what UIProcess wants.
+
+ We start removing the concept of a RealtimeMediaSource be interrupted. Instead we use muting of sources.
+ This allows UIProcess or the page to override any muted state, for instance if page goes in foreground again.
+
+ We update the AudioSharedUnit to allow restarting capture even if suspended.
+ This ensures that we are able to restart capturing even if we do not receive the audio session end of interruption.
+ Also, this notification sometimes takes a long time to happen and we do not want to wait for it when user is interacting with the page.
+ A future refactoring will further remove RealtimeMediaSource interrupted-related code.
+
+ Manually tested.
+
+ * dom/Document.cpp:
+ (WebCore::Document::visibilityStateChanged):
+ * page/Page.cpp:
+ (WebCore::Page::setMuted):
+ * platform/audio/PlatformMediaSessionManager.h:
+ (WebCore::PlatformMediaSessionManager::isInterrupted const):
+ * platform/mediastream/RealtimeMediaSource.cpp:
+ (WebCore::RealtimeMediaSource::setInterrupted):
+ (WebCore::RealtimeMediaSource::setMuted):
+ * platform/mediastream/mac/BaseAudioSharedUnit.cpp:
+ (WebCore::BaseAudioSharedUnit::startProducingData):
+ (WebCore::BaseAudioSharedUnit::resume):
+ (WebCore::BaseAudioSharedUnit::suspend):
+
2020-05-07 Russell Epstein <[email protected]>
Cherry-pick r261208. rdar://problem/62978272
Modified: branches/safari-609-branch/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp (261524 => 261525)
--- branches/safari-609-branch/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp 2020-05-12 00:22:08 UTC (rev 261524)
+++ branches/safari-609-branch/Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp 2020-05-12 00:22:12 UTC (rev 261525)
@@ -461,13 +461,43 @@
return state;
}
+#if PLATFORM(IOS_FAMILY)
+static MediaStreamTrack* findActiveCaptureTrackForDocument(Document& document, RealtimeMediaSource* activeSource, RealtimeMediaSource::Type type)
+{
+ MediaStreamTrack* selectedTrack = nullptr;
+ for (auto* captureTrack : allCaptureTracks()) {
+ if (captureTrack->document() != &document || captureTrack->ended())
+ continue;
+
+ if (&captureTrack->source() == activeSource)
+ return captureTrack;
+
+ // If the document has a live capture track, which is not the active one, we pick the first one.
+ // FIXME: We should probably store per page active audio/video capture tracks.
+ if (!selectedTrack && captureTrack->privateTrack().type() == type)
+ selectedTrack = captureTrack;
+ }
+ return selectedTrack;
+}
+#endif
+
void MediaStreamTrack::updateCaptureAccordingToMutedState(Document& document)
{
+#if PLATFORM(IOS_FAMILY)
+ auto* activeAudioSource = RealtimeMediaSourceCenter::singleton().audioCaptureFactory().activeSource();
+ if (auto* audioCaptureTrack = findActiveCaptureTrackForDocument(document, activeAudioSource, RealtimeMediaSource::Type::Audio))
+ audioCaptureTrack->setMuted(document.page()->mutedState());
+
+ auto* activeVideoSource = RealtimeMediaSourceCenter::singleton().videoCaptureFactory().activeSource();
+ if (auto* videoCaptureTrack = findActiveCaptureTrackForDocument(document, activeVideoSource, RealtimeMediaSource::Type::Video))
+ videoCaptureTrack->setMuted(document.page()->mutedState());
+#else
for (auto* captureTrack : allCaptureTracks()) {
if (captureTrack->document() != &document || captureTrack->ended())
continue;
captureTrack->setMuted(document.page()->mutedState());
}
+#endif
}
void MediaStreamTrack::endCapture(Document& document)
Modified: branches/safari-609-branch/Source/WebCore/Modules/mediastream/MediaStreamTrack.h (261524 => 261525)
--- branches/safari-609-branch/Source/WebCore/Modules/mediastream/MediaStreamTrack.h 2020-05-12 00:22:08 UTC (rev 261524)
+++ branches/safari-609-branch/Source/WebCore/Modules/mediastream/MediaStreamTrack.h 2020-05-12 00:22:12 UTC (rev 261525)
@@ -154,6 +154,8 @@
void setIdForTesting(String&& id) { m_private->setIdForTesting(WTFMove(id)); }
+ Document* document() const;
+
#if !RELEASE_LOG_DISABLED
const Logger& logger() const final { return m_private->logger(); }
const void* logIdentifier() const final { return m_private->logIdentifier(); }
@@ -171,8 +173,6 @@
void configureTrackRendering();
- Document* document() const;
-
// ActiveDOMObject API.
void stop() final { stopTrack(); }
const char* activeDOMObjectName() const override;
Modified: branches/safari-609-branch/Source/WebCore/dom/Document.cpp (261524 => 261525)
--- branches/safari-609-branch/Source/WebCore/dom/Document.cpp 2020-05-12 00:22:08 UTC (rev 261524)
+++ branches/safari-609-branch/Source/WebCore/dom/Document.cpp 2020-05-12 00:22:12 UTC (rev 261525)
@@ -1729,9 +1729,13 @@
for (auto* client : m_visibilityStateCallbackClients)
client->visibilityStateChanged();
-#if ENABLE(MEDIA_STREAM)
- if (auto* page = this->page())
- RealtimeMediaSourceCenter::singleton().setCapturePageState(hidden(), page->isMediaCaptureMuted());
+#if PLATFORM(IOS_FAMILY)
+ if (hidden()) {
+ RealtimeMediaSourceCenter::singleton().setCapturePageState(hidden(), page()->isMediaCaptureMuted());
+ return;
+ }
+ if (!PlatformMediaSessionManager::sharedManager().isInterrupted())
+ MediaStreamTrack::updateCaptureAccordingToMutedState(*this);
#endif
}
Modified: branches/safari-609-branch/Source/WebCore/page/Page.cpp (261524 => 261525)
--- branches/safari-609-branch/Source/WebCore/page/Page.cpp 2020-05-12 00:22:08 UTC (rev 261524)
+++ branches/safari-609-branch/Source/WebCore/page/Page.cpp 2020-05-12 00:22:12 UTC (rev 261525)
@@ -1761,9 +1761,6 @@
void Page::setMuted(MediaProducer::MutedStateFlags muted)
{
- if (m_mutedState == muted)
- return;
-
m_mutedState = muted;
forEachDocument([] (Document& document) {
Modified: branches/safari-609-branch/Source/WebCore/platform/audio/PlatformMediaSessionManager.h (261524 => 261525)
--- branches/safari-609-branch/Source/WebCore/platform/audio/PlatformMediaSessionManager.h 2020-05-12 00:22:08 UTC (rev 261524)
+++ branches/safari-609-branch/Source/WebCore/platform/audio/PlatformMediaSessionManager.h 2020-05-12 00:22:12 UTC (rev 261525)
@@ -137,6 +137,8 @@
bool processIsSuspended() const { return m_processIsSuspended; }
+ bool isInterrupted() const { return m_interrupted; }
+
protected:
friend class PlatformMediaSession;
explicit PlatformMediaSessionManager();
Modified: branches/safari-609-branch/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp (261524 => 261525)
--- branches/safari-609-branch/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp 2020-05-12 00:22:08 UTC (rev 261524)
+++ branches/safari-609-branch/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp 2020-05-12 00:22:12 UTC (rev 261525)
@@ -76,25 +76,12 @@
void RealtimeMediaSource::setInterrupted(bool interrupted, bool pageMuted)
{
- if (interrupted == m_interrupted)
- return;
-
ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, interrupted, ", page muted : ", pageMuted);
-
- m_interrupted = interrupted;
- if (!interrupted && pageMuted)
- return;
-
setMuted(interrupted);
}
void RealtimeMediaSource::setMuted(bool muted)
{
- if (!muted && interrupted()) {
- ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, "ignoring unmute because of interruption");
- return;
- }
-
ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, muted);
// Changed m_muted before calling start/stop so muted() will reflect the correct state.
Modified: branches/safari-609-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h (261524 => 261525)
--- branches/safari-609-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h 2020-05-12 00:22:08 UTC (rev 261524)
+++ branches/safari-609-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h 2020-05-12 00:22:12 UTC (rev 261525)
@@ -43,7 +43,6 @@
WEBCORE_EXPORT void setActiveSource(RealtimeMediaSource&);
WEBCORE_EXPORT void unsetActiveSource(RealtimeMediaSource&);
-protected:
RealtimeMediaSource* activeSource() { return m_activeSource; }
private:
Modified: branches/safari-609-branch/Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp (261524 => 261525)
--- branches/safari-609-branch/Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp 2020-05-12 00:22:08 UTC (rev 261524)
+++ branches/safari-609-branch/Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp 2020-05-12 00:22:12 UTC (rev 261525)
@@ -73,6 +73,9 @@
{
ASSERT(isMainThread());
+ if (m_suspended)
+ resume();
+
if (++m_producingCount != 1)
return;
@@ -79,11 +82,6 @@
if (isProducingData())
return;
- if (m_suspended) {
- RELEASE_LOG_INFO(WebRTC, "BaseAudioSharedUnit::startProducingData - exiting early as suspended");
- return;
- }
-
if (hasAudioUnit()) {
cleanupAudioUnit();
ASSERT(!hasAudioUnit());
@@ -160,7 +158,9 @@
OSStatus BaseAudioSharedUnit::resume()
{
ASSERT(isMainThread());
- ASSERT(m_suspended);
+ if (!m_suspended)
+ return 0;
+
ASSERT(!isProducingData());
RELEASE_LOG_INFO(WebRTC, "BaseAudioSharedUnit::resume");
@@ -181,7 +181,7 @@
}
forEachClient([](auto& client) {
- client.notifyMutedChange(false);
+ client.setMuted(false);
});
return 0;
@@ -197,7 +197,7 @@
stopInternal();
forEachClient([](auto& client) {
- client.notifyMutedChange(true);
+ client.setMuted(true);
});
return 0;
