Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 4918b7c46048ecbb3de1f268c7f424fc7bb5dca3
https://github.com/WebKit/WebKit/commit/4918b7c46048ecbb3de1f268c7f424fc7bb5dca3
Author: Youenn Fablet <[email protected]>
Date: 2024-09-06 (Fri, 06 Sep 2024)
Changed paths:
M LayoutTests/http/wpt/mediasession/resources/toggleCapture-iframe.html
A LayoutTests/http/wpt/mediasession/voiceActivityDetection-expected.txt
A LayoutTests/http/wpt/mediasession/voiceActivityDetection.html
M LayoutTests/platform/glib/TestExpectations
M LayoutTests/platform/mac-ventura/TestExpectations
M Source/WTF/wtf/PlatformHave.h
M Source/WebCore/Modules/mediasession/MediaSession.cpp
M Source/WebCore/Modules/mediasession/MediaSessionAction.h
M Source/WebCore/Modules/mediasession/MediaSessionAction.idl
M Source/WebCore/Modules/mediasession/MediaSessionActionDetails.h
M Source/WebCore/Modules/mediastream/UserMediaClient.h
M Source/WebCore/Modules/mediastream/UserMediaController.cpp
M Source/WebCore/Modules/mediastream/UserMediaController.h
M Source/WebCore/WebCore.xcodeproj/project.pbxproj
M Source/WebCore/dom/Document.cpp
M Source/WebCore/dom/Document.h
M Source/WebCore/page/Page.cpp
M Source/WebCore/page/Page.h
M Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h
M Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp
M Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.h
M Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp
M Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h
A Source/WebCore/platform/mediastream/mac/CoreAudioSharedInternalUnit.h
M Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.cpp
M Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.h
A Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.mm
M Source/WebCore/platform/mediastream/mac/MockAudioSharedUnit.mm
M Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp
M Source/WebKit/GPUProcess/GPUProcess.cpp
M Source/WebKit/GPUProcess/GPUProcess.h
M Source/WebKit/GPUProcess/GPUProcess.messages.in
M Source/WebKit/UIProcess/GPU/GPUProcessProxy.cpp
M Source/WebKit/UIProcess/GPU/GPUProcessProxy.h
M Source/WebKit/UIProcess/GPU/GPUProcessProxy.messages.in
M Source/WebKit/UIProcess/WebPageProxy.cpp
M Source/WebKit/UIProcess/WebPageProxy.h
M Source/WebKit/UIProcess/WebPageProxy.messages.in
M Source/WebKit/WebProcess/WebCoreSupport/WebUserMediaClient.cpp
M Source/WebKit/WebProcess/WebCoreSupport/WebUserMediaClient.h
M Source/WebKit/WebProcess/WebPage/WebPage.cpp
M Source/WebKit/WebProcess/WebPage/WebPage.h
M Source/WebKit/WebProcess/WebPage/WebPage.messages.in
Log Message:
-----------
Implement MediaSession voiceactivity action
rdar://133054919
https://bugs.webkit.org/show_bug.cgi?id=277545
Reviewed by Eric Carlson.
We implement voiceactivity detection on macOS and iOS based on OS APIs.
This is done in GPU process and triggers voiceactivity action for documents
having muted microphone tracks.
This detection works even if microphone is not in use, as long as the audio
unit is running.
When being asked to stop the audio unit, we check whether voiceactivity
detection is on.
If so, we mute microphone capture and let the audio unit running.
Added a mock based test.
Manually tested.
* LayoutTests/http/wpt/mediasession/resources/toggleCapture-iframe.html:
* LayoutTests/http/wpt/mediasession/voiceActivityDetection-expected.txt: Added.
* LayoutTests/http/wpt/mediasession/voiceActivityDetection.html: Added.
* LayoutTests/platform/glib/TestExpectations:
* LayoutTests/platform/mac-ventura/TestExpectations:
* Source/WTF/wtf/PlatformHave.h:
* Source/WebCore/Modules/mediasession/MediaSession.cpp:
(WebCore::platformCommandForMediaSessionAction):
(WebCore::MediaSession::setActionHandler):
* Source/WebCore/Modules/mediasession/MediaSessionAction.h:
* Source/WebCore/Modules/mediasession/MediaSessionAction.idl:
* Source/WebCore/Modules/mediastream/UserMediaClient.h:
* Source/WebCore/Modules/mediastream/UserMediaController.cpp:
(WebCore::UserMediaController::setShouldListenToVoiceActivity):
(WebCore::UserMediaController::checkDocumentForVoiceActivity):
(WebCore::UserMediaController::voiceActivityDetected):
* Source/WebCore/Modules/mediastream/UserMediaController.h:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/dom/Document.cpp:
(WebCore::Document::updateIsPlayingMedia):
(WebCore::Document::hasMutedAudioCaptureDevice const):
(WebCore::Document::setShouldListenToVoiceActivity):
(WebCore::Document::voiceActivityDetected):
* Source/WebCore/dom/Document.h:
* Source/WebCore/page/Page.cpp:
(WebCore::Page::voiceActivityDetected):
* Source/WebCore/page/Page.h:
* Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h:
(WebCore::AudioCaptureFactory::enableMutedSpeechActivityEventListener):
(WebCore::AudioCaptureFactory::disableMutedSpeechActivityEventListener):
* Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.cpp:
(WebCore::BaseAudioSharedUnit::stopProducingData):
* Source/WebCore/platform/mediastream/mac/BaseAudioSharedUnit.h:
(WebCore::BaseAudioSharedUnit::captureDeviceID const):
(WebCore::BaseAudioSharedUnit::persistentID const):
(WebCore::BaseAudioSharedUnit::setVoiceActivityListenerCallback):
(WebCore::BaseAudioSharedUnit::voiceActivityDetected):
(WebCore::BaseAudioSharedUnit::isListeningToVoiceActivity const):
* Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.cpp:
(WebCore::CoreAudioCaptureSourceFactory::enableMutedSpeechActivityEventListener):
(WebCore::CoreAudioCaptureSourceFactory::disableMutedSpeechActivityEventListener):
* Source/WebCore/platform/mediastream/mac/CoreAudioCaptureSource.h:
* Source/WebCore/platform/mediastream/mac/CoreAudioSharedInternalUnit.h: Added.
* Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.cpp:
(WebCore::CoreAudioSharedUnit::startInternal):
(WebCore::CoreAudioSharedUnit::isProducingMicrophoneSamplesChanged):
(WebCore::CoreAudioSharedUnit::prewarmAudioUnitCreation):
(): Deleted.
* Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.h:
* Source/WebCore/platform/mediastream/mac/CoreAudioSharedUnit.mm: Added.
(WebCore::speechActivityListenerCallback):
(WebCore::manageSpeechActivityListener):
(WebCore::CoreAudioSharedUnit::processVoiceActivityEvent):
(WebCore::CoreAudioSharedInternalUnit::setVoiceActivityDetection):
(WebCore::CoreAudioSharedUnit::enableMutedSpeechActivityEventListener):
(WebCore::CoreAudioSharedUnit::disableMutedSpeechActivityEventListener):
* Source/WebCore/platform/mediastream/mac/MockAudioSharedUnit.mm:
(WebCore::m_voiceDetectionTimer):
(WebCore::MockAudioSharedInternalUnit::setVoiceActivityDetection):
(WebCore::MockAudioSharedInternalUnit::voiceDetected):
(WebCore::MockAudioSharedInternalUnit::set):
* Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp:
* Source/WebKit/GPUProcess/GPUProcess.cpp:
(WebKit::GPUProcess::setShouldListenToVoiceActivity):
* Source/WebKit/GPUProcess/GPUProcess.h:
* Source/WebKit/GPUProcess/GPUProcess.messages.in:
* Source/WebKit/UIProcess/GPU/GPUProcessProxy.cpp:
(WebKit::GPUProcessProxy::setShouldListenToVoiceActivity):
(WebKit::GPUProcessProxy::voiceActivityDetected):
* Source/WebKit/UIProcess/GPU/GPUProcessProxy.h:
* Source/WebKit/UIProcess/GPU/GPUProcessProxy.messages.in:
* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::resetState):
(WebKit::WebPageProxy::setShouldListenToVoiceActivity):
(WebKit::WebPageProxy::voiceActivityDetected):
(WebKit::WebPageProxy::gpuProcessExited):
* Source/WebKit/UIProcess/WebPageProxy.h:
* Source/WebKit/UIProcess/WebPageProxy.messages.in:
* Source/WebKit/WebProcess/WebCoreSupport/WebUserMediaClient.cpp:
(WebKit::WebUserMediaClient::setShouldListenToVoiceActivity):
* Source/WebKit/WebProcess/WebCoreSupport/WebUserMediaClient.h:
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::voiceActivityDetected):
* Source/WebKit/WebProcess/WebPage/WebPage.h:
* Source/WebKit/WebProcess/WebPage/WebPage.messages.in:
Canonical link: https://commits.webkit.org/283278@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
