Diff
Modified: branches/safari-611-branch/LayoutTests/ChangeLog (271840 => 271841)
--- branches/safari-611-branch/LayoutTests/ChangeLog 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/LayoutTests/ChangeLog 2021-01-25 22:12:26 UTC (rev 271841)
@@ -1,5 +1,89 @@
2021-01-25 Alan Coon <[email protected]>
+ Cherry-pick r271471. rdar://problem/73477068
+
+ Source/WebCore:
+ [HANG] 496ms to 1360ms in WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices()
+ https://bugs.webkit.org/show_bug.cgi?id=220471
+
+ Reviewed by Youenn Fablet.
+
+ Refactor RealtimeMediaSourceCenter::getMediaStreamDevices() to take a completion handler, rather than
+ synchronously return a Vector of CaptureDevices. This also requires all CaptureDeviceManager subclasses
+ to support taking a completion handler themselves. By default, all CaptureDeviceManagers will support
+ the CompletionHandler path by just synchronously calling the completion handler with the existing
+ synchronous method. But for AVAudioSessionCaptureDeviceManager, override that default implementation by
+ activating the AVAudioSession on a background thread, and querying that session's inputs on a background
+ thread as well.
+
+ * platform/mediastream/CaptureDeviceManager.h:
+ * platform/mediastream/RealtimeMediaSourceCenter.cpp:
+ (WebCore::RealtimeMediaSourceCenter::getMediaStreamDevices):
+ * platform/mediastream/RealtimeMediaSourceCenter.h:
+ * platform/mediastream/RealtimeMediaSourceFactory.h:
+ (WebCore::AudioCaptureFactory::getSpeakerDevices const):
+ * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h:
+ * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:
+ (WebCore::AVAudioSessionCaptureDeviceManager::audioSessionDeviceWithUID):
+ (WebCore::AVAudioSessionCaptureDeviceManager::scheduleUpdateCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::getCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::activateAudioSession):
+ (WebCore::AVAudioSessionCaptureDeviceManager::retrieveAudioSessionCaptureDevices const):
+ (WebCore::AVAudioSessionCaptureDeviceManager::setAudioCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::audioSessionCaptureDevices): Deleted.
+
+ Source/WebKit:
+ [HANG] 496ms to 1360ms in WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices()
+ https://bugs.webkit.org/show_bug.cgi?id=220471
+
+ Reviewed by Youenn Fablet.
+
+ Use the completion-handler version of RealtimeMediaSourceCenter::getMediaStreamDevices().
+
+ * UIProcess/UserMediaPermissionRequestManagerProxy.cpp:
+ (WebKit::UserMediaPermissionRequestManagerProxy::computeFilteredDeviceList):
+ (WebKit::UserMediaPermissionRequestManagerProxy::enumerateMediaDevicesForFrame):
+ * UIProcess/UserMediaPermissionRequestManagerProxy.h:
+ * UIProcess/UserMediaProcessManager.cpp:
+ (WebKit::UserMediaProcessManager::updateCaptureDevices):
+ (WebKit::UserMediaProcessManager::beginMonitoringCaptureDevices):
+ * UIProcess/UserMediaProcessManager.h:
+
+ LayoutTests:
+ [Cocoa] Support key rotation with HLS-backed encrypted media streams
+ https://bugs.webkit.org/show_bug.cgi?id=220493
+ <rdar://68227709>
+
+ Reviewed by Youenn Fablet.
+
+ Fix a broken layout test; the test enumerates devices, then uses the deviceIds returned to generate
+ constraints for a call to getUserMedia(). However, it assumes all devices will either be of kind
+ 'audioinput' or 'videoinput'. If an 'audiooutput' device is returned (as the MockRealtimeMediaSourceCenter
+ does), then the test turns that into a video capture constraint, which fails.
+
+ * fast/mediastream/get-user-media-device-id.html:
+
+
+ git-svn-id: https://svn.webkit.org/repository/webkit/trunk@271471 268f45cc-cd09-0410-ab3c-d52691b4dbfc
+
+ 2021-01-13 Jer Noble <[email protected]>
+
+ [Cocoa] Support key rotation with HLS-backed encrypted media streams
+ https://bugs.webkit.org/show_bug.cgi?id=220493
+ <rdar://68227709>
+
+ Reviewed by Youenn Fablet.
+
+ Fix a broken layout test; the test enumerates devices, then uses the deviceIds returned to generate
+ constraints for a call to getUserMedia(). However, it assumes all devices will either be of kind
+ 'audioinput' or 'videoinput'. If an 'audiooutput' device is returned (as the MockRealtimeMediaSourceCenter
+ does), then the test turns that into a video capture constraint, which fails.
+
+ * fast/mediastream/get-user-media-device-id.html:
+
+2021-01-25 Alan Coon <[email protected]>
+
Cherry-pick r271453. rdar://problem/73477471
Fix for LayoutTests/accessibility/mac/details-summary.html in isolated tree mode.
Modified: branches/safari-611-branch/LayoutTests/fast/mediastream/get-user-media-device-id.html (271840 => 271841)
--- branches/safari-611-branch/LayoutTests/fast/mediastream/get-user-media-device-id.html 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/LayoutTests/fast/mediastream/get-user-media-device-id.html 2021-01-25 22:12:26 UTC (rev 271841)
@@ -25,6 +25,8 @@
return navigator.mediaDevices.enumerateDevices();
}).then(devices => {
devices.forEach((device) => {
+ if (device.kind == "audiooutput")
+ return;
let kind = device.kind == "audioinput" ? "audio" : "video";
deviceIds.push({ type: kind, id : device.deviceId});
});
Modified: branches/safari-611-branch/Source/WebCore/ChangeLog (271840 => 271841)
--- branches/safari-611-branch/Source/WebCore/ChangeLog 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebCore/ChangeLog 2021-01-25 22:12:26 UTC (rev 271841)
@@ -1,5 +1,106 @@
2021-01-25 Alan Coon <[email protected]>
+ Cherry-pick r271471. rdar://problem/73477068
+
+ Source/WebCore:
+ [HANG] 496ms to 1360ms in WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices()
+ https://bugs.webkit.org/show_bug.cgi?id=220471
+
+ Reviewed by Youenn Fablet.
+
+ Refactor RealtimeMediaSourceCenter::getMediaStreamDevices() to take a completion handler, rather than
+ synchronously return a Vector of CaptureDevices. This also requires all CaptureDeviceManager subclasses
+ to support taking a completion handler themselves. By default, all CaptureDeviceManagers will support
+ the CompletionHandler path by just synchronously calling the completion handler with the existing
+ synchronous method. But for AVAudioSessionCaptureDeviceManager, override that default implementation by
+ activating the AVAudioSession on a background thread, and querying that session's inputs on a background
+ thread as well.
+
+ * platform/mediastream/CaptureDeviceManager.h:
+ * platform/mediastream/RealtimeMediaSourceCenter.cpp:
+ (WebCore::RealtimeMediaSourceCenter::getMediaStreamDevices):
+ * platform/mediastream/RealtimeMediaSourceCenter.h:
+ * platform/mediastream/RealtimeMediaSourceFactory.h:
+ (WebCore::AudioCaptureFactory::getSpeakerDevices const):
+ * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h:
+ * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:
+ (WebCore::AVAudioSessionCaptureDeviceManager::audioSessionDeviceWithUID):
+ (WebCore::AVAudioSessionCaptureDeviceManager::scheduleUpdateCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::getCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::activateAudioSession):
+ (WebCore::AVAudioSessionCaptureDeviceManager::retrieveAudioSessionCaptureDevices const):
+ (WebCore::AVAudioSessionCaptureDeviceManager::setAudioCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::audioSessionCaptureDevices): Deleted.
+
+ Source/WebKit:
+ [HANG] 496ms to 1360ms in WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices()
+ https://bugs.webkit.org/show_bug.cgi?id=220471
+
+ Reviewed by Youenn Fablet.
+
+ Use the completion-handler version of RealtimeMediaSourceCenter::getMediaStreamDevices().
+
+ * UIProcess/UserMediaPermissionRequestManagerProxy.cpp:
+ (WebKit::UserMediaPermissionRequestManagerProxy::computeFilteredDeviceList):
+ (WebKit::UserMediaPermissionRequestManagerProxy::enumerateMediaDevicesForFrame):
+ * UIProcess/UserMediaPermissionRequestManagerProxy.h:
+ * UIProcess/UserMediaProcessManager.cpp:
+ (WebKit::UserMediaProcessManager::updateCaptureDevices):
+ (WebKit::UserMediaProcessManager::beginMonitoringCaptureDevices):
+ * UIProcess/UserMediaProcessManager.h:
+
+ LayoutTests:
+ [Cocoa] Support key rotation with HLS-backed encrypted media streams
+ https://bugs.webkit.org/show_bug.cgi?id=220493
+ <rdar://68227709>
+
+ Reviewed by Youenn Fablet.
+
+ Fix a broken layout test; the test enumerates devices, then uses the deviceIds returned to generate
+ constraints for a call to getUserMedia(). However, it assumes all devices will either be of kind
+ 'audioinput' or 'videoinput'. If an 'audiooutput' device is returned (as the MockRealtimeMediaSourceCenter
+ does), then the test turns that into a video capture constraint, which fails.
+
+ * fast/mediastream/get-user-media-device-id.html:
+
+
+ git-svn-id: https://svn.webkit.org/repository/webkit/trunk@271471 268f45cc-cd09-0410-ab3c-d52691b4dbfc
+
+ 2021-01-13 Jer Noble <[email protected]>
+
+ [HANG] 496ms to 1360ms in WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices()
+ https://bugs.webkit.org/show_bug.cgi?id=220471
+
+ Reviewed by Youenn Fablet.
+
+ Refactor RealtimeMediaSourceCenter::getMediaStreamDevices() to take a completion handler, rather than
+ synchronously return a Vector of CaptureDevices. This also requires all CaptureDeviceManager subclasses
+ to support taking a completion handler themselves. By default, all CaptureDeviceManagers will support
+ the CompletionHandler path by just synchronously calling the completion handler with the existing
+ synchronous method. But for AVAudioSessionCaptureDeviceManager, override that default implementation by
+ activating the AVAudioSession on a background thread, and querying that session's inputs on a background
+ thread as well.
+
+ * platform/mediastream/CaptureDeviceManager.h:
+ * platform/mediastream/RealtimeMediaSourceCenter.cpp:
+ (WebCore::RealtimeMediaSourceCenter::getMediaStreamDevices):
+ * platform/mediastream/RealtimeMediaSourceCenter.h:
+ * platform/mediastream/RealtimeMediaSourceFactory.h:
+ (WebCore::AudioCaptureFactory::getSpeakerDevices const):
+ * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h:
+ * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:
+ (WebCore::AVAudioSessionCaptureDeviceManager::audioSessionDeviceWithUID):
+ (WebCore::AVAudioSessionCaptureDeviceManager::scheduleUpdateCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::getCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::activateAudioSession):
+ (WebCore::AVAudioSessionCaptureDeviceManager::retrieveAudioSessionCaptureDevices const):
+ (WebCore::AVAudioSessionCaptureDeviceManager::setAudioCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::audioSessionCaptureDevices): Deleted.
+
+2021-01-25 Alan Coon <[email protected]>
+
Cherry-pick r271436. rdar://problem/73477448
REGRESSION(r268666) Incorrect vertical position inside grid items with padding
Modified: branches/safari-611-branch/Source/WebCore/platform/mediastream/CaptureDeviceManager.h (271840 => 271841)
--- branches/safari-611-branch/Source/WebCore/platform/mediastream/CaptureDeviceManager.h 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebCore/platform/mediastream/CaptureDeviceManager.h 2021-01-25 22:12:26 UTC (rev 271841)
@@ -35,6 +35,7 @@
class WEBCORE_EXPORT CaptureDeviceManager {
public:
virtual const Vector<CaptureDevice>& captureDevices() = 0;
+ virtual void getCaptureDevices(CompletionHandler<void(Vector<CaptureDevice>&&)>&& callback) { callback(copyToVector(captureDevices())); };
virtual Optional<CaptureDevice> captureDeviceWithPersistentID(CaptureDevice::DeviceType, const String&) { return WTF::nullopt; }
protected:
Modified: branches/safari-611-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp (271840 => 271841)
--- branches/safari-611-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp 2021-01-25 22:12:26 UTC (rev 271841)
@@ -112,27 +112,48 @@
audioSource->whenReady(WTFMove(whenAudioSourceReady));
}
-Vector<CaptureDevice> RealtimeMediaSourceCenter::getMediaStreamDevices()
+void RealtimeMediaSourceCenter::getMediaStreamDevices(CompletionHandler<void(Vector<CaptureDevice>&&)>&& completion)
{
- Vector<CaptureDevice> result;
- for (auto& device : audioCaptureFactory().audioCaptureDeviceManager().captureDevices()) {
- if (device.enabled())
- result.append(device);
- }
- for (auto& device : audioCaptureFactory().speakerDevices()) {
- if (device.enabled())
- result.append(device);
- }
- for (auto& device : videoCaptureFactory().videoCaptureDeviceManager().captureDevices()) {
- if (device.enabled())
- result.append(device);
- }
- for (auto& device : displayCaptureFactory().displayCaptureDeviceManager().captureDevices()) {
- if (device.enabled())
- result.append(device);
- }
+ class CaptureDeviceAccumulator : public RefCounted<CaptureDeviceAccumulator> {
+ public:
+ static Ref<CaptureDeviceAccumulator> create(CompletionHandler<void(Vector<CaptureDevice>&&)>&& completion)
+ {
+ return adoptRef(*new CaptureDeviceAccumulator(WTFMove(completion)));
+ }
- return result;
+ ~CaptureDeviceAccumulator()
+ {
+ m_completionHandler(WTFMove(m_results));
+ }
+
+ CompletionHandler<void(Vector<CaptureDevice>&&)> accumulate()
+ {
+ return [this, protectedThis = makeRef(*this)] (Vector<CaptureDevice>&& result) {
+ m_results.appendVector(WTFMove(result));
+ };
+ }
+
+ private:
+ explicit CaptureDeviceAccumulator(CompletionHandler<void(Vector<CaptureDevice>&&)>&& completion)
+ : m_completionHandler(WTFMove(completion))
+ {
+ }
+
+ CompletionHandler<void(Vector<CaptureDevice>&&)> m_completionHandler;
+ Vector<CaptureDevice> m_results;
+ };
+
+ auto accumulator = CaptureDeviceAccumulator::create([completion = WTFMove(completion)] (auto&& devices) mutable {
+ devices.removeAllMatching([] (auto& captureDevice) {
+ return !captureDevice.enabled();
+ });
+ completion(WTFMove(devices));
+ });
+
+ audioCaptureFactory().audioCaptureDeviceManager().getCaptureDevices(accumulator->accumulate());
+ videoCaptureFactory().videoCaptureDeviceManager().getCaptureDevices(accumulator->accumulate());
+ displayCaptureFactory().displayCaptureDeviceManager().getCaptureDevices(accumulator->accumulate());
+ audioCaptureFactory().getSpeakerDevices(accumulator->accumulate());
}
static void addStringToSHA1(SHA1& sha1, const String& string)
Modified: branches/safari-611-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.h (271840 => 271841)
--- branches/safari-611-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.h 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.h 2021-01-25 22:12:26 UTC (rev 271841)
@@ -66,7 +66,7 @@
using NewMediaStreamHandler = Function<void(Expected<Ref<MediaStreamPrivate>, String>&&)>;
void createMediaStream(Ref<const Logger>&&, NewMediaStreamHandler&&, String&&, CaptureDevice&& audioDevice, CaptureDevice&& videoDevice, const MediaStreamRequest&);
- WEBCORE_EXPORT Vector<CaptureDevice> getMediaStreamDevices();
+ WEBCORE_EXPORT void getMediaStreamDevices(CompletionHandler<void(Vector<CaptureDevice>&&)>&&);
const RealtimeMediaSourceSupportedConstraints& supportedConstraints() { return m_supportedConstraints; }
Modified: branches/safari-611-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h (271840 => 271841)
--- branches/safari-611-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebCore/platform/mediastream/RealtimeMediaSourceFactory.h 2021-01-25 22:12:26 UTC (rev 271841)
@@ -27,6 +27,7 @@
#if ENABLE(MEDIA_STREAM)
+#include <wtf/CompletionHandler.h>
#include <wtf/WeakPtr.h>
#include <wtf/text/WTFString.h>
@@ -62,6 +63,7 @@
virtual CaptureSourceOrError createAudioCaptureSource(const CaptureDevice&, String&&, const MediaConstraints*) = 0;
virtual CaptureDeviceManager& audioCaptureDeviceManager() = 0;
virtual const Vector<CaptureDevice>& speakerDevices() const = 0;
+ virtual void getSpeakerDevices(CompletionHandler<void(Vector<CaptureDevice>&&)>&& completion) const { completion(copyToVector(speakerDevices())); }
class ExtensiveObserver : public CanMakeWeakPtr<ExtensiveObserver> { };
virtual void addExtensiveObserver(ExtensiveObserver&) { };
Modified: branches/safari-611-branch/Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h (271840 => 271841)
--- branches/safari-611-branch/Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h 2021-01-25 22:12:26 UTC (rev 271841)
@@ -48,6 +48,7 @@
static AVAudioSessionCaptureDeviceManager& singleton();
const Vector<CaptureDevice>& captureDevices() final;
+ void getCaptureDevices(CompletionHandler<void(Vector<CaptureDevice>&&)>&&) final;
const Vector<CaptureDevice>& speakerDevices() const { return m_speakerDevices; }
Optional<CaptureDevice> captureDeviceWithPersistentID(CaptureDevice::DeviceType, const String&);
@@ -63,8 +64,10 @@
~AVAudioSessionCaptureDeviceManager();
void createAudioSession();
+ void activateAudioSession();
void refreshAudioCaptureDevices();
- Vector<AVAudioSessionCaptureDevice>& audioSessionCaptureDevices();
+ Vector<AVAudioSessionCaptureDevice> retrieveAudioSessionCaptureDevices() const;
+ void setAudioCaptureDevices(Vector<AVAudioSessionCaptureDevice>&&);
enum class AudioSessionState { NotNeeded, Inactive, Active };
Modified: branches/safari-611-branch/Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm (271840 => 271841)
--- branches/safari-611-branch/Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebCore/platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm 2021-01-25 22:12:26 UTC (rev 271841)
@@ -136,16 +136,12 @@
return WTF::nullopt;
}
-Vector<AVAudioSessionCaptureDevice>& AVAudioSessionCaptureDeviceManager::audioSessionCaptureDevices()
+Optional<AVAudioSessionCaptureDevice> AVAudioSessionCaptureDeviceManager::audioSessionDeviceWithUID(const String& deviceID)
{
if (!m_audioSessionCaptureDevices)
refreshAudioCaptureDevices();
- return m_audioSessionCaptureDevices.value();
-}
-Optional<AVAudioSessionCaptureDevice> AVAudioSessionCaptureDeviceManager::audioSessionDeviceWithUID(const String& deviceID)
-{
- for (auto& device : audioSessionCaptureDevices()) {
+ for (auto& device : *m_audioSessionCaptureDevices) {
if (device.persistentId() == deviceID)
return device;
}
@@ -154,42 +150,71 @@
void AVAudioSessionCaptureDeviceManager::scheduleUpdateCaptureDevices()
{
- if (m_updateDeviceStateQueue.hasPendingTasks())
- return;
-
- m_updateDeviceStateQueue.enqueueTask([this] {
- refreshAudioCaptureDevices();
- });
+ getCaptureDevices([] (auto) { });
}
void AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices()
{
- {
- // Make sure we have created the audio session.
- auto locker = holdLock(m_lock);
+ if (m_audioSessionState == AudioSessionState::Inactive) {
+ m_audioSessionState = AudioSessionState::Active;
+
+ dispatch_sync(m_dispatchQueue, makeBlockPtr([this] {
+ activateAudioSession();
+ }).get());
}
- bool firstTime = !m_devices;
+
+ Vector<AVAudioSessionCaptureDevice> newAudioDevices;
+ dispatch_sync(m_dispatchQueue, makeBlockPtr([&] {
+ newAudioDevices = retrieveAudioSessionCaptureDevices();
+ }).get());
+ setAudioCaptureDevices(WTFMove(newAudioDevices));
+}
+
+void AVAudioSessionCaptureDeviceManager::getCaptureDevices(CompletionHandler<void(Vector<CaptureDevice>&&)>&& completion)
+{
if (m_audioSessionState == AudioSessionState::Inactive) {
m_audioSessionState = AudioSessionState::Active;
- if (!m_listener)
- m_listener = adoptNS([[WebAVAudioSessionAvailableInputsListener alloc] initWithCallback:this audioSession:m_audioSession.get()]);
-
- NSError *error = nil;
- [m_audioSession setActive:YES withOptions:0 error:&error];
- if (error)
- RELEASE_LOG_ERROR(WebRTC, "Failed to activate audio session with error: %@.", error.localizedDescription);
+ dispatch_async(m_dispatchQueue, makeBlockPtr([this] {
+ activateAudioSession();
+ }).get());
}
+ dispatch_async(m_dispatchQueue, makeBlockPtr([this, completion = WTFMove(completion)] () mutable {
+ auto newAudioDevices = retrieveAudioSessionCaptureDevices();
+ callOnWebThreadOrDispatchAsyncOnMainThread(makeBlockPtr([this, completion = WTFMove(completion), newAudioDevices = WTFMove(newAudioDevices)] () mutable {
+ setAudioCaptureDevices(WTFMove(newAudioDevices));
+ completion(copyToVector(*m_devices));
+ }).get());
+ }).get());
+}
+
+void AVAudioSessionCaptureDeviceManager::activateAudioSession()
+{
+ if (!m_listener)
+ m_listener = adoptNS([[WebAVAudioSessionAvailableInputsListener alloc] initWithCallback:this audioSession:m_audioSession.get()]);
+
+ NSError *error = nil;
+ [m_audioSession setActive:YES withOptions:0 error:&error];
+ if (error)
+ RELEASE_LOG_ERROR(WebRTC, "Failed to activate audio session with error: %@.", error.localizedDescription);
+}
+
+Vector<AVAudioSessionCaptureDevice> AVAudioSessionCaptureDeviceManager::retrieveAudioSessionCaptureDevices() const
+{
Vector<AVAudioSessionCaptureDevice> newAudioDevices;
- Vector<CaptureDevice> newDevices;
auto *defaultInput = [m_audioSession currentRoute].inputs.firstObject;
- for (AVAudioSessionPortDescription *portDescription in [m_audioSession availableInputs]) {
- auto audioDevice = AVAudioSessionCaptureDevice::create(portDescription, defaultInput);
- newDevices.append(audioDevice);
- newAudioDevices.append(WTFMove(audioDevice));
- }
+ newAudioDevices.reserveInitialCapacity([m_audioSession availableInputs].count);
+ for (AVAudioSessionPortDescription *portDescription in [m_audioSession availableInputs])
+ newAudioDevices.uncheckedAppend(AVAudioSessionCaptureDevice::create(portDescription, defaultInput));
+
+ return newAudioDevices;
+}
+
+void AVAudioSessionCaptureDeviceManager::setAudioCaptureDevices(Vector<AVAudioSessionCaptureDevice>&& newAudioDevices)
+{
+ bool firstTime = !m_devices;
bool haveDeviceChanges = !m_devices || newAudioDevices.size() != m_devices->size();
if (!haveDeviceChanges) {
for (size_t i = 0; i < newAudioDevices.size(); ++i) {
@@ -203,6 +228,7 @@
if (!haveDeviceChanges && !firstTime)
return;
+ auto newDevices = copyToVectorOf<CaptureDevice>(newAudioDevices);
m_audioSessionCaptureDevices = WTFMove(newAudioDevices);
std::sort(newDevices.begin(), newDevices.end(), [] (auto& first, auto& second) -> bool {
return first.isDefault() && !second.isDefault();
Modified: branches/safari-611-branch/Source/WebKit/ChangeLog (271840 => 271841)
--- branches/safari-611-branch/Source/WebKit/ChangeLog 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebKit/ChangeLog 2021-01-25 22:12:26 UTC (rev 271841)
@@ -1,5 +1,92 @@
2021-01-25 Alan Coon <[email protected]>
+ Cherry-pick r271471. rdar://problem/73477068
+
+ Source/WebCore:
+ [HANG] 496ms to 1360ms in WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices()
+ https://bugs.webkit.org/show_bug.cgi?id=220471
+
+ Reviewed by Youenn Fablet.
+
+ Refactor RealtimeMediaSourceCenter::getMediaStreamDevices() to take a completion handler, rather than
+ synchronously return a Vector of CaptureDevices. This also requires all CaptureDeviceManager subclasses
+ to support taking a completion handler themselves. By default, all CaptureDeviceManagers will support
+ the CompletionHandler path by just synchronously calling the completion handler with the existing
+ synchronous method. But for AVAudioSessionCaptureDeviceManager, override that default implementation by
+ activating the AVAudioSession on a background thread, and querying that session's inputs on a background
+ thread as well.
+
+ * platform/mediastream/CaptureDeviceManager.h:
+ * platform/mediastream/RealtimeMediaSourceCenter.cpp:
+ (WebCore::RealtimeMediaSourceCenter::getMediaStreamDevices):
+ * platform/mediastream/RealtimeMediaSourceCenter.h:
+ * platform/mediastream/RealtimeMediaSourceFactory.h:
+ (WebCore::AudioCaptureFactory::getSpeakerDevices const):
+ * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.h:
+ * platform/mediastream/ios/AVAudioSessionCaptureDeviceManager.mm:
+ (WebCore::AVAudioSessionCaptureDeviceManager::audioSessionDeviceWithUID):
+ (WebCore::AVAudioSessionCaptureDeviceManager::scheduleUpdateCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::getCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::activateAudioSession):
+ (WebCore::AVAudioSessionCaptureDeviceManager::retrieveAudioSessionCaptureDevices const):
+ (WebCore::AVAudioSessionCaptureDeviceManager::setAudioCaptureDevices):
+ (WebCore::AVAudioSessionCaptureDeviceManager::audioSessionCaptureDevices): Deleted.
+
+ Source/WebKit:
+ [HANG] 496ms to 1360ms in WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices()
+ https://bugs.webkit.org/show_bug.cgi?id=220471
+
+ Reviewed by Youenn Fablet.
+
+ Use the completion-handler version of RealtimeMediaSourceCenter::getMediaStreamDevices().
+
+ * UIProcess/UserMediaPermissionRequestManagerProxy.cpp:
+ (WebKit::UserMediaPermissionRequestManagerProxy::computeFilteredDeviceList):
+ (WebKit::UserMediaPermissionRequestManagerProxy::enumerateMediaDevicesForFrame):
+ * UIProcess/UserMediaPermissionRequestManagerProxy.h:
+ * UIProcess/UserMediaProcessManager.cpp:
+ (WebKit::UserMediaProcessManager::updateCaptureDevices):
+ (WebKit::UserMediaProcessManager::beginMonitoringCaptureDevices):
+ * UIProcess/UserMediaProcessManager.h:
+
+ LayoutTests:
+ [Cocoa] Support key rotation with HLS-backed encrypted media streams
+ https://bugs.webkit.org/show_bug.cgi?id=220493
+ <rdar://68227709>
+
+ Reviewed by Youenn Fablet.
+
+ Fix a broken layout test; the test enumerates devices, then uses the deviceIds returned to generate
+ constraints for a call to getUserMedia(). However, it assumes all devices will either be of kind
+ 'audioinput' or 'videoinput'. If an 'audiooutput' device is returned (as the MockRealtimeMediaSourceCenter
+ does), then the test turns that into a video capture constraint, which fails.
+
+ * fast/mediastream/get-user-media-device-id.html:
+
+
+ git-svn-id: https://svn.webkit.org/repository/webkit/trunk@271471 268f45cc-cd09-0410-ab3c-d52691b4dbfc
+
+ 2021-01-13 Jer Noble <[email protected]>
+
+ [HANG] 496ms to 1360ms in WebCore::AVAudioSessionCaptureDeviceManager::refreshAudioCaptureDevices()
+ https://bugs.webkit.org/show_bug.cgi?id=220471
+
+ Reviewed by Youenn Fablet.
+
+ Use the completion-handler version of RealtimeMediaSourceCenter::getMediaStreamDevices().
+
+ * UIProcess/UserMediaPermissionRequestManagerProxy.cpp:
+ (WebKit::UserMediaPermissionRequestManagerProxy::computeFilteredDeviceList):
+ (WebKit::UserMediaPermissionRequestManagerProxy::enumerateMediaDevicesForFrame):
+ * UIProcess/UserMediaPermissionRequestManagerProxy.h:
+ * UIProcess/UserMediaProcessManager.cpp:
+ (WebKit::UserMediaProcessManager::updateCaptureDevices):
+ (WebKit::UserMediaProcessManager::beginMonitoringCaptureDevices):
+ * UIProcess/UserMediaProcessManager.h:
+
+2021-01-25 Alan Coon <[email protected]>
+
Cherry-pick r271467. rdar://problem/73477045
Crash at SOAuthorizationSession::dismissViewController
Modified: branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp (271840 => 271841)
--- branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp 2021-01-25 22:12:26 UTC (rev 271841)
@@ -654,40 +654,42 @@
});
}
-Vector<CaptureDevice> UserMediaPermissionRequestManagerProxy::computeFilteredDeviceList(bool revealIdsAndLabels)
+void UserMediaPermissionRequestManagerProxy::computeFilteredDeviceList(bool revealIdsAndLabels, CompletionHandler<void(Vector<CaptureDevice>&&)>&& completion)
{
- static const int defaultMaximumCameraCount = 1;
- static const int defaultMaximumMicrophoneCount = 1;
+ static const unsigned defaultMaximumCameraCount = 1;
+ static const unsigned defaultMaximumMicrophoneCount = 1;
- auto devices = RealtimeMediaSourceCenter::singleton().getMediaStreamDevices();
- int cameraCount = 0;
- int microphoneCount = 0;
+ RealtimeMediaSourceCenter::singleton().getMediaStreamDevices([this, weakThis = makeWeakPtr(this), revealIdsAndLabels, completion = WTFMove(completion)] (auto&& devices) mutable {
+ unsigned cameraCount = 0;
+ unsigned microphoneCount = 0;
- Vector<CaptureDevice> filteredDevices;
- for (const auto& device : devices) {
- if (!device.enabled() || (device.type() != WebCore::CaptureDevice::DeviceType::Camera && device.type() != WebCore::CaptureDevice::DeviceType::Microphone && device.type() != WebCore::CaptureDevice::DeviceType::Speaker))
- continue;
+ Vector<CaptureDevice> filteredDevices;
+ for (const auto& device : devices) {
+ if (!device.enabled() || (device.type() != WebCore::CaptureDevice::DeviceType::Camera && device.type() != WebCore::CaptureDevice::DeviceType::Microphone && device.type() != WebCore::CaptureDevice::DeviceType::Speaker))
+ continue;
- if (!revealIdsAndLabels) {
- if (device.type() == WebCore::CaptureDevice::DeviceType::Camera && ++cameraCount > defaultMaximumCameraCount)
- continue;
- if (device.type() == WebCore::CaptureDevice::DeviceType::Microphone && ++microphoneCount > defaultMaximumMicrophoneCount)
- continue;
- if (device.type() != WebCore::CaptureDevice::DeviceType::Camera && device.type() != WebCore::CaptureDevice::DeviceType::Microphone)
- continue;
- } else {
- // We only expose speakers tied to a microphone for the moment.
- if (device.type() == WebCore::CaptureDevice::DeviceType::Speaker && !haveMicrophoneDevice(devices, device.groupId()))
- continue;
+ if (!revealIdsAndLabels) {
+ if (device.type() == WebCore::CaptureDevice::DeviceType::Camera && ++cameraCount > defaultMaximumCameraCount)
+ continue;
+ if (device.type() == WebCore::CaptureDevice::DeviceType::Microphone && ++microphoneCount > defaultMaximumMicrophoneCount)
+ continue;
+ if (device.type() != WebCore::CaptureDevice::DeviceType::Camera && device.type() != WebCore::CaptureDevice::DeviceType::Microphone)
+ continue;
+ } else {
+ // We only expose speakers tied to a microphone for the moment.
+ if (device.type() == WebCore::CaptureDevice::DeviceType::Speaker && !haveMicrophoneDevice(devices, device.groupId()))
+ continue;
+ }
+
+ filteredDevices.append(revealIdsAndLabels ? device : CaptureDevice({ }, device.type(), { }, { }));
}
- filteredDevices.append(revealIdsAndLabels ? device : CaptureDevice({ }, device.type(), { }, { }));
- }
+ if (weakThis)
+ m_hasFilteredDeviceList = !revealIdsAndLabels;
- m_hasFilteredDeviceList = !revealIdsAndLabels;
-
- ALWAYS_LOG(LOGIDENTIFIER, filteredDevices.size(), " devices revealed");
- return filteredDevices;
+ ALWAYS_LOG(LOGIDENTIFIER, filteredDevices.size(), " devices revealed");
+ completion(WTFMove(filteredDevices));
+ });
}
#endif
@@ -739,7 +741,9 @@
bool revealIdsAndLabels = originHasPersistentAccess || wasGrantedVideoOrAudioAccess(frameID, userMediaDocumentOrigin.get(), topLevelDocumentOrigin.get());
callCompletionHandler.release();
- completionHandler(computeFilteredDeviceList(revealIdsAndLabels), deviceIDHashSalt);
+ computeFilteredDeviceList(revealIdsAndLabels, [completionHandler = WTFMove(completionHandler), deviceIDHashSalt = WTFMove(deviceIDHashSalt)] (auto&& devices) mutable {
+ completionHandler(devices, deviceIDHashSalt);
+ });
});
};
Modified: branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.h (271840 => 271841)
--- branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.h 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.h 2021-01-25 22:12:26 UTC (rev 271841)
@@ -119,7 +119,7 @@
bool wasGrantedVideoOrAudioAccess(WebCore::FrameIdentifier, const WebCore::SecurityOrigin& userMediaDocumentOrigin, const WebCore::SecurityOrigin& topLevelDocumentOrigin);
- Vector<WebCore::CaptureDevice> computeFilteredDeviceList(bool revealIdsAndLabels);
+ void computeFilteredDeviceList(bool revealIdsAndLabels, CompletionHandler<void(Vector<WebCore::CaptureDevice>&&)>&&);
void processUserMediaPermissionRequest();
void processUserMediaPermissionInvalidRequest(const String& invalidConstraint);
Modified: branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaProcessManager.cpp (271840 => 271841)
--- branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaProcessManager.cpp 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaProcessManager.cpp 2021-01-25 22:12:26 UTC (rev 271841)
@@ -217,41 +217,51 @@
});
}
-void UserMediaProcessManager::beginMonitoringCaptureDevices()
+void UserMediaProcessManager::updateCaptureDevices(ShouldNotify shouldNotify)
{
- static std::once_flag onceFlag;
+ WebCore::RealtimeMediaSourceCenter::singleton().getMediaStreamDevices([this, shouldNotify] (auto&& newDevices) mutable {
+ auto oldDevices = WTFMove(m_captureDevices);
+ m_captureDevices = WTFMove(newDevices);
- std::call_once(onceFlag, [this] {
- m_captureDevices = WebCore::RealtimeMediaSourceCenter::singleton().getMediaStreamDevices();
+ if (shouldNotify == ShouldNotify::No)
+ return;
- WebCore::RealtimeMediaSourceCenter::singleton().setDevicesChangedObserver([this]() {
- auto oldDevices = WTFMove(m_captureDevices);
- m_captureDevices = WebCore::RealtimeMediaSourceCenter::singleton().getMediaStreamDevices();
+ if (m_captureDevices.size() == oldDevices.size()) {
+ bool haveChanges = false;
+ for (auto &newDevice : m_captureDevices) {
+ if (newDevice.type() != WebCore::CaptureDevice::DeviceType::Camera && newDevice.type() != WebCore::CaptureDevice::DeviceType::Microphone)
+ continue;
- if (m_captureDevices.size() == oldDevices.size()) {
- bool haveChanges = false;
- for (auto &newDevice : m_captureDevices) {
- if (newDevice.type() != WebCore::CaptureDevice::DeviceType::Camera && newDevice.type() != WebCore::CaptureDevice::DeviceType::Microphone)
- continue;
+ auto index = oldDevices.findMatching([&newDevice] (auto& oldDevice) {
+ return newDevice.persistentId() == oldDevice.persistentId() && newDevice.enabled() != oldDevice.enabled();
+ });
- auto index = oldDevices.findMatching([&newDevice] (auto& oldDevice) {
- return newDevice.persistentId() == oldDevice.persistentId() && newDevice.enabled() != oldDevice.enabled();
- });
-
- if (index == notFound) {
- haveChanges = true;
- break;
- }
+ if (index == notFound) {
+ haveChanges = true;
+ break;
}
-
- if (!haveChanges)
- return;
}
- // When a device with camera and microphone is attached or detached, the CaptureDevice notification for
- // the different devices won't arrive at the same time so delay a bit so we can coalesce the callbacks.
- if (!m_debounceTimer.isActive())
- m_debounceTimer.startOneShot(deviceChangeDebounceTimerInterval);
+ if (!haveChanges)
+ return;
+ }
+
+ // When a device with camera and microphone is attached or detached, the CaptureDevice notification for
+ // the different devices won't arrive at the same time so delay a bit so we can coalesce the callbacks.
+ if (!m_debounceTimer.isActive())
+ m_debounceTimer.startOneShot(deviceChangeDebounceTimerInterval);
+ });
+}
+
+void UserMediaProcessManager::beginMonitoringCaptureDevices()
+{
+ static std::once_flag onceFlag;
+
+ std::call_once(onceFlag, [this] {
+ updateCaptureDevices(ShouldNotify::No);
+
+ WebCore::RealtimeMediaSourceCenter::singleton().setDevicesChangedObserver([this]() {
+ updateCaptureDevices(ShouldNotify::Yes);
});
});
}
Modified: branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaProcessManager.h (271840 => 271841)
--- branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaProcessManager.h 2021-01-25 22:12:20 UTC (rev 271840)
+++ branches/safari-611-branch/Source/WebKit/UIProcess/UserMediaProcessManager.h 2021-01-25 22:12:26 UTC (rev 271841)
@@ -48,6 +48,8 @@
void beginMonitoringCaptureDevices();
private:
+ enum class ShouldNotify { Yes, No };
+ void updateCaptureDevices(ShouldNotify);
void captureDevicesChanged();
Vector<WebCore::CaptureDevice> m_captureDevices;
