- Revision
- 209010
- Author
- commit-qu...@webkit.org
- Date
- 2016-11-28 12:46:44 -0800 (Mon, 28 Nov 2016)
Log Message
Support for HTML Media Capture API
https://bugs.webkit.org/show_bug.cgi?id=43239
Patch by Andrew Gold <ag...@apple.com> on 2016-11-28
Reviewed by Tim Horton.
In order to enable media capture on iOS, we must first use the AVFoundation API to
check/request capture permission from the user for Safari. Then, Safari must request
permission on behalf of the webpage to use the capture devices. Additionally, Safari
must present UI when the capture is taking place, so WebKit needs to notify the client
of a capture session beginning and ending. To do this, we added four methods to
WKUIDelegatePrivate to request permission from the user, check for permissions, notify
that a capture session has begun, and notify that a capture session has ended. Additionally,
we added a private method to WKWebView that allows the client to stop a capture session.
* UIProcess/API/APIUIClient.h:
(API::UIClient::didBeginCaptureSession): Notifies the client of a capture session beginning.
(API::UIClient::didEndCaptureSession): Notifies the client of a capture session ending.
* UIProcess/API/Cocoa/WKUIDelegatePrivate.h: Added new delegate methods to request permission,
check for permission, notify of a capture session beginning, and notify of a capture session
ending.
* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _stopMediaCapture]): Cancels a media capture session.
* UIProcess/API/Cocoa/WKWebViewPrivate.h:
* UIProcess/Cocoa/UIDelegate.h:
* UIProcess/Cocoa/UIDelegate.mm:
(WebKit::UIDelegate::setDelegate): Added the new delegate methods.
(WebKit::UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest):
First checks if the user has authorized the client application access capture devices,
then uses the WKUIDelegate to request permission for a site from the user.
(WebKit::UIDelegate::UIClient::checkUserMediaPermissionForOrigin): Checks the client
for permission to access capture devices.
(WebKit::UIDelegate::UIClient::didBeginCaptureSession): Notifies the client of a capture
session beginning.
(WebKit::UIDelegate::UIClient::didEndCaptureSession): Notifies the client of a capture
session ending.
* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::isPlayingMediaDidChange): Calls UIDelegate method to notify the
client of a capture session beginning/ending.
Modified Paths
Diff
Modified: trunk/Source/WebKit2/ChangeLog (209009 => 209010)
--- trunk/Source/WebKit2/ChangeLog 2016-11-28 20:45:27 UTC (rev 209009)
+++ trunk/Source/WebKit2/ChangeLog 2016-11-28 20:46:44 UTC (rev 209010)
@@ -1,3 +1,48 @@
+2016-11-28 Andrew Gold <ag...@apple.com>
+
+ Support for HTML Media Capture API
+ https://bugs.webkit.org/show_bug.cgi?id=43239
+
+ Reviewed by Tim Horton.
+
+ In order to enable media capture on iOS, we must first use the AVFoundation API to
+ check/request capture permission from the user for Safari. Then, Safari must request
+ permission on behalf of the webpage to use the capture devices. Additionally, Safari
+ must present UI when the capture is taking place, so WebKit needs to notify the client
+ of a capture session beginning and ending. To do this, we added four methods to
+ WKUIDelegatePrivate to request permission from the user, check for permissions, notify
+ that a capture session has begun, and notify that a capture session has ended. Additionally,
+ we added a private method to WKWebView that allows the client to stop a capture session.
+
+ * UIProcess/API/APIUIClient.h:
+ (API::UIClient::didBeginCaptureSession): Notifies the client of a capture session beginning.
+ (API::UIClient::didEndCaptureSession): Notifies the client of a capture session ending.
+
+ * UIProcess/API/Cocoa/WKUIDelegatePrivate.h: Added new delegate methods to request permission,
+ check for permission, notify of a capture session beginning, and notify of a capture session
+ ending.
+
+ * UIProcess/API/Cocoa/WKWebView.mm:
+ (-[WKWebView _stopMediaCapture]): Cancels a media capture session.
+ * UIProcess/API/Cocoa/WKWebViewPrivate.h:
+
+ * UIProcess/Cocoa/UIDelegate.h:
+ * UIProcess/Cocoa/UIDelegate.mm:
+ (WebKit::UIDelegate::setDelegate): Added the new delegate methods.
+ (WebKit::UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest):
+ First checks if the user has authorized the client application access capture devices,
+ then uses the WKUIDelegate to request permission for a site from the user.
+ (WebKit::UIDelegate::UIClient::checkUserMediaPermissionForOrigin): Checks the client
+ for permission to access capture devices.
+ (WebKit::UIDelegate::UIClient::didBeginCaptureSession): Notifies the client of a capture
+ session beginning.
+ (WebKit::UIDelegate::UIClient::didEndCaptureSession): Notifies the client of a capture
+ session ending.
+
+ * UIProcess/WebPageProxy.cpp:
+ (WebKit::WebPageProxy::isPlayingMediaDidChange): Calls UIDelegate method to notify the
+ client of a capture session beginning/ending.
+
2016-11-28 Eric Carlson <eric.carl...@apple.com>
[MediaStream] Don't request user permission for a device if it has already been granted in the current browsing context
Modified: trunk/Source/WebKit2/UIProcess/API/APIUIClient.h (209009 => 209010)
--- trunk/Source/WebKit2/UIProcess/API/APIUIClient.h 2016-11-28 20:45:27 UTC (rev 209009)
+++ trunk/Source/WebKit2/UIProcess/API/APIUIClient.h 2016-11-28 20:46:44 UTC (rev 209010)
@@ -150,6 +150,8 @@
virtual void pinnedStateDidChange(WebKit::WebPageProxy&) { }
virtual void isPlayingAudioDidChange(WebKit::WebPageProxy&) { }
+ virtual void didBeginCaptureSession() { }
+ virtual void didEndCaptureSession() { }
#if ENABLE(MEDIA_SESSION)
virtual void mediaSessionMetadataDidChange(WebKit::WebPageProxy&, WebKit::WebMediaSessionMetadata*) { }
Modified: trunk/Source/WebKit2/UIProcess/API/Cocoa/WKUIDelegatePrivate.h (209009 => 209010)
--- trunk/Source/WebKit2/UIProcess/API/Cocoa/WKUIDelegatePrivate.h 2016-11-28 20:45:27 UTC (rev 209009)
+++ trunk/Source/WebKit2/UIProcess/API/Cocoa/WKUIDelegatePrivate.h 2016-11-28 20:46:44 UTC (rev 209010)
@@ -60,6 +60,10 @@
- (void)_webView:(WKWebView *)webView imageOrMediaDocumentSizeChanged:(CGSize)size WK_API_AVAILABLE(macosx(10.12), ios(10.0));
- (NSDictionary *)_dataDetectionContextForWebView:(WKWebView *)webView WK_API_AVAILABLE(macosx(10.12), ios(10.0));
+- (void)_webView:(WKWebView *)webView requestUserMediaAuthorizationForMicrophone:(BOOL)microphone camera:(BOOL)camera url:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL decisionHandler:(void (^)(BOOL authorizedMicrophone, BOOL authorizedCamera))decisionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_webView:(WKWebView *)webView checkUserMediaPermissionForURL:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL frameIdentifier:(NSUInteger)frameIdentifier decisionHandler:(void (^)(NSString *salt, BOOL authorized))decisionHandler WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_webViewDidBeginCaptureSession:(WKWebView *)webView WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
+- (void)_webViewDidEndCaptureSession:(WKWebView *)webView WK_API_AVAILABLE(macosx(WK_MAC_TBA), ios(WK_IOS_TBA));
#if TARGET_OS_IPHONE
- (BOOL)_webView:(WKWebView *)webView shouldIncludeAppLinkActionsForElement:(_WKActivatedElementInfo *)element WK_API_AVAILABLE(ios(9.0));
- (NSArray *)_webView:(WKWebView *)webView actionsForElement:(_WKActivatedElementInfo *)element defaultActions:(NSArray<_WKElementAction *> *)defaultActions;
Modified: trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebView.mm (209009 => 209010)
--- trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebView.mm 2016-11-28 20:45:27 UTC (rev 209009)
+++ trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebView.mm 2016-11-28 20:46:44 UTC (rev 209010)
@@ -4040,6 +4040,11 @@
#endif
}
+- (void)_stopMediaCapture
+{
+ _page->setMuted(WebCore::MediaProducer::CaptureDevicesAreMuted);
+}
+
#pragma mark iOS-specific methods
#if PLATFORM(IOS)
Modified: trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebViewPrivate.h (209009 => 209010)
--- trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebViewPrivate.h 2016-11-28 20:45:27 UTC (rev 209009)
+++ trunk/Source/WebKit2/UIProcess/API/Cocoa/WKWebViewPrivate.h 2016-11-28 20:46:44 UTC (rev 209010)
@@ -258,6 +258,8 @@
@property (nonatomic, setter=_setFullscreenDelegate:) id<_WKFullscreenDelegate> _fullscreenDelegate WK_API_AVAILABLE(macosx(10.13));
@property (nonatomic, readonly) BOOL _isInFullscreen WK_API_AVAILABLE(macosx(WK_MAC_TBA));
+- (void)_stopMediaCapture;
+
@end
#if !TARGET_OS_IPHONE
Modified: trunk/Source/WebKit2/UIProcess/Cocoa/UIDelegate.h (209009 => 209010)
--- trunk/Source/WebKit2/UIProcess/Cocoa/UIDelegate.h 2016-11-28 20:45:27 UTC (rev 209009)
+++ trunk/Source/WebKit2/UIProcess/Cocoa/UIDelegate.h 2016-11-28 20:46:44 UTC (rev 209010)
@@ -89,6 +89,10 @@
#if PLATFORM(MAC)
bool runOpenPanel(WebPageProxy*, WebFrameProxy*, const WebCore::SecurityOriginData&, API::OpenPanelParameters*, WebOpenPanelResultListenerProxy*) override;
#endif
+ bool decidePolicyForUserMediaPermissionRequest(WebKit::WebPageProxy&, WebKit::WebFrameProxy&, API::SecurityOrigin&, API::SecurityOrigin&, WebKit::UserMediaPermissionRequestProxy&) override;
+ bool checkUserMediaPermissionForOrigin(WebKit::WebPageProxy&, WebKit::WebFrameProxy&, API::SecurityOrigin&, API::SecurityOrigin&, WebKit::UserMediaPermissionCheckProxy&) override;
+ void didBeginCaptureSession() override;
+ void didEndCaptureSession() override;
void printFrame(WebKit::WebPageProxy*, WebKit::WebFrameProxy*) override;
#if PLATFORM(IOS)
#if HAVE(APP_LINKS)
@@ -130,6 +134,10 @@
bool webViewFullscreenMayReturnToInline : 1;
bool webViewDidEnterFullscreen : 1;
bool webViewDidExitFullscreen : 1;
+ bool webViewRequestUserMediaAuthorizationForMicrophoneCameraURLMainFrameURLDecisionHandler : 1;
+ bool webViewCheckUserMediaPermissionForURLMainFrameURLFrameIdentifierDecisionHandler : 1;
+ bool webViewDidBeginCaptureSession : 1;
+ bool webViewDidEndCaptureSession : 1;
#if PLATFORM(IOS)
#if HAVE(APP_LINKS)
bool webViewShouldIncludeAppLinkActionsForElement : 1;
Modified: trunk/Source/WebKit2/UIProcess/Cocoa/UIDelegate.mm (209009 => 209010)
--- trunk/Source/WebKit2/UIProcess/Cocoa/UIDelegate.mm 2016-11-28 20:45:27 UTC (rev 209009)
+++ trunk/Source/WebKit2/UIProcess/Cocoa/UIDelegate.mm 2016-11-28 20:46:44 UTC (rev 209010)
@@ -30,6 +30,8 @@
#import "CompletionHandlerCallChecker.h"
#import "NavigationActionData.h"
+#import "UserMediaPermissionCheckProxy.h"
+#import "UserMediaPermissionRequestProxy.h"
#import "WKFrameInfoInternal.h"
#import "WKNavigationActionInternal.h"
#import "WKOpenPanelParametersInternal.h"
@@ -46,6 +48,17 @@
#import <WebCore/URL.h>
#import <wtf/BlockPtr.h>
+#if PLATFORM(IOS)
+#import <AVFoundation/AVCaptureDevice.h>
+#import <AVFoundation/AVMediaFormat.h>
+#import <WebCore/SoftLinking.h>
+
+SOFT_LINK_FRAMEWORK(AVFoundation);
+SOFT_LINK_CLASS(AVFoundation, AVCaptureDevice);
+SOFT_LINK_CONSTANT(AVFoundation, AVMediaTypeAudio, NSString *);
+SOFT_LINK_CONSTANT(AVFoundation, AVMediaTypeVideo, NSString *);
+#endif
+
namespace WebKit {
UIDelegate::UIDelegate(WKWebView *webView)
@@ -101,6 +114,10 @@
#endif
m_delegateMethods.webViewActionsForElementDefaultActions = [delegate respondsToSelector:@selector(_webView:actionsForElement:defaultActions:)];
m_delegateMethods.webViewDidNotHandleTapAsClickAtPoint = [delegate respondsToSelector:@selector(_webView:didNotHandleTapAsClickAtPoint:)];
+ m_delegateMethods.webViewRequestUserMediaAuthorizationForMicrophoneCameraURLMainFrameURLDecisionHandler = [delegate respondsToSelector:@selector(_webView:requestUserMediaAuthorizationForMicrophone:camera:url:mainFrameURL:decisionHandler:)];
+ m_delegateMethods.webViewCheckUserMediaPermissionForURLMainFrameURLFrameIdentifierDecisionHandler = [delegate respondsToSelector:@selector(_webView:checkUserMediaPermissionForURL:mainFrameURL:frameIdentifier:decisionHandler:)];
+ m_delegateMethods.webViewDidBeginCaptureSession = [delegate respondsToSelector:@selector(_webViewDidBeginCaptureSession:)];
+ m_delegateMethods.webViewDidEndCaptureSession = [delegate respondsToSelector:@selector(_webViewDidEndCaptureSession:)];
m_delegateMethods.presentingViewControllerForWebView = [delegate respondsToSelector:@selector(_presentingViewControllerForWebView:)];
#endif
m_delegateMethods.dataDetectionContextForWebView = [delegate respondsToSelector:@selector(_dataDetectionContextForWebView:)];
@@ -303,6 +320,129 @@
}
#endif
+bool UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest(WebKit::WebPageProxy& page, WebKit::WebFrameProxy& frame, API::SecurityOrigin& userMediaOrigin, API::SecurityOrigin& topLevelOrigin, WebKit::UserMediaPermissionRequestProxy& request)
+{
+ auto delegate = m_uiDelegate.m_delegate.get();
+ if (!delegate || !m_uiDelegate.m_delegateMethods.webViewRequestUserMediaAuthorizationForMicrophoneCameraURLMainFrameURLDecisionHandler) {
+ request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::UserMediaDisabled);
+ return true;
+ }
+
+ bool requiresAudio = request.requiresAudio();
+ bool requiresVideo = request.requiresVideo();
+ if (!requiresAudio && !requiresVideo) {
+ request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::NoConstraints);
+ return true;
+ }
+
+ __block WKWebView *webView = m_uiDelegate.m_webView;
+ void (^uiDelegateAuthorizationBlock)(void) = ^ {
+ const WebFrameProxy* mainFrame = frame.page()->mainFrame();
+ WebCore::URL requestFrameURL(WebCore::URL(), frame.url());
+ WebCore::URL mainFrameURL(WebCore::URL(), mainFrame->url());
+
+ [(id <WKUIDelegatePrivate>)delegate _webView:webView requestUserMediaAuthorizationForMicrophone:requiresAudio camera:requiresVideo url:requestFrameURL mainFrameURL:mainFrameURL decisionHandler:^(BOOL authorizedMicrophone, BOOL authorizedCamera) {
+ if ((requiresAudio != authorizedMicrophone) || (requiresVideo != authorizedCamera)) {
+ request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
+ return;
+ }
+ const String& videoDeviceUID = requiresVideo ? request.videoDeviceUIDs().first() : String();
+ const String& audioDeviceUID = requiresAudio ? request.audioDeviceUIDs().first() : String();
+ request.allow(audioDeviceUID, videoDeviceUID);
+ }];
+ };
+
+#if PLATFORM(IOS)
+ void (^cameraAuthorizationBlock)(void) = ^ {
+ if (requiresVideo) {
+ AVAuthorizationStatus cameraAuthorizationStatus = [getAVCaptureDeviceClass() authorizationStatusForMediaType:getAVMediaTypeVideo()];
+ switch (cameraAuthorizationStatus) {
+ case AVAuthorizationStatusDenied:
+ case AVAuthorizationStatusRestricted:
+ request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
+ return;
+ case AVAuthorizationStatusNotDetermined:
+ [getAVCaptureDeviceClass() requestAccessForMediaType:getAVMediaTypeVideo() completionHandler:^(BOOL authorized) {
+ if (!authorized) {
+ request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
+ return;
+ }
+ uiDelegateAuthorizationBlock();
+ }];
+ break;
+ default:
+ uiDelegateAuthorizationBlock();
+ }
+ } else
+ uiDelegateAuthorizationBlock();
+ };
+
+ if (requiresAudio) {
+ AVAuthorizationStatus microphoneAuthorizationStatus = [getAVCaptureDeviceClass() authorizationStatusForMediaType:getAVMediaTypeAudio()];
+ switch (microphoneAuthorizationStatus) {
+ case AVAuthorizationStatusDenied:
+ case AVAuthorizationStatusRestricted:
+ request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
+ return true;
+ case AVAuthorizationStatusNotDetermined:
+ [getAVCaptureDeviceClass() requestAccessForMediaType:getAVMediaTypeAudio() completionHandler:^(BOOL authorized) {
+ if (!authorized) {
+ request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
+ return;
+ }
+ cameraAuthorizationBlock();
+ }];
+ break;
+ default:
+ cameraAuthorizationBlock();
+ }
+ } else
+ cameraAuthorizationBlock();
+#else
+ uiDelegateAuthorizationBlock();
+#endif
+
+ return true;
+}
+
+bool UIDelegate::UIClient::checkUserMediaPermissionForOrigin(WebKit::WebPageProxy& page, WebKit::WebFrameProxy& frame, API::SecurityOrigin& userMediaOrigin, API::SecurityOrigin& topLevelOrigin, WebKit::UserMediaPermissionCheckProxy& request)
+{
+ auto delegate = m_uiDelegate.m_delegate.get();
+ if (!delegate || !m_uiDelegate.m_delegateMethods.webViewCheckUserMediaPermissionForURLMainFrameURLFrameIdentifierDecisionHandler) {
+ request.setUserMediaAccessInfo(String(), false);
+ return true;
+ }
+
+ WKWebView *webView = m_uiDelegate.m_webView;
+ const WebFrameProxy* mainFrame = frame.page()->mainFrame();
+ WebCore::URL requestFrameURL(WebCore::URL(), frame.url());
+ WebCore::URL mainFrameURL(WebCore::URL(), mainFrame->url());
+
+ [(id <WKUIDelegatePrivate>)delegate _webView:webView checkUserMediaPermissionForURL:requestFrameURL mainFrameURL:mainFrameURL frameIdentifier:frame.frameID() decisionHandler:^(NSString *salt, BOOL authorized) {
+ request.setUserMediaAccessInfo(String(salt), authorized);
+ }];
+
+ return true;
+}
+
+void UIDelegate::UIClient::didBeginCaptureSession()
+{
+ auto delegate = m_uiDelegate.m_delegate.get();
+ if (!delegate || !m_uiDelegate.m_delegateMethods.webViewDidBeginCaptureSession)
+ return;
+
+ [(id <WKUIDelegatePrivate>)delegate _webViewDidBeginCaptureSession:m_uiDelegate.m_webView];
+}
+
+void UIDelegate::UIClient::didEndCaptureSession()
+{
+ auto delegate = m_uiDelegate.m_delegate.get();
+ if (!delegate || !m_uiDelegate.m_delegateMethods.webViewDidEndCaptureSession)
+ return;
+
+ [(id <WKUIDelegatePrivate>)delegate _webViewDidEndCaptureSession:m_uiDelegate.m_webView];
+}
+
void UIDelegate::UIClient::reachedApplicationCacheOriginQuota(WebPageProxy*, const WebCore::SecurityOrigin& securityOrigin, uint64_t currentQuota, uint64_t totalBytesNeeded, Function<void (unsigned long long)>&& completionHandler)
{
if (!m_uiDelegate.m_delegateMethods.webViewDecideWebApplicationCacheQuotaForSecurityOriginCurrentQuotaTotalBytesNeeded) {
Modified: trunk/Source/WebKit2/UIProcess/WebPageProxy.cpp (209009 => 209010)
--- trunk/Source/WebKit2/UIProcess/WebPageProxy.cpp 2016-11-28 20:45:27 UTC (rev 209009)
+++ trunk/Source/WebKit2/UIProcess/WebPageProxy.cpp 2016-11-28 20:46:44 UTC (rev 209010)
@@ -6409,6 +6409,13 @@
if (state == m_mediaState)
return;
+ WebCore::MediaProducer::MediaStateFlags oldMediaStateHasActiveCapture = m_mediaState & (WebCore::MediaProducer::HasActiveAudioCaptureDevice | WebCore::MediaProducer::HasActiveVideoCaptureDevice);
+ WebCore::MediaProducer::MediaStateFlags newMediaStateHasActiveCapture = state & (WebCore::MediaProducer::HasActiveAudioCaptureDevice | WebCore::MediaProducer::HasActiveVideoCaptureDevice);
+ if (!oldMediaStateHasActiveCapture && newMediaStateHasActiveCapture)
+ m_uiClient->didBeginCaptureSession();
+ if (oldMediaStateHasActiveCapture && !newMediaStateHasActiveCapture)
+ m_uiClient->didEndCaptureSession();
+
MediaProducer::MediaStateFlags playingMediaMask = MediaProducer::IsPlayingAudio | MediaProducer::IsPlayingVideo;
MediaProducer::MediaStateFlags oldState = m_mediaState;
m_mediaState = state;
