Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 5ddb3547ed33c90b6fc9d152aeabf20335323002
https://github.com/WebKit/WebKit/commit/5ddb3547ed33c90b6fc9d152aeabf20335323002
Author: Jean-Yves Avenard <[email protected]>
Date: 2026-03-02 (Mon, 02 Mar 2026)
Changed paths:
M Source/WebCore/html/HTMLMediaElement.h
M Source/WebKit/SourcesCocoa.txt
M Source/WebKit/UIProcess/WebPageProxy.cpp
M Source/WebKit/UIProcess/WebPageProxy.h
M Source/WebKit/UIProcess/WebPageProxy.messages.in
M Source/WebKit/WebKit.xcodeproj/project.pbxproj
M Source/WebKit/WebProcess/WebPage/WebPage.cpp
M Source/WebKit/WebProcess/WebPage/WebPage.h
M Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.h
M Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.mm
A Source/WebKit/WebProcess/cocoa/TextRecognitionRequest.cpp
A Source/WebKit/WebProcess/cocoa/TextRecognitionRequest.h
M Source/WebKit/WebProcess/cocoa/VideoPresentationManager.mm
Log Message:
-----------
Unable to select live text on paused video on youtube, khan academy,
marketwatch and cnbc
https://bugs.webkit.org/show_bug.cgi?id=308914
rdar://170817667
Reviewed by Wenson Hsieh.
In 305540@main; code thought to be redundant was incorrectly removed.
Some of the code removed handled the case where a video element was in native
fullscreen and handling of selected text in a paused video was handled
by the AVPlayerViewController.
We continue on the logic flow used in 305540@main (where the content process
instruct the UI process on when to start text recognition analysis) but
now when a video enters native fullscreen, the web process sends
the associated bitmap to a CocoaImageAnalyzerRequest in the UIP to be
ultimately delegated to the AVPlayerViewController.
We do this by having TextRecognitionRequest class instance, controlled by the
web process' PlaybackSessionManager. Using the same heuristics as the logic
pre-305540@main for when to start the text recognition analysis:
1- The video is in native full screen
2- The video is paused and not seeking
A new analysis will be performed when we either:
- enter native fullscreen,
- the currentTime of the video has changed or the video has been paused for
over 250ms.
Manually tested on iPad and iPhone.
On mac, covered by API tests.
* Source/WebCore/html/HTMLMediaElement.h:
* Source/WebKit/SourcesCocoa.txt:
* Source/WebKit/UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::beginTextRecognitionForVideoInElementFullScreen):
(WebKit::WebPageProxy::cancelTextRecognitionForVideoInElementFullScreen):
* Source/WebKit/UIProcess/WebPageProxy.h:
* Source/WebKit/UIProcess/WebPageProxy.messages.in:
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::beginTextRecognitionForVideoInElementFullScreen):
(WebKit::WebPage::cancelTextRecognitionForVideoInElementFullScreen):
* Source/WebKit/WebProcess/WebPage/WebPage.h:
* Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.h:
(WebKit::PlaybackSessionManager::textRecognitionRequest):
* Source/WebKit/WebProcess/cocoa/PlaybackSessionManager.mm:
(WebKit::PlaybackSessionManager::PlaybackSessionManager):
(WebKit::PlaybackSessionManager::currentTimeChanged):
(WebKit::PlaybackSessionManager::rateChanged):
* Source/WebKit/WebProcess/cocoa/TextRecognitionRequest.cpp: Added.
(WebKit::TextRecognitionRequest::TextRecognitionRequest):
(WebKit::TextRecognitionRequest::~TextRecognitionRequest):
(WebKit::TextRecognitionRequest::textRecognitionFor):
(WebKit::TextRecognitionRequest::cancel):
* Source/WebKit/WebProcess/cocoa/TextRecognitionRequest.h: Added.
* Source/WebKit/WebProcess/cocoa/VideoPresentationManager.mm:
(WebKit::VideoPresentationManager::didEnterFullscreen):
(WebKit::VideoPresentationManager::didExitFullscreen):
Canonical link: https://commits.webkit.org/308498@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
