Title: [271219] trunk/Source
Revision
271219
Author
[email protected]
Date
2021-01-06 16:16:30 -0800 (Wed, 06 Jan 2021)

Log Message

[Cocoa] Revert audioTimePitchAlgorithm to "TimeDomain" from "Spectral"
https://bugs.webkit.org/show_bug.cgi?id=220341

Reviewed by Youenn Fablet.

Source/WebCore:

The "Spectral" pitch correction algorithm has notable deficiencies when applied to spoken
content; notably, transient propagation is missing, and the resulting audio sounds "robotic".

Revert to "TimeDomain", which is both more computationally inexpensive, and both propogates
transients and is optimized for non-harmonic content like speech.

Add a setting to allow clients to customize this behavior, e.g., if those clients would prefer
optimizing for music over speech.

Drive-by fix: re-apply the pitch algorithm after the AVPlayerItem reaches the "readyToPlay"
status to work around an AVFoundation bug.

* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::loadResource):
* page/Settings.yaml:
* page/SettingsBase.h:
* platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::setPitchCorrectionAlgorithm):
* platform/graphics/MediaPlayer.h:
* platform/graphics/MediaPlayerEnums.h:
* platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::setPitchCorrectionAlgorithm):
* platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
* platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
(WebCore::audioTimePitchAlgorithmForMediaPlayerPitchCorrectionAlgorithm):
(WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerItem):
(WebCore::MediaPlayerPrivateAVFoundationObjC::setPreservesPitch):
(WebCore::MediaPlayerPrivateAVFoundationObjC::setPitchCorrectionAlgorithm):
(WebCore::MediaPlayerPrivateAVFoundationObjC::playerItemStatusDidChange):

Source/WebCore/PAL:

* pal/cocoa/AVFoundationSoftLink.h:
* pal/cocoa/AVFoundationSoftLink.mm:

Source/WebKit:

Add a private preference to control what pitch correction algorithm will be used by MediaPlayer.

* GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::setPitchCorrectionAlgorithm):
* GPUProcess/media/RemoteMediaPlayerProxy.h:
* GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
* UIProcess/API/Cocoa/WKPreferences.mm:
(-[WKPreferences _pitchCorrectionAlgorithm]):
(-[WKPreferences _setPitchCorrectionAlgorithm:]):
* UIProcess/API/Cocoa/WKPreferencesPrivate.h:
* WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::setPitchCorrectionAlgorithm):
* WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::updatePreferences):

Source/WebKitLegacy/mac:

Add a private preference to control what pitch correction algorithm will be used by MediaPlayer.

* WebView/WebPreferenceKeysPrivate.h:
* WebView/WebPreferences.mm:
(-[WebPreferences _pitchCorrectionAlgorithm]):
(-[WebPreferences _setPitchCorrectionAlgorithm:]):
* WebView/WebPreferencesPrivate.h:
* WebView/WebView.mm:
(-[WebView _preferencesChanged:]):

Source/WTF:

Add a private preference to control what pitch correction algorithm will be used by MediaPlayer.

* Scripts/Preferences/WebPreferences.yaml:

Modified Paths

Diff

Modified: trunk/Source/WTF/ChangeLog (271218 => 271219)


--- trunk/Source/WTF/ChangeLog	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WTF/ChangeLog	2021-01-07 00:16:30 UTC (rev 271219)
@@ -1,3 +1,14 @@
+2021-01-06  Jer Noble  <[email protected]>
+
+        [Cocoa] Revert audioTimePitchAlgorithm to "TimeDomain" from "Spectral"
+        https://bugs.webkit.org/show_bug.cgi?id=220341
+
+        Reviewed by Youenn Fablet.
+
+        Add a private preference to control what pitch correction algorithm will be used by MediaPlayer.
+
+        * Scripts/Preferences/WebPreferences.yaml:
+
 2021-01-06  Andy Estes  <[email protected]>
 
         [Mac] Replace most uses of HAVE(MT_PLUGIN_FORMAT_READER) with ENABLE(WEBM_FORMAT_READER)

Modified: trunk/Source/WTF/Scripts/Preferences/WebPreferences.yaml (271218 => 271219)


--- trunk/Source/WTF/Scripts/Preferences/WebPreferences.yaml	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WTF/Scripts/Preferences/WebPreferences.yaml	2021-01-07 00:16:30 UTC (rev 271219)
@@ -1717,6 +1717,17 @@
     WebCore:
       default: true
 
+PitchCorrectionAlgorithm:
+  type: uint32_t
+  webcoreBinding: custom
+  defaultValue:
+    WebKitLegacy:
+      "PLATFORM(COCOA)": static_cast<uint32_t>(WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm::BestForSpeech)
+      default: static_cast<uint32_t>(WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm::BestAllAround)
+    WebKit:
+      "PLATFORM(COCOA)": static_cast<uint32_t>(WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm::BestForSpeech)
+      default: static_cast<uint32_t>(WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm::BestAllAround)
+
 PluginsEnabled:
   type: bool
   webcoreGetter: arePluginsEnabled

Modified: trunk/Source/WebCore/ChangeLog (271218 => 271219)


--- trunk/Source/WebCore/ChangeLog	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/ChangeLog	2021-01-07 00:16:30 UTC (rev 271219)
@@ -1,3 +1,40 @@
+2021-01-06  Jer Noble  <[email protected]>
+
+        [Cocoa] Revert audioTimePitchAlgorithm to "TimeDomain" from "Spectral"
+        https://bugs.webkit.org/show_bug.cgi?id=220341
+
+        Reviewed by Youenn Fablet.
+
+        The "Spectral" pitch correction algorithm has notable deficiencies when applied to spoken
+        content; notably, transient propagation is missing, and the resulting audio sounds "robotic".
+
+        Revert to "TimeDomain", which is both more computationally inexpensive, and both propogates
+        transients and is optimized for non-harmonic content like speech.
+
+        Add a setting to allow clients to customize this behavior, e.g., if those clients would prefer
+        optimizing for music over speech.
+
+        Drive-by fix: re-apply the pitch algorithm after the AVPlayerItem reaches the "readyToPlay"
+        status to work around an AVFoundation bug.
+
+        * html/HTMLMediaElement.cpp:
+        (WebCore::HTMLMediaElement::loadResource):
+        * page/Settings.yaml:
+        * page/SettingsBase.h:
+        * platform/graphics/MediaPlayer.cpp:
+        (WebCore::MediaPlayer::setPitchCorrectionAlgorithm):
+        * platform/graphics/MediaPlayer.h:
+        * platform/graphics/MediaPlayerEnums.h:
+        * platform/graphics/MediaPlayerPrivate.h:
+        (WebCore::MediaPlayerPrivateInterface::setPitchCorrectionAlgorithm):
+        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h:
+        * platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm:
+        (WebCore::audioTimePitchAlgorithmForMediaPlayerPitchCorrectionAlgorithm):
+        (WebCore::MediaPlayerPrivateAVFoundationObjC::createAVPlayerItem):
+        (WebCore::MediaPlayerPrivateAVFoundationObjC::setPreservesPitch):
+        (WebCore::MediaPlayerPrivateAVFoundationObjC::setPitchCorrectionAlgorithm):
+        (WebCore::MediaPlayerPrivateAVFoundationObjC::playerItemStatusDidChange):
+
 2021-01-06  Aditya Keerthi  <[email protected]>
 
         [macOS] Text inside form controls is off center on burton.com

Modified: trunk/Source/WebCore/PAL/ChangeLog (271218 => 271219)


--- trunk/Source/WebCore/PAL/ChangeLog	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/PAL/ChangeLog	2021-01-07 00:16:30 UTC (rev 271219)
@@ -1,3 +1,13 @@
+2021-01-06  Jer Noble  <[email protected]>
+
+        [Cocoa] Revert audioTimePitchAlgorithm to "TimeDomain" from "Spectral"
+        https://bugs.webkit.org/show_bug.cgi?id=220341
+
+        Reviewed by Youenn Fablet.
+
+        * pal/cocoa/AVFoundationSoftLink.h:
+        * pal/cocoa/AVFoundationSoftLink.mm:
+
 2020-12-15  Jer Noble  <[email protected]>
 
         [Cocoa] Adopt -externalContentProtectionStatus

Modified: trunk/Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.h (271218 => 271219)


--- trunk/Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -100,6 +100,8 @@
 
 SOFT_LINK_CONSTANT_FOR_HEADER(PAL, AVFoundation, AVAudioTimePitchAlgorithmSpectral, NSString *)
 #define AVAudioTimePitchAlgorithmSpectral PAL::get_AVFoundation_AVAudioTimePitchAlgorithmSpectral()
+SOFT_LINK_CONSTANT_FOR_HEADER(PAL, AVFoundation, AVAudioTimePitchAlgorithmTimeDomain, NSString *)
+#define AVAudioTimePitchAlgorithmTimeDomain PAL::get_AVFoundation_AVAudioTimePitchAlgorithmTimeDomain()
 SOFT_LINK_CONSTANT_FOR_HEADER(PAL, AVFoundation, AVAudioTimePitchAlgorithmVarispeed, NSString *)
 #define AVAudioTimePitchAlgorithmVarispeed PAL::get_AVFoundation_AVAudioTimePitchAlgorithmVarispeed()
 SOFT_LINK_CONSTANT_FOR_HEADER(PAL, AVFoundation, AVMediaCharacteristicVisual, NSString *)

Modified: trunk/Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.mm (271218 => 271219)


--- trunk/Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.mm	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/PAL/pal/cocoa/AVFoundationSoftLink.mm	2021-01-07 00:16:30 UTC (rev 271219)
@@ -126,6 +126,7 @@
 
 SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVAssetImageGeneratorApertureModeCleanAperture, NSString *, PAL_EXPORT)
 SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVAudioTimePitchAlgorithmSpectral, NSString *, PAL_EXPORT)
+SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVAudioTimePitchAlgorithmTimeDomain, NSString *, PAL_EXPORT)
 SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVAudioTimePitchAlgorithmVarispeed, NSString *, PAL_EXPORT)
 SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVCaptureDeviceWasConnectedNotification, NSString *, PAL_EXPORT)
 SOFT_LINK_CONSTANT_FOR_SOURCE_WITH_EXPORT(PAL, AVFoundation, AVCaptureDeviceWasDisconnectedNotification, NSString *, PAL_EXPORT)

Modified: trunk/Source/WebCore/html/HTMLMediaElement.cpp (271218 => 271219)


--- trunk/Source/WebCore/html/HTMLMediaElement.cpp	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/html/HTMLMediaElement.cpp	2021-01-07 00:16:30 UTC (rev 271219)
@@ -1447,6 +1447,7 @@
     if (!autoplay() && !m_havePreparedToPlay)
         m_player->setPreload(m_mediaSession->effectivePreloadForElement());
     m_player->setPreservesPitch(m_webkitPreservesPitch);
+    m_player->setPitchCorrectionAlgorithm(document().settings().pitchCorrectionAlgorithm());
 
     if (!m_explicitlyMuted) {
         m_explicitlyMuted = true;

Modified: trunk/Source/WebCore/page/Settings.yaml (271218 => 271219)


--- trunk/Source/WebCore/page/Settings.yaml	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/page/Settings.yaml	2021-01-07 00:16:30 UTC (rev 271219)
@@ -410,6 +410,13 @@
     WebCore:
       default: false
 
+PitchCorrectionAlgorithm:
+  type: uint32_t
+  refinedType: MediaPlayerEnums::PitchCorrectionAlgorithm
+  defaultValue:
+    WebCore:
+      default: MediaPlayerEnums::PitchCorrectionAlgorithm::BestAllAround
+
 PDFImageCachingPolicy:
   type: uint32_t
   refinedType: PDFImageCachingPolicy

Modified: trunk/Source/WebCore/page/SettingsBase.h (271218 => 271219)


--- trunk/Source/WebCore/page/SettingsBase.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/page/SettingsBase.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -35,6 +35,7 @@
 #include "ForcedAccessibilityValue.h"
 #include "FrameFlattening.h"
 #include "HTMLParserScriptingFlagPolicy.h"
+#include "MediaPlayerEnums.h"
 #include "PDFImageCachingPolicy.h"
 #include "StorageBlockingPolicy.h"
 #include "StorageMap.h"

Modified: trunk/Source/WebCore/platform/graphics/MediaPlayer.cpp (271218 => 271219)


--- trunk/Source/WebCore/platform/graphics/MediaPlayer.cpp	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/platform/graphics/MediaPlayer.cpp	2021-01-07 00:16:30 UTC (rev 271219)
@@ -931,6 +931,15 @@
     m_private->setPreservesPitch(preservesPitch);
 }
 
+void MediaPlayer::setPitchCorrectionAlgorithm(PitchCorrectionAlgorithm pitchCorrectionAlgorithm)
+{
+    if (m_pitchCorrectionAlgorithm == pitchCorrectionAlgorithm)
+        return;
+
+    m_pitchCorrectionAlgorithm = pitchCorrectionAlgorithm;
+    m_private->setPitchCorrectionAlgorithm(pitchCorrectionAlgorithm);
+}
+
 std::unique_ptr<PlatformTimeRanges> MediaPlayer::buffered()
 {
     return m_private->buffered();

Modified: trunk/Source/WebCore/platform/graphics/MediaPlayer.h (271218 => 271219)


--- trunk/Source/WebCore/platform/graphics/MediaPlayer.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/platform/graphics/MediaPlayer.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -398,6 +398,9 @@
     bool preservesPitch() const;
     void setPreservesPitch(bool);
 
+    void setPitchCorrectionAlgorithm(PitchCorrectionAlgorithm);
+    PitchCorrectionAlgorithm pitchCorrectionAlgorithm() const { return m_pitchCorrectionAlgorithm; }
+
     std::unique_ptr<PlatformTimeRanges> buffered();
     std::unique_ptr<PlatformTimeRanges> seekable();
     void bufferedTimeRangesChanged();
@@ -671,6 +674,7 @@
     bool m_contentMIMETypeWasInferredFromExtension { false };
     bool m_initializingMediaEngine { false };
     DynamicRangeMode m_preferredDynamicRangeMode { DynamicRangeMode::Standard };
+    PitchCorrectionAlgorithm m_pitchCorrectionAlgorithm { PitchCorrectionAlgorithm::BestAllAround };
 
 #if ENABLE(MEDIA_SOURCE)
     RefPtr<MediaSourcePrivateClient> m_mediaSource;

Modified: trunk/Source/WebCore/platform/graphics/MediaPlayerEnums.h (271218 => 271219)


--- trunk/Source/WebCore/platform/graphics/MediaPlayerEnums.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/platform/graphics/MediaPlayerEnums.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -106,6 +106,11 @@
         TargetTypeTVOut
     };
 
+    enum class PitchCorrectionAlgorithm : uint8_t {
+        BestAllAround,
+        BestForMusic,
+        BestForSpeech,
+    };
 };
 
 String convertEnumerationToString(MediaPlayerEnums::ReadyState);
@@ -240,4 +245,13 @@
     >;
 };
 
+template<> struct EnumTraits<WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm> {
+using values = EnumValues<
+    WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm,
+    WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm::BestAllAround,
+    WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm::BestForMusic,
+    WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm::BestForSpeech
+    >;
+};
+
 }; // namespace WTF

Modified: trunk/Source/WebCore/platform/graphics/MediaPlayerPrivate.h (271218 => 271219)


--- trunk/Source/WebCore/platform/graphics/MediaPlayerPrivate.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/platform/graphics/MediaPlayerPrivate.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -123,6 +123,7 @@
     virtual double rate() const { return 0; }
 
     virtual void setPreservesPitch(bool) { }
+    virtual void setPitchCorrectionAlgorithm(MediaPlayer::PitchCorrectionAlgorithm) { }
 
     virtual bool paused() const = 0;
 

Modified: trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h (271218 => 271219)


--- trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -198,6 +198,7 @@
     void setRateDouble(double) final;
     double rate() const final;
     void setPreservesPitch(bool) final;
+    void setPitchCorrectionAlgorithm(MediaPlayer::PitchCorrectionAlgorithm) final;
     void seekToTime(const MediaTime&, const MediaTime& negativeTolerance, const MediaTime& positiveTolerance) final;
     unsigned long long totalBytes() const final;
     std::unique_ptr<PlatformTimeRanges> platformBufferedTimeRanges() const final;

Modified: trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm (271218 => 271219)


--- trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateAVFoundationObjC.mm	2021-01-07 00:16:30 UTC (rev 271219)
@@ -1035,6 +1035,20 @@
     setDelayCallbacks(false);
 }
 
+static NSString* audioTimePitchAlgorithmForMediaPlayerPitchCorrectionAlgorithm(MediaPlayer::PitchCorrectionAlgorithm pitchCorrectionAlgorithm, bool preservesPitch)
+{
+    if (!preservesPitch)
+        return AVAudioTimePitchAlgorithmVarispeed;
+
+    switch (pitchCorrectionAlgorithm) {
+    case MediaPlayer::PitchCorrectionAlgorithm::BestAllAround:
+    case MediaPlayer::PitchCorrectionAlgorithm::BestForMusic:
+        return AVAudioTimePitchAlgorithmSpectral;
+    case MediaPlayer::PitchCorrectionAlgorithm::BestForSpeech:
+        return AVAudioTimePitchAlgorithmTimeDomain;
+    }
+}
+
 void MediaPlayerPrivateAVFoundationObjC::createAVPlayerItem()
 {
     if (m_avPlayerItem)
@@ -1053,7 +1067,7 @@
     for (NSString *keyName in itemKVOProperties())
         [m_avPlayerItem.get() addObserver:m_objcObserver.get() forKeyPath:keyName options:options context:(void *)MediaPlayerAVFoundationObservationContextPlayerItem];
 
-    [m_avPlayerItem setAudioTimePitchAlgorithm:(player()->preservesPitch() ? AVAudioTimePitchAlgorithmSpectral : AVAudioTimePitchAlgorithmVarispeed)];
+    [m_avPlayerItem setAudioTimePitchAlgorithm:audioTimePitchAlgorithmForMediaPlayerPitchCorrectionAlgorithm(player()->pitchCorrectionAlgorithm(), player()->preservesPitch())];
 
     if (m_avPlayer)
         setAVPlayerItem(m_avPlayerItem.get());
@@ -1461,9 +1475,15 @@
 void MediaPlayerPrivateAVFoundationObjC::setPreservesPitch(bool preservesPitch)
 {
     if (m_avPlayerItem)
-        [m_avPlayerItem setAudioTimePitchAlgorithm:(preservesPitch ? AVAudioTimePitchAlgorithmSpectral : AVAudioTimePitchAlgorithmVarispeed)];
+        [m_avPlayerItem setAudioTimePitchAlgorithm:audioTimePitchAlgorithmForMediaPlayerPitchCorrectionAlgorithm(player()->pitchCorrectionAlgorithm(), preservesPitch)];
 }
 
+void MediaPlayerPrivateAVFoundationObjC::setPitchCorrectionAlgorithm(MediaPlayer::PitchCorrectionAlgorithm pitchCorrectionAlgorithm)
+{
+    if (m_avPlayerItem)
+        [m_avPlayerItem setAudioTimePitchAlgorithm:audioTimePitchAlgorithmForMediaPlayerPitchCorrectionAlgorithm(pitchCorrectionAlgorithm, player()->preservesPitch())];
+}
+
 std::unique_ptr<PlatformTimeRanges> MediaPlayerPrivateAVFoundationObjC::platformBufferedTimeRanges() const
 {
     auto timeRanges = makeUnique<PlatformTimeRanges>();
@@ -2947,6 +2967,10 @@
 {
     m_cachedItemStatus = status;
 
+    // FIXME(rdar://72829354): Remove after AVFoundation radar is fixed.
+    if (status == AVPlayerItemStatusReadyToPlay)
+        [m_avPlayerItem setAudioTimePitchAlgorithm:audioTimePitchAlgorithmForMediaPlayerPitchCorrectionAlgorithm(player()->pitchCorrectionAlgorithm(), player()->preservesPitch())];
+
     updateStates();
 }
 

Modified: trunk/Source/WebKit/ChangeLog (271218 => 271219)


--- trunk/Source/WebKit/ChangeLog	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKit/ChangeLog	2021-01-07 00:16:30 UTC (rev 271219)
@@ -1,3 +1,26 @@
+2021-01-06  Jer Noble  <[email protected]>
+
+        [Cocoa] Revert audioTimePitchAlgorithm to "TimeDomain" from "Spectral"
+        https://bugs.webkit.org/show_bug.cgi?id=220341
+
+        Reviewed by Youenn Fablet.
+
+        Add a private preference to control what pitch correction algorithm will be used by MediaPlayer.
+
+        * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
+        (WebKit::RemoteMediaPlayerProxy::setPitchCorrectionAlgorithm):
+        * GPUProcess/media/RemoteMediaPlayerProxy.h:
+        * GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
+        * UIProcess/API/Cocoa/WKPreferences.mm:
+        (-[WKPreferences _pitchCorrectionAlgorithm]):
+        (-[WKPreferences _setPitchCorrectionAlgorithm:]):
+        * UIProcess/API/Cocoa/WKPreferencesPrivate.h:
+        * WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
+        (WebKit::MediaPlayerPrivateRemote::setPitchCorrectionAlgorithm):
+        * WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::updatePreferences):
+
 2021-01-06  Yusuke Suzuki  <[email protected]>
 
         [JSC] Replace JSBigInt::toUint64 with JSBigInt::toBigUInt64

Modified: trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp (271218 => 271219)


--- trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp	2021-01-07 00:16:30 UTC (rev 271219)
@@ -206,6 +206,11 @@
     m_player->setPreservesPitch(preservesPitch);
 }
 
+void RemoteMediaPlayerProxy::setPitchCorrectionAlgorithm(WebCore::MediaPlayer::PitchCorrectionAlgorithm algorithm)
+{
+    m_player->setPitchCorrectionAlgorithm(algorithm);
+}
+
 void RemoteMediaPlayerProxy::prepareForRendering()
 {
     m_player->prepareForRendering();

Modified: trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h (271218 => 271219)


--- trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -133,6 +133,7 @@
     void setPreload(WebCore::MediaPlayerEnums::Preload);
     void setPrivateBrowsingMode(bool);
     void setPreservesPitch(bool);
+    void setPitchCorrectionAlgorithm(WebCore::MediaPlayer::PitchCorrectionAlgorithm);
 
     void setVisible(bool);
     void setShouldMaintainAspectRatio(bool);

Modified: trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in (271218 => 271219)


--- trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in	2021-01-07 00:16:30 UTC (rev 271219)
@@ -46,6 +46,7 @@
     SetPreload(enum:uint8_t WebCore::MediaPlayerEnums::Preload preload)
     SetPrivateBrowsingMode(bool privateMode)
     SetPreservesPitch(bool preservesPitch)
+    SetPitchCorrectionAlgorithm(WebCore::MediaPlayer::PitchCorrectionAlgorithm algorithm)
 
     PrepareForRendering()
     SetVisible(bool visible)

Modified: trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferences.mm (271218 => 271219)


--- trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferences.mm	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferences.mm	2021-01-07 00:16:30 UTC (rev 271219)
@@ -1469,6 +1469,16 @@
     _preferences->setPrivateClickMeasurementEnabled(privateClickMeasurementEnabled);
 }
 
+- (_WKPitchCorrectionAlgorithm)_pitchCorrectionAlgorithm
+{
+    return static_cast<_WKPitchCorrectionAlgorithm>(_preferences->pitchCorrectionAlgorithm());
+}
+
+- (void)_setPitchCorrectionAlgorithm:(_WKPitchCorrectionAlgorithm)pitchCorrectionAlgorithm
+{
+    _preferences->setPitchCorrectionAlgorithm(pitchCorrectionAlgorithm);
+}
+
 @end
 
 

Modified: trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferencesPrivate.h (271218 => 271219)


--- trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferencesPrivate.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferencesPrivate.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -51,6 +51,12 @@
     _WKEditableLinkBehaviorNeverLive,
 } WK_API_AVAILABLE(macos(10.13.4), ios(11.3));
 
+typedef NS_ENUM(NSInteger, _WKPitchCorrectionAlgorithm) {
+    _WKPitchCorrectionAlgorithmBestAllAround,
+    _WKPitchCorrectionAlgorithmBestForMusic,
+    _WKPitchCorrectionAlgorithmBestForSpeech,
+} WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+
 @class _WKExperimentalFeature;
 @class _WKInternalDebugFeature;
 
@@ -163,6 +169,7 @@
 @property (nonatomic, setter=_setAccessibilityIsolatedTreeEnabled:) BOOL _accessibilityIsolatedTreeEnabled WK_API_AVAILABLE(macos(10.16));
 @property (nonatomic, setter=_setSpeechRecognitionEnabled:) BOOL _speechRecognitionEnabled WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 @property (nonatomic, setter=_setPrivateClickMeasurementEnabled:) BOOL _privateClickMeasurementEnabled WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+@property (nonatomic, setter=_setPitchCorrectionAlgorithm:) _WKPitchCorrectionAlgorithm _pitchCorrectionAlgorithm WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 
 #if !TARGET_OS_IPHONE
 @property (nonatomic, setter=_setWebGLEnabled:) BOOL _webGLEnabled WK_API_AVAILABLE(macos(10.13.4));

Modified: trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp (271218 => 271219)


--- trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp	2021-01-07 00:16:30 UTC (rev 271219)
@@ -213,6 +213,11 @@
     connection().send(Messages::RemoteMediaPlayerProxy::SetPreservesPitch(preservesPitch), m_id);
 }
 
+void MediaPlayerPrivateRemote::setPitchCorrectionAlgorithm(WebCore::MediaPlayer::PitchCorrectionAlgorithm algorithm)
+{
+    connection().send(Messages::RemoteMediaPlayerProxy::SetPitchCorrectionAlgorithm(algorithm), m_id);
+}
+
 void MediaPlayerPrivateRemote::setVolumeDouble(double volume)
 {
     connection().send(Messages::RemoteMediaPlayerProxy::SetVolume(volume), m_id);

Modified: trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h (271218 => 271219)


--- trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -206,6 +206,7 @@
     void setMuted(bool) final;
     void setPrivateBrowsingMode(bool) final;
     void setPreservesPitch(bool) final;
+    void setPitchCorrectionAlgorithm(WebCore::MediaPlayer::PitchCorrectionAlgorithm) final;
 
     bool shouldIgnoreIntrinsicSize() final;
 

Modified: trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp (271218 => 271219)


--- trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp	2021-01-07 00:16:30 UTC (rev 271219)
@@ -3755,6 +3755,7 @@
 #if ENABLE(DATA_DETECTION)
     settings.setDataDetectorTypes(static_cast<DataDetectorType>(store.getUInt32ValueForKey(WebPreferencesKey::dataDetectorTypesKey())));
 #endif
+    settings.setPitchCorrectionAlgorithm(static_cast<MediaPlayerEnums::PitchCorrectionAlgorithm>(store.getUInt32ValueForKey(WebPreferencesKey::pitchCorrectionAlgorithmKey())));
 
     DatabaseManager::singleton().setIsAvailable(store.getBoolValueForKey(WebPreferencesKey::databasesEnabledKey()));
 

Modified: trunk/Source/WebKitLegacy/mac/ChangeLog (271218 => 271219)


--- trunk/Source/WebKitLegacy/mac/ChangeLog	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKitLegacy/mac/ChangeLog	2021-01-07 00:16:30 UTC (rev 271219)
@@ -1,3 +1,20 @@
+2021-01-06  Jer Noble  <[email protected]>
+
+        [Cocoa] Revert audioTimePitchAlgorithm to "TimeDomain" from "Spectral"
+        https://bugs.webkit.org/show_bug.cgi?id=220341
+
+        Reviewed by Youenn Fablet.
+
+        Add a private preference to control what pitch correction algorithm will be used by MediaPlayer.
+
+        * WebView/WebPreferenceKeysPrivate.h:
+        * WebView/WebPreferences.mm:
+        (-[WebPreferences _pitchCorrectionAlgorithm]):
+        (-[WebPreferences _setPitchCorrectionAlgorithm:]):
+        * WebView/WebPreferencesPrivate.h:
+        * WebView/WebView.mm:
+        (-[WebView _preferencesChanged:]):
+
 2021-01-04  Jeff Miller  <[email protected]>
 
         Update user-visible copyright strings to include 2021

Modified: trunk/Source/WebKitLegacy/mac/WebView/WebPreferenceKeysPrivate.h (271218 => 271219)


--- trunk/Source/WebKitLegacy/mac/WebView/WebPreferenceKeysPrivate.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKitLegacy/mac/WebView/WebPreferenceKeysPrivate.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -288,6 +288,7 @@
 #define WebKitCSSIndividualTransformPropertiesEnabledPreferenceKey @"WebKitCSSIndividualTransformPropertiesEnabled"
 #define WebKitContactPickerAPIEnabledPreferenceKey @"WebKitContactPickerAPIEnabled"
 #define WebKitSpeechRecognitionEnabledPreferenceKey @"WebKitSpeechRecognitionEnabled"
+#define WebKitPitchCorrectionAlgorithmPreferenceKey @"WebKitPitchCorrectionAlgorithm"
 
 // The preference keys below this point are deprecated and have no effect. They should
 // be removed when it is considered safe to do so.

Modified: trunk/Source/WebKitLegacy/mac/WebView/WebPreferences.mm (271218 => 271219)


--- trunk/Source/WebKitLegacy/mac/WebView/WebPreferences.mm	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKitLegacy/mac/WebView/WebPreferences.mm	2021-01-07 00:16:30 UTC (rev 271219)
@@ -45,6 +45,7 @@
 #import <WebCore/ApplicationCacheStorage.h>
 #import <WebCore/AudioSession.h>
 #import <WebCore/DeprecatedGlobalSettings.h>
+#import <WebCore/MediaPlayerEnums.h>
 #import <WebCore/NetworkStorageSession.h>
 #import <WebCore/RuntimeApplicationChecks.h>
 #import <WebCore/Settings.h>
@@ -3378,6 +3379,16 @@
     [self _setBoolValue:flag forKey:WebKitSpeechRecognitionEnabledPreferenceKey];
 }
 
+- (WebKitPitchCorrectionAlgorithm)_pitchCorrectionAlgorithm
+{
+    return static_cast<WebKitPitchCorrectionAlgorithm>([self _unsignedIntValueForKey:WebKitPitchCorrectionAlgorithmPreferenceKey]);
+}
+
+- (void)_setPitchCorrectionAlgorithm:(WebKitPitchCorrectionAlgorithm)pitchCorrectionAlgorithm
+{
+    [self _setUnsignedIntValue:pitchCorrectionAlgorithm forKey:WebKitPitchCorrectionAlgorithmPreferenceKey];
+}
+
 @end
 
 @implementation WebPreferences (WebPrivateObsolete)

Modified: trunk/Source/WebKitLegacy/mac/WebView/WebPreferencesPrivate.h (271218 => 271219)


--- trunk/Source/WebKitLegacy/mac/WebView/WebPreferencesPrivate.h	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKitLegacy/mac/WebView/WebPreferencesPrivate.h	2021-01-07 00:16:30 UTC (rev 271219)
@@ -73,6 +73,12 @@
     WebKitAudioSessionCategoryAudioProcessing = 'proc',
 } WebKitAudioSessionCategory;
 
+typedef enum {
+    WebKitPitchCorrectionAlgorithmBestAllAround = 0,
+    WebKitPitchCorrectionAlgorithmBestForMusic,
+    WebKitPitchCorrectionAlgorithmBestForSpeech,
+} WebKitPitchCorrectionAlgorithm;
+
 extern NSString *WebPreferencesChangedNotification WEBKIT_DEPRECATED_MAC(10_3, 10_14);
 extern NSString *WebPreferencesRemovedNotification WEBKIT_DEPRECATED_MAC(10_3, 10_14);
 extern NSString *WebPreferencesChangedInternalNotification WEBKIT_DEPRECATED_MAC(10_3, 10_14);
@@ -350,6 +356,7 @@
 @property (nonatomic) BOOL CSSIndividualTransformPropertiesEnabled;
 @property (nonatomic) BOOL contactPickerAPIEnabled;
 @property (nonatomic, setter=_setSpeechRecognitionEnabled:) BOOL _speechRecognitionEnabled;
+@property (nonatomic, setter=_setPitchCorrectionAlgorithm:) WebKitPitchCorrectionAlgorithm _pitchCorrectionAlgorithm;
 @end
 
 @interface WebPreferences (WebPrivateDeprecated)

Modified: trunk/Source/WebKitLegacy/mac/WebView/WebView.mm (271218 => 271219)


--- trunk/Source/WebKitLegacy/mac/WebView/WebView.mm	2021-01-06 23:45:44 UTC (rev 271218)
+++ trunk/Source/WebKitLegacy/mac/WebView/WebView.mm	2021-01-07 00:16:30 UTC (rev 271219)
@@ -2924,6 +2924,7 @@
     settings.setFrameFlattening((const WebCore::FrameFlattening)[preferences frameFlattening]);
     settings.setTextDirectionSubmenuInclusionBehavior(core([preferences textDirectionSubmenuInclusionBehavior]));
     settings.setBackForwardCacheExpirationInterval(Seconds { [preferences _backForwardCacheExpirationInterval] });
+    settings.setPitchCorrectionAlgorithm(static_cast<WebCore::MediaPlayerEnums::PitchCorrectionAlgorithm>([preferences _pitchCorrectionAlgorithm]));
 
     BOOL mediaPlaybackRequiresUserGesture = [preferences mediaPlaybackRequiresUserGesture];
     settings.setVideoPlaybackRequiresUserGesture(mediaPlaybackRequiresUserGesture || [preferences videoPlaybackRequiresUserGesture]);
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to