Title: [271946] trunk/Source
Revision
271946
Author
[email protected]
Date
2021-01-27 04:39:27 -0800 (Wed, 27 Jan 2021)

Log Message

Implement visibility based resource load scheduling for low priority resources
https://bugs.webkit.org/show_bug.cgi?id=220728

Reviewed by Geoff Garen.

Source/WebCore:

Track load scheduling mode during on per page basis.
Add support for prioritizing resources based on their visibility.

* Headers.cmake:
* WebCore.xcodeproj/project.pbxproj:
* loader/LoaderStrategy.cpp:
(WebCore::LoaderStrategy::setResourceLoadSchedulingMode):
(WebCore::LoaderStrategy::prioritizeResourceLoads):
* loader/LoaderStrategy.h:
* page/Page.cpp:
(WebCore::Page::didStartProvisionalLoad):
(WebCore::Page::didFinishLoad):

Sheduled modes is is disabled at latest when the load completes.

(WebCore::Page::doAfterUpdateRendering):
(WebCore::Page::prioritizeVisibleResources):

After rendering update, if in scheduled mode, check the visibility status of images resources.
Prioritize loads for visible resources.
Move out of the shceduled mode if the document is fully parsed and all visibile loads are completed.

(WebCore::Page::setLoadSchedulingMode):
* page/Page.h:
(WebCore::Page::loadSchedulingMode const):

Source/WebKit:

Add a simple network process side resource load scheduler. When active it limits the number of low priority resource loads
(mostly images) that are passed to network layer. The current limit is 6 per host, reduced by any ongoing higher priority
loads. This reduces impact of low priority loads to higher priority ones, espcially on HTTP/2.

The scheduler also supports reprioritizing loads. This is used to let images currently in the viewport to skip the queue
and start their loads faster.

The feature is not enabled in this patch.

* NetworkProcess/Downloads/Download.cpp:
(WebKit::Download::cancel):
(WebKit::Download::didReceiveData):
(WebKit::Download::didFinish):
(WebKit::Download::didFail):
* NetworkProcess/NetworkConnectionToWebProcess.cpp:
(WebKit::NetworkConnectionToWebProcess::clearPageSpecificData):
(WebKit::NetworkConnectionToWebProcess::removeStorageAccessForFrame):
(WebKit::NetworkConnectionToWebProcess::setResourceLoadSchedulingMode):
(WebKit::NetworkConnectionToWebProcess::prioritizeResourceLoads):
(WebKit::NetworkConnectionToWebProcess::clearPageSpecificDataForResourceLoadStatistics): Deleted.

Rename to clearPageSpecificData and move out of #if.

* NetworkProcess/NetworkConnectionToWebProcess.h:
* NetworkProcess/NetworkConnectionToWebProcess.messages.in:
* NetworkProcess/NetworkLoad.cpp:
(WebKit::NetworkLoad::start):
(WebKit::NetworkLoad::startWithScheduling):
(WebKit::NetworkLoad::~NetworkLoad):
(WebKit::NetworkLoad::didCompleteWithError):
* NetworkProcess/NetworkLoad.h:
* NetworkProcess/NetworkLoadScheduler.cpp: Added.
(WebKit::NetworkLoadScheduler::HostContext::schedule):
(WebKit::NetworkLoadScheduler::HostContext::unschedule):
(WebKit::NetworkLoadScheduler::HostContext::prioritize):
(WebKit::NetworkLoadScheduler::HostContext::start):
(WebKit::NetworkLoadScheduler::HostContext::~HostContext):
(WebKit::NetworkLoadScheduler::schedule):
(WebKit::NetworkLoadScheduler::unschedule):
(WebKit::NetworkLoadScheduler::setResourceLoadSchedulingMode):
(WebKit::NetworkLoadScheduler::prioritizeLoads):
(WebKit::NetworkLoadScheduler::clearPageData):
(WebKit::NetworkLoadScheduler::contextForLoad):
* NetworkProcess/NetworkLoadScheduler.h: Copied from Source/WebCore/loader/LoaderStrategy.cpp.
* NetworkProcess/NetworkResourceLoader.cpp:
(WebKit::NetworkResourceLoader::startNetworkLoad):
* NetworkProcess/NetworkSession.cpp:
(WebKit::NetworkSession::networkLoadScheduler):
* NetworkProcess/NetworkSession.h:
* NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp:
(WebKit::NetworkCache::SpeculativeLoad::SpeculativeLoad):
* Sources.txt:
* WebKit.xcodeproj/project.pbxproj:
* WebProcess/Network/WebLoaderStrategy.cpp:
(WebKit::WebLoaderStrategy::setResourceLoadSchedulingMode):
(WebKit::WebLoaderStrategy::prioritizeResourceLoads):
* WebProcess/Network/WebLoaderStrategy.h:
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::close):

Source/WTF:

* Scripts/Preferences/WebPreferencesInternal.yaml:

Add an internal setting, default to off for now.

Modified Paths

Added Paths

Diff

Modified: trunk/Source/WTF/ChangeLog (271945 => 271946)


--- trunk/Source/WTF/ChangeLog	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WTF/ChangeLog	2021-01-27 12:39:27 UTC (rev 271946)
@@ -1,3 +1,14 @@
+2021-01-27  Antti Koivisto  <[email protected]>
+
+        Implement visibility based resource load scheduling for low priority resources
+        https://bugs.webkit.org/show_bug.cgi?id=220728
+
+        Reviewed by Geoff Garen.
+
+        * Scripts/Preferences/WebPreferencesInternal.yaml:
+
+        Add an internal setting, default to off for now.
+
 2021-01-26  BJ Burg  <[email protected]>
 
         REGRESSION(r267641): WebKitDeveloperExtras preference has no effect for WebKitLegacy clients

Modified: trunk/Source/WTF/Scripts/Preferences/WebPreferencesInternal.yaml (271945 => 271946)


--- trunk/Source/WTF/Scripts/Preferences/WebPreferencesInternal.yaml	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WTF/Scripts/Preferences/WebPreferencesInternal.yaml	2021-01-27 12:39:27 UTC (rev 271946)
@@ -599,6 +599,18 @@
     WebCore:
       default: false
 
+ResourceLoadSchedulingEnabled:
+  type: bool
+  humanReadableName: "Resource Load Scheduling"
+  humanReadableDescription: "Network process side priority and visibility based resource load scheduling"
+  defaultValue:
+    WebKitLegacy:
+      default: false
+    WebKit:
+      default: false
+    WebCore:
+      default: false
+
 # FIXME: This is not relevent for WebKitLegacy, so should be excluded from WebKitLegacy entirely.
 RestrictedHTTPResponseAccess:
   type: bool

Modified: trunk/Source/WebCore/ChangeLog (271945 => 271946)


--- trunk/Source/WebCore/ChangeLog	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebCore/ChangeLog	2021-01-27 12:39:27 UTC (rev 271946)
@@ -1,3 +1,36 @@
+2021-01-27  Antti Koivisto  <[email protected]>
+
+        Implement visibility based resource load scheduling for low priority resources
+        https://bugs.webkit.org/show_bug.cgi?id=220728
+
+        Reviewed by Geoff Garen.
+
+        Track load scheduling mode during on per page basis.
+        Add support for prioritizing resources based on their visibility.
+
+        * Headers.cmake:
+        * WebCore.xcodeproj/project.pbxproj:
+        * loader/LoaderStrategy.cpp:
+        (WebCore::LoaderStrategy::setResourceLoadSchedulingMode):
+        (WebCore::LoaderStrategy::prioritizeResourceLoads):
+        * loader/LoaderStrategy.h:
+        * page/Page.cpp:
+        (WebCore::Page::didStartProvisionalLoad):
+        (WebCore::Page::didFinishLoad):
+
+        Sheduled modes is is disabled at latest when the load completes.
+
+        (WebCore::Page::doAfterUpdateRendering):
+        (WebCore::Page::prioritizeVisibleResources):
+
+        After rendering update, if in scheduled mode, check the visibility status of images resources.
+        Prioritize loads for visible resources.
+        Move out of the shceduled mode if the document is fully parsed and all visibile loads are completed.
+
+        (WebCore::Page::setLoadSchedulingMode):
+        * page/Page.h:
+        (WebCore::Page::loadSchedulingMode const):
+
 2021-01-27  Ryosuke Niwa  <[email protected]>
 
         REGRESSION(r271929): MobileSafari hangs upon launch due to a trivial deadlock in IOSurfacePool::setPoolSize

Modified: trunk/Source/WebCore/Headers.cmake (271945 => 271946)


--- trunk/Source/WebCore/Headers.cmake	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebCore/Headers.cmake	2021-01-27 12:39:27 UTC (rev 271946)
@@ -775,6 +775,7 @@
     loader/ImageLoader.h
     loader/LinkLoader.h
     loader/LinkLoaderClient.h
+    loader/LoadSchedulingMode.h
     loader/LoadTiming.h
     loader/LoaderStrategy.h
     loader/MediaResourceLoader.h

Modified: trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj (271945 => 271946)


--- trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebCore/WebCore.xcodeproj/project.pbxproj	2021-01-27 12:39:27 UTC (rev 271946)
@@ -5146,6 +5146,7 @@
 		E47C39321FE6E10500BBBC6B /* RenderTreeBuilderList.h in Headers */ = {isa = PBXBuildFile; fileRef = E47C39281FE6E0DD00BBBC6B /* RenderTreeBuilderList.h */; };
 		E47C39331FE6E10800BBBC6B /* RenderTreeBuilderMultiColumn.h in Headers */ = {isa = PBXBuildFile; fileRef = E47C39271FE6E0DC00BBBC6B /* RenderTreeBuilderMultiColumn.h */; };
 		E47E276516036ED200EE2AFB /* ExtensionStyleSheets.h in Headers */ = {isa = PBXBuildFile; fileRef = E47E276416036ED200EE2AFB /* ExtensionStyleSheets.h */; settings = {ATTRIBUTES = (Private, ); }; };
+		E47FC8A325B84D72005495FC /* LoadSchedulingMode.h in Headers */ = {isa = PBXBuildFile; fileRef = E47FC8A125B84D71005495FC /* LoadSchedulingMode.h */; settings = {ATTRIBUTES = (Private, ); }; };
 		E48137B91DB3B526005C59BF /* StyleValidity.h in Headers */ = {isa = PBXBuildFile; fileRef = E48137B81DB3B526005C59BF /* StyleValidity.h */; settings = {ATTRIBUTES = (Private, ); }; };
 		E4863CFE23842E9E00972158 /* RuleData.h in Headers */ = {isa = PBXBuildFile; fileRef = E4863CFD23842E9E00972158 /* RuleData.h */; };
 		E4946EAF156E64DD00D3297F /* StyleRuleImport.h in Headers */ = {isa = PBXBuildFile; fileRef = E4946EAD156E64DD00D3297F /* StyleRuleImport.h */; settings = {ATTRIBUTES = (Private, ); }; };
@@ -16338,6 +16339,7 @@
 		E47C392C1FE6E0DF00BBBC6B /* RenderTreeUpdater.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RenderTreeUpdater.h; sourceTree = "<group>"; };
 		E47E276416036ED200EE2AFB /* ExtensionStyleSheets.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ExtensionStyleSheets.h; sourceTree = "<group>"; };
 		E47E276716036EDC00EE2AFB /* ExtensionStyleSheets.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ExtensionStyleSheets.cpp; sourceTree = "<group>"; };
+		E47FC8A125B84D71005495FC /* LoadSchedulingMode.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = LoadSchedulingMode.h; sourceTree = "<group>"; };
 		E48137B81DB3B526005C59BF /* StyleValidity.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = StyleValidity.h; sourceTree = "<group>"; };
 		E4863CFA23842E8700972158 /* RuleData.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RuleData.cpp; sourceTree = "<group>"; };
 		E4863CFD23842E9E00972158 /* RuleData.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RuleData.h; sourceTree = "<group>"; };
@@ -27400,6 +27402,7 @@
 				51E6820F16387302003BBF3C /* LoaderStrategy.h */,
 				861C2EA313FB4FDD00062ABB /* LoadTiming.cpp */,
 				8A12E35C11FA33280025836A /* LoadTiming.h */,
+				E47FC8A125B84D71005495FC /* LoadSchedulingMode.h */,
 				CEEFCD7719DB31F7003876D7 /* MediaResourceLoader.cpp */,
 				CEEFCD7819DB31F7003876D7 /* MediaResourceLoader.h */,
 				52F10862162B6D82009AC81E /* MixedContentChecker.cpp */,
@@ -34946,6 +34949,7 @@
 				71A57DF2154BE25C0009D120 /* SVGPathUtilities.h in Headers */,
 				B2227A880D00BF220071B782 /* SVGPatternElement.h in Headers */,
 				7CE58D4A1DD64A5B00128552 /* SVGPoint.h in Headers */,
+				E47FC8A325B84D72005495FC /* LoadSchedulingMode.h in Headers */,
 				B2227A8C0D00BF220071B782 /* SVGPointList.h in Headers */,
 				B2227A8F0D00BF220071B782 /* SVGPolyElement.h in Headers */,
 				B2227A910D00BF220071B782 /* SVGPolygonElement.h in Headers */,

Copied: trunk/Source/WebCore/loader/LoadSchedulingMode.h (from rev 271945, trunk/Source/WebCore/loader/LoaderStrategy.cpp) (0 => 271946)


--- trunk/Source/WebCore/loader/LoadSchedulingMode.h	                        (rev 0)
+++ trunk/Source/WebCore/loader/LoadSchedulingMode.h	2021-01-27 12:39:27 UTC (rev 271946)
@@ -0,0 +1,47 @@
+/*
+ * Copyright (C) 2021 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include <wtf/EnumTraits.h>
+
+namespace WebCore {
+
+enum class LoadSchedulingMode : uint8_t {
+    Direct, // All loads are issued directly to network layer.
+    Prioritized // Low priority loads may get delayed in favor of higher priority ones.
+};
+
+}
+
+namespace WTF {
+template<> struct EnumTraits<WebCore::LoadSchedulingMode> {
+    using values = EnumValues<
+        WebCore::LoadSchedulingMode,
+        WebCore::LoadSchedulingMode::Direct,
+        WebCore::LoadSchedulingMode::Prioritized
+    >;
+};
+}

Modified: trunk/Source/WebCore/loader/LoaderStrategy.cpp (271945 => 271946)


--- trunk/Source/WebCore/loader/LoaderStrategy.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebCore/loader/LoaderStrategy.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -32,6 +32,15 @@
 
 LoaderStrategy::~LoaderStrategy() = default;
 
+
+void LoaderStrategy::setResourceLoadSchedulingMode(Page&, LoadSchedulingMode)
+{
+}
+
+void LoaderStrategy::prioritizeResourceLoads(const Vector<SubresourceLoader*>&)
+{
+}
+
 ResourceResponse LoaderStrategy::responseFromResourceLoadIdentifier(uint64_t /* resourceLoadIdentifier */)
 {
     return { };

Modified: trunk/Source/WebCore/loader/LoaderStrategy.h (271945 => 271946)


--- trunk/Source/WebCore/loader/LoaderStrategy.h	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebCore/loader/LoaderStrategy.h	2021-01-27 12:39:27 UTC (rev 271946)
@@ -26,6 +26,7 @@
 #pragma once
 
 #include "FetchOptions.h"
+#include "LoadSchedulingMode.h"
 #include "PageIdentifier.h"
 #include "ResourceLoadPriority.h"
 #include "ResourceLoaderOptions.h"
@@ -69,6 +70,9 @@
     virtual void suspendPendingRequests() = 0;
     virtual void resumePendingRequests() = 0;
 
+    virtual void setResourceLoadSchedulingMode(Page&, LoadSchedulingMode);
+    virtual void prioritizeResourceLoads(const Vector<SubresourceLoader*>&);
+
     virtual bool usePingLoad() const { return true; }
     using PingLoadCompletionHandler = WTF::Function<void(const ResourceError&, const ResourceResponse&)>;
     virtual void startPingLoad(Frame&, ResourceRequest&, const HTTPHeaderMap& originalRequestHeaders, const FetchOptions&, ContentSecurityPolicyImposition, PingLoadCompletionHandler&& = { }) = 0;

Modified: trunk/Source/WebCore/page/Page.cpp (271945 => 271946)


--- trunk/Source/WebCore/page/Page.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebCore/page/Page.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -29,6 +29,7 @@
 #include "BackForwardClient.h"
 #include "BackForwardController.h"
 #include "CacheStorageProvider.h"
+#include "CachedImage.h"
 #include "CachedResourceLoader.h"
 #include "Chrome.h"
 #include "ChromeClient.h"
@@ -1232,6 +1233,9 @@
 {
     if (m_performanceMonitor)
         m_performanceMonitor->didStartProvisionalLoad();
+
+    if (m_settings->resourceLoadSchedulingEnabled())
+        setLoadSchedulingMode(LoadSchedulingMode::Prioritized);
 }
 
 void Page::didCommitLoad()
@@ -1249,6 +1253,8 @@
 
     if (m_performanceMonitor)
         m_performanceMonitor->didFinishLoad();
+
+    setLoadSchedulingMode(LoadSchedulingMode::Direct);
 }
 
 bool Page::isOnlyNonUtilityPage() const
@@ -1607,6 +1613,8 @@
     });
 #endif
 
+    prioritizeVisibleResources();
+
     m_renderingUpdateRemainingSteps.last().remove(RenderingUpdateStep::EventRegionUpdate);
 
 #if ENABLE(IOS_TOUCH_EVENTS)
@@ -1676,6 +1684,44 @@
     }
 }
 
+void Page::prioritizeVisibleResources()
+{
+    if (loadSchedulingMode() == LoadSchedulingMode::Direct)
+        return;
+    if (!mainFrame().document())
+        return;
+
+    Vector<CachedResource*> toPrioritize;
+
+    forEachDocument([&] (Document& document) {
+        toPrioritize.appendVector(document.cachedResourceLoader().visibleResourcesToPrioritize());
+    });
+
+    if (toPrioritize.isEmpty()) {
+        if (!mainFrame().document()->parsing()) {
+            // All visible resources have been loaded and we are done with parsing. Stop scheduling.
+            setLoadSchedulingMode(LoadSchedulingMode::Direct);
+        }
+        return;
+    }
+
+    auto resourceLoaders = toPrioritize.map([](auto* resource) {
+        return resource->loader();
+    });
+
+    platformStrategies()->loaderStrategy()->prioritizeResourceLoads(resourceLoaders);
+}
+
+void Page::setLoadSchedulingMode(LoadSchedulingMode mode)
+{
+    if (m_loadSchedulingMode == mode)
+        return;
+
+    m_loadSchedulingMode = mode;
+
+    platformStrategies()->loaderStrategy()->setResourceLoadSchedulingMode(*this, m_loadSchedulingMode);
+}
+
 void Page::suspendScriptedAnimations()
 {
     m_scriptedAnimationsSuspended = true;

Modified: trunk/Source/WebCore/page/Page.h (271945 => 271946)


--- trunk/Source/WebCore/page/Page.h	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebCore/page/Page.h	2021-01-27 12:39:27 UTC (rev 271946)
@@ -29,6 +29,7 @@
 #include "LayoutMilestone.h"
 #include "LayoutRect.h"
 #include "LengthBox.h"
+#include "LoadSchedulingMode.h"
 #include "MediaProducer.h"
 #include "MediaSessionGroupIdentifier.h"
 #include "Pagination.h"
@@ -834,6 +835,9 @@
     bool textInteractionEnabled() { return m_textInteractionEnabled; }
     void setTextInteractionEnabled(bool value) { m_textInteractionEnabled = value; }
 
+    LoadSchedulingMode loadSchedulingMode() const { return m_loadSchedulingMode; }
+    void setLoadSchedulingMode(LoadSchedulingMode);
+
 private:
     struct Navigation {
         RegistrableDomain domain;
@@ -873,6 +877,7 @@
     void renderingUpdateCompleted();
     void computeUnfulfilledRenderingSteps(OptionSet<RenderingUpdateStep>);
     void scheduleRenderingUpdateInternal();
+    void prioritizeVisibleResources();
 
     RenderingUpdateScheduler& renderingUpdateScheduler();
 
@@ -1135,6 +1140,7 @@
     bool m_loadsFromNetwork { true };
     bool m_canUseCredentialStorage { true };
     ShouldRelaxThirdPartyCookieBlocking m_shouldRelaxThirdPartyCookieBlocking { ShouldRelaxThirdPartyCookieBlocking::No };
+    LoadSchedulingMode m_loadSchedulingMode { LoadSchedulingMode::Direct };
     bool m_hasBeenNotifiedToInjectUserScripts { false };
 
     MonotonicTime m_lastRenderingUpdateTimestamp;

Modified: trunk/Source/WebKit/ChangeLog (271945 => 271946)


--- trunk/Source/WebKit/ChangeLog	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/ChangeLog	2021-01-27 12:39:27 UTC (rev 271946)
@@ -1,3 +1,70 @@
+2021-01-27  Antti Koivisto  <[email protected]>
+
+        Implement visibility based resource load scheduling for low priority resources
+        https://bugs.webkit.org/show_bug.cgi?id=220728
+
+        Reviewed by Geoff Garen.
+
+        Add a simple network process side resource load scheduler. When active it limits the number of low priority resource loads
+        (mostly images) that are passed to network layer. The current limit is 6 per host, reduced by any ongoing higher priority
+        loads. This reduces impact of low priority loads to higher priority ones, espcially on HTTP/2.
+
+        The scheduler also supports reprioritizing loads. This is used to let images currently in the viewport to skip the queue
+        and start their loads faster.
+
+        The feature is not enabled in this patch.
+
+        * NetworkProcess/Downloads/Download.cpp:
+        (WebKit::Download::cancel):
+        (WebKit::Download::didReceiveData):
+        (WebKit::Download::didFinish):
+        (WebKit::Download::didFail):
+        * NetworkProcess/NetworkConnectionToWebProcess.cpp:
+        (WebKit::NetworkConnectionToWebProcess::clearPageSpecificData):
+        (WebKit::NetworkConnectionToWebProcess::removeStorageAccessForFrame):
+        (WebKit::NetworkConnectionToWebProcess::setResourceLoadSchedulingMode):
+        (WebKit::NetworkConnectionToWebProcess::prioritizeResourceLoads):
+        (WebKit::NetworkConnectionToWebProcess::clearPageSpecificDataForResourceLoadStatistics): Deleted.
+
+        Rename to clearPageSpecificData and move out of #if.
+
+        * NetworkProcess/NetworkConnectionToWebProcess.h:
+        * NetworkProcess/NetworkConnectionToWebProcess.messages.in:
+        * NetworkProcess/NetworkLoad.cpp:
+        (WebKit::NetworkLoad::start):
+        (WebKit::NetworkLoad::startWithScheduling):
+        (WebKit::NetworkLoad::~NetworkLoad):
+        (WebKit::NetworkLoad::didCompleteWithError):
+        * NetworkProcess/NetworkLoad.h:
+        * NetworkProcess/NetworkLoadScheduler.cpp: Added.
+        (WebKit::NetworkLoadScheduler::HostContext::schedule):
+        (WebKit::NetworkLoadScheduler::HostContext::unschedule):
+        (WebKit::NetworkLoadScheduler::HostContext::prioritize):
+        (WebKit::NetworkLoadScheduler::HostContext::start):
+        (WebKit::NetworkLoadScheduler::HostContext::~HostContext):
+        (WebKit::NetworkLoadScheduler::schedule):
+        (WebKit::NetworkLoadScheduler::unschedule):
+        (WebKit::NetworkLoadScheduler::setResourceLoadSchedulingMode):
+        (WebKit::NetworkLoadScheduler::prioritizeLoads):
+        (WebKit::NetworkLoadScheduler::clearPageData):
+        (WebKit::NetworkLoadScheduler::contextForLoad):
+        * NetworkProcess/NetworkLoadScheduler.h: Copied from Source/WebCore/loader/LoaderStrategy.cpp.
+        * NetworkProcess/NetworkResourceLoader.cpp:
+        (WebKit::NetworkResourceLoader::startNetworkLoad):
+        * NetworkProcess/NetworkSession.cpp:
+        (WebKit::NetworkSession::networkLoadScheduler):
+        * NetworkProcess/NetworkSession.h:
+        * NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp:
+        (WebKit::NetworkCache::SpeculativeLoad::SpeculativeLoad):
+        * Sources.txt:
+        * WebKit.xcodeproj/project.pbxproj:
+        * WebProcess/Network/WebLoaderStrategy.cpp:
+        (WebKit::WebLoaderStrategy::setResourceLoadSchedulingMode):
+        (WebKit::WebLoaderStrategy::prioritizeResourceLoads):
+        * WebProcess/Network/WebLoaderStrategy.h:
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::close):
+
 2021-01-27  Youenn Fablet  <[email protected]>
 
         [MacOS] Enable NSURLSession WebSocket code path in WebKitTestRunner

Modified: trunk/Source/WebKit/NetworkProcess/Downloads/Download.cpp (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/Downloads/Download.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/Downloads/Download.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -44,7 +44,7 @@
 #include "NetworkDataTaskCocoa.h"
 #endif
 
-#define RELEASE_LOG_IF_ALLOWED(fmt, ...) RELEASE_LOG_IF(isAlwaysOnLoggingAllowed(), Network, "%p - Download::" fmt, this, ##__VA_ARGS__)
+#define DOWNLOAD_RELEASE_LOG_IF_ALLOWED(fmt, ...) RELEASE_LOG_IF(isAlwaysOnLoggingAllowed(), Network, "%p - Download::" fmt, this, ##__VA_ARGS__)
 
 namespace WebKit {
 using namespace WebCore;
@@ -98,7 +98,7 @@
         completionHandler(resumeData);
         if (!weakThis || m_ignoreDidFailCallback == IgnoreDidFailCallback::No)
             return;
-        RELEASE_LOG_IF_ALLOWED("didCancel: (id = %" PRIu64 ")", downloadID().toUInt64());
+        DOWNLOAD_RELEASE_LOG_IF_ALLOWED("didCancel: (id = %" PRIu64 ")", downloadID().toUInt64());
         if (auto extension = std::exchange(m_sandboxExtension, nullptr))
             extension->revoke();
         m_downloadManager.downloadFinished(*this);
@@ -130,7 +130,7 @@
 void Download::didReceiveData(uint64_t bytesWritten, uint64_t totalBytesWritten, uint64_t totalBytesExpectedToWrite)
 {
     if (!m_hasReceivedData) {
-        RELEASE_LOG_IF_ALLOWED("didReceiveData: Started receiving data (id = %" PRIu64 ")", downloadID().toUInt64());
+        DOWNLOAD_RELEASE_LOG_IF_ALLOWED("didReceiveData: Started receiving data (id = %" PRIu64 ")", downloadID().toUInt64());
         m_hasReceivedData = true;
     }
     
@@ -141,7 +141,7 @@
 
 void Download::didFinish()
 {
-    RELEASE_LOG_IF_ALLOWED("didFinish: (id = %" PRIu64 ")", downloadID().toUInt64());
+    DOWNLOAD_RELEASE_LOG_IF_ALLOWED("didFinish: (id = %" PRIu64 ")", downloadID().toUInt64());
 
     send(Messages::DownloadProxy::DidFinish());
 
@@ -158,7 +158,7 @@
     if (m_ignoreDidFailCallback == IgnoreDidFailCallback::Yes)
         return;
 
-    RELEASE_LOG_IF_ALLOWED("didFail: (id = %" PRIu64 ", isTimeout = %d, isCancellation = %d, errCode = %d)",
+    DOWNLOAD_RELEASE_LOG_IF_ALLOWED("didFail: (id = %" PRIu64 ", isTimeout = %d, isCancellation = %d, errCode = %d)",
         downloadID().toUInt64(), error.isTimeout(), error.isCancellation(), error.errorCode());
 
     send(Messages::DownloadProxy::DidFail(error, resumeData));
@@ -202,4 +202,4 @@
 
 } // namespace WebKit
 
-#undef RELEASE_LOG_IF_ALLOWED
+#undef DOWNLOAD_RELEASE_LOG_IF_ALLOWED

Modified: trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.cpp (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -32,6 +32,8 @@
 #include "Logging.h"
 #include "NetworkCache.h"
 #include "NetworkConnectionToWebProcessMessages.h"
+#include "NetworkLoad.h"
+#include "NetworkLoadScheduler.h"
 #include "NetworkMDNSRegisterMessages.h"
 #include "NetworkProcess.h"
 #include "NetworkProcessConnectionMessages.h"
@@ -853,17 +855,22 @@
         loader->disableExtraNetworkLoadMetricsCapture();
 }
 
+void NetworkConnectionToWebProcess::clearPageSpecificData(PageIdentifier pageID)
+{
+    if (auto* session = networkSession())
+        session->networkLoadScheduler().clearPageData(pageID);
+
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
-void NetworkConnectionToWebProcess::removeStorageAccessForFrame(FrameIdentifier frameID, PageIdentifier pageID)
-{
     if (auto* storageSession = networkProcess().storageSession(m_sessionID))
-        storageSession->removeStorageAccessForFrame(frameID, pageID);
+        storageSession->clearPageSpecificDataForResourceLoadStatistics(pageID);
+#endif
 }
 
-void NetworkConnectionToWebProcess::clearPageSpecificDataForResourceLoadStatistics(PageIdentifier pageID)
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+void NetworkConnectionToWebProcess::removeStorageAccessForFrame(FrameIdentifier frameID, PageIdentifier pageID)
 {
     if (auto* storageSession = networkProcess().storageSession(m_sessionID))
-        storageSession->clearPageSpecificDataForResourceLoadStatistics(pageID);
+        storageSession->removeStorageAccessForFrame(frameID, pageID);
 }
 
 void NetworkConnectionToWebProcess::logUserInteraction(const RegistrableDomain& domain)
@@ -1170,6 +1177,33 @@
     networkProcess().setCORSDisablingPatterns(pageIdentifier, WTFMove(patterns));
 }
 
+void NetworkConnectionToWebProcess::setResourceLoadSchedulingMode(WebCore::PageIdentifier pageIdentifier, WebCore::LoadSchedulingMode mode)
+{
+    auto* session = networkSession();
+    if (!session)
+        return;
+
+    session->networkLoadScheduler().setResourceLoadSchedulingMode(pageIdentifier, mode);
+}
+
+void NetworkConnectionToWebProcess::prioritizeResourceLoads(Vector<ResourceLoadIdentifier> loadIdentifiers)
+{
+    auto* session = networkSession();
+    if (!session)
+        return;
+
+    Vector<NetworkLoad*> loads;
+    for (auto identifier : loadIdentifiers) {
+        auto* loader = m_networkResourceLoaders.get(identifier);
+        if (!loader || !loader->networkLoad())
+            continue;
+        loads.append(loader->networkLoad());
+    }
+
+    session->networkLoadScheduler().prioritizeLoads(loads);
+}
+
+
 } // namespace WebKit
 
 #undef NETWORK_PROCESS_MESSAGE_CHECK_COMPLETION

Modified: trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.h (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.h	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.h	2021-01-27 12:39:27 UTC (rev 271946)
@@ -40,6 +40,8 @@
 #include "WebResourceLoadObserver.h"
 #include <_javascript_Core/ConsoleTypes.h>
 #include <WebCore/FrameIdentifier.h>
+#include <WebCore/LayoutMilestone.h>
+#include <WebCore/LoadSchedulingMode.h>
 #include <WebCore/MessagePortChannelProvider.h>
 #include <WebCore/MessagePortIdentifier.h>
 #include <WebCore/NetworkLoadInformation.h>
@@ -48,6 +50,7 @@
 #include <WebCore/ProcessIdentifier.h>
 #include <WebCore/RegistrableDomain.h>
 #include <WebCore/WebSocketIdentifier.h>
+#include <wtf/OptionSet.h>
 #include <wtf/RefCounted.h>
 
 namespace PAL {
@@ -266,9 +269,10 @@
 
     CacheStorageEngineConnection& cacheStorageConnection();
 
+    void clearPageSpecificData(WebCore::PageIdentifier);
+
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     void removeStorageAccessForFrame(WebCore::FrameIdentifier, WebCore::PageIdentifier);
-    void clearPageSpecificDataForResourceLoadStatistics(WebCore::PageIdentifier);
 
     void logUserInteraction(const RegistrableDomain&);
     void resourceLoadStatisticsUpdated(Vector<WebCore::ResourceLoadStatistics>&&, CompletionHandler<void()>&&);
@@ -325,6 +329,9 @@
 
     void hasUploadStateChanged(bool);
 
+    void setResourceLoadSchedulingMode(WebCore::PageIdentifier, WebCore::LoadSchedulingMode);
+    void prioritizeResourceLoads(Vector<ResourceLoadIdentifier>);
+
 #if ENABLE(APPLE_PAY_REMOTE_UI)
     WebPaymentCoordinatorProxy& paymentCoordinator();
 

Modified: trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.messages.in (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.messages.in	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/NetworkConnectionToWebProcess.messages.in	2021-01-27 12:39:27 UTC (rev 271946)
@@ -62,9 +62,10 @@
     CreateSocketStream(URL url, String cachePartition, WebCore::WebSocketIdentifier identifier)
     CreateSocketChannel(WebCore::ResourceRequest request, String protocol, WebCore::WebSocketIdentifier identifier)
 
+    ClearPageSpecificData(WebCore::PageIdentifier pageID);
+
 #if ENABLE(RESOURCE_LOAD_STATISTICS)
     RemoveStorageAccessForFrame(WebCore::FrameIdentifier frameID, WebCore::PageIdentifier pageID);
-    ClearPageSpecificDataForResourceLoadStatistics(WebCore::PageIdentifier pageID);
     LogUserInteraction(WebCore::RegistrableDomain domain)
     ResourceLoadStatisticsUpdated(Vector<WebCore::ResourceLoadStatistics> statistics) -> () Async
     HasStorageAccess(WebCore::RegistrableDomain subFrameDomain, WebCore::RegistrableDomain topFrameDomain, WebCore::FrameIdentifier frameID, WebCore::PageIdentifier pageID) -> (bool hasStorageAccess) Async
@@ -101,4 +102,6 @@
 #if PLATFORM(MAC)
     UpdateActivePages(String name, Vector<String> activePagesOrigins, audit_token_t auditToken)
 #endif
+    SetResourceLoadSchedulingMode(WebCore::PageIdentifier webPageID, enum:uint8_t WebCore::LoadSchedulingMode mode)
+    PrioritizeResourceLoads(Vector<uint64_t> loadIdentifiers)
 }

Modified: trunk/Source/WebKit/NetworkProcess/NetworkLoad.cpp (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkLoad.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/NetworkLoad.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -29,6 +29,7 @@
 #include "AuthenticationChallengeDisposition.h"
 #include "AuthenticationManager.h"
 #include "NetworkDataTaskBlob.h"
+#include "NetworkLoadScheduler.h"
 #include "NetworkProcess.h"
 #include "NetworkProcessProxyMessages.h"
 #include "NetworkSession.h"
@@ -36,6 +37,8 @@
 #include "WebErrors.h"
 #include <WebCore/ResourceRequest.h>
 #include <WebCore/SharedBuffer.h>
+#include <wtf/ListHashSet.h>
+#include <wtf/NeverDestroyed.h>
 #include <wtf/Seconds.h>
 
 namespace WebKit {
@@ -60,9 +63,19 @@
         m_task->resume();
 }
 
+void NetworkLoad::startWithScheduling()
+{
+    if (!m_task || !m_task->networkSession())
+        return;
+    m_scheduler = makeWeakPtr(m_task->networkSession()->networkLoadScheduler());
+    m_scheduler->schedule(*this);
+}
+
 NetworkLoad::~NetworkLoad()
 {
     ASSERT(RunLoop::isMain());
+    if (m_scheduler)
+        m_scheduler->unschedule(*this);
     if (m_redirectCompletionHandler)
         m_redirectCompletionHandler({ });
     if (m_task)
@@ -225,6 +238,11 @@
 
 void NetworkLoad::didCompleteWithError(const ResourceError& error, const WebCore::NetworkLoadMetrics& networkLoadMetrics)
 {
+    if (m_scheduler) {
+        m_scheduler->unschedule(*this);
+        m_scheduler = nullptr;
+    }
+
     if (error.isNull())
         m_client.get().didFinishLoading(networkLoadMetrics);
     else

Modified: trunk/Source/WebKit/NetworkProcess/NetworkLoad.h (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkLoad.h	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/NetworkLoad.h	2021-01-27 12:39:27 UTC (rev 271946)
@@ -39,6 +39,7 @@
 
 namespace WebKit {
 
+class NetworkLoadScheduler;
 class NetworkProcess;
 
 class NetworkLoad final : private NetworkDataTaskClient {
@@ -48,6 +49,7 @@
     ~NetworkLoad();
 
     void start();
+    void startWithScheduling();
     void cancel();
 
     bool isAllowedToAskUserForCredentials() const;
@@ -56,6 +58,7 @@
     void updateRequestAfterRedirection(WebCore::ResourceRequest&) const;
 
     const NetworkLoadParameters& parameters() const { return m_parameters; }
+    const URL& url() const { return parameters().request.url(); }
 
     void continueWillSendRequest(WebCore::ResourceRequest&&);
 
@@ -90,6 +93,7 @@
     const NetworkLoadParameters m_parameters;
     CompletionHandler<void(WebCore::ResourceRequest&&)> m_redirectCompletionHandler;
     RefPtr<NetworkDataTask> m_task;
+    WeakPtr<NetworkLoadScheduler> m_scheduler;
 
     WebCore::ResourceRequest m_currentRequest; // Updated on redirects.
 };

Added: trunk/Source/WebKit/NetworkProcess/NetworkLoadScheduler.cpp (0 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkLoadScheduler.cpp	                        (rev 0)
+++ trunk/Source/WebKit/NetworkProcess/NetworkLoadScheduler.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -0,0 +1,172 @@
+/*
+ * Copyright (C) 2021 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "NetworkLoadScheduler.h"
+
+#include "NetworkLoad.h"
+
+namespace WebKit {
+
+static constexpr size_t maximumActiveCountForLowPriority = 6;
+
+class NetworkLoadScheduler::HostContext {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    HostContext() = default;
+    ~HostContext();
+
+    void schedule(NetworkLoad&);
+    void unschedule(NetworkLoad&);
+    void prioritize(NetworkLoad&);
+
+    void finishPage(WebCore::PageIdentifier);
+
+private:
+    void start(NetworkLoad&);
+
+    HashSet<NetworkLoad*> m_activeLoads;
+    ListHashSet<NetworkLoad*> m_pendingLoads;
+};
+
+void NetworkLoadScheduler::HostContext::schedule(NetworkLoad& load)
+{
+    auto startImmediately = [&] {
+        auto priority = load.parameters().request.priority();
+        if (priority > ResourceLoadPriority::Low)
+            return true;
+
+        if (m_activeLoads.size() < maximumActiveCountForLowPriority)
+            return true;
+
+        return false;
+    }();
+
+    if (startImmediately) {
+        start(load);
+        return;
+    }
+
+    m_pendingLoads.add(&load);
+}
+
+void NetworkLoadScheduler::HostContext::unschedule(NetworkLoad& load)
+{
+    m_activeLoads.remove(&load);
+    m_pendingLoads.remove(&load);
+
+    if (m_pendingLoads.isEmpty())
+        return;
+    if (m_activeLoads.size() >= maximumActiveCountForLowPriority)
+        return;
+
+    start(*m_pendingLoads.takeFirst());
+}
+
+void NetworkLoadScheduler::HostContext::prioritize(NetworkLoad& load)
+{
+    if (!m_pendingLoads.remove(&load))
+        return;
+
+    start(load);
+}
+
+void NetworkLoadScheduler::HostContext::start(NetworkLoad& load)
+{
+    m_activeLoads.add(&load);
+
+    load.start();
+}
+
+NetworkLoadScheduler::HostContext::~HostContext()
+{
+    for (auto* load : m_pendingLoads)
+        start(*load);
+}
+
+NetworkLoadScheduler::NetworkLoadScheduler() = default;
+NetworkLoadScheduler::~NetworkLoadScheduler() = default;
+
+void NetworkLoadScheduler::schedule(NetworkLoad& load)
+{
+    auto* context = contextForLoad(load);
+
+    if (!context) {
+        load.start();
+        return;
+    }
+
+    context->schedule(load);
+}
+
+void NetworkLoadScheduler::unschedule(NetworkLoad& load)
+{
+    if (auto* context = contextForLoad(load))
+        context->unschedule(load);
+}
+
+void NetworkLoadScheduler::setResourceLoadSchedulingMode(WebCore::PageIdentifier pageIdentifier, WebCore::LoadSchedulingMode mode)
+{
+    switch (mode) {
+    case WebCore::LoadSchedulingMode::Prioritized:
+        m_pageContexts.ensure(pageIdentifier, [&] {
+            return makeUnique<PageContext>();
+        });
+        break;
+    case WebCore::LoadSchedulingMode::Direct:
+        m_pageContexts.remove(pageIdentifier);
+        break;
+    }
+}
+
+void NetworkLoadScheduler::prioritizeLoads(const Vector<NetworkLoad*>& loads)
+{
+    for (auto* load : loads) {
+        if (auto* context = contextForLoad(*load))
+            context->prioritize(*load);
+    }
+}
+
+void NetworkLoadScheduler::clearPageData(WebCore::PageIdentifier pageIdentifier)
+{
+    m_pageContexts.remove(pageIdentifier);
+}
+
+auto NetworkLoadScheduler::contextForLoad(const NetworkLoad& load) -> HostContext*
+{
+    if (!load.url().protocolIsInHTTPFamily())
+        return nullptr;
+
+    auto* pageContext = m_pageContexts.get(load.parameters().webPageID);
+    if (!pageContext)
+        return nullptr;
+
+    auto host = load.url().host().toString();
+    return pageContext->ensure(host, [&] {
+        return makeUnique<HostContext>();
+    }).iterator->value.get();
+}
+
+}

Copied: trunk/Source/WebKit/NetworkProcess/NetworkLoadScheduler.h (from rev 271945, trunk/Source/WebCore/loader/LoaderStrategy.cpp) (0 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkLoadScheduler.h	                        (rev 0)
+++ trunk/Source/WebKit/NetworkProcess/NetworkLoadScheduler.h	2021-01-27 12:39:27 UTC (rev 271946)
@@ -0,0 +1,58 @@
+/*
+ * Copyright (C) 2021 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include <WebCore/LoadSchedulingMode.h>
+#include <WebCore/PageIdentifier.h>
+#include <wtf/HashMap.h>
+#include <wtf/WeakPtr.h>
+
+namespace WebKit {
+
+class NetworkLoad;
+
+class NetworkLoadScheduler : public CanMakeWeakPtr<NetworkLoadScheduler> {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    NetworkLoadScheduler();
+    ~NetworkLoadScheduler();
+
+    void schedule(NetworkLoad&);
+    void unschedule(NetworkLoad&);
+
+    void setResourceLoadSchedulingMode(WebCore::PageIdentifier, WebCore::LoadSchedulingMode);
+    void prioritizeLoads(const Vector<NetworkLoad*>&);
+    void clearPageData(WebCore::PageIdentifier);
+
+private:
+    class HostContext;
+    HostContext* contextForLoad(const NetworkLoad&);
+
+    using PageContext = HashMap<String, std::unique_ptr<HostContext>>;
+    HashMap<WebCore::PageIdentifier, std::unique_ptr<PageContext>> m_pageContexts;
+};
+
+}

Modified: trunk/Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -336,7 +336,7 @@
     parameters.request = WTFMove(request);
     parameters.isNavigatingToAppBoundDomain = m_parameters.isNavigatingToAppBoundDomain;
     m_networkLoad = makeUnique<NetworkLoad>(*this, &networkSession->blobRegistry(), WTFMove(parameters), *networkSession);
-    m_networkLoad->start();
+    m_networkLoad->startWithScheduling();
 
     RELEASE_LOG_IF_ALLOWED("startNetworkLoad: Going to the network (description=%" PUBLIC_LOG_STRING ")", m_networkLoad->description().utf8().data());
 }

Modified: trunk/Source/WebKit/NetworkProcess/NetworkSession.cpp (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkSession.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/NetworkSession.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -27,6 +27,7 @@
 #include "NetworkSession.h"
 
 #include "Logging.h"
+#include "NetworkLoadScheduler.h"
 #include "NetworkProcess.h"
 #include "NetworkProcessProxyMessages.h"
 #include "NetworkResourceLoadParameters.h"
@@ -382,4 +383,11 @@
     m_dataTaskSet.remove(task);
 }
 
+NetworkLoadScheduler& NetworkSession::networkLoadScheduler()
+{
+    if (!m_networkLoadScheduler)
+        m_networkLoadScheduler = makeUnique<NetworkLoadScheduler>();
+    return *m_networkLoadScheduler;
+}
+
 } // namespace WebKit

Modified: trunk/Source/WebKit/NetworkProcess/NetworkSession.h (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/NetworkSession.h	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/NetworkSession.h	2021-01-27 12:39:27 UTC (rev 271946)
@@ -53,6 +53,7 @@
 
 class PrivateClickMeasurementManager;
 class NetworkDataTask;
+class NetworkLoadScheduler;
 class NetworkProcess;
 class NetworkResourceLoader;
 class NetworkSocketChannel;
@@ -150,6 +151,8 @@
     void removeSoftUpdateLoader(ServiceWorkerSoftUpdateLoader* loader) { m_softUpdateLoaders.remove(loader); }
 #endif
 
+    NetworkLoadScheduler& networkLoadScheduler();
+
 protected:
     NetworkSession(NetworkProcess&, const NetworkSessionCreationParameters&);
 
@@ -186,6 +189,7 @@
     bool m_isInvalidated { false };
 #endif
     RefPtr<NetworkCache::Cache> m_cache;
+    std::unique_ptr<NetworkLoadScheduler> m_networkLoadScheduler;
     WebCore::BlobRegistryImpl m_blobRegistry;
     unsigned m_testSpeedMultiplier { 1 };
     bool m_allowsServerPreconnect { true };

Modified: trunk/Source/WebKit/NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/cache/NetworkCacheSpeculativeLoad.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -69,7 +69,7 @@
     parameters.request = m_originalRequest;
     parameters.isNavigatingToAppBoundDomain = isNavigatingToAppBoundDomain;
     m_networkLoad = makeUnique<NetworkLoad>(*this, nullptr, WTFMove(parameters), *networkSession);
-    m_networkLoad->start();
+    m_networkLoad->startWithScheduling();
 }
 
 SpeculativeLoad::~SpeculativeLoad()

Modified: trunk/Source/WebKit/NetworkProcess/soup/WebKitDirectoryInputStream.cpp (271945 => 271946)


--- trunk/Source/WebKit/NetworkProcess/soup/WebKitDirectoryInputStream.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/NetworkProcess/soup/WebKitDirectoryInputStream.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -28,6 +28,7 @@
 
 #include "WebKitDirectoryInputStreamData.h"
 #include <glib/gi18n-lib.h>
+#include <wtf/glib/GUniquePtr.h>
 #include <wtf/glib/WTFGType.h>
 
 struct _WebKitDirectoryInputStreamPrivate {

Modified: trunk/Source/WebKit/Sources.txt (271945 => 271946)


--- trunk/Source/WebKit/Sources.txt	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/Sources.txt	2021-01-27 12:39:27 UTC (rev 271946)
@@ -65,6 +65,7 @@
 NetworkProcess/NetworkHTTPSUpgradeChecker.cpp
 NetworkProcess/NetworkLoad.cpp
 NetworkProcess/NetworkLoadChecker.cpp
+NetworkProcess/NetworkLoadScheduler.cpp
 NetworkProcess/NetworkProcess.cpp
 NetworkProcess/NetworkProcessCreationParameters.cpp
 NetworkProcess/NetworkProcessPlatformStrategies.cpp

Modified: trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj (271945 => 271946)


--- trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/WebKit.xcodeproj/project.pbxproj	2021-01-27 12:39:27 UTC (rev 271946)
@@ -1945,6 +1945,7 @@
 		E4436ECC1A0D040B00EAD204 /* NetworkCache.h in Headers */ = {isa = PBXBuildFile; fileRef = E4436EBF1A0CFDB200EAD204 /* NetworkCache.h */; };
 		E4436ECE1A0D040B00EAD204 /* NetworkCacheKey.h in Headers */ = {isa = PBXBuildFile; fileRef = E4436EC11A0CFDB200EAD204 /* NetworkCacheKey.h */; };
 		E4436ECF1A0D040B00EAD204 /* NetworkCacheStorage.h in Headers */ = {isa = PBXBuildFile; fileRef = E4436EC21A0CFDB200EAD204 /* NetworkCacheStorage.h */; };
+		E47FC8A025B8331C005495FC /* NetworkLoadScheduler.h in Headers */ = {isa = PBXBuildFile; fileRef = E47FC89F25B8331C005495FC /* NetworkLoadScheduler.h */; };
 		E489D28C1A0A2DB80078C06A /* NetworkCacheCoders.h in Headers */ = {isa = PBXBuildFile; fileRef = E489D2851A0A2DB80078C06A /* NetworkCacheCoders.h */; };
 		E49D40D71AD3FB170066B7B9 /* NetworkCacheBlobStorage.h in Headers */ = {isa = PBXBuildFile; fileRef = E49D40D61AD3FB170066B7B9 /* NetworkCacheBlobStorage.h */; };
 		E4D54D0421F1D72D007E3C36 /* ScrollingTreeFrameScrollingNodeRemoteIOS.h in Headers */ = {isa = PBXBuildFile; fileRef = E40C1F9321F0B96E00530718 /* ScrollingTreeFrameScrollingNodeRemoteIOS.h */; };
@@ -5744,6 +5745,8 @@
 		E4436EC31A0CFDB200EAD204 /* NetworkCacheStorage.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkCacheStorage.cpp; sourceTree = "<group>"; };
 		E4697CCC1B25EB8F001B0A6C /* NetworkCacheFileSystem.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkCacheFileSystem.cpp; sourceTree = "<group>"; };
 		E47D1E961B062B66002676A8 /* NetworkCacheData.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkCacheData.cpp; sourceTree = "<group>"; };
+		E47FC89D25B83313005495FC /* NetworkLoadScheduler.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkLoadScheduler.cpp; sourceTree = "<group>"; };
+		E47FC89F25B8331C005495FC /* NetworkLoadScheduler.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkLoadScheduler.h; sourceTree = "<group>"; };
 		E489D2841A0A2DB80078C06A /* NetworkCacheCoders.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkCacheCoders.cpp; sourceTree = "<group>"; };
 		E489D2851A0A2DB80078C06A /* NetworkCacheCoders.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkCacheCoders.h; sourceTree = "<group>"; };
 		E49D40D61AD3FB170066B7B9 /* NetworkCacheBlobStorage.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkCacheBlobStorage.h; sourceTree = "<group>"; };
@@ -8328,6 +8331,8 @@
 				4103FBA12061BDB700C2EAF8 /* NetworkLoadChecker.h */,
 				83D454D61BE9D3C4006C93BD /* NetworkLoadClient.h */,
 				839149631BEA838500D2D953 /* NetworkLoadParameters.h */,
+				E47FC89D25B83313005495FC /* NetworkLoadScheduler.cpp */,
+				E47FC89F25B8331C005495FC /* NetworkLoadScheduler.h */,
 				510CC7DF16138E2900D03ED3 /* NetworkProcess.cpp */,
 				510CC7E016138E2900D03ED3 /* NetworkProcess.h */,
 				51A8A6171627F5BB000D90E9 /* NetworkProcess.messages.in */,
@@ -12091,6 +12096,7 @@
 				93A88B331BC6E9CD00ABA5C2 /* WebHitTestResultData.h in Headers */,
 				F44DFEB21E9E752F0038D196 /* WebIconUtilities.h in Headers */,
 				93261161214054F4000806E7 /* WebIDBConnectionToClient.h in Headers */,
+				E47FC8A025B8331C005495FC /* NetworkLoadScheduler.h in Headers */,
 				514129941C6428BB0059E714 /* WebIDBConnectionToServer.h in Headers */,
 				510523741C73D38B007993CB /* WebIDBConnectionToServerMessages.h in Headers */,
 				933E835523A07C0600DEF289 /* WebIDBServer.h in Headers */,

Modified: trunk/Source/WebKit/WebProcess/Network/WebLoaderStrategy.cpp (271945 => 271946)


--- trunk/Source/WebKit/WebProcess/Network/WebLoaderStrategy.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/WebProcess/Network/WebLoaderStrategy.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -851,6 +851,22 @@
     return false;
 }
 
+void WebLoaderStrategy::setResourceLoadSchedulingMode(WebCore::Page& page, WebCore::LoadSchedulingMode mode)
+{
+    auto& connection = WebProcess::singleton().ensureNetworkProcessConnection().connection();
+    connection.send(Messages::NetworkConnectionToWebProcess::SetResourceLoadSchedulingMode(WebPage::fromCorePage(page).identifier(), mode), 0);
+}
+
+void WebLoaderStrategy::prioritizeResourceLoads(const Vector<WebCore::SubresourceLoader*>& resources)
+{
+    auto identifiers = resources.map([](auto* loader) -> ResourceLoadIdentifier {
+        return loader->identifier();
+    });
+
+    auto& connection = WebProcess::singleton().ensureNetworkProcessConnection().connection();
+    connection.send(Messages::NetworkConnectionToWebProcess::PrioritizeResourceLoads(identifiers), 0);
+}
+
 } // namespace WebKit
 
 #undef RELEASE_LOG_IF_ALLOWED

Modified: trunk/Source/WebKit/WebProcess/Network/WebLoaderStrategy.h (271945 => 271946)


--- trunk/Source/WebKit/WebProcess/Network/WebLoaderStrategy.h	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/WebProcess/Network/WebLoaderStrategy.h	2021-01-27 12:39:27 UTC (rev 271946)
@@ -114,6 +114,9 @@
 
     void isResourceLoadFinished(WebCore::CachedResource&, CompletionHandler<void(bool)>&&) final;
 
+    void setResourceLoadSchedulingMode(WebCore::Page&, WebCore::LoadSchedulingMode) final;
+    void prioritizeResourceLoads(const Vector<WebCore::SubresourceLoader*>&) final;
+
     Vector<uint64_t> ongoingLoads() const final
     {
         return WTF::map(m_webResourceLoaders, [](auto&& keyValue) -> uint64_t {

Modified: trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp (271945 => 271946)


--- trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp	2021-01-27 12:12:29 UTC (rev 271945)
+++ trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp	2021-01-27 12:39:27 UTC (rev 271946)
@@ -1409,9 +1409,7 @@
 
     RELEASE_LOG_IF_ALLOWED(Loading, "close:");
 
-#if ENABLE(RESOURCE_LOAD_STATISTICS)
-    WebProcess::singleton().ensureNetworkProcessConnection().connection().send(Messages::NetworkConnectionToWebProcess::ClearPageSpecificDataForResourceLoadStatistics(m_identifier), 0);
-#endif
+    WebProcess::singleton().ensureNetworkProcessConnection().connection().send(Messages::NetworkConnectionToWebProcess::ClearPageSpecificData(m_identifier), 0);
 
     m_isClosed = true;
 
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to