Diff
Modified: branches/safari-609-branch/LayoutTests/ChangeLog (254999 => 255000)
--- branches/safari-609-branch/LayoutTests/ChangeLog 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/LayoutTests/ChangeLog 2020-01-23 21:42:59 UTC (rev 255000)
@@ -1,3 +1,114 @@
+2020-01-23 Russell Epstein <[email protected]>
+
+ Cherry-pick r254296. rdar://problem/58606275
+
+ Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
+ https://bugs.webkit.org/show_bug.cgi?id=205522
+ <rdar://problem/58125759>
+
+ Reviewed by Chris Dumez.
+
+ Source/WebKit:
+
+ The purpose of this patch is to capture navigations that happen programmatically
+ after the document has loaded. These are delayed redirects and should be counted
+ as redirects in ITP.
+
+ To achieve this, a timestamp is captured in
+ WebPageProxy::didFinishDocumentLoadForFrame() to be able to calculate how much time
+ has passed since that timestamp in the IPC sent from
+ WebPageProxy::logFrameNavigation(). The IPC also gets information on whether
+ userInitiatedActivity exists so that ITP can decide whether to treat the navigation
+ as triggered by the user or not.
+
+ A new test was added and existing test expectations were updated.
+
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+ (WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+ (WebKit::ResourceLoadStatisticsMemoryStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
+ * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+ (WebKit::WebResourceLoadStatisticsStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ The deleted WebResourceLoadStatisticsStore::logFrameNavigation() was dead code.
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+ * NetworkProcess/NetworkProcess.cpp:
+ (WebKit::NetworkProcess::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/NetworkProcess.h:
+ * NetworkProcess/NetworkProcess.messages.in:
+ * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
+ (-[WKWebsiteDataStore _resourceLoadStatisticsSetShouldSubmitTelemetry:]): Deleted.
+ Dead code.
+ * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
+ * UIProcess/WebPageProxy.cpp:
+ (WebKit::WebPageProxy::didFinishDocumentLoadForFrame):
+ Now captures a timestamp.
+ (WebKit::WebPageProxy::decidePolicyForNavigationAction):
+ Removed dead code.
+ (WebKit::WebPageProxy::logFrameNavigation):
+ Now sends the diff between now and the timestamp captured in
+ WebPageProxy::didFinishDocumentLoadForFrame().
+ * UIProcess/WebPageProxy.h:
+ * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
+ (WebKit::WebsiteDataStore::platformInitialize):
+ Removed dead code.
+ (WebKit::WebsiteDataStore::platformDestroy):
+ Removed dead code.
+ * UIProcess/WebsiteData/WebsiteDataStore.h:
+ (WebKit::WebsiteDataStore::resourceLoadStatistics const): Deleted.
+ Dead code.
+
+ Tools:
+
+ * WebKitTestRunner/cocoa/TestControllerCocoa.mm:
+ (WTR::initializeWebViewConfiguration):
+ Removed dead code.
+
+ LayoutTests:
+
+ Results updated with additional data now that delayed redirects are captured.
+
+ * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt:
+ * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt:
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database.html: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects.html: Added.
+ * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt:
+ * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt:
+
+
+ git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254296 268f45cc-cd09-0410-ab3c-d52691b4dbfc
+
+ 2020-01-09 John Wilander <[email protected]>
+
+ Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
+ https://bugs.webkit.org/show_bug.cgi?id=205522
+ <rdar://problem/58125759>
+
+ Reviewed by Chris Dumez.
+
+ Results updated with additional data now that delayed redirects are captured.
+
+ * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt:
+ * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt:
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database.html: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects.html: Added.
+ * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt:
+ * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt:
+
2020-01-15 Alan Coon <[email protected]>
Cherry-pick r254187. rdar://problem/58605950
Modified: branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt (254999 => 255000)
--- branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt 2020-01-23 21:42:59 UTC (rev 255000)
@@ -6,6 +6,8 @@
hadUserInteraction: No
mostRecentUserInteraction: -1
grandfathered: No
+ TopFrameUniqueRedirectsTo:
+ localhost
IsScheduledForAllButCookieDataRemoval: No
isPrevalentResource: Yes
isVeryPrevalentResource: No
@@ -22,6 +24,8 @@
hadUserInteraction: No
mostRecentUserInteraction: -1
grandfathered: No
+ TopFrameUniqueRedirectsFrom:
+ 127.0.0.1
TopFrameLinkDecorationsFrom:
127.0.0.1
127.0.0.2
Modified: branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt (254999 => 255000)
--- branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt 2020-01-23 21:42:59 UTC (rev 255000)
@@ -6,6 +6,8 @@
hadUserInteraction: No
mostRecentUserInteraction: -1
grandfathered: No
+ topFrameUniqueRedirectsFrom:
+ 127.0.0.1
topFrameLinkDecorationsFrom:
127.0.0.1
gotLinkDecorationFromPrevalentResource: Yes
@@ -16,6 +18,8 @@
hadUserInteraction: No
mostRecentUserInteraction: -1
grandfathered: No
+ topFrameUniqueRedirectsTo:
+ localhost
gotLinkDecorationFromPrevalentResource: No
isPrevalentResource: Yes
isVeryPrevalentResource: No
Copied: branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt (from rev 254865, branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt) (0 => 255000)
--- branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt (rev 0)
+++ branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt 2020-01-23 21:42:59 UTC (rev 255000)
@@ -0,0 +1,24 @@
+Non-user initiated, delayed cross-site navigations done client-side should be detected as top frame redirects by ITP.
+
+Resource load statistics:
+
+Registrable domain: 127.0.0.1
+ hadUserInteraction: No
+ mostRecentUserInteraction: -1
+ grandfathered: No
+ TopFrameUniqueRedirectsTo:
+ localhost
+ IsScheduledForAllButCookieDataRemoval: No
+ isPrevalentResource: No
+ isVeryPrevalentResource: No
+ dataRecordsRemoved: 0
+Registrable domain: localhost
+ hadUserInteraction: No
+ mostRecentUserInteraction: -1
+ grandfathered: No
+ TopFrameUniqueRedirectsFrom:
+ 127.0.0.1
+ IsScheduledForAllButCookieDataRemoval: No
+ isPrevalentResource: No
+ isVeryPrevalentResource: No
+ dataRecordsRemoved: 0
Added: branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database.html (0 => 255000)
--- branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database.html (rev 0)
+++ branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database.html 2020-01-23 21:42:59 UTC (rev 255000)
@@ -0,0 +1,35 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <script src=""
+</head>
+<body _onload_="setTimeout('runTest()', 100)">
+<div id="description">Non-user initiated, delayed cross-site navigations done client-side should be detected as top frame redirects by ITP.</div>
+<div id="output"></div>
+<script>
+ if (window.testRunner) {
+ testRunner.setUseITPDatabase(true);
+ testRunner.waitUntilDone();
+ testRunner.dumpAsText();
+ }
+
+ const navigationEndOrigin = "http://localhost:8000";
+ const navigationStartOrigin = "http://127.0.0.1:8000";
+
+ function navigateCrossSite() {
+ document.location.href = "" + "/resourceLoadStatistics/log-delayed-client-side-redirects.html?dummyParam=" + Math.random();
+ }
+
+ function runTest() {
+ if (document.location.origin === navigationStartOrigin) {
+ setEnableFeature(true, navigateCrossSite);
+ } else {
+ testRunner.dumpResourceLoadStatistics();
+ setEnableFeature(false, function () {
+ testRunner.notifyDone();
+ });
+ }
+ }
+</script>
+</body>
+</html>
Copied: branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt (from rev 254865, branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt) (0 => 255000)
--- branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt (rev 0)
+++ branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt 2020-01-23 21:42:59 UTC (rev 255000)
@@ -0,0 +1,24 @@
+Non-user initiated, delayed cross-site navigations done client-side should be detected as top frame redirects by ITP.
+
+Resource load statistics:
+
+Registrable domain: localhost
+ hadUserInteraction: No
+ mostRecentUserInteraction: -1
+ grandfathered: No
+ topFrameUniqueRedirectsFrom:
+ 127.0.0.1
+ gotLinkDecorationFromPrevalentResource: No
+ isPrevalentResource: No
+ isVeryPrevalentResource: No
+ dataRecordsRemoved: 0
+Registrable domain: 127.0.0.1
+ hadUserInteraction: No
+ mostRecentUserInteraction: -1
+ grandfathered: No
+ topFrameUniqueRedirectsTo:
+ localhost
+ gotLinkDecorationFromPrevalentResource: No
+ isPrevalentResource: No
+ isVeryPrevalentResource: No
+ dataRecordsRemoved: 0
Added: branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects.html (0 => 255000)
--- branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects.html (rev 0)
+++ branches/safari-609-branch/LayoutTests/http/tests/resourceLoadStatistics/log-delayed-client-side-redirects.html 2020-01-23 21:42:59 UTC (rev 255000)
@@ -0,0 +1,34 @@
+<!DOCTYPE html>
+<html>
+<head>
+ <script src=""
+</head>
+<body _onload_="setTimeout('runTest()', 100)">
+<div id="description">Non-user initiated, delayed cross-site navigations done client-side should be detected as top frame redirects by ITP.</div>
+<div id="output"></div>
+<script>
+ if (window.testRunner) {
+ testRunner.waitUntilDone();
+ testRunner.dumpAsText();
+ }
+
+ const navigationEndOrigin = "http://localhost:8000";
+ const navigationStartOrigin = "http://127.0.0.1:8000";
+
+ function navigateCrossSite() {
+ document.location.href = "" + "/resourceLoadStatistics/log-delayed-client-side-redirects.html?dummyParam=" + Math.random();
+ }
+
+ function runTest() {
+ if (document.location.origin === navigationStartOrigin) {
+ setEnableFeature(true, navigateCrossSite);
+ } else {
+ testRunner.dumpResourceLoadStatistics();
+ setEnableFeature(false, function () {
+ testRunner.notifyDone();
+ });
+ }
+ }
+</script>
+</body>
+</html>
Modified: branches/safari-609-branch/LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt (254999 => 255000)
--- branches/safari-609-branch/LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt 2020-01-23 21:42:59 UTC (rev 255000)
@@ -14,8 +14,12 @@
hadUserInteraction: Yes
mostRecentUserInteraction: within 24 hours
grandfathered: No
+ TopFrameUniqueRedirectsTo:
+ localhost
+ TopFrameUniqueRedirectsFrom:
+ localhost
IsScheduledForAllButCookieDataRemoval: No
- isPrevalentResource: No
+ isPrevalentResource: Yes
isVeryPrevalentResource: No
dataRecordsRemoved: 0
Registrable domain: localhost
@@ -24,6 +28,10 @@
grandfathered: No
StorageAccessUnderTopFrameDomains:
127.0.0.1
+ TopFrameUniqueRedirectsTo:
+ 127.0.0.1
+ TopFrameUniqueRedirectsFrom:
+ 127.0.0.1
IsScheduledForAllButCookieDataRemoval: No
SubframeUnderTopFrameDomains:
127.0.0.1
Modified: branches/safari-609-branch/LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt (254999 => 255000)
--- branches/safari-609-branch/LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/LayoutTests/http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt 2020-01-23 21:42:59 UTC (rev 255000)
@@ -16,6 +16,10 @@
grandfathered: No
storageAccessUnderTopFrameDomains:
127.0.0.1
+ topFrameUniqueRedirectsTo:
+ 127.0.0.1
+ topFrameUniqueRedirectsFrom:
+ 127.0.0.1
gotLinkDecorationFromPrevalentResource: No
subframeUnderTopFrameDomains:
127.0.0.1
@@ -26,8 +30,12 @@
hadUserInteraction: Yes
mostRecentUserInteraction: within 24 hours
grandfathered: No
+ topFrameUniqueRedirectsTo:
+ localhost
+ topFrameUniqueRedirectsFrom:
+ localhost
gotLinkDecorationFromPrevalentResource: No
- isPrevalentResource: No
+ isPrevalentResource: Yes
isVeryPrevalentResource: No
dataRecordsRemoved: 0
Modified: branches/safari-609-branch/Source/WebKit/ChangeLog (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/ChangeLog 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/ChangeLog 2020-01-23 21:42:59 UTC (rev 255000)
@@ -1,3 +1,161 @@
+2020-01-23 Russell Epstein <[email protected]>
+
+ Cherry-pick r254296. rdar://problem/58606275
+
+ Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
+ https://bugs.webkit.org/show_bug.cgi?id=205522
+ <rdar://problem/58125759>
+
+ Reviewed by Chris Dumez.
+
+ Source/WebKit:
+
+ The purpose of this patch is to capture navigations that happen programmatically
+ after the document has loaded. These are delayed redirects and should be counted
+ as redirects in ITP.
+
+ To achieve this, a timestamp is captured in
+ WebPageProxy::didFinishDocumentLoadForFrame() to be able to calculate how much time
+ has passed since that timestamp in the IPC sent from
+ WebPageProxy::logFrameNavigation(). The IPC also gets information on whether
+ userInitiatedActivity exists so that ITP can decide whether to treat the navigation
+ as triggered by the user or not.
+
+ A new test was added and existing test expectations were updated.
+
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+ (WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+ (WebKit::ResourceLoadStatisticsMemoryStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
+ * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+ (WebKit::WebResourceLoadStatisticsStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ The deleted WebResourceLoadStatisticsStore::logFrameNavigation() was dead code.
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+ * NetworkProcess/NetworkProcess.cpp:
+ (WebKit::NetworkProcess::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/NetworkProcess.h:
+ * NetworkProcess/NetworkProcess.messages.in:
+ * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
+ (-[WKWebsiteDataStore _resourceLoadStatisticsSetShouldSubmitTelemetry:]): Deleted.
+ Dead code.
+ * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
+ * UIProcess/WebPageProxy.cpp:
+ (WebKit::WebPageProxy::didFinishDocumentLoadForFrame):
+ Now captures a timestamp.
+ (WebKit::WebPageProxy::decidePolicyForNavigationAction):
+ Removed dead code.
+ (WebKit::WebPageProxy::logFrameNavigation):
+ Now sends the diff between now and the timestamp captured in
+ WebPageProxy::didFinishDocumentLoadForFrame().
+ * UIProcess/WebPageProxy.h:
+ * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
+ (WebKit::WebsiteDataStore::platformInitialize):
+ Removed dead code.
+ (WebKit::WebsiteDataStore::platformDestroy):
+ Removed dead code.
+ * UIProcess/WebsiteData/WebsiteDataStore.h:
+ (WebKit::WebsiteDataStore::resourceLoadStatistics const): Deleted.
+ Dead code.
+
+ Tools:
+
+ * WebKitTestRunner/cocoa/TestControllerCocoa.mm:
+ (WTR::initializeWebViewConfiguration):
+ Removed dead code.
+
+ LayoutTests:
+
+ Results updated with additional data now that delayed redirects are captured.
+
+ * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt:
+ * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt:
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database.html: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects.html: Added.
+ * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt:
+ * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt:
+
+
+ git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254296 268f45cc-cd09-0410-ab3c-d52691b4dbfc
+
+ 2020-01-09 John Wilander <[email protected]>
+
+ Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
+ https://bugs.webkit.org/show_bug.cgi?id=205522
+ <rdar://problem/58125759>
+
+ Reviewed by Chris Dumez.
+
+ The purpose of this patch is to capture navigations that happen programmatically
+ after the document has loaded. These are delayed redirects and should be counted
+ as redirects in ITP.
+
+ To achieve this, a timestamp is captured in
+ WebPageProxy::didFinishDocumentLoadForFrame() to be able to calculate how much time
+ has passed since that timestamp in the IPC sent from
+ WebPageProxy::logFrameNavigation(). The IPC also gets information on whether
+ userInitiatedActivity exists so that ITP can decide whether to treat the navigation
+ as triggered by the user or not.
+
+ A new test was added and existing test expectations were updated.
+
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+ (WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+ (WebKit::ResourceLoadStatisticsMemoryStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
+ * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+ (WebKit::WebResourceLoadStatisticsStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ The deleted WebResourceLoadStatisticsStore::logFrameNavigation() was dead code.
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+ * NetworkProcess/NetworkProcess.cpp:
+ (WebKit::NetworkProcess::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/NetworkProcess.h:
+ * NetworkProcess/NetworkProcess.messages.in:
+ * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
+ (-[WKWebsiteDataStore _resourceLoadStatisticsSetShouldSubmitTelemetry:]): Deleted.
+ Dead code.
+ * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
+ * UIProcess/WebPageProxy.cpp:
+ (WebKit::WebPageProxy::didFinishDocumentLoadForFrame):
+ Now captures a timestamp.
+ (WebKit::WebPageProxy::decidePolicyForNavigationAction):
+ Removed dead code.
+ (WebKit::WebPageProxy::logFrameNavigation):
+ Now sends the diff between now and the timestamp captured in
+ WebPageProxy::didFinishDocumentLoadForFrame().
+ * UIProcess/WebPageProxy.h:
+ * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
+ (WebKit::WebsiteDataStore::platformInitialize):
+ Removed dead code.
+ (WebKit::WebsiteDataStore::platformDestroy):
+ Removed dead code.
+ * UIProcess/WebsiteData/WebsiteDataStore.h:
+ (WebKit::WebsiteDataStore::resourceLoadStatistics const): Deleted.
+ Dead code.
+
2020-01-21 Alan Coon <[email protected]>
Cherry-pick r254125. rdar://problem/58353217
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp 2020-01-23 21:42:59 UTC (rev 255000)
@@ -1508,7 +1508,7 @@
return primaryDomainsToBlock;
}
-void ResourceLoadStatisticsDatabaseStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
+void ResourceLoadStatisticsDatabaseStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
{
ASSERT(!RunLoop::isMain());
@@ -1523,19 +1523,23 @@
statisticsWereUpdated = true;
}
- if (isRedirect && !areTargetAndSourceDomainsSameSite) {
+ if (!areTargetAndSourceDomainsSameSite) {
if (isMainFrame) {
+ bool wasNavigatedAfterShortDelayWithoutUserInteraction = !wasPotentiallyInitiatedByUser && delayAfterMainFrameDocumentLoad < parameters().minDelayAfterMainFrameDocumentLoadToNotBeARedirect;
+ if (isRedirect || wasNavigatedAfterShortDelayWithoutUserInteraction) {
+ auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
+ auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
+ insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
+ insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
+ statisticsWereUpdated = true;
+ }
+ } else if (isRedirect) {
auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
- insertDomainRelationshipList(topFrameUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
- insertDomainRelationshipList(topFrameUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
- } else {
- auto redirectingDomainResult = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
- auto targetResult = ensureResourceStatisticsForRegistrableDomain(targetDomain);
insertDomainRelationshipList(subresourceUniqueRedirectsToQuery, HashSet<RegistrableDomain>({ targetDomain }), redirectingDomainResult.second);
insertDomainRelationshipList(subresourceUniqueRedirectsFromQuery, HashSet<RegistrableDomain>({ sourceDomain }), targetResult.second);
+ statisticsWereUpdated = true;
}
- statisticsWereUpdated = true;
}
if (statisticsWereUpdated)
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h 2020-01-23 21:42:59 UTC (rev 255000)
@@ -124,7 +124,7 @@
void requestStorageAccess(SubFrameDomain&&, TopFrameDomain&&, WebCore::FrameIdentifier, WebCore::PageIdentifier, CompletionHandler<void(StorageAccessStatus)>&&) override;
void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebCore::StorageAccessPromptWasShown, CompletionHandler<void(WebCore::StorageAccessWasGranted)>&&) override;
- void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame) override;
+ void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser) override;
void logUserInteraction(const TopFrameDomain&, CompletionHandler<void()>&&) override;
void logCrossSiteLoadWithLinkDecoration(const NavigatedFromDomain&, const NavigatedToDomain&) override;
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp 2020-01-23 21:42:59 UTC (rev 255000)
@@ -428,7 +428,7 @@
return domainsToBlock;
}
-void ResourceLoadStatisticsMemoryStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
+void ResourceLoadStatisticsMemoryStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
{
ASSERT(!RunLoop::isMain());
@@ -443,15 +443,18 @@
statisticsWereUpdated = true;
}
- if (isRedirect && !areTargetAndSourceDomainsSameSite) {
+ if (!areTargetAndSourceDomainsSameSite) {
if (isMainFrame) {
auto& redirectingDomainStatistics = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
- if (redirectingDomainStatistics.topFrameUniqueRedirectsTo.add(targetDomain).isNewEntry)
- statisticsWereUpdated = true;
- auto& targetStatistics = ensureResourceStatisticsForRegistrableDomain(targetDomain);
- if (targetStatistics.topFrameUniqueRedirectsFrom.add(sourceDomain).isNewEntry)
- statisticsWereUpdated = true;
- } else {
+ bool wasNavigatedAfterShortDelayWithoutUserInteraction = !wasPotentiallyInitiatedByUser && delayAfterMainFrameDocumentLoad < parameters().minDelayAfterMainFrameDocumentLoadToNotBeARedirect;
+ if (isRedirect || wasNavigatedAfterShortDelayWithoutUserInteraction) {
+ if (redirectingDomainStatistics.topFrameUniqueRedirectsTo.add(targetDomain).isNewEntry)
+ statisticsWereUpdated = true;
+ auto& targetStatistics = ensureResourceStatisticsForRegistrableDomain(targetDomain);
+ if (targetStatistics.topFrameUniqueRedirectsFrom.add(sourceDomain).isNewEntry)
+ statisticsWereUpdated = true;
+ }
+ } else if (isRedirect) {
auto& redirectingDomainStatistics = ensureResourceStatisticsForRegistrableDomain(sourceDomain);
if (redirectingDomainStatistics.subresourceUniqueRedirectsTo.add(targetDomain).isNewEntry)
statisticsWereUpdated = true;
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h 2020-01-23 21:42:59 UTC (rev 255000)
@@ -104,7 +104,7 @@
void requestStorageAccess(SubFrameDomain&&, TopFrameDomain&&, WebCore::FrameIdentifier, WebCore::PageIdentifier, CompletionHandler<void(StorageAccessStatus)>&&) override;
void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebCore::StorageAccessPromptWasShown, CompletionHandler<void(WebCore::StorageAccessWasGranted)>&&) override;
- void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame) override;
+ void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser) override;
void logUserInteraction(const TopFrameDomain&, CompletionHandler<void()>&&) override;
void logCrossSiteLoadWithLinkDecoration(const NavigatedFromDomain&, const NavigatedToDomain&) override;
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsStore.h 2020-01-23 21:42:59 UTC (rev 255000)
@@ -171,7 +171,7 @@
virtual void requestStorageAccess(SubFrameDomain&&, TopFrameDomain&&, WebCore::FrameIdentifier, WebCore::PageIdentifier, CompletionHandler<void(StorageAccessStatus)>&&) = 0;
virtual void grantStorageAccess(SubFrameDomain&&, TopFrameDomain&&, WebCore::FrameIdentifier, WebCore::PageIdentifier, WebCore::StorageAccessPromptWasShown, CompletionHandler<void(WebCore::StorageAccessWasGranted)>&&) = 0;
- virtual void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame) = 0;
+ virtual void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser) = 0;
virtual void logUserInteraction(const TopFrameDomain&, CompletionHandler<void()>&&) = 0;
virtual void logCrossSiteLoadWithLinkDecoration(const NavigatedFromDomain&, const NavigatedToDomain&) = 0;
@@ -222,6 +222,7 @@
Seconds grandfatheringTime { 24_h * 7 };
Seconds cacheMaxAgeCapTime { 24_h * 7 };
Seconds clientSideCookiesAgeCapTime { 24_h * 7 };
+ Seconds minDelayAfterMainFrameDocumentLoadToNotBeARedirect { 5_s };
bool shouldNotifyPagesWhenDataRecordsWereScanned { false };
bool shouldClassifyResourcesBeforeDataRecordsRemoval { true };
bool shouldSubmitTelemetry { true };
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp 2020-01-23 21:42:59 UTC (rev 255000)
@@ -554,43 +554,13 @@
});
}
-void WebResourceLoadStatisticsStore::logFrameNavigation(const WebFrameProxy& frame, const URL& pageURL, const WebCore::ResourceRequest& request, const URL& redirectURL)
+void WebResourceLoadStatisticsStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
{
ASSERT(RunLoop::isMain());
- auto sourceURL = redirectURL;
- bool isRedirect = !redirectURL.isNull();
- if (!isRedirect) {
- sourceURL = frame.url();
- if (sourceURL.isNull())
- sourceURL = pageURL;
- }
-
- auto& targetURL = request.url();
-
- if (!targetURL.isValid() || !pageURL.isValid())
- return;
-
- auto targetHost = targetURL.host();
- auto mainFrameHost = pageURL.host();
-
- if (targetHost.isEmpty() || mainFrameHost.isEmpty() || targetHost == sourceURL.host())
- return;
-
- RegistrableDomain targetDomain { targetURL };
- RegistrableDomain topFrameDomain { pageURL };
- RegistrableDomain sourceDomain { sourceURL };
-
- logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, frame.isMainFrame());
-}
-
-void WebResourceLoadStatisticsStore::logFrameNavigation(const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
-{
- ASSERT(RunLoop::isMain());
-
- postTask([this, targetDomain = targetDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), sourceDomain = sourceDomain.isolatedCopy(), isRedirect, isMainFrame] {
+ postTask([this, targetDomain = targetDomain.isolatedCopy(), topFrameDomain = topFrameDomain.isolatedCopy(), sourceDomain = sourceDomain.isolatedCopy(), isRedirect, isMainFrame, delayAfterMainFrameDocumentLoad, wasPotentiallyInitiatedByUser] {
if (m_statisticsStore)
- m_statisticsStore->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame);
+ m_statisticsStore->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame, delayAfterMainFrameDocumentLoad, wasPotentiallyInitiatedByUser);
});
}
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h 2020-01-23 21:42:59 UTC (rev 255000)
@@ -210,8 +210,7 @@
void applicationWillTerminate();
- void logFrameNavigation(const WebFrameProxy&, const URL& pageURL, const WebCore::ResourceRequest&, const URL& redirectURL);
- void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame);
+ void logFrameNavigation(const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser);
void logUserInteraction(const TopFrameDomain&, CompletionHandler<void()>&&);
void logCrossSiteLoadWithLinkDecoration(const NavigatedFromDomain&, const NavigatedToDomain&, CompletionHandler<void()>&&);
void clearUserInteraction(const TopFrameDomain&, CompletionHandler<void()>&&);
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/NetworkProcess.cpp (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/NetworkProcess.cpp 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/NetworkProcess.cpp 2020-01-23 21:42:59 UTC (rev 255000)
@@ -984,11 +984,11 @@
}
}
-void NetworkProcess::logFrameNavigation(PAL::SessionID sessionID, const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame)
+void NetworkProcess::logFrameNavigation(PAL::SessionID sessionID, const RegistrableDomain& targetDomain, const RegistrableDomain& topFrameDomain, const RegistrableDomain& sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
{
if (auto* networkSession = this->networkSession(sessionID)) {
if (auto* resourceLoadStatistics = networkSession->resourceLoadStatistics())
- resourceLoadStatistics->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame);
+ resourceLoadStatistics->logFrameNavigation(targetDomain, topFrameDomain, sourceDomain, isRedirect, isMainFrame, delayAfterMainFrameDocumentLoad, wasPotentiallyInitiatedByUser);
} else
ASSERT_NOT_REACHED();
}
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/NetworkProcess.h (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/NetworkProcess.h 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/NetworkProcess.h 2020-01-23 21:42:59 UTC (rev 255000)
@@ -225,7 +225,7 @@
void isRelationshipOnlyInDatabaseOnce(PAL::SessionID, const RegistrableDomain& subDomain, const RegistrableDomain& topDomain, CompletionHandler<void(bool)>&&);
void hasLocalStorage(PAL::SessionID, const RegistrableDomain&, CompletionHandler<void(bool)>&&);
void getAllStorageAccessEntries(PAL::SessionID, CompletionHandler<void(Vector<String> domains)>&&);
- void logFrameNavigation(PAL::SessionID, const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame);
+ void logFrameNavigation(PAL::SessionID, const NavigatedToDomain&, const TopFrameDomain&, const NavigatedFromDomain&, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser);
void logUserInteraction(PAL::SessionID, const TopFrameDomain&, CompletionHandler<void()>&&);
void removePrevalentDomains(PAL::SessionID, const Vector<RegistrableDomain>&);
void resetCacheMaxAgeCapForPrevalentResources(PAL::SessionID, CompletionHandler<void()>&&);
Modified: branches/safari-609-branch/Source/WebKit/NetworkProcess/NetworkProcess.messages.in (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/NetworkProcess/NetworkProcess.messages.in 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/NetworkProcess/NetworkProcess.messages.in 2020-01-23 21:42:59 UTC (rev 255000)
@@ -102,7 +102,7 @@
IsRegisteredAsRedirectingTo(PAL::SessionID sessionID, WebCore::RegistrableDomain redirectedFromDomain, WebCore::RegistrableDomain redirectedToDomain) -> (bool isRedirectingTo) Async
IsRegisteredAsSubFrameUnder(PAL::SessionID sessionID, WebCore::RegistrableDomain subFrameDomain, WebCore::RegistrableDomain topFrameDomain) -> (bool isSubframeUnder) Async
IsRegisteredAsSubresourceUnder(PAL::SessionID sessionID, WebCore::RegistrableDomain subresourceDomain, WebCore::RegistrableDomain topFrameDomain) -> (bool isSubresourceUnder) Async
- LogFrameNavigation(PAL::SessionID sessionID, WebCore::RegistrableDomain targetDomain, WebCore::RegistrableDomain topFrameDomain, WebCore::RegistrableDomain sourceDomain, bool isRedirect, bool isMainFrame)
+ LogFrameNavigation(PAL::SessionID sessionID, WebCore::RegistrableDomain targetDomain, WebCore::RegistrableDomain topFrameDomain, WebCore::RegistrableDomain sourceDomain, bool isRedirect, bool isMainFrame, Seconds delayAfterMainFrameDocumentLoad, bool wasPotentiallyInitiatedByUser)
LogUserInteraction(PAL::SessionID sessionID, WebCore::RegistrableDomain topFrameDomain) -> () Async
RemovePrevalentDomains(PAL::SessionID sessionID, Vector<WebCore::RegistrableDomain> domainsWithInteraction)
ResetParametersToDefaultValues(PAL::SessionID sessionID) -> () Async
Modified: branches/safari-609-branch/Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStore.mm (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStore.mm 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStore.mm 2020-01-23 21:42:59 UTC (rev 255000)
@@ -405,17 +405,6 @@
return [NSURL fileURLWithPath:_websiteDataStore->configuration().indexedDBDatabaseDirectory() isDirectory:YES];
}
-- (void)_resourceLoadStatisticsSetShouldSubmitTelemetry:(BOOL)value
-{
-#if ENABLE(RESOURCE_LOAD_STATISTICS)
- auto* store = _websiteDataStore->resourceLoadStatistics();
- if (!store)
- return;
-
- store->setShouldSubmitTelemetry(value);
-#endif
-}
-
- (void)_setResourceLoadStatisticsTestingCallback:(void (^)(WKWebsiteDataStore *, NSString *))callback
{
#if ENABLE(RESOURCE_LOAD_STATISTICS)
Modified: branches/safari-609-branch/Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h 2020-01-23 21:42:59 UTC (rev 255000)
@@ -62,7 +62,6 @@
@property (nonatomic, readonly) NSURL *_indexedDBDatabaseDirectory;
-- (void)_resourceLoadStatisticsSetShouldSubmitTelemetry:(BOOL)value WK_API_AVAILABLE(macos(10.13), ios(11.0));
- (void)_setResourceLoadStatisticsTestingCallback:(nullable void (^)(WKWebsiteDataStore *, NSString *))callback WK_API_AVAILABLE(macos(10.13), ios(11.0));
- (void)_getAllStorageAccessEntriesFor:(WKWebView *)webView completionHandler:(void (^)(NSArray<NSString *> *domains))completionHandler WK_API_AVAILABLE(macos(10.14), ios(12.0));
+ (void)_allowWebsiteDataRecordsForAllOrigins WK_API_AVAILABLE(macos(10.13.4), ios(11.3));
Modified: branches/safari-609-branch/Source/WebKit/UIProcess/WebPageProxy.cpp (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/UIProcess/WebPageProxy.cpp 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/UIProcess/WebPageProxy.cpp 2020-01-23 21:42:59 UTC (rev 255000)
@@ -4562,8 +4562,12 @@
if (frame->isMainFrame() && navigationID)
navigation = navigationState().navigation(navigationID);
- if (frame->isMainFrame())
+ if (frame->isMainFrame()) {
m_navigationClient->didFinishDocumentLoad(*this, navigation.get(), m_process->transformHandlesToObjects(userData.object()).get());
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+ m_didFinishDocumentLoadForMainFrameTimestamp = MonotonicTime::now();
+#endif
+ }
}
void WebPageProxy::didFinishLoadForFrame(FrameIdentifier frameID, uint64_t navigationID, const UserData& userData)
@@ -4976,10 +4980,9 @@
API::Navigation* mainFrameNavigation = frame.isMainFrame() ? navigation.get() : nullptr;
WebFrameProxy* originatingFrame = originatingFrameInfoData.frameID ? process->webFrame(*originatingFrameInfoData.frameID) : nullptr;
+ auto userInitiatedActivity = process->userInitiatedActivity(navigationActionData.userGestureTokenIdentifier);
#if ENABLE(RESOURCE_LOAD_STATISTICS)
- if (auto* resourceLoadStatisticsStore = websiteDataStore().resourceLoadStatistics())
- resourceLoadStatisticsStore->logFrameNavigation(frame, URL(URL(), m_pageLoadState.url()), request, redirectResponse.url());
- logFrameNavigation(frame, URL(URL(), m_pageLoadState.url()), request, redirectResponse.url());
+ logFrameNavigation(frame, URL(URL(), m_pageLoadState.url()), request, redirectResponse.url(), !!userInitiatedActivity);
#endif
if (m_policyClient)
@@ -4994,7 +4997,6 @@
sourceFrameInfo = API::FrameInfo::create(WTFMove(originatingFrameInfoData), originatingPage);
}
- auto userInitiatedActivity = process->userInitiatedActivity(navigationActionData.userGestureTokenIdentifier);
bool shouldOpenAppLinks = !m_shouldSuppressAppLinksInNextNavigationPolicyDecision
&& destinationFrameInfo->isMainFrame()
&& (m_mainFrame ? !hostsAreEqual(m_mainFrame->url(), request.url()) : false)
@@ -5038,7 +5040,7 @@
}
#if ENABLE(RESOURCE_LOAD_STATISTICS)
-void WebPageProxy::logFrameNavigation(const WebFrameProxy& frame, const URL& pageURL, const WebCore::ResourceRequest& request, const URL& redirectURL)
+void WebPageProxy::logFrameNavigation(const WebFrameProxy& frame, const URL& pageURL, const WebCore::ResourceRequest& request, const URL& redirectURL, bool wasPotentiallyInitiatedByUser)
{
ASSERT(RunLoop::isMain());
@@ -5061,7 +5063,7 @@
if (targetHost.isEmpty() || mainFrameHost.isEmpty() || targetHost == sourceURL.host())
return;
- m_process->processPool().sendToNetworkingProcess(Messages::NetworkProcess::LogFrameNavigation(m_websiteDataStore->sessionID(), RegistrableDomain { targetURL }, RegistrableDomain { pageURL }, RegistrableDomain { sourceURL }, isRedirect, frame.isMainFrame()));
+ m_process->processPool().sendToNetworkingProcess(Messages::NetworkProcess::LogFrameNavigation(m_websiteDataStore->sessionID(), RegistrableDomain { targetURL }, RegistrableDomain { pageURL }, RegistrableDomain { sourceURL }, isRedirect, frame.isMainFrame(), MonotonicTime::now() - m_didFinishDocumentLoadForMainFrameTimestamp, wasPotentiallyInitiatedByUser));
}
#endif
Modified: branches/safari-609-branch/Source/WebKit/UIProcess/WebPageProxy.h (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/UIProcess/WebPageProxy.h 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/UIProcess/WebPageProxy.h 2020-01-23 21:42:59 UTC (rev 255000)
@@ -2148,7 +2148,7 @@
void didAttachToRunningProcess();
#if ENABLE(RESOURCE_LOAD_STATISTICS)
- void logFrameNavigation(const WebFrameProxy&, const URL& pageURL, const WebCore::ResourceRequest&, const URL& redirectURL);
+ void logFrameNavigation(const WebFrameProxy&, const URL& pageURL, const WebCore::ResourceRequest&, const URL& redirectURL, bool wasPotentiallyInitiatedByUser);
#endif
// WebPaymentCoordinatorProxy::Client
@@ -2651,6 +2651,10 @@
#if PLATFORM(IOS_FAMILY) && ENABLE(DEVICE_ORIENTATION)
std::unique_ptr<WebDeviceOrientationUpdateProviderProxy> m_webDeviceOrientationUpdateProviderProxy;
#endif
+
+#if ENABLE(RESOURCE_LOAD_STATISTICS)
+ MonotonicTime m_didFinishDocumentLoadForMainFrameTimestamp;
+#endif
};
} // namespace WebKit
Modified: branches/safari-609-branch/Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm 2020-01-23 21:42:59 UTC (rev 255000)
@@ -52,8 +52,6 @@
namespace WebKit {
-static id terminationObserver;
-
static HashSet<WebsiteDataStore*>& dataStores()
{
static NeverDestroyed<HashSet<WebsiteDataStore*>> dataStores;
@@ -229,24 +227,6 @@
void WebsiteDataStore::platformInitialize()
{
- if (!terminationObserver) {
- ASSERT(dataStores().isEmpty());
-
-#if PLATFORM(MAC)
- NSString *notificationName = NSApplicationWillTerminateNotification;
-#else
- NSString *notificationName = UIApplicationWillTerminateNotification;
-#endif
- terminationObserver = [[NSNotificationCenter defaultCenter] addObserverForName:notificationName object:nil queue:nil usingBlock:^(NSNotification *note) {
- for (auto& dataStore : dataStores()) {
-#if ENABLE(RESOURCE_LOAD_STATISTICS)
- if (dataStore->m_resourceLoadStatistics)
- dataStore->m_resourceLoadStatistics->applicationWillTerminate();
-#endif
- }
- }];
- }
-
ASSERT(!dataStores().contains(this));
dataStores().add(this);
}
@@ -253,18 +233,8 @@
void WebsiteDataStore::platformDestroy()
{
-#if ENABLE(RESOURCE_LOAD_STATISTICS)
- if (m_resourceLoadStatistics)
- m_resourceLoadStatistics->applicationWillTerminate();
-#endif
-
ASSERT(dataStores().contains(this));
dataStores().remove(this);
-
- if (dataStores().isEmpty()) {
- [[NSNotificationCenter defaultCenter] removeObserver:terminationObserver];
- terminationObserver = nil;
- }
}
void WebsiteDataStore::platformRemoveRecentSearches(WallTime oldestTimeToRemove)
Modified: branches/safari-609-branch/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h (254999 => 255000)
--- branches/safari-609-branch/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Source/WebKit/UIProcess/WebsiteData/WebsiteDataStore.h 2020-01-23 21:42:59 UTC (rev 255000)
@@ -131,7 +131,6 @@
void setServiceWorkerRegistrationDirectory(String&& directory) { m_resolvedConfiguration->setServiceWorkerRegistrationDirectory(WTFMove(directory)); }
#if ENABLE(RESOURCE_LOAD_STATISTICS)
- WebResourceLoadStatisticsStore* resourceLoadStatistics() const { return m_resourceLoadStatistics.get(); }
void clearResourceLoadStatisticsInWebProcesses(CompletionHandler<void()>&&);
#endif
@@ -336,7 +335,6 @@
const Ref<DeviceIdHashSaltStorage> m_deviceIdHashSaltStorage;
#if ENABLE(RESOURCE_LOAD_STATISTICS)
- RefPtr<WebResourceLoadStatisticsStore> m_resourceLoadStatistics;
bool m_resourceLoadStatisticsDebugMode { false };
bool m_resourceLoadStatisticsEnabled { false };
WTF::Function<void(const String&)> m_statisticsTestingCallback;
Modified: branches/safari-609-branch/Tools/ChangeLog (254999 => 255000)
--- branches/safari-609-branch/Tools/ChangeLog 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Tools/ChangeLog 2020-01-23 21:42:59 UTC (rev 255000)
@@ -1,3 +1,107 @@
+2020-01-23 Russell Epstein <[email protected]>
+
+ Cherry-pick r254296. rdar://problem/58606275
+
+ Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
+ https://bugs.webkit.org/show_bug.cgi?id=205522
+ <rdar://problem/58125759>
+
+ Reviewed by Chris Dumez.
+
+ Source/WebKit:
+
+ The purpose of this patch is to capture navigations that happen programmatically
+ after the document has loaded. These are delayed redirects and should be counted
+ as redirects in ITP.
+
+ To achieve this, a timestamp is captured in
+ WebPageProxy::didFinishDocumentLoadForFrame() to be able to calculate how much time
+ has passed since that timestamp in the IPC sent from
+ WebPageProxy::logFrameNavigation(). The IPC also gets information on whether
+ userInitiatedActivity exists so that ITP can decide whether to treat the navigation
+ as triggered by the user or not.
+
+ A new test was added and existing test expectations were updated.
+
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+ (WebKit::ResourceLoadStatisticsDatabaseStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.cpp:
+ (WebKit::ResourceLoadStatisticsMemoryStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/Classifier/ResourceLoadStatisticsMemoryStore.h:
+ * NetworkProcess/Classifier/ResourceLoadStatisticsStore.h:
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+ (WebKit::WebResourceLoadStatisticsStore::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ The deleted WebResourceLoadStatisticsStore::logFrameNavigation() was dead code.
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.h:
+ * NetworkProcess/NetworkProcess.cpp:
+ (WebKit::NetworkProcess::logFrameNavigation):
+ Now takes the extra parameters delayAfterMainFrameDocumentLoad
+ and wasPotentiallyInitiatedByUser.
+ * NetworkProcess/NetworkProcess.h:
+ * NetworkProcess/NetworkProcess.messages.in:
+ * UIProcess/API/Cocoa/WKWebsiteDataStore.mm:
+ (-[WKWebsiteDataStore _resourceLoadStatisticsSetShouldSubmitTelemetry:]): Deleted.
+ Dead code.
+ * UIProcess/API/Cocoa/WKWebsiteDataStorePrivate.h:
+ * UIProcess/WebPageProxy.cpp:
+ (WebKit::WebPageProxy::didFinishDocumentLoadForFrame):
+ Now captures a timestamp.
+ (WebKit::WebPageProxy::decidePolicyForNavigationAction):
+ Removed dead code.
+ (WebKit::WebPageProxy::logFrameNavigation):
+ Now sends the diff between now and the timestamp captured in
+ WebPageProxy::didFinishDocumentLoadForFrame().
+ * UIProcess/WebPageProxy.h:
+ * UIProcess/WebsiteData/Cocoa/WebsiteDataStoreCocoa.mm:
+ (WebKit::WebsiteDataStore::platformInitialize):
+ Removed dead code.
+ (WebKit::WebsiteDataStore::platformDestroy):
+ Removed dead code.
+ * UIProcess/WebsiteData/WebsiteDataStore.h:
+ (WebKit::WebsiteDataStore::resourceLoadStatistics const): Deleted.
+ Dead code.
+
+ Tools:
+
+ * WebKitTestRunner/cocoa/TestControllerCocoa.mm:
+ (WTR::initializeWebViewConfiguration):
+ Removed dead code.
+
+ LayoutTests:
+
+ Results updated with additional data now that delayed redirects are captured.
+
+ * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-database-expected.txt:
+ * http/tests/resourceLoadStatistics/log-cross-site-load-with-link-decoration-expected.txt:
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database-expected.txt: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-database.html: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects-expected.txt: Added.
+ * http/tests/resourceLoadStatistics/log-delayed-client-side-redirects.html: Added.
+ * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-database-expected.txt:
+ * http/tests/storageAccess/aggregate-sorted-data-with-storage-access-expected.txt:
+
+
+ git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254296 268f45cc-cd09-0410-ab3c-d52691b4dbfc
+
+ 2020-01-09 John Wilander <[email protected]>
+
+ Resource Load Statistics: Add timing information to WebPageProxy::logFrameNavigation() to detect delayed client-side redirects
+ https://bugs.webkit.org/show_bug.cgi?id=205522
+ <rdar://problem/58125759>
+
+ Reviewed by Chris Dumez.
+
+ * WebKitTestRunner/cocoa/TestControllerCocoa.mm:
+ (WTR::initializeWebViewConfiguration):
+ Removed dead code.
+
2020-01-21 Alan Coon <[email protected]>
Cherry-pick r254632. rdar://problem/58764714
Modified: branches/safari-609-branch/Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm (254999 => 255000)
--- branches/safari-609-branch/Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm 2020-01-23 21:14:54 UTC (rev 254999)
+++ branches/safari-609-branch/Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm 2020-01-23 21:42:59 UTC (rev 255000)
@@ -81,7 +81,6 @@
}
[globalWebViewConfiguration.websiteDataStore _setResourceLoadStatisticsEnabled:YES];
- [globalWebViewConfiguration.websiteDataStore _resourceLoadStatisticsSetShouldSubmitTelemetry:NO];
[globalWebsiteDataStoreDelegateClient release];
globalWebsiteDataStoreDelegateClient = [[TestWebsiteDataStoreDelegate alloc] init];
