- Revision
- 254397
- Author
- [email protected]
- Date
- 2020-01-11 06:57:57 -0800 (Sat, 11 Jan 2020)
Log Message
Add correct grandfathering functionality to the ITP database backend
https://bugs.webkit.org/show_bug.cgi?id=205844
<rdar://problem/58360450>
Reviewed by John Wilander.
Source/WebKit:
This patch adds functionality for grandfathering statistics upon
creation of a new ITP database file. The memory store relies on
the presence of the ITP plist to indicate whether the data should
be grandfathered. Since the database does not use a plist, this patch
adds a boolean variable to track whether the database file previously
existed or not to indicate when to grandfather.
* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
(WebKit::ResourceLoadStatisticsDatabaseStore::openITPDatabase):
* NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
The boolean variable m_isNewResourceLoadStatisticDatabaseFile is set
to true if the database file did not previously exist, indicating ITP
is being used for the first time and data should be grandfathered.
* NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
(WebKit::WebResourceLoadStatisticsStore::populateMemoryStoreFromDisk):
This is where the grandfathering happens. The call to logTestingEvent
is for API testing purposes only.
(WebKit::WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent):
If a scheduleClearInMemoryAndPersistent is called, the plist is
deleted, and data will be grandfathered upon recreation. To mimic
this, the ITP database store is cleared and becomes "new" again, so
the boolean is set to true.
* UIProcess/API/C/WKPreferences.cpp:
(WKPreferencesSetIsITPDatabaseEnabled):
(WKPreferencesGetIsITPDatabaseEnabled):
* UIProcess/API/C/WKPreferencesRef.h:
* UIProcess/API/Cocoa/WKPreferences.mm:
(-[WKPreferences _isITPDatabaseEnabled]):
* UIProcess/API/Cocoa/WKPreferencesPrivate.h:
In order to properly run the grandfathering
API tests, the 'isITPDatabaseEnabled' boolean needs to be exposed from
Web Preferences.
* UIProcess/Cocoa/WebProcessPoolCocoa.mm:
The API tests rely on the ITP database flag being set correctly. Since
being on-by-default, the ITP database flag does not show up in the
user defaults menu if enabled. This patch fixes this by first checking
to see if the value is in the defaults menu, otherwise returning the
default preferences value. This ensures the preferences value matches
the actual store being initialized.
Tools:
In order to test both the ITP database and memory stores, this patch
had to restructure the grandfathering API tests.
If the test is for the database backend (ends in *Database), it will
check if the default store matches. If so, it continues on with the
normal test. If not, it has to use an API call to reconstruct the
new store and ensure the grandfathering still happens. Vice versa for
the memory store tests.
* TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm:
(isITPDatabaseEnabled):
(TEST):
Modified Paths
Diff
Modified: trunk/Source/WebKit/ChangeLog (254396 => 254397)
--- trunk/Source/WebKit/ChangeLog 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Source/WebKit/ChangeLog 2020-01-11 14:57:57 UTC (rev 254397)
@@ -1,3 +1,55 @@
+2020-01-11 Kate Cheney <[email protected]>
+
+ Add correct grandfathering functionality to the ITP database backend
+ https://bugs.webkit.org/show_bug.cgi?id=205844
+ <rdar://problem/58360450>
+
+ Reviewed by John Wilander.
+
+ This patch adds functionality for grandfathering statistics upon
+ creation of a new ITP database file. The memory store relies on
+ the presence of the ITP plist to indicate whether the data should
+ be grandfathered. Since the database does not use a plist, this patch
+ adds a boolean variable to track whether the database file previously
+ existed or not to indicate when to grandfather.
+
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp:
+ (WebKit::ResourceLoadStatisticsDatabaseStore::openITPDatabase):
+ * NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h:
+ The boolean variable m_isNewResourceLoadStatisticDatabaseFile is set
+ to true if the database file did not previously exist, indicating ITP
+ is being used for the first time and data should be grandfathered.
+
+ * NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp:
+ (WebKit::WebResourceLoadStatisticsStore::populateMemoryStoreFromDisk):
+ This is where the grandfathering happens. The call to logTestingEvent
+ is for API testing purposes only.
+
+ (WebKit::WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent):
+ If a scheduleClearInMemoryAndPersistent is called, the plist is
+ deleted, and data will be grandfathered upon recreation. To mimic
+ this, the ITP database store is cleared and becomes "new" again, so
+ the boolean is set to true.
+
+ * UIProcess/API/C/WKPreferences.cpp:
+ (WKPreferencesSetIsITPDatabaseEnabled):
+ (WKPreferencesGetIsITPDatabaseEnabled):
+ * UIProcess/API/C/WKPreferencesRef.h:
+ * UIProcess/API/Cocoa/WKPreferences.mm:
+ (-[WKPreferences _isITPDatabaseEnabled]):
+ * UIProcess/API/Cocoa/WKPreferencesPrivate.h:
+ In order to properly run the grandfathering
+ API tests, the 'isITPDatabaseEnabled' boolean needs to be exposed from
+ Web Preferences.
+
+ * UIProcess/Cocoa/WebProcessPoolCocoa.mm:
+ The API tests rely on the ITP database flag being set correctly. Since
+ being on-by-default, the ITP database flag does not show up in the
+ user defaults menu if enabled. This patch fixes this by first checking
+ to see if the value is in the defaults menu, otherwise returning the
+ default preferences value. This ensures the preferences value matches
+ the actual store being initialized.
+
2020-01-10 Eric Carlson <[email protected]>
[Media in GPU process] Extend the GPU process sandbox to allow access to local files when necessary
Modified: trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp (254396 => 254397)
--- trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.cpp 2020-01-11 14:57:57 UTC (rev 254397)
@@ -309,6 +309,11 @@
void ResourceLoadStatisticsDatabaseStore::openITPDatabase()
{
+ if (!FileSystem::fileExists(m_storageDirectoryPath))
+ m_isNewResourceLoadStatisticsDatabaseFile = true;
+ else
+ m_isNewResourceLoadStatisticsDatabaseFile = false;
+
if (!m_database.open(m_storageDirectoryPath)) {
RELEASE_LOG_ERROR(Network, "%p - ResourceLoadStatisticsDatabaseStore::open failed, error message: %{public}s, database path: %{public}s", this, m_database.lastErrorMsg(), m_storageDirectoryPath.utf8().data());
ASSERT_NOT_REACHED();
Modified: trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h (254396 => 254397)
--- trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Source/WebKit/NetworkProcess/Classifier/ResourceLoadStatisticsDatabaseStore.h 2020-01-11 14:57:57 UTC (rev 254397)
@@ -135,6 +135,8 @@
bool isCorrectSubStatisticsCount(const RegistrableDomain&, const TopFrameDomain&);
void resourceToString(StringBuilder&, const String&) const;
Seconds getMostRecentlyUpdatedTimestamp(const RegistrableDomain&, const TopFrameDomain&) const;
+ bool isNewResourceLoadStatisticsDatabaseFile() const { return m_isNewResourceLoadStatisticsDatabaseFile; }
+ void setIsNewResourceLoadStatisticsDatabaseFile(bool isNewResourceLoadStatisticsDatabaseFile) { m_isNewResourceLoadStatisticsDatabaseFile = isNewResourceLoadStatisticsDatabaseFile; }
private:
void openITPDatabase();
@@ -256,6 +258,7 @@
mutable WebCore::SQLiteStatement m_storageAccessExistsStatement;
mutable WebCore::SQLiteStatement m_getMostRecentlyUpdatedTimestampStatement;
PAL::SessionID m_sessionID;
+ bool m_isNewResourceLoadStatisticsDatabaseFile { false };
};
} // namespace WebKit
Modified: trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp (254396 => 254397)
--- trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Source/WebKit/NetworkProcess/Classifier/WebResourceLoadStatisticsStore.cpp 2020-01-11 14:57:57 UTC (rev 254397)
@@ -230,7 +230,19 @@
m_persistentStorage->populateMemoryStoreFromDisk([protectedThis = WTFMove(protectedThis), completionHandler = WTFMove(completionHandler)]() mutable {
postTaskReply(WTFMove(completionHandler));
});
- else
+ else if (is<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore)) {
+ auto& databaseStore = downcast<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore);
+ if (databaseStore.isNewResourceLoadStatisticsDatabaseFile()) {
+ m_statisticsStore->grandfatherExistingWebsiteData([protectedThis = WTFMove(protectedThis), completionHandler = WTFMove(completionHandler)]() mutable {
+ postTaskReply(WTFMove(completionHandler));
+ });
+ databaseStore.setIsNewResourceLoadStatisticsDatabaseFile(false);
+ } else
+ postTaskReply([this, protectedThis = WTFMove(protectedThis), completionHandler = WTFMove(completionHandler)]() mutable {
+ logTestingEvent("PopulatedWithoutGrandfathering"_s);
+ completionHandler();
+ });
+ } else
postTaskReply(WTFMove(completionHandler));
});
}
@@ -924,9 +936,11 @@
m_statisticsStore->clear([this, protectedThis = protectedThis.copyRef(), shouldGrandfather, callbackAggregator = callbackAggregator.copyRef()] () mutable {
if (shouldGrandfather == ShouldGrandfatherStatistics::Yes) {
- if (m_statisticsStore)
+ if (m_statisticsStore) {
m_statisticsStore->grandfatherExistingWebsiteData([callbackAggregator = WTFMove(callbackAggregator)]() mutable { });
- else
+ if (is<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore))
+ downcast<ResourceLoadStatisticsDatabaseStore>(*m_statisticsStore).setIsNewResourceLoadStatisticsDatabaseFile(true);
+ } else
RELEASE_LOG(ResourceLoadStatistics, "WebResourceLoadStatisticsStore::scheduleClearInMemoryAndPersistent After being cleared, m_statisticsStore is null when trying to grandfather data.");
}
});
Modified: trunk/Source/WebKit/UIProcess/API/C/WKPreferences.cpp (254396 => 254397)
--- trunk/Source/WebKit/UIProcess/API/C/WKPreferences.cpp 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Source/WebKit/UIProcess/API/C/WKPreferences.cpp 2020-01-11 14:57:57 UTC (rev 254397)
@@ -118,6 +118,16 @@
return toImpl(preferencesRef)->loadsSiteIconsIgnoringImageLoadingPreference();
}
+void WKPreferencesSetIsITPDatabaseEnabled(WKPreferencesRef preferencesRef, bool isITPDatabaseEnabled)
+{
+ toImpl(preferencesRef)->setIsITPDatabaseEnabled(isITPDatabaseEnabled);
+}
+
+bool WKPreferencesGetIsITPDatabaseEnabled(WKPreferencesRef preferencesRef)
+{
+ return toImpl(preferencesRef)->isITPDatabaseEnabled();
+}
+
void WKPreferencesSetOfflineWebApplicationCacheEnabled(WKPreferencesRef preferencesRef, bool offlineWebApplicationCacheEnabled)
{
toImpl(preferencesRef)->setOfflineWebApplicationCacheEnabled(offlineWebApplicationCacheEnabled);
Modified: trunk/Source/WebKit/UIProcess/API/C/WKPreferencesRef.h (254396 => 254397)
--- trunk/Source/WebKit/UIProcess/API/C/WKPreferencesRef.h 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Source/WebKit/UIProcess/API/C/WKPreferencesRef.h 2020-01-11 14:57:57 UTC (rev 254397)
@@ -78,6 +78,10 @@
WK_EXPORT bool WKPreferencesGetLoadsSiteIconsIgnoringImageLoadingPreference(WKPreferencesRef preferences);
// Defaults to true.
+WK_EXPORT void WKPreferencesSetIsITPDatabaseEnabled(WKPreferencesRef preferences, bool isITPDatabaseEnabled);
+WK_EXPORT bool WKPreferencesGetIsITPDatabaseEnabled(WKPreferencesRef preferences);
+
+// Defaults to true.
WK_EXPORT void WKPreferencesSetOfflineWebApplicationCacheEnabled(WKPreferencesRef preferences, bool offlineWebApplicationCacheEnabled);
WK_EXPORT bool WKPreferencesGetOfflineWebApplicationCacheEnabled(WKPreferencesRef preferences);
Modified: trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferences.mm (254396 => 254397)
--- trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferences.mm 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferences.mm 2020-01-11 14:57:57 UTC (rev 254397)
@@ -902,6 +902,11 @@
_preferences->setRemotePlaybackEnabled(enabled);
}
+- (BOOL)_isITPDatabaseEnabled
+{
+ return _preferences->isITPDatabaseEnabled();
+}
+
#if PLATFORM(MAC)
- (void)_setJavaEnabledForLocalFiles:(BOOL)enabled
{
Modified: trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferencesPrivate.h (254396 => 254397)
--- trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferencesPrivate.h 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Source/WebKit/UIProcess/API/Cocoa/WKPreferencesPrivate.h 2020-01-11 14:57:57 UTC (rev 254397)
@@ -161,6 +161,7 @@
@property (nonatomic, setter=_setWebAudioEnabled:) BOOL _webAudioEnabled WK_API_AVAILABLE(macos(10.14), ios(WK_IOS_TBA));
@property (nonatomic, setter=_setAcceleratedCompositingEnabled:) BOOL _acceleratedCompositingEnabled WK_API_AVAILABLE(macos(10.13.4), ios(WK_IOS_TBA));
@property (nonatomic, setter=_setRequestAnimationFrameEnabled:) BOOL _requestAnimationFrameEnabled WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
+@property (nonatomic, readonly) BOOL _isITPDatabaseEnabled WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
#if !TARGET_OS_IPHONE
@property (nonatomic, setter=_setWebGLEnabled:) BOOL _webGLEnabled WK_API_AVAILABLE(macos(10.13.4));
Modified: trunk/Source/WebKit/UIProcess/Cocoa/WebProcessPoolCocoa.mm (254396 => 254397)
--- trunk/Source/WebKit/UIProcess/Cocoa/WebProcessPoolCocoa.mm 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Source/WebKit/UIProcess/Cocoa/WebProcessPoolCocoa.mm 2020-01-11 14:57:57 UTC (rev 254397)
@@ -401,7 +401,11 @@
parameters.storageAccessAPIEnabled = storageAccessAPIEnabled();
- parameters.shouldEnableITPDatabase = [defaults boolForKey:[NSString stringWithFormat:@"InternalDebug%@", WebPreferencesKey::isITPDatabaseEnabledKey().createCFString().get()]];
+ NSNumber *databaseEnabledValue = [defaults objectForKey:[NSString stringWithFormat:@"InternalDebug%@", WebPreferencesKey::isITPDatabaseEnabledKey().createCFString().get()]];
+ if (databaseEnabledValue)
+ parameters.shouldEnableITPDatabase = databaseEnabledValue.boolValue;
+ else
+ parameters.shouldEnableITPDatabase = m_defaultPageGroup->preferences().isITPDatabaseEnabled();
parameters.enableAdClickAttributionDebugMode = [defaults boolForKey:[NSString stringWithFormat:@"Experimental%@", WebPreferencesKey::adClickAttributionDebugModeEnabledKey().createCFString().get()]];
}
Modified: trunk/Tools/ChangeLog (254396 => 254397)
--- trunk/Tools/ChangeLog 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Tools/ChangeLog 2020-01-11 14:57:57 UTC (rev 254397)
@@ -1,3 +1,24 @@
+2020-01-11 Kate Cheney <[email protected]>
+
+ Add correct grandfathering functionality to the ITP database backend
+ https://bugs.webkit.org/show_bug.cgi?id=205844
+ <rdar://problem/58360450>
+
+ Reviewed by John Wilander.
+
+ In order to test both the ITP database and memory stores, this patch
+ had to restructure the grandfathering API tests.
+
+ If the test is for the database backend (ends in *Database), it will
+ check if the default store matches. If so, it continues on with the
+ normal test. If not, it has to use an API call to reconstruct the
+ new store and ensure the grandfathering still happens. Vice versa for
+ the memory store tests.
+
+ * TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm:
+ (isITPDatabaseEnabled):
+ (TEST):
+
2020-01-10 Myles C. Maxfield <[email protected]>
REGRESSION(r185816): In the Hong Kong locale, navigator.language reports it's in the Taiwan locale
Modified: trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm (254396 => 254397)
--- trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm 2020-01-11 08:27:55 UTC (rev 254396)
+++ trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/ResourceLoadStatistics.mm 2020-01-11 14:57:57 UTC (rev 254397)
@@ -69,6 +69,19 @@
@end
+bool isITPDatabaseEnabled()
+{
+ NSUserDefaults *defaults = [NSUserDefaults standardUserDefaults];
+ auto configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+ auto defaultDatabaseEnabled = [configuration preferences]._isITPDatabaseEnabled;
+
+ NSNumber *databaseEnabledValue = [defaults objectForKey:@"InternalDebugIsITPDatabaseEnabled"];
+ if (databaseEnabledValue)
+ return databaseEnabledValue.boolValue;
+
+ return defaultDatabaseEnabled;
+}
+
TEST(ResourceLoadStatistics, GrandfatherCallback)
{
auto *dataStore = [WKWebsiteDataStore defaultDataStore];
@@ -83,7 +96,9 @@
static bool grandfatheredFlag;
[dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
- ASSERT_TRUE([message isEqualToString:@"Grandfathered"]);
+ // Only if the database store is not default-enabled do we need to check that this first message is correct.
+ if (!isITPDatabaseEnabled())
+ ASSERT_TRUE([message isEqualToString:@"Grandfathered"]);
grandfatheredFlag = true;
}];
@@ -92,6 +107,25 @@
[dataStore _setResourceLoadStatisticsEnabled:YES];
[webView loadRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:@"about:blank"]]];
+ if (isITPDatabaseEnabled()) {
+ TestWebKitAPI::Util::run(&grandfatheredFlag);
+
+ // Since the database store is enabled, we need to disable it to test this functionality with the plist.
+ // We already have a NetworkProcess up and running with ITP enabled. We now make another call initializing
+ // the memory store and testing grandfathering.
+ grandfatheredFlag = false;
+ [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+ ASSERT_TRUE([message isEqualToString:@"Grandfathered"]);
+ grandfatheredFlag = true;
+ }];
+
+ [dataStore _setUseITPDatabase:false completionHandler: ^(void) {
+ doneFlag = true;
+ }];
+
+ TestWebKitAPI::Util::run(&doneFlag);
+ }
+
TestWebKitAPI::Util::run(&grandfatheredFlag);
// Spin the runloop until the resource load statistics file has written to disk.
@@ -103,6 +137,7 @@
}
grandfatheredFlag = false;
+ doneFlag = false;
[dataStore removeDataOfTypes:[WKWebsiteDataStore _allWebsiteDataTypesIncludingPrivate] modifiedSince:[NSDate distantPast] completionHandler:^ {
doneFlag = true;
}];
@@ -134,6 +169,83 @@
}
}
+TEST(ResourceLoadStatistics, GrandfatherCallbackDatabase)
+{
+ auto *dataStore = [WKWebsiteDataStore defaultDataStore];
+
+ NSURL *statisticsDirectoryURL = [NSURL fileURLWithPath:[@"~/Library/WebKit/TestWebKitAPI/WebsiteData/ResourceLoadStatistics" stringByExpandingTildeInPath] isDirectory:YES];
+ NSURL *fileURL = [statisticsDirectoryURL URLByAppendingPathComponent:@"observations.db"];
+ [[NSFileManager defaultManager] removeItemAtURL:fileURL error:nil];
+ [[NSFileManager defaultManager] removeItemAtURL:statisticsDirectoryURL error:nil];
+ EXPECT_FALSE([[NSFileManager defaultManager] fileExistsAtPath:statisticsDirectoryURL.path]);
+
+ static bool grandfatheredFlag;
+ static bool doneFlag;
+
+ [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+ // Only if the database store is default-enabled do we need to check that this first message is correct.
+ if (isITPDatabaseEnabled())
+ ASSERT_TRUE([message isEqualToString:@"Grandfathered"]);
+ grandfatheredFlag = true;
+ }];
+
+ auto webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600)]);
+ [dataStore _setResourceLoadStatisticsEnabled:YES];
+ [webView loadRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:@"about:blank"]]];
+
+ if (!isITPDatabaseEnabled()) {
+ TestWebKitAPI::Util::run(&grandfatheredFlag);
+
+ // Since the database store is not enabled, we need to update the store to use a database instead of a plist for this test.
+ // We already have a NetworkProcess up and running with ITP enabled. We now make another call initializing
+ // the database store and testing grandfathering.
+ grandfatheredFlag = false;
+ [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+ ASSERT_TRUE([message isEqualToString:@"Grandfathered"]);
+ grandfatheredFlag = true;
+ }];
+
+ [dataStore _setUseITPDatabase:true completionHandler: ^(void) {
+ doneFlag = true;
+ }];
+
+ TestWebKitAPI::Util::run(&doneFlag);
+ }
+
+ TestWebKitAPI::Util::run(&grandfatheredFlag);
+
+ // Spin the runloop until the resource load statistics file has written to disk.
+ // If the test enters a spin loop here, it has failed.
+ while (true) {
+ if ([[NSFileManager defaultManager] fileExistsAtPath:fileURL.path])
+ break;
+ TestWebKitAPI::Util::spinRunLoop(1);
+ }
+
+ grandfatheredFlag = false;
+ doneFlag = false;
+ [dataStore removeDataOfTypes:[WKWebsiteDataStore _allWebsiteDataTypesIncludingPrivate] modifiedSince:[NSDate distantPast] completionHandler:^ {
+ doneFlag = true;
+ }];
+
+ TestWebKitAPI::Util::run(&doneFlag);
+ TestWebKitAPI::Util::spinRunLoop(10);
+
+ // The website data store remove should have completed, but since we removed all of the data types that are monitored by resource load statistics,
+ // no grandfathering call should have been made. Note that the database file will not be deleted like in the persistent storage case, only cleared.
+ EXPECT_FALSE(grandfatheredFlag);
+
+ doneFlag = false;
+ [dataStore removeDataOfTypes:[NSSet setWithObjects:WKWebsiteDataTypeCookies, _WKWebsiteDataTypeResourceLoadStatistics, nil] modifiedSince:[NSDate distantPast] completionHandler:^ {
+ doneFlag = true;
+ }];
+
+ // Since we did not remove every data type covered by resource load statistics, we do expect that grandfathering took place again.
+ // If the test hangs waiting on either of these conditions, it has failed.
+ TestWebKitAPI::Util::run(&grandfatheredFlag);
+ TestWebKitAPI::Util::run(&doneFlag);
+}
+
TEST(ResourceLoadStatistics, ShouldNotGrandfatherOnStartup)
{
auto *dataStore = [WKWebsiteDataStore defaultDataStore];
@@ -148,8 +260,12 @@
EXPECT_TRUE([[NSFileManager defaultManager] fileExistsAtPath:targetURL.path]);
static bool callbackFlag;
+ static bool doneFlag;
+
[dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
- ASSERT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
+ // Only if the database store is not default-enabled do we need to check that this first message is correct.
+ if (!isITPDatabaseEnabled())
+ ASSERT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
callbackFlag = true;
}];
@@ -158,9 +274,78 @@
[dataStore _setResourceLoadStatisticsEnabled:YES];
[webView loadRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:@"about:blank"]]];
+ if (isITPDatabaseEnabled()) {
+ TestWebKitAPI::Util::run(&callbackFlag);
+
+ // Since the database store is enabled, we need to disable it to test this functionality with the plist.
+ // We already have a NetworkProcess up and running with ITP enabled. We now make another call initializing
+ // the memory store and testing grandfathering.
+ callbackFlag = false;
+ [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+ ASSERT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
+ callbackFlag = true;
+ }];
+
+ [dataStore _setUseITPDatabase:false completionHandler: ^(void) {
+ doneFlag = true;
+ }];
+
+ TestWebKitAPI::Util::run(&doneFlag);
+ }
+
TestWebKitAPI::Util::run(&callbackFlag);
}
+TEST(ResourceLoadStatistics, ShouldNotGrandfatherOnStartupDatabase)
+{
+ auto *dataStore = [WKWebsiteDataStore defaultDataStore];
+
+ NSURL *statisticsDirectoryURL = [NSURL fileURLWithPath:[@"~/Library/WebKit/TestWebKitAPI/WebsiteData/ResourceLoadStatistics" stringByExpandingTildeInPath] isDirectory:YES];
+ NSURL *targetURL = [statisticsDirectoryURL URLByAppendingPathComponent:@"observations.db"];
+ NSURL *testResourceURL = [[NSBundle mainBundle] URLForResource:@"EmptyGrandfatheredResourceLoadStatistics" withExtension:@"plist" subdirectory:@"TestWebKitAPI.resources"];
+
+ [[NSFileManager defaultManager] createDirectoryAtURL:statisticsDirectoryURL withIntermediateDirectories:YES attributes:nil error:nil];
+ [[NSFileManager defaultManager] copyItemAtURL:testResourceURL toURL:targetURL error:nil];
+
+ EXPECT_TRUE([[NSFileManager defaultManager] fileExistsAtPath:targetURL.path]);
+
+ static bool callbackFlag;
+ static bool doneFlag;
+
+ [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+ // Only if the database store is default-enabled do we need to check that this first message is correct.
+ if (isITPDatabaseEnabled())
+ ASSERT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
+ callbackFlag = true;
+ }];
+
+ // We need an active NetworkProcess to perform ResourceLoadStatistics operations.
+ auto webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600)]);
+ [dataStore _setResourceLoadStatisticsEnabled:YES];
+ [webView loadRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:@"about:blank"]]];
+
+ if (!isITPDatabaseEnabled()) {
+ TestWebKitAPI::Util::run(&callbackFlag);
+
+ // Since the database store is not enabled, we need to update the store to use a database instead of a plist for this test.
+ // We already have a NetworkProcess up and running with ITP enabled. We now make another call initializing
+ // the database store and testing grandfathering.
+ callbackFlag = false;
+ [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+ ASSERT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
+ callbackFlag = true;
+ }];
+
+ [dataStore _setUseITPDatabase:true completionHandler: ^(void) {
+ doneFlag = true;
+ }];
+
+ TestWebKitAPI::Util::run(&doneFlag);
+ }
+
+ TestWebKitAPI::Util::run(&callbackFlag);
+}
+
TEST(ResourceLoadStatistics, ChildProcessesNotLaunched)
{
// Ensure the shared process pool exists so the data store operations we're about to do work with it.
@@ -176,13 +361,78 @@
[[NSFileManager defaultManager] createDirectoryAtURL:statisticsDirectoryURL withIntermediateDirectories:YES attributes:nil error:nil];
[[NSFileManager defaultManager] copyItemAtURL:testResourceURL toURL:targetURL error:nil];
+
+ EXPECT_TRUE([[NSFileManager defaultManager] fileExistsAtPath:targetURL.path]);
+ static bool callbackFlag;
+ static bool doneFlag;
+ [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+ // Only if the database store is not default-enabled do we need to check that this first message is correct.
+ if (!isITPDatabaseEnabled())
+ EXPECT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
+ callbackFlag = true;
+ }];
+
+ // We need an active NetworkProcess to perform ResourceLoadStatistics operations.
+ auto webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600)]);
+ [dataStore _setResourceLoadStatisticsEnabled:YES];
+ [webView loadRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:@"about:blank"]]];
+
+ if (isITPDatabaseEnabled()) {
+ TestWebKitAPI::Util::run(&callbackFlag);
+
+ // Since the database store is enabled, we need to disable it to test this functionality with the plist.
+ // We already have a NetworkProcess up and running with ITP enabled. We now make another call initializing
+ // the memory store and testing grandfathering.
+ callbackFlag = false;
+ [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+ ASSERT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
+ callbackFlag = true;
+ }];
+
+ [dataStore _setUseITPDatabase:false completionHandler: ^(void) {
+ doneFlag = true;
+ }];
+
+ TestWebKitAPI::Util::run(&doneFlag);
+ }
+
+ TestWebKitAPI::Util::run(&callbackFlag);
+
EXPECT_TRUE([[NSFileManager defaultManager] fileExistsAtPath:targetURL.path]);
+ webView.clear();
+
+ EXPECT_EQ((size_t)0, [sharedProcessPool _pluginProcessCount]);
+}
+
+TEST(ResourceLoadStatistics, ChildProcessesNotLaunchedDatabase)
+{
+ // Ensure the shared process pool exists so the data store operations we're about to do work with it.
+ WKProcessPool *sharedProcessPool = [WKProcessPool _sharedProcessPool];
+
+ EXPECT_EQ((size_t)0, [sharedProcessPool _pluginProcessCount]);
+
+ auto *dataStore = [WKWebsiteDataStore defaultDataStore];
+
+ NSURL *statisticsDirectoryURL = [NSURL fileURLWithPath:[@"~/Library/WebKit/TestWebKitAPI/WebsiteData/ResourceLoadStatistics" stringByExpandingTildeInPath] isDirectory:YES];
+ NSURL *targetURL = [statisticsDirectoryURL URLByAppendingPathComponent:@"observations.db"];
+
+ NSURL *testResourceURL = [[NSBundle mainBundle] URLForResource:@"EmptyGrandfatheredResourceLoadStatistics" withExtension:@"plist" subdirectory:@"TestWebKitAPI.resources"];
+
+ [[NSFileManager defaultManager] createDirectoryAtURL:statisticsDirectoryURL withIntermediateDirectories:YES attributes:nil error:nil];
+ [[NSFileManager defaultManager] copyItemAtURL:testResourceURL toURL:targetURL error:nil];
+
+ EXPECT_TRUE([[NSFileManager defaultManager] fileExistsAtPath:targetURL.path]);
+
+ static bool callbackFlag;
static bool doneFlag;
+
[dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
- EXPECT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
- doneFlag = true;
+ // Only if the database store is default-enabled do we need to check that this first message is correct.
+ if (isITPDatabaseEnabled())
+ EXPECT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
+ callbackFlag = true;
}];
// We need an active NetworkProcess to perform ResourceLoadStatistics operations.
@@ -190,8 +440,27 @@
[dataStore _setResourceLoadStatisticsEnabled:YES];
[webView loadRequest:[NSURLRequest requestWithURL:[NSURL URLWithString:@"about:blank"]]];
- TestWebKitAPI::Util::run(&doneFlag);
+ if (!isITPDatabaseEnabled()) {
+ TestWebKitAPI::Util::run(&callbackFlag);
+ // Since the database store is not enabled, we need to update the store to use a database instead of a plist for this test.
+ // We already have a NetworkProcess up and running with ITP enabled. We now make another call initializing
+ // the database store and testing grandfathering.
+ callbackFlag = false;
+ [dataStore _setResourceLoadStatisticsTestingCallback:^(WKWebsiteDataStore *, NSString *message) {
+ ASSERT_TRUE([message isEqualToString:@"PopulatedWithoutGrandfathering"]);
+ callbackFlag = true;
+ }];
+
+ [dataStore _setUseITPDatabase:true completionHandler: ^(void) {
+ doneFlag = true;
+ }];
+
+ TestWebKitAPI::Util::run(&doneFlag);
+ }
+
+ TestWebKitAPI::Util::run(&callbackFlag);
+
EXPECT_TRUE([[NSFileManager defaultManager] fileExistsAtPath:targetURL.path]);
webView.clear();
