- Revision
- 190257
- Author
- [email protected]
- Date
- 2015-09-25 15:36:09 -0700 (Fri, 25 Sep 2015)
Log Message
[Mac] Media tests start to time out after a few days of bot uptime
https://bugs.webkit.org/show_bug.cgi?id=149468
Reviewed by Darin Adler.
Source/WebKit2:
Make the fix work with WebKit2, and avoid touching user's media cache for WebKit1.
When UI process is not sandboxed, inherit DIRHELPER_USER_DIR_SUFFIX from it.
This means that during testing, WebContent uses a temporary directory of the form
/private/var/folders/.../T/WebKitTestRunner-VKjmz0/com.apple.WebKit.WebContent.Development,
instead of /private/var/folders/.../T/com.apple.WebKit.WebContent.Development+WebKitTestRunner.
WebKit1 now also better respects a temporary directory of the form
/private/var/folders/.../T/DumpRenderTree-VKjmz0, instead of the default, which
makes CoreMedia put its cache inside it.
As a result, almost everything is isolated per testing process, and deleted at the end.
Same with cache directories. The only outliers that I know of are plug-in directories,
which are per-process, but are not under the testing root, and thus aren't deleted.
* PluginProcess/mac/PluginProcessMac.mm:
(WebKit::PluginProcess::initializeSandbox):
* Shared/ChildProcessProxy.cpp:
(WebKit::ChildProcessProxy::fromConnection):
(WebKit::ChildProcessProxy::getLaunchOptions):
(WebKit::ChildProcessProxy::connect):
* Shared/ChildProcessProxy.h:
* Shared/EntryPointUtilities/mac/XPCService/XPCServiceEntryPoint.mm:
(WebKit::XPCServiceInitializerDelegate::getExtraInitializationData):
* Shared/SandboxInitializationParameters.h:
(WebKit::SandboxInitializationParameters::sandboxProfile):
(WebKit::SandboxInitializationParameters::setUserDirectorySuffix):
(WebKit::SandboxInitializationParameters::userDirectorySuffix):
(WebKit::SandboxInitializationParameters::setSystemDirectorySuffix): Deleted.
(WebKit::SandboxInitializationParameters::systemDirectorySuffix): Deleted.
* Shared/ios/ChildProcessIOS.mm:
(WebKit::ChildProcess::initializeSandbox):
* Shared/mac/ChildProcessMac.mm:
(WebKit::ChildProcess::initializeSandbox):
* UIProcess/Databases/DatabaseProcessProxy.cpp:
(WebKit::DatabaseProcessProxy::getLaunchOptions):
* UIProcess/Network/NetworkProcessProxy.cpp:
(WebKit::NetworkProcessProxy::getLaunchOptions):
* UIProcess/Plugins/PluginProcessProxy.cpp:
(WebKit::PluginProcessProxy::getLaunchOptions):
* UIProcess/WebProcessProxy.cpp:
(WebKit::WebProcessProxy::getLaunchOptions):
Tools:
Delete cache directory in addition to the temporary one. Don't delete /private/tmp/MediaCache,
as it's not longer used by tests. Set TMPDIR and DIRHELPER_USER_DIR_SUFFIX to make
all temporary and cache files go to a custom location.
* Scripts/webkitpy/port/base.py:
* Scripts/webkitpy/port/driver.py:
* Scripts/webkitpy/port/mac.py:
* Scripts/webkitpy/port/win.py:
(WinPort._driver_tempdir_for_environment): Deleted unused function.
Modified Paths
Diff
Modified: trunk/Source/WebKit2/ChangeLog (190256 => 190257)
--- trunk/Source/WebKit2/ChangeLog 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/ChangeLog 2015-09-25 22:36:09 UTC (rev 190257)
@@ -1,3 +1,53 @@
+2015-09-25 Alexey Proskuryakov <[email protected]>
+
+ [Mac] Media tests start to time out after a few days of bot uptime
+ https://bugs.webkit.org/show_bug.cgi?id=149468
+
+ Reviewed by Darin Adler.
+
+ Make the fix work with WebKit2, and avoid touching user's media cache for WebKit1.
+
+ When UI process is not sandboxed, inherit DIRHELPER_USER_DIR_SUFFIX from it.
+ This means that during testing, WebContent uses a temporary directory of the form
+ /private/var/folders/.../T/WebKitTestRunner-VKjmz0/com.apple.WebKit.WebContent.Development,
+ instead of /private/var/folders/.../T/com.apple.WebKit.WebContent.Development+WebKitTestRunner.
+
+ WebKit1 now also better respects a temporary directory of the form
+ /private/var/folders/.../T/DumpRenderTree-VKjmz0, instead of the default, which
+ makes CoreMedia put its cache inside it.
+
+ As a result, almost everything is isolated per testing process, and deleted at the end.
+ Same with cache directories. The only outliers that I know of are plug-in directories,
+ which are per-process, but are not under the testing root, and thus aren't deleted.
+
+ * PluginProcess/mac/PluginProcessMac.mm:
+ (WebKit::PluginProcess::initializeSandbox):
+ * Shared/ChildProcessProxy.cpp:
+ (WebKit::ChildProcessProxy::fromConnection):
+ (WebKit::ChildProcessProxy::getLaunchOptions):
+ (WebKit::ChildProcessProxy::connect):
+ * Shared/ChildProcessProxy.h:
+ * Shared/EntryPointUtilities/mac/XPCService/XPCServiceEntryPoint.mm:
+ (WebKit::XPCServiceInitializerDelegate::getExtraInitializationData):
+ * Shared/SandboxInitializationParameters.h:
+ (WebKit::SandboxInitializationParameters::sandboxProfile):
+ (WebKit::SandboxInitializationParameters::setUserDirectorySuffix):
+ (WebKit::SandboxInitializationParameters::userDirectorySuffix):
+ (WebKit::SandboxInitializationParameters::setSystemDirectorySuffix): Deleted.
+ (WebKit::SandboxInitializationParameters::systemDirectorySuffix): Deleted.
+ * Shared/ios/ChildProcessIOS.mm:
+ (WebKit::ChildProcess::initializeSandbox):
+ * Shared/mac/ChildProcessMac.mm:
+ (WebKit::ChildProcess::initializeSandbox):
+ * UIProcess/Databases/DatabaseProcessProxy.cpp:
+ (WebKit::DatabaseProcessProxy::getLaunchOptions):
+ * UIProcess/Network/NetworkProcessProxy.cpp:
+ (WebKit::NetworkProcessProxy::getLaunchOptions):
+ * UIProcess/Plugins/PluginProcessProxy.cpp:
+ (WebKit::PluginProcessProxy::getLaunchOptions):
+ * UIProcess/WebProcessProxy.cpp:
+ (WebKit::WebProcessProxy::getLaunchOptions):
+
2015-09-25 Beth Dakin <[email protected]>
Clicking on a data detected item inside a form control always pops up a map
Modified: trunk/Source/WebKit2/PluginProcess/mac/PluginProcessMac.mm (190256 => 190257)
--- trunk/Source/WebKit2/PluginProcess/mac/PluginProcessMac.mm 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/PluginProcess/mac/PluginProcessMac.mm 2015-09-25 22:36:09 UTC (rev 190257)
@@ -508,7 +508,7 @@
exit(EX_OSERR);
}
- sandboxParameters.setSystemDirectorySuffix([[[[NSFileManager defaultManager] stringWithFileSystemRepresentation:temporaryDirectory length:strlen(temporaryDirectory)] lastPathComponent] fileSystemRepresentation]);
+ sandboxParameters.setUserDirectorySuffix([[[[NSFileManager defaultManager] stringWithFileSystemRepresentation:temporaryDirectory length:strlen(temporaryDirectory)] lastPathComponent] fileSystemRepresentation]);
sandboxParameters.addPathParameter("PLUGIN_PATH", m_pluginPath);
sandboxParameters.addPathParameter("NSURL_CACHE_DIR", m_nsurlCacheDirectory);
Modified: trunk/Source/WebKit2/Shared/ChildProcessProxy.cpp (190256 => 190257)
--- trunk/Source/WebKit2/Shared/ChildProcessProxy.cpp 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/Shared/ChildProcessProxy.cpp 2015-09-25 22:36:09 UTC (rev 190257)
@@ -56,6 +56,12 @@
return childProcessProxy;
}
+void ChildProcessProxy::getLaunchOptions(ProcessLauncher::LaunchOptions& launchOptions)
+{
+ if (const char* userDirectorySuffix = getenv("DIRHELPER_USER_DIR_SUFFIX"))
+ launchOptions.extraInitializationData.add(ASCIILiteral("user-directory-suffix"), userDirectorySuffix);
+}
+
void ChildProcessProxy::connect()
{
ASSERT(!m_processLauncher);
Modified: trunk/Source/WebKit2/Shared/ChildProcessProxy.h (190256 => 190257)
--- trunk/Source/WebKit2/Shared/ChildProcessProxy.h 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/Shared/ChildProcessProxy.h 2015-09-25 22:36:09 UTC (rev 190257)
@@ -82,8 +82,9 @@
bool dispatchMessage(IPC::Connection&, IPC::MessageDecoder&);
bool dispatchSyncMessage(IPC::Connection&, IPC::MessageDecoder&, std::unique_ptr<IPC::MessageEncoder>&);
+ virtual void getLaunchOptions(ProcessLauncher::LaunchOptions&);
+
private:
- virtual void getLaunchOptions(ProcessLauncher::LaunchOptions&) = 0;
virtual void connectionWillOpen(IPC::Connection&);
virtual void processWillShutDown(IPC::Connection&) = 0;
Modified: trunk/Source/WebKit2/Shared/EntryPointUtilities/mac/XPCService/XPCServiceEntryPoint.mm (190256 => 190257)
--- trunk/Source/WebKit2/Shared/EntryPointUtilities/mac/XPCService/XPCServiceEntryPoint.mm 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/Shared/EntryPointUtilities/mac/XPCService/XPCServiceEntryPoint.mm 2015-09-25 22:36:09 UTC (rev 190257)
@@ -97,6 +97,12 @@
if (!inspectorProcess.isEmpty())
extraInitializationData.add(ASCIILiteral("inspector-process"), inspectorProcess);
+ if (!isClientSandboxed()) {
+ String userDirectorySuffix = xpc_dictionary_get_string(extraDataInitializationDataObject, "user-directory-suffix");
+ if (!userDirectorySuffix.isEmpty())
+ extraInitializationData.add(ASCIILiteral("user-directory-suffix"), userDirectorySuffix);
+ }
+
return true;
}
Modified: trunk/Source/WebKit2/Shared/SandboxInitializationParameters.h (190256 => 190257)
--- trunk/Source/WebKit2/Shared/SandboxInitializationParameters.h 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/Shared/SandboxInitializationParameters.h 2015-09-25 22:36:09 UTC (rev 190257)
@@ -86,8 +86,8 @@
return m_overrideSandboxProfilePathOrSandboxProfile;
}
- void setSystemDirectorySuffix(const String& suffix) { m_systemDirectorySuffix = suffix; }
- const String& systemDirectorySuffix() const { return m_systemDirectorySuffix; }
+ void setUserDirectorySuffix(const String& suffix) { m_userDirectorySuffix = suffix; }
+ const String& userDirectorySuffix() const { return m_userDirectorySuffix; }
#endif
private:
@@ -95,7 +95,7 @@
void appendPathInternal(const char* name, const char* path);
mutable Vector<const char*> m_namedParameters;
- String m_systemDirectorySuffix;
+ String m_userDirectorySuffix;
ProfileSelectionMode m_profileSelectionMode;
String m_overrideSandboxProfilePathOrSandboxProfile;
Modified: trunk/Source/WebKit2/Shared/ios/ChildProcessIOS.mm (190256 => 190257)
--- trunk/Source/WebKit2/Shared/ios/ChildProcessIOS.mm 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/Shared/ios/ChildProcessIOS.mm 2015-09-25 22:36:09 UTC (rev 190257)
@@ -73,9 +73,9 @@
#if ENABLE(MANUAL_SANDBOXING)
NSBundle *webkit2Bundle = [NSBundle bundleForClass:NSClassFromString(@"WKWebView")];
String defaultProfilePath = [webkit2Bundle pathForResource:[[NSBundle mainBundle] bundleIdentifier] ofType:@"sb"];
- if (sandboxParameters.systemDirectorySuffix().isNull()) {
- String defaultSystemDirectorySuffix = String([[NSBundle mainBundle] bundleIdentifier]) + "+" + parameters.clientIdentifier;
- sandboxParameters.setSystemDirectorySuffix(defaultSystemDirectorySuffix);
+ if (sandboxParameters.userDirectorySuffix().isNull()) {
+ String defaultUserDirectorySuffix = makeString(String([[NSBundle mainBundle] bundleIdentifier]), '+', parameters.clientIdentifier);
+ sandboxParameters.setUserDirectorySuffix(defaultUserDirectorySuffix);
}
String sandboxImportPath = "/usr/local/share/sandbox/imports";
Modified: trunk/Source/WebKit2/Shared/mac/ChildProcessMac.mm (190256 => 190257)
--- trunk/Source/WebKit2/Shared/mac/ChildProcessMac.mm 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/Shared/mac/ChildProcessMac.mm 2015-09-25 22:36:09 UTC (rev 190257)
@@ -90,9 +90,14 @@
NSBundle *webkit2Bundle = [NSBundle bundleForClass:NSClassFromString(@"WKView")];
String defaultProfilePath = [webkit2Bundle pathForResource:[[NSBundle mainBundle] bundleIdentifier] ofType:@"sb"];
- if (sandboxParameters.systemDirectorySuffix().isNull()) {
- String defaultSystemDirectorySuffix = String([[NSBundle mainBundle] bundleIdentifier]) + "+" + parameters.clientIdentifier;
- sandboxParameters.setSystemDirectorySuffix(defaultSystemDirectorySuffix);
+ if (sandboxParameters.userDirectorySuffix().isNull()) {
+ auto userDirectorySuffix = parameters.extraInitializationData.find("user-directory-suffix");
+ if (userDirectorySuffix != parameters.extraInitializationData.end())
+ sandboxParameters.setUserDirectorySuffix([makeString(userDirectorySuffix->value, '/', String([[NSBundle mainBundle] bundleIdentifier])) fileSystemRepresentation]);
+ else {
+ String defaultUserDirectorySuffix = makeString(String([[NSBundle mainBundle] bundleIdentifier]), '+', parameters.clientIdentifier);
+ sandboxParameters.setUserDirectorySuffix(defaultUserDirectorySuffix);
+ }
}
Vector<String> osVersionParts;
@@ -106,7 +111,7 @@
sandboxParameters.addParameter("_OS_VERSION", osVersion.utf8().data());
// Use private temporary and cache directories.
- setenv("DIRHELPER_USER_DIR_SUFFIX", fileSystemRepresentation(sandboxParameters.systemDirectorySuffix()).data(), 0);
+ setenv("DIRHELPER_USER_DIR_SUFFIX", fileSystemRepresentation(sandboxParameters.userDirectorySuffix()).data(), 1);
char temporaryDirectory[PATH_MAX];
if (!confstr(_CS_DARWIN_USER_TEMP_DIR, temporaryDirectory, sizeof(temporaryDirectory))) {
WTFLogAlways("%s: couldn't retrieve private temporary directory path: %d\n", getprogname(), errno);
Modified: trunk/Source/WebKit2/UIProcess/Databases/DatabaseProcessProxy.cpp (190256 => 190257)
--- trunk/Source/WebKit2/UIProcess/Databases/DatabaseProcessProxy.cpp 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/UIProcess/Databases/DatabaseProcessProxy.cpp 2015-09-25 22:36:09 UTC (rev 190257)
@@ -67,6 +67,7 @@
void DatabaseProcessProxy::getLaunchOptions(ProcessLauncher::LaunchOptions& launchOptions)
{
launchOptions.processType = ProcessLauncher::DatabaseProcess;
+ ChildProcessProxy::getLaunchOptions(launchOptions);
platformGetLaunchOptions(launchOptions);
}
Modified: trunk/Source/WebKit2/UIProcess/Network/NetworkProcessProxy.cpp (190256 => 190257)
--- trunk/Source/WebKit2/UIProcess/Network/NetworkProcessProxy.cpp 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/UIProcess/Network/NetworkProcessProxy.cpp 2015-09-25 22:36:09 UTC (rev 190257)
@@ -83,6 +83,7 @@
void NetworkProcessProxy::getLaunchOptions(ProcessLauncher::LaunchOptions& launchOptions)
{
launchOptions.processType = ProcessLauncher::NetworkProcess;
+ ChildProcessProxy::getLaunchOptions(launchOptions);
platformGetLaunchOptions(launchOptions);
}
Modified: trunk/Source/WebKit2/UIProcess/Plugins/PluginProcessProxy.cpp (190256 => 190257)
--- trunk/Source/WebKit2/UIProcess/Plugins/PluginProcessProxy.cpp 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/UIProcess/Plugins/PluginProcessProxy.cpp 2015-09-25 22:36:09 UTC (rev 190257)
@@ -85,6 +85,7 @@
void PluginProcessProxy::getLaunchOptions(ProcessLauncher::LaunchOptions& launchOptions)
{
launchOptions.processType = ProcessLauncher::PluginProcess;
+ ChildProcessProxy::getLaunchOptions(launchOptions);
platformGetLaunchOptions(launchOptions, m_pluginProcessAttributes);
}
Modified: trunk/Source/WebKit2/UIProcess/WebProcessProxy.cpp (190256 => 190257)
--- trunk/Source/WebKit2/UIProcess/WebProcessProxy.cpp 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Source/WebKit2/UIProcess/WebProcessProxy.cpp 2015-09-25 22:36:09 UTC (rev 190257)
@@ -130,6 +130,8 @@
{
launchOptions.processType = ProcessLauncher::WebProcess;
+ ChildProcessProxy::getLaunchOptions(launchOptions);
+
if (WebInspectorProxy::isInspectorProcessPool(m_processPool))
launchOptions.extraInitializationData.add(ASCIILiteral("inspector-process"), ASCIILiteral("1"));
Modified: trunk/Tools/ChangeLog (190256 => 190257)
--- trunk/Tools/ChangeLog 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Tools/ChangeLog 2015-09-25 22:36:09 UTC (rev 190257)
@@ -1,3 +1,20 @@
+2015-09-25 Alexey Proskuryakov <[email protected]>
+
+ [Mac] Media tests start to time out after a few days of bot uptime
+ https://bugs.webkit.org/show_bug.cgi?id=149468
+
+ Reviewed by Darin Adler.
+
+ Delete cache directory in addition to the temporary one. Don't delete /private/tmp/MediaCache,
+ as it's not longer used by tests. Set TMPDIR and DIRHELPER_USER_DIR_SUFFIX to make
+ all temporary and cache files go to a custom location.
+
+ * Scripts/webkitpy/port/base.py:
+ * Scripts/webkitpy/port/driver.py:
+ * Scripts/webkitpy/port/mac.py:
+ * Scripts/webkitpy/port/win.py:
+ (WinPort._driver_tempdir_for_environment): Deleted unused function.
+
2015-09-25 Dan Bernstein <[email protected]>
Removed MiniBrowser.entitlements.
Modified: trunk/Tools/Scripts/webkitpy/port/base.py (190256 => 190257)
--- trunk/Tools/Scripts/webkitpy/port/base.py 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Tools/Scripts/webkitpy/port/base.py 2015-09-25 22:36:09 UTC (rev 190257)
@@ -1154,8 +1154,8 @@
def _driver_tempdir(self):
return self._filesystem.mkdtemp(prefix='%s-' % self.driver_name())
- def _driver_tempdir_for_environment(self):
- return self._driver_tempdir()
+ def remove_cache_directory(self, name):
+ pass
def _path_to_webcore_library(self):
"""Returns the full path to a built copy of WebCore."""
Modified: trunk/Tools/Scripts/webkitpy/port/driver.py (190256 => 190257)
--- trunk/Tools/Scripts/webkitpy/port/driver.py 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Tools/Scripts/webkitpy/port/driver.py 2015-09-25 22:36:09 UTC (rev 190257)
@@ -313,10 +313,11 @@
def _setup_environ_for_driver(self, environment):
environment['DYLD_LIBRARY_PATH'] = self._port._build_path()
environment['DYLD_FRAMEWORK_PATH'] = self._port._build_path()
- # FIXME: We're assuming that WebKitTestRunner checks this DumpRenderTree-named environment variable.
- # FIXME: Commented out for now to avoid tests breaking. Re-enable after
- # we cut over to NRWT
- #environment['DUMPRENDERTREE_TEMP'] = str(self._port._driver_tempdir_for_environment())
+ # Use an isolated temp directory that can be deleted after testing (especially important on Mac, as
+ # CoreMedia disk cache is in the temp directory).
+ environment['TMPDIR'] = str(self._driver_tempdir)
+ environment['DIRHELPER_USER_DIR_SUFFIX'] = os.path.basename(str(self._driver_tempdir))
+ # Put certain normally persistent files into the temp directory (e.g. IndexedDB storage).
environment['DUMPRENDERTREE_TEMP'] = str(self._driver_tempdir)
environment['LOCAL_RESOURCE_ROOT'] = self._port.layout_tests_dir()
environment['ASAN_OPTIONS'] = "allocator_may_return_null=1"
@@ -358,6 +359,7 @@
if self._driver_tempdir:
self._port._filesystem.rmtree(str(self._driver_tempdir))
+ self._port.remove_cache_directory(os.path.basename(str(self._driver_tempdir)))
self._driver_tempdir = None
def cmd_line(self, pixel_tests, per_test_args):
Modified: trunk/Tools/Scripts/webkitpy/port/mac.py (190256 => 190257)
--- trunk/Tools/Scripts/webkitpy/port/mac.py 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Tools/Scripts/webkitpy/port/mac.py 2015-09-25 22:36:09 UTC (rev 190257)
@@ -108,12 +108,14 @@
return env
def _clear_global_caches_and_temporary_files(self):
- self._filesystem.rmtree('/private/tmp/MediaCache')
self._filesystem.rmtree(os.path.expanduser('~/Library/' + self.driver_name()))
self._filesystem.rmtree(os.path.expanduser('~/Library/Application Support/' + self.driver_name()))
self._filesystem.rmtree(os.path.expanduser('~/Library/Caches/' + self.driver_name()))
self._filesystem.rmtree(os.path.expanduser('~/Library/WebKit/' + self.driver_name()))
+ def remove_cache_directory(self, name):
+ self._filesystem.rmtree(os.confstr(65538) + name)
+
def operating_system(self):
return 'mac'
Modified: trunk/Tools/Scripts/webkitpy/port/win.py (190256 => 190257)
--- trunk/Tools/Scripts/webkitpy/port/win.py 2015-09-25 22:06:09 UTC (rev 190256)
+++ trunk/Tools/Scripts/webkitpy/port/win.py 2015-09-25 22:36:09 UTC (rev 190257)
@@ -138,9 +138,6 @@
def _path_to_lighttpd_php(self):
return "/usr/bin/php-cgi"
- def _driver_tempdir_for_environment(self):
- return cygpath(self._driver_tempdir())
-
def test_search_path(self):
test_fallback_names = [path for path in self.baseline_search_path() if not path.startswith(self._webkit_baseline_path('mac'))]
return map(self._webkit_baseline_path, test_fallback_names)