Diff
Modified: trunk/Websites/perf.webkit.org/ChangeLog (190644 => 190645)
--- trunk/Websites/perf.webkit.org/ChangeLog 2015-10-06 20:57:22 UTC (rev 190644)
+++ trunk/Websites/perf.webkit.org/ChangeLog 2015-10-06 21:04:03 UTC (rev 190645)
@@ -1,3 +1,28 @@
+2015-10-06 Ryosuke Niwa <rn...@webkit.org>
+
+ Perf dashboard should have the capability to test local UI with production data
+ https://bugs.webkit.org/show_bug.cgi?id=149834
+
+ Reviewed by Chris Dumez.
+
+ Added tools/run-with-remote-server.py which runs a local httpd server and pulls data from a remote server.
+
+ * Install.md: Added the instruction on how to use the script. Also updated the remaining instructions
+ for El Capitan.
+ * config.json: Added remote server configurations.
+ * public/admin/fetch-from-remote.php: Added. This script fetches JSON from the remote server specified in
+ config.json and caches the results in the location specified as "cacheDirectory" in config.json.
+ (main):
+ (fetch_remote):
+ * public/include/db.php:
+ (config_path): Extracted from generate_data_file.
+ (generate_data_file):
+ * tools/remote-server-relay.conf: Added. Apache 2.4 configuration file for a local http server launched by
+ run-with-remote-server.py.
+ * tools/run-with-remote-server.py: Added. Launches Apache with the right set of directives.
+ (main):
+ (abspath_from_root):
+
2015-07-13 Ryosuke Niwa <rn...@webkit.org>
Fix a typo.
Modified: trunk/Websites/perf.webkit.org/Install.md (190644 => 190645)
--- trunk/Websites/perf.webkit.org/Install.md 2015-10-06 20:57:22 UTC (rev 190644)
+++ trunk/Websites/perf.webkit.org/Install.md 2015-10-06 21:04:03 UTC (rev 190645)
@@ -1,7 +1,6 @@
# Checking Out the Code and Installing Required Applications
-The instructions assume you're using Mac OS X (Mavericks for Server.app case and Mountain Lion without Server.app) as the
-host server, and assume that we're installing this application at `/Volumes/Data/perf.webkit.org`.
+The instructions assume you're using Mac OS X as the host server and installing this application at `/Volumes/Data/perf.webkit.org`.
You can choose between using Server.app or install the required tools separately
@@ -9,9 +8,35 @@
2. Install node.
3. Install Xcode with command line tools (only needed for svn)
4. `svn co https://svn.webkit.org/repository/webkit/trunk/Websites/perf.webkit.org /Volumes/Data/perf.webkit.org`
-5. Inside `/Volumes/Data/perf.webkit.org`, run `npm install pg`.
+5. Inside `/Volumes/Data/perf.webkit.org`, run `npm install pg` and `mkdir -m 755 public/data/`
+# Testing Local UI Changes with Production Data
+The front end has the capability to pull data from a production server without replicating the database locally on OS X (Yosemite and later).
+To use this feature, modify `config.json`'s `remoteServer` entry so that "remoteServer.url" points to your production server,
+and "remoteServer.basicAuth" specifies the username and the password that is used by the production sever.
+
+Remove "basicAuth" entry for production servers that doesn't require a basic authentication (e.g. perf.webkit.org).
+
+```json
+{
+ "url": "http://perf.webkit.org",
+ "basicAuth": {
+ "username": "webkitten",
+ "password": "webkitten's secret password"
+ }
+}
+```
+
+Then run `tools/run-with-remote-server.py`. This launches a httpd server on port 8080.
+
+The initial few page loads after starting the script could take as much as a few minutes depending on your production sever's configurations
+since Apache needs to start a pool of processes. Reloading the dashboards few times should bring the load time under control.
+
+The script caches remote server's responses under `public/data/remote-cache` and never revalidates them (to allow offline work).
+If you needed the latest content, delete caches stored in this directory by running `rm -rf public/data/remote-cache`.
+
+
# Configuring Apache
You can use apachectl to start/stop/restart apache server from the command line:
@@ -20,30 +45,31 @@
- Stopping httpd: `sudo apachectl stop`
- Restarting httpd: `sudo apachectl restart`
+The apache logs are located at `/private/var/log/apache2`.
+
## Instructions if you're using Server.app
- Enable PHP web applications
- - Go to Server Website / Store Site Files In, change it to /Volumes/Data/perf.webkit.org/public/`
+ - Go to Server Website / Store Site Files In, change it to `/Volumes/Data/perf.webkit.org/public/`
- Go to Server Website / Edit advanced settings, enable Allow overrides using .htaccess files
+ - httpd config file is located at `/Library/Server/Web/Config/apache2/sites/0000_any_80.conf` (and/or 0000_any_`PORT#`.conf)
## Instructions if you're not using Server.app
- Edit /private/etc/apache2/httpd.conf
1. Change DocumentRoot to `/Volumes/Data/perf.webkit.org/public/`
- 2. Uncomment `"LoadModule php5_module libexec/apache2/libphp5.so"`
+ 2. Uncomment `LoadModule php5_module libexec/apache2/libphp5.so`
+ 3. Uncomment `LoadModule rewrite_module libexec/apache2/mod_rewrite.so`
+ 4. Uncomment `LoadModule deflate_module libexec/apache2/mod_deflate.so`
- In Mavericks and later, copy php.ini to load pdo_pgsql.so pgsql.so.
`sudo cp /Applications/Server.app/Contents/ServerRoot/etc/php.ini /etc/`
+ - In El Capitan and later, comment out the `LockFile` directive in `/private/etc/apache2/extra/httpd-mpm.conf`
+ since the directive has been superseded by `Mutex` directive.
+## Production Configurations
-## Common directives for the related apache config file
-
- httpd config file is located at:
-
- - With Server.app: /Library/Server/Web/Config/apache2/sites/0000_any_80.conf (and/or 0000_any_`PORT#`.conf)
- - Without: /private/etc/apache2/httpd.conf
-
1. Update ServerAdmin to your email address
2. Add the following directives to enable gzip:
@@ -56,13 +82,10 @@
Options Indexes MultiViews
php_flag zlib.output_compression on
-The apache logs are located at `/private/var/log/apache2`.
+### Protecting the Administrative Pages to Prevent Execution of Arbitrary Code
-
-# Protecting the Administrative Pages to Prevent Execution of Arbitrary Code
-
By default, the application gives the administrative privilege to everyone. Anyone can add, remove, or edit tests,
-builders, and other entities in the database and may even execute arbitrary _javascript_ on the server via aggregators.
+builders, and other entities in the database.
We recommend protection via Digest Auth on https connection.
@@ -97,8 +120,9 @@
Run `database/init-database.sql` in psql as `webkit-perf-db-user`:
`/Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user -f init-database.sql`
-## Making a Backup of the Database
+## Making a Backup and Restoring
Run `/Applications/Server.app/Contents/ServerRoot/usr/bin/pg_dump -h localhost --no-owner -f <filepath> webkit-perf-db | gzip > backup.gz`
-To restore, setup a new database and run `gunzip backup.gz | /Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user`
+To restore, setup a new database and run
+`gunzip backup.gz | /Applications/Server.app/Contents/ServerRoot/usr/bin/psql webkit-perf-db -h localhost --username webkit-perf-db-user`
Modified: trunk/Websites/perf.webkit.org/config.json (190644 => 190645)
--- trunk/Websites/perf.webkit.org/config.json 2015-10-06 20:57:22 UTC (rev 190644)
+++ trunk/Websites/perf.webkit.org/config.json 2015-10-06 21:04:03 UTC (rev 190645)
@@ -13,6 +13,17 @@
"hostname": "localhost",
"port": 80
},
+ "cacheDirectory": "public/data/remote-cache/",
+ "remoteServer": {
+ "httpdConfig": "tools/remote-server-relay.conf",
+ "httpdPID": "tools/remote-server-relay.pid",
+ "httpdErroLog": "tools/remote-server-relay.log",
+ "url": "http://perf.webkit.org",
+ "basicAuth": {
+ "username": "username",
+ "password": "password"
+ }
+ },
"defaultDashboard": [[]],
"dashboards": {}
}
Added: trunk/Websites/perf.webkit.org/public/admin/fetch-from-remote.php (0 => 190645)
--- trunk/Websites/perf.webkit.org/public/admin/fetch-from-remote.php (rev 0)
+++ trunk/Websites/perf.webkit.org/public/admin/fetch-from-remote.php 2015-10-06 21:04:03 UTC (rev 190645)
@@ -0,0 +1,43 @@
+<?php
+
+require('../include/db.php');
+
+function main($path)
+{
+ $remote_server = config('remoteServer');
+
+ $remote_url = $remote_server['url'] . $path;
+ $cache_path = config_path('cacheDirectory', hash("sha256", $remote_url));
+
+ $content = @file_get_contents($cache_path);
+ if ($content === FALSE) {
+ $content = fetch_remote($remote_server, $remote_url);
+ if ($content === FALSE) {
+ header('HTTP/1.0 404 Not Found');
+ echo 'NotFound';
+ exit(1);
+ }
+ file_put_contents($cache_path, $content);
+ }
+
+ header('Content-Type: application/json');
+ header('Content-Length: ' . strlen($content));
+ echo $content;
+}
+
+function fetch_remote($remote_server, $remote_url)
+{
+ $auth = array_get($remote_server, 'basicAuth');
+
+ $header = '';
+ if ($auth)
+ $header = 'Authorization: Basic ' . base64_encode($auth['username'] . ':' . $auth['password']);
+
+ $context = stream_context_create(array('http' => array('method' => 'GET', 'header' => $header)));
+
+ return @file_get_contents($remote_url, false, $context);
+}
+
+main(array_get($_SERVER, 'PATH_INFO', ''));
+
+?>
Modified: trunk/Websites/perf.webkit.org/public/include/db.php (190644 => 190645)
--- trunk/Websites/perf.webkit.org/public/include/db.php 2015-10-06 20:57:22 UTC (rev 190644)
+++ trunk/Websites/perf.webkit.org/public/include/db.php 2015-10-06 21:04:03 UTC (rev 190645)
@@ -36,10 +36,14 @@
return $_config[$key];
}
+function config_path($key, $path) {
+ return CONFIG_DIR . config($key) . '/' . $path;
+}
+
function generate_data_file($filename, $content) {
if (!assert(ctype_alnum(str_replace(array('-', '_', '.'), '', $filename))))
return FALSE;
- return file_put_contents(CONFIG_DIR . config('dataDirectory') . '/' . $filename, $content);
+ return file_put_contents(config_path('dataDirectory'. $filename), $content);
}
if (config('debug')) {
Added: trunk/Websites/perf.webkit.org/tools/remote-server-relay.conf (0 => 190645)
--- trunk/Websites/perf.webkit.org/tools/remote-server-relay.conf (rev 0)
+++ trunk/Websites/perf.webkit.org/tools/remote-server-relay.conf 2015-10-06 21:04:03 UTC (rev 190645)
@@ -0,0 +1,63 @@
+ServerRoot "/usr"
+Listen 8080
+
+LoadModule authn_core_module libexec/apache2/mod_authn_core.so
+LoadModule authz_core_module libexec/apache2/mod_authz_core.so
+LoadModule deflate_module libexec/apache2/mod_deflate.so
+LoadModule log_config_module libexec/apache2/mod_log_config.so
+LoadModule log_forensic_module libexec/apache2/mod_log_forensic.so
+LoadModule env_module libexec/apache2/mod_env.so
+LoadModule headers_module libexec/apache2/mod_headers.so
+LoadModule setenvif_module libexec/apache2/mod_setenvif.so
+LoadModule mime_module libexec/apache2/mod_mime.so
+LoadModule unixd_module libexec/apache2/mod_unixd.so
+LoadModule status_module libexec/apache2/mod_status.so
+LoadModule negotiation_module libexec/apache2/mod_negotiation.so
+LoadModule dir_module libexec/apache2/mod_dir.so
+LoadModule alias_module libexec/apache2/mod_alias.so
+LoadModule rewrite_module libexec/apache2/mod_rewrite.so
+LoadModule mime_module libexec/apache2/mod_mime.so
+LoadModule php5_module libexec/apache2/libphp5.so
+LoadModule negotiation_module libexec/apache2/mod_negotiation.so
+
+<Directory />
+ Options Indexes FollowSymLinks MultiViews
+ AllowOverride None
+ Require all granted
+</Directory>
+
+<IfModule dir_module>
+ DirectoryIndex index.html index.php
+</IfModule>
+
+LogLevel warn
+
+RewriteEngine On
+RewriteRule "/data/(.*)$" "/admin/fetch-from-remote.php/data/$1"
+RewriteRule "/api/(.*)$" "/admin/fetch-from-remote.php/api/$1"
+
+<IfModule log_config_module>
+ LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
+ LogFormat "%h %l %u %t \"%r\" %>s %b" common
+
+ <IfModule logio_module>
+ LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
+ </IfModule>
+
+ CustomLog |/usr/bin/tee common
+ ErrorLog |/usr/bin/tee
+</IfModule>
+
+<IfModule mime_module>
+ AddType text/html .html
+ AddType text/html .htm
+ AddType text/css .css
+ AddType text/_javascript_ .js
+</IfModule>
+
+<IfModule php5_module>
+ AddType application/x-httpd-php .php
+ AddType application/x-httpd-php-source .phps
+</IfModule>
+
+Include /private/etc/apache2/extra/httpd-mpm.conf
Added: trunk/Websites/perf.webkit.org/tools/run-with-remote-server.py (0 => 190645)
--- trunk/Websites/perf.webkit.org/tools/run-with-remote-server.py (rev 0)
+++ trunk/Websites/perf.webkit.org/tools/run-with-remote-server.py 2015-10-06 21:04:03 UTC (rev 190645)
@@ -0,0 +1,40 @@
+#!/usr/bin/python
+
+import json
+import os
+import subprocess
+import tempfile
+
+
+def main():
+ with open(abspath_from_root('config.json')) as config_file:
+ config = json.load(config_file)
+ cache_dir = abspath_from_root(config['cacheDirectory'])
+ httpd_config_file = abspath_from_root(config['remoteServer']['httpdConfig'])
+ httpd_pid_file = abspath_from_root(config['remoteServer']['httpdPID'])
+ httpd_error_log_file = abspath_from_root(config['remoteServer']['httpdErroLog'])
+ doc_root = abspath_from_root('public')
+
+ if not os.path.isdir(cache_dir):
+ os.makedirs(cache_dir)
+ os.chmod(cache_dir, 0755)
+
+ httpd_mutax_dir = tempfile.mkdtemp()
+ try:
+ subprocess.call(['httpd',
+ '-f', httpd_config_file,
+ '-c', 'PidFile ' + httpd_pid_file,
+ '-c', 'Mutex file:' + httpd_mutax_dir,
+ '-c', 'DocumentRoot ' + doc_root,
+ '-c', 'ErrorLog ' + httpd_error_log_file,
+ '-X'])
+ finally:
+ os.rmdir(httpd_mutax_dir)
+
+
+def abspath_from_root(relpath):
+ return os.path.abspath(os.path.join(os.path.dirname(__file__), '../', relpath))
+
+
+if __name__ == "__main__":
+ main()
Property changes on: trunk/Websites/perf.webkit.org/tools/run-with-remote-server.py
___________________________________________________________________
Added: svn:executable
