Diff
Modified: trunk/PerformanceTests/SunSpider/ChangeLog (155382 => 155383)
--- trunk/PerformanceTests/SunSpider/ChangeLog 2013-09-09 21:32:48 UTC (rev 155382)
+++ trunk/PerformanceTests/SunSpider/ChangeLog 2013-09-09 21:36:54 UTC (rev 155383)
@@ -1,3 +1,14 @@
+2013-09-09 Filip Pizlo <[email protected]>
+
+ Stress tests should test the jsc profiler (-p)
+ https://bugs.webkit.org/show_bug.cgi?id=121043
+
+ Reviewed by Mark Hahnenberg.
+
+ Add a jsc-stress-test that tries to profile SunSpider.
+
+ * profiler-test.yaml: Added.
+
2013-08-08 Cosmin Truta <[email protected]>
SunSpider: Move the 3d-morph fix from version 1.0 to version 1.0.1
Added: trunk/PerformanceTests/SunSpider/profiler-test.yaml (0 => 155383)
--- trunk/PerformanceTests/SunSpider/profiler-test.yaml (rev 0)
+++ trunk/PerformanceTests/SunSpider/profiler-test.yaml 2013-09-09 21:36:54 UTC (rev 155383)
@@ -0,0 +1,28 @@
+# Copyright (C) 2013 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# Specifies that we want to run all tests in test/sunspider-1.0/*.js
+# with the runProfiler command.
+
+- path: tests/sunspider-1.0
+ cmd: runProfiler
Modified: trunk/Tools/ChangeLog (155382 => 155383)
--- trunk/Tools/ChangeLog 2013-09-09 21:32:48 UTC (rev 155382)
+++ trunk/Tools/ChangeLog 2013-09-09 21:36:54 UTC (rev 155383)
@@ -1,3 +1,24 @@
+2013-09-09 Filip Pizlo <[email protected]>
+
+ Stress tests should test the jsc profiler (-p)
+ https://bugs.webkit.org/show_bug.cgi?id=121043
+
+ Reviewed by Mark Hahnenberg.
+
+ Add a runProfiler command that all tests could use. This requires profiler-test-helper,
+ which first runs the JS test and then tries the output with display-profiler-output.
+ But if any of the things required for this to work aren't present, we just do a simpler
+ test that just uses "-p".
+
+ Because I didn't want to pollute SunSpider with "//@ runProfiler", I added the ability
+ to create test collections using a yaml file that specifies the test path and the
+ command to run.
+
+ * Scripts/jsc-stress-test-helpers: Added.
+ * Scripts/jsc-stress-test-helpers/profiler-test-helper: Added.
+ * Scripts/run-_javascript_core-tests:
+ * Scripts/run-jsc-stress-tests:
+
2013-09-09 Zan Dobersek <[email protected]>
[webkitpy] Remove the DeprecatedPort class
Added: trunk/Tools/Scripts/jsc-stress-test-helpers/profiler-test-helper (0 => 155383)
--- trunk/Tools/Scripts/jsc-stress-test-helpers/profiler-test-helper (rev 0)
+++ trunk/Tools/Scripts/jsc-stress-test-helpers/profiler-test-helper 2013-09-09 21:36:54 UTC (rev 155383)
@@ -0,0 +1,34 @@
+#!/usr/bin/env ruby
+
+# Copyright (C) 2013 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+require 'json'
+require 'shellwords'
+
+def mysys(*cmd)
+ raise "Command #{cmd.inspect} failed: #{$?.inspect}" unless system(*cmd)
+end
+
+mysys(*ARGV[2..-1])
+mysys("echo quit | #{Shellwords.shellescape(ARGV[0])} #{Shellwords.shellescape(ARGV[1])}")
Modified: trunk/Tools/Scripts/run-_javascript_core-tests (155382 => 155383)
--- trunk/Tools/Scripts/run-_javascript_core-tests 2013-09-09 21:32:48 UTC (rev 155382)
+++ trunk/Tools/Scripts/run-_javascript_core-tests 2013-09-09 21:36:54 UTC (rev 155383)
@@ -243,7 +243,8 @@
"-j", jscPath($productDir), "-o", $jscStressResultsDir,
"PerformanceTests/SunSpider/tests/sunspider-1.0",
"PerformanceTests/SunSpider/tests/v8-v6",
- "LayoutTests/fast/js/regress/script-tests"
+ "LayoutTests/fast/js/regress/script-tests",
+ "PerformanceTests/SunSpider/profiler-test.yaml"
);
if ($enableFTL) {
push(@jscStressDriverCmd, "--ftl-jit");
Modified: trunk/Tools/Scripts/run-jsc-stress-tests (155382 => 155383)
--- trunk/Tools/Scripts/run-jsc-stress-tests 2013-09-09 21:32:48 UTC (rev 155382)
+++ trunk/Tools/Scripts/run-jsc-stress-tests 2013-09-09 21:36:54 UTC (rev 155383)
@@ -25,7 +25,15 @@
require 'getoptlong'
require 'pathname'
+require 'yaml'
+THIS_SCRIPT_PATH = Pathname.new(__FILE__).realpath
+SCRIPTS_PATH = THIS_SCRIPT_PATH.dirname
+raise unless SCRIPTS_PATH.basename.to_s == "Scripts"
+raise unless SCRIPTS_PATH.dirname.basename.to_s == "Tools"
+
+HELPERS_PATH = SCRIPTS_PATH + "jsc-stress-test-helpers"
+
$haveShellwords = false
begin
@@ -36,13 +44,28 @@
$stderr.puts "Error: #{e.inspect}"
end
-def mysys(*cmd)
+$canRunDisplayProfilerOutput = false
+
+begin
+ require 'json'
+ require 'highline'
+ $canRunDisplayProfilerOutput = true
+rescue => e
+ $stderr.puts "Warning: did not find json or highline; some features will be disabled."
+ $stderr.puts "Error: #{e.inspect}"
+end
+
+def printCommandArray(*cmd)
begin
commandArray = cmd.each{|value| Shellwords.shellescape(value.to_s)}.join(' ')
rescue
commandArray = cmd.join(' ')
end
$stderr.puts ">> #{commandArray}"
+end
+
+def mysys(*cmd)
+ printCommandArray(*cmd)
raise "Command failed: #{$?.inspect}" unless system(*cmd)
end
@@ -53,6 +76,7 @@
$collections = []
$outputDir = Pathname.new("results")
$parallel = ($haveShellwords and numProcessors > 1)
+$verbosity = 0
def usage
puts "run-jsc-stress-tests -j <shell path> <collections path> [<collections path> ...]"
@@ -61,6 +85,7 @@
puts "--ftl-jit Indicate that we have the FTL JIT."
puts "--[no-]parallel Run in parallel, or not. Default is #{$parallel}."
puts "--output-dir (-o) Path where to put results. Default is #{$outputDir}."
+ puts "--verbose (-v) Print more things while running."
puts "--help (-h) Print this message."
exit 1
end
@@ -70,7 +95,8 @@
['--ftl-jit', GetoptLong::NO_ARGUMENT],
['--parallel', GetoptLong::NO_ARGUMENT],
['--no-parallel', GetoptLong::NO_ARGUMENT],
- ['--output-dir', '-o', GetoptLong::REQUIRED_ARGUMENT]).each {
+ ['--output-dir', '-o', GetoptLong::REQUIRED_ARGUMENT],
+ ['--verbose', '-v', GetoptLong::NO_ARGUMENT]).each {
| opt, arg |
case opt
when '--help'
@@ -85,6 +111,8 @@
$parallel = true
when '--no-parallel'
$parallel = false
+ when '--verbose'
+ $verbosity += 1
end
}
@@ -113,17 +141,32 @@
File.open(filename, "w") {
| outp |
outp.puts "echo Running #{Shellwords.shellescape(@name)}"
- outp.puts("(cd #{Shellwords.shellescape(@directory.to_s)} && " +
- @arguments.map{|v| Shellwords.shellescape(v)}.join(' ') +
- ") || #{failCommand}")
+ cmd = ("(cd #{Shellwords.shellescape(@directory.to_s)} && " +
+ @arguments.map{|v| Shellwords.shellescape(v)}.join(' ') +
+ ") || #{failCommand}")
+ if $verbosity >= 1
+ outp.puts "echo #{Shellwords.shellescape(cmd)}"
+ end
+ outp.puts cmd
}
end
end
+$uniqueFilenameCounter = 0
+def uniqueFilename(extension)
+ payloadDir = $outputDir + "_payload"
+ Dir.mkdir payloadDir unless payloadDir.directory?
+ result = payloadDir.realpath + "temp-#{$uniqueFilenameCounter}#{extension}"
+ $uniqueFilenameCounter += 1
+ result
+end
+
+def addRunCommand(kind, command)
+ $runlist << Plan.new($benchmarkDirectory, command, "#{$collectionName}/#{$benchmark}.#{kind}")
+end
+
def run(kind, *options)
- $runlist << Plan.new($collection,
- [$jscPath.to_s] + options + [$benchmark],
- "#{$collectionName}/#{$benchmark}.#{kind}")
+ addRunCommand(kind, [$jscPath.to_s] + options + [$benchmark.to_s])
end
def runDefault
@@ -150,6 +193,15 @@
run("ftl-eager", "--useExperimentalFTL=true", *EAGER_OPTIONS)
end
+def runProfiler
+ profilerOutput = uniqueFilename(".json")
+ if $haveShellwords and $canRunDisplayProfilerOutput
+ addRunCommand("profiler", ["ruby", (HELPERS_PATH + "profiler-test-helper").to_s, (SCRIPTS_PATH + "display-profiler-output").to_s, profilerOutput.to_s, $jscPath.to_s, "-p", profilerOutput.to_s, $benchmark.to_s])
+ else
+ run("profiler-simple", "-p", profilerOutput.to_s)
+ end
+end
+
def defaultRun
runDefault
runNoCJIT
@@ -166,8 +218,6 @@
puts "Skipping #{$collectionName}/#{$benchmark}"
end
-collectionNames = {}
-
Dir.mkdir($outputDir) unless $outputDir.directory?
begin
File.delete($outputDir + "failed")
@@ -176,51 +226,107 @@
$outputDir = $outputDir.realpath
-ARGV.each {
- | collection |
- $collection = Pathname.new(collection)
- outerDir = $collection.dirname
- name = $collection.basename
+def allJSFiles(path)
+ if path.file?
+ [path]
+ else
+ result = []
+ Dir.foreach(path) {
+ | filename |
+ next unless filename =~ /\.js$/
+ next unless (path + filename).file?
+ result << path + filename
+ }
+ result
+ end
+end
+
+# Returns [collectionPath, collectionName]
+def simplifyCollectionName(collectionNames, collectionPath)
+ outerDir = collectionPath.dirname
+ name = collectionPath.basename
lastName = name
- while lastName.to_s =~ /test/
- lastName = outerDir.basename
- name = lastName + name
- outerDir = outerDir.dirname
+ if collectionPath.directory?
+ while lastName.to_s =~ /test/
+ lastName = outerDir.basename
+ name = lastName + name
+ outerDir = outerDir.dirname
+ end
end
- $collectionName = name.to_s
+ collectionName = name.to_s
toAdd = 1
- while collectionNames[$collectionName]
- $collectionName = File.basename(name.to_s) + "-#{toAdd}"
+ while collectionNames[collectionName]
+ collectionName = File.basename(name.to_s) + "-#{toAdd}"
toAdd += 1
end
- collectionNames[$collectionName] = true
+ collectionNames[collectionName] = true
+ [collectionPath, collectionName]
+end
+
+def prepareCollection(name)
dir = $outputDir
- name.each_filename {
+ Pathname.new(name).each_filename {
| filename |
dir = dir + filename
Dir.mkdir(dir) unless dir.directory?
}
+end
- Dir.foreach($collection) {
- | benchmark |
- next unless benchmark =~ /\.js$/
- next unless ($collection + benchmark).file?
+collectionNames = {}
+
+ARGV.each {
+ | collection |
+ collection, collectionName = simplifyCollectionName(collectionNames, Pathname.new(collection))
+
+ if collection.file?
+ subCollectionNames = {}
+ YAML::load(IO::read(collection)).each {
+ | entry |
+ path = collection.dirname + entry["path"]
+
+ subCollection, subCollectionName = simplifyCollectionName(subCollectionNames, path)
+
+ $collection = subCollection
+ $collectionName = (Pathname.new(collectionName) + subCollectionName).to_s
+
+ prepareCollection($collectionName)
+
+ allJSFiles(path).each {
+ | path |
+
+ path = path.realpath
+
+ $benchmark = path.basename
+ $benchmarkDirectory = path.dirname
+
+ eval entry["cmd"]
+ }
+ }
+ else
+ prepareCollection(collectionName)
- $benchmark = benchmark
-
- didRun = false
- File.open($collection + benchmark) {
- | inp |
- inp.each_line {
- | line |
- next unless line =~ /^\/\/@/
- eval $~.post_match
- didRun = true
+ $collection = collection
+ $collectionName = collectionName
+ $benchmarkDirectory = $collection
+ allJSFiles($collection).each {
+ | path |
+
+ $benchmark = path.basename
+
+ didRun = false
+ File.open($collection + $benchmark) {
+ | inp |
+ inp.each_line {
+ | line |
+ next unless line =~ /^\/\/@/
+ eval $~.post_match
+ didRun = true
+ }
}
+
+ defaultRun unless didRun
}
-
- defaultRun unless didRun
- }
+ end
}
def appendFailure(plan)
@@ -343,6 +449,9 @@
print "#{plan.name}: "
Dir.chdir(plan.directory) {
+ if $verbosity >= 1
+ printCommandArray(*plan.arguments)
+ end
if system(*plan.arguments)
puts "OK."
else
