Title: [86766] trunk/Tools
Revision
86766
Author
[email protected]
Date
2011-05-18 08:38:14 -0700 (Wed, 18 May 2011)

Log Message

Add a new page to build.webkit.org to help find when tests started failing

The page is accessible at <http://build.webkit.org/TestFailures/>. It is pretty minimalist
right now, but already shows some useful information. It's somewhat similar to webkit-patch
failure-reason and sheriffbot, and perhaps can be combined with them eventually. It's a
little more convenient than either of them, though, because it's all done in the browser
(and thus it's easy to go directly to the relevant test results).

* BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Buildbot.js: Added.
(Buildbot): This class represents a Buildbot server.
(Buildbot.prototype.buildURL): Returns the URL for the summary page for a particular build.
(Buildbot.prototype.builderNamed): Returns a Builder with the given name.
(Buildbot.prototype.getTesterNames): Fetches the names of all testers and passes them to the
callback.
(Buildbot.prototype.parseBuildName): Breaks up a build name into its constituent parts. Must
be implemented by a derived class that understands this server's build naming scheme.
(Buildbot.prototype.resultsDirectoryURL): Returns the URL for the results directory for a
particular build.

* BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Builder.js: Added.
(Builder): This class represents one builder on the buildbot.
(Builder.prototype.buildURL): Returns the URL for the summary page for a particular build.
(Builder.prototype.failureDiagnosisTextAndURL): Returns data that provides a little more
information about a particular test failure.
(Builder.prototype.startFetchingBuildHistory): Periodically calls the callback with
information about when tests started failing.
(Builder.prototype.resultsDirectoryURL): Returns the URL for the results directory for a
particular build.
(Builder.prototype._getBuildNames): Fetches the names of all builds and passes them to the
callback.
(Builder.prototype._getFailingTests): Fetches the results.html page for the given build and
extracts all the failing tests listed in it, passing them to the callback.
(Builder.prototype._incorporateBuildHistory): Gets the failing tests for the specified
build, merges them into the build history, and calls the callback telling it whether the
next build should be fetched to provide more information.

* BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/TestFailures.css:
Added. Just some simple styles.

* BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Utilities.js: Added.
(createDefinitionList): Takes an array of pairs and turns them into a DL element.
(getResource): Wrapper around XMLHttpRequest.
(Array.prototype.findFirst): Finds the first element matching the given predicate and
returns it.
(Array.prototype.last): Returns the last element of the array.

* BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/ViewController.js: Added.
(ViewController): This class contains the main logic for displaying the page.
(ViewController.loaded): Just calls through to parseHash.
(ViewController.parseHash): Either starts analyzing failures on a particular builder, or
shows the list of all testers so one can be chosen. This function is called when the page
loads and whenever we get a hashchange event.
(ViewController._displayBuilder): Asks the builder to fetch build history, and displays it
as it is fetched. The display ends up grouping tests by when they started failing.
(ViewController._displayTesters): Gets the list of testers and displays it.

(ViewController._domForBuildName):
(ViewController._domForFailedTest):
Helper functions to create descriptions and links for a particular build or failed test.

* BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/WebKitBuildbot.js: Added.
(WebKitBuildbot): Calls up to the base class constructor with the correct base URL.
(WebKitBuildbot.prototype.parseBuildName): Parses a build.webkit.org-style build name.

* BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/index.html: Added. Just
loads all the files and sets up a ViewController, which does the rest.

* BuildSlaveSupport/build.webkit.org-config/templates/root.html: Added a link to the new
page.

Modified Paths

Added Paths

Diff

Added: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Buildbot.js (0 => 86766)


--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Buildbot.js	                        (rev 0)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Buildbot.js	2011-05-18 15:38:14 UTC (rev 86766)
@@ -0,0 +1,74 @@
+/*
+ * Copyright (C) 2011 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+function Buildbot(baseURL) {
+    this.baseURL = baseURL;
+    this._cache = {};
+}
+
+Buildbot.prototype = {
+    _builders: {},
+
+    buildURL: function(builderName, buildName) {
+        return this.baseURL + 'builders/' + builderName + '/builds/' + buildName;
+    },
+
+    builderNamed: function(name) {
+        if (!(name in this._builders))
+            this._builders[name] = new Builder(name, this);
+        return this._builders[name];
+    },
+
+    getTesterNames: function(callback) {
+        var cacheKey = 'getTesterNames';
+        if (cacheKey in this._cache) {
+            callback(this._cache[cacheKey]);
+            return;
+        }
+
+        var self = this;
+        getResource(this.baseURL + 'results/', function(xhr) {
+            var root = document.createElement('html');
+            root.innerHTML = xhr.responseText;
+            var names = Array.prototype.map.call(root.querySelectorAll('td:first-child > a > b'), function(elem) {
+                return elem.innerText.replace(/\/$/, '');
+            });
+
+            self._cache[cacheKey] = names;
+            callback(names);
+        });
+    },
+
+    // Returns an object with at least the following properties:
+    //   revision: source revision number for this build (integer)
+    //   buildNumber: number of this build (integer)
+    parseBuildName: function(buildName) {
+        throw "Derived classes must implement";
+    },
+
+    resultsDirectoryURL: function(builderName, buildName) {
+        return this.baseURL + 'results/' + builderName + '/' + buildName + '/';
+    },
+};

Added: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Builder.js (0 => 86766)


--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Builder.js	                        (rev 0)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Builder.js	2011-05-18 15:38:14 UTC (rev 86766)
@@ -0,0 +1,222 @@
+/*
+ * Copyright (C) 2011 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+function Builder(name, buildbot) {
+    this.name = name;
+    this.buildbot = buildbot;
+    this._cache = {};
+}
+
+Builder.prototype = {
+    buildURL: function(buildName) {
+        return this.buildbot.buildURL(this.name, buildName);
+    },
+
+    failureDiagnosisTextAndURL: function(buildName, testName, failureType) {
+        var urlStem = this.resultsDirectoryURL(buildName) + testName.replace(/\.[^.]+$/, '');
+        var diagnosticInfo = {
+            fail: {
+                text: 'pretty diff',
+                url: urlStem + '-pretty-diff.html',
+            },
+            timeout: {
+                text: 'timed out',
+            },
+            crash: {
+                text: 'crash log',
+                url: urlStem + '-crash-log.txt',
+            },
+            'webprocess crash': {
+                text: 'web process crash log',
+                url: urlStem + '-crash-log.txt',
+            },
+        };
+
+        return diagnosticInfo[failureType];
+    },
+
+    /*
+     * Preiodically calls callback until all current failures have been explained. Callback is
+     * passed an object like the following:
+     * {
+     *     'r2_1 (1)': {
+     *         'css1/basic/class_as_selector2.html': 'fail',
+     *     },
+     *     'r1_1 (0)': {
+     *         'css1/basic/class_as_selector.html': 'crash',
+     *     },
+     * },
+     * Each build contains just the failures that a) are still occuring on the bots, and b) were new
+     * in that build.
+     */
+    startFetchingBuildHistory: function(callback) {
+        var cacheKey = '_startFetchingBuildHistory';
+        if (!(cacheKey in this._cache))
+            this._cache[cacheKey] = {};
+
+        var history = this._cache[cacheKey];
+
+        var self = this;
+        self._getBuildNames(function(buildNames) {
+            function inner(buildIndex) {
+                self._incorporateBuildHistory(buildNames, buildIndex, history, function(callAgain) {
+                    callback(history);
+                    if (!callAgain)
+                        return;
+                    var nextIndex = buildIndex + 1;
+                    if (nextIndex >= buildNames.length)
+                        return;
+                    setTimeout(function() { inner(nextIndex) }, 0);
+                });
+            }
+            inner(0);
+        });
+    },
+
+    resultsDirectoryURL: function(buildName) {
+        return this.buildbot.resultsDirectoryURL(this.name, buildName);
+    },
+
+    _getBuildNames: function(callback) {
+        var cacheKey = '_getBuildNames';
+        if (cacheKey in this._cache) {
+            callback(this._cache[cacheKey]);
+            return;
+        }
+
+        var self = this;
+        getResource(this.buildbot.baseURL + 'results/' + this.name, function(xhr) {
+            var root = document.createElement('html');
+            root.innerHTML = xhr.responseText;
+
+            var buildNames = Array.prototype.map.call(root.querySelectorAll('td:first-child > a > b'), function(elem) {
+                return elem.innerText.replace(/\/$/, '');
+            }).filter(function(filename) {
+                return !/\.zip$/.test(filename);
+            });
+            buildNames.reverse();
+
+            self._cache[cacheKey] = buildNames;
+            callback(buildNames);
+        });
+    },
+
+    _getFailingTests: function(buildName, callback, errorCallback) {
+        var cacheKey = '_getFailingTests_' + buildName;
+        if (cacheKey in this._cache) {
+            callback(this._cache[cacheKey]);
+            return;
+        }
+
+        var tests = {};
+        this._cache[cacheKey] = tests;
+
+        var self = this;
+        getResource(self.buildbot.baseURL + 'json/builders/' + self.name + '/builds/' + self.buildbot.parseBuildName(buildName).buildNumber, function(xhr) {
+            var data = ""
+            var layoutTestStep = data.steps.findFirst(function(step) { return step.name === 'layout-test'; });
+            if (!('isStarted' in layoutTestStep)) {
+                // run-webkit-tests never even ran.
+                errorCallback(tests);
+                return;
+            }
+
+            if (!('results' in layoutTestStep) || layoutTestStep.results[0] === 0) {
+                // All tests passed.
+                callback(tests);
+                return;
+            }
+
+            if (/^Exiting early/.test(layoutTestStep.results[1][0])) {
+                // Too many tests crashed or timed out. We can't use this test run.
+                errorCallback(tests);
+                return;
+            }
+
+            // Find out which tests failed.
+            getResource(self.resultsDirectoryURL(buildName) + 'results.html', function(xhr) {
+                var root = document.createElement('html');
+                root.innerHTML = xhr.responseText;
+
+                function testsForResultTable(regex) {
+                    var paragraph = Array.prototype.findFirst.call(root.querySelectorAll('p'), function(paragraph) {
+                        return regex.test(paragraph.innerText);
+                    });
+                    if (!paragraph)
+                        return [];
+                    var table = paragraph.nextElementSibling;
+                    console.assert(table.nodeName === 'TABLE');
+                    return Array.prototype.map.call(table.querySelectorAll('td:first-child > a'), function(elem) {
+                        return elem.innerText;
+                    });
+                }
+
+                testsForResultTable(/did not match expected results/).forEach(function(name) {
+                    tests[name] = 'fail';
+                });
+                testsForResultTable(/timed out/).forEach(function(name) {
+                    tests[name] = 'timeout';
+                });
+                testsForResultTable(/tool to crash/).forEach(function(name) {
+                    tests[name] = 'crash';
+                });
+                testsForResultTable(/Web process to crash/).forEach(function(name) {
+                    tests[name] = 'webprocess crash';
+                });
+
+                callback(tests);
+            },
+            function(xhr) {
+                // We failed to fetch results.html. run-webkit-tests must have aborted early.
+                errorCallback(tests);
+            });
+        });
+    },
+
+    _incorporateBuildHistory: function(buildNames, buildIndex, history, callback) {
+        var previousBuildName = Object.keys(history).last();
+        var nextBuildName = buildNames[buildIndex];
+
+        this._getFailingTests(nextBuildName, function(tests) {
+            history[nextBuildName] = {};
+
+            for (var testName in tests) {
+                if (previousBuildName) {
+                    if (!(testName in history[previousBuildName]))
+                        continue;
+                    delete history[previousBuildName][testName];
+                }
+                history[nextBuildName][testName] = tests[testName];
+            }
+
+            callback(Object.keys(history[nextBuildName]).length);
+        },
+        function(tests) {
+            // Some tests failed, but we couldn't fetch results.html (perhaps because the test
+            // run aborted early for some reason). Just skip this build entirely.
+            callback(true);
+        });
+    },
+};

Added: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/TestFailures.css (0 => 86766)


--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/TestFailures.css	                        (rev 0)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/TestFailures.css	2011-05-18 15:38:14 UTC (rev 86766)
@@ -0,0 +1,11 @@
+ol {
+    list-style-type: none;
+}
+dt {
+    float: left;
+    font-weight: bold;
+    margin-right: 3px;
+}
+dt::after {
+    content: ':';
+}

Added: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Utilities.js (0 => 86766)


--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Utilities.js	                        (rev 0)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Utilities.js	2011-05-18 15:38:14 UTC (rev 86766)
@@ -0,0 +1,66 @@
+/*
+ * Copyright (C) 2011 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+function createDefinitionList(items) {
+    var list = document.createElement('dl');
+    items.forEach(function(pair) {
+        var dt = document.createElement('dt');
+        dt.appendChild(pair[0]);
+        var dd = document.createElement('dd');
+        dd.appendChild(pair[1]);
+        list.appendChild(dt);
+        list.appendChild(dd);
+    });
+    return list;
+}
+
+function getResource(url, callback, errorCallback) {
+    var xhr = new XMLHttpRequest();
+    xhr._onreadystatechange_ = function() {
+        if (this.readyState !== 4)
+            return;
+        // Allow a status of 0 for easier testing with local files.
+        if (!this.status || this.status === 200)
+            callback(this);
+        else if (errorCallback)
+            errorCallback(this);
+    };
+    xhr.open("GET", url);
+    xhr.send();
+}
+
+Array.prototype.findFirst = function(predicate) {
+    for (var i = 0; i < this.length; ++i) {
+        if (predicate(this[i]))
+            return this[i];
+    }
+    return null;
+}
+
+Array.prototype.last = function() {
+    if (!this.length)
+        return undefined;
+    return this[this.length - 1];
+}

Added: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/ViewController.js (0 => 86766)


--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/ViewController.js	                        (rev 0)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/ViewController.js	2011-05-18 15:38:14 UTC (rev 86766)
@@ -0,0 +1,146 @@
+/*
+ * Copyright (C) 2011 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+function ViewController(buildbot) {
+    this._buildbot = buildbot;
+
+    var self = this;
+    addEventListener('load', function() { self.loaded() }, false);
+    addEventListener('hashchange', function() { self.parseHash(location.hash) }, false);
+}
+
+ViewController.prototype = {
+    loaded: function() {
+        this.parseHash(location.hash);
+    },
+
+    parseHash: function(hash) {
+        var match = /#\/(.*)/.exec(hash);
+        if (match)
+            this._displayBuilder(this._buildbot.builderNamed(match[1]));
+        else
+            this._displayTesters();
+    },
+
+    _displayBuilder: function(builder) {
+        var self = this;
+        builder.startFetchingBuildHistory(function(history) {
+            var list = document.createElement('ol');
+            Object.keys(history).forEach(function(buildName, buildIndex, buildNameArray) {
+                if (!Object.keys(history[buildName]).length)
+                    return;
+                var dlItems = [
+                    [document.createTextNode('Failed'), self._domForBuildName(builder, buildName)],
+                ];
+                if (buildIndex + 1 < buildNameArray.length)
+                    dlItems.push([document.createTextNode('Passed'), self._domForBuildName(builder, buildNameArray[buildIndex + 1])]);
+
+                var item = document.createElement('li');
+                item.appendChild(createDefinitionList(dlItems));
+                list.appendChild(item);
+
+                var testList = document.createElement('ol');
+                for (var testName in history[buildName]) {
+                    var testItem = document.createElement('li');
+                    testItem.appendChild(self._domForFailedTest(builder, buildName, testName, history[buildName][testName]));
+                    testList.appendChild(testItem);
+                }
+                item.appendChild(testList);
+            });
+
+            var header = document.createElement('h1');
+            header.appendChild(document.createTextNode(builder.name));
+            document.body.innerHTML = '';
+            document.title = builder.name;
+            document.body.appendChild(header);
+            document.body.appendChild(list);
+        });
+    },
+
+    _displayTesters: function() {
+        this._buildbot.getTesterNames(function(names) {
+            var list = document.createElement('ul');
+            names.forEach(function(name) {
+                var link = document.createElement('a');
+                link.href = '' + name;
+                link.appendChild(document.createTextNode(name));
+                var item = document.createElement('li');
+                item.appendChild(link);
+                list.appendChild(item);
+            });
+
+            document.body.innerHTML = '';
+            document.title = 'Testers';
+            document.body.appendChild(list);
+        });
+    },
+
+    _domForBuildName: function(builder, buildName) {
+        var parsed = this._buildbot.parseBuildName(buildName);
+
+        var sourceLink = document.createElement('a');
+        sourceLink.href = '' + parsed.revision;
+        sourceLink.appendChild(document.createTextNode('r' + parsed.revision));
+
+        var buildLink = document.createElement('a');
+        buildLink.href = ""
+        buildLink.appendChild(document.createTextNode(parsed.buildNumber));
+
+        var resultsLink = document.createElement('a');
+        resultsLink.href = "" + 'results.html';
+        resultsLink.appendChild(document.createTextNode('results.html'));
+
+        var result = document.createDocumentFragment();
+        result.appendChild(sourceLink);
+        result.appendChild(document.createTextNode(' ('));
+        result.appendChild(buildLink);
+        result.appendChild(document.createTextNode(') ('));
+        result.appendChild(resultsLink);
+        result.appendChild(document.createTextNode(')'));
+
+        return result;
+    },
+
+    _domForFailedTest: function(builder, buildName, testName, failureType) {
+        var diagnosticInfo = builder.failureDiagnosisTextAndURL(buildName, testName, failureType);
+
+        var result = document.createDocumentFragment();
+        result.appendChild(document.createTextNode(testName));
+        result.appendChild(document.createTextNode(' ('));
+
+        var textNode = document.createTextNode(diagnosticInfo.text);
+        if ('url' in diagnosticInfo) {
+            var link = document.createElement('a');
+            link.href = ""
+            link.appendChild(textNode);
+            result.appendChild(link);
+        } else
+            result.appendChild(textNode);
+
+        result.appendChild(document.createTextNode(')'));
+
+        return result;
+    },
+};

Added: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/WebKitBuildbot.js (0 => 86766)


--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/WebKitBuildbot.js	                        (rev 0)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/WebKitBuildbot.js	2011-05-18 15:38:14 UTC (rev 86766)
@@ -0,0 +1,40 @@
+/*
+ * Copyright (C) 2011 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+function WebKitBuildbot() {
+    Buildbot.call(this, 'http://build.webkit.org/');
+}
+
+WebKitBuildbot.prototype = {
+    parseBuildName: function(buildName) {
+        var match = /^r(\d+) \((\d+)\)$/.exec(buildName);
+        return {
+            revision: parseInt(match[1], 10),
+            buildNumber: parseInt(match[2], 10),
+        };
+    },
+};
+
+WebKitBuildbot.prototype.__proto__ = Buildbot.prototype;

Added: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/index.html (0 => 86766)


--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/index.html	                        (rev 0)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/index.html	2011-05-18 15:38:14 UTC (rev 86766)
@@ -0,0 +1,42 @@
+<!--
+Copyright (C) 2011 Apple Inc. All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+THE POSSIBILITY OF SUCH DAMAGE.
+-->
+
+<!DOCTYPE html>
+<html>
+<head>
+    <link rel="stylesheet" href=""
+    <script src=""
+    <script src=""
+    <script src=""
+    <script src=""
+
+    <script src=""
+    <script>
+        var viewController = new ViewController(new WebKitBuildbot());
+    </script>
+</head>
+<body>
+</body>
+</html>

Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/templates/root.html (86765 => 86766)


--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/templates/root.html	2011-05-18 15:35:36 UTC (rev 86765)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/templates/root.html	2011-05-18 15:38:14 UTC (rev 86766)
@@ -20,6 +20,7 @@
   <li><a href="" information.</li>
   <li><a href="" Results</a></li>
   <li><a href="" Viewer</a></li>
+  <li><a href="" Failures</a></li>
 </ul>
 </body> </html>
 

Modified: trunk/Tools/ChangeLog (86765 => 86766)


--- trunk/Tools/ChangeLog	2011-05-18 15:35:36 UTC (rev 86765)
+++ trunk/Tools/ChangeLog	2011-05-18 15:38:14 UTC (rev 86766)
@@ -1,3 +1,75 @@
+2011-05-18  Adam Roben  <[email protected]>
+
+        Add a new page to build.webkit.org to help find when tests started failing
+
+        The page is accessible at <http://build.webkit.org/TestFailures/>. It is pretty minimalist
+        right now, but already shows some useful information. It's somewhat similar to webkit-patch
+        failure-reason and sheriffbot, and perhaps can be combined with them eventually. It's a
+        little more convenient than either of them, though, because it's all done in the browser
+        (and thus it's easy to go directly to the relevant test results).
+
+        * BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Buildbot.js: Added.
+        (Buildbot): This class represents a Buildbot server.
+        (Buildbot.prototype.buildURL): Returns the URL for the summary page for a particular build.
+        (Buildbot.prototype.builderNamed): Returns a Builder with the given name.
+        (Buildbot.prototype.getTesterNames): Fetches the names of all testers and passes them to the
+        callback.
+        (Buildbot.prototype.parseBuildName): Breaks up a build name into its constituent parts. Must
+        be implemented by a derived class that understands this server's build naming scheme.
+        (Buildbot.prototype.resultsDirectoryURL): Returns the URL for the results directory for a
+        particular build.
+
+        * BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Builder.js: Added.
+        (Builder): This class represents one builder on the buildbot.
+        (Builder.prototype.buildURL): Returns the URL for the summary page for a particular build.
+        (Builder.prototype.failureDiagnosisTextAndURL): Returns data that provides a little more
+        information about a particular test failure.
+        (Builder.prototype.startFetchingBuildHistory): Periodically calls the callback with
+        information about when tests started failing.
+        (Builder.prototype.resultsDirectoryURL): Returns the URL for the results directory for a
+        particular build.
+        (Builder.prototype._getBuildNames): Fetches the names of all builds and passes them to the
+        callback.
+        (Builder.prototype._getFailingTests): Fetches the results.html page for the given build and
+        extracts all the failing tests listed in it, passing them to the callback.
+        (Builder.prototype._incorporateBuildHistory): Gets the failing tests for the specified
+        build, merges them into the build history, and calls the callback telling it whether the
+        next build should be fetched to provide more information.
+
+        * BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/TestFailures.css:
+        Added. Just some simple styles.
+
+        * BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/Utilities.js: Added.
+        (createDefinitionList): Takes an array of pairs and turns them into a DL element.
+        (getResource): Wrapper around XMLHttpRequest.
+        (Array.prototype.findFirst): Finds the first element matching the given predicate and
+        returns it.
+        (Array.prototype.last): Returns the last element of the array.
+
+        * BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/ViewController.js: Added.
+        (ViewController): This class contains the main logic for displaying the page.
+        (ViewController.loaded): Just calls through to parseHash.
+        (ViewController.parseHash): Either starts analyzing failures on a particular builder, or
+        shows the list of all testers so one can be chosen. This function is called when the page
+        loads and whenever we get a hashchange event.
+        (ViewController._displayBuilder): Asks the builder to fetch build history, and displays it
+        as it is fetched. The display ends up grouping tests by when they started failing.
+        (ViewController._displayTesters): Gets the list of testers and displays it.
+
+        (ViewController._domForBuildName):
+        (ViewController._domForFailedTest):
+        Helper functions to create descriptions and links for a particular build or failed test.
+
+        * BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/WebKitBuildbot.js: Added.
+        (WebKitBuildbot): Calls up to the base class constructor with the correct base URL.
+        (WebKitBuildbot.prototype.parseBuildName): Parses a build.webkit.org-style build name.
+
+        * BuildSlaveSupport/build.webkit.org-config/public_html/TestFailures/index.html: Added. Just
+        loads all the files and sets up a ViewController, which does the rest.
+
+        * BuildSlaveSupport/build.webkit.org-config/templates/root.html: Added a link to the new
+        page.
+
 2011-05-18  Kristóf Kosztyó  <[email protected]>
 
         Reviewed by Csaba Osztrogonác.
_______________________________________________
webkit-changes mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo.cgi/webkit-changes

Reply via email to