Re: [webmessaging] Need to analyze test failures

2014-12-17 Thread Arthur Barstow

Update below ...

Hi All,

The Web Messaging testing data now shows relatively good 
interoperability (see [All],[2]). However, there are two tests that 
have only one pass:


1. http://www.w3c-test.org/webmessaging/with-ports/026.html. Only 
passes on FF.


2. http://www.w3c-test.org/webmessaging/without-ports/025.html. Only 
passes on IE.


I welcome feedback on whether these failures are considered minor/major 
implementation bugs, minor/major interoperability problems; that is, 
should we propose moving [CR] to Proposed Recommendation with these 
failures, or block until there are two or more passes.


-Thanks, AB

[All] http://w3c.github.io/test-results/webmessaging/all.html
[2] http://w3c.github.io/test-results/webmessaging/less-than-2.html
[CR] http://www.w3.org/TR/2012/CR-webmessaging-20120501/


On 11/23/14 6:54 PM, Arthur Barstow wrote:

Hi Cindy, Kris, All,

Earlier today I ran the webmessaging tests on Chrome, FF, IE and Opera 
(12.16) and uploaded the results plus the wptreport files to 
[test-results]. The good news is that 104/118 tests (~90%) have 2 or 
more passes, however, there are also 14 tests with less than passes.


Do you have some time to look at the [2] failures? For each of these 
failures, it would be helpful to know which ones are actually test 
case failures versus implementation bugs, and in the case of impl 
bugs, to get a sense of their priority (f.ex. {low,medium,high} impact 
on interoperability.


Naturally, we would appreciate help from anyone that is willing to 
review these failures.


-Thanks, ArtB

[test-results] 
https://github.com/w3c/test-results/tree/gh-pages/webmessaging

[2] http://w3c.github.io/test-results/webmessaging/less-than-2.html





[webmessaging] Need to analyze test failures

2014-11-23 Thread Arthur Barstow

Hi Cindy, Kris, All,

Earlier today I ran the webmessaging tests on Chrome, FF, IE and Opera 
(12.16) and uploaded the results plus the wptreport files to 
[test-results]. The good news is that 104/118 tests (~90%) have 2 or 
more passes, however, there are also 14 tests with less than passes.


Do you have some time to look at the [2] failures? For each of these 
failures, it would be helpful to know which ones are actually test case 
failures versus implementation bugs, and in the case of impl bugs, to 
get a sense of their priority (f.ex. {low,medium,high} impact on 
interoperability.


Naturally, we would appreciate help from anyone that is willing to 
review these failures.


-Thanks, ArtB

[test-results] 
https://github.com/w3c/test-results/tree/gh-pages/webmessaging

[2] http://w3c.github.io/test-results/webmessaging/less-than-2.html