>TL;DR - Firefox does pretty well when compared to Chrome. Excellent! Cool stats!
>We've begun this project by doing a set of performance runs on >WebPageTest.com in order to compare Firefox's performance against Chrome on >the home pages of the Alexa 100 (the top 100 web sites worldwide). We've >made a visualization tool which plots all of the runs on a graph, so we may >easily compare them: > > http://valenting.github.io/presto-plot/plot.html > > - click on a domain link to see results for each site. > - you are able to view the results sorted or unsorted, or filter by the >browser version > - you can also compare metrics other than load time - such as speed >Index, number of DNS queries, etc > - you can also compare the browsers on several connectivity profiles >(Cable, 3G, Dialup) > - You can click on each individual point to go to the WPT run and view >the results in greater detail What does "run index" mean in the graphs? The values appear to be sorted from best to worst; so it's comparing best to best, next-best to next-best, etc? Ah, and I see "sorted" undoes the sorting. I'd think displaying mean/median and std-deviation (or a bell-curve-ish) might be easier to understand. But I'm no statistician. :-) It also likely is easier to read when the numbers of samples don't match (or you need to stretch them all to the same "width"; using a bell-curve plot of median/mean/std-dev avoids that problem. Thumbnails, or columns on the right for each selected browser with median (or mean), with the best (for that site) in green, the worst in red would allow eyeballing the results and finding interesting differences without clicking on 100 links....... (please!) Or to avoid overloading the page, one page with graphs like today, another with the columns I indicated (where clicking on the row takes you to the graph page for that side). >------------------ >Error sources: > >Websites may return different content depending on the UA string. While >this optimization makes sense for a lot of websites, in this situation it >is difficult to determine if the browser's performance or the website's >optimizations have more impact on the page load. Might be interesting to force our UA for a series of tests to match theirs, or vice-versa, just to check which sites appear to care (and then mark them). Great! It'll be interesting to track how these change over time as well (or as versions get added to the list). Again, medians/means/etc may help with evaluating and tracking this (or automating notices, ala Talos) -- Randell Jesup, Mozilla Corp remove "news" for personal email _______________________________________________ dev-platform mailing list [email protected] https://lists.mozilla.org/listinfo/dev-platform
