Hi folks, I now have some preliminary numbers to share with the group based on our Firefox experiments. The executive summary is that our data confirms Google's results. More detail below.
EXPERIMENTAL DESIGN This is a forced experiment in which each client tries all the variants. The experiment is deployed via a system add-on (a remotely deployable, centrally managed piece of JavaScript code), and then takes measurements by trying to do an XHR to a given URL (https://mail.google.com/robots.txt) with a specific set of flags. We do the following three measurements: - TLS 1.2 - TLS 1.3 draft-18 - TLS 1.3 draft-18 with (approximately) PR#1092 ("7e02") We take five trials for each measurement, randomly shuffling the measurement order and then repeating the shuffled pattern five times. Each trial is done with a different connection and we declare "success" when any of the five trials succeeds. RESULTS This experiment was run on a 2% sample of the Firefox Beta population who have locale set to en-US, which we selected because of very high GMail blocking rates in some locales, which is a potential confounding factor. The experimen started 11/27 and has been running through today. This gave us an initial population of 161578, of whom 160809 (99.5% completed the experiment and reported results). This produced the following results: Success Failure Fail Rate -------------------------------------------------------- TLS 1.2 158260 2549 .0158 TLS 1.3-18 158194 4743 .0291 TLS 1.3-Experiment 158194 2615 .0163 For the statistics minded, the difference between -18 and 1.2 is significant at p < .001 and the 95% confidence interval of the failure rate difference is .0122-.0143 (using R's prop.test). There is no significant difference between 1.2 and 1.3-experiment (p = .36). We've got a -22 experiment in flight now, but it will only be on Nightly, so this is probably the strongest data we will have for a while. -Ekr ADDITIONAL DETAILS The relevant NSS version: https://dxr.mozilla.org/mozilla-beta/source/security/nss/lib/ssl Experimental code: https://github.com/mozilla/one-off-system-add-ons/tree/master/addons/tls13-middlebox-ghack iPython Notebook with analysis: https://gist.github.com/ekr/598208b5399faf303453b10cb11647bf
_______________________________________________ TLS mailing list TLS@ietf.org https://www.ietf.org/mailman/listinfo/tls
