On Saturday, September 21, 2019 at 6:19:29 PM UTC-7, Ryan Sleevi wrote: > On Sat, Sep 21, 2019 at 7:52 PM Kirk Hall via dev-security-policy < > dev-security-policy@lists.mozilla.org > <mailto:dev-security-policy@lists.mozilla.org>> wrote: > >> To remedy this, Entrust Datacard surveyed all of its TLS/SSL web server >> certificate customers over three days (19-21 September 2019) concerning >> website identity in browsers, browser UIs in general, and EV browser UIs in >> particular. We have received 504 responses from customers to date, and >> more responses are still coming in. Respondent company size ranged all the >> way from 1-99 employees to over 20,000 employees.
[snip] > 3) Are the numbers Entrust DataCard provided in > https://cabforum.org/wp-content/uploads/23.-Update-on-London-Protocol.pdf > <https://cabforum.org/wp-content/uploads/23.-Update-on-London-Protocol.pdf> > still accurate? That is, do EV certificates account for only 0.48% of the > certificate population? > > If those numbers are correct, this seems like it's a survey that represents > a small fraction of Entrust DataCard's customers (unless Entrust DataCard > only a few thousand customers), which represents a small fraction of > connections in Mozilla Firefox (approximately 0.3% over a 2 month period), > regarding certificates that account for only 0.48% of the certificate > population. > > Is that the correct perspective? [PW] The following response is to address the questions/comments regarding dataset type and size. Sean Ellis [1] was the head of marketing at LogMeIn and Uproar from launch to IPO. He was the first marketer at Dropbox, Lookout and Xobni, and he coined the term "growth hacker" in 2010. So, he knows a thing or two when it comes to product/market fit research - including the type of questions to ask, and the size of the dataset required to derive a good understanding of the responses. According to Ellis, the goal for a customer survey is to get feedback from people who had recently experienced "real usage" of the product. The key question in the survey for these people according to Ellis, is: "How would you feel if you could no longer rely on MetaCert's green shield? a) Very disappointed b) Somewhat disappointed c) Not disappointed d) N/A I no longer use the product According to Ellis, to get an indication of product/market fit, you'll want to know the percentage of people who would be "very disappointed" if they could no longer use your product. In his experience, it becomes possible to sustainably grow a product when it reaches around 40% of users who try it that would be "very disappointed" if they could no longer use it. For this percentage to be meaningful, you need to have a fairly large sample size. In Ellis' experience, a minimum of 30 responses is needed before the survey becomes directionally useful. At 100+ responses he is much more confident in the results. Based on Ellis' observations, it would appear that Entrust DataCard's dataset is big enough. I'm not debating the merits of the research, as I have my own research to prove that browser-based visual indicators for website identity does protect end-users - but only when designed properly. [1] https://blog.growthhackers.com/using-product-market-fit-to-drive-sustainable-growth-58e9124ee8db <https://blog.growthhackers.com/using-product-market-fit-to-drive-sustainable-growth-58e9124ee8db> Regards, Paul _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
