On 19 September 2017 at 07:58, Leif Oines <loi...@mozilla.com> wrote:
> here's the same FxA-based table as above, but only for multi-device users. > pretty similar - if anything, these people seem to be slightly more outdated > https://sql.telemetry.mozilla.org/queries/38604 > > Note that this query is also counting non-sync users of FxA. I don't expect that to skew the numbers too much, but worth being aware of. Ryan > version count prop > 40 and below 16,696.00 0.01 > 41.00 3,372.00 0.00 > 42.00 3,802.00 0.00 > 43.00 9,763.00 0.00 > 44.00 4,082.00 0.00 > 45.00 19,436.00 0.01 > 46.00 4,758.00 0.00 > 47.00 25,820.00 0.01 > 48.00 21,457.00 0.01 > 49.00 16,159.00 0.01 > 50.00 25,007.00 0.01 > 51.00 22,702.00 0.01 > 52.00 156,080.00 0.06 > 53.00 54,776.00 0.02 > 54.00 293,116.00 0.12 > 55.00 1,743,521.00 0.69 > 56.00 68,620.00 0.03 > 57.00 20,506.00 0.01 > > On Mon, Sep 18, 2017 at 11:54 AM, Alex Davis <ada...@mozilla.com> wrote: > >> Apparently we didn't land any client-side telemetry for sync until >>> version 47 or so (i think?). that means that when we count users based on >>> FxA (not sync) stats, we will see a longer tail of users with older >>> versions. so when making this decision maybe we should pay attention to >>> that? >> >> >> This seems about right. Since Sync telemetry is client side, we won't >> have the full history. >> >> @leif, I was reminded that we probably mostly care about multi-device >> users. If users are on Fx <40 but don't sync between 2 devices, do we >> really care? We wouldn't be providing much value to them. Any chance we can >> break this down by multi-device users instead of devices? >> >> * We stop proactively testing on versions < X, but accept and fix >>> reported bugs on that version >>> * We wont fix bugs that only affect versions < Y, but don't prevent >>> users from attempting to use them >>> * We proactively error out when we detect a login attempt with version < >>> Z >>> >> >> Talking to Karlof in SF on Friday, he proposed something similar. Seems >> like the second bullet would make the most sense. We stop fixing old >> versions but don't prevent users from using them. This way we can be a bit >> more aggressive with which version we want to support and it encourages >> users to update. >> >> >> -- >> Alex Davis // Mountain View >> Product Manager // FxA & Sync >> (415) 769-9247 >> IRC & Slack: adavis >> >> On Mon, Sep 18, 2017 at 10:54 AM, Leif Oines <loi...@mozilla.com> wrote: >> >>> Thanks for running those numbers Mark. I've tried to recreate the same >>> thing using the sync_summary_v2 derived dataset in re:dash. I suspect that >>> since you're pulling raw pings perhaps we should trust those more... but I >>> could be wrong. for comparison, below are the analogous numbers I'm getting >>> from re:dash (https://sql.telemetry.mozilla.org/queries/38529). they >>> are based on the entire sync_summary_v2 dataset for the past 7 days. >>> >>> version counts prop >>> 48 1 0 >>> 50 47,569 0.01 >>> 51 53,285 0.01 >>> 52 500,793 0.06 >>> 53 136,974 0.02 >>> 54 906,476 0.11 >>> 55 5,987,420 0.76 >>> 56 92,960 0.01 >>> 57 19,784 0 >>> >>> so, according to these numbers versions 52 and up should cover 98.7% . >>> that's pretty close to what Mark estimates. however I have a couple >>> questions/comments: >>> >>> 1. Mark you said that your numbers are based on a 10% sample, yet they >>> seem to be around the same order of magnitude as my numbers. as far as i >>> know sync_summary_v2 is not a sampled dataset, so do we know what the >>> difference there might be? >>> 2. Apparently we didn't land any client-side telemetry for sync until >>> version 47 or so (i think?). that means that when we count users based on >>> FxA (not sync) stats, we will see a longer tail of users with older >>> versions. so when making this decision maybe we should pay attention to >>> that? >>> >>> with that in mind here are the comparable numbers for the FxA data. I >>> omitted ios devices and versions with only 1 user: >>> https://sql.telemetry.mozilla.org/queries/38604 >>> >>> version counts prop >>> 40 and below 57,869.00 0.01 >>> 41.00 15,443.00 0.00 >>> 42.00 14,849.00 0.00 >>> 43.00 37,472.00 0.00 >>> 44.00 17,411.00 0.00 >>> 45.00 55,595.00 0.01 >>> 46.00 21,085.00 0.00 >>> 47.00 108,109.00 0.01 >>> 48.00 113,114.00 0.01 >>> 49.00 68,451.00 0.01 >>> 50.00 81,087.00 0.01 >>> 51.00 82,610.00 0.01 >>> 52.00 626,233.00 0.06 >>> 53.00 207,903.00 0.02 >>> 54.00 1,037,261.00 0.10 >>> 55.00 7,741,729.00 0.74 >>> 56.00 153,630.00 0.01 >>> 57.00 32,955.00 0.00 >>> >>> Here, 52 and up covers only 93.5% of users. to cover 99% we would need >>> to go back to 43. to cover ~97%, version 48. >>> >>> hope that helps! >>> -Leif >>> >>> On Mon, Sep 18, 2017 at 12:55 AM, Mark Hammond <mhamm...@mozilla.com> >>> wrote: >>> >>>> On 18/09/2017 10:20 AM, Ryan Kelly wrote: >>>> > On 15 September 2017 at 10:35, Alex Davis <ada...@mozilla.com >>>> > <mailto:ada...@mozilla.com>> wrote: >>>> > >>>> > If we were more aggressive and did 52, we'd cover 87.7% of >>>> > users... which seems *too* aggressive but perhaps we can see >>>> if >>>> > we can try to nudge people to upgrade first. >>>> > >>>> > >>>> > I'd like to make a correction. It's 92.62% of users that are 52 or >>>> > higher. >>>> > >>>> > >>>> > That does seem aggressive, but we could also consider a nuanced >>>> approach >>>> > to phasing out support, because we have the following possibilities >>>> for >>>> > what "removing support" means: >>>> >>>> I'm slightly skeptical of that data (although a quick look doesn't make >>>> it clear there's anything wrong with it, if there actually is) - but my >>>> understanding has always been that the uptake of new Firefox versions is >>>> slower than we'd like, but generally successful within the 6 week cycle. >>>> >>>> By contrast, if I look at 10% of all desktop Firefoxes that submitted a >>>> sync ping in the last 7 days I get: >>>> >>>> 49 - 1, 0.00 >>>> 50 - 26527, 0.34 >>>> 51 - 27602, 0.35 >>>> 52 - 419814, 5.31 >>>> 53 - 56468, 0.71 >>>> 54 - 257171, 3.26 >>>> 55 - 7002788, 88.63 >>>> 56 - 83928, 1.06 >>>> 57 - 26413, 0.33 >>>> >>>> so in that 10% sample, there was exactly 1 version 49. Looking at 52 >>>> (current ESR) and up gives ~99.3% of all pings. >>>> >>>> https://gist.github.com/mhammond/46906eb3de269f3969e2fffc6dd801eb >>>> >>>> FWIW, going back 28 days (still a 10% sample) shows the same basic >>>> pattern, although obviously 54 is higher due to it being the current >>>> release for some of that period. >>>> >>>> 49 - 5, 0.00 >>>> 50 - 109896, 0.36 >>>> 51 - 117934, 0.39 >>>> 52 - 1829602, 6.03 >>>> 53 - 231241, 0.76 >>>> 54 - 6847774, 22.58 >>>> 55 - 20751445, 68.42 >>>> 56 - 343482, 1.13 >>>> 57 - 98950, 0.33 >>>> >>>> (this is the same gist as above, but ndays=28 instead of ndays=7) >>>> >>>> Mark >>>> >>>> >>> >> >
_______________________________________________ Dev-fxacct mailing list Dev-fxacct@mozilla.org https://mail.mozilla.org/listinfo/dev-fxacct
