This thread spawned a metrics deep dive, I'm going to sum those up and give a concrete proposal for Firefox desktop.
TL;DR - I propose we support down to Firefox desktop 45 and at this time, only remove support for `context=fx_desktop_v1`. Leif created [1] to show the percentage of users with activity events in the past 7 days, broken down by browser. Looking at the chart, the usage is heavily tilted to Fx 52 and above. Looking at the raw numbers, Firefox 45 is a local maximum. Numbers trail off below that. None of this is too surprising. 36 984 37 1,021 38 3,561 39 1,860 40 2,233 41 3,192 42 3,485 43 9,136 44 3,782 45 16,696 46 4,311 47 22,946 48 19,792 49 14,017 50 21,541 51 19,140 52 156,994 53 35,692 54 93,583 55 310,620 56 1,780,673 57 125,916 58 25,743 I created [2] to see which `context` query parameters are being specified on this /signin page for the past 7 days. fx_desktop_v1 9,487 fx_desktop_v2 3,342 fx_desktop_v3 282,131 fx_fennec_v1 384,930 fx_fennec_v1email 34 fx_firstrun_v2 5,410,661 fx_ios_v1 17,175 fx_sync 82 iframe 469,764 oauth 415 web 15,410 Far and away the vast majority of visits to /signin come from the newest firstrun page. This is expected. What is suprising is the #2 context is `iframe`, at nearly 10%! `context=iframe` is used by the firstrun page for Firefox <= 46. 10% is nothing to just shrug off, so I created [3] in DataDog to see the UA breakdown of the users visiting /signin with `context=iframe`. 45 176.4K 43 99.5K 41 54.2K 44 51.2K 40 41.5K 42 30.5K 47 2.97K 52 0.15K 56 0.12K 48 0.1K Fx 45 came out on top at almost 2x the nearest neighbor, Fx 43. We still have a *lot* of folks viewing the /signin page using `context=iframe`. But how many sign in? Turns out, not very many [4]. 45 550 43 130 44 100 41 70 42 50 56 50 40 20 Of the 176.4K Fx 45 users that viewed the signin page in the last week, ~550 signed in. The current ESR is based on Firefox 52. ESR - 1 is based on Firefox 45. This is one explaination why there is a local maximum around Firefox 45 in the pure counts - some folks are still on the old ESR. The good news is, the vast majority (>98.3% based on Leif's #s) are on Firefox 45 or above. ================================================ I propose we support "ESR - 1." ESR - 1, with a minimum of Firefox 45. Users below Firefox 45 will see a screen that allows them to upgrade their Firefox [6]. This will enable us to remove support for `context=fx_desktop_v1` and that's it. We went from `context=fx_desktop_v1` to `fx_desktop_v2` with Firefox 40. The change from `context=fx_desktop_v2` was in Firefox 46 [5]. `context=iframe` was used until Firefox 46. The next ESR comes out in March, based on Firefox 59 [7]. At that time, ESR - 1 will be based on Firefox 52. We'll be able to drop support for `context=iframe` and `context=fx_desktop_v2`. While dropping support for `context=fx_desktop_v1` might seem modest, it's something we haven't done before and we'll have to work through a lot of details. I think in the near term, even this modest step will pay dividends, especially in regards to our functional tests. A few folks have asked, will we actively disconnect these old browsers if they are already connected to Sync? No. They'll only be asked to sign in if they for some reason lose their session and must sign in again. Now, there is some nuance to how to approach this. The two options I see: * Any visitor with Firefox < 45 will be asked to upgrade. * Any visitor with `context=fx_desktop_v1` will be asked to upgrade. These are *not* equivalent statements. The first will ask more users to upgrade because in the second, users on Fx >= 40 but <= 46 who open FxA with `context=fx_desktop_v2` will not be asked to upgrade. I'm pretty ambivalent to tell the truth. I can make a case either way. We'll help more users update and secure themselves with the first. Thoughts? Shane [1] - https://sql.telemetry.mozilla.org/queries/38529 [2] - https://sql.telemetry.mozilla.org/queries/47220 [3] - https://app.datadoghq.com/dash/54208/fxa-content--browser-details?live=true&page=0&is_auto=false&from_ts=1508238635135&to_ts=1508843435135&tile_size=m&fullscreen=249448170 [4] - https://app.datadoghq.com/dash/54208/fxa-content--browser-details?live=true&page=0&is_auto=false&from_ts=1508238717877&to_ts=1508843517877&tile_size=m&fullscreen=249456359 [5] - https://hg.mozilla.org/mozilla-central/rev/f1573412a4ad [6] - https://github.com/mozilla/fxa-content-server/issues/5558 [7] - https://wiki.mozilla.org/RapidRelease/Calendar On Mon, Sep 25, 2017 at 6:21 PM, Alex Davis <[email protected]> wrote: > If 46 makes the most sense from an engineering standpoint, it seems to > make a lot of sense from a metrics standpoint. We'd continue to support 98% > of active users from the last 7 days. > > Questions: > - Can we easily display a message to pre-56 users to upgrade? (IIUC, we > seem to already do it in some cases) > - What quantity of work is required to remove support for old browsers? (1 > engineer in 2 weeks? 4 weeks? 6 weeks?) > - I suppose that the answer of the previous question will influence this > one but... Is this work that we want/need to plan for in Q4 OKRs? In our > draft Q4 OKRs, we didn't plan to this. We thought Q1 would make sense > combined with some performance work. Do we still feel that this makes sense? > > > -- > Alex Davis // Mountain View > Product Manager // FxA & Sync > (415) 769-9247 > IRC & Slack: adavis > > On Mon, Sep 25, 2017 at 9:39 AM, Shane Tomlinson <[email protected]> > wrote: > >> >> >> On Mon, Sep 25, 2017 at 6:50 AM, Ryan Kelly <[email protected]> wrote: >> >> >>> AFAICT, Firefox used context=fx_desktop_v1 up until Firefox 45 and our >>> switch to webchannels in this bug: >>> >>> https://bugzilla.mozilla.org/show_bug.cgi?id=1218022 >>> >>> If then used context=fx_desktop_v2 for a while one release, when we >>> landed this bug for Firefox 46: >>> >>> https://bugzilla.mozilla.org/show_bug.cgi?id=1204714 >>> >>> So IIUC dropping support for those two integrations would disable Fxa >>> login on Firefox 45 and earlier, which seems reasonable to me. >>> >> >> fx_desktop_v1 and fx_desktop_v2 are the two that cause the most slowdown. >> I'd also like to get rid of fx_firstrun_v1. fx_firstrun_v2 was used >> beginning with Firefox 46: >> >> https://github.com/mozilla/bedrock/pull/3851 >> >> >>> >>> That said, I'll note that the fx_ios_v1 broker currently extends >>> fx_desktop_v1: >>> >>> https://github.com/mozilla/fxa-content-server/blob/master/ap >>> p/scripts/models/auth_brokers/fx-ios-v1.js >>> >>> So we'll have to double-check that we don't accidentally lose test >>> coverage for iOS if we remove the desktop version. >>> >> >> >> This also needs to be considered for fx_desktop_v2=>fx_desktop_v3 and >> fx_firstrun_v1=>fx_firstrun_v2. I want to speed up development, but not >> at the expense of broken supported flows. >> >> Shane >> > > > _______________________________________________ > Dev-fxacct mailing list > [email protected] > https://mail.mozilla.org/listinfo/dev-fxacct > >
_______________________________________________ Sync-dev mailing list [email protected] https://mail.mozilla.org/listinfo/sync-dev
