That makes sense, I'll work on a revert of the enabled-by-default soon. By the looks of this comment <https://github.com/twilio/twilio-video.js/issues/1968#issuecomment-1412982869> it sounds like Twilio has released a fix to the bug, but I assume it will take some time for users to update their library.
On Wednesday, February 1, 2023 at 4:35:03 PM UTC+1 Yoav Weiss wrote: > Enabling by default on Canary and then turning it off right before branch > could definitely be a good way to get lab testers that test on Canary be > aware of this. > At the same time, the release managers may not be excited about it, > because it reduces Canary's testing relevance. > > Maybe we can settle on having this enabled at the early stages of the > Canary milestone, and then revert as we approach Beta? > > On Wed, Feb 1, 2023 at 11:49 AM Henrik Boström <h...@chromium.org> wrote: > >> Ping. Does my proposal to do enabled-by-default (so that testing >> environments don't miss this again) prior to having ramped this up to 100% >> make sense or is that considered bad practise? >> Current status is ENABLED_BY_DEFAULT in M111 (currently Canary), which >> would need to be reverted if we either want to delay this to M112 or not do >> ENABLED_BY_DEFAULT at all until 100% launch. >> >> >> On Tuesday, January 31, 2023 at 12:45:20 PM UTC+1 Henrik Boström wrote: >> >>> On Tuesday, January 31, 2023 at 11:39:36 AM UTC+1 Yoav Weiss wrote: >>> Thanks for filing an intent and moving the trial to 0% on stable. >>> >>> From responses on the other thread, it seems like there may be a few >>> months of lag between the time developers notice this upcoming change and >>> the time it'd reach users. >>> Do you know if a 3P deprecation trial would have better deployment >>> latency? >>> >>> If a 3P deprecation trial still requires the affected websites to get >>> the latest version of the affected library in order to get that trial then, >>> if I understand correctly, I don't think this will help because it sounds >>> like the next version of Twilio will contain the fix for this in which case >>> the trial would not be needed. >>> >>> It sounds like it is just a matter of waiting for updates to be pushed, >>> but please correct me if I'm wrong about this +Anna and if a trial would >>> help. >>> >>> >>> On Mon, Jan 30, 2023 at 11:20 AM Henrik Boström <h...@chromium.org> >>> wrote: >>> >>> >>> On Monday, January 30, 2023 at 11:16:59 AM UTC+1 Harald Alvestrand wrote: >>> I'm not sure an enterprise policy is appropriate - I see the same >>> problem with sunsetting the policy as with sunsetting the stat in general, >>> and usage of enterprise policies is (as far as I know) far more opaque to >>> us than origin trials or Finch feature usage. >>> >>> Oh, we definitely don't want that. If a flag is needed (other than the >>> Finch one) then I would much rather do a Reverse Origin Trial in that case. >>> I still think that has limited value but if that mitigates concerns then >>> I'm supportive of it. >>> >>> >>> >>> On Mon, Jan 30, 2023 at 11:13 AM Henrik Boström <h...@chromium.org> >>> wrote: >>> >>> >>> On Friday, January 27, 2023 at 7:24:58 PM UTC+1 Johnny Stenback wrote: >>> Is there an enterprise policy in place for this deprecation already? If >>> not, adding one seems appropriate given the challenges of rolling out even >>> simple fixes in some enterprise environments. >>> >>> One does not exist at the moment but I can add one >>> <https://chromium.googlesource.com/chromium/src/+/HEAD/docs/enterprise/add_new_policy.md> >>> . >>> >>> >>> Thanks, >>> Johnny >>> >>> On Fri, Jan 27, 2023 at 5:16 AM Henrik Boström <h...@chromium.org> >>> wrote: >>> Delaying the enabled-by-default to M112 is fine by me but I'll wait for >>> a resolution here before taking action. Currently it is enabled-by-default >>> in Canary. >>> >>> On Friday, January 27, 2023 at 12:41:23 PM UTC+1 >>> philipp...@googlemail.com wrote: >>> Am Fr., 27. Jan. 2023 um 11:49 Uhr schrieb Henrik Boström < >>> h...@chromium.org>: >>> *Contact emails* >>> h...@chromium.org, h...@chromium.org >>> >>> *Background* >>> I attempted to remove this feature before but had forgotten to file an >>> intent to deprecate, background here >>> <https://groups.google.com/a/chromium.org/g/blink-dev/c/RsIktnGhHqw/>. >>> >>> *Specification* >>> The getStats() API spec is here <https://w3c.github.io/webrtc-stats/> and >>> it contains all the metrics. The deprecated metrics are also listed, but in >>> the obsolete section >>> <https://w3c.github.io/webrtc-stats/#obsolete-rtcmediastreamtrackstats-members>. >>> >>> There's an open issue to remove obsolete metrics from the spec as soon as >>> they are unshipped from modern browsers. This is considered a blocker for >>> the document to reach Proposed Recommendation status. >>> >>> *Summary* >>> WebRTC is a set of JavaScript APIs (spec >>> <https://w3c.github.io/webrtc-pc/>) that allow real-time communication >>> between browsers. For the relevant metrics being removed, we're only >>> talking about the WebRTC use case that is sending or receiving audio or >>> video (typically Video Conferencing use cases), not the data channel use >>> cases that is a popular WebRTC use case, since data channel only use cases >>> would never have any tracks/streams. >>> >>> RTCPeerConnection.getStats() returns a map of string-to-objects, where >>> each object is one of the dictionaries defined in the stats spec. The >>> reason an app calls getStats() is mostly to report quality metrics (send >>> and receive resolutions, bitrates, glitches, video QP, etc) which can be >>> important for A/B experimentation. It can also be used in a way that >>> impacts app logic or even UX inside the app. Most common use case I can >>> think of: poll getStats() at 10 Hz and render volume bars for each >>> participant based on volume levels from stats objects. >>> >>> The deprecation in question is to remove some stats objects that were >>> made obsolete several years ago: all metrics on the "track" dictionary have >>> been moved to non-obsolete objects ("inbound-rtp", "outbound-rtp", >>> "media-source"). Reasons for wanting to deprecate include: >>> >>> - Spec-compliance: needed for browser implementations to align and >>> for the spec to become Proposed Recommendation. >>> - Web compat: Firefox never implement "track" or "steam" >>> >>> <https://wpt.fyi/results/webrtc-stats/supported-stats.https.html?label=experimental&label=master&aligned> >>> due >>> to them being obsolete. >>> - Performance: the duplicated metrics make up ~40% of the stats >>> report size, which can be a significant number of bytes in larger >>> meetings >>> and it is common for apps to poll getStats() 10 times per second. >>> - Tech debt: unblock removal of 1400 LOC. >>> >>> In the meantime, the obsolete metrics is duplicated in several places of >>> the stats report. >>> >>> *Risks* >>> *- Impossible to properly measure usage* >>> Because stats objects are exposed as JavaScript dictionaries, and >>> because apps have to iterate all objects of the stats report in order to >>> find the ones they are interested in or if they just dump all the data >>> without filtering, there is no way to measure how big the dependency is on >>> track in the real world. >>> >>> While we lack use counters, we have some positive signs: >>> >>> - Because Firefox does not have "track" or "stream" stats, any app >>> that can run on Firefox already exercises the paths of these not >>> existing. >>> >>> >>> - An experiment to "unship deprecated metrics" has been running at 50% >>> Canary since October >>> >>> <https://groups.google.com/a/chromium.org/g/blink-dev/c/RsIktnGhHqw/m/3iqjODsMBwAJ>, >>> >>> giving developers testing Canary a heads-up. Nobody complained until the >>> experiment reached Stable. >>> - We got to 50% Stable in M109 and while we're in the process of >>> rolling it back now due to breaking twilio-video.js >>> <https://github.com/twilio/twilio-video.js/issues/1968>, it's >>> interesting to note that this is the only breakage we are aware of (that >>> does not mean there aren't more breakages, but I believe this at least >>> says >>> something about the severity). >>> >>> *- Selenium et al typically starts browsers from fresh profiles and >>> hence does not know the finch trial seed* >>> The most likely explanation for breakage is not testing Canary or test >>> environments not having access to Finch experiments. This makes the >>> behavior on Stable a surprise. >>> >>> *- To have a Reverse Origin Trial or not to have a Reverse Origin Trial?* >>> Migrating should require so few lines of code (look for stats.type == >>> 'inbound-rtp' instead of stats.type == 'track', for example) that it seems >>> to be a bigger hurdle for a developer to enroll in the trial than to simply >>> fix their code. >>> >>> *- Compatiblity risk* >>> There is one particular metric out of all metrics that, if you run >>> Safari, does not exist on "inbound-rtp" yet. This can be a problem, but >>> again is probably not a big problem because this particular metric was >>> never implemented on Firefox so apps already need to survive without it, >>> and it is very easy to write a fallback path for the Safari case: >>> >>> let trackIdentifer = null; // In Firefox this will never be set >>> regardless. >>> if (inboundRtp.trackIdentifier) { >>> // Spec-compliant browser. >>> trackIdentifier = inboundRtp.trackIdentifier; >>> } else if (inboundRtp.trackId) { >>> // Fallback-path for Safari or 1+ year old Chromium browsers. >>> trackIdentifier = report.get(inboundRtp.trackId).trackIdentifier; >>> } >>> >>> *Proposal* >>> Rollback to 0% Stable but keep the "unship deprecated" experiment at 50% >>> Canary/Beta. Wait for Twilio to fix their issue and do another rollout >>> attempt. Keep a slower rollout pace next time. >>> >>> I see limited amount of value in a Reverse Origin Trial since it appears >>> to be more effort to register to the trial than to fix the issue, if you >>> are affected. >>> >>> I do prefer to have the feature enabled-by-default in M111+ and >>> overwrite that default via Finch rather than the other way around as to not >>> "turn off the fire alarm" for non-Finch testing environments. I realize >>> that is not perfect (what if you run in a non-Finch environment) but it >>> would reduce overall risk. >>> >>> Thank you Henrik. I agree with one suggestion: only do default-off in >>> M112+ (which is branching so you would just need to revert this commit >>> <https://chromiumdash.appspot.com/commit/3a4d52f365df03413a856ea20366b36e8fb8ea0b> >>> on >>> the M111 branch). >>> This gives developers another month to update (which itself should be >>> quick) and then rolling it out to their customers and users (which takes >>> time). >>> >>> I hope that the 50% rollout caused enough incidents (even though you may >>> never hear about some of them) to get the fixes in ASAP. >>> >>> cheers >>> >>> Philipp >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+unsubscr...@chromium.org. >>> To view this discussion on the web visit https://groups.google.com/a/ch >>> romium.org/d/msgid/blink-dev/5ecf1ea6-c16c-464a-b529-439e05e >>> 4feedn%40chromium.org >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/5ecf1ea6-c16c-464a-b529-439e05e4feedn%40chromium.org?utm_medium=email&utm_source=footer> >>> . >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to blink-dev+unsubscr...@chromium.org. >>> To view this discussion on the web visit https://groups.google.com/a/ch >>> romium.org/d/msgid/blink-dev/6a0b5bb9-addd-4a56-b053-1429bba >>> abe2dn%40chromium.org >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/6a0b5bb9-addd-4a56-b053-1429bbaabe2dn%40chromium.org?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/1d988518-52d2-4910-ab81-72b2198c7c83n%40chromium.org.
