+1 that providing temporary enterprise policy exceptions is standard practice <https://www.chromium.org/developers/enterprise-changes/> for breaking changes that we predict may have enterprise impact.
Rick On Mon, Mar 3, 2025 at 2:24 PM Erik Anderson <erik.ander...@microsoft.com> wrote: > Hi Geoff, > > > > My suggestion re: a policy was not to have one that is supported > indefinitely. > > > > Many high-risk deprecations have had a policy lasting for, I believe, as > little as 3 major version releases. Having such a thing helps mitigate the > concern that the risk analysis was way off (which could then mean needing > to do a stable respin if your risk analysis was off). If a policy is > available, impacted enterprises can quickly self-remediate, report what > broke once you flip over the default, and have a little bit more of a > window to plan mitigations tied to the removal of the policy (since they’d > now be aware of what broke and why). > > > > Thanks, > > Erik > > > > *From:* Ken Russell <k...@chromium.org> > *Sent:* Monday, March 3, 2025 10:39 AM > *To:* Ashley Gullen <ash...@scirra.com> > *Cc:* Geoff Lang <geoffl...@google.com>; Erik Anderson < > erik.ander...@microsoft.com>; Rick Byers <rby...@chromium.org>; David > Adrian <dadr...@google.com>; blink-dev <blink-dev@chromium.org>; > geof...@chromium.org <geoffl...@chromium.org> > *Subject:* Re: [EXTERNAL] Re: [blink-dev] Intent to Remove: SwiftShader > Fallback > > > > It's feasible, but a significant amount of engineering work that our > (Chrome Graphics) team would not be able to prioritize versus other current > work that would impact a larger user base. > > > > -Ken > > > > > > > > On Fri, Feb 28, 2025 at 9:45 AM 'Ashley Gullen' via blink-dev < > blink-dev@chromium.org> wrote: > > Is it feasible to have SwiftShader (or WARP) run in its own process with a > stronger sandbox? > > > > > > On Fri, 28 Feb 2025 at 15:25, Geoff Lang <geoffl...@google.com> wrote: > > Hey Erik, Ashley, Rick, > > > > I want to be clear that I think having high WebGL availability is a good > thing. I don't think that users with software WebGL have a great experience > but it's likely better than no availability, at least for drawing static > things. What pushes this over the line and warrants this discussion is that > JITing code in the GPU process is a huge vulnerability and is a rapidly > increasing attack target. > > > > We're investigating WARP as an alternative on Windows. You are right that > a large portion of the SwiftShader fallback is on machines with no GPUs > (headless or VMs). There are just many unknowns about the quality and > security of WARP, it will take a while to be confident in such a change and > it still does not resolve the issue of JITing code in the weakly sandboxed > GPU process. > > > > Regarding corporate policy, I'd much rather have these users fall back in > the same way as everyone else and work towards lowering the number of users > in this position. It would mean supporting and testing a feature only used > by enterprise users when we have no visibility into crashes, bugs or > vulnerabilities that they face. > > > > We're also disabling software fallback due to a crashes in the GPU driver > (as opposed to blocklisted GPU). Right now any user can fairly easily > trigger a GPU crash and fall back to software WebGL which opens up > vulnerabilities to all users instead of the 2.7%. > > > > Geoff > > > > On Thu, Feb 27, 2025 at 3:28 PM Erik Anderson <erik.ander...@microsoft.com> > wrote: > > Hi David, > > The initial message states that SwiftShader primarily covers older Windows > devices. Beyond those, there are a non-trivial set of enterprise users that > use thin clients to connect to a remote Windows device which is often > running in a VM without access to a physical GPU. > > For example, this applies to the Microsoft Dev Box offering ( > https://azure.microsoft.com/en-us/products/dev-box/). > > > > Unfortunately, enterprise clients often turn off telemetry. So, I would > assume any UMA-derived metrics to be undercounting the population. > > It’s likely there are certain line-of-business and/or consumer-oriented > sites that have a hard dependency on WebGL to be fully functional. > > Have you considered, on Windows, targeting WARP ( > https://learn.microsoft.com/en-us/windows/win32/direct3darticles/directx-warp) > instead? I don’t know if there are other viable alternatives on other OSes, > but if the primary impacted clients are Windows perhaps that would be a > sufficient mitigation. > > To help enterprise customers reason about how much this is going to impact > them, it would be helpful to have an enterprise policy to control this. > This is a common pattern for potentially high-impact changes. > > In its initial phase, the policy would enable motivated enterprises to > forcibly disable SwiftShader as a scream test. And after you switch over > the default, it could enable enterprises caught unaware to have some > additional window of time to plan mitigations (by re-enabling it via > policy) before you proceed with fully deprecating support and remove the > policy. > > Can you comment on if you plan to add such a policy or, if not, why not? > > Thanks! > > > > *From:* 'Ashley Gullen' via blink-dev <blink-dev@chromium.org> > *Sent:* Thursday, February 27, 2025 4:14 AM > *To:* Rick Byers <rby...@chromium.org> > *Cc:* David Adrian <dadr...@google.com>; blink-dev <blink-dev@chromium.org>; > geof...@chromium.org <geoffl...@chromium.org> > *Subject:* [EXTERNAL] Re: [blink-dev] Intent to Remove: SwiftShader > Fallback > > > > Thanks for the response Rick, I agree with much of what you've said and I > think your views and suggested workarounds are all generally reasonable. > > > > I just realised I previously responded to this thread but only replied to > David - for transparency I've copied my previous response below. > > > > I can confirm all content made with Construct since about 2018 requires > WebGL to work and will show an error message if WebGL is unavailable. I've > included a screenshot of the message Construct content published to the web > will display when WebGL is not supported, saying "Software update needed", > since that has usually been the best advice in that situation in the past. > As my previous message says we long ago removed any other fallback and are > now likely too dependent on WebGL to feasibly reimplement a canvas2d > fallback. > > > > Some other thoughts about workarounds/mitigations: > > - A swiftshader WASM module would at least give us a workaround, but > if that was something like a ~10 MB+ module it would be a very high > download overhead which we'd be obligated to include in every Construct > export for compatibility > - Swiftshader could be removed from insecure origins with little > impact to us, and using a permission policy for cross-site iframes should > be straightforward to work with > - If it helps reduce the attack surface, we could live with > SwiftShader support for WebGL 1 only (no WebGL 2) with minimum capabilities > (no extensions). > - A permission prompt to the user is not ideal but better than > nothing, and I imagine it would be tricky to explain to a normal web user > though the prompt message (and makes obtaining a WebGL context async...) > - Regarding getting WebGL to work on more devices, as I mentioned in > my previous message, reviewing the GPU blocklist to re-enable WebGL for > older devices if drivers have been updated or workarounds for issues can be > found would help reduce the number of devices subject to SwiftShader. Being > able to enable at least WebGL 1 will still help with Construct content. > - If a software fallback can be securely implemented for WebGPU, > Construct has a WebGPU renderer too now so that would give us a workaround > (and potentially for any other WebGL content - AFAIK many widely used > libraries like three.js now either support WebGPU or are working on it) > > Thanks for the consideration all. > > > > Copy of my previous message: > > ----- > > > > OK, thanks for the information. I just want to point out that even > stopping WebGL content for only 2.7% of users is still potentially very > disruptive. Consider a web game on Poki that requires WebGL and gets a > million players. With this change, now 27,000 users will see a "WebGL not > supported" error message. That's then potentially a huge number of new > support requests to deal with. > > > > > Can you share the number for Construct about what percentage of your > users are using the SwiftShader fallback? Again, our numbers indicate that > these are primarily older Windows workstations. > > > > For the Construct editor itself, it is around 3%, so that seems in line. > But the key point here is that Construct is middleware: it is a tool our > users develop web games in and then publish independently of us. It is much > more important that WebGL works for players of those games than it does for > Construct itself. > > > > Lots of people use older Windows workstations. We've had issues before > where for example a graphics driver bug affecting WebGL 1 caused a great > deal of trouble in a South American market, even though I suspect it only > affected a small percentage of devices - see > https://issues.chromium.org/issues/40941645 which was never resolved. > There are probably places in the world where there are large numbers of > people using older Windows workstations. I fear that pulling WebGL support > from those devices may result in much higher numbers of unsupported users, > and many more support requests, in the specific markets where such devices > are common. > > > > Is there anything that can be done to mitigate this change? Given > SwiftShader allowed WebGL to be considered ubiquitous for many years, > engines like Construct long ago removed any fallback for systems that do > not support WebGL; we moved forward assuming we could rely on WebGL, and so > now it's probably infeasible to bring back any fallback as we have too many > key features that fundamentally require WebGL. Could SwiftShader be adapted > to not use JIT? Could some other fallback be found? Could the GPU blocklist > be revised to enable WebGL on as many older devices as possible? > > > > I think the number of affected users should be <1% to minimise the impact > from such a change. At web scale 2.7% is still a lot. Perhaps with revising > the GPU blocklist and adding more workarounds this is feasible. I fear if > this goes ahead without any mitigation, it will cause a great deal of > trouble and is exactly the kind of thing sceptics of the web will bring up > to say that web technology sucks, browsers can't be trusted, and people > should just develop desktop games instead. > > > > > > On Tue, 25 Feb 2025 at 22:31, Rick Byers <rby...@chromium.org> wrote: > > Sorry for the delay from API owners, as discussed on chat the chromestatus > entry wasn't set up properly to request API owner review (now fixed). > > > > This is a tricky one indeed (thanks for your input Ashley!). It looks like > <https://chromestatus.com/metrics/feature/timeline/popularity/4026> WebGL > is used on about 20% of page loads, so 2.7% of that is ~0.5% of page loads > which is very high risk according to our rules of thumb > <https://docs.google.com/document/d/1RC-pBBvsazYfCNNUSkPqAVpSpNJ96U8trhNkfV0v9fk/edit?tab=t.0#heading=h.mqfkui78vo5z> > . > > > > Of course that's an upper-bound, how many will have a fallback? One option > would be to collect some UKM data for SwiftShader usage and review a random > ~50 sites to observe the user experience in practice. That could give us a > better sense of what the real user impact would likely be. Or Maybe Ashley > can give us some examples of some web games just to confirm they indeed go > from being playable to unplayable without swiftshader on some specific > devices? David, do you have a device yourself you can test with that > doesn't support GPU WebGL? > > > > Regardless, unless sites have been really good about almost always falling > back somehow, I suspect we'll find that there's enough end-user impact to > make this a high-risk change (but I could be convinced otherwise such as > via a thorough UKM analysis). In which case then we could start working > through our playbook for a phased plan for risky breaking changes. Not > unlike what we did for flash removal > <https://www.chromium.org/flash-roadmap/>, or WebSQL > <https://developer.chrome.com/blog/deprecating-web-sql> (both big > security benefit but big web compat risk). For example: > > - Explore whether we can build swiftshader into a wasm module that > sites can use as a (probably even slower) fallback themselves. This turned > out to be the key to making WebSQL deprecation tractable. In general our > policy > > <https://docs.google.com/document/d/1RC-pBBvsazYfCNNUSkPqAVpSpNJ96U8trhNkfV0v9fk/edit?tab=t.0#heading=h.x5bhg5grhfeo> > is that we don't take functionality away that developers can't replace with > some other substitute except in pretty extreme circumstances. > - Prompt the user on whether or not to enable it per-origin (like a > permission) > - Put 3p usage behind a permission policy so the top-level site has to > opt-in to allow 3p iframes to use swiftshader > - Rely on some heuristics, (perhaps crowd-sourced signals) to try to > find a sweet spot in the safety vs. functionality tradeoff space. We did > this for flash initially with things like blocking it for very small > canvases. > - Anything we can do to make WebGL work on a larger set of devices? > - Probably lots of other ideas that aren't occurring to me right now, > more examples in bit.ly/blink.compat. > > On the other side of the equation, API owners can be convinced to accept > more compat risk the more significant the security benefits are. Are there > more details you can share? Such as: > > - Are we confident that an attacker can only trigger swiftshader on > somewhere around 3% of users (vs. some knob which can force it to be used > on a larger fraction)? To what extent do we have reason to believe that the > vulnerable population size is large enough to be a plausible target for > attackers? Is there anything we can do to make the vulnerable user base > more reliably contained? > - How does swiftshader compare to other areas in terms of the number > of vulnerabilities we've found in practice? Are there any reports of ITW > exploits of it? It looks like > > <https://chrome-commit-tracker.arthursonzogni.com/cve/reward_per_components?start=2019-12-27&end=2025-02-25> > since 2020 SwiftShader has been about 8% of Chrome's VRP spend - that seems > quite significant to me, but probably not in the top 5 areas of concern. > This was obviously key to the immense cost and pain of Flash removal - we > kept having severe security incidents in practice. > > So assuming Ashley and I are right that this isn't likely to be easy, that > means it's likely quite a lot of work to attempt to phase-out SwiftShader > in a responsible fashion. But with luck maybe we can find a first step that > is a good cost-benefit tradeoff (like putting 3P usage behind a permission > prompt)? Or maybe it's just a better cost-benefit tradeoff to invest in > other areas which pose a threat to a greater number users (hardening ANGLE > perhaps)? But of course I will defer to the judgement of security and GPU > experts like yourself on that question, I'm happy to consult and support if > you want to invest in a plan that API owners can approve. > > > > Rick > > > > > > > > On Wed, Feb 19, 2025 at 2:48 PM 'David Adrian' via blink-dev < > blink-dev@chromium.org> wrote: > > > I wrote about this previously but I'm still concerned this is a major > breaking change for existing published WebGL content on the web. If the > figure of 2.7% comes from my previous citing of Web3DSurvey > > It does, not it comes from Chrome's metrics system. > > > Does Google have their own internal data about the usage of SwiftShader? > > It is the 2.7% number. > > > Suppose this change rolls out and we get reports that say our WebGL > content no longer works for 10% of users in a South American market. Then > what? There is nothing feasible we can do about it. These customers were > previously getting by with SwiftShader, but now they get an error message. > So I fear this risks disaster for web games in some markets. > > > I mentioned I don't think it should be used as evidence to make such a > big change as this. Maybe in some places it will affect 25% or 50% of users > - who knows? How can we be sure? > > Can you share the number for Construct about what percentage of your users > are using the SwiftShader fallback? Again, our numbers indicate that these > are primarily older Windows workstations. Notably, SwiftShader is not used > at all on mobile. > > > V8 does JIT with untrusted JavaScript code and that is generally > considered reasonably secure, is there any particular technical reason > SwiftShader is not considered as secure? > > Yes. The GPU process is shared between all sites, whereas the V8 JIT is > per-site. This means compromising the GPU process can be enough to bypass > site isolation protections with a single bug. Additionally, V8 bugs can be > reliably patched in the browser, whereas SwiftShader "bugs" can be > user-mode graphics driver bugs that are simply more exposed via SwiftShader > than they would be otherwise. In this case, the browser can't patch the bug > because it's in the driver. > > > On Thursday, February 13, 2025 at 12:12:07 PM UTC-5 ash...@scirra.com > wrote: > > I wrote about this previously but I'm still concerned this is a major > breaking change for existing published WebGL content on the web. If the > figure of 2.7% comes from my previous citing of Web3DSurvey ( > https://web3dsurvey.com/) then this should be seen as very much an > underestimate, because that site uses a relatively small sample size that > is more likely to be focused on high-end devices (samples are taken from > developer-focused sites like the three.js website, WebGPU fundamentals > etc). I would not be surprised if the real worldwide average was more like > 4-5%. Then if that's a worldwide average, there will probably be some > specific countries or markets where the figure could be more like 10%. > > > > Suppose this change rolls out and we get reports that say our WebGL > content no longer works for 10% of users in a South American market. Then > what? There is nothing feasible we can do about it. These customers were > previously getting by with SwiftShader, but now they get an error message. > So I fear this risks disaster for web games in some markets. > > > > Does Google have their own internal data about the usage of SwiftShader? > Can more data about this be shared? I respect the work done by Web3DSurvey > but unfortunately for the reasons I mentioned I don't think it should be > used as evidence to make such a big change as this. Maybe in some places it > will affect 25% or 50% of users - who knows? How can we be sure? > > > > Can there not be some other fallback implemented? V8 does JIT with > untrusted JavaScript code and that is generally considered reasonably > secure, is there any particular technical reason SwiftShader is not > considered as secure? > > > > I'd also point out that any website that has a poor experience with > SwiftShader can already opt-out using the failIfMajorPerformanceCaveat > context flag. If there is some other mode that can be used instead, or just > showing an error message is acceptable, then websites can already implement > that. In our case with Construct we specifically attempt to obtain > hardware-accelerated WebGPU, WebGL 2, or WebGL 1; only failing that do we > resort to using SwiftShader on the basis that showing the content with > potentially poor performance is better than not showing it at all. > > > > > > On Thu, 13 Feb 2025 at 15:46, 'David Adrian' via blink-dev < > blin...@chromium.org> wrote: > > Contact emails > > dad...@google.com, geof...@chromium.org > > > Summary > > Allowing automatic fallback to WebGL backed by SwiftShader is deprecated > and will be removed. This has been noted in DevTools since Chrome 130. > > > > WebGL context creation will fail instead of falling back to SwiftShader. > This is for two primary reasons: > > > > 1. SwiftShader is a high security risk due to JIT-ed code running in > Chromium's GPU process. > > 2. Users have a poor experience when falling back from a high-performance > GPU-backed WebGL to a CPU-backed implementation. Users have no control over > this behavior and it is difficult to describe in bug reports. > > > > SwiftShader is a useful tool for web developers to test their sites on > systems that are headless or do not have a supported GPU. This use case > will still be supported by opting in but is not intended for running > untrusted content. > > > > To opt-in to lower security guarantees and allow SwiftShader for WebGL, > run the chrome executable with the --enable-unsafe-swiftshader command-line > switch. > > > > During the deprecation period, a warning will appear in the javascript > console when a WebGL context is created and backed with SwiftShader. > Passing --enable-unsafe-swiftshader will remove this warning message. This > deprecation period began in Chrome 130. > > > > Chromium and other browsers do not guarantee WebGL availability. Please > test and handle WebGL context creation failure and fall back to other web > APIs such as Canvas2D or an appropriate message to the user. > > > > SwiftShader is an internal implementation detail of Chromium, not a public > web standard, therefore buy-in from other browsers is not required. The > devices covered by SwiftShader (primarily older Windows devices) are likely > already incompatible with WebGL in other browsers. > > > > SwiftShader is not used on mobile; this only applies to Desktop platforms. > > > Blink component > > Blink>WebGL > <https://issues.chromium.org/issues?q=customfield1222907:%22Blink%3EWebGL%22> > > > Motivation > > > https://chromium.googlesource.com/chromium/src/+/main/docs/gpu/swiftshader.md#automatic-swiftshader-webgl-fallback-is-deprecated > > > Risks > > > > SwiftShader is used by devices without hardware acceleration for WebGL. > This is approximately 2.7% of WebGL contexts. However, WebGL is considered > fallible and in many cases, these draws are not performant and provide an > effectively unusable experience for users. Many applications, such as > Google Maps, prefer to fail out rather than use SwiftShader. > > > Debuggability > > None > > > Flag name on about://flags > > --enable-unsafe-swiftshader command-line switch. > > > Finch feature name > > AllowSwiftShaderFallback > > > Tracking bug > > https://issues.chromium.org/40277080 > > > Launch bug > > https://launch.corp.google.com/launch/4351104 > > > Estimated milestones > > Shipping on Desktop 137 > > > Link to entry on the Chrome Platform Status > > https://chromestatus.com/feature/5166674414927872?gate=5188866041184256 > > > > This intent message was generated by Chrome Platform Status > <https://chromestatus.com/>. > > > > -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to blink-dev+...@chromium.org. > To view this discussion visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGkh42KV4DrSSyEgJaF4DnFOXAye-wRLrfD-LKGNkWhyWzshLA%40mail.gmail.com > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAGkh42KV4DrSSyEgJaF4DnFOXAye-wRLrfD-LKGNkWhyWzshLA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to blink-dev+unsubscr...@chromium.org. > To view this discussion visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c5131675-dff4-4aa0-8e84-4cdc373e3035n%40chromium.org > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/c5131675-dff4-4aa0-8e84-4cdc373e3035n%40chromium.org?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to blink-dev+unsubscr...@chromium.org. > To view this discussion visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAABs73jWBkuxvj%3DDDXmEQNwLfCa_uV5OZZ5nZJRj9ZMgP9yk7Q%40mail.gmail.com > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAABs73jWBkuxvj%3DDDXmEQNwLfCa_uV5OZZ5nZJRj9ZMgP9yk7Q%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- > You received this message because you are subscribed to the Google Groups > "blink-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to blink-dev+unsubscr...@chromium.org. > To view this discussion visit > https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAABs73hYs9O-2hdmfn37fQb-U-8m_-08i3Qg9dkUhKNQQvNLSg%40mail.gmail.com > <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAABs73hYs9O-2hdmfn37fQb-U-8m_-08i3Qg9dkUhKNQQvNLSg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+unsubscr...@chromium.org. To view this discussion visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAFUtAY_L3Xrt6Sk%3DDfnvVzTA9ASU6yTjt-2z5u4Gdfgs_RYdrw%40mail.gmail.com.
