Hi everyone! Last week a GNOME Shell hackfest was held at the Canonical office in Taipei, Taiwan. The topic was fractional scaling, and this E-mail aims to summarize what was done, and future tasks. The hackfest was attended by me, Marco Trevisan and Shih-Yuan Lee.
Prior to the hackfest, mutter had already been adapted to support very rudimentary fractional scaling; mostly involving changing 'int' to 'float', hard coding the scales 1, 1.5 and 2, and various minor adaptations. During the hackfest, we made progress in many places, and I'll try to split them up into topics, and summarize the current state and future problems. 1) Overall approach In GNOME 3.24, HiDPI scaling of window content is managed simply resizing the window actor (for Wayland clients) or completely client side (for X11 clients). Currently on master, hidden behind a gsetting, this has changed to decoupling the stage coordinate space from physical pixels. In other words, depending on the scale a monitor is configured to have, the pixel density of that area of the stage may be different from other areas. This is done by transforming the viewport given a scale when drawing each monitor, causing clutter to draw things bigger. We introduce two types of coordinate spaces; logical pixel coordinate space - the coordinate space of the stage, and physical pixel coordinate space - the coordinate space of each framebuffer. Windows, actors, cursor position etc are all in logical pixel coordinate space. The framebuffer coordinate space depends on what framebuffer is referenced. It can be the framebuffer flipped to the monitor; it can be a framebuffer of a client surface buffer. For X11 clients, this means that every client will be scaled according to what monitor it is drawn on; but it also means that without further changes, X11 style client side scaling is no longer possible. Thus, until a solution to this is developed, gnome-settings-daemon will configure X11 clients that are aware of the 'Gdk/WindowScalingFactor' XSetting to never draw with anything other than scale 1. In order to support fractional scaling, to make this work there are various things to consider. For example, we wanted monitors in the stage to be positioned on the integer grid of the logical coordinate space. This means that we can't just use any arbitrary fractional scale for any monitor mode, as when scaling using such a scale would result in a non-integer-sized rectangle representing the monitor area on the stage. To solve this, we define what scales we want to support, and find scaling factors that result in integer monitor rectangles. For example, we might want to support ~1.375 scaling on a 1920x1080 resolution, and for that we find the scaling factor ~1.37931, resulting in a 1392x783 rectangle in logical pixel coordinate space. 2) Configuration API For 3.26, Mutter will expose a different configuration API than before. This work has been going on for quite a while, and is now fairly ready for actual usage. In short, it moves all complex low level monitor hardware management to mutter, exposing a higher level API. This new API allows the API user to place monitors in logical monitors (making them clones) and logical monitors in a global screen space. During the hackfest we updated the API to pass a list of supported scales for every supported mode of a monitor. Mutter was updated to expose the calculated modes (see part 1), gnome-settings-daemon was updated to deal with the API change, and gnome-control-center was updated to be able to deal with supported scales being different for each mode of a monitor. As a proof of concept, and for allowing to test many different scales, in the current branch, we list 8 scales per int, ranging from 0.5 to 4.0. This will of course be tweaked in some way or another. 3) Mutter related work Mutter was updated here and there to support fractional scaling better as well as improvements to the new way of dealing with different monitor scales. For example, taking screenshots were fixed, sub-logical-pixel cursor positioning was added, the new configuration system (D-Bus API implementation, configuration file reading/writing) was updated to deal with fractional scaling, and the accompanying precision loss issues. 4) GNOME Shell UI related work Some time was spent looking into how to make GNOME Shell look good. The issue is mainly that clutter thinks it's drawing in scale 1, but in fact everything will be scaled according to the monitor scale when being painted. The result is that low resolution images are scaled up. A proof of concept of how to deal with this had been implemented some weeks before, by exposing a new concept called "resource scale" to clutter actors. The resource scale refers to how much any resource should be scaled with to look good. For example, StIcon is changed to fetch larger icons given the resource scale, and there were promising (yet to be finalized) results of how to deal with all of the StThemeNode related drawing without having to do any Javascript changes. 5) Calculation of a default scale We spent some time discussing (internally and also with System 76 via IRC) and testing out adding automatic calculation of default scales including fractional ones. We had various hardware to test on, and have a proof of concept algorithm. This is also a topic that of course will be tweaked, given input received from tests. Canonical Taiwan provided us with various hardware to test on, so we also started a list of hardware and "nice" default scales, to get data on what magical values to use some algorithm. The test result table can be found here <https://wiki.gnome.org/Projects/GnomeShell/ScaleFactorTests>. Feel free to add more data into the table. We also discussed whether it would be beneficial to allow vendors to override any automatically calculated default scale using something similar to how libinput deals with hardware differences via hwdb. We wouldn't be able to use hwdb directly I'm assuming, as these things would use data from EDID for identifying hardware. 6) Client side support Previously, an idea has been that for scales like 1.5, tell clients to scale by 3, then downscale by 2, resulting in a target framebuffer scale of 1.5, but the benefits of this has not seemed to be worth it, thus we ended up for monitor scales between 1 and 2, tell clients to scale using scale 2 and then downscale accordingly. In practice we'll most often never scale above 2, so we did not prioritize fixing client side bugs related to drawing using scale > 2. There were various issues, such as problems with drawing shadow, warnings and possibly crashes, that needs to be dealt with before we can allow > 2 scales with good conscience. 7) Wayland protocol improvements We touched on this, and concluded that the most important use case where we cannot let clients draw larger, having the compositor later downscale the client buffer, is for performance critical applications such as games. For this, a future Wayland fullscreen protocol needs to take this into account, by for example specifying the expected framebuffer resolution, instead of specifying an expected window size in logical pixel coordinate space. Whether to add some way to let toolkits who are able to deal with fractional scaling client side avoid any compositor side scaling is an open question, and it's not clear whether the benefits are worth the needed complications. 8) XWayland support X11 clients, as mentioned previously, are now always considered scale 1 and are scaled according to the monitor scale. There have been various discussions on this previously in some places, but we didn't spend much time on this. One possible approach discussed is to add a new X11 specification allowing clients to do things similar to how Wayland surface buffer scales are handled, allowing the Wayland compositor to scale the window, while at the same time know how to transform child windows. For now XWayland completely ignores any wl_output::scale set, and directly take the current mode and geometry of a wl_output and use this to describe the X11 screen and monitor dimensions. For X11 clients to be able to maximize, fullscreen etc, they need to have the correct idea of the geometry of the monitors and screen in the logical pixel coordinate space, so mutter will lie about the current mode and say it is size the scaled logical monitor layout. To support things such as X11 clients (games) setting a temporary resolution, we probably need a custom XWayland protocol for communicating the monitor and screen dimensions, so we can pass "real" data about monitors. Further reading on HiDPI XWayland support: https://bugs.freedesktop.org/show_bug.cgi?id=93315 This more or less summarizes the high level things that were discussed and worked on during the hackfest. There is a more detailed log available here: <https://hackmd.io/KwQwxiAsCmCMwFoCcJiwTAHAEwSESiAbAOwDMYJcAZtWWdkA> On that page, you can also find various git branches with actual things to inspect and test, as well as a more detailed list of issues and things to do. Finally, many thanks to the Canonical Taipei office and staff for hosting us last week! Jonas _______________________________________________ gnome-shell-list mailing list gnome-shell-list@gnome.org https://mail.gnome.org/mailman/listinfo/gnome-shell-list
