On Wednesday, April 3, 2019 5:36 AM, Peter Hutterer <[email protected]> wrote: > I've been trying to sort out the new hi-res wheel scrolling that was added to > linux 5.0 but it's been a bit of a struggle to say the least. For the > impatient, skip forward to the protocol diff but the background info is > important.
Thanks for taking the time to write this big summary. It's pretty helpful! > Everything below applies equally to horizontal and vertical scrolling, both > directions and wheels and wheel tilts. > > The APIs that are involved for low-resolution wheel scrolling: > > kernel: > - REL_WHEEL for every wheel click > > libinput uses pointer axis events for wheels: > - pointer axis value: the movement of the wheel in degrees > - pointer axis discrete value: the number of wheel clicks > - pointer axis source: set to 'wheel' > > Usally this means you get discrete 1, value 15 for the standard 15-degree > mouse > wheel, discrete 2 value 30 is two wheel clicks within one hw frame. > > There is one implicit assumption here: the discrete value must > be 1 or more for every event. Otherwise it's impossible to use the value and > calculate the angle. More on that later. > > This API is effectively the same in the wayland protocol except for the unit > of > the 'value' which is is in screen coordinates, not degrees. > > wayland protocol: > - pointer axis value: the movement in screen coordinates > - pointer axis discrete: the number of wheel clicks > - pointer axis source: set to 'wheel' > > The big difference here though: libinput's API is per-device, wayland's API is > per wl-seat pointer. There's no way to tell in the protocol that you're > scrolling simultaneously with wheels on two different mice. > > Let's see how the common compositors implement this: > > weston: > - for source wheel, only the discrete value is handled and: > - pointer axis discrete is sent as-is from libinput > - pointer axis source is sent as-is from libinput > - pointer axis value is always (discrete * 10) > - otherwise the libinput pointer value (degrees) from a source wheel is > ignored > > The magic multiplier 10 is afaik for historical reasons, weston used to do 10 > so early libinput took that. When libinput switched to use physical dimensions > instead of 10, that value was kept for backwards compatibility with existing > wayland clients. > > mutter: > - for source wheel, only the discrete value is handled and: > - clutter-internal event for the discrete **direction** from libinput > - i.e. discrete > 1 will still only generate one event,some events > drop to the floor here for fast wheel movements > - clutter-internal event with (value * 10) marked as 'emulated' but > that event is ignored later when we're writing to the protocol > - the discrete event is written as wl_pointer.axis_discrete and > wl_pointer.axis with a value of 10 > > So, the same multiplier of 10 but fast scroll events with more than one click > per frame appear to get dropped. > > kwin (through qtwayland): > - doesn't handle discrete at all afaict > - pointer axis events are passed through but I got a bit confused over the > actual value written on the wire here. > > wlroots: > - pointer axis discrete is sent as-is from libinput > - pointer axis source is sent as-is from libinput > - pointer axis value is sent as-is from libinput > > Because the physical angle is passed on from libinput as screen coordinates, > this means wlroots has different scroll behaviour to GNOME/sway and it does > scroll a larger distance for wheels with higher click angles. This is the > opposite of the intended behaviour, manufacturers advertise these wheels > as slow-scrolling wheels. Realistically though the physical movements is the > same-ish and if a client uses discrete it doesn't matter anyway. Hmm, that's interesting. We wlroots devs have had assumed you'd want a given angle to always scroll the same amount of pixels. But this doesn't matter that much because we have a configuration option for the scroll multiplier anyway. We still should have good defaults of course. > xf86-input-libinput: > - X devices need to set up a scroll distance for scroll motion, the driver > uses 120 (since Feb, before it was 15). This means any multiple of 120.0 > triggers legacy button emulation in the server, otherwise the number has no > meaning. > - for source wheel it uses the discrete value only, so effectively 120 * > discrete. > > This driver had a division by 0 for discrete values of 0, fixed in the > current release. More on this later. > > > > Let's look at how wheel events are processed in some of the clients: > > GTK: > - for a discrete event, GDK_SCROLL_UP is passed on flagged as 'emulated' > - the axis value is passed on as GDK_SCROLL_SMOOTH > > Qt: > - doesn't handle discrete > - multiplies the value by -12. This gives it a 120-based value which > matches the Windows API provided the axis value is 10 of course. > Qt on wlroots is probably off here. > > Xwayland: > - X devices need to set up a scroll distance for scroll motion, Xwayland > currently uses 1.0 for that. This means any multiple of 1.0 triggers legacy > button emulation in the server. > - wl_pointer.axis_discrete are used where available > - wl_pointer.axis uses the value * 0.1. This effectively means Xwayland > relies on the deltas to be 10 like in mutter/weston, X clients under > wlroots will have off wheel click emulation. Xwayland uses axis_discrete when available, so wlroots should be fine (we sent the patch doing that when we decided not to implement axis like Weston IIRC). > With me so far? Hooray, now let's introduce hi-res scrolling. > > All the examples below are for a quarter scroll wheel movement, i.e. you get > 4 hi-res events for every 1 lo-res event. The kernel uses a fraction/multiple > of 120 for logical clicks, same as the Windows API. So one wheel click is 120, > half a click is 60, etc. I'm calling this 120-based value the v120 because I'm > very creative. > > kernel: > - REL_WHEEL for every wheel click > - REL_WHEEL_HI_RES for every fraction of a wheel click (as portion of 120), > so a quarter-wheel stop gives you a value of 30. > - there is no guarantee that REL_WHEEL is sent every full 120 because e.g. > Logitech mice may reset halfway through a wheel motion. So the two axes > must be considered completely independent. > > libinput: > > Basically: the current API is fairly useless in handling fractional scroll > because libinput uses physical distances. libinput could send four events: > - value 3.75, discrete 0 > - value 3.75, discrete 0 > - value 3.75, discrete 0 > - value 3.75, discrete 1 > > Which is technically correct. But callers have no reliable way how much of a > fraction of a wheel the 3.75 represents until the first discrete event comes > in. Guessing is unreliable, if you scroll faster you may get 2+ fractional > units in one event. And mice may reset halfway through a scroll motion so you > don't always get the same number of events per discrete. So libinput needs > some > extra API that gives us some baseline to compare values against, i.e. same as > the kernel's 120-based API does. > > There are other problems, specifically related to sending discrete 0 events: > > weston: > - Current weston: a discrete value of 0 ends up as value of 0 which weston > treats as axis_stop event. So our event sequence is axis_source, > axis_stop, pointer_frame. this breaks scrolling. > > mutter: > - Current mutter: a discrete value of 0 ends up in noops. > mutter still generates internal clutter events but they are never pushed > onto the wire. The normal values are ignored, so we basically > get low-res scrolling just as before. > Note: this is based on reading the code, not test runs > > wlroots: > - will forward value/discrete as it comes in from libinput We're in the same boat as Weston here: the special value 0 will be sent as axis_stop. > > kwin: > - doesn't use discrete so won't be affected by changes > > xf86-input-libinput: > - anything pre 0.28.2 will get a division by 0 > - current state: discrete values of 0 end up as noops and we we get lo-res > scrolling only > > So basically: if we change libinput to send discrete 0 events, we'll break > weston and all but the most recent xf86-input-libinput. Code changes are > needed > just to cope with the different event sequence, let alone the actual hi-res > bits. Agreed. > The solution to this is to add a new event, LIBINPUT_EVENT_POINTER_AXIS_WHEEL. > That event provides libinput_event_pointer_axis_value_v120() which is > basically > a mirror of the kernel API. This event obsoletes POINTER_AXIS events for > sources WHEEL and WHEEL_TILT, so callers should just ignore those if they > support the new event. A branch for this is available here: > https://gitlab.freedesktop.org/whot/libinput/commits/wip/hi-res-scrolling > > The event sequence would now be: > > - WHEEL: value 3.75, v120 30 > - WHEEL: value 3.75, v120 30 > - WHEEL: value 3.75, v120 30 > - WHEEL: value 3.75, v120 30 > - AXIS: value 3.75, discrete 1, v120 120 > > Implementation-wise: AXIS and WHEEL are independent, so there's no guarantee > that they add up to the same 120 values. This new event is easy to add to > compositors. This makes sense to me. It's unfortunate to introduce a new API, but I guess as discussed before there's no way around it. > Let's look at the Wayland protocol with the current API given hi-res scroll > events: > - value 3.75, no discrete event > - value 3.75, no discrete event > - value 3.75, no discrete event > - value 3.75, discrete 1 > > This is assuming 15 degrees/4 == 3.75 but if the compositor force wheels > to a scale of 10, the value would be 2.5. > > Qt: > - doesn't use discrete so won't be affected by changes > > GTK: > - generates 4 GDK_SCROLL_SMOOTH events, > - generates 1 GDK_SCROLL_DOWN event ('emulated') > - GTK doesn't really care as such about whether the distance-to-wheel is 10 or > something else, it'll just have different scroll distances (e.g. under > wlroots). > > Afaict this works correctly (tested with stock F29 nautilus) > > Xwayland: > - current Xwayland: values are handled as fraction as before, so we get 3 > smooth-scroll events at 25%. Event 4 has a discrete event which is handled > as full event, so we scroll by a logical click. Total distance is 175%. Hmm. Is a logical click 150%? I think I'm missing something here. But yeah, if we stop sending axis_discrete and continue sending axis, it'll break clients. > This means: Xwayland cannot work at all without any extra wayland protocol > unless it assumes factor 10 is the base unit. > > > So we have at least one ubiquitous Wayland client that cannot handle it. > And the issues are very similar to the ones libinput's API has, added to that > is that you cannot guarantee that two events come from the same device. > What we need is a new event, but I'm struggling to add this to the wayland > protocol in a good way, mostly for backwards-compatibility reasons. > > > Much of the below would be easier to solve if we can say "if you claim to > support wl_pointer version 8, you get different behaviour for axis events". I > don't think we can trust clients to handle this correctly, I got burned by > those assumptions before (in XI2). So I'm ruling out changing any of the > existing protocol behaviour. Hmm. We have to be careful -- anything we decide will be set in stone. The protocol would look less ugly if we change wl_pointer's meaning in a new version. (A similar change has already been done to the wl_data_offer API) Though it's true that until we still have users of the old version we'll need to be careful. > My current idea is to add a wl_pointer.axis_v120 event: > > wl_pointer.axis_source wheel > wl_pointer.axis_v120 30 > wl_pointer.frame > wl_pointer.axis_source wheel > wl_pointer.axis_v120 30 > wl_pointer.frame > wl_pointer.axis_source wheel > wl_pointer.axis_v120 30 > wl_pointer.frame > wl_pointer.axis_source wheel > wl_pointer.axis 10 > wl_pointer.axis_discrete 1 > wl_pointer.axis_v120 30 > wl_pointer.frame I can think of a few issues with this: * Can the compositor still have a scroll multiplier setting? Is a 120 granularity enough? (ie. what happens if the user has a very precise mouse and configures a scale multiplier of 0.5?) Maybe this value could be a wl_fixed and named e.g. axis_fraction? * What do we do about wl_pointer.axis? Maybe we can just phase it out? I'm not a fan of having to support two APIs forever. Plus the current axis event is ill-defined. * Can't we just define a axis_full_turn event sent after binding to wl_pointer, which defines what is the amount of axis you need to wait for until you reach a full turn? (The axis event already carries a wl_fixed value) > Main problems here: this puts a fair bit of implementation requirements on the > compositor - skip source/v120/frame for clients < supported version. Well this is inevitable anyway -- having a new protocol version will require compositors to add more version checks. > wl_pointer.source cannot currently occur on its own, it requires a > wl_pointer.axis event in the current protocol. Same with axis_discrete. Empty > frames are allowed, I think, but pointless. > > It also puts a bit of extra logic in the client, you need to remember if > you've > seen a v120 in this frame and discard the axis/axis_discrete events. If not, > then axis events must be processed as normal. > > A possible workaround would be to add v120 to all axis frames (including > touchpad scrolling) but that would then put the compositor in charge of > deciding what distance is a wheel click and encodes that information in the > protocol. Side note, the current protocol already requires compositors to decide of an axis event value for touchpads. > The v120 event must also carry the equivalent to the wl_pointer.axis value so > that can be passed on into existing code. So the above would have a v120 event > with v120 value of 30 and an axis value of 2.5 (if enforcing 10 units per > wheel click). > > Because the hi-res and lo-res sources are independent, we need to allow for a > 0-value v120 event, which basically just tags that frame as as low-res? > The compositor also need to *always* send v120 events for wheel/tilt sources > because otherwise the client code becomes even more of a nightmare. > Where not backed by libinput the compositor needs to handle that itself. Agreed. > Anyway, the below is the best I've come up with so far though I haven't > implemented it client-side yet. I'd really appreciate any feedback and/or > epiphanies. > > Cheers, > Peter > > diff --git a/protocol/wayland.xml b/protocol/wayland.xml > index b1c930d..cd45407 100644 > --- a/protocol/wayland.xml > +++ b/protocol/wayland.xml > @@ -1662,7 +1662,7 @@ > </request> > </interface> > > - <interface name="wl_seat" version="7"> > + <interface name="wl_seat" version="8"> > <description summary="group of input devices"> > A seat is a group of keyboards, pointer and touch devices. This > object is published as a global during start up, or when such a > @@ -1771,7 +1771,7 @@ > > </interface> > > - <interface name="wl_pointer" version="7"> > + <interface name="wl_pointer" version="8"> > <description summary="pointer input device"> > The wl_pointer interface represents one or more input devices, > such as mice, which control the pointer location and pointer_focus > @@ -2092,9 +2092,97 @@ > <arg name="axis" type="uint" enum="axis" summary="axis type"/> > <arg name="discrete" type="int" summary="number of steps"/> > </event> Nit: we generally add a <!-- Version 8 additions --> comment here. > + <event name="axis_v120" since="8"> > + <description summary="axis high-resolution scroll event"> > + Discrete high-resolution scroll information. > + > + This event carries high-resolution wheel scroll information, > + with each multiple of 120 representing one logical scroll step. > + For example, a v120 of 30 is one-quarter of a logical > + scroll step in the positive direction, the v120 of -60 is > + one-half of a logical scroll step in the negative direction. > + Clients that rely on discrete scrolling should accumulate the > + v120's to multiples of 120 before processing the event. > + > + This event also carries the axis motion value in the same coordinate > + space as the wl_pointer.axis event's value. See the > + wl_pointer.axis documentation for details. Why do we need to duplicate this information? > + This event is guaranteed for axis movement from axis sources wheel > + and wheel_tilt even if the underlying device does not support > + high-resolution scrolling. > + > + Where a wl_pointer.axis_source event occurs in the same > + wl_pointer.frame, the axis source applies to this event. > + > + Where wl_pointer.axis and/or wl_pointer.axis_discrete events > + occur in the same wl_pointer.frame, those events reflect an > + accumulated scroll amount. Clients handling axis_v120 events can > + usually ignore the axis and axis_discrete events. > + > + For example, a wheel generating events every 25% of each physical > + wheel detent may create the following event sequence: > + > + wl_pointer.axis_source wheel > + wl_pointer.axis_v120 30 > + wl_pointer.frame > + wl_pointer.axis_source wheel > + wl_pointer.axis_v120 30 > + wl_pointer.frame > + wl_pointer.axis_source wheel > + wl_pointer.axis_v120 30 > + wl_pointer.frame > + wl_pointer.axis_source wheel > + wl_pointer.axis_v120 30 > + wl_pointer.axis 10 > + wl_pointer.axis_discrete 1 > + wl_pointer.frame > + > + For backwards compatibility, the wl_pointer.axis event for a wheel > + or wheel tilt must contain the value of the full wheel movement. It > + must not be sent for a fraction of a physical detent. > + > + For backwards compatibility, the wl_pointer.discrete event for a wheel > + must contain the value of the full wheel movement. It must not be > + sent for a fraction of a physical detent and it must not be 0. > + > + For backwards compatibility, the wl_pointer.axis_source event in a > + frame containing only a wl_pointer.axis_v120 event must not be sent > + to clients that do not support wl_pointer version 8 or later. In > + other words, the above sequence to such clients is reduced to: > + wl_pointer.axis_source wheel > + wl_pointer.axis 10 > + wl_pointer.axis_discrete 1 > + wl_pointer.frame > + > + Clients must treat wl_pointer.axis_v120 and > + wl_pointer.axis/axis_discrete as two independent event sources. > + There is no guarantee that the steps indicated by the v120 events > + match the steps by wl_pointer.axis_discrete events. In other words, > + an accumulated v120 value of 360 does not guarantee an accumulated > + axis_discrete value of 3. > + > + The value of a wl_pointer.axis_v120 event may be 0. > + This is the case where a low-resolution event has no equivalent > + high-resolution event. The value of 0 indicates that any > + wl_pointer.axis/axis_discrete events in the same event can be > + ignored. > + > + The axis motion's coordinate space is identical to the > + wl_pointer.axis event. > + > + The order of wl_pointer.axis_v120 and wl_pointer.axis_source is > + not guaranteed. > + </description> > + <arg name="time" type="uint" summary="timestamp with millisecond > granularity"/> > + <arg name="axis" type="uint" enum="axis" summary="axis type"/> > + <arg name="motion" type="fixed" summary="length of vector in > surface-local coordinate space"/> > + <arg name="v120" type="int" summary="scroll distance as fraction of > 120"/> > + </event> > </interface> > > - <interface name="wl_keyboard" version="7"> > + <interface name="wl_keyboard" version="8"> > <description summary="keyboard input device"> > The wl_keyboard interface represents one or more keyboards > associated with a seat. > @@ -2208,7 +2296,7 @@ > </event> > </interface> > > - <interface name="wl_touch" version="7"> > + <interface name="wl_touch" version="8"> > <description summary="touchscreen input device"> > The wl_touch interface represents a touchscreen > associated with a seat. > > _______________________________________________ > wayland-devel mailing list > [email protected] > https://lists.freedesktop.org/mailman/listinfo/wayland-devel _______________________________________________ wayland-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/wayland-devel
