We've been chatting some more around the Oslo office about the fact that even on platforms where the touchscreen is the primary pointing device, both QWidgets and Qt Quick components are handling mostly mouse events. In order to make that work, the Qt 4 approach (which is still in place now) is for each incoming touch event, first let it propagate as a touch event; then, if the event was not accepted, synthesize a mouse event and let that propagate through. This method of achieving backwards compatibility discourages handling touch events though: if any variety of button widget or QML component, which handles mouse events only, is a child (or descendant) of any component which handles touch events, then when the touch event is accepted by the parent component, the mouse event will not be synthesized. So the button (or other mouse-only component) cannot be pressed.
The WebKit team has this problem, in that they want the QML web view to be flickable, but obviously the user needs to be able to interact with any mouse-only components (such as buttons) which might be on top. So WebKit needs to separately synthesize mouse events from touch events because the main synthesis mechanism doesn't work in that case. In src/quick/items/qquickcanvas.cpp, there is a very recent new method translateTouchToMouse which generates a new QQuickMouseEventEx event type containing velocity information. (In fact maybe it makes sense to just put the velocity in the base QMouseEvent, but that's optional to what I'm about to propose.) This is IMO another case where mouse event synthesis should not be necessary. I suspect the reason for it is the same as the WebKit case. Graphics View has yet another way, but handling touch events there is lower-priority than for QML. If we set aside all the Qt history and think about what would have been ideal if we were starting over from scratch, I think I'd have wanted a "pointing" event type which has a bit less than what QMouseEvent does: at least the coordinates and an enum for "what happened" (pressed, released, clicked, double-clicked, entering, leaving and so on). The mouse event could inherit that and add mouse-specific concepts like having multiple buttons, and the touch event could inherit and add the multiple-finger concept. The point being that naive widgets like Button should not need to care where the event came from, just that it was clicked or tapped, but not dragged; so QPushButton would just handle the hypothetical Pointing event. Then most of the third-party legacy apps would have already been doing the same thing, and we wouldn't have trouble at this stage to introduce a touch event as a different specialization of the Pointing event. (Alternatively multiple fingers could be treated just like multiple mice: separate press/release events for the independent fingers. But actually it's useful to group multiple touch points together as long as they come from the same user; it facilitates gestural UIs, in that the UI does not need to gather up multiple points from multiple events occurring at different times, and figure out that they are part of one gesture.) Anyway, back to reality... my next idea was let's have a flag to enable the mouse event synthesis, which should be true by default, so that we can at least turn it off and try to develop pure-touch UIs. But it turns out this flag already exists: AA_SynthesizeMouseForUnhandledTouchEvents, which is true by default. And there is even the reverse one: AA_SynthesizeTouchForUnhandledMouseEvents. So that's a good start. The proposal is this: I think we need to have a QML PointingArea element which looks just like MouseArea except that it handles both mouse events and single-touch events the same way. Then we need to start using it instead of MouseArea in almost every case. That way AA_SynthesizeMouseForUnhandledTouchEvents can eventually be set to false for some applications. We also need the exsisting QWidgets to start handling touch events too. After that is done, individual QWidget-based apps in the field (especially those with custom widgets) can set the AA_SynthesizeMouseForUnhandledTouchEvents flag or not, depending on what works better for them; but we need to move towards a future in which we do not need to synthesize mouse events. Some apps may eventually have a use for the distinction between mouse and touch, too. One reason I got interested again at this particular time is that I was thinking it would be nice if the KDE Konsole (terminal application) was flickable. But dragging with the left mouse button is the way that you select text, and that is also useful. So on my touchscreen laptop, I can select text by dragging, regardless whether I drag with a finger on the screen, with the touchpad, or with an external mouse; but with a finger on the screen, selecting text is not really what I expect. If Konsole handled both mouse events and touch events, it could do something appropriate for each of them, and there could be some useful multi-finger gestures too. This is just an example, and in fact it may already be possible to implement this use case without changes to Qt (except for the lack of support for XInput 2.2 and/or direct support for the evdev driver on my laptop, which I'm also interested in looking at separately.) But in a more complex case which has more widgets or Qt Quick elements, if touch events are being accepted at all, you need to have them understood everywhere. But we have the issue that the QML MouseArea component is in very widespread use. This is because QML started life on the desktop. There is already QQuickMultiPointTouchArea and QQuickPinchArea; so in applications which intend to be portable between touchscreen devices and conventional desktop usage, it should be OK to have overlapping touch areas and MouseAreas, and this will enable the app developer to customize the behavior depending on whether the user is interacting with a mouse or a finger. But naive components like any of the many Button implementations should not necessarily need to care. They should be using the proposed PointingArea instead of MouseArea. Alternatively MouseArea could handle touch events itself, but then pretty soon we will start thinking the name sounds rather dated. It wouldn't even surprise me if people stop using mice in a decade or so; whereas we will probably always have some kind of "pointing device", thus the need for a generic name which won't be obsolete later. And, we still need a mouse-only Area which can be used in combination with the touch-only Areas so that it's possible to make a cross-platform UI. In summary the consequences of mouse event synthesis present some real problems, and I think we need to get the device-agnostic PointingArea into Qt5 ASAP. -- MVH, Shawn Rutledge ❖ "ecloud" on IRC _______________________________________________ Development mailing list [email protected] http://lists.qt-project.org/mailman/listinfo/development
