Hi Anton, Thanks for this information and please go ahead with your fix.

 

As earlier Alexey pointed out in his email about the 
"WM_LBUTTONDOWN/WM_LBUTTONUP"  messages mess up being a possibility and I was 
debugging on that and did found that it is an issue with these 
"WM_LBUTTONDOWN/WM_LBUTTONUP"  messages but could not find out the root cause 
of it though, hence I was unable to reply to his earlier email comments.

 

Thanks and regards,

Shashi

 

From: Anton Litvinov 
Sent: Tuesday, October 3, 2017 7:38 PM
To: Shashidhara Veerabhadraiah <shashidhara.veerabhadra...@oracle.com>; Alexey 
Ivanov <alexey.iva...@oracle.com>; Sergey Bylokhov <sergey.bylok...@oracle.com>
Cc: awt-dev@openjdk.java.net
Subject: Re: <AWT Dev> [10] JDK-8178361: JFileChooser does not allow to open 
folders with a double tap when using the touch screen interface

 

Hello Shashi, Alexey and Sergey,

Shashi, I know that "WM_GESTURE" is mutually exclusive with "WM_TOUCH" which is 
used by my fix for JDK-8166772, according to the next quotation from the MSDN 
page 
(https://msdn.microsoft.com/en-us/library/windows/desktop/dd371581(v=vs.85).aspx).
"Note  By default, you receive WM_GESTURE messages instead of WM_TOUCH 
messages. If you call RegisterTouchWindow, you will stop receiving WM_GESTURE 
messages."

Sergey, Alexey and Shashi,

I debugged this issue (JDK-8178361), identified the root cause of this bug and 
created my own 2 line length alternative version of the fix for this bug. This 
bug is a bug in JDK and is not caused by lack of support of the touch input by 
JDK on Windows OS. My fix does not involve any touch input API. Please review 
my fix and find the explanation of the root cause of the bug.

Webrev: http://cr.openjdk.java.net/~alitvinov/8178361/jdk10/webrev.00

THE ROOT CAUSE OF THE BUG:
The event "java.awt.event.MouseEvent.MOUSE_CLICKED" is not sent from the native 
method "AwtComponent::WmMouseUp" to the event queue for "WM_LBUTTONUP" message 
related to the second touch (tap) of the double touch user action. Tracing 
shows that in reality:

1.  During a double click and a double touch "AwtComponent" window always 
receives 2 pairs of "WM_LBUTTONDOWN/WM_LBUTTONUP" messages, thus JDK already 
has support of the double touch action via "WM_LBUTTONDOWN/WM_LBUTTONUP" 
messages.
2.1.  For each "WM_LBUTTONDOWN" in "AwtComponent::WmMouseDown" JDK sends 1 
"java.awt.event.MouseEvent" (MOUSE_PRESSED).
2.2.  For each "WM_LBUTTONUP" in "AwtComponent::WmMouseUp" JDK sends 2 
"java.awt.event.MouseEvent" (MOUSE_RELEASED, MOUSE_CLICKED).

However, when the double touch is done "MOUSE_CLICKED" event is not sent for 
"WM_LBUTTONUP" from the 2nd pair of "WM_LBUTTONDOWN/WM_LBUTTONUP" messages, 
because "m_mouseButtonClickAllowed" equals "0" and the next condition in 
"AwtComponent::WmMouseUp" is not satisfied:

    if ((m_mouseButtonClickAllowed & GetButtonMK(button)) != 0) { //CLICK 
allowed
        SendMouseEvent(java_awt_event_MouseEvent_MOUSE_CLICKED,
                       ::JVM_CurrentTimeMillis(NULL, 0), x, y, 
GetJavaModifiers(),
                       clickCount, JNI_FALSE, GetButton(button));
    }

It occurs, because between the 2nd pair of the messages 
"WM_LBUTTONDOWN/WM_LBUTTONUP" the messages "WM_MOUSEMOVE" arrive, so "x", "y" 
coordinates of "WM_LBUTTONDOWN", "WM_LBUTTONUP" are different. JDK handles 
"WM_MOUSEMOVE", and in "AwtComponent::WmMouseMove" this event is treated as 
"dragging" and the next expression is executed.

    "m_mouseButtonClickAllowed = 0;"

Therefore in "AwtComponent::WmMouseUp" it is decided not to send 
"java.awt.event.MouseEvent.MOUSE_CLICKED" for the end of dragging.

THE FIX:
The fix allows to send "java.awt.event.MouseEvent.MOUSE_CLICKED" event from 
"AwtComponent::WmMouseUp" for the case, when "clickCount" exceeds "1". If JDK 
already associates 2 "WM_LBUTTONDOWN" messages in "AwtComponent::WmMouseDown" 
as a double or triple click, then it is possible to consider that no dragging 
happened between these messages.

Thank you,
Anton

On 28/09/2017 03:11, Shashidhara Veerabhadraiah wrote:

Hi Anton, Thank you for your review comments. Please see below for my replies.

 

Thanks and regards,

Shashi

 

From: Anton Litvinov 
Sent: Tuesday, September 26, 2017 6:55 AM
To: Shashidhara Veerabhadraiah HYPERLINK 
"mailto:shashidhara.veerabhadra...@oracle.com";<shashidhara.veerabhadra...@oracle.com>;
 Alexey Ivanov HYPERLINK 
"mailto:alexey.iva...@oracle.com";<alexey.iva...@oracle.com>; Sergey Bylokhov 
HYPERLINK "mailto:sergey.bylok...@oracle.com";<sergey.bylok...@oracle.com>
Cc: HYPERLINK "mailto:awt-dev@openjdk.java.net"awt-dev@openjdk.java.net
Subject: Re: <AWT Dev> [10] JDK-8178361: JFileChooser does not allow to open 
folders with a double tap when using the touch screen interface

 

Hi Shashi,

I am also reviewing your fix since last week. I built "jdk10/client" with your 
fix and was able to see, how the 1st and the 2nd versions of your fix work. I 
have the following remarks and questions to you:

1. The fix causes a rather significant regression, which is a loss of the drag 
functionality using the touch screen. To reproduce it you can use "SwingSet2" 
demo application, where try to move any of the internal frames on the first tab 
of the demo application using the touch screen. Also you can try to select text 
in JTextField in that demo application using the touch screen. I think that 
this regression is serious and should be addressed in this fix.

[Shashi] I found the problem that was causing this issue. It is to do with the 
drag capture/release implemented in the WmMouseDown() and WmMouseUp() 
functions. This does not handles the touch drag as such and we need to 
implement a new touch gesture to enable the same functionality.

2. Why is it necessary for your fix to handle "WM_POINTERENTER", 
"WM_POINTERLEAVE" messages?

[Shashi] Currently we are mapping the touch input to mouse input and the mouse 
input follows a protocol of sending a mouse enter to and exit from a window 
event before processing further events. This same is implemented thro’ Pointer 
enter and pointer leave messages.

3. According to MSDN the used by you Win32 API function "GetPointerInfo" and 
"WM_POINTER*" messages are available on Windows 8 or later OS versions, what 
means that they are not available on older OS versions, for example on Windows 
7, and what means that your fix will not resolve the bug for Windows 7.

Windows 7 is supported by JDK 9 
(http://www.oracle.com/technetwork/java/javase/jdk9certconfig-3761018.html), 
and obviously JDK 10 also supports Windows 7, therefore the bug should be fixed 
also for Windows 7, if it is reproducible on that OS.

[Shashi] Thanks for pointing this out. Windows 7 offers a different touch input 
method via the WM_GESTURE and here is the link for it: HYPERLINK 
"https://msdn.microsoft.com/en-us/library/windows/desktop/dd940543%28v=vs.85%29.aspx"https://msdn.microsoft.com/en-us/library/windows/desktop/dd940543(v=vs.85).aspx.
 I am exploring a way to implement the same in the coming update.

As one can see, there are a lot of major updates that needs to be done for this 
feature to be complete. It will take some more days to do this, hence please 
wait for some time till I produce a new update.

Thank you,
Anton

On 25/09/2017 07:29, Shashidhara Veerabhadraiah wrote:

Hi Alexey, Thank you for pointing that out. I misread the info that 
EnableMouseInPointer() is a must call and would enable the WM_POINTER messages 
being sent to the component. But actually it is not required to be enabled and 
as you rightly pointed out that it would convert the WM_MOUSE to the WM_POINTER 
messages.

 

Below is the Webrev that does this change. Now since mouse in pointer is 
disabled, the mouse behavior should not have any effect at all by this software 
updates.

 

Webrev: HYPERLINK 
"http://cr.openjdk.java.net/%7Esveerabhadra/8178361/webrev.01/"http://cr.openjdk.java.net/~sveerabhadra/8178361/webrev.01/

 

For additional replies, please see below.

 

Thanks and regards,

Shashi

 

From: Alexey Ivanov 
Sent: Friday, September 22, 2017 5:19 PM
To: Shashidhara Veerabhadraiah HYPERLINK 
"mailto:shashidhara.veerabhadra...@oracle.com";<shashidhara.veerabhadra...@oracle.com>;
 Sergey Bylokhov HYPERLINK 
"mailto:sergey.bylok...@oracle.com";<sergey.bylok...@oracle.com>
Cc: HYPERLINK "mailto:awt-dev@openjdk.java.net"awt-dev@openjdk.java.net
Subject: Re: <AWT Dev> [10] JDK-8178361: JFileChooser does not allow to open 
folders with a double tap when using the touch screen interface

 

Hi Shashi,

You use EnableMouseInPointer which converts classical WM_MOUSE events to 
WM_POINTER events. This could have substantial effect on AWT components 
behaviour. I am worried about High DPI support, drag-and-drop functionality etc.

[Shashi] This is not required and hence removed.

Is there a simpler way to handle double-tap?
Windows converts taps to clicks, i.e. the app receives WM_LBUTTONDOWN and 
WM_LBUTTONUP.
Have you tried to find out why double-tap is not converted to double-click?

[Shashi] Since the mouse in pointer is disabled and a windows icon behavior is 
different from a windows button behavior(Because the a touch on the icon does 
not translate to a left button down event whereas it does for a windows 
button(file name!!)), it is required to bring in the WM_POINTER message type 
into the component message handling routine.

Do right and middle mouse buttons continue to work as expected? Mouse wheel? 
Any additional mouse buttons?

[Shashi] It should not have any effect after mouse in pointer is disabled.

The documentation for EnableMouseInPointer [1] says: “This function can be 
called only once in the context of a process lifetime.” You call it for every 
component created. It would have been enough to call it once during Toolkit 
initialization.


Regards,
Alexey

[1] HYPERLINK 
"https://msdn.microsoft.com/en-us/library/windows/desktop/hh447467%28v=vs.85%29.aspx"https://msdn.microsoft.com/en-us/library/windows/desktop/hh447467(v=vs.85).aspx

On 19/09/2017 04:35, Shashidhara Veerabhadraiah wrote:

Hi Sergey, When I checked it was not working on the Java applications. None of 
the controls could take touch click events but it used to highlight because 
when we touch on the touch screen interface the system moves the cursor over to 
the touch point thereby enabling mouse focus events being sent.
 
Thanks and regards,
Shashi
 
-----Original Message-----
From: Sergey Bylokhov 
Sent: Tuesday, September 19, 2017 3:39 AM
To: HYPERLINK 
"mailto:shashidhara.veerabhadra...@oracle.com"shashidhara.veerabhadra...@oracle.com
 >> Shashidhara Veerabhadraiah HYPERLINK 
"mailto:shashidhara.veerabhadra...@oracle.com";<shashidhara.veerabhadra...@oracle.com>
Cc: HYPERLINK "mailto:awt-dev@openjdk.java.net"awt-dev@openjdk.java.net
Subject: Re: <AWT Dev> [10] JDK-8178361: JFileChooser does not allow to open 
folders with a double tap when using the touch screen interface
 
Hi, Shashi.
Why the bug is JFileChooser specific?
Does it means that it works for other elements(buttons/lists/menu/etc)?
 
On 9/18/17 08:58, Shashidhara Veerabhadraiah wrote:

Hi All, Please review this software changes for the /_enhancement_/ 
JDK-8178361.
 
Issue: Request was filed to enable the touch event processing for Java 
client applications.
 
Fix: Windows platform offers 'pointer' implementation thro' which one 
can tap for the touch inputs as well. A typical 'pointer' function may 
contain touch, pen, touch pad or mouse inputs. This allows for a 
uniform input processing though the event source may defer. This fix 
enables the 'touch' events(via TOUCH pointer) to be passed to the 
components using the other type of pointer 'mouse'. Essentially there 
is a conversion put in place to convert a touch event input into a 
left click mouse event. I think this is the right thing to do 
considering desktop scenarios to which the typical Java applications 
that gets exposed to. This is also the same behavior on my touch 
enabled windows 10 laptop as well. Please let me know if a different behavior 
is expected. Below is the output:
 
Now touch event works as a typical mouse left click as in general. But 
there is an issue because the icon size of the Java UI is typically 
small, touching that icon exactly is sometimes difficult. This is 
handled by an Windows application by having larger icon size or icon 
view as a standard. Hence I believe this is outside the scope this 
software update. Any changes to it would be done later as a new bug or 
an enhancement.
 
Bug ID: https://bugs.openjdk.java.net/browse/JDK-8178361
 
Webrev: HYPERLINK 
"http://cr.openjdk.java.net/%7Esveerabhadra/8178361/webrev.00/"http://cr.openjdk.java.net/~sveerabhadra/8178361/webrev.00/
 
Thanks and regards,
 
Shashi
 

 
--
Best regards, Sergey.

 

 

 

Reply via email to