Clipboard API: Proposal for asynchronous API

[Кошмарчик]

I just sent a proposal to the WICG for an asynchronous Clipboard API:
https://discourse.wicg.io/t/proposal-modern-asynchronous-clipboard-api/1513

(yes, I know. Not the first time something like this has been proposed
)

This is a follow-up to an earlier thread
Clipboard API: remove dangerous formats from mandatory data types

where Chrome is unable to safely add clipboard support for a number of
image formats.

An async API would allow these image formats to be supported (because the
sanitization could happen without blocking the main thread).

Re: [clipboard] kill onbefore* events?

[Кошмарчик]

On Wed, Mar 9, 2016 at 3:40 PM, Hallvord Reiar Michaelsen Steen <
hst...@mozilla.com> wrote:

>
> On Wed, Mar 9, 2016 at 11:19 PM, Gary Kacmarcik (Кошмарчик)
>  wrote:
>
> > There is a design tension between the 2 extreme approaches, each with
> pros
> > and cons:
> >
> > (1) A single 'beforeinput' event that fires for all "input" activities.
> > Pros: Developers only need to support this and don't worry about the
> > details. Basic support for new input types just work without additional
> > work.
>
> I don't see how this logic holds if you add something like beforeInput
> for paste. Suppose that I already wrote a beforeInput - based editor.
> Suppose it's pretty good at handling the events I already know - I've
> carefully implemented typing, deletion etc based on beforeInput
> events. Now all of a sudden a modern browser is sending me beforeInput
> with eventType set to "paste" and with an event.clipboardData property
> I'd have to pull data from to insert into the DOM to make the paste
> work. How is that something that will "just work" in my app? It
> clearly won't.


Unless the `beforeinput` event is canceled, the DOM will be updated without
any special handling. Most webapps don't need to do anything to support
typing, deletion, et al, and they wouldn't need to do anything to support
paste. A rich editing app may need/choose to do extra work, but pasting
into a regular  should work without any extra effort.

Consider `beforeinput` (as it replaces `keypress`):
* `beforeinput` event is sent with key info. If I ignore it, the DOM will
be updated with the key value, if I cancel it, it doesn't. If the
programmer does nothing, key presses just work.

Now a paste action happens:
* `beforeinput` event is sent with the paste data (just in case the app
wants to do something with it). If ignored, the DOM will be updated, If
canceled, the DOM won't be updated. If the programmer does nothing, pasting
just works.

Now a brand-new-event-that-updates-the-DOM is sent:
* Same thing. `beforeinput` event is sent. DOM is updated (or not, if
canceled). It just works.

In fact, that's pretty much how the Clipboard API spec says the 'paste'
event should work:

7.1.3 The paste action

If the cursor is in an editable element, the paste action will insert
> clipboard data in the most suitable format (if any) supported for the given
> context.


This means that it updates the DOM, which means that a `beforeinput` event
needs to fire.  And if we're firing a cancelable `beforeinput` event, then
there's no point also sending a `paste` event that does basically the
thing. This is the exact same situation that we had with `keypress` and it
was deprecated in favor of `beforeinput`.

Re: [clipboard] kill onbefore* events?

[Кошмарчик]

On Wed, Mar 9, 2016 at 5:29 AM, Hallvord Reiar Michaelsen Steen <
hst...@mozilla.com> wrote:

> On Mon, Mar 7, 2016 at 11:00 PM, Gary Kacmarcik (Кошмарчик)
>  wrote:
>
> > Agreed. That is the major benefit of 'beforeinput' and 'input' -- you
> will
> > receive a 'beforeinput' event just before any DOM update (with an option
> to
> > cancel it), and you'll also receive an 'input' event just after the DOM
> was
> > updated.  If you want to track all DOM updates, then you only need to
> listen
> > to these two events.
>
> In my personal (and humble) opinion it's not actually a benefit for
> developers to have only one event that will fire a lot and indicate
> lots of different things. I accept that library authors want a
> convenient "point of overview" regarding what happens in the DOM (but
> wasn't mutation observers supposed to do that job?). However, making
> one single event type carry lots of different information and adding
> stuff like clipboardData and/or dataTransfer (because drag-and-drop
> changes the DOM too, right?) to the InputEvent interface seems like we
> force authors to
>
> 1) Write code with lots of if(e.eventType === foo) and
> switch(e.eventType) branching, which is hard to read
> 2) Debug code that gets run a lot all the time - event loops that fire
> often are annoying when debugging
>

Yes, that's one of the trade-offs that needs to be considered when
designing these APIs.

There is a design tension between the 2 extreme approaches, each with pros
and cons:

(1) A single 'beforeinput' event that fires for all "input" activities.
Pros: Developers only need to support this and don't worry about the
details. Basic support for new input types just work without additional
work.
Cons: Developers that care only about a particular type need to filter out
the other event types.

(2) Separate events for each type of input activity.
Pros: Developers that care about specific event types can easily catch the
cases they care about.
Cons: Developers have to explicitly add support for each event type if they
want to catch all input.

(Hybrid approaches are also possible: add some more specific events + a
generic catch-all, but that means a lot more events are being sent.)

There are issues with either approach:
(1) Problem with single event: Anne mentioned XMLHttpRequest
readystatechange as an example of going too far down the path with a
generic "something-happened" event.
(2) Problem with separate events: Many developers didn't support
composition events because their native language didn't use them, so parts
of the web were broken for CJK users. We don't want similar things to
happen for voice or accessibility (or other) inputs. Note that this is part
of the reason why 'keypress' was deprecated.

Personally, I'm more concerned about developers forgetting to add (or not
knowing they need to add) support for uncommon things like IMEs, voice and
accessibility, than I am about a developer sometimes needing to write a bit
of extra code to filter out things they're not immediately interested in.
The latter is an inconvenience for the developer, but the former breaks
part of the web for some users.

Re: [clipboard] kill onbefore* events?

[Кошмарчик]

On Mon, Mar 7, 2016 at 12:49 PM, Johannes Wilm 
wrote:

>
> On Mon, Mar 7, 2016 at 3:52 PM, Hallvord Reiar Michaelsen Steen <
> hst...@mozilla.com> wrote:
>
>>
>> On Mon, Mar 7, 2016 at 2:50 PM, Johannes Wilm 
>> wrote:
>>
>>
> > At the beginning of this year, the browser makers except Mozilla had an
>> > informal meeting about editing, and from what I gather, they did not
>> know
>> > about this. So their proposals seems to include copy/paste in
>> beforeInput
>> > and they have proposed another attribute to hold rich text content to be
>> > used for dragging/pasting.
>>
>> Hm..? I don't know anything about this. No biggie, we'll figure this
>> out. What exactly do you mean by  "include copy/paste in beforeInput"
>> - that for example Ctrl-C according to their view should trigger not a
>> "copy" event but a "beforeInput" event?
>
>
> The way we currently have it is that any user initiated editing operation
> triggers a beforeInput event with an inputType attribute specifying exactly
> what type of change the user is asking for. For many operations this is is
> in practice another event in addition to an already existing event
> (keydown, etc.), but for actions there is no good event to listen to.
>
> An exception up to this point was pasting/cutting -- for those we thought
> we would only listen to the events you were working on in the clipboard API.
>
> Now what the browser makers seem to have counted on is that the
> beforeInput event is also triggered for cut and paste within both cE=true
> and the other types of contenteditable. It would be a beforeInput event
> with the inputType attribute set to "cut" or "paste".
>
> And it doesn't seem like such an awful idea to me. For someone creating an
> editor, it should in theory mean that they only have to listen to the
> beforeInput event and based on that do all the required actions.
>

Agreed. That is the major benefit of 'beforeinput' and 'input' -- you will
receive a 'beforeinput' event just before any DOM update (with an option to
cancel it), and you'll also receive an 'input' event just after the DOM was
updated.  If you want to track all DOM updates, then you only need to
listen to these two events.

Other events could also be sent before/after the DOM is updated, but it's
better to avoid sending events around needlessly. FWIW, this is why
'keypress' was deprecated -- there's no need for it since its functionality
is covered by 'beforeinput'.

I don't know the details and subtleties of the cut and paste events, but it
would be great if they could coordinate with (or be folded into)
'beforeinput'. Especially since a 'beforeinput' event should be sent before
the cut/paste DOM update occurs in any case. But the issues with 'copy' and
the formatting commands that Johannes mentions would need to be addressed.

Re: [clipboard] kill onbefore* events?

[Кошмарчик]

I'm not very familiar with onbefore{cut|paste|copy}, but they sound like
very specialized versions of the beforeinput event.

What do these events provide that you don't get from handling beforeinput?
(well, other than the upcoming context "I'm about to do a cut/paste/copy")

On Tue, Feb 2, 2016 at 2:00 PM, Hallvord Reiar Michaelsen Steen <
hst...@mozilla.com> wrote:

> Hi,
> there's some scepticism about implementing
> onbeforecut/onbeforepaste/onbeforecopy in Gecko [1], IE's implementation
> seems considerably more limited than I expected (maybe because of bugs?),
> and it doesn't really seem like an elegant solution to the use case it is
> meant to solve.
>
> Would anybody mind if we killed those events?
>
> [1] https://bugzilla.mozilla.org/show_bug.cgi?id=596764
>
> -Hallvord R
>

Re: time at TPAC other than Wednesday?

[Кошмарчик]

+public-webapps since these these notes are relevant for both groups

On Fri, Jan 8, 2016 at 5:03 PM, Johannes Wilm 
wrote:

> Just so we don't have too many different ideas about what is going on.
>
> This is the last I remember having heard from the taskforce. We were
> waiting on answers from the different browser vendors on the issues under
> "open issues" before we could move on finishing the most important parts of
> the spec, AFAIR. One of the big questions here was what kind of "dumb
> ranges" the browser vendors would be willing to support.
>

The good news is that the range issue has been resolved by removing them
from the event. See the notes below for more info.

But maybe you found answers to those issues at the Redmond meeting? If so,
> could you please send over the minutes?
>

Good timing. I was just finishing up my notes (see below) from the meeting.

-Gary


This is a summary of my notes from the beforeinput/editing summit hosted by
Microsoft on 7 Jan 2016.

These notes focus on the UIEvent/beforeinput discussions, but some of the
editing discussions are included here as well.


Spec location

==

UIEvent spec will contain core description of beforeinput

data, isComposing

the data property should contain the text/plain version

will define the event ordering

Editing spec will contain partial interface that adds editing-specific
stuff:

editType (renamed to inputType)

dataTransfer field (for handling rich data)


What about selection ranges for beforeinput?

==

We determined that this was not needed as an attribute on the InputEvent.

If you need the range, you call a getTargetRanges() method and get a static
snapshot of the current ranges

Usually there will be only 1 range, but an array is returned to handle
multirange.

This avoids many problems associated with including targetRanges on the
event, e,g:

Should the range be live or static?

Live ranges can be expensive to upkeep and they are often held onto longer
than intended

A lighter-weight StaticRange would need to be defined

How to handle multiple handlers for an event, when the first handler in
chain modifies DOM or selection.

What range should be sent to the 2nd handler?

We need to send same event to each handler in the chain

Therefore followup events will need to be sent an incorrect range.

Ugh


Order of compositionupdate and beforeinput events

==

Current UIEvent spec says compositionupdate fires before beforeinput

What is the point of beforeinput *after* compositionend?

it seems unnecessary

It makes more sense for the event order to be:

beforeinput

compositionupdate

input

Update UIEvent spec with the new event ordering


When should DOM be updated during composition?

==

Current browser behavior:

IE/edge fire compositionupdate after the DOM has been modified

Moz/Saf/Chrome fire it before the DOM has been modified.

We should define when the DOM is updated explicltly in the spec:

beforeinput

compositionupdate

-- update DOM

input


beforeinput/input during composition

==

WRT composition, beforeinput and input should not fire:

before compositionstart

after compositionend

beforeinput/input events should wrap all compositionupdates


Composition events

==

There must be at least one compositionupdate after compositionstart

There must be at least one compositionupdate before compositionend

The same compositionupdate can satisfy both requirements


keyboard events during composition

==

general thought was that it didn't make sense to suppress these

isComposing flag can be used to ignore if needed

currenly only FireFox suppresses keydown/keyup during composition

Mozilla was not present, so we need to follow-up


dead keys as composition events

==

current browser behavior:

on Mac, all browsers generate composition events for dead keys

on Win, no browsers generate composition events for dead keys

current spec says that dead keys should generate composition events

need to confirm that there are technical issues making this difficult to do
on Win




>
> ------ Forwarded message --
> From: Gary Kačmarčík (Кошмарчик) 
> Date: Wed, Nov 4, 2015 at 11:32 PM
> Subject: Re: time at TPAC other than Wednesday?
> To: Johannes Wilm 
> Cc: Travis Leithead , cha...@yandex-team.ru,
> Ryosuke Niwa , Koji Ishii , Xiaoqian
> Cindy Wu , Grisha Lyukshin , Ojan
> Vafai 
>
>
> These are my notes, with the final Summary at the top as a tl;dr. Sorry
> for any oddities in formatting.  Please respond with any
> corrections/clarifications.
>
> -Gary
>
>
>
> *Before input and Composition Events*
> 9:30 Wed
> Present: Ryosuke Niwa, Travis Leithead, Takayoshi Kochi, Johannes Wilm,
> Ojan Vafai, Koji Ishii, Gary Kacmarcik, Yoshifume Inoue, Yoichi Osato,
> Dominic Cooney
> Dialed in: Grisha Lyukshin
>

Re: PSA: publishing new WD of UI Events and D3E Keyboard specs April 28/30

[Кошмарчик]

On Fri, Apr 24, 2015 at 7:26 AM, Arthur Barstow 
wrote:

>
> A few notes about these specs:
>
> * The permissions of the specs' now obsolete Github repository will be set
> to read-only.
>

s/Github/Mercurial/ ?

Re: DOM L3 Events Input Events Work to the Editing Task Force

[Кошмарчик]

|input| and |beforeinput| were only brought into D3E because they were
1) Required since |keypress| was being deprecated (with
|input|/|beforeinput| intended as a replacement)
2) Not being worked on by anyone.

>From that POV, it sounds like a reasonable idea. Esp. since one of the
places (IIRC) where we thought it was supposed to be was in the Editing
spec.

However, the biggest issues we had with |beforeinput| and |input| was
determining the correct event order relative to composition and keyboard
events. So it seems like a cross-dependency for D3E to depend on the
Editing spec to get |input| and |beforeinput| and for the Editing spec to
need D3E's composition and keyboard events (to properly specify the event
ordering).

Thus, I wonder if a separate Input Event spec (that both D3E and Editing
would refer to) would make more sense.

As for "easier to finish DOM L3", that depends. D2E defines keypress, which
is being deprecated in favor or input/beforeinput. Thus, D3E will not be a
complete functional replacement for D2E without a proper spec for
input/beforeinput.

-Gary


On Mon, Dec 8, 2014 at 2:05 PM, Ben Peters  wrote:

>  Since the Editing Task Force is working on several input-related issues,
> we would like to take over the work on Input Events. I have created an
> unofficial document for that purpose at
> http://w3c.github.io/editing-explainer/input-events.html. It is based on
> the work in DOM L3. Can we move the DOM L3 Input Events design to be only
> what is already implemented in browsers (which should make it easier to
> finish DOM L3, right?), and work on it going forward in the Editing Task
> Force?
>
>
>
> Ben
>

Re: CfC: publish WG Note of UI Events; deadline November 14

[Кошмарчик]

On Fri, Nov 7, 2014 at 7:36 AM, Anne van Kesteren  wrote:

> On Fri, Nov 7, 2014 at 4:28 PM, Arthur Barstow 
> wrote:
> > If anyone has comments or concerns about this CfC, please reply by
> November
> > 14 at the latest.
>
> My concern is that we previously agreed that UI Events would be a much
> more suitable name for the contents of DOM Level 3 Events.


I agree. "UI Events" is a much more descriptive name for the content.

My primary concern is that we (specifically, "I") have been telling people
that UI Events is not the same as D3E. If we change this, then I'll have to
have those conversations all over again, but reversed. ^_^

But we
> would keep using DOM Level 3 Events because it would be done quickly
> and then we'd move on to UI Events. As we now know we did not finish
> DOM Level 3 Events quickly.


FWIW, we pushed to have it done quickly and it was delayed:
(1) once because the spec was a step backward from DOM2 in some regards and
that needed to be fixed,
(2) again because there was feedback that style and presentation should be
updated to match more recent specs.

#2 is when the WG effectively decided that cleaning up the presentation was
more important than releasing it quickly.

So I would like us to abandon that name
> and settle on UI Events.
>

SGTM.

With regards to the current contents of UI Events, I assume that publishing
a "gutted WD Note" is meant simply to establish a historical record of what
was worked on before the content is deleted?  When we were focusing on
completing the D3E spec quickly, this is where we sent items that we felt
should be part of D3E, but would take too much time to finalize. We'll want
to reconsider some of these items for inclusion back in D3E (er... I mean
UI Events).

-Gary

Re: Proposal for a DOM L3 Events Telecon

[Кошмарчик]

On Tue, May 7, 2013 at 1:39 AM, Masayuki Nakano wrote:

> Hello, sorry for the big delay due to my business trip and holiday week of
> Japan.
>
> I'm available on either schedule. When I join the telecon, how can I join
> it? Skype or something?
>

Excellent question - I'm wondering the same thing. Will this be using Zakim
(http://www.w3.org/2002/01/UsingZakim) and will we have an IRC channel as
well?

-Gary

Re: Proposal for a DOM L3 Events Telecon

[Кошмарчик]

On Mon, Apr 29, 2013 at 12:59 PM, Travis Leithead <
travis.leith...@microsoft.com> wrote:

>  I’d like to propose we start a call to begin to work toward resolving
> the final bugs in the spec and for other business related to getting DOM L3
> Events to CR. On the call we can workout what subsequent meetings we should
> also arrange.
>
> ** **
>
> Does next Tuesday (May 7th), at 11 am PST work your you?
>

Note that 11am PDT = 3am JST.  If Masayuki is interested in joining, we
should pick a late afternoon PDT time:
   4pm PDT (Tue) = 8am JST (Wed)
   5pm PDT (Tue) = 9am JST (Wed)

Re: publish FPWD of UI Events; deadline May 4

[Кошмарчик]

Google supports this.


On Mon, Apr 29, 2013 at 11:03 AM, Travis Leithead <
travis.leith...@microsoft.com> wrote:

> Microsoft supports this.
>
>
> -Original Message-
> From: Arthur Barstow [mailto:art.bars...@nokia.com]
> Sent: Saturday, April 27, 2013 7:31 AM
> To: public-webapps
> Subject: CfC: publish FPWD of UI Events; deadline May 4
>
> As discussed during WebApps' April 25 meeting, this is a Call for
> Consensus to publish a First Public Working Draft of the UI Events spec
> using the following ED as the basis:
>
>
>
> This CfC satisfies the group's requirement to "record the group's decision
> to request advancement".
>
> By publishing this FPWD, the group sends a signal to the community to
> begin reviewing the document. The FPWD reflects where the group is on this
> spec at the time of publication; it does _not_ necessarily mean there is
> consensus on the spec's contents.
>
> If you have any comments or concerns about this CfC, please reply to this
> e-mail by May 4 at the latest. Positive response is preferred and
> encouraged, and silence will be considered as agreement with the proposal.
>
> -Thanks, AB
>
>  Original Message 
> Subject:ACTION-682: Start a CfC for FPWD of UI Events (and make
> sure
> it has a Bugzilla component) (Web Applications Working Group)
> Date:   Thu, 25 Apr 2013 17:29:27 +
> From:   ext Web Applications Working Group Issue Tracker
> 
> Reply-To:   Web Applications Working Group 
> To: 
>
>
>
> ACTION-682: Start a CfC for FPWD of UI Events (and make sure it has a
> Bugzilla component) (Web Applications Working Group)
>
> http://www.w3.org/2008/webapps/track/actions/682
>
> On: Arthur Barstow
> Due: 2013-05-02
>
> If you do not want to be notified on new action items for this group,
> please update your settings at:
> http://www.w3.org/2008/webapps/track/users/7672#settings
>
>
>
>
>
>
>
>
>

Re: Proposal for a DOM L3 Events Telecon

[Кошмарчик]

On Mon, Apr 29, 2013 at 12:59 PM, Travis Leithead <
travis.leith...@microsoft.com> wrote:

>  I’d like to propose we start a call to begin to work toward resolving
> the final bugs in the spec and for other business related to getting DOM L3
> Events to CR. On the call we can workout what subsequent meetings we should
> also arrange.
>
> ** **
>
> Does next Tuesday (May 7th), at 11 am PST work your you?
>

sgtm

Re: Draft minutes: 26 April 2013 f2f meeting

[Кошмарчик]

 = comments directly from people on IRC
person: = comments transcribed from people live in the room or on the phone.


On Sat, Apr 27, 2013 at 8:04 PM, Rick Waldron wrote:

> This is hard to follow... Almost like two different discussions are
> happening at the same time. One discussion has "person: ..." the other
> " ..."
>
> Rick
>
>
> On Saturday, April 27, 2013, Arthur Barstow wrote:
>
>> The Draft minutes from WebApps' April 26 f2f meeting are at the following
>> and copied below:
>>
>>  
>> 
>> >
>>
>> If you have corrections, comments, etc., please reply by May 2.
>>
>> Thanks very much to Josh Soref for once again being an excellent Scribe!
>>
>> And a Huge thanks to Daniel Austin and PayPal for hosting the meeting!
>>
>> -AB
>>
>> W3C 
>>
>>
>>  - DRAFT -
>>
>>
>>  Web Applications WG F2F Meeting
>>
>>
>>26 Apr 2013
>>
>> Agenda 
>> 
>> >
>>
>> See also:IRC log 
>> 
>> >
>>
>>
>>Attendees
>>
>> Present
>>Art_Barstow, Charles_McCathieNevile, Josh_Soref, Robin_Berjon,
>>Yves_Lafon, Ted_Oconnor, Laszlo_Gombos, Tyler_Barton,
>>Adrian_Bateman, Glenn_Adams, Doug_Turner, Bryan_Sullivan, Bin_Hu,
>>Arnaud_Braud, Yosuke_Funahasi, Jae_Won_Chung, Hiroyuki_Aizu,
>>adrianba, Lyle_Troxell, eliot_graff, Yosuke_Funahashi, krisk,
>>Jungkee_Song, Eduardo_Fullea, Jonghong_Jeon, Mike_Smith,
>>Arun_Ranganathan, Wonsuk_Lee, MikeSmith!, hober
>> Regrets
>> Chair
>>Art, Charles
>> Scribe
>>Josh_Soref
>>
>>
>>Contents
>>
>>  * Topics 
>> 
>> >
>> 1. testing > webapps-minutes.html#item01
>> >
>> 2. Move to Github
>>
>> 
>> >
>> 3. Progress Events
>>
>> 
>> >
>> 4. XHR Status
>>
>> 
>> >
>> 5. Coordination (TC39)
>>
>> 
>> >
>>  * Summary of Action Items
>>
>> 
>> >
>>
>> --**--**
>> 
>>
>>  Scribe: Josh_Soref
>>
>>  ScribeNick: timeless
>>
>>  i
>>
>>  trackbot, start telcon
>>
>>  Meeting: Web Applications Working Group Teleconference
>>
>>  Date: 26 April 2013
>>
>>  Ta
>>
>>  Shenzhen
>>
>> *ACTION:*barstow announce the WG will meet during TPAC 2013 in
>> November [recorded inhttp://www.w3.org/2013/04/**26-webapps-minutes.html#
>> **action01 ]
>>
>>  Created ACTION-692 - Announce the WG will meet during TPAC
>> 2013 in November [on Arthur Barstow - due 2013-05-03].
>>
>>  ScribeNick: darobin
>>
>> chaals:that was painless
>> ... the chartering stuff
>> ... what do we need to add or remove
>> ... we have a spec called URL, we asked if people would work on it
>>
>> Inventory of Charter updates that need to be made or might be made <
>> http://www.w3.org/wiki/**Webapps/Charter
>> >
>>
>> chaals:Anne is not in WebApps, so can't edit
>> ... no one in WebApps is editing it
>> ... Anne is working on it outside W3C
>> ... should we keep it in our charter
>>
>>  Hi Jungkee
>>
>> Robin:if we're just republishing, why not automate it?
>>
>> chaals:uh, we actually want a responsible editor
>>
>>  I note that chaals said "Anne puts his spec in the public
>> domain, so you can just copy it"
>>
>> dougt:how about we could just resolve the dispute over the licensing
>> instead?
>>
>> chaals:we could, but that's outside the scope of this group
>> ... if the AC get a consensus, then the problem goes away
>> ... until that happens, we need to figure out what to do with that spec
>> ... if no one is committing to it, seems pointless to have it in the
>> charter
>>
>> robin:just pointing out that this is pretty fundamental
>>
>> bryan:is this really cut and paste?
>>
>> group:yeah
>>
>> bryan:so why not just do it?
>>
>> chaals:because no one volunteers
>>
>> dougt:the problem is the license, so we should transition to an open
>> license
>>
>> ArtB:do we want to start a CfC to dr

Re: CfC: publish WD of Clipboard API and events; deadline March 26

[Кошмарчик]

Formatting comment: There are hard-coded section numbers in the doc and
respec auto-numbers the first  tag inside  tags. The result of
this is that most of the sections in this spec have 2 (different!) section
numbers. It seems like the hard-coded section numbers should probably be
removed, and sub-s added in the few places where they are missing.


On Tue, Mar 19, 2013 at 3:12 PM, Arthur Barstow wrote:

> Hallvord has made a number of changes since the last publication of
> Clipboard API and events (last published in Feb 2012). As such, this is a
> Call for Consensus to publish a new WD of this spec using the ED as the
> basis:
>
>   
> 
> >
>
> Agreement to this proposal: a) indicates support for publishing a new WD;
> and b) does not necessarily indicate support of the _contents_ of the WD.
>
> If you have any comments or concerns about this proposal, please reply to
> this e-mail by March 26 at the latest. Positive response to this CfC is
> preferred and encouraged and silence will be assumed to mean agreement with
> the proposal.
>
> -Thanks, AB
>
>

Re: Re: Keyboard events for accessible RIAs and Games

[Кошмарчик]

I've updated the UIEvents document with an initial draft for
queryKeyCap/queryLocale

https://dvcs.w3.org/hg/d4e/raw-file/tip/source_respec.htm (Section 4.1)


On Mon, Feb 18, 2013 at 8:09 AM, Gary Kacmarcik (Кошмарчик) <
gary...@chromium.org> wrote:

> I'll be updating the document this week. I'll send an update to the list
> after that happens.
>
>
> On Sat, Feb 16, 2013 at 7:40 AM, Florian Bösch  wrote:
>
>> Any progress on the speccing of queryKeyCap?
>>
>>
>> On Fri, Feb 1, 2013 at 9:14 PM, Gary Kacmarcik (Кошмарчик) <
>> gary...@chromium.org> wrote:
>>
>>> On Fri, Feb 1, 2013 at 11:42 AM, Travis Leithead <
>>> travis.leith...@microsoft.com> wrote:
>>>
>>>>  I think we should give it another try by including it in our UI
>>>> Events spec. I like the idea of adding the static queryKeyCap(code) API to
>>>> Keyboard events. I wonder about the name though. A "key capability" doesn't
>>>> sound right. Are we querying for a key's locale name? e.g.,
>>>> queryKeyLocaleName(code)?
>>>>
>>>
>>>  SGTM. I'll add a section to the spec for this.
>>>
>>> The KeyCap name refers to the "cap" placed over the keyswitch of the
>>> physical keyboard.  It's not a great name since there's no guarantee that
>>> the physical keyboard matches the current locale (although they usually
>>> do). However, the other (more accurate) names that I was able to come up
>>> with at the time were all rather unwieldy.
>>>
>>> Taking your name as a base, I think we'd need something like
>>> queryLocaleChar(locale, code) or queryLocaleKey since we'd be returning the
>>> equivalent of the 'char' (or 'key') attribute.
>>>
>>> Thinking briefly about this now:
>>> * If we return 'char' equivalents, we won't return dead keys or other
>>> virtual keys.
>>> * If we return 'key' values, then we need to address how to handle
>>> non-printable keys like "Shift" and "Home" that people might expect to be
>>> translated.
>>> * I think we'll also want the ability to specify modifier keys to apply
>>> to the 'code' (to generate shifted or AltGr'ed versions).
>>>
>>> I'll formalize this a bit more and send something out for comments.
>>>
>>> -Gary
>>>
>>>
>>
>

Re: Re: Keyboard events for accessible RIAs and Games

[Кошмарчик]

I'll be updating the document this week. I'll send an update to the list
after that happens.


On Sat, Feb 16, 2013 at 7:40 AM, Florian Bösch  wrote:

> Any progress on the speccing of queryKeyCap?
>
>
> On Fri, Feb 1, 2013 at 9:14 PM, Gary Kacmarcik (Кошмарчик) <
> gary...@chromium.org> wrote:
>
>> On Fri, Feb 1, 2013 at 11:42 AM, Travis Leithead <
>> travis.leith...@microsoft.com> wrote:
>>
>>>  I think we should give it another try by including it in our UI Events
>>> spec. I like the idea of adding the static queryKeyCap(code) API to
>>> Keyboard events. I wonder about the name though. A "key capability" doesn't
>>> sound right. Are we querying for a key's locale name? e.g.,
>>> queryKeyLocaleName(code)?
>>>
>>
>>  SGTM. I'll add a section to the spec for this.
>>
>> The KeyCap name refers to the "cap" placed over the keyswitch of the
>> physical keyboard.  It's not a great name since there's no guarantee that
>> the physical keyboard matches the current locale (although they usually
>> do). However, the other (more accurate) names that I was able to come up
>> with at the time were all rather unwieldy.
>>
>> Taking your name as a base, I think we'd need something like
>> queryLocaleChar(locale, code) or queryLocaleKey since we'd be returning the
>> equivalent of the 'char' (or 'key') attribute.
>>
>> Thinking briefly about this now:
>> * If we return 'char' equivalents, we won't return dead keys or other
>> virtual keys.
>> * If we return 'key' values, then we need to address how to handle
>> non-printable keys like "Shift" and "Home" that people might expect to be
>> translated.
>> * I think we'll also want the ability to specify modifier keys to apply
>> to the 'code' (to generate shifted or AltGr'ed versions).
>>
>> I'll formalize this a bit more and send something out for comments.
>>
>> -Gary
>>
>>
>

Re: Re: Keyboard events for accessible RIAs and Games

[Кошмарчик]

On Fri, Feb 1, 2013 at 11:42 AM, Travis Leithead <
travis.leith...@microsoft.com> wrote:

>  I think we should give it another try by including it in our UI Events
> spec. I like the idea of adding the static queryKeyCap(code) API to
> Keyboard events. I wonder about the name though. A "key capability" doesn't
> sound right. Are we querying for a key's locale name? e.g.,
> queryKeyLocaleName(code)?
>

SGTM. I'll add a section to the spec for this.

The KeyCap name refers to the "cap" placed over the keyswitch of the
physical keyboard.  It's not a great name since there's no guarantee that
the physical keyboard matches the current locale (although they usually
do). However, the other (more accurate) names that I was able to come up
with at the time were all rather unwieldy.

Taking your name as a base, I think we'd need something like
queryLocaleChar(locale, code) or queryLocaleKey since we'd be returning the
equivalent of the 'char' (or 'key') attribute.

Thinking briefly about this now:
* If we return 'char' equivalents, we won't return dead keys or other
virtual keys.
* If we return 'key' values, then we need to address how to handle
non-printable keys like "Shift" and "Home" that people might expect to be
translated.
* I think we'll also want the ability to specify modifier keys to apply to
the 'code' (to generate shifted or AltGr'ed versions).

I'll formalize this a bit more and send something out for comments.

-Gary

Re: Keyboard events for accessible RIAs and Games

[Кошмарчик]

I was wondering that myself...

On Wed, Jan 30, 2013 at 12:58 PM, Florian Bösch  wrote:

> What/where would be a good place to put the API for say queryKeyCap(code) ?
>
>
> On Wed, Jan 30, 2013 at 9:38 PM, Florian Bösch  wrote:
>
>> On Wed, Jan 30, 2013 at 9:02 PM, Gary Kacmarcik (Кошмарчик) <
>> gary...@chromium.org> wrote:
>>
>>> Yes, trying to match the current (virtual) keycap was not a direct goal,
>>> but adding the 'code' attribute makes it possible (when combined with the
>>> current layout) to calculate a keycap value.
>>>
>> I think that would probably be the most preferable solution. The code
>> would be stored (in presets, preferences etc.) and the keycap (primary
>> unmodified key display for a printable key) would be queried for a code
>> whenever it's necessary to display to a user. Let's set aside that
>> consideration for a numeric keycode (it's a minor concern anyway).
>>
>
>

Re: Keyboard events for accessible RIAs and Games

[Кошмарчик]

On Wed, Jan 30, 2013 at 9:31 AM, Florian Bösch  wrote:

> To the first comment: The persistent description might not offer a user
> something he recognizes. This becomes relevant in the case I'm arguing for
> when a Web-Developer wants to construct a mapping dialog (example
> screenshot
> http://codeflow.org/entries/2013/jan/30/keyboard-events-in-javascript-are-broken/blender.png
> ).
>
> So for instance the (slightly contrieved) combination ctrl+shift+/+1 would
> read as: CtrlKey+ShiftKey+ForwardSlash+Digit1
>
> A possible translation to a users locale primary unmodified symbol for a
> printable key is still something that is desired (and is practised by any
> shortcut mapping dialog in existence).
>

Yes, trying to match the current (virtual) keycap was not a direct goal,
but adding the 'code' attribute makes it possible (when combined with the
current layout) to calculate a keycap value.

But I don't believe that this info belongs directly in the key event:
* We already have 2 (or 3) key attributes that encode the key information
and this would require adding one more. Adding yet another would make the
key events even more confusing to users.
* This functionality is useful outside of key events. For the key pref
dialog example that you gave, I assume that I'd have the 'code' (saved in a
pref file or something) and I'd combine it with the current layout to get a
value to display in the dialog. This would work even if the user changed
keyboard layouts.

So I definitely think we should provide the ability to get the keycap from
the key event, but I think a general API is preferable to encoding it in
the key event.


> The second comment: It is often desirable to have a numeric identifier (as
> well as a string) in certain combinations of resolving events to actions. A
> string necessitates usage of an object whereas a numeric identifier can be
> mapped trough an array. The latter is often preferred due to JS not having
> true hashes, and it might offer faster lookup (which while not particularly
> relevant in realtime, may become very relevant at event replays).
>

We initially proposed numeric identifiers (seeded with USB usage codes to
start), but got feedback that in the Web Platform constants should be
string rather than ints. Using strings also makes the code self-documenting
(compare "if (key.code == 49)..." with "if (key.code == 'Digit1')...").

Javascript performance is really a separate issue from this spec. Given the
tradeoff between code-readability and possible performance issues, I will
generally prefer code readability. In this case, even if I saw measured
performance issues with current JS implementation, I would still probably
opt for readability because I expect that JS perf (in all browsers) will
increase faster than this spec will be updated.

-Gary

Re: Keyboard events for accessible RIAs and Games

[Кошмарчик]

This is not the first time these issues have been raised. The following
threads can provide some context to the current state of the discussion:
http://lists.w3.org/Archives/Public/www-dom/2012JulSep/0103.html (obsolete
proposal)
http://lists.w3.org/Archives/Public/public-webapps/2012OctDec/0623.html

Please also take a look at the UIEvents (formerly "DOM Level 4 Events")
Editor's Draft at:
https://dvcs.w3.org/hg/d4e/raw-file/tip/source_respec.htm
I believe that it addresses all of your concerns but I'm interested in any
comments that you might have.

Thanks,
-Gary


On Wed, Jan 30, 2013 at 6:05 AM, Florian Bösch  wrote:

> I have written a blog post at length about this issue here:
> http://codeflow.org/entries/2013/jan/30/keyboard-events-in-javascript-are-broken/
>
> In short the problem is the following:
> - RIAs (shortcut systems) and Games (shortcut systems and action systems)
> need to be able to identify each key uniquely regardless of modifier keys
> pressed.
> - To make things mappable an unmodified primary symbol or description
> needs to be available to display to the user.
> - To make things speedy to handle there should also be an accompanying
> unmodified keyCode (numeric)
>
> The current event model (keyCode, charCode and which) do not satisfy that
> requirement:
> - The same keyCode appears on multiple keys
> - The same charCode appears (differently from keyCode) on multiple keys
> - charCode and keyCode change based on modifiers
> - keyCodes cannot be converted to a users locale
> - no non printable key descriptions are offered
> - "which" is a combination of all the problems above.
>
> It is recognized that this is not a good way to do things as for instance
> the MDN already lists "keyCode", "charCode" and "which" as deprecated.
>
> The DOM Event Level 3 does not solve all these problems:
> - The char member is still being delivered modified
> - The key member has to match the char member for printable keys
> - There is no numeric unmodified keyCode
>
> I would suggest:
> - Add a primary key symbol (printable, from the users locale) umodified
> and for non printable keys add the human readable description ('backspace',
> 'enter' etc.)
> - Add an additional unmodified key code member holding the numeric key
> code unmodified
>

DOM Level 3 Events - minor comments

[Кошмарчик]

This is a list of typos and other minor formatting problems that I
encountered while going through the current DOM Level 3 Event spec. A few
of the things listed here are actual errors, but the majority fall into two
categories:
* Initial capital letter missing at the start of a note or warning.
* Sentences glued together with a semi-colon instead of using periods and
starting a new sentence. I didn't call out all of these.

Many of these are perhaps a bit nit-picky, so please consider them
suggestions or feel free to ignore if you disagree.

-Gary


1.3 Stylistic Conventions
=

I'm not sure if it's worth listing them here or not, but the following
stylistic conventions are not mentioned in this section even through they
are used in the document:
* Light green for constants (not to be confused with the brighter green
highlighting used for character values.
* Yellow for attributes
* Blue for methods
* Pink for parameters
* Orange (with red text) for event types. Actually, these are somewhat hard
to read. Has this particular text formatting been verified to be readable
by people with color vision problems?

2. Glossary
===

In *character value*, "Note: in source code..." should begin with a capital

In *candidate event handlers*, "...released or un-frozen after this set of
of candidate event handlers..." has duplicate 'of'

In *event listener*, "Note: ...will be provided as the first paramter to
the user-defined function..." -> spelling of parameter

In *event order*, "Note: there can be..." needs initial cap

In *focus ring*, "A focus ring is a an ordered set of..." remove 'a'

In *Unicode character categories", The term "General Category" is
capitalized in official Unicode documentation. Suggest rewrite as "A subset
of the General Category values that are defined for each Unicode code
point. This subset contains all the Letter (Ll, Lm, Lo, Lt, Lu), Number
(Nd, Nl, No), Punctuation (Pc, Pd, Pe, Pf, Pi, Po, Ps) and Symbol (Sc, Sk,
Sm, So) category values."

3.1 Event dispatch and DOM event flow
=

Caption for Figure 1 should be centered and start with a capital letter.

"... The exception itself must not propogate outside the scope of the event
handler." Spelling of 'propagate'

In Example: "In the following code, the exception thrown from the call to
throw "Error" does not propogate into the parent scope, (which would
prevent the console.log statement from executing):" Comma before paren
looks awkward. Spelling of 'propagate'.

"This specification defines three event phases: capture phase; target
phase; and bubble phase." semi-colons should be commas

"The capture phase: the event object must propagate..." initial cap since
it starts a sentence. Same for 1, 2 and 3 in this list.

3.2 Default actions and cancelable events
=

In first Example, "...depends on what happens next--for example, if the
user's pointing..." double hyphen should be proper em-dash.

In second Example, "...event on  elements..." the
 tag and attributes should be in a monospace font

3.3 Synchronous and asynchronous events
===

"...ordered by sequence of temporal occurrence, with respect to other
events, to changes in the DOM..." comma after occurrence makes this harder
to read.

In first Example, 'keydown' event is listed, but there is no corresponding
'keyup'. Is this intentional?

3.5.1 Activation event synthesis


"...of the default actions of that activation trigger; the value of the
Event.target..." Period instead of semi-colon to ease readability.

3.6 Event exceptions


"Note: the exception EventException..." initial cap.

4. Basic Event Interfaces
=

"Figure 2: graphical representation of the DOM3 Events interface
inheritance" should have initial capital for 'Graphical'

4.1 Interface Event
===

For eventPhase, "The un-initialized value of this attribute must be 0."
could mention the defined constant NONE. Suggest: "The un-initialized value
of this attribute must be 0 (NONE)."

For stopImmediatePropagation, I think it would look better if the
"introduced in DOM Level 3" was enclosed in parentheses, as is done above
for "Interface Event (introduced in DOM Level 2)". But that change would
need to be made throughout this document for consistency.

4.3 Interface EventTarget
=

In the first Note, "...event type (see the List of DOM3 Event Types); for
example, a mouseover event type..." Semi-colon -> period. The sentence is
long enough already.

The second Note uses italics for the code in "...use of attributes, e.g.,
onclick="handleClick()" (see [HTML5]...". This isn't consistent with code
elsewhere, so I don't believe it was intentional.

addEventListener and removeEventListener should have a "modified in DOM
Level 3" note just like dispatchEvent.

dispatchEvent might benefit f

Re: Re: Event.key complaints?

[Кошмарчик]

On Fri, Nov 30, 2012 at 4:29 PM, Travis Leithead <
travis.leith...@microsoft.com> wrote:

>  Awesome stuff Gary.
>
> ** **
>
> (And I like that we won't need to change the behavior of key or char in
> your proposal—that part made me really nervous, since IE has shipped this
> stuff since 9, and I know our new Win8 app model is using it.)
>
> ** **
>
> I'm planning in the short term to start a new DOM4 Events spec, which will
> be the successor to DOM3 Events. I brought this up at TPAC and there were
> no objections. Gary, I'd love you collaboration on specifying your new
> "code" property in that spec.
>

Sounds good to me.  I still have some comments to make on the DOM3 Events
spec, but I'll still send them out knowing that some of them will need to
be punted to DOM4.

When were you thinking of kicking off the DOM4 Events process?

-Gary

Re: Re: Event.key complaints?

[Кошмарчик]

On Thu, Nov 8, 2012 at 6:37 AM, Hallvord Reiar Michaelsen Steen wrote:

>
>>> Hence, what I think would be most usable in the real world would be
>>> making event.key a mapping back to un-shifted character values of a
>>> normal QUERTY (en-US) layout. Authors are asking for stable reference
>>> values for identifying keys, and that's the most stable and widely
>>> known reference keyboard layout.

Gary responded:

>> The main problem with using unmodified 'en-US' values is that it doesn't
>> define values for keys that are not found on US keyboards. So, it's great
>> for US keys, but completely ignores the rest of the world.

Hallvord replied:

> Yep, and the solution to that is listing those keys and figuring out what
> their preferred key name should be. [...]
>
> Thanks for coming up with your list :-)

That wasn't really supposed to be a list, just a few obvious problematic
examples. ^_^

However, I've spent some time over the past few weeks working to identify
all the problem cases so that we can have a more concrete proposal.

But first (for background), I extended your key event "stories" into 3
basic scenarios that I refer to (for lack of better, unambiguous terms) as
CHAR, MEANING and CODE.


CHAR (high-level character info)

The CHAR use-case is for users who are interested only in the printable
final character that the user typed (or entered through some other means).
The user gets the char info by handling 'keypress' (deprecated) or HTML5
'input' events and reading the 'char' field from the KeyboardEvent.
Example use-case: Detect character input (to validate field after each char
is entered)

MEANING (high-level key info - virtual key meaning)
---
The MEANING use-case is for users who are interested in the meaning of the
key being pressed, taking into account the current keyboard layout (and IME
and dead keys).
Example use-case: Detect modified keys or bare modifier keys (to perform an
action in response to shortcut)

CODE (low-level key info - physical keys)
-
The CODE use-case is for users who are interested in the key that was
pressed by the user, without any layout modifications applied.
Example use-cases: Games (detect WASD keys for movement), Remote desktop
client (trap all keys to send to remote host)


I was hoping that the MEANING and CODE scenarios could be satisfied with a
single field in the KeyboardEvent structure, but further reflection has
convinced me that this is not possible.

Consider these examples that demonstrate the difference between MEANING and
CODE, and why both are needed:

Left/Right Alt key:
CHAR MEANING   CODElocation
   US Keyboard   ""   "Alt"   "AltLeft"left
   US Keyboard   ""   "Alt"   "AltRight"   right
   French Keyboard   ""   "Alt"   "AltLeft"standard
   French Keyboard   ""  "AltGr"  "AltRight"   standard
   Notes:
  MEANING permits matching "Alt" regardless of which Alt key was
pressed.
  CODE permits matching "AltRight" regardless of which layout is in
effect.
  The location for "Alt" and "AltGr" is 'standard' since there's only
one
of each key. Arguably, these values could be 'left'/'right' to match
the US layout.

Single Quote key:
CHAR MEANING   CODE
   US Keyboard   "'"   "'""Quote"
   Japanese Kbd  ":"   ":""Quote"
   US Intl Kbd   """DeadAcute""Quote"
   Notes:
  For US Intl Kbd, we can't use the unmodified US key ("'") in the
MEANING since that's where the dead key is specified.

Equal Sign (with Left Alt pressed):
CHAR MEANING   CODEmodifiers
   US Keyboard   """=""EqualSign"  Alt
   Korean Kbd"""JunjaMode""EqualSign"  Alt

Two (with Shift key pressed):
CHAR MEANING   CODE
   US Keyboard   "@"  "@" "Digit2"
   UK Keyboard   """  """ "Digit2"
   French Keyboard   "2"  "2" "Digit2"
   Notes:
  If we adopted MEANING = unmodified US key, then the MEANINGs would
all be "2" instead of varying. That would fix this case, but
wouldn't help the other examples given here.


Note that neither the MEANING nor the CODE is sufficient by itself to
encode the info needed for all the desired use cases.

As for where to store this information, it's clear that we need 3 separate
fields. We already have 'char' and 'key' in the spec with 'char'
corresponding to CHAR and 'key' corresponding to MEANING. I propose a new
field 'code' which encodes the CODE information.

Note that, wherever possible, the CODE names come from the US keyboard
layout. So these values will be familiar to most developers while still
satisfying the requirement to provide for a unique mapping from code name
t

Re: Event.key complaints?

[Кошмарчик]

On Thu 1 Nov 2012, Hallvord R. M. Steen wrote:
> I would like the "story" of event.char and event.key to be that
> event.char describes the generated character (if any) in its
> shifted/unshifted/modified/localized glory while event.key describes
> the key (perhaps on a best-effort basis, but in a way that is at least
> as stable and usable as event.keyCode).

I think we're mostly in agreement here (except for being satisfied with
"best-effort" key codes ^_^).

> Hence, what I think would be most usable in the real world would be
> making event.key a mapping back to un-shifted character values of a
> normal QUERTY (en-US) layout. Authors are asking for stable reference
> values for identifying keys, and that's the most stable and widely
> known reference keyboard layout.

The main problem with using unmodified 'en-US' values is that it doesn't
define values for keys that are not found on US keyboards. So, it's great
for US keys, but completely ignores the rest of the world. And once you
start looking at adding support for these other keys, you find that things
aren't so simple.

This is why we proposed using the USB codes - it defines unique values that
cover pretty much every modern keyboard that's in use.

Consider the following problems with using 'en-US':
* What should the 'key' value be for the "B00" key (located next to the
left shift - see the ISO/IEC9995-2 spec[1])? This is used in UK, Italian,
Spanish, French, German and many other keyboards.
* What should the 'key' value be for "B11" key (next to the right shift)?
This is used on Brazilian and Japanese keyboards.
* And "C12" (tucked under Enter key when Enter takes 2 rows)? Keyboards
with B00 usually have C12 as well.
* And "E13" (between += and Backspace)? Found on Japanese (Yen key) and
Russian keyboards.
None of these keys exist on a US keyboard.

The USB codes solve all of these problems because they've already thought
about all the keyboard variation that exists in the world:
* For B00, the USB code = 0x64, name = "Non-US \ and |".
* For B11, the USB code = 0x87, name = "International1".
* For C12, the USB code = 0x32, name = "Non-US # and ~".
* For E13, the USB code = 0x89, name = "International3".

Simply specifying the 'key' codes as coming from the 'en-US' layout leaves
too many details undefined and (while it may be marginally better than the
legacy keyCode values) will most likely result in a lot of browser
variation for these non-US "edge cases".

Because of these problems, specifying this field as containing unmodified
'en-US' key is not an adequate solution.

As for the solution we need to come up with, it doesn't matter to me
if it's:
* encoded in the current 'key' field, or in a new field (although it'd be
nice to have the 'key' field "do the right thing").
* a numeric value or a string (although I think a numeric value is
preferable to avoid confusion with the 'char' value).
* the exact USB codes or something similar that we derive from them.

But, we do need it to:
* be able to uniquely identify each key on the keyboard.
* be useful for all keys on all keyboards (and not just those that map
nicely to 'en-US')
* be straightforward to implement on all platforms.

-Gary

[1] See http://en.wikipedia.org/wiki/ISO/IEC_9995

Re: Proposal to add USB keycodes to the current DOM3 key events

[Кошмарчик]

On Thu, Sep 13, 2012 at 12:15 AM, Masayuki Nakano wrote:

> Hi, I'm a developer of Gecko. So, I'd like to comments to this document.


Thanks for your comments and my apologies for taking so long to respond.


> On 2012/09/08 3:37, Gary Kacmarcik (Кошмарчик) wrote:
>
>> Problems with the current DOM Level 3 key events
>> ==**==
>>
>> Problem 1: Matching keydown and keyup events
>> --**--
>>
>> The main problem with the current proposal is that it doesn't provide
>> enough information so that keyup events can be matched with their
>> corresponding keydown event. In this regard, it is unlike the legacy
>> events, where the keyCode value was usually sufficient to match the
>> keydown/keyup events.
>>
>
> Yeah, but there are some cases that only keydown or only keyup event is
> fired.
>
> * If focus is moved to another window or something while some keys are
> pressed, keyup event are never fired if the keys are released while the
> browser is deactive.
> * If the browser becomes active while some keys are pressed, only keyup
> events are fired if the keys are released while the browser is active.
>

These are both true and valid. However, they describe a separate issue from
what we're trying to solve here.

Even in the best case scenario where you have not lost any key events, we
can not reliably match them with the current specification.
Even if we solve the problem of how to handle "lost" keyups/keydowns when a
window loses/gains focus, we still can not reliably match them with the
current specification.

* If a utility software or IME generates fake native key event to the
> browser, it may cause only keydown or keyup event if the native key event
> isn't one or more sets of native keydown and keyup events.
>

While I don't fully understand the intricacies of how events are fired from
IMEs, I don't believe that we're any worse off in this case.  But we're not
trying to solve the "missing key event" problem, we're trying to solve the
problem where both the keydown and the keyup are present but cannot be
matched because the |key| value is different for the keydown and keyup
event.

So, anyway, it's very difficult issue to know if a key is actually pressed.


Yes. But even if you haven't lost any events, the current spec does not
allow events to be reliably matched.

 For examples of why this is a problem, consider the following scenarios:
>>
>> * Games and other applications which treat the keyboard as a large
>> collection of buttons need to be able to keep track of which keys are
>> currently being held down. Since there is no API to query the current
>> keyboard state, it is crucial that apps be able to match every keyup
>> event with the originating keydown event.
>>
>
> Only Windows (Win32, I'm not sure for Metro) has such native API. So, even
> if web specs defines such API, browsers cannot implement it on some
> platforms, unfortunately.


The lack of a nice cross-platform way of getting keyboard state means that
webapp developers will be relying on manual tracking of key up/down events.
I believe that makes it more important to make is possible to reliably
match a keyup event with its originating keydown event (again, assuming it
hasn't been lost or sent to another element).

 * Applications that provide remote access functionality often need to
>> keep track of the sequence of events that generated a particular
>> character. This is so that they can handle the case where the local
>> and remote systems have different keyboard layouts (since they may
>> require a different sequence of keys or modifiers to be injected on
>> the remote machine).
>>
>
> I don't understand this case. I think that applications should prefer the
> actual key event's information rather than the different keyboard layout...


Remote access systems need to match keydown and keyup events so that they
can (if necessary) inject appropriate modifier keys on the remote computer
so the correct key event is generated. A simple case would be a user at a
client machine (with a French keyboard) connecting to a remote host (with a
US keyboard). To type a '2', the user needs to hold the shift key on the
French keyboard. To properly generate this using low-level events on the
host (with the US layout), the shift key would need to be unpressed before
the '2' key event is injected.

This example is a bit esoteric, but basically remote access apps need to
track the keyboard state by matching keydown and keyup events.

 Proposal
>> 
>>
>
>  A more complete mapping table between USB usage codes and
&

Re: Proposal to add USB keycodes to the current DOM3 key events

[Кошмарчик]

On Fri, Sep 7, 2012 at 11:44 AM, Travis Leithead
 wrote:
>> From: gary...@google.com [mailto:gary...@google.com] On Behalf Of Gary
>>
>> I've written up a brief proposal to enhance the current DOM Level 3
>> key events by adding USB keycodes.
>
> Nice. Yes, getting the keyboard layout-independent position of a key was 
> specifically out-of-scope for DOM Level 3 Events. I don't know much about the 
> USB key codes, but it looks promising as a standard way of getting key 
> location interop across OS/languages.
>
> I did want to call out a sentence from you problem statement:
>
> "Games and other applications which treat the keyboard as a large collection 
> of buttons need to be able to keep track of which keys are currently being 
> held down. ***Since there is no API to query the current keyboard state***, 
> it is crucial that apps be able to match every keyup event with the 
> originating keydown event."
>
> An old strawman was proposed in 2009 for this: 
> http://www.w3.org/2008/webapps/wiki/Query_Key_Status, but is still worth 
> pursuing/revising.

Thanks for the comments!

I agree that an API to query the keyboard state is definitely
worthwhile, but I feel this proposal is independent from whatever
decision we eventually make with regard to a QueryKeyStatus API.

The most serious problem with the current key event spec is that there
is no reliable way to link keyup events with the corresponding keydown
event. Even if we added a QueryKeyStatus API, this would still be
broken.

-Gary

Re: Proposal to add USB keycodes to the current DOM3 key events

[Кошмарчик]

(Including the contents of entire document inline so that the info is
easier to skim/comment on.)

-Gary Kacmarcik (gary...@chromium.org


Proposal for enhancing the current DOM Level 3 key events with USB scancodes

Status: DRAFT
Authors: Gary Kacmarcik, James Weatherall


The current DOM Level 3 key event spec improves considerably upon the
"legacy" HTML4 key events that are currently used by most browsers.
The HTML4 events have always been problematic because the lack of a
detailed specification for the events led to implementation
inconsistencies on the various platform/browser combinations. The DOM
Level 3 proposal seeks to address this problem by specifying exactly
how these key events should be encoded.

However, there are still cases where the current proposal does not
provide enough information about the key event to perform desirable
tasks. In some cases, the legacy event values actually provide more
(internally) consistent information about the events.


Problems with the current DOM Level 3 key events


Problem 1: Matching keydown and keyup events


The main problem with the current proposal is that it doesn't provide
enough information so that keyup events can be matched with their
corresponding keydown event. In this regard, it is unlike the legacy
events, where the keyCode value was usually sufficient to match the
keydown/keyup events.

For examples of why this is a problem, consider the following scenarios:

* Games and other applications which treat the keyboard as a large
collection of buttons need to be able to keep track of which keys are
currently being held down. Since there is no API to query the current
keyboard state, it is crucial that apps be able to match every keyup
event with the originating keydown event.

* Applications that provide remote access functionality often need to
keep track of the sequence of events that generated a particular
character. This is so that they can handle the case where the local
and remote systems have different keyboard layouts (since they may
require a different sequence of keys or modifiers to be injected on
the remote machine).

Appendix A (at the end of this document) contains some sample key
event sequences that demonstrate this problem. The simple case of
typing a shifted character (the '@' sign) works fine in the normal
case, but if the Shift key is released too early then the keyup will
have a different value than the original keydown. This inconsistency
makes it difficult to match the keyup with the originating keydown.

Note that since these event values are dependent on the current
keyboard layout, there is no simple mapping table between the shifted
and unshifted version of each key. The international keyboard layout
examples in Appendix A demonstrate this complication.


Problem 2: Identifying keys by their position
-

While not as critical as the first problem, another common task for
some applications (typically games) is to handle key events based on
the key's position on the keyboard. For example, a game may want to
detect when the key next to the CapsLock key is pressed so it can be
associated with some player action. With a US keyboard, this would be
the key with the 'A' keycap, but on an AZERTY keyboard, this key would
have a 'Q' keycap.

A layout-independent way of identifying keys would be useful to games,
music/beatbox players, and other apps that prefer to consider the
keyboard as a set of buttons. A touch-typing app could also use
information to this target lessons for the home row or for the
left/right hand.

(Note that some implementations of HTML4 key events suffer from this
problem as well. For example, the keycodes in WebKit-based browsers
are based on Windows VKEY values, which are layout-dependent.)


Proposal


Our proposal to address these problems is to add a new field that
contains the USB Usage Page and Usage ID codes for keydown and keyup
events.

In this proposal, the KeyboardEvent would be extended to include:

   interface KeyboardEvent : UIEvent
   {
  ...
  readonly attribute unsigned long usbUsage;
  ...
   };

The usbUsage field is a combination of the USB Usage Page (upper
16-bits) and the USB Usage ID (lower 16-bits).  Some example usbUsage
codes are given in Appendix B, but a complete list can be found in the
USB HID Usage Tables documentation (see References).

The vast majority of these codes would come from Usage Page 0x07,
which contains Keyboard/Keypad values. However, many modern keyboards
contain specialized "media" keys which would have values defined in
other Usage Pages.

Providing this information in addition to the currently specified
fields would be address the problems described earlier:


Solving Problem 1 : Easy to match keydown with keyup

Since the USB key values are associated with the individual keys, the
value would not depend 

Proposal to add USB keycodes to the current DOM3 key events

[Кошмарчик]

Hi all,

I've written up a brief proposal to enhance the current DOM Level 3
key events by adding USB keycodes.

Here is a link to the proposal document:
https://docs.google.com/document/d/1eJvlUaTBsWa71hIc0X4s6SrCopX9LCPs1YOAuVluxBs/edit


_In summary_

*Why this proposal?*

With the current DOM3 key events:
Problem 1: Keyup events cannot be reliably matched with the
originating keydown event. This is our primary motivation for this
proposal since this causes problems for a webapp that we are working
on.
Problem 2: Keys cannot be identified by their position on the keyboard.

*The proposal, in a nutshell*

Add a new usbUsage field to the KeyboardEvent. The value of this field
would contain the USB Usage Page and Usage ID for the key. These
values are consistent and identify keys by their location on the
keyboard without being affected by modifier keys or the current
keyboard layout.


Please see the linked document for a more detailed description and
motivating examples.

Thank you,
Gary Kacmarcik
gary...@chromium.org