[gamepad] Allow standard-mapped devices to have extra buttons or axes allow multiple mappings per device
Hi all, I recently was asked to review the Gamepad API draft specification. My background is games though I've done some scientific computing with alternate input devices too. http://chadaustin.me/2014/10/the-gamepad-api/ I'd like to suggest two simple changes to the Gamepad API. In tandem, these APIs would allow applications to have automatic compatibility with a broader range of devices. The changes are: 1. Allow standard-mapped devices to have extra buttons or axes 2. Add support for multiple standard mappings per device. Together, these changes would allow a simple application that only needs a d-pad and a few buttons to work automatically with a Wiimote, XInput device, and a racing wheel (many racing wheels have d-pads on them). Even without the proposal #2, proposal #1 would allow optional access to extra capabilities on a device without having to sacrifice the standard mapping. This feels noncontroversial to me because mappings only assign meaning to otherwise neutral indices in the buttons[] and axes[] arrays. For a given device, there can be many such meanings. -- Chad Austin http://chadaustin.me
[gamepad] Add an event-based input mechanism
Just as I mentioned in my previous email to this list, I recently was asked to review the Gamepad API draft specification. My background is games though I've done some scientific computing with alternate input devices too. http://chadaustin.me/2014/10/the-gamepad-api/ I'd like to make a second proposed change to the gamepad API, an event-based mechanism for receiving button and axis changes. The full rationale is explained in the linked article, but the summary is: 1) an event-based API entirely avoids the issue of missed button presses 2) event-based APIs don't require non-animating web pages to use requestAnimationFrame just to poll gamepad state 3) an event-based API could (and should!) give access to high-precision event timing information for use in gesture recognition and scientific computing 4) event-based APIs can reduce processing latency when a button changes state rapidly in low-frame-rate situations -- Chad Austin http://chadaustin.me
Re: [gamepad] Add an event-based input mechanism
On Mon, Oct 13, 2014 at 1:06 AM, Florian Bösch pya...@gmail.com wrote: Note that events for axis input can (when wrongly handled) lead to undesirable behavior. For instance, suppose you have a 2-axis input you use to plot a line on screen. If you poll the positions and then draw a line from the last position to the current position, you will get a smooth line. However if you receive events for axes, each arriving separately, and use each to plot a line, you will get a stepped/stairs line. If nothing but an event based mechanism is present, this would force an author to catch all axes events and collate them when no more events are queued. That'd require knowing when are no more events coming, which cannot be done in a callback based mechanism (as the event queue can't be queried). You are absolutely correct, and I did not necessarily mean gamepad events should look like onmousemove or onkeydown. I was imagining an event more like device has new data, after which a non-empty list of events would be available from the device. Presumably, events in said list can be coalesced to fit within a fixed buffer size if necessary. On the specific use case of drawing a smooth line from rough axis input, an appropriate smoothing algorithm should not rely on polling frequency -- especially requestAnimationFrame frequency -- but instead should generate the smooth curve (or sequence of line segments or whatever) from high-resolution event timing information if possible. Of course, this is where the discussion starts to approach the intersection of how humans perceive input and output, and why mice are sampled at hundreds of Hz rather than the 60 Hz of requestAnimationFrame. It's conceivable that some use-cases of buttons can run into similar scenarios (combos and whatnot). Although in these cases, a developer could probably keep track of the pressed buttons themselves. However, this means that not only button presses have to be provided, but also button releases, such that combos could be reliably detected by the developer. Yes, definitely. Button up is as important as button down.
Re: [gamepad] Allow standard-mapped devices to have extra buttons or axes allow multiple mappings per device
On Mon, Oct 13, 2014 at 3:55 PM, Brandon Jones bajo...@google.com wrote: Re: change #1, with standard gamepad mappings today (at least in Chrome) we map any buttons that don't correspond to the official standard mapping to button[17] and up. This, of course, depends on which buttons are actually visible to the browser (many devices expose buttons that can't be seen by normal applications, like the home button on Xbox controllers). A good example is the Touchpad on the PS4 controller. We can't see actual touch inputs without writing a custom driver, but we detect clicks on the touchpad and report them as button[17]. The same would go for axes, but I've yet to encounter a standard-looking gamepad that exposed four axes worth of usable data. Perhaps the spec should be slightly reworded in this regard, since it currently states that The standard gamepad has 4 axes, and up to 17 buttons, which could be read as explicitly disallowing buttons or axes beyond that. Great! Then I agree - the behavior of Chrome then doesn't match the exact wording of the spec. The spec should be updated to reflect reality. I'm not very supportive of suggestion #2 since we don't actually HAVE any other mappings defined (joystick/flightstick would be the next most natural candidate, I think, but they can vary significantly). If we did, I'm not sure how you would reasonably stack mappings without breaking an awful lot of code. The mappings field would have to be either space/comma delimited (which breaks things) or turned into an array (which breaks things) or you would expose the same device twice under different mappings (which is really weird and probably breaks things.) Beyond that, there's no way that the layouts of two mapping types could every overlay completely cleanly, and we really shouldn't be claiming that something that looks like a racing wheel is actually a standard gamepad anyway. I just can't see how it would work. What's the status of the gamepad API? It reads like it's in a draft state, yet it has some implementations in the wild... Is it de facto frozen? Assuming it's not de facto frozen -- and temporarily sidestepping the problem of maintaining compatibility with existing implementations -- the issue with the mapping API _as written today_ is that there's only one standard mapping, meaning that if you plug in any non-XInput-style device that has a d-pad, you'll need the customer to specify their own bindings. Your device is nonstandard. Press left. Press up. Press right. Press down. Press OK. Beyond that, there's no way that the layouts of two mapping types could every overlay completely cleanly, and we really shouldn't be claiming that something that looks like a racing wheel is actually a standard gamepad anyway. I just can't see how it would work. The concern that two mappings that each specify a directional pad would not have overlapping button IDs is valid. Fundamentally, what I'd like is access to a richer form of per-button and per-axis metadata than is provided by the API today. The standard mapping API provided today is rigid and limiting. However, I don't have time to come up with a proposal right now - perhaps coming soon. +Katelyn as she expressed similar concerns. Thanks, Chad
=[xhr] Expose XMLHttpRequest priority
Hi all, I would like to see a priority field added to XMLHttpRequest. Mike Belshe's proposal here is a great start: http://www.mail-archive.com/public-webapps@w3.org/msg08218.html *Motivation* Browsers already prioritize network requests. By giving XMLHttpRequest access to the same machinery, the page or application can reduce overall latency and make better use of available bandwidth. I wrote about our specific use case (efficiently streaming hundreds of 3D assets into WebGL) in detail at http://chadaustin.me/2014/08/web-platform-limitations-xmlhttprequest-priority/ Gecko appears to support a general 32-bit priority: http://lxr.mozilla.org/mozilla-central/source/xpcom/threads/nsISupportsPriority.idl and http://lxr.mozilla.org/mozilla-central/source/netwerk/protocol/http/HttpBaseChannel.cpp#45 Chrome appears to be limited to five priorities: https://code.google.com/p/chromium/codesearch#chromium/src/net/base/request_priority.hsq=package:chromiumtype=csrcl=1411964872 but seems to have a fairly general priority queue implementation. https://code.google.com/p/chromium/codesearch#chromium/src/content/browser/loader/resource_scheduler.ccsq=package:chromiumtype=csrcl=1411964872l=206 SPDY exposes 3 bits of priority per stream. *Proposal* Add a numeric priority property to XMLHttpRequest. It is a 3-bit integer from 0 to 7. Default to 3. 0 is most important. Why integers and not strings, as others have proposed? Because priority arithmetic is convenient. For example, in our use case, we might say The top bit is set by whether an asset is high-resolution or low-resolution. Low-resolution assets would be loaded first. The bottom two bits are used to group request priorities by object. The 3D scene might be the most important resource, followed by my avatar, followed by closer objects, followed by farther objects. Note that, with a very simple use case, we've just consumed all three bits. There's some vague argument that having fewer priorities makes implementing prioritization easier, but as we've seen, the browsers just have a priority queue anyway. Allow priorities to change after send() is called. The browser may ignore this change. It could also ignore the priority property entirely. I propose XMLHttpRequest priority not be artificially limited to a range of priorities relative to other resources the browser might initiate. That is, the API should expose the full set of priorities the browser supports. If my application wants to prioritize an XHR over some browser-initiated request, it should be allowed to do so. The more control over priority available, the better a customer experience can be built. For example, at the logical extreme, fine-grained priority levels and mutable priority values would allow dynamically streaming and reprioritizing texture mip levels as objects approach the camera. If there's enough precision, the application could set priority of an object to the distance from the camera. Or, in a non-WebGL scenario, an image load's priority could be set to the distance from the current viewport. I believe this proposal is very easy to implement: just plumb the priority value through to the prioritizing network layer browsers already implement. What will it take to get this added to the spec? Thanks, -- Chad Austin Technical Director, IMVU http://chadaustin.me
Re: =[xhr] Expose XMLHttpRequest priority
On Tue, Sep 30, 2014 at 5:28 AM, Anne van Kesteren ann...@annevk.nl wrote: On Tue, Sep 30, 2014 at 10:25 AM, Chad Austin caus...@gmail.com wrote: What will it take to get this added to the spec? There's been a pretty long debate on the WHATWG mailing list how to prioritize fetches of all things. I recommend contributing there. I don't think we should focus on just XMLHttpRequest. Hi Anne, Thanks for the quick response. Is this something along the lines of a SupportsPriority interface that XHR, various DOM nodes, and such would implement? Can you point me to the existing discussion so I have context? Thanks, Chad
