Re: [gamepad] Allow standard-mapped devices to have extra buttons or axes allow multiple mappings per device
Hi, I agree with Chad's suggestions: the spec should be expanded to expose more than the core set of axes buttons, and richer metadata would be extremely useful to developers. I've been using the gamepad API (as shipped in Chrome, at least - I haven't had time to adapt to Firefox's implementation, because I used gamepad.js and it's a broken pile...) for a while now, in a few shipped JSIL-based game ports. My experiences opinions inform the following. I apologize if this is a bit hard to read, by the way - all this crunch time for the game I'm working on has fried my brain a bit. :-) To try and express my PoV on this (previously stated to WG members/participants, but not directly on this list - sorry, too much on my plate to add another workgroup to my subscriptions): I think best-effort mapping to 'standard' layouts is a great idea. I have some objections to how it is done, but it is an improvement over what we had before. My main concern is that the current design for remapping sets us up for a descent into a quagmire that can't easily be fixed. The scenario I anticipate happening: Gamepads only have one standard mapping, supported by the vast majority of available controllers (let's just say 360 pads and dualshock 3/4 controllers, since those put together probably cover at least 90% of the market). If you expand the remapping to all XInput devices, that widens it even further - though this is problematic in a way I will get to later. If there is one standard mapping, and it covers most users, this means the natural path for developers is to only support the standard mapping. They won't add keybinding/button remapping, they won't add support for multiple layouts, they might not even add support for choosing which of your gamepads you want to use. This is not an unrealistic fear: It is the status quo in virtually all console games and in many PC games. In some cases you can rebind by editing .ini files, but that is out of the question for web games since their configuration typically lies in not-user-editable JS on the server or in blobs stored somewhere (localStorage, indexed db, cookies) that are not easily user-editable. This 'developers only support the easiest thing' future is relatively likely and is actually worse than the previous status quo of raw-controllers-only. I think the solution to this is simple: Ensure that there is more than one standard mapping. Find a way to make sure that developers have to do at least a basic amount of effort to handle things correctly. If the simple hack isn't sufficient for wider audiences, they will have an incentive to do a good job. This doesn't have to be a punitive measure - there are diverse controllers out there that people might want to use, from standard gamepads to racing wheels to flight sticks. Something as simple as supporting the additional inputs of the dualshock 3 4 is a good rationale here (they both have gyros accelerometers; the 4 has a touchpad and touchpanel button.) When the common case is slightly more complicated, it forces developers to at least *understand* the breadth of the problem and make rational choices. It also encourages the development of high-quality, userspace JS wrappers/abstraction layers, so fixes improvements can happen in those libraries. I would argue that the success of XInput is not due to its excessive, somewhat-detrimental 'least common denominator' design, but because it aggressively targeted fixed the most common problems with DirectInput. OK, so, there's my rationale for why the current approach to mapping should change. Now to dive in deeper to some other issues I see with the gamepad API. My first objection to the design implementation is that it is based on passive enumeration device acquisition. You have to poll to find out what gamepads there are and interact with them. This obviously produces the threat of fingerprinting, and to mitigate fingerprinting we have the *miserable* constraint where you have to push a button to make your controller show up. This changes the ordinal identifiers of gamepads from session to session, and - even worse - means that a controller cannot be activated by analog stick inputs, which is a real problem for games that start with a menu (like two of my major ports). This defies user expectations and is just plain confusing. Polling an actual device to get its state is perfectly reasonable, but polling the overall state of your input devices is not so. Device availability is something that should be event-based (it is at a driver level anyway) and it should be exposed to content as events. The fingerprinting problem is addressed with the solution I proposed here: Content should *request* the input it needs, at which point the useragent is able to figure out how to provide it. To mitigate fingerprinting concerns, this can be done based on user consent, and the input presented to the content can end up fully sanitized, because the user content has no need to
Re: [gamepad] Allow standard-mapped devices to have extra buttons or axes allow multiple mappings per device
Re: change #1, with standard gamepad mappings today (at least in Chrome) we map any buttons that don't correspond to the official standard mapping to button[17] and up. This, of course, depends on which buttons are actually visible to the browser (many devices expose buttons that can't be seen by normal applications, like the home button on Xbox controllers). A good example is the Touchpad on the PS4 controller. We can't see actual touch inputs without writing a custom driver, but we detect clicks on the touchpad and report them as button[17]. The same would go for axes, but I've yet to encounter a standard-looking gamepad that exposed four axes worth of usable data. Perhaps the spec should be slightly reworded in this regard, since it currently states that The standard gamepad has 4 axes, and up to 17 buttons, which could be read as explicitly disallowing buttons or axes beyond that. I'm not very supportive of suggestion #2 since we don't actually HAVE any other mappings defined (joystick/flightstick would be the next most natural candidate, I think, but they can vary significantly). If we did, I'm not sure how you would reasonably stack mappings without breaking an awful lot of code. The mappings field would have to be either space/comma delimited (which breaks things) or turned into an array (which breaks things) or you would expose the same device twice under different mappings (which is really weird and probably breaks things.) Beyond that, there's no way that the layouts of two mapping types could every overlay completely cleanly, and we really shouldn't be claiming that something that looks like a racing wheel is actually a standard gamepad anyway. I just can't see how it would work. --Brandon On Sun Oct 12 2014 at 11:59:16 PM Chad Austin caus...@gmail.com wrote: Hi all, I recently was asked to review the Gamepad API draft specification. My background is games though I've done some scientific computing with alternate input devices too. http://chadaustin.me/2014/10/the-gamepad-api/ I'd like to suggest two simple changes to the Gamepad API. In tandem, these APIs would allow applications to have automatic compatibility with a broader range of devices. The changes are: 1. Allow standard-mapped devices to have extra buttons or axes 2. Add support for multiple standard mappings per device. Together, these changes would allow a simple application that only needs a d-pad and a few buttons to work automatically with a Wiimote, XInput device, and a racing wheel (many racing wheels have d-pads on them). Even without the proposal #2, proposal #1 would allow optional access to extra capabilities on a device without having to sacrifice the standard mapping. This feels noncontroversial to me because mappings only assign meaning to otherwise neutral indices in the buttons[] and axes[] arrays. For a given device, there can be many such meanings. -- Chad Austin http://chadaustin.me
Re: [gamepad] Allow standard-mapped devices to have extra buttons or axes allow multiple mappings per device
On Mon, Oct 13, 2014 at 3:55 PM, Brandon Jones bajo...@google.com wrote: Re: change #1, with standard gamepad mappings today (at least in Chrome) we map any buttons that don't correspond to the official standard mapping to button[17] and up. This, of course, depends on which buttons are actually visible to the browser (many devices expose buttons that can't be seen by normal applications, like the home button on Xbox controllers). A good example is the Touchpad on the PS4 controller. We can't see actual touch inputs without writing a custom driver, but we detect clicks on the touchpad and report them as button[17]. The same would go for axes, but I've yet to encounter a standard-looking gamepad that exposed four axes worth of usable data. Perhaps the spec should be slightly reworded in this regard, since it currently states that The standard gamepad has 4 axes, and up to 17 buttons, which could be read as explicitly disallowing buttons or axes beyond that. Great! Then I agree - the behavior of Chrome then doesn't match the exact wording of the spec. The spec should be updated to reflect reality. I'm not very supportive of suggestion #2 since we don't actually HAVE any other mappings defined (joystick/flightstick would be the next most natural candidate, I think, but they can vary significantly). If we did, I'm not sure how you would reasonably stack mappings without breaking an awful lot of code. The mappings field would have to be either space/comma delimited (which breaks things) or turned into an array (which breaks things) or you would expose the same device twice under different mappings (which is really weird and probably breaks things.) Beyond that, there's no way that the layouts of two mapping types could every overlay completely cleanly, and we really shouldn't be claiming that something that looks like a racing wheel is actually a standard gamepad anyway. I just can't see how it would work. What's the status of the gamepad API? It reads like it's in a draft state, yet it has some implementations in the wild... Is it de facto frozen? Assuming it's not de facto frozen -- and temporarily sidestepping the problem of maintaining compatibility with existing implementations -- the issue with the mapping API _as written today_ is that there's only one standard mapping, meaning that if you plug in any non-XInput-style device that has a d-pad, you'll need the customer to specify their own bindings. Your device is nonstandard. Press left. Press up. Press right. Press down. Press OK. Beyond that, there's no way that the layouts of two mapping types could every overlay completely cleanly, and we really shouldn't be claiming that something that looks like a racing wheel is actually a standard gamepad anyway. I just can't see how it would work. The concern that two mappings that each specify a directional pad would not have overlapping button IDs is valid. Fundamentally, what I'd like is access to a richer form of per-button and per-axis metadata than is provided by the API today. The standard mapping API provided today is rigid and limiting. However, I don't have time to come up with a proposal right now - perhaps coming soon. +Katelyn as she expressed similar concerns. Thanks, Chad
