Hi, I agree with Chad's suggestions: the spec should be expanded to expose more than the core set of axes & buttons, and richer metadata would be extremely useful to developers.
I've been using the gamepad API (as shipped in Chrome, at least - I haven't had time to adapt to Firefox's implementation, because I used gamepad.js and it's a broken pile...) for a while now, in a few shipped JSIL-based game ports. My experiences & opinions inform the following. I apologize if this is a bit hard to read, by the way - all this crunch time for the game I'm working on has fried my brain a bit. :-) To try and express my PoV on this (previously stated to WG members/participants, but not directly on this list - sorry, too much on my plate to add another workgroup to my subscriptions): I think best-effort mapping to 'standard' layouts is a great idea. I have some objections to how it is done, but it is an improvement over what we had before. My main concern is that the current design for remapping sets us up for a descent into a quagmire that can't easily be fixed. The scenario I anticipate happening: Gamepads only have one standard mapping, supported by the vast majority of available controllers (let's just say 360 pads and dualshock 3/4 controllers, since those put together probably cover at least 90% of the market). If you expand the remapping to all XInput devices, that widens it even further - though this is problematic in a way I will get to later. If there is one standard mapping, and it covers most users, this means the natural path for developers is to only support the standard mapping. They won't add keybinding/button remapping, they won't add support for multiple layouts, they might not even add support for choosing which of your gamepads you want to use. This is not an unrealistic fear: It is the status quo in virtually all console games and in many PC games. In some cases you can rebind by editing .ini files, but that is out of the question for web games since their configuration typically lies in not-user-editable JS on the server or in blobs stored somewhere (localStorage, indexed db, cookies) that are not easily user-editable. This 'developers only support the easiest thing' future is relatively likely and is actually worse than the previous status quo of raw-controllers-only. I think the solution to this is simple: Ensure that there is more than one standard mapping. Find a way to make sure that developers have to do at least a basic amount of effort to handle things correctly. If the simple hack isn't sufficient for wider audiences, they will have an incentive to do a good job. This doesn't have to be a punitive measure - there are diverse controllers out there that people might want to use, from standard gamepads to racing wheels to flight sticks. Something as simple as supporting the additional inputs of the dualshock 3 & 4 is a good rationale here (they both have gyros & accelerometers; the 4 has a touchpad and touchpanel button.) When the common case is slightly more complicated, it forces developers to at least *understand* the breadth of the problem and make rational choices. It also encourages the development of high-quality, userspace JS wrappers/abstraction layers, so fixes & improvements can happen in those libraries. I would argue that the success of XInput is not due to its excessive, somewhat-detrimental 'least common denominator' design, but because it aggressively targeted & fixed the most common problems with DirectInput. OK, so, there's my rationale for why the current approach to mapping should change. Now to dive in deeper to some other issues I see with the gamepad API. My first objection to the design & implementation is that it is based on passive enumeration & device acquisition. You have to poll to find out what gamepads there are and interact with them. This obviously produces the threat of fingerprinting, and to mitigate fingerprinting we have the *miserable* constraint where you have to push a button to make your controller show up. This changes the ordinal identifiers of gamepads from session to session, and - even worse - means that a controller cannot be activated by analog stick inputs, which is a real problem for games that start with a menu (like two of my major ports). This defies user expectations and is just plain confusing. Polling an actual device to get its state is perfectly reasonable, but polling the overall state of your input devices is not so. Device availability is something that should be event-based (it is at a driver level anyway) and it should be exposed to content as events. The fingerprinting problem is addressed with the solution I proposed here: Content should *request* the input it needs, at which point the useragent is able to figure out how to provide it. To mitigate fingerprinting concerns, this can be done based on user consent, and the input presented to the content can end up fully sanitized, because the user content has no need to identify devices and make choices about how to map inputs. The request can clearly specify the *semantic* inputs the game requires, like an 'accept button', instead of just identifying buttons by ordinal or some rigid layout with buttons called things like 'a' and 'x'. Semantic requests can easily be an optional thing, at which point the common case is roughly identical to the present. Semantic requests for buttons also mean that the game has a user-friendly name to display for each button, and that name can be properly localized. This leads into my second objection: I think it is unwise to straddle the boundary between useragent & content the way the current approach does. The browser makes a 'best effort' at remapping, but there is no real facility for the content to figure out how good the 'best effort' was, or to opt into getting the raw state so they can do a better job. The fact that a 'best effort' will not always be possible means that a bunch of the work here - arguably some of the most important work - is shoved off into content, but the common case is easy enough that content may not cover the less common cases. This really sucks. I think any solution that can't exist entirely in the useragent must exist in content. I argue that the correct approach to remapping is to do it in userspace JS. The fact that remapping has already shipped prevents this, but perhaps there is some sort of compromise that can be reached - on first use of the gamepad api, perhaps, a blessed userspace JS library is loaded that provides remapping, and there is a trivial way for content to request that a content-provided version of the library is used instead. This would end up working sort of like a polyfill in reverse, where if you've loaded a polyfill the browser does not trample over it. I think a relatively ideal - not perfect, realistic - solution in content can also solve problems that the browser can't solve by itself. In detail: * The blessed polyfill can be officially hosted on a community-controlled domain - perhaps shepherded by a known good actor like Mozilla or Google - that acts as a canonical location for the gamepad mapping support. * As a result, the community domain can act as a storage repository for the user's button mappings & other information, perhaps only hosting a simple script that exposes this shared data to individual games and applications. This storage doesn't have to exist in the cloud - the domain just acts as an origin any localStorage or indexedDB data is tied to, so all content can access it. * Given the shared storage repository, the burden of custom button mapping & device identification/remapping can be entirely offloaded to content while simultaneously improving the experience for users. * The blessed library can provide a user-friendly API & interface that covers all the corner cases - a good UI that shows the inputs the game wants and allows the user to map them, along with an easy 'default' that shows you the best-fit mapping so you can decide whether to use it. This means that a user who doesn't want the best-fit mapping - they want to invert y, they want to rearrange the buttons because they have a disability that makes the standard layout difficult - can easily opt out of it. I believe my proposed approach addresses all the problems the current gamepad API solves, while offloading a lot of the burden from the browser developer to content. In addition, it solves some problems the current API does not. The ability to save and customize bindings on a global basis instead of on a per-game basis would be a HUGE usability improvement for web games, and more importantly it would be an advantage traditional PC games can't offer. In practice, for developers this would mean a new API sitting on top of the existing gamepad API, where instead of polling for gamepads, you ask for one or more virtualized gamepads that fit your game's requirements. If the user has already opted into it before, you just instantly get the devices you want, if they're available. No need to cope with fingerprinting protections. If the user has existing bindings for your game or another game with the same semantic requirements, they have no need to bind inputs again. If you need a new 'standard' layout like a flightstick or a racing wheel, you ask for that instead. The part where you've already got a gamepad remains the same: None of that API has to change. Just the part where you acquire devices. This means the existing implementation (warts and all) can remain and the new, ideal design can replace it side-by-side. I'm aware that my comments here & my proposal are rather complicated so I'd be happy to explain them in more detail. Full disclosure: I was involved in the initial prototyping & spec work for the gamepad API that happened at Mozilla, up until shortly before Google's shipped prototype API became the de-facto standard. I'm slightly bitter, if only because the request-oriented approach to acquiring gamepads is what I proposed originally and fought hard to make happen on our side. I'm a bit sad that the current approach shipped because while it solves important problems, it sets us up for some pain in the future. On 13 October 2014 17:58, Chad Austin <caus...@gmail.com> wrote: > On Mon, Oct 13, 2014 at 3:55 PM, Brandon Jones <bajo...@google.com> wrote: >> >> Re: change #1, with standard gamepad mappings today (at least in Chrome) >> we map any buttons that don't correspond to the official standard mapping to >> button and up. This, of course, depends on which buttons are actually >> visible to the browser (many devices expose buttons that can't be "seen" by >> normal applications, like the home button on Xbox controllers). A good >> example is the Touchpad on the PS4 controller. We can't see actual touch >> inputs without writing a custom driver, but we detect clicks on the touchpad >> and report them as button. The same would go for axes, but I've yet to >> encounter a standard-looking gamepad that exposed > four axes worth of >> usable data. Perhaps the spec should be slightly reworded in this regard, >> since it currently states that "The standard gamepad has 4 axes, and up to >> 17 buttons", which could be read as explicitly disallowing buttons or axes >> beyond that. > > > Great! Then I agree - the behavior of Chrome then doesn't match the exact > wording of the spec. The spec should be updated to reflect reality. > >> I'm not very supportive of suggestion #2 since we don't actually HAVE any >> other mappings defined (joystick/flightstick would be the next most natural >> candidate, I think, but they can vary significantly). If we did, I'm not >> sure how you would reasonably "stack" mappings without breaking an awful lot >> of code. The "mappings" field would have to be either space/comma delimited >> (which breaks things) or turned into an array (which breaks things) or you >> would expose the same device twice under different mappings (which is really >> weird and probably breaks things.) Beyond that, there's no way that the >> layouts of two mapping types could every overlay completely cleanly, and we >> really shouldn't be claiming that something that looks like a racing wheel >> is actually a "standard gamepad" anyway. I just can't see how it would work. > > > What's the status of the gamepad API? It reads like it's in a draft state, > yet it has some implementations in the wild... Is it de facto frozen? > > Assuming it's not de facto frozen -- and temporarily sidestepping the > problem of maintaining compatibility with existing implementations -- the > issue with the mapping API _as written today_ is that there's only one > standard mapping, meaning that if you plug in any non-XInput-style device > that has a d-pad, you'll need the customer to specify their own bindings. > "Your device is nonstandard. Press left. Press up. Press right. Press > down. Press OK." > >> Beyond that, there's no way that the layouts of two mapping types could >> every overlay completely cleanly, and we really shouldn't be claiming that >> something that looks like a racing wheel is actually a "standard gamepad" >> anyway. I just can't see how it would work. > > > The concern that two mappings that each specify a directional pad would not > have overlapping button IDs is valid. > > Fundamentally, what I'd like is access to a richer form of per-button and > per-axis metadata than is provided by the API today. The "standard mapping" > API provided today is rigid and limiting. However, I don't have time to > come up with a proposal right now - perhaps coming soon. > > +Katelyn as she expressed similar concerns. > > Thanks, > Chad