Re: Gamepad focus model (Re: Input and games.)
Daniel Stone daniel@... writes: I know I've had some trouble expressing my problems with the every-gamepad-in-a-seat proposal, but this pretty much hits the nail on the head. How does the gamepad's focus get assigned (and change) if it's in its own seat? Does it always follow the focus of another wl_seat? If so, why have we created another wl_seat? Is it only to contain a wl_gamepad and nothing else - if so, why are we using seats as an indirection layer? Or do some gamepads go into the seat with the pointer and keyboard and others go into their own seats - if so, isn't that totally arbitrary and more than a little confusing? Okay, so the most common configuration would be one seat with a mouse, keyboard, and gamepad. If you only ever play single player games, this is all you would see. When you have multiplayer games, you just want to be able to launch the game and have everybody automatically in it. You don't want your extra players to have to think about focusing the app or anything like that. That's the crux of the problem: how do you have one focus for multiple players? The many-gamepads-per-seat model solves this, but it has a couple problems: 1) Some compositors may put every gamepad in one seat, and some compositors may assign one gamepad per seat. The game programmer has to support both code paths. And unfortunately, when game programmers inevitably neglect to do that, the gamers are going to blame the compositors rather than the games. Wayland will get a reputation as a display server where you have to configure your gamepads differently for different games. 2) Gamepads aren't the only scenario where you want to have one focus for multiple players. Here's a couple scenarios that are complicated by breaking the seat == user model: A) You're playing split-screen Halo. Two users have a mouse/keyboard, and two users have gamepads. In the seat == user world, we know how this is set up: seat 1 has a mouse/keyboard, seat 2 has a mouse/keyboard, seat 3 has a gamepad, and seat 4 has a gamepad. In the many-gamepads-per-seat model, I don't know how to set this up. The first two players would have to have their own seats, but since the player ID is in the gamepad, where do we get their player numbers from? And what about players 3 and 4? Where do their gamepads go? They can go in seat 1, but then how do you distinguish player 1 from players 3 and 4? And why does player 2 have to manually focus the game? (I don't believe this scenario is inherently exotic. Lemmings, Settlers, and Hired Guns all did this back in the DOS and Amiga days, before OSs made it harder to distinguish mice from each other.) B) Imagine that, with the current trend of adding touchpads to gamepads, Fruit Ninja releases a multi-player version. All players play on the same field, but it tracks their scores independently; whomever slashes the fruit first gets the points. Let's imagine we have player 1 with a separate touchpad, and players 2 and 3 with gamepads that have touchpads built in. In the seat == user model, it's obvious how this should be set up: seat 1 gets a touchpad, seat 2 gets a gamepad with the associated touchpad, and so does seat 3. In the many-gamepads-per-seat model, the question rises again of where you put the gamepads. Do they go in the same seat as the touchpad they're attached to? Or do they all end up in seat 1? If they're all in seat 1, does the player indicator on their controller match up with their actual player number? And again, they all have to individually focus the game. If the focus can be changed independently, how does this happen? Focus can only be changed independently for collaborative seats. Collaborative mode means I want to interact with the desktop independently. When collaborative mode is turned off, your focus is derived from a seat which *is* in collaborative mode. If the keyboard/pointer seat switches focus to another game, do both seats switch, or does the other stay behind? Assuming seat 2 is following seat 1, they both switch. If both switch - why are we complicating the focus model rather than just adding both to a seat? Because this problem doesn't start and end with gamepads. If we take the multiple-gamepads-per-seat model to its logical conclusion, then to support the scenarios above, you would also need multiple mice/keyboards/touchpads per seat. And then how do you determine which devices belong to which players? The wl_seat == one player idea is a nice little mental model, but it's not in any way worth complicating our core input/focus management model for. I believe we're both trying to avoid complication :) -Rick- ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On Tue, 14 May 2013 04:49:39 + (UTC) Rick Yorgason r...@firefang.com wrote: Daniel Stone daniel@... writes: I know I've had some trouble expressing my problems with the every-gamepad-in-a-seat proposal, but this pretty much hits the nail on the head. How does the gamepad's focus get assigned (and change) if it's in its own seat? Does it always follow the focus of another wl_seat? If so, why have we created another wl_seat? Is it only to contain a wl_gamepad and nothing else - if so, why are we using seats as an indirection layer? Or do some gamepads go into the seat with the pointer and keyboard and others go into their own seats - if so, isn't that totally arbitrary and more than a little confusing? Okay, so the most common configuration would be one seat with a mouse, keyboard, and gamepad. If you only ever play single player games, this is all you would see. When you have multiplayer games, you just want to be able to launch the game and have everybody automatically in it. You don't want your extra players to have to think about focusing the app or anything like that. That's the crux of the problem: how do you have one focus for multiple players? The many-gamepads-per-seat model solves this, but it has a couple problems: 1) Some compositors may put every gamepad in one seat, and some compositors may assign one gamepad per seat. The game programmer has to support both code paths. And unfortunately, when game programmers inevitably neglect to do that, the gamers are going to blame the compositors rather than the games. Wayland will get a reputation as a display server where you have to configure your gamepads differently for different games. 2) Gamepads aren't the only scenario where you want to have one focus for multiple players. Here's a couple scenarios that are complicated by breaking the seat == user model: A) You're playing split-screen Halo. Two users have a mouse/keyboard, and two users have gamepads. In the seat == user world, we know how this is set up: seat 1 has a mouse/keyboard, seat 2 has a mouse/keyboard, seat 3 has a gamepad, and seat 4 has a gamepad. In the many-gamepads-per-seat model, I don't know how to set this up. The first two players would have to have their own seats, but since the player ID is in the gamepad, where do we get their player numbers from? And what about players 3 and 4? Where do their gamepads go? They can go in seat 1, but then how do you distinguish player 1 from players 3 and 4? And why does player 2 have to manually focus the game? (I don't believe this scenario is inherently exotic. Lemmings, Settlers, and Hired Guns all did this back in the DOS and Amiga days, before OSs made it harder to distinguish mice from each other.) B) Imagine that, with the current trend of adding touchpads to gamepads, Fruit Ninja releases a multi-player version. All players play on the same field, but it tracks their scores independently; whomever slashes the fruit first gets the points. Let's imagine we have player 1 with a separate touchpad, and players 2 and 3 with gamepads that have touchpads built in. In the seat == user model, it's obvious how this should be set up: seat 1 gets a touchpad, seat 2 gets a gamepad with the associated touchpad, and so does seat 3. In the many-gamepads-per-seat model, the question rises again of where you put the gamepads. Do they go in the same seat as the touchpad they're attached to? Or do they all end up in seat 1? If they're all in seat 1, does the player indicator on their controller match up with their actual player number? And again, they all have to individually focus the game. If the focus can be changed independently, how does this happen? Focus can only be changed independently for collaborative seats. Collaborative mode means I want to interact with the desktop independently. When collaborative mode is turned off, your focus is derived from a seat which *is* in collaborative mode. If the keyboard/pointer seat switches focus to another game, do both seats switch, or does the other stay behind? Assuming seat 2 is following seat 1, they both switch. If both switch - why are we complicating the focus model rather than just adding both to a seat? Because this problem doesn't start and end with gamepads. If we take the multiple-gamepads-per-seat model to its logical conclusion, then to support the scenarios above, you would also need multiple mice/keyboards/touchpads per seat. And then how do you determine which devices belong to which players? The wl_seat == one player idea is a nice little mental model, but it's not in any way worth complicating our core input/focus management model for. I believe we're both trying to avoid complication :) Rick, this makes sense to me, FWIW. Having proper concepts for describing how a server could assign foci is what was missing. The protocol is a separate thing
Re: Gamepad focus model (Re: Input and games.)
(Still digesting the rest of the thread.) On 14 May 2013 08:11, Peter Hutterer peter.hutte...@who-t.net wrote: the other thing that made me thing about your approach: with the gamepad_manager, you've got an extra layer in the tree for gamepads that pointers/keyboards don't have. that's not a bad thing and focus handling is easier this way, but now you could also look at pointers and keyboards and see if that extra layer is useful for those two. And I think it would be - e.g. wacom devices share their focus with the pointer, but also have their device-specific stuff that clients may need. The reason that's there is because Wayland is heavily object-driven. Rather than firing out event selections, you mostly register for events by binding to interfaces which send them. So, you need wl_gamepad_manager to support more than one gamepad per seat; you could have the seat send out new gamepad events directly, but then you don't have any way to unregister for those events - so we use wl_gamepad_manager, and when it's destroyed, you don't get new events from it. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Hi, On 14 May 2013 05:49, Rick Yorgason r...@firefang.com wrote: Okay, so the most common configuration would be one seat with a mouse, keyboard, and gamepad. If you only ever play single player games, this is all you would see. When you have multiplayer games, you just want to be able to launch the game and have everybody automatically in it. You don't want your extra players to have to think about focusing the app or anything like that. Yes, 100%. That's the crux of the problem: how do you have one focus for multiple players? The many-gamepads-per-seat model solves this, but it has a couple problems: 1) Some compositors may put every gamepad in one seat, and some compositors may assign one gamepad per seat. The game programmer has to support both code paths. And unfortunately, when game programmers inevitably neglect to do that, the gamers are going to blame the compositors rather than the games. Wayland will get a reputation as a display server where you have to configure your gamepads differently for different games. Compositors can implement maximise badly too, or multi-display support. But it's not an argument to push the complexity out. It's an argument for not running broken compositors. 2) Gamepads aren't the only scenario where you want to have one focus for multiple players. Here's a couple scenarios that are complicated by breaking the seat == user model: It's not seat == user, it's seat == focus. A) You're playing split-screen Halo. Two users have a mouse/keyboard, and two users have gamepads. In the seat == user world, we know how this is set up: seat 1 has a mouse/keyboard, seat 2 has a mouse/keyboard, seat 3 has a gamepad, and seat 4 has a gamepad. We know how it's set up, but we don't know how it works. How does the compositor know to focus 3 and 4 at Halo, rather than at the browser or whatever else is running at the time? You can't have the keyboard and mouse tell you, because they're different seats. So it's just manual then? Or the compositor making random guesses? In the many-gamepads-per-seat model, I don't know how to set this up. The first two players would have to have their own seats, but since the player ID is in the gamepad, where do we get their player numbers from? And what about players 3 and 4? Where do their gamepads go? They can go in seat 1, but then how do you distinguish player 1 from players 3 and 4? And why does player 2 have to manually focus the game? The gamepads go to whichever seat it is that's directing their focus. If their focus is entirely manually set, then they go in their own seat. Simple, no? Or am I totally missing something, and do gamepads under Wayland typically not follow keyboard/mouse focii? (I don't mean in really contrived theoretical cases, I mean in the overwhelming majority of situations.) (I don't believe this scenario is inherently exotic. Lemmings, Settlers, and Hired Guns all did this back in the DOS and Amiga days, before OSs made it harder to distinguish mice from each other.) Yes, that's the tradeoff you get for multitasking, and having the OS know where your mice are, rather than having to tell every single game where every single mouse is. B) Imagine that, with the current trend of adding touchpads to gamepads, Fruit Ninja releases a multi-player version. All players play on the same field, but it tracks their scores independently; whomever slashes the fruit first gets the points. Let's imagine we have player 1 with a separate touchpad, and players 2 and 3 with gamepads that have touchpads built in. In the seat == user model, it's obvious how this should be set up: seat 1 gets a touchpad, seat 2 gets a gamepad with the associated touchpad, and so does seat 3. In the many-gamepads-per-seat model, the question rises again of where you put the gamepads. Do they go in the same seat as the touchpad they're attached to? Or do they all end up in seat 1? If they're all in seat 1, does the player indicator on their controller match up with their actual player number? And again, they all have to individually focus the game. Again, how does the focus get changed? Where are the gamepads deriving their focus from? Put them in that seat, and add a wl_touch object for every gamepad. If the focus can be changed independently, how does this happen? Focus can only be changed independently for collaborative seats. Collaborative mode means I want to interact with the desktop independently. When collaborative mode is turned off, your focus is derived from a seat which *is* in collaborative mode. OK, so what's the downside from having the gamepad embedded inside the seat it derives its focus from? If the keyboard/pointer seat switches focus to another game, do both seats switch, or does the other stay behind? Assuming seat 2 is following seat 1, they both switch. If both switch - why are we complicating the focus model rather than just adding both to a seat?
Re: Gamepad focus model (Re: Input and games.)
On Tue, May 14, 2013 at 10:18 AM, Daniel Stone dan...@fooishbar.org wrote: Hi, On 14 May 2013 05:49, Rick Yorgason r...@firefang.com wrote: Okay, so the most common configuration would be one seat with a mouse, keyboard, and gamepad. If you only ever play single player games, this is all you would see. When you have multiplayer games, you just want to be able to launch the game and have everybody automatically in it. You don't want your extra players to have to think about focusing the app or anything like that. Yes, 100%. That's the crux of the problem: how do you have one focus for multiple players? The many-gamepads-per-seat model solves this, but it has a couple problems: 1) Some compositors may put every gamepad in one seat, and some compositors may assign one gamepad per seat. The game programmer has to support both code paths. And unfortunately, when game programmers inevitably neglect to do that, the gamers are going to blame the compositors rather than the games. Wayland will get a reputation as a display server where you have to configure your gamepads differently for different games. Compositors can implement maximise badly too, or multi-display support. But it's not an argument to push the complexity out. It's an argument for not running broken compositors. 2) Gamepads aren't the only scenario where you want to have one focus for multiple players. Here's a couple scenarios that are complicated by breaking the seat == user model: It's not seat == user, it's seat == focus. A) You're playing split-screen Halo. Two users have a mouse/keyboard, and two users have gamepads. In the seat == user world, we know how this is set up: seat 1 has a mouse/keyboard, seat 2 has a mouse/keyboard, seat 3 has a gamepad, and seat 4 has a gamepad. We know how it's set up, but we don't know how it works. How does the compositor know to focus 3 and 4 at Halo, rather than at the browser or whatever else is running at the time? You can't have the keyboard and mouse tell you, because they're different seats. So it's just manual then? Or the compositor making random guesses? In the many-gamepads-per-seat model, I don't know how to set this up. The first two players would have to have their own seats, but since the player ID is in the gamepad, where do we get their player numbers from? And what about players 3 and 4? Where do their gamepads go? They can go in seat 1, but then how do you distinguish player 1 from players 3 and 4? And why does player 2 have to manually focus the game? The gamepads go to whichever seat it is that's directing their focus. If their focus is entirely manually set, then they go in their own seat. Simple, no? Or am I totally missing something, and do gamepads under Wayland typically not follow keyboard/mouse focii? (I don't mean in really contrived theoretical cases, I mean in the overwhelming majority of situations.) (I don't believe this scenario is inherently exotic. Lemmings, Settlers, and Hired Guns all did this back in the DOS and Amiga days, before OSs made it harder to distinguish mice from each other.) Yes, that's the tradeoff you get for multitasking, and having the OS know where your mice are, rather than having to tell every single game where every single mouse is. B) Imagine that, with the current trend of adding touchpads to gamepads, Fruit Ninja releases a multi-player version. All players play on the same field, but it tracks their scores independently; whomever slashes the fruit first gets the points. Let's imagine we have player 1 with a separate touchpad, and players 2 and 3 with gamepads that have touchpads built in. In the seat == user model, it's obvious how this should be set up: seat 1 gets a touchpad, seat 2 gets a gamepad with the associated touchpad, and so does seat 3. In the many-gamepads-per-seat model, the question rises again of where you put the gamepads. Do they go in the same seat as the touchpad they're attached to? Or do they all end up in seat 1? If they're all in seat 1, does the player indicator on their controller match up with their actual player number? And again, they all have to individually focus the game. Again, how does the focus get changed? Where are the gamepads deriving their focus from? Put them in that seat, and add a wl_touch object for every gamepad. If the focus can be changed independently, how does this happen? Focus can only be changed independently for collaborative seats. Collaborative mode means I want to interact with the desktop independently. When collaborative mode is turned off, your focus is derived from a seat which *is* in collaborative mode. OK, so what's the downside from having the gamepad embedded inside the seat it derives its focus from? If the keyboard/pointer seat switches focus to another game, do both seats switch, or does the
Re: Gamepad focus model (Re: Input and games.)
Rick Yorgason wrote: When you have multiplayer games, you just want to be able to launch the game and have everybody automatically in it. You don't want your extra players to have to think about focusing the app or anything like that. That's the crux of the problem: how do you have one focus for multiple players? I feel the underlying function should be this: 1. The game can enumerate all the gamepads. 2. The game can grab the focus for any gamepads it wants. Once it is grabbed, the pointer motion makes NO difference, removing a lot of the questions people are asking here. If the device was moving the pointer, it stops moving the pointer. If another device like a mouse moves the pointer, it continues to do so, but it does not matter if the mouse is moved outside the game and clicked elsewhere: the mouse works like normal sending clicks to that other client, and the gamepad events continue going to the game. The only complexity is to prevent clients from grabbing the gamepad when they should not. For this the grab the focus does not work unless the shell allows it to. As I see it the gamepad is attached to at most one pointer. The gamepad can be grabbed if the client has that pointer's focus. The gamepad can also be attached to any number of other devices. It can also be grabbed if the event that triggered the grab request is a focus-in or event for a related device (this is so keyboard keys can cause a gamepad grab). Specifying this attachement is where the seat and hierarchy comes in. I very much agree that the seat==user idea is best. Otherwise it seems like seats are pretty useless, and users would have to be added to wayland anyway, making it more complex for no good reason. To get the one seat per gamepad but they all share the same pointer the parent-seat proposal seems like as easy fit. I am having a hard time coming up with a plausible setup that this would not cover. Grabbing a gamepad does not change this parenting in any way. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Hi Bill, In reply to everything below, I don't think there's anything in any of the proposals that prevents compositors from implementing pointer emulation for gamepads, or from suspending that emulation when a game has the gamepad's focus. -Rick- On 2013-05-14 15:08, Bill Spitzak wrote: I feel the underlying function should be this: 1. The game can enumerate all the gamepads. 2. The game can grab the focus for any gamepads it wants. Once it is grabbed, the pointer motion makes NO difference, removing a lot of the questions people are asking here. If the device was moving the pointer, it stops moving the pointer. If another device like a mouse moves the pointer, it continues to do so, but it does not matter if the mouse is moved outside the game and clicked elsewhere: the mouse works like normal sending clicks to that other client, and the gamepad events continue going to the game. The only complexity is to prevent clients from grabbing the gamepad when they should not. For this the grab the focus does not work unless the shell allows it to. As I see it the gamepad is attached to at most one pointer. The gamepad can be grabbed if the client has that pointer's focus. The gamepad can also be attached to any number of other devices. It can also be grabbed if the event that triggered the grab request is a focus-in or event for a related device (this is so keyboard keys can cause a gamepad grab). Specifying this attachement is where the seat and hierarchy comes in. I very much agree that the seat==user idea is best. Otherwise it seems like seats are pretty useless, and users would have to be added to wayland anyway, making it more complex for no good reason. To get the one seat per gamepad but they all share the same pointer the parent-seat proposal seems like as easy fit. I am having a hard time coming up with a plausible setup that this would not cover. Grabbing a gamepad does not change this parenting in any way. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On Mon, 13 May 2013 09:12:03 +1000 Peter Hutterer peter.hutte...@who-t.net wrote: On Fri, May 10, 2013 at 10:41:45AM +0300, Pekka Paalanen wrote: On Thu, 9 May 2013 16:44:09 +1000 Peter Hutterer peter.hutte...@who-t.net wrote: On Mon, May 06, 2013 at 03:36:20PM +0300, Pekka Paalanen wrote: [...] I had a private chat with Daniel, and we came to an understanding, which I try to describe below. The interface names below are more like placeholders for now. Into wl_seat, we should add a capability bit for gamepad. When the bit is set, a client can send wl_seat::get_gamepad_manager request, which creates a new wl_gamepad_manager object. (Do we actually need a capability bit?) A wl_gamepad_manager will send an event for each physical gamepad (as it dynamically appears, if hotplugged later) associated with this particular wl_seat, creating a wl_gamepad object for each. A wl_gamepad object will send an event about the player id as the first thing, and also if it later changes. do you expect the player ID to be exposed on the protocol? what does it contain and who manages the association? is it related to any physical ID of the gamepad, and if not, can such ID be provided (if the device supports it)? I imagined the player ID to be sent explicitly in the protocol, with its very own event. The server would manage the association between physical devices and the player ID, according to built-in heuristics and user preferences. E.g. if one goes from 0 gamepads to 1 gamepad, it will always be player 1; or maybe take the first player slot that does not have a gamepad, unless we identify the device as the same that was recently unplugged in which case take the player slot that it used to have. Or something, with user preferences, and some UI to fix the association in the rare case when the heuristics choose wrong. Recognizing a gamepad could be done via physical identification, if one is available, or by usb port/hub, or whatever platform specific way. The clients would not have to care, they would only work with unique player IDs, which are integers created in sequence starting from 1 (or 0). first - I agree with player ID being assigned to a gamepad for the simple reason of giving all clients a baseline. I think that phys information should be exposed in some way. use-case here is a specific controller that is always owned by the same human player. the game could then associate this automatically with the player. this could be a later addition to the protocol though, because right now I have no idea how to do this sensibly. Wouldn't that be better configured in the server, rather than in every game? But yeah, I don't even have any idea what should be exposed as physical info. Using integers starting from 1 makes it trivial for the server to update the player ID display on the gamepads, if they have one. The server could also implement a global UI to reassing gamepads. For instance, press the famous home button for 5 secs on any gamepad, get some gamepad controllable GUI to reassing them, or maybe just a request to press the home button again on each gamepad in sequence. this UI could also handle the seat assignment, see below. Doing all this client side would mean we need to find a way to communicate all the device and platform specific hints for the gamepad assignment heuristics somehow, and each client instance would have its own gamepad assignments. is the player ID not better represented through a wl_seat? - if you're playing a full-screen game, both seats can safely have the same focus, otherwise, players can play games independently. plus, if both players have a separate seat, they can use their keyboard independently for data entry. Those were exactly my thoughts in the beginning, however during the way long email thread, I got convinced otherwise for now. There are a few issues that come mind: - how to know, when one seat should follow the keyboard(?) focus of another seat? I wonder when this is necessary. unless I'm misunderstanding something this should only be of concern when both users want to change games (or otherwise between clients)? I would imagine, that two players each with own gamepads is a more common case when playing the same game on split-screen or alike, than playing two different games. Therefore, when the game is started, the display server would ideally somehow know to assing both gamepad foci to the game window. The basic assumption with seats is that each is independent, but the server is of course free to assign foci as it chooses. This is just a server implementation detail. But I guess it does not really matter how it is on the protocol level, there would need to be a focus group configuration in the server anyway; either by protocol allowing several gamepad in one
Re: Gamepad focus model (Re: Input and games.)
On Mon, May 13, 2013 at 2:33 AM, David Herrmann dh.herrm...@gmail.com wrote: That is why the kernel provides PHYS and UNIQ fields for every input device (they might be empty if not implemented, but at least they're supposed to be there..). PHYS provides the physical location for the device. UNIQ provides a unique identification. PHYS is potentially useful, but not reliable; there are often several USB ports near each other, and the player may not re-plug the device in the same one, especially if they are on the back of the machine. With wireless setups we can't be sure that PHYS will be identical between connections. Still, it's useful as a hint. UNIQ has been an empty string on every USB input device I've encountered. I need to get bluetooth set up (my PC has no bluetooth hardware, I need to buy a card or something), so i can't say whether wireless devices provide useful UNIQ values. Todd. -- Todd Showalter, President, Electron Jump Games, Inc. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Hi, On 13 May 2013 07:14, Pekka Paalanen ppaala...@gmail.com wrote: On Mon, 13 May 2013 09:12:03 +1000 Peter Hutterer peter.hutte...@who-t.net wrote: Those were exactly my thoughts in the beginning, however during the way long email thread, I got convinced otherwise for now. There are a few issues that come mind: - how to know, when one seat should follow the keyboard(?) focus of another seat? I wonder when this is necessary. unless I'm misunderstanding something this should only be of concern when both users want to change games (or otherwise between clients)? I would imagine, that two players each with own gamepads is a more common case when playing the same game on split-screen or alike, than playing two different games. Therefore, when the game is started, the display server would ideally somehow know to assing both gamepad foci to the game window. The basic assumption with seats is that each is independent, but the server is of course free to assign foci as it chooses. This is just a server implementation detail. But I guess it does not really matter how it is on the protocol level, there would need to be a focus group configuration in the server anyway; either by protocol allowing several gamepad in one wl_seat, or a purely server detail by focus groups containing wl_seats. I think this is not a protocol question as much as a server implementation consideration, so... *shrug*. I know I've had some trouble expressing my problems with the every-gamepad-in-a-seat proposal, but this pretty much hits the nail on the head. How does the gamepad's focus get assigned (and change) if it's in its own seat? Does it always follow the focus of another wl_seat? If so, why have we created another wl_seat? Is it only to contain a wl_gamepad and nothing else - if so, why are we using seats as an indirection layer? Or do some gamepads go into the seat with the pointer and keyboard and others go into their own seats - if so, isn't that totally arbitrary and more than a little confusing? If the focus can be changed independently, how does this happen? If the keyboard/pointer seat switches focus to another game, do both seats switch, or does the other stay behind? If both switch - why are we complicating the focus model rather than just adding both to a seat? The wl_seat == one player idea is a nice little mental model, but it's not in any way worth complicating our core input/focus management model for. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On Fri, May 10, 2013 at 10:41:45AM +0300, Pekka Paalanen wrote: On Thu, 9 May 2013 16:44:09 +1000 Peter Hutterer peter.hutte...@who-t.net wrote: On Mon, May 06, 2013 at 03:36:20PM +0300, Pekka Paalanen wrote: [...] I had a private chat with Daniel, and we came to an understanding, which I try to describe below. The interface names below are more like placeholders for now. Into wl_seat, we should add a capability bit for gamepad. When the bit is set, a client can send wl_seat::get_gamepad_manager request, which creates a new wl_gamepad_manager object. (Do we actually need a capability bit?) A wl_gamepad_manager will send an event for each physical gamepad (as it dynamically appears, if hotplugged later) associated with this particular wl_seat, creating a wl_gamepad object for each. A wl_gamepad object will send an event about the player id as the first thing, and also if it later changes. do you expect the player ID to be exposed on the protocol? what does it contain and who manages the association? is it related to any physical ID of the gamepad, and if not, can such ID be provided (if the device supports it)? I imagined the player ID to be sent explicitly in the protocol, with its very own event. The server would manage the association between physical devices and the player ID, according to built-in heuristics and user preferences. E.g. if one goes from 0 gamepads to 1 gamepad, it will always be player 1; or maybe take the first player slot that does not have a gamepad, unless we identify the device as the same that was recently unplugged in which case take the player slot that it used to have. Or something, with user preferences, and some UI to fix the association in the rare case when the heuristics choose wrong. Recognizing a gamepad could be done via physical identification, if one is available, or by usb port/hub, or whatever platform specific way. The clients would not have to care, they would only work with unique player IDs, which are integers created in sequence starting from 1 (or 0). first - I agree with player ID being assigned to a gamepad for the simple reason of giving all clients a baseline. I think that phys information should be exposed in some way. use-case here is a specific controller that is always owned by the same human player. the game could then associate this automatically with the player. this could be a later addition to the protocol though, because right now I have no idea how to do this sensibly. Using integers starting from 1 makes it trivial for the server to update the player ID display on the gamepads, if they have one. The server could also implement a global UI to reassing gamepads. For instance, press the famous home button for 5 secs on any gamepad, get some gamepad controllable GUI to reassing them, or maybe just a request to press the home button again on each gamepad in sequence. this UI could also handle the seat assignment, see below. Doing all this client side would mean we need to find a way to communicate all the device and platform specific hints for the gamepad assignment heuristics somehow, and each client instance would have its own gamepad assignments. is the player ID not better represented through a wl_seat? - if you're playing a full-screen game, both seats can safely have the same focus, otherwise, players can play games independently. plus, if both players have a separate seat, they can use their keyboard independently for data entry. Those were exactly my thoughts in the beginning, however during the way long email thread, I got convinced otherwise for now. There are a few issues that come mind: - how to know, when one seat should follow the keyboard(?) focus of another seat? I wonder when this is necessary. unless I'm misunderstanding something this should only be of concern when both users want to change games (or otherwise between clients)? - a client still needs to map seats into players; how? does it need an UI to fix it in case a user is not happy with the result? well, tbh if you're planning for multiple seats at some point you'll need a UI to handle it anyway. whether this is for ptr/kbd seats or gamepads, it comes down to the same. but even with the proposal above you still need some UI anyway. also, afaict this is what some game consoles have, e.g. the Wii associates your controller with your Me. Cheers, Peter Also, allowing multiple gamepads in one seat does not exclude the seat approach. A server could still assign every gamepad to a different seat, provided it had some way to solve the focus assignment. In any case, I think the player ID assigned to each gamepad by the server is useful, since it would solve the seat-player mapping, too. Although, in a strict one gamepad per seat setting it would be better suited as a seat event than a gamepad event, perhaps. Thanks, pq If a
Re: Gamepad focus model (Re: Input and games.)
On Thu, 9 May 2013 16:44:09 +1000 Peter Hutterer peter.hutte...@who-t.net wrote: On Mon, May 06, 2013 at 03:36:20PM +0300, Pekka Paalanen wrote: [...] I had a private chat with Daniel, and we came to an understanding, which I try to describe below. The interface names below are more like placeholders for now. Into wl_seat, we should add a capability bit for gamepad. When the bit is set, a client can send wl_seat::get_gamepad_manager request, which creates a new wl_gamepad_manager object. (Do we actually need a capability bit?) A wl_gamepad_manager will send an event for each physical gamepad (as it dynamically appears, if hotplugged later) associated with this particular wl_seat, creating a wl_gamepad object for each. A wl_gamepad object will send an event about the player id as the first thing, and also if it later changes. do you expect the player ID to be exposed on the protocol? what does it contain and who manages the association? is it related to any physical ID of the gamepad, and if not, can such ID be provided (if the device supports it)? I imagined the player ID to be sent explicitly in the protocol, with its very own event. The server would manage the association between physical devices and the player ID, according to built-in heuristics and user preferences. E.g. if one goes from 0 gamepads to 1 gamepad, it will always be player 1; or maybe take the first player slot that does not have a gamepad, unless we identify the device as the same that was recently unplugged in which case take the player slot that it used to have. Or something, with user preferences, and some UI to fix the association in the rare case when the heuristics choose wrong. Recognizing a gamepad could be done via physical identification, if one is available, or by usb port/hub, or whatever platform specific way. The clients would not have to care, they would only work with unique player IDs, which are integers created in sequence starting from 1 (or 0). Using integers starting from 1 makes it trivial for the server to update the player ID display on the gamepads, if they have one. The server could also implement a global UI to reassing gamepads. For instance, press the famous home button for 5 secs on any gamepad, get some gamepad controllable GUI to reassing them, or maybe just a request to press the home button again on each gamepad in sequence. Doing all this client side would mean we need to find a way to communicate all the device and platform specific hints for the gamepad assignment heuristics somehow, and each client instance would have its own gamepad assignments. is the player ID not better represented through a wl_seat? - if you're playing a full-screen game, both seats can safely have the same focus, otherwise, players can play games independently. plus, if both players have a separate seat, they can use their keyboard independently for data entry. Those were exactly my thoughts in the beginning, however during the way long email thread, I got convinced otherwise for now. There are a few issues that come mind: - how to know, when one seat should follow the keyboard(?) focus of another seat? - a client still needs to map seats into players; how? does it need an UI to fix it in case a user is not happy with the result? Also, allowing multiple gamepads in one seat does not exclude the seat approach. A server could still assign every gamepad to a different seat, provided it had some way to solve the focus assignment. In any case, I think the player ID assigned to each gamepad by the server is useful, since it would solve the seat-player mapping, too. Although, in a strict one gamepad per seat setting it would be better suited as a seat event than a gamepad event, perhaps. Thanks, pq If a gamepad is hot-unplugged, a wl_gamepad event will notify about that, and the wl_gamepad object becomes inert (does not send any events, ignores all but the destroy request). Associating physical devices into wl_seats is left as a server detail. The only rule is, that a physical device can appear at most in one wl_seat at a time. The server is free to move e.g. gamepads from one seat to another at runtime, clients will just see one gamepad hot-unplugged, and one hotplugged later under a different seat. Gamepad input events are delivered according to the keyboard focus of the related wl_seat. If there is no keyboard to focus, then use the pointer focus, or something. It doesn't really affect the protocol design how the focus is assigned. However, would we need a wl_gamepad::enter,leave events? Probably, along with events for initial state. Or maybe enter/leave should be wl_gamepad_manager events? This design allows several gamepads associated with one wl_seat, and thus one focus. It also allows gamepads to be assigned to different seats, but then we will have more problems on managing the foci, not unlike with keyboards.
Re: Gamepad focus model (Re: Input and games.)
On Thu, 9 May 2013 10:49:03 + (UTC) Rick Yorgason r...@firefang.com wrote: Pekka Paalanen ppaalanen@... writes: From the game's point of view, it will need to iterate over all wl_seats. For each seat with the gamepad capability bit set, create a wl_gamepad_manager, receive all wl_gamepad objects, and for each wl_gamepad receive the player id. Create your surfaces, wait for foci to arrive, and fire away. So, the purpose of wl_gamepad_manager is to allow multiple gamepads to share one application focus. That seemed weird to me (why is it only gamepads that would want that ability?) which is why I started thinking about wl_seat parenting, and I apologize if that caused more heat than light. But now I'm seriously wondering, does the compositor really need *any* protocol support to handle this case? I think we've been assuming that each seat will get its own free reign over the desktop, but isn't the compositor free to *not* do that? And instead to have some seats which only interact with the focused application? We could have one wl_gamepad per wl_seat, just like wl_pointer, wl_keyboard, or wl_touch, put the player ID in wl_seat instead of wl_gamepad, and give compositor writers a few suggestions: * Not all seats need to have a desktop pointer or the ability to control their own focus. It can be useful to allow second class seats which follow the focus of another seat, in which case neither the pointer nor any other event from that seat should be allowed outside of the focused application. * Each gamepad should automatically be put in a separate seat, either by putting it in an existing seat without a gamepad, or automatically creating a new seat. * If a seat is automatically created for a gamepad, it should ideally be of the second class type by default. Users can always reconfigure their seat if they want control over the desktop. Wouldn't that allow everything we want? It allows every user to have a full set of devices, and users don't have to worry about focus issues unless they want to. It also means the protocol doesn't need the contrivances of wl_gamepad_manager or seat parenting. Yes. If I recall correctly, I originally started with a similar scheme, but Daniel did not like it. Daniel? Thanks, pq ps. please don't cut the cc list unless explicitly requested. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Hi, On 9 May 2013 11:49, Rick Yorgason r...@firefang.com wrote: But now I'm seriously wondering, does the compositor really need *any* protocol support to handle this case? I think we've been assuming that each seat will get its own free reign over the desktop, but isn't the compositor free to *not* do that? And instead to have some seats which only interact with the focused application? Yes, the mechanics of how focus is set and transferred are unspecified, thankfully. We could have one wl_gamepad per wl_seat, just like wl_pointer, wl_keyboard, or wl_touch, put the player ID in wl_seat instead of wl_gamepad, and give compositor writers a few suggestions: * Not all seats need to have a desktop pointer or the ability to control their own focus. Seats without wl_pointer already don't have a desktop pointer, and seats don't necessarily control their own focus. What happens when the compositor transfers focus is very well-defined. But nowhere in the protocol does it specify when focus is transferred, because without a view of the full window tree, clients don't have enough context to know what would happen anyway. So this is a moot point. It can be useful to allow second class seats which follow the focus of another seat, in which case neither the pointer nor any other event from that seat should be allowed outside of the focused application. That's not the point of a seat right now. But this is totally doable without any compositor changes if you want, and the client doesn't ever need to know about it. * Each gamepad should automatically be put in a separate seat, either by putting it in an existing seat without a gamepad, or automatically creating a new seat. Why put it in a seat, then? If it's not going to go in with a keyboard, mouse or touch device, don't bother with the seats, just keep it as a separate object. The purpose of seats was to aggregate and relate input devices. If all you're doing with wl_seat is using it as a shim to carry one (_exactly_ one) object, why bother? * If a seat is automatically created for a gamepad, it should ideally be of the second class type by default. Users can always reconfigure their seat if they want control over the desktop. Again, doesn't require protocol changes. Wouldn't that allow everything we want? It allows every user to have a full set of devices, and users don't have to worry about focus issues unless they want to. It also means the protocol doesn't need the contrivances of wl_gamepad_manager or seat parenting. I don't see what it brings us, if I'm honest. I guess it all hinges on how the focus model is implemented, and to what extent we want to express that through the protocol. *shrug* Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Pekka Paalanen ppaalanen@... writes: Also, allowing multiple gamepads in one seat does not exclude the seat approach. A server could still assign every gamepad to a different seat, provided it had some way to solve the focus assignment. Unfortunately, that comes at the expense of requiring clients to support two code paths. For better or worse, a lot of game developers prefer to write close to the metal, so if their compositor of choice prefers one method, there's a very real chance that the game developer will forget to support the other method. is the player ID not better represented through a wl_seat? - if you're playing a full-screen game, both seats can safely have the same focus, otherwise, players can play games independently. plus, if both players have a separate seat, they can use their keyboard independently for data entry. Those were exactly my thoughts in the beginning, however during the way long email thread, I got convinced otherwise for now. There are a few issues that come mind: - how to know, when one seat should follow the keyboard(?) focus of another seat? - a client still needs to map seats into players; how? does it need an UI to fix it in case a user is not happy with the result? You would definitely need a UI for corner cases and advanced users, but I think you could design a heuristic that's good enough for games most of the time. Here's an example heuristic: A) Whenever a keyboard or mouse is plugged into the system, it gets added to the first seat. B) Whenever a gamepad is plugged into the system, it gets added to the first seat without a gamepad. If all the seats already have a gamepad assigned, a new seat is automatically created. C) If a user explicitly creates a seat, that seat gets full access to the desktop by default, including their own desktop-level cursor. (I call this collaboration below.) D) If the compositor has to automatically create a new seat to deal with a new gamepad, that seat has collaboration turned off, and follows the first seat's focus. E) If a keyboard, mouse, or touch screen is attached or plugged into the gamepad, it goes in the same seat as the gamepad. This does not override rule D. F) There must always be at least one seat. G) At least one seat must be in collaborative mode. I've put together an example scenario below. Excuse the ASCII art :) -Rick- 1) The users plug two keyboards and two mice into the computer. They all get assigned to seat 1. 2) The users open a seat configuration dialogue. They see this: Seat 1 [Configure] [Renumber] [Add a new seat] 3) They click Configure. They see this: Seat 1 -- Choose the devices you want in this seat: [X] Keyboard 1 [X] Keyboard 2 [X] Mouse 1 [X] Mouse 2 [X] This seat can collaboratively share the desktop (disabled) Derive focus from [N/A] (disabled) 4) They go back and click Add a new seat. They see this: Seat 2 -- Choose the devices you want in this seat: [ ] Keyboard 1 [ ] Keyboard 2 [ ] Mouse 1 [ ] Mouse 2 [X] This seat can collaboratively share the desktop (disabled) Derive focus from [Seat 1] 5) They reconfigure it like so: Seat 2 -- Choose the devices you want in this seat: [ ] Keyboard 1 [X] Keyboard 2 [ ] Mouse 1 [X] Mouse 2 [X] This seat can collaboratively share the desktop Derive focus from [N/A] (disabled) 6) This automatically removes Keyboard 2 and Mouse 2 from Seat 1. It also un-disables the collaborative line. If they go back to Seat 1's configuration screen, they'll see this: Seat 1 -- Choose the devices you want in this seat: [X] Keyboard 1 [ ] Keyboard 2 [X] Mouse 1 [ ] Mouse 2 [X] This seat can collaboratively share the desktop Derive focus from [N/A] (disabled) 7) They plug a gamepad into the computer. It automatically gets assigned to seat 1: Seat 1 -- Choose the devices you want in this seat: [X] Keyboard 1 [ ] Keyboard 2 [X] Mouse 1 [ ] Mouse 2 [X] Gamepad 1 [X] This seat can collaboratively share the desktop Derive focus from [N/A] (disabled) 8) They plug another gamepad into the computer. It automatically gets assigned to seat 2: Seat 2 -- Choose the devices you want in this seat: [ ] Keyboard 1 [X] Keyboard 2 [ ] Mouse 1 [X] Mouse 2 [ ] Gamepad 1 [X] Gamepad 2 [X] This seat can collaboratively share the desktop Derive focus from [N/A] (disabled) 9) They plug a third gamepad into the computer. The compositor automatically creates a seat with collaboration turned off. The seat list now looks like this: Seat 1 [Configure] [Renumber] [Delete] Seat 2 [Configure] [Renumber] [Delete] Seat 3 [Configure] [Renumber] [Delete] [Add a new seat] 10) They configure seat 3, and see this: Seat 3 -- Choose the devices you want in this seat: [ ] Keyboard 1 [ ] Keyboard 2 [ ] Mouse 1 [ ] Mouse 2 [ ] Gamepad 1 [ ] Gamepad 2 [X] Gamepad 3 [ ] This seat can collaboratively share the desktop Derive focus from [Seat 1] 11) The users decide that player 3 should follow player
Re: Gamepad focus model (Re: Input and games.)
Daniel Stone daniel@... writes: Why put it in a seat, then? If it's not going to go in with a keyboard, mouse or touch device, don't bother with the seats, just keep it as a separate object. The purpose of seats was to aggregate and relate input devices. If all you're doing with wl_seat is using it as a shim to carry one (_exactly_ one) object, why bother? For the same reason you would put a keyboard in a seat by itself: just because it's the only device in that seat right now, doesn't mean it will always be the only device in its seat. Hopefully my last email with the scenario makes this more clear. Keep in mind that both the OUYA and PS4 controllers are coming with touch pads built in. Seats would be a perfect way of grouping the two devices together. Also, while we're talking about aggregation, earlier in the thread we were assuming that it never makes sense to aggregate controllers, but now I think it's safer to say that it *rarely* makes sense to aggregate controllers. Consider a scenario where a user has a really nice wireless controller, but they also have a crappy controller that's plugged directly into their computer which they only use when they run out of batteries. This is advanced user territory, but it's conceivable to think that the user would want to permanently assign both gamepads to the same seat. The only tricky part about aggregating gamepads is dealing with the axes, but all you really have to do is add each axis component together and ensure that the component doesn't go beyond the max size. -Rick- ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On Tue, May 07, 2013 at 11:14:08AM -0400, Todd Showalter wrote: On Tue, May 7, 2013 at 3:23 AM, Pekka Paalanen ppaala...@gmail.com wrote: Yeah, like Daniel said, there is no concept of a return value. When a client creates a new object, the server can only either agree, or disconnect the client with a protocol error. Any other behaviour requires specialized handling, and causes a roundtrip, where the client must either wait for a reply before continuing, or risk having further requests ignored without any obvious way to know what got ignored in the end. Both cases are unacceptable. Ok. I was assuming that cases where you had fundamental capability change in the server (ie: input devices appearing or disappearing) were rare and special enough to warrant a round trip. When a client sends a request, that creates a new protocol object, then from the client's point of view, the object is created on that instant, before the request has even been submitted to the wire. This allows the client to immediately send more requests on that new object, without waiting for a roundtrip in between. The same works also in the reverse direction, when the server creates protocol objects by sending events. A major design principle in Wayland is to minimize roundtrips, as it leads to better performance and lower overhead. Fair enough. We're talking about rare events here, so I wouldn't have called it essential, but if that's an organizing principle of the project then so be it. It's not about the gamepad capabilities at all. It's just an assignment, configured in the server: this input device belongs to player N. The place where that becomes a problem is with controller batteries. As an example, I've got a PS3, and my wife uses it to watch netflix (it's a streaming tv/movie service, for those who haven't heard of it). It uses the PS3 controller as a remote, to do things like play/pause. It's not uncommon for the battery in the controller to run flat while she's watching. I've got a second controller, and we typically charge one while the other is in use, but fairly often the controller she's using runs flat. When that happens, we have a second charged controller, but to use it we have to reboot the PS3, because without rebooting it connects as Player 2, and netflix only listens to Player 1. As far as I know there's no simple way to tell the gamepad to reconnect as Player 1, short of rebooting the machine and rerunning all the controller handshaking. When a gamepad goes away and then it reappears or another appears, it's *probably* the same player. So what I'm thinking is that it makes more sense to have the wl_gamepad go into a disconnected state, and then reactivate when the next gamepad appears, rather than creating a new wl_gamepad. in that case you'd be prone to having any number of gamepads in a disconnected state until the server dies (unless you add timeouts, etc). tbh, I see the above as an application bug. Netflix is not a multiplayer game, so it should handle a new gamepad gracefully (especially when the first one re-appears). and games should too and give you the option of re-configuring to match the target. If the gamepad later comes back online, it is like it was hotplugged again: a new wl_gamepad object is sent, with the same player id as before. This would work too. The main thing is dealing well with the single player case where the player is replacing a gamepad. This could be because: - they wandered out of RF range when they were getting a drink - they want to play the game with a different gamepad - the gamepad they were using ran out of power and is now plugged in via usb - the gamepad they were using ran out of power and is being replaced with a charged gamepad - someone tripped over the usb cord and yanked it out and then plugged it back in all of these should be handled in the client, IMO. the only thing wayland should attempt is keep the identification stable if the gamepad provides some physical ability to do so (i.e. so you can recognise the same gamepad coming back). Cheers, Peter Yeah, the main point of the leave event is to say you don't get any more input events from this device, until it comes back, and it also implies that the client should forget all temporary state of the gamepad, like which buttons were down. Yes. Immediately following an enter event, or in the enter event, a new set of current state is sent. Notice, that this should not be done by sending e.g. fake button-down events. We have a protocol design policy, that input events from user actions are never manufactured. My temptation would actually be to say that when focus goes to a new application, we treat buttons that are down as if they were up; don't send a release when they are lifted. So, if I'm holding down SELECT when focus enters the client
Re: Gamepad focus model (Re: Input and games.)
On Mon, May 06, 2013 at 03:36:20PM +0300, Pekka Paalanen wrote: [...] I had a private chat with Daniel, and we came to an understanding, which I try to describe below. The interface names below are more like placeholders for now. Into wl_seat, we should add a capability bit for gamepad. When the bit is set, a client can send wl_seat::get_gamepad_manager request, which creates a new wl_gamepad_manager object. (Do we actually need a capability bit?) A wl_gamepad_manager will send an event for each physical gamepad (as it dynamically appears, if hotplugged later) associated with this particular wl_seat, creating a wl_gamepad object for each. A wl_gamepad object will send an event about the player id as the first thing, and also if it later changes. do you expect the player ID to be exposed on the protocol? what does it contain and who manages the association? is it related to any physical ID of the gamepad, and if not, can such ID be provided (if the device supports it)? is the player ID not better represented through a wl_seat? - if you're playing a full-screen game, both seats can safely have the same focus, otherwise, players can play games independently. plus, if both players have a separate seat, they can use their keyboard independently for data entry. Cheers, Peter If a gamepad is hot-unplugged, a wl_gamepad event will notify about that, and the wl_gamepad object becomes inert (does not send any events, ignores all but the destroy request). Associating physical devices into wl_seats is left as a server detail. The only rule is, that a physical device can appear at most in one wl_seat at a time. The server is free to move e.g. gamepads from one seat to another at runtime, clients will just see one gamepad hot-unplugged, and one hotplugged later under a different seat. Gamepad input events are delivered according to the keyboard focus of the related wl_seat. If there is no keyboard to focus, then use the pointer focus, or something. It doesn't really affect the protocol design how the focus is assigned. However, would we need a wl_gamepad::enter,leave events? Probably, along with events for initial state. Or maybe enter/leave should be wl_gamepad_manager events? This design allows several gamepads associated with one wl_seat, and thus one focus. It also allows gamepads to be assigned to different seats, but then we will have more problems on managing the foci, not unlike with keyboards. Hopefully there are no protocol design implications, though. From the game's point of view, it will need to iterate over all wl_seats. For each seat with the gamepad capability bit set, create a wl_gamepad_manager, receive all wl_gamepad objects, and for each wl_gamepad receive the player id. Create your surfaces, wait for foci to arrive, and fire away. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On Wed, May 08, 2013 at 11:46:43AM +, Rick Yorgason wrote: Rick Yorgason rick@... writes: Having the two controllers paired doesn't solve 3. There are a lot of UI problems with having two pointers running around the screen that share a focus. Let's say they're one of those crazy users that like sloppy- focus. What happens when the two cursors are on different windows? Does the primary cursor over-ride? Does the secondary cursor work in windows that aren't focused? Even if you have click-to-focus, you still have problems with the two cursors fighting. What happens if they both try to do a drag-and-drop? The only solution to this is probably to make the secondary pointer inert in windows that aren't focussed. That said, I'm not sure if that 100% fixes it either. I was thinking that a desktop environment would probably only have one cursor. In other words, if any wl_seats are sharing focus, most apps would treat them as aggregate, and it would only be more specialized apps (like games) that might want to separate them. In thinking more about this some more, I don't even think these seats need to be aggregated. A second-class wl_seat would just mean This seat is intended to be used at the application level rather than the compositor level, and it will send enter/leave events to the same surfaces as its parent. In other words, if you move the mouse attached to a second-class wl_seat, the compositor completely ignores that motion. Now, when you read what I wrote about how enter/leave events get propagated to second-class wl_seats, you probably thought Wait, enter/leave events are produced by wl_pointer, wl_keyboard, and maybe wl_gamepad. Are you suggesting *all* of those get propagated? Well, I don't think it makes sense to propagate wl_pointer::enter/leave, and this leads me to what I believe is an unfortunate design choice in the current protocol, which is that wl_keyboard::enter/leave doesn't make much sense. It should really be wl_seat::enter/leave. wl_pointer::enter/leave makes sense because pointers are special. They can freely move outside of surfaces, and can affect things in unfocused applications. But the keyboard focus is pretty much identical to the application focus: in a typical compositor, when an application gets keyboard focus, that application's title bar will light up. Currently it would make perfect sense for wl_gamepad to use wl_keyboard's focus except... what do you do if there is no keyboard? This would have been an easier problem to solve if we could just say that wl_keyboard and wl_gamepad both use wl_seat's focus. if you have more than one application open, you may want the keyboard focus in a different window than gamepad focus. think of 'taking notes while playing a game' from what I understand nothing currently stops the compositor from setting the gamepad focus to the same surface as the keyboard focus. encoding that in the protocol however is not a good idea. Cheers, Peter ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Pekka Paalanen ppaalanen@... writes: From the game's point of view, it will need to iterate over all wl_seats. For each seat with the gamepad capability bit set, create a wl_gamepad_manager, receive all wl_gamepad objects, and for each wl_gamepad receive the player id. Create your surfaces, wait for foci to arrive, and fire away. So, the purpose of wl_gamepad_manager is to allow multiple gamepads to share one application focus. That seemed weird to me (why is it only gamepads that would want that ability?) which is why I started thinking about wl_seat parenting, and I apologize if that caused more heat than light. But now I'm seriously wondering, does the compositor really need *any* protocol support to handle this case? I think we've been assuming that each seat will get its own free reign over the desktop, but isn't the compositor free to *not* do that? And instead to have some seats which only interact with the focused application? We could have one wl_gamepad per wl_seat, just like wl_pointer, wl_keyboard, or wl_touch, put the player ID in wl_seat instead of wl_gamepad, and give compositor writers a few suggestions: * Not all seats need to have a desktop pointer or the ability to control their own focus. It can be useful to allow second class seats which follow the focus of another seat, in which case neither the pointer nor any other event from that seat should be allowed outside of the focused application. * Each gamepad should automatically be put in a separate seat, either by putting it in an existing seat without a gamepad, or automatically creating a new seat. * If a seat is automatically created for a gamepad, it should ideally be of the second class type by default. Users can always reconfigure their seat if they want control over the desktop. Wouldn't that allow everything we want? It allows every user to have a full set of devices, and users don't have to worry about focus issues unless they want to. It also means the protocol doesn't need the contrivances of wl_gamepad_manager or seat parenting. -Rick- ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On Thu, May 9, 2013 at 5:49 AM, Rick Yorgason r...@firefang.com wrote: Pekka Paalanen ppaalanen@... writes: From the game's point of view, it will need to iterate over all wl_seats. For each seat with the gamepad capability bit set, create a wl_gamepad_manager, receive all wl_gamepad objects, and for each wl_gamepad receive the player id. Create your surfaces, wait for foci to arrive, and fire away. So, the purpose of wl_gamepad_manager is to allow multiple gamepads to share one application focus. That seemed weird to me (why is it only gamepads that would want that ability?) which is why I started thinking about wl_seat parenting, and I apologize if that caused more heat than light. But now I'm seriously wondering, does the compositor really need *any* protocol support to handle this case? I think we've been assuming that each seat will get its own free reign over the desktop, but isn't the compositor free to *not* do that? And instead to have some seats which only interact with the focused application? We could have one wl_gamepad per wl_seat, just like wl_pointer, wl_keyboard, or wl_touch, put the player ID in wl_seat instead of wl_gamepad, and give compositor writers a few suggestions: * Not all seats need to have a desktop pointer or the ability to control their own focus. It can be useful to allow second class seats which follow the focus of another seat, in which case neither the pointer nor any other event from that seat should be allowed outside of the focused application. * Each gamepad should automatically be put in a separate seat, either by putting it in an existing seat without a gamepad, or automatically creating a new seat. * If a seat is automatically created for a gamepad, it should ideally be of the second class type by default. Users can always reconfigure their seat if they want control over the desktop. Wouldn't that allow everything we want? It allows every user to have a full set of devices, and users don't have to worry about focus issues unless they want to. It also means the protocol doesn't need the contrivances of wl_gamepad_manager or seat parenting. The more I think about this solution, the more I like it. You're right that clients have no reason to know why a seat gets focus, simply that it does. Also, it's worth remembering that whatever GUI we present to the user doesn't have to directly correspond to the protocol. From the user's perspective, this would probably look like picking which pointing devices actually get assigned to a pointer, and which seats correspond to which pointer. --Jason Ekstrand ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Rick Yorgason rick@... writes: Having the two controllers paired doesn't solve 3. There are a lot of UI problems with having two pointers running around the screen that share a focus. Let's say they're one of those crazy users that like sloppy- focus. What happens when the two cursors are on different windows? Does the primary cursor over-ride? Does the secondary cursor work in windows that aren't focused? Even if you have click-to-focus, you still have problems with the two cursors fighting. What happens if they both try to do a drag-and-drop? The only solution to this is probably to make the secondary pointer inert in windows that aren't focussed. That said, I'm not sure if that 100% fixes it either. I was thinking that a desktop environment would probably only have one cursor. In other words, if any wl_seats are sharing focus, most apps would treat them as aggregate, and it would only be more specialized apps (like games) that might want to separate them. In thinking more about this some more, I don't even think these seats need to be aggregated. A second-class wl_seat would just mean This seat is intended to be used at the application level rather than the compositor level, and it will send enter/leave events to the same surfaces as its parent. In other words, if you move the mouse attached to a second-class wl_seat, the compositor completely ignores that motion. Now, when you read what I wrote about how enter/leave events get propagated to second-class wl_seats, you probably thought Wait, enter/leave events are produced by wl_pointer, wl_keyboard, and maybe wl_gamepad. Are you suggesting *all* of those get propagated? Well, I don't think it makes sense to propagate wl_pointer::enter/leave, and this leads me to what I believe is an unfortunate design choice in the current protocol, which is that wl_keyboard::enter/leave doesn't make much sense. It should really be wl_seat::enter/leave. wl_pointer::enter/leave makes sense because pointers are special. They can freely move outside of surfaces, and can affect things in unfocused applications. But the keyboard focus is pretty much identical to the application focus: in a typical compositor, when an application gets keyboard focus, that application's title bar will light up. Currently it would make perfect sense for wl_gamepad to use wl_keyboard's focus except... what do you do if there is no keyboard? This would have been an easier problem to solve if we could just say that wl_keyboard and wl_gamepad both use wl_seat's focus. -Rick- ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Rick Yorgason rick@... writes: In thinking more about this some more, I don't even think these seats need to be aggregated. A second-class wl_seat would just mean This seat is intended to be used at the application level rather than the compositor level, and it will send enter/leave events to the same surfaces as its parent. Once more in spec format, to be more explicit: New requests provided by wl_seat: wl_seat::get_gamepad - return gamepad object The ID provided will be initialized to the wl_pointer interface for this seat. This request only takes effect if the seat has the gamepad capability. Arguments: id -- new_id New events provided by wl_seat: wl_seat::secondclass - Seat has become second-class Second-class seats are intended for client consumption rather than server consumption. A second-class seat is always parented to a first-class seat and follows its parent's focus. A second-class seat's pointer is trapped by the focused surface. Arguments: parent -- object -- The parent seat. wl_seat::firstclass - Seat has become first-class First-class seats are the default scenario. Multiple first-class seats can be used to allow different users to control different applications on the same desktop. No arguments. wl_seat::index - Seat has a new friendly identifier The index is intended to be a user-friendly identifier to differentiate seats. It starts from 1 and increments or reuses indices as necessary. Devices such as gamepads with player indicators are expected to display this value when possible. Arguments: index -- uint -- The new friendly identifier wl_seat::enter - enter event Notification that this seat's focus is on a certain surface. Arguments: serial -- uint surface -- object wl_seat::leave - leave event Notification that the seat's focus is no longer on a certain surface. The leave notification is sent before the enter notification for the new focus. Arguments: serial -- uint surface -- object --- New wl_seat::capability values: gamepad Value: 8 This seat has a gamepad ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Rick Yorgason wrote: Currently it would make perfect sense for wl_gamepad to use wl_keyboard's focus except... what do you do if there is no keyboard? This would have been an easier problem to solve if we could just say that wl_keyboard and wl_gamepad both use wl_seat's focus. If the gamepad can move the pointer then I think the buttons should go to the pointer focus. Otherwise I agree with the idea that the keyboard focus should be the seat focus. All devices not moving the pointer should go to the seat focus. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Bill Spitzak spitzak@... writes: If the gamepad can move the pointer then I think the buttons should go to the pointer focus. I think if the gamepad is emulating a pointer, it should go all the way and send wl_pointer events. I believe this is perfectly workable with the proposed design. The compositor could simply use the gamepad to emulate a mouse and aggregate the results into wl_pointer. If an active surface has created a wl_gamepad object, it suspends aggregation for that device. Whether or not you'll like the results is another matter. A joystick is a terrible way of controlling a cursor. Although I have used such a setup to control the next track button in a music player on a computer across the room, only because I didn't have a wireless mouse on hand. -Rick- ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Martin Minarik minarik11@... writes: Most common scenario would be: A: joypad 1 on mother seat with keyboard and pointer B: joypad 2 on child seat Let's say: - user plugs another keyboard and pointer, udev assigns it to B: - weston promotes B: to mother seat The situation is as follows: A: joypad 1 on mother seat with keyboard and pointer B: joypad 2 on mother seat with keyboard and pointer The seats are now independent and can control the UI, user 2 can now quit the focus on the fly. But the question is.. Is this the expected? Let's say there is a full screen application and the user 2 presses alt+tab, now we have a surface stacking/ordering race. To answer your question, yes, that is the expected behaviour. The original purpose for wl_seat was to allow multiple users to have their own cursors, their own window focus, and to share a desktop collaboratively. You've described that scenario precisely. Also, when you said weston promotes B: to mother seat, were you implying that this is an automatic response to the keyboard being assigned to B? Because there's no reason for that to be the case. B could happily remain as a child seat, and would simply have no way to change window focus. This would be a perfectly sane thing to do if, for instance, you plugged a keyboard into your gamepad. I suspect most compositors would only create mother seats if the user explicitly sets them up. If you plug a second keyboard into your computer, there's no way for the compositor to guess whether you want it aggregated with the first keyboard, or whether you want a second user, and aggregating is the status quo right now. -Rick- ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Hi Todd, Daniel nicely replied to the most important comments, here are a few more. On Mon, 6 May 2013 09:48:47 -0400 Todd Showalter t...@electronjump.com wrote: On Mon, May 6, 2013 at 8:36 AM, Pekka Paalanen ppaala...@gmail.com wrote: Into wl_seat, we should add a capability bit for gamepad. When the bit is set, a client can send wl_seat::get_gamepad_manager request, which creates a new wl_gamepad_manager object. (Do we actually need a capability bit?) There are options here: - have the capability bit, if the bit is set the client can request a manager -- has to deal with the case where the client sent the request but the caps bit wasn't set, presumably by returning NULL or -1 the protocol equivalent - leave out the caps bit, client requests the manager if they want it, they get NULL equivalent if there are no gamepads - leave out the caps bit, gampad manager is always there, but can be expected to return 0 if asked to enumerate gamepads when none are connected Yeah, like Daniel said, there is no concept of a return value. When a client creates a new object, the server can only either agree, or disconnect the client with a protocol error. Any other behaviour requires specialized handling, and causes a roundtrip, where the client must either wait for a reply before continuing, or risk having further requests ignored without any obvious way to know what got ignored in the end. Both cases are unacceptable. When a client sends a request, that creates a new protocol object, then from the client's point of view, the object is created on that instant, before the request has even been submitted to the wire. This allows the client to immediately send more requests on that new object, without waiting for a roundtrip in between. The same works also in the reverse direction, when the server creates protocol objects by sending events. A major design principle in Wayland is to minimize roundtrips, as it leads to better performance and lower overhead. A wl_gamepad_manager will send an event for each physical gamepad (as it dynamically appears, if hotplugged later) associated with this particular wl_seat, creating a wl_gamepad object for each. A wl_gamepad object will send an event about the player id as the first thing, and also if it later changes. Some gamepads don't have player id controls, so we can't rely on them, but supporting them where we can is useful. I think it's best viewed as a really forceful hint as to the player's ID, where otherwise we're stuck doing heuristics with plugging. It's not about the gamepad capabilities at all. It's just an assignment, configured in the server: this input device belongs to player N. If a gamepad is hot-unplugged, a wl_gamepad event will notify about that, and the wl_gamepad object becomes inert (does not send any events, ignores all but the destroy request). Dealing gracefully with things like wireless gamepads running their batteries flat or moving out of radio range is important, which is what I assume this is to deal with. I presume the idea here is that if the player moves back into range or replaces the batteries, the wl_gamepad object revives? No, that's not what I had in mind. An inert protocol object is permanently dead by definition, and is only waiting for destruction by the client. This is just a convenience to avoid races between the server and the client. If the gamepad later comes back online, it is like it was hotplugged again: a new wl_gamepad object is sent, with the same player id as before. Gamepad input events are delivered according to the keyboard focus of the related wl_seat. If there is no keyboard to focus, then use the pointer focus, or something. It doesn't really affect the protocol design how the focus is assigned. However, would we need a wl_gamepad::enter,leave events? Probably, along with events for initial state. Or maybe enter/leave should be wl_gamepad_manager events? I think we need enter/leave events. The client can be responsible for cleaning up its own state, though if an initial state is sent on focus gain that makes things much easier. Yeah, the main point of the leave event is to say you don't get any more input events from this device, until it comes back, and it also implies that the client should forget all temporary state of the gamepad, like which buttons were down. Immediately following an enter event, or in the enter event, a new set of current state is sent. Notice, that this should not be done by sending e.g. fake button-down events. We have a protocol design policy, that input events from user actions are never manufactured. I don't see anything here that raises any flags for me; at least at first reading it seems quite usable. Cool. There are lots of details to get right, but those are easier to tune with some XML at hand. Thanks, pq ___ wayland-devel mailing
Re: Gamepad focus model (Re: Input and games.)
On Tue, May 7, 2013 at 3:23 AM, Pekka Paalanen ppaala...@gmail.com wrote: Yeah, like Daniel said, there is no concept of a return value. When a client creates a new object, the server can only either agree, or disconnect the client with a protocol error. Any other behaviour requires specialized handling, and causes a roundtrip, where the client must either wait for a reply before continuing, or risk having further requests ignored without any obvious way to know what got ignored in the end. Both cases are unacceptable. Ok. I was assuming that cases where you had fundamental capability change in the server (ie: input devices appearing or disappearing) were rare and special enough to warrant a round trip. When a client sends a request, that creates a new protocol object, then from the client's point of view, the object is created on that instant, before the request has even been submitted to the wire. This allows the client to immediately send more requests on that new object, without waiting for a roundtrip in between. The same works also in the reverse direction, when the server creates protocol objects by sending events. A major design principle in Wayland is to minimize roundtrips, as it leads to better performance and lower overhead. Fair enough. We're talking about rare events here, so I wouldn't have called it essential, but if that's an organizing principle of the project then so be it. It's not about the gamepad capabilities at all. It's just an assignment, configured in the server: this input device belongs to player N. The place where that becomes a problem is with controller batteries. As an example, I've got a PS3, and my wife uses it to watch netflix (it's a streaming tv/movie service, for those who haven't heard of it). It uses the PS3 controller as a remote, to do things like play/pause. It's not uncommon for the battery in the controller to run flat while she's watching. I've got a second controller, and we typically charge one while the other is in use, but fairly often the controller she's using runs flat. When that happens, we have a second charged controller, but to use it we have to reboot the PS3, because without rebooting it connects as Player 2, and netflix only listens to Player 1. As far as I know there's no simple way to tell the gamepad to reconnect as Player 1, short of rebooting the machine and rerunning all the controller handshaking. When a gamepad goes away and then it reappears or another appears, it's *probably* the same player. So what I'm thinking is that it makes more sense to have the wl_gamepad go into a disconnected state, and then reactivate when the next gamepad appears, rather than creating a new wl_gamepad. If the gamepad later comes back online, it is like it was hotplugged again: a new wl_gamepad object is sent, with the same player id as before. This would work too. The main thing is dealing well with the single player case where the player is replacing a gamepad. This could be because: - they wandered out of RF range when they were getting a drink - they want to play the game with a different gamepad - the gamepad they were using ran out of power and is now plugged in via usb - the gamepad they were using ran out of power and is being replaced with a charged gamepad - someone tripped over the usb cord and yanked it out and then plugged it back in Yeah, the main point of the leave event is to say you don't get any more input events from this device, until it comes back, and it also implies that the client should forget all temporary state of the gamepad, like which buttons were down. Yes. Immediately following an enter event, or in the enter event, a new set of current state is sent. Notice, that this should not be done by sending e.g. fake button-down events. We have a protocol design policy, that input events from user actions are never manufactured. My temptation would actually be to say that when focus goes to a new application, we treat buttons that are down as if they were up; don't send a release when they are lifted. So, if I'm holding down SELECT when focus enters the client window and then release it, press it and release it, the client sees the press and the second release, but not the initial release. That doesn't work with axis values, but if the client cares about deltas it's going to have to clear them on focus change anyways, since it has already been said that the protocol will not be sending deltas. If we were sending deltas we could make things a little cleaner in some ways, but it does expand the protocol and I'm not sure it does so usefully. Todd. -- Todd Showalter, President, Electron Jump Games, Inc. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On Tue, 7 May 2013 11:14:08 -0400 Todd Showalter t...@electronjump.com wrote: On Tue, May 7, 2013 at 3:23 AM, Pekka Paalanen ppaala...@gmail.com wrote: Yeah, like Daniel said, there is no concept of a return value. When a client creates a new object, the server can only either agree, or disconnect the client with a protocol error. Any other behaviour requires specialized handling, and causes a roundtrip, where the client must either wait for a reply before continuing, or risk having further requests ignored without any obvious way to know what got ignored in the end. Both cases are unacceptable. Ok. I was assuming that cases where you had fundamental capability change in the server (ie: input devices appearing or disappearing) were rare and special enough to warrant a round trip. When a client sends a request, that creates a new protocol object, then from the client's point of view, the object is created on that instant, before the request has even been submitted to the wire. This allows the client to immediately send more requests on that new object, without waiting for a roundtrip in between. The same works also in the reverse direction, when the server creates protocol objects by sending events. A major design principle in Wayland is to minimize roundtrips, as it leads to better performance and lower overhead. Fair enough. We're talking about rare events here, so I wouldn't have called it essential, but if that's an organizing principle of the project then so be it. They can be rare, but they can also happen any time. It's not about the gamepad capabilities at all. It's just an assignment, configured in the server: this input device belongs to player N. The place where that becomes a problem is with controller batteries. As an example, I've got a PS3, and my wife uses it to watch netflix (it's a streaming tv/movie service, for those who haven't heard of it). It uses the PS3 controller as a remote, to do things like play/pause. It's not uncommon for the battery in the controller to run flat while she's watching. I've got a second controller, and we typically charge one while the other is in use, but fairly often the controller she's using runs flat. When that happens, we have a second charged controller, but to use it we have to reboot the PS3, because without rebooting it connects as Player 2, and netflix only listens to Player 1. As far as I know there's no simple way to tell the gamepad to reconnect as Player 1, short of rebooting the machine and rerunning all the controller handshaking. When a gamepad goes away and then it reappears or another appears, it's *probably* the same player. So what I'm thinking is that it makes more sense to have the wl_gamepad go into a disconnected state, and then reactivate when the next gamepad appears, rather than creating a new wl_gamepad. If the gamepad later comes back online, it is like it was hotplugged again: a new wl_gamepad object is sent, with the same player id as before. This would work too. The main thing is dealing well with the single player case where the player is replacing a gamepad. This could be because: - they wandered out of RF range when they were getting a drink - they want to play the game with a different gamepad - the gamepad they were using ran out of power and is now plugged in via usb - the gamepad they were using ran out of power and is being replaced with a charged gamepad - someone tripped over the usb cord and yanked it out and then plugged it back in Yeah, sure, and that's all just heuristics inside the server. The server needs to make sure the player id becomes what the user wants, even if one wl_gamepad object is deleted and another created. The server will have the same problem even if it was supposed to revive a dead wl_gamepad object, anyway. The problem you described with PS3 should be solvable with the mysterious gamepad configuration GUI I talked about before, somehow. Yeah, the main point of the leave event is to say you don't get any more input events from this device, until it comes back, and it also implies that the client should forget all temporary state of the gamepad, like which buttons were down. Yes. Immediately following an enter event, or in the enter event, a new set of current state is sent. Notice, that this should not be done by sending e.g. fake button-down events. We have a protocol design policy, that input events from user actions are never manufactured. My temptation would actually be to say that when focus goes to a new application, we treat buttons that are down as if they were up; don't send a release when they are lifted. So, if I'm holding down SELECT when focus enters the client window and then release it, press it and release it, the client sees the press and the second release, but not the initial release. It depends.
Re: Gamepad focus model (Re: Input and games.)
On Tue, May 7, 2013 at 1:02 PM, Pekka Paalanen ppaala...@gmail.com wrote: This would work too. The main thing is dealing well with the single player case where the player is replacing a gamepad. This could be because: - they wandered out of RF range when they were getting a drink - they want to play the game with a different gamepad - the gamepad they were using ran out of power and is now plugged in via usb - the gamepad they were using ran out of power and is being replaced with a charged gamepad - someone tripped over the usb cord and yanked it out and then plugged it back in Yeah, sure, and that's all just heuristics inside the server. The server needs to make sure the player id becomes what the user wants, even if one wl_gamepad object is deleted and another created. The client needs to look at a new wl_gamepad when it shows up and decide whether it's a new player or an existing player who is reconnecting, As long as it's easy for the client to do that, I think we're good. The problem you described with PS3 should be solvable with the mysterious gamepad configuration GUI I talked about before, somehow. Partly, though I think the default case should be that if a controller disappears and another (or the same one) appears, the assumption is it's the player that just left coming back. The number of times that isn't true isn't likely to be statistically significant. My temptation would actually be to say that when focus goes to a new application, we treat buttons that are down as if they were up; don't send a release when they are lifted. So, if I'm holding down SELECT when focus enters the client window and then release it, press it and release it, the client sees the press and the second release, but not the initial release. It depends. If a gamepad enters with button A down, and then the user presses button B down, is the application supposed to respond to B or A+B? In my experience games that use gamepads don't usually use the gamepad buttons as modifiers; it can happen, but it's awkward to explain to the player and often awkward to actually perform with the hands. What you get more often is some sort of lockon, where holding a button down makes player motion relative to a target (so you can circle-strafe around an opponent, for example). In cases like this the focus switch is likely to have broken the player's context anyways. Todd. -- Todd Showalter, President, Electron Jump Games, Inc. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Hi, On 7 May 2013 16:14, Todd Showalter t...@electronjump.com wrote: On Tue, May 7, 2013 at 3:23 AM, Pekka Paalanen ppaala...@gmail.com wrote: Yeah, like Daniel said, there is no concept of a return value. When a client creates a new object, the server can only either agree, or disconnect the client with a protocol error. Any other behaviour requires specialized handling, and causes a roundtrip, where the client must either wait for a reply before continuing, or risk having further requests ignored without any obvious way to know what got ignored in the end. Both cases are unacceptable. Ok. I was assuming that cases where you had fundamental capability change in the server (ie: input devices appearing or disappearing) were rare and special enough to warrant a round trip. There's another reason too: unifying the cold- and hotplug cases. With the X model, where you have a roundtrip to enumerate devices and their details, plus hotplug events, you have two codepaths and it gets quite annoying. Wayland follows the udev model, where coldplug is just a storm of hotplug events. If you really need to do strict enumeration, you could do a roundtrip immediately after creating the gamepad object in which all you do is destroy it, knowing that in between those two happening, you've got all your device creation events. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Todd Showalter wrote: My temptation would actually be to say that when focus goes to a new application, we treat buttons that are down as if they were up; don't send a release when they are lifted. So, if I'm holding down SELECT when focus enters the client window and then release it, press it and release it, the client sees the press and the second release, but not the initial release. Pointer motion while a button is held down does not change the focus, right? (ie same as mice). So the only way this happens is if the client or server actively changes the focus or if the current focus is destroyed. Any solution should be similar to any solution for keyboard keys and mouse buttons (even if it is just these buttons are in class 1, these in class 2, these in class 3, etc...), so that it is possible to talk about all devices with buttons in the same documentation. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On Mon, May 6, 2013 at 8:36 AM, Pekka Paalanen ppaala...@gmail.com wrote: Into wl_seat, we should add a capability bit for gamepad. When the bit is set, a client can send wl_seat::get_gamepad_manager request, which creates a new wl_gamepad_manager object. (Do we actually need a capability bit?) There are options here: - have the capability bit, if the bit is set the client can request a manager -- has to deal with the case where the client sent the request but the caps bit wasn't set, presumably by returning NULL or -1 the protocol equivalent - leave out the caps bit, client requests the manager if they want it, they get NULL equivalent if there are no gamepads - leave out the caps bit, gampad manager is always there, but can be expected to return 0 if asked to enumerate gamepads when none are connected A wl_gamepad_manager will send an event for each physical gamepad (as it dynamically appears, if hotplugged later) associated with this particular wl_seat, creating a wl_gamepad object for each. A wl_gamepad object will send an event about the player id as the first thing, and also if it later changes. Some gamepads don't have player id controls, so we can't rely on them, but supporting them where we can is useful. I think it's best viewed as a really forceful hint as to the player's ID, where otherwise we're stuck doing heuristics with plugging. If a gamepad is hot-unplugged, a wl_gamepad event will notify about that, and the wl_gamepad object becomes inert (does not send any events, ignores all but the destroy request). Dealing gracefully with things like wireless gamepads running their batteries flat or moving out of radio range is important, which is what I assume this is to deal with. I presume the idea here is that if the player moves back into range or replaces the batteries, the wl_gamepad object revives? Gamepad input events are delivered according to the keyboard focus of the related wl_seat. If there is no keyboard to focus, then use the pointer focus, or something. It doesn't really affect the protocol design how the focus is assigned. However, would we need a wl_gamepad::enter,leave events? Probably, along with events for initial state. Or maybe enter/leave should be wl_gamepad_manager events? I think we need enter/leave events. The client can be responsible for cleaning up its own state, though if an initial state is sent on focus gain that makes things much easier. I don't see anything here that raises any flags for me; at least at first reading it seems quite usable. Todd. -- Todd Showalter, President, Electron Jump Games, Inc. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On 6 May 2013 14:48, Todd Showalter t...@electronjump.com wrote: On Mon, May 6, 2013 at 8:36 AM, Pekka Paalanen ppaala...@gmail.com wrote: Into wl_seat, we should add a capability bit for gamepad. When the bit is set, a client can send wl_seat::get_gamepad_manager request, which creates a new wl_gamepad_manager object. (Do we actually need a capability bit?) There are options here: - have the capability bit, if the bit is set the client can request a manager -- has to deal with the case where the client sent the request but the caps bit wasn't set, presumably by returning NULL or -1 the protocol equivalent - leave out the caps bit, client requests the manager if they want it, they get NULL equivalent if there are no gamepads Wayland doesn't have a 'return NULL' facility: the client creates an entry for the object, and a proxy, and then the server later instantiates that object. The only way to return NULL is an asynchronous error which requires specialised handling. - leave out the caps bit, gampad manager is always there, but can be expected to return 0 if asked to enumerate gamepads when none are connected Similarly, there's no round trip to enumerate gamepads. The wl_gamepad_manager would advertise new (to the client) gamepads, i.e. when the client creates the gamepad manager, the manager immediately advertises all existing devices, and then later advertises new devices as they're added. Gamepad input events are delivered according to the keyboard focus of the related wl_seat. If there is no keyboard to focus, then use the pointer focus, or something. It doesn't really affect the protocol design how the focus is assigned. However, would we need a wl_gamepad::enter,leave events? Probably, along with events for initial state. Or maybe enter/leave should be wl_gamepad_manager events? I think we need enter/leave events. The client can be responsible for cleaning up its own state, though if an initial state is sent on focus gain that makes things much easier. Yeah, I think we definitely need enter/leave with current state. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
Pekka Paalanen ppaalanen@... writes: This design allows several gamepads associated with one wl_seat, and thus one focus. It also allows gamepads to be assigned to different seats, but then we will have more problems on managing the foci, not unlike with keyboards. Hopefully there are no protocol design implications, though. From the game's point of view, it will need to iterate over all wl_seats. For each seat with the gamepad capability bit set, create a wl_gamepad_manager, receive all wl_gamepad objects, and for each wl_gamepad receive the player id. Create your surfaces, wait for foci to arrive, and fire away. This all sounds good, and I certainly wouldn't try to block it, but I do have some thought experiments that lead to another solution. Incoming thought experiments, from least to most theoretical: Scenario 1) A typical single-user display server would likely only support one wl_seat, and assign all wl_gamepads to that seat. Games can iterate over them using wl_gamepad_manager. Great! Scenario 2) Two users are using a multi-user display server. Each user has a keyboard, mouse, and gamepad. User 2 has to set up their wl_seat using some configuration window built into the display server, but once that's done each user can jump in and out of a game as they see fit, assuming the game was written to iterate over wl_seats properly. Neat! Scenario 3) Two users are using gamepads with built-in touch screens (like the PS4 and OUYA controllers). The players don't care about the ability to have their own window focus, but the game wants to associate each touchpad with its gamepad. Player 2 must set up their own wl_seat in the display server, and must focus the game separately using the touchpad. Not ideal. You might say, yeah, it's a bit of extra effort to set up player 2, but the cool thing is that player 2 can jump in and out of the game as they see fit! Only that's not necessarily true either. Since many games steal the cursor, and player 2 has no keyboard, player 2 can jump into the game (using the touchpad on the controller) but can't jump out. Scenario 4) In this scenario we imagine that wl_seat has grown support for wl_headset. Player 2 has an Xbox 360 controller with a headset plugged in. To support this, player 2 must go through the display server's wl_seat configuration to set up their own seat. Like in scenario 3, this is less than ideal. And how does player 2 focus the game without a keyboard or mouse? Now, here's my alternate suggestion: * Drop wl_gamepad_manager. * Give wl_seat the ability to share focus with the default seat. * A typical display server would add each gamepad to a new seat that shares focus with the default seat. * Multiple wl_seats is *the* way to support multiple controllers. Going through the scenarios again: Scenario 1) A single-user display would assign each wl_gamepad to a unique wl_seat that shares focus with the default seat. Games iterate through wl_seats to get each player. Cool. Scenario 2) On a multi-user display server, users can still use some configuration screen to set up their own mouse/keyboard/gamepad/focus, allowing them to jump in and out of the game as they see fit. As an added bonus, any multi-controller game is *guaranteed* to work properly under this setup, since the game programmer can't wrongly assume there's only one wl_seat. Great! Scenarios 3, 4) The display server can assume that devices attached to a gamepad should go into the same seat as that gamepad, and don't need to have their own focus. Player 2 is free to use their PS4/OUYA controller or headset in a game without any extra configuration. Cool! I don't know how much would get upset by allowing wl_seats to share focus, but it seems to be more general and future-proof than the wl_gamepad_manager solution. -Rick- ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Gamepad focus model (Re: Input and games.)
On Mon, May 6, 2013 at 5:10 PM, Rick Yorgason r...@firefang.com wrote: Pekka Paalanen ppaalanen@... writes: This design allows several gamepads associated with one wl_seat, and thus one focus. It also allows gamepads to be assigned to different seats, but then we will have more problems on managing the foci, not unlike with keyboards. Hopefully there are no protocol design implications, though. From the game's point of view, it will need to iterate over all wl_seats. For each seat with the gamepad capability bit set, create a wl_gamepad_manager, receive all wl_gamepad objects, and for each wl_gamepad receive the player id. Create your surfaces, wait for foci to arrive, and fire away. This all sounds good, and I certainly wouldn't try to block it, but I do have some thought experiments that lead to another solution. Incoming thought experiments, from least to most theoretical: Scenario 1) A typical single-user display server would likely only support one wl_seat, and assign all wl_gamepads to that seat. Games can iterate over them using wl_gamepad_manager. Great! Scenario 2) Two users are using a multi-user display server. Each user has a keyboard, mouse, and gamepad. User 2 has to set up their wl_seat using some configuration window built into the display server, but once that's done each user can jump in and out of a game as they see fit, assuming the game was written to iterate over wl_seats properly. Neat! Scenario 3) Two users are using gamepads with built-in touch screens (like the PS4 and OUYA controllers). The players don't care about the ability to have their own window focus, but the game wants to associate each touchpad with its gamepad. Player 2 must set up their own wl_seat in the display server, and must focus the game separately using the touchpad. Not ideal. You might say, yeah, it's a bit of extra effort to set up player 2, but the cool thing is that player 2 can jump in and out of the game as they see fit! Only that's not necessarily true either. Since many games steal the cursor, and player 2 has no keyboard, player 2 can jump into the game (using the touchpad on the controller) but can't jump out. I think this is where a judicious use of the home button could come in very handy. As long as the game is nice and releases the mouse on home, or the compositor takes over or something, the problem is solved. Scenario 4) In this scenario we imagine that wl_seat has grown support for wl_headset. Player 2 has an Xbox 360 controller with a headset plugged in. To support this, player 2 must go through the display server's wl_seat configuration to set up their own seat. Like in scenario 3, this is less than ideal. And how does player 2 focus the game without a keyboard or mouse? Now, here's my alternate suggestion: * Drop wl_gamepad_manager. Personally, I don't see why we need wl_gamepad_manager in order to have multiple gamepads. Why can't we have a get_gamepads request to which the seat respons with a gamepad event. Then again, the extra protocol object isn't that bad either. * Give wl_seat the ability to share focus with the default seat. * A typical display server would add each gamepad to a new seat that shares focus with the default seat. I don't think we should restrict this to default. Effectively, we would need a way for a seat to have a parent seat from which it derives its focus. This may get to be a mess especially from a user-interface perspective. Perhaps you could have a automatically parent to whatever my default is paradigm. However, for more interesting configs, it get a lot more confusing. See below. * Multiple wl_seats is *the* way to support multiple controllers. Going through the scenarios again: Scenario 1) A single-user display would assign each wl_gamepad to a unique wl_seat that shares focus with the default seat. Games iterate through wl_seats to get each player. Cool. Scenario 2) On a multi-user display server, users can still use some configuration screen to set up their own mouse/keyboard/gamepad/focus, allowing them to jump in and out of the game as they see fit. As an added bonus, any multi-controller game is *guaranteed* to work properly under this setup, since the game programmer can't wrongly assume there's only one wl_seat. Great! Ok, let's bump this a step further and say that you have four controllers and two mice/keyboards. You want to have 2 gamepads for each mouse. Now we have a problem since only the default focus can have multiple gamepads associated with it. Like I said above, this can be solved by having a parenting structure for seats. Scenarios 3, 4) The display server can assume that devices attached to a gamepad should go into the same seat as that gamepad, and don't need to have their own focus. Player 2 is free to use their PS4/OUYA controller or headset in a game without any extra configuration. Cool! Having the two controllers
Re: Gamepad focus model (Re: Input and games.)
Jason Ekstrand jason@... writes: Scenario 2) Two users are using a multi-user display server. Each user has a keyboard, mouse, and gamepad. User 2 has to set up their wl_seat using some configuration window built into the display server, but once that's done each user can jump in and out of a game as they see fit, assuming the game was written to iterate over wl_seats properly. Neat! Scenario 3) Two users are using gamepads with built-in touch screens (like the PS4 and OUYA controllers). The players don't care about the ability to have their own window focus, but the game wants to associate each touchpad with its gamepad. Player 2 must set up their own wl_seat in the display server, and must focus the game separately using the touchpad. Not ideal. You might say, yeah, it's a bit of extra effort to set up player 2, but the cool thing is that player 2 can jump in and out of the game as they see fit! Only that's not necessarily true either. Since many games steal the cursor, and player 2 has no keyboard, player 2 can jump into the game (using the touchpad on the controller) but can't jump out. I think this is where a judicious use of the home button could come in very handy. As long as the game is nice and releases the mouse on home, or the compositor takes over or something, the problem is solved. Well, *a* problem is solved, but if you read carefully you'll see that's not actually the problem the user was trying to solve. You're still saying Yeah, I know it's a pain to set up your controller, but look at this neat thing you can do that you never really cared about! Scenario 4) In this scenario we imagine that wl_seat has grown support for wl_headset. Player 2 has an Xbox 360 controller with a headset plugged in. To support this, player 2 must go through the display server's wl_seat configuration to set up their own seat. Like in scenario 3, this is less than ideal. And how does player 2 focus the game without a keyboard or mouse? I don't think we should restrict this to default. Effectively, we would need a way for a seat to have a parent seat from which it derives its focus. This may get to be a mess especially from a user-interface perspective. Perhaps you could have a automatically parent to whatever my default is paradigm. However, for more interesting configs, it get a lot more confusing. See below. Yes, a parent seat is definitely more general. I was originally thinking of something like that, but a use for it didn't immediately come to mind. The two pairs of players scenario is interesting. Having the two controllers paired doesn't solve 3. There are a lot of UI problems with having two pointers running around the screen that share a focus. Let's say they're one of those crazy users that like sloppy- focus. What happens when the two cursors are on different windows? Does the primary cursor over-ride? Does the secondary cursor work in windows that aren't focused? Even if you have click-to-focus, you still have problems with the two cursors fighting. What happens if they both try to do a drag-and-drop? The only solution to this is probably to make the secondary pointer inert in windows that aren't focussed. That said, I'm not sure if that 100% fixes it either. I was thinking that a desktop environment would probably only have one cursor. In other words, if any wl_seats are sharing focus, most apps would treat them as aggregate, and it would only be more specialized apps (like games) that might want to separate them. To tie up my comments, I think this can get far too complicated fast. I think it's better to have a seat correspond to a focus and allow for the fact that we may have more than one person to a seat. It's not 100% ideal, but it's a lot simpler than some strange tree of seats. Besides, if some compositor wants to do a strange tree of seats, there's nothing in pq's proposed protocol that would prevent that and games shouldn't care. pq's proposed protocol is certainly good enough. My two concerns are: 1) Does this make it a pain to give players more than one IO device, like gamepad+touchscreen or gamepad+headset. 2) If multiple wl_seats are only used in exotic setups, then programmers are unlikely to discover that they're not enumerating them properly. Both of these concerns are somewhat exotic, and I'm personally unlikely to run into these problems, so I'm not going to lose any sleep if they're not addressed. It just seems like wl_seats would be a perfect abstraction for multiple players if only they didn't have to be so tightly tied to focus. -Rick- ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
