Re: Mouse Lock
WebEvents WG is taking up this spec. I've created a new Mouse Lock thread on public-webevents: http://lists.w3.org/Archives/Public/public-webevents/2011OctDec/0066.html
Re: Mouse Lock
WebEvents WG has proposed a charter revision that includes Mouse Lock: http://lists.w3.org/Archives/Public/public-webevents/2011JulSep/0087.html Also, Minor update to spec: 0.71, 2011-09-27 Preventing malicious 'click jacking' behavior has been curtailed by requiring the target of mouse lock events to be in the top-level origin (as the call to lockMouse() must also be). This replaces previous text requiring mouse lock events to not be considered user gestures. This was instigated by comments on webkit-dev https://lists.webkit.org/pipermail/webkit-dev/2011-September/018028.html On Mon, Sep 12, 2011 at 2:11 PM, Vincent Scheib sch...@google.com wrote: I've updated the draft spec http://goo.gl/9G8pd with these points. - Removed Touch related components to simplify initial version of specification and because they do not add the same level of new application capability as mouse related events. - MouseLockable Interface moved from element to document, mouselock takes target as parameter, mouseLocked returns current target element. - movementX/Y behavior when system cursor reenters window specified as zeros - lockMouse must be called from the same origin as the top level document - Cases of target element not being in DOM tree specified to fail or exit lock - Mouse user gestures suppressed while under mouse lock - Client and screen coordinates when under lock report the last value they had when lock was entered. Clarification on events that should and should not fire when under mouse lock. - Added example Example handling failure to lock - Use case rewrite: View-port panning by moving a mouse cursor against the bounds of a view-port. - Merged use cases Free three dimensional model rotation Move layers of information without bound limits - Added FAQ Why bundle all functionality (hiding cursor, providing mouse deltas) instead of using CSS to hide the cursor, always providing delta values, and offering an API to restrict the cursor movement to a portion of the webpage? - Added FAQ Why modify MouseEvent and reuse existing mouse events instead of creating a mousedelta event? - Added FAQ Why separate targets for mouse events under mouse lock and keyboard input focus? On Fri, Aug 26, 2011 at 4:56 PM, Vincent Scheib sch...@google.com wrote: -- What if target is removed from DOM, or is not currently in DOM? -- Additional issues from Olli Pettay on w3 bug. I believe mouse lock should fail to acquire if the target is not in the DOM tree, and should be exited if the target of an existing lock is removed from the tree. -- Are user gestures generated while under mouse lock? -- From a security point of view, should we sanitize the events generated when in mouse lock (click, down, up, ...) to be 'synthetic' and not 'user gestures'? E.g. in Chrome I believe you can only pop up new windows with window.open from script code triggered by a user gesture event. If we don't, and assuming security model allowing only mouse lock for DOM within top level document origin, then a page could direct user gesture events to any DOM element in their origin they wanted. That may be used for a form of 'click jacking'. On Thu, Aug 25, 2011 at 5:54 PM, Vincent Scheib sch...@google.comwrote: Continuing the discussion of some of the open issues I'm aware of: -- If MouseEvent types are returned under mouse lock, what should .clientX/Y and screenX/Y be? -- Spec as drafted states they should be the center of the target element. That's likely a poor idea, as discussed, and a better solution is to freeze .clientX/Y and screenX/Y to whatever their last value from the user agent was. (Pending feedback, I will update spec to state that .clientX/Y .screenX/Y will report the same constant values of the last known cursor position just as mouse lock was entered. This will also be the location the system cursor will be placed at when mouse lock is exited) -- Modify MouseEvent, or a new type returned by new mouse events? -- Spec as drafted now has simply added .movementX/Y data members to the MouseEvent type which is returned by mouse events: click, mousedown, mouseup, mouseover, mousemove, mouseout. No new mouse event types are introduced. Several, including Glenn Maynard, Robert O'Callahan have brought up points against this, suggesting different events and types. Let me enumerate options and my perceived pro/cons: Option A: as currently drafted: reuse existing mouse events, make minimal modification to MouseEvent to include movementX/Y valid regardles of mouse lock state, mouseover mouseout will never be fired under mouse lock. pro 1. minimal change to existing events and types pro 2. movementX/Y available even when mouse is not locked. Purely a convenience, apps could compute similar data by tracking the last position of a mouse event and computing deltas (resetting last position on a mouseover event). pro 3. event handling code that can operate both with and without
Re: Mouse Lock
On Mon, Aug 29, 2011 at 2:29 AM, Vincent Scheib sch...@google.com wrote:
I believe first developers seek a specifiable solution that will provide a
consistent application experience on all platforms and API implementations.
Yes, the raw input method provides this. (I don't like calling it that,
since it's not necessarily anything of the sort and it suggests that it's
lower level than it is, but I don't have another name off-hand to
distinguish these.)
Finally, I believe that it is worthwhile to consider adding some rawer form
of the input data if possible in a v2 implementation and or spec.
The raw input implementation requires a different interface. What you
suggest would require implementing and supporting two completely different
approaches to delta input. This seems shortsighted.
Movement values will stop when the cursor hits a screen edge.
This defeats the purpose.
I'm deferring consideration for higher resolution mice data and sampling
frequency to a future version of this specification. This was discussed some
in Provide mouse deltas without hiding the cursor, alternatively provide a
bounding rectangle for mouse. But a summary of my position is that:
- Specification is difficult and may not be possible.
- Implementation is more complex and doesn't seem justified yet.
As I pointed out before, both specification and implementation is simpler; a
laundry list of complexities go away.
- Graceful addition of more 'raw' data and higher sampling frequencies is
possible. I'm not saying We shouldn't implement raw data, but I am saying
we should defer it.
I don't believe graceful addition is possible; an entirely separate
interface, using a separate event, would be necessary. As a result, if this
approach is implemented, it'd be much less likely that the other would ever
happen. Half-solutions have a tendency to prevent the creation of a full
solution, even when it's claimed beforehand that it's only being deferred.
This was vaguely suggested in that no one made a clear proposal of what
separate events would look like. click, down, and up events are needed. The
current events use the MouseEvent type. I believe that for the same reason
mousedown includes more context than just the mouse button was pressed
that the mouse lock equivalent would too. Thus, the mouse lock version of
click, down, and up events should contain .movementX/Y data.
This doesn't make sense to me. Movement is unrelated to clicks. There's no
need to have special locked versions for these events; there's no movement
data associated with a click. If they had movement attributes, they would
always be zero.
If you want a concrete proposal, I'd start with something like this:
When a delta input device is moved, fire an event named mousedelta,
which bubbles and is cancelable, and uses the MouseDeltaEvent interface, at
the Window object. The deltaX attribute must be set to the horizontal
distance moved, with positive values indicating motion to the right. The
deltaY attribute must be set to the vertical distance moved, with positive
values indicating motion downwards. A delta value of 400 must indicate a
movement of one inch.
interface MouseDeltaEvent : Event {
readonly attribute float deltaX;
readonly attribute float deltaY;
};
The value of 400 DPI is suggested because, according to [1], that's the most
common resolution for mice. Implementations scale values to that resolution
if necessary. The interface uses floats, to support arbitrary-resolution
input.
[1] http://msdn.microsoft.com/en-us/library/ee418864%28v=vs.85%29.aspx
--
Glenn Maynard
Re: Mouse Lock
But do you agree with me that post ballistics, accelerated and low resolution is exactly what you don't want for most use cases? Why should it not be possible to specify the units I described in my last mail: http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/0978.html. That email describes computing distance traveled of input device presuming DPI, sampling rate, and a sample's delta data. DPI is unknown (unless I'm not finding it in APIs e.g. around http://msdn.microsoft.com/en-us/library/ms645589.aspx). I believe first developers seek a specifiable solution that will provide a consistent application experience on all platforms and API implementations. I further believe that post ballistics data will satisfy most developers and users; I have asked some e.g. Unity developers about their implementations and needs, and checked source of other games (e.g. SDL library). Finally, I believe that it is worthwhile to consider adding some rawer form of the input data if possible in a v2 implementation and or spec. Option A: as currently drafted: reuse existing mouse events, make minimal modification to MouseEvent to include movementX/Y valid regardles of mouse lock state, mouseover mouseout will never be fired under mouse lock. pro 2. movementX/Y available even when mouse is not locked. Purely a convenience, apps could compute similar data by tracking the last position of a mouse event and computing deltas (resetting last position on a mouseover event). Purely a convenience is incorrect. Once the mouse cursor reaches the edge of the screen, you can no longer detect motion in that direction by examining the cursor position. That's why you either need to warp the mouse or use raw input events. (Remember, mousemove events are still sent when the mouse is outside of the window, if the user is dragging.) Sorry, some confusion here. This topic Modify MouseEvent, or a new type returned by new mouse events? presumes the hide cursor and warp method, as do all other sections in my recent summary except section Provide mouse deltas without hiding the cursor, alternatively provide a bounding rectangle for mouse. This section is trying to lay out the pros/cons of where to put movement data. The Option A is to provide movementX/Y when locked and unlocked. When locked, the cursor disappears and movement data is provided without limit. When unlocked the mouse events continue to contain that data member and values are provided with best effort. The system cursor exists and mouse events are not available when the cursor is not over the window (unless the mouse is captured). Movement values will stop when the cursor hits a screen edge. The movement members then become only a convenience because javascript code can compute the same values by storing last known positions of screenX/Y and tracking motion with mousemove mouseover. MouseEvent to include movementX/Y valid regardles of mouse lock state I don't think it's possible to both 1: use the same units as the cursor (screen pixels) and 2: get this information when the cursor is visible. If you use the raw APIs (eg. WM_INPUT) Correct, the proposal is NOT to use raw APIs as described above. You forgot a con with this approach: There's no way of exposing functionality of high-res mice. You can only detect mouse motion when it's far enough to cause the (unseen) cursor to move a pixel, because you're reporting motion in terms of cursor movement, rather than the higher-resolution data provided by the device. You're also forced to receive mouse acceleration and I believe axis locking (improve pointer precision in Windows), which may not always be wanted for first-person games. (You also can't expose the ability of modern mice to report at higher rates, eg. 1khz, for the same reason; I'm personally ambivalent about this since I don't have any use cases for it, but others may.) I'm deferring consideration for higher resolution mice data and sampling frequency to a future version of this specification. This was discussed some in Provide mouse deltas without hiding the cursor, alternatively provide a bounding rectangle for mouse. But a summary of my position is that: - Specification is difficult and may not be possible. - Implementation is more complex and doesn't seem justified yet. - Graceful addition of more 'raw' data and higher sampling frequencies is possible. I'm not saying We shouldn't implement raw data, but I am saying we should defer it. I'm happy to review any prototype patches to the chromium/webkit projects that answer these issues. New mouse events created with unique names for click, down, up, move, e.g. mouselockclick, mouselockdown, mouselockup, mouselockmove. I'm confused: was this suggested? I suggested a single event fired for delta movement, eg. mousedelta, but of course there wouldn't be separate events for clicks, too; clicks are orthogonal to motion. (I'm confused that you dismissed
Re: Mouse Lock
-- Provide mouse deltas without hiding the cursor, alternatively provide a bounding rectangle for mouse -- (See Klaas's recent suggestions) It may be possible, but significant implementation hurdles remain for this. First, we must be able to specify the units that mouse deltas will be provided in. That does not appear straight forward. E.g. the windows API offers only the movements, sampling frequency, and HID id. Offering data in post ballistics / accelerated and configured mouse pointer units is straight forward to specify and implement across platforms and implementations of this API. If we find implementation routes we can add access to this data in a later revision, the two API versions are not mutually exclusive and implementation work would be straight forward to share. All use cases are possible even with the 'bundled' proposal as currently spec-ed though they may need to create their own cursor for a visible cursor. But do you agree with me that post ballistics, accelerated and low resolution is exactly what you don't want for most use cases? Why should it not be possible to specify the units I described in my last mail: http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/0978.html. 2011/8/26 Vincent Scheib sch...@google.com Continuing the discussion of some of the open issues I'm aware of: -- If MouseEvent types are returned under mouse lock, what should .clientX/Y and screenX/Y be? -- Spec as drafted states they should be the center of the target element. That's likely a poor idea, as discussed, and a better solution is to freeze .clientX/Y and screenX/Y to whatever their last value from the user agent was. (Pending feedback, I will update spec to state that .clientX/Y .screenX/Y will report the same constant values of the last known cursor position just as mouse lock was entered. This will also be the location the system cursor will be placed at when mouse lock is exited) -- Modify MouseEvent, or a new type returned by new mouse events? -- Spec as drafted now has simply added .movementX/Y data members to the MouseEvent type which is returned by mouse events: click, mousedown, mouseup, mouseover, mousemove, mouseout. No new mouse event types are introduced. Several, including Glenn Maynard, Robert O'Callahan have brought up points against this, suggesting different events and types. Let me enumerate options and my perceived pro/cons: Option A: as currently drafted: reuse existing mouse events, make minimal modification to MouseEvent to include movementX/Y valid regardles of mouse lock state, mouseover mouseout will never be fired under mouse lock. pro 1. minimal change to existing events and types pro 2. movementX/Y available even when mouse is not locked. Purely a convenience, apps could compute similar data by tracking the last position of a mouse event and computing deltas (resetting last position on a mouseover event). pro 3. event handling code that can operate both with and without mouse lock being acquired can do so easily using the same functions and taking the same MouseEvent data type input. e.g. rotateView(mouse_event) could be called from a mousemove event regardless of mouse lock state and still use .movementX/Y. con 1. When under mouse lock, clientX/Y screenX/Y will not offer any useful data - or worse implementations may not follow spec and provide misleading data of the actual hidden cursor location. con 2. Not all mouse events using MouseEvent are appropriate, mouseover and mouseout should be suppressed. Glen continued that it is probably cleaner to stop firing *all* mouse movement events entirely, as if the mouse isn't moving, and to use a separate mousedelta event when locked which only has deltaX and deltaY. Robert reiterated that point. Option B: make no use of or modification to any existing mouse event or the MouseEvent type. New mouse events created with unique names for click, down, up, move, e.g. mouselockclick, mouselockdown, mouselockup, mouselockmove. New events return a MouseLockEvent type derived from UIEvent that contains movementX/Y, ctrlKey, shiftKey, altKey, metaKey, button. The events and data types are completely mutually exclusive based on the state of mouse lock. pro 1. (dual of Option A con 1): clientX/Y screenX/Y do not exist under mouse lock. pro 2. (dual of Option A pro 3): Functions designed specifically to operate only under mouse lock, or only when not under mouse lock, are only triggered when appropriate. con 1. (dual of Option A pro 1): Larger change, introducing 4 new events and a new return type. con 2. (dual of Option A pro 2): movementX/Y must be calculated manually by apps when not under lock. Some hybrid could be considered, e.g. only make a new mouselockmove event but reuse the click/down/up events. I don't think that hybrid is worth discussing. My opinion: - Option A con 1 (.client .screen not useful, could be wrong): I don't perceive
Re: Mouse Lock
On Mon, Aug 22, 2011 at 8:13 PM, Vincent Scheib sch...@google.com wrote: What are the units of .movementX/Y if we're reading 'raw' input device coordinates? Are they consistent across devices and platforms? Can we specify them? I don't think any units can be specified here, since it's device-dependent and possibly OS-dependent (even if all systems give access to unprocessed USB units, units from devices are probably device-specific anyway). I don't think it's important, as long as it's clear to developers that delta units are *not* the same as coordinate units: they can't assume that a dX of 1 is the same as moving the mouse cursor one pixel to the right. In that case, it seems preferable for the delta units to *always* have different units. If some implementations have the same units in both, people would start depending on that, which would obviously cause problems. Option A: as currently drafted: reuse existing mouse events, make minimal modification to MouseEvent to include movementX/Y valid regardles of mouse lock state, mouseover mouseout will never be fired under mouse lock. pro 2. movementX/Y available even when mouse is not locked. Purely a convenience, apps could compute similar data by tracking the last position of a mouse event and computing deltas (resetting last position on a mouseover event). Purely a convenience is incorrect. Once the mouse cursor reaches the edge of the screen, you can no longer detect motion in that direction by examining the cursor position. That's why you either need to warp the mouse or use raw input events. (Remember, mousemove events are still sent when the mouse is outside of the window, if the user is dragging.) MouseEvent to include movementX/Y valid regardles of mouse lock state I don't think it's possible to both 1: use the same units as the cursor (screen pixels) and 2: get this information when the cursor is visible. If you use the raw APIs (eg. WM_INPUT), you get untouched device units, not screen units; so you lose #1. If you use the higher-level APIs, you need to warp the cursor to the center continually (to prevent clamping against the edge of the screen), in which case you lose #2 instead. (An OS could provide the necessary APIs to implement this, but I don't think they do.) You forgot a con with this approach: There's no way of exposing functionality of high-res mice. You can only detect mouse motion when it's far enough to cause the (unseen) cursor to move a pixel, because you're reporting motion in terms of cursor movement, rather than the higher-resolution data provided by the device. You're also forced to receive mouse acceleration and I believe axis locking (improve pointer precision in Windows), which may not always be wanted for first-person games. (You also can't expose the ability of modern mice to report at higher rates, eg. 1khz, for the same reason; I'm personally ambivalent about this since I don't have any use cases for it, but others may.) New mouse events created with unique names for click, down, up, move, e.g. mouselockclick, mouselockdown, mouselockup, mouselockmove. I'm confused: was this suggested? I suggested a single event fired for delta movement, eg. mousedelta, but of course there wouldn't be separate events for clicks, too; clicks are orthogonal to motion. (I'm confused that you dismissed what seems like the obvious approach as not worth discussing.) Also note that in this model, there would be no mouse lock to begin with; mousedelta events would simply be fired all the time, alongside and independent from mousemove events, presumably on the document or the window. A separate feature would be needed to contain the mouse to a rectangle (to prevent the cursor from drifting into another window where clicks might do something unwanted); and if the application wants to hide the cursor it would need to do so itself. In summary, pros: - High-resolution mouse movements can be reported accurately, which isn't possible with anything based on cursor movement. - Axis-locking won't be applied; games don't want this. - Mouse acceleration won't be applied; games may not want this. - High-frequency data can be reported (if anyone cares). - Mouse deltas can be used without invoking any API that might prompt the user. It's okay to ask allow this site to lock the mouse? when you start a game for the first time, but it would be unacceptable for GMaps when the user simply tries to drag the map around. (Note that the abovementioned mouse containment API would ask permission, but you wouldn't need to use that for the dragging use case.) - This is much simpler. There's no need to talk about locking screenX, etc. to some position, whether mouseover events are fired, restoring the cursor position on unlock, and so on; all of that just goes away. Cons: - Units don't match cursor movement. The only case I can think where this matters is dragging, eg. GMaps: dragging would move the map at a different rate than the
Re: Mouse Lock
-- What if target is removed from DOM, or is not currently in DOM? -- Additional issues from Olli Pettay on w3 bug. I believe mouse lock should fail to acquire if the target is not in the DOM tree, and should be exited if the target of an existing lock is removed from the tree. -- Are user gestures generated while under mouse lock? -- From a security point of view, should we sanitize the events generated when in mouse lock (click, down, up, ...) to be 'synthetic' and not 'user gestures'? E.g. in Chrome I believe you can only pop up new windows with window.open from script code triggered by a user gesture event. If we don't, and assuming security model allowing only mouse lock for DOM within top level document origin, then a page could direct user gesture events to any DOM element in their origin they wanted. That may be used for a form of 'click jacking'. On Thu, Aug 25, 2011 at 5:54 PM, Vincent Scheib sch...@google.com wrote: Continuing the discussion of some of the open issues I'm aware of: -- If MouseEvent types are returned under mouse lock, what should .clientX/Y and screenX/Y be? -- Spec as drafted states they should be the center of the target element. That's likely a poor idea, as discussed, and a better solution is to freeze .clientX/Y and screenX/Y to whatever their last value from the user agent was. (Pending feedback, I will update spec to state that .clientX/Y .screenX/Y will report the same constant values of the last known cursor position just as mouse lock was entered. This will also be the location the system cursor will be placed at when mouse lock is exited) -- Modify MouseEvent, or a new type returned by new mouse events? -- Spec as drafted now has simply added .movementX/Y data members to the MouseEvent type which is returned by mouse events: click, mousedown, mouseup, mouseover, mousemove, mouseout. No new mouse event types are introduced. Several, including Glenn Maynard, Robert O'Callahan have brought up points against this, suggesting different events and types. Let me enumerate options and my perceived pro/cons: Option A: as currently drafted: reuse existing mouse events, make minimal modification to MouseEvent to include movementX/Y valid regardles of mouse lock state, mouseover mouseout will never be fired under mouse lock. pro 1. minimal change to existing events and types pro 2. movementX/Y available even when mouse is not locked. Purely a convenience, apps could compute similar data by tracking the last position of a mouse event and computing deltas (resetting last position on a mouseover event). pro 3. event handling code that can operate both with and without mouse lock being acquired can do so easily using the same functions and taking the same MouseEvent data type input. e.g. rotateView(mouse_event) could be called from a mousemove event regardless of mouse lock state and still use .movementX/Y. con 1. When under mouse lock, clientX/Y screenX/Y will not offer any useful data - or worse implementations may not follow spec and provide misleading data of the actual hidden cursor location. con 2. Not all mouse events using MouseEvent are appropriate, mouseover and mouseout should be suppressed. Glen continued that it is probably cleaner to stop firing *all* mouse movement events entirely, as if the mouse isn't moving, and to use a separate mousedelta event when locked which only has deltaX and deltaY. Robert reiterated that point. Option B: make no use of or modification to any existing mouse event or the MouseEvent type. New mouse events created with unique names for click, down, up, move, e.g. mouselockclick, mouselockdown, mouselockup, mouselockmove. New events return a MouseLockEvent type derived from UIEvent that contains movementX/Y, ctrlKey, shiftKey, altKey, metaKey, button. The events and data types are completely mutually exclusive based on the state of mouse lock. pro 1. (dual of Option A con 1): clientX/Y screenX/Y do not exist under mouse lock. pro 2. (dual of Option A pro 3): Functions designed specifically to operate only under mouse lock, or only when not under mouse lock, are only triggered when appropriate. con 1. (dual of Option A pro 1): Larger change, introducing 4 new events and a new return type. con 2. (dual of Option A pro 2): movementX/Y must be calculated manually by apps when not under lock. Some hybrid could be considered, e.g. only make a new mouselockmove event but reuse the click/down/up events. I don't think that hybrid is worth discussing. My opinion: - Option A con 1 (.client .screen not useful, could be wrong): I don't perceive much damage here, and if implementation are making mistakes much more could go wrong elsewhere, but they don't get new data even with faulty client and screen data. - Option A con 2 (not all events returning MouseEvent are appropriate in mouse lock) isn't substantial... 4 of the events are and they happen to share a MouseEvent type. -
Re: Mouse Lock
Continuing the discussion of some of the open issues I'm aware of: -- If MouseEvent types are returned under mouse lock, what should .clientX/Y and screenX/Y be? -- Spec as drafted states they should be the center of the target element. That's likely a poor idea, as discussed, and a better solution is to freeze .clientX/Y and screenX/Y to whatever their last value from the user agent was. (Pending feedback, I will update spec to state that .clientX/Y .screenX/Y will report the same constant values of the last known cursor position just as mouse lock was entered. This will also be the location the system cursor will be placed at when mouse lock is exited) -- Modify MouseEvent, or a new type returned by new mouse events? -- Spec as drafted now has simply added .movementX/Y data members to the MouseEvent type which is returned by mouse events: click, mousedown, mouseup, mouseover, mousemove, mouseout. No new mouse event types are introduced. Several, including Glenn Maynard, Robert O'Callahan have brought up points against this, suggesting different events and types. Let me enumerate options and my perceived pro/cons: Option A: as currently drafted: reuse existing mouse events, make minimal modification to MouseEvent to include movementX/Y valid regardles of mouse lock state, mouseover mouseout will never be fired under mouse lock. pro 1. minimal change to existing events and types pro 2. movementX/Y available even when mouse is not locked. Purely a convenience, apps could compute similar data by tracking the last position of a mouse event and computing deltas (resetting last position on a mouseover event). pro 3. event handling code that can operate both with and without mouse lock being acquired can do so easily using the same functions and taking the same MouseEvent data type input. e.g. rotateView(mouse_event) could be called from a mousemove event regardless of mouse lock state and still use .movementX/Y. con 1. When under mouse lock, clientX/Y screenX/Y will not offer any useful data - or worse implementations may not follow spec and provide misleading data of the actual hidden cursor location. con 2. Not all mouse events using MouseEvent are appropriate, mouseover and mouseout should be suppressed. Glen continued that it is probably cleaner to stop firing *all* mouse movement events entirely, as if the mouse isn't moving, and to use a separate mousedelta event when locked which only has deltaX and deltaY. Robert reiterated that point. Option B: make no use of or modification to any existing mouse event or the MouseEvent type. New mouse events created with unique names for click, down, up, move, e.g. mouselockclick, mouselockdown, mouselockup, mouselockmove. New events return a MouseLockEvent type derived from UIEvent that contains movementX/Y, ctrlKey, shiftKey, altKey, metaKey, button. The events and data types are completely mutually exclusive based on the state of mouse lock. pro 1. (dual of Option A con 1): clientX/Y screenX/Y do not exist under mouse lock. pro 2. (dual of Option A pro 3): Functions designed specifically to operate only under mouse lock, or only when not under mouse lock, are only triggered when appropriate. con 1. (dual of Option A pro 1): Larger change, introducing 4 new events and a new return type. con 2. (dual of Option A pro 2): movementX/Y must be calculated manually by apps when not under lock. Some hybrid could be considered, e.g. only make a new mouselockmove event but reuse the click/down/up events. I don't think that hybrid is worth discussing. My opinion: - Option A con 1 (.client .screen not useful, could be wrong): I don't perceive much damage here, and if implementation are making mistakes much more could go wrong elsewhere, but they don't get new data even with faulty client and screen data. - Option A con 2 (not all events returning MouseEvent are appropriate in mouse lock) isn't substantial... 4 of the events are and they happen to share a MouseEvent type. - I see no logical argument leading us to one or the other option. I'd prefer the minimal change, the convenience of code that can use the same MouseEvent structure, and .movementX/Y even when not in mouse lock. (Pending feedback, I will leave spec at Option A but add clarification about e.g. mouseover/mouseout not being called) -- Separate targets for keyboard and mouse locked events? -- Robert: ... is there a need for mouse-lock motion events to go to one element while keyboard input goes elsewhere? ... I'm asking whether we can reuse the existing notion of the currently focused element and send mouse events to that element while the mouse is locked. I argue yes, there's a good cause for mouse events to a different target than keyboard input. Status quo enables typing into an input field while mousemove events are sent to different targets the mouse happens to be moving over. In mouse lock, consider a game with a 3D viewport controlled by mouse and a chat window with the active keyboard
Re: Mouse Lock
I agree with you that the deltas should be in a separate event. That would
also be the most logical because moving the cursor(corresponding to a
mousemove event) and physically moving your mouse are separate things. If
mousemove wasn't already used for cursor movement(mousemove imo should
have been called cursormoveor pointermove), that would be the most
appropriate name for this new mouse delta event. Unfortunately that isn't
the case so a different name has to be chosen. Maybe:
mousepositionchanged? This event should have the same security
restrictions as mousemove.
Constraining the cursor to an element could work almost the same as the
proposed lockMouse method. The only difference is that the cursor isn't
locked to the center, but limited to the bounds of some element so you get
something like this:
x = document.getElementById(some_element);
x.clipCursor(
function() {
x.freeCursor();
couldLock = true;
}
);
ClipCursor is the name Windows is using for this, but is could also be
called something like wallCursor.
2011/8/13 Glenn Maynard gl...@zewt.org
On Fri, Aug 12, 2011 at 7:50 PM, Vincent Scheib sch...@google.com wrote:
BTW, draft spec currently states, When mouse lock is enabled clientX,
clientY, screenX, and screenY must hold constant values as if the mouse were
located at the center of the mouse lock target element I chose this to
pick something concrete, but not all zeros which may be used as magic values
in JS code, etc. I'm not strong in this opinion.
Some things that come to mind:
- Delta movement may be received separately from cursor movement. For
example, an implementation may use WM_MOUSEMOVE to generate mousemove
events, and WM_INPUT to generate mouse delta events, as Klaas pointed out.
These window messages aren't defined relative to each other; a WM_INPUT
event may or may not be followed by a WM_MOUSEMOVE event. A mouse may
generate WM_INPUT events at 1kHz, but WM_MOUSEMOVE events may only be
generated at 60 or 120Hz.
This means that you'd either end up generating far more mousemove events,
eg. three events with delta information followed by a single event with
cursor information (possibly placing unwanted stress on unrelated mousemove
event handlers--this is a particular problem with Klaas's model, see below);
or discarding information, which also seems undesirable. Using a separate
event avoids this issue.
- The deltas may not be comparable to corresponding cursor movement. For
example, mouse cursors often have acceleration and snap-to-axis filters
applied. I think these filters aren't applied to the data received by
WM_INPUT. Is this a good thing or not?
It's probably a good thing for an FPS, to avoid snap-to-axis cursor filters
from affecting the viewport.
It's probably a bad thing for dragging Google Maps around. Having the map
drag at a different rate than the cursor normally moves may be strange.
(Using deltas for dragging is useful: you can continually drag the map,
without having to reposition the mouse cursor every time you reach the edge
of the screen.)
- If mousemove events when locked put clientX, etc. at some arbitrary
position, are mouseover events generated, as if the cursor had actually
moved there? If deltas do end up packed into mousemove events, I'd suggest
that these properties not change while locked: continue reporting the
last-generated position, as if the unseen cursor is stationary while
locked. When the mouse is unlocked, the cursor is back where it was
originally; no mouseover events are generated at all, as the cursor
(distinct from the mouse) never moved.
- Klaas suggests an entirely different approach. Instead of having a
separate delta mode, mouse deltas would always be available. Mouse lock
would be achieved by locking the mouse within a rectangle (a separate
feature entirely from mouse deltas), and hiding the cursor with CSS.
There's some appeal to having delta information always available, without
forcing the cursor to be hidden and locked to use them. It makes the
dragging (Google Maps) use case simpler: no actual locking is needed, as you
don't care if the mouse cursor stays within the window--so this use case
wouldn't be burdened by any security mechanisms required by locking. The
first two points above still apply here; using a separate mousedelta event
would still make sense in this model.
--
Glenn Maynard
Re: Mouse Lock
You could constrain the mouse cursor on linux using a combination of XQueryPointer and XWarpPointer. By moving the cursor when it hits the edge of an element. The same solution could be used for Mac where there also doesn't seem to be a direct method for constraining the mouse cursor. When searching for a solution for Mac the first thing I got was the question you posted on stack overflow a couple of hours ago :P. The linux equivalent of WM_INPUT mouse data would be /dev/input/mice. On Mac it appears to be a little more difficult, but I think it could be implemented by using what's described at: http://developer.apple.com/library/mac/#documentation/DeviceDrivers/Conceptual/USBBook/USBDeviceInterfaces/USBDevInterfaces.html . Concerning your question about the units of raw mouse input. The mouse sends the number of movements that happened since the last time the mouse was polled(by default I believe the polling rate for USB is 100Hz, but with some gaming mice you can increase this to 1000Hz). The maximum number of movements the mouse can record during that interval depends on the DPI. When you have a mouse of 2000dpi and the polling rate is 100Hz and you move it 1 inch in a certain direction, during that 1 second you get 100 times an average value of 2000/100 = 20. If the polling rate is increased to 1000Hz you would get 1000 times a value of 2 per second. This polling rate is the number of times per second the WM_INPUT is called for mouse movement on windows and on linux its the frequency at which /dev/input/mice is updated. In this case using raw input is the only way to take optimal advantage of high resolution gaming mice and because of this possible high frequency IMO it should be in a separate event as I pointed out in my mail an hour ago. 2011/8/23 Vincent Scheib sch...@google.com The 'component' version (separate orthogonal features for mouse deltas, constraining movement of mouse, and hiding cursor) seems preferable to the 'composite' api as currently drafted. That is, if implementable and specifiable. We need to answer a few questions (I'll research this, but if anyone already has pointers/answers to the information it will accelerate the process): Can the mouse be constrained on all platforms? If so, can it be constrained to arbitrary rectangles or only a window (this is a nice to have, would be good to have web apps able to confine the mouse to only a region of a page). - Windows: ClipCursor provides this. Does it work over remote desktop (just a nice to have? warping may not either)? - Linux: XGrabPointer can constrain to a window's area (is there a way to specify arbitrary rects?) - Mac: ? What are the units of .movementX/Y if we're reading 'raw' input device coordinates? Are they consistent across devices and platforms? Can we specify them? If not, I'm hesitate to cause web application developers to deal with calibrating to arbitrary and unknown units for .movementX/Y. - Windows WM_INPUT does not include ballistics (mouse acceleration), and that's OK/arguably preferable. But what are the units? Using the WM_MOUSEMOVE data and warping the cursor position allows us to specify precisely what the units are of .movementX/Y, in current spec draft, The members movementX and movementY must provide the change in position of the mouse, as if the values of e.g. clientX were stored between two subsequent mousemove events e1 and e2 and the difference taken e2.clientX-e1.clientX.
Re: Mouse Lock
On Thu, Aug 11, 2011 at 7:27 PM, Vincent Scheib sch...@google.com wrote: Re Rob: Is there a need to provide mouse-locking on a per-element basis? It seems to me it would be enough for mouse-locking to be per-DOM-window (or per-DOM-document) and deliver events to the focused element. This simplifies the model a little bit by not having to define new state for the mouse-locked element. Or is there a need for mouse-lock motion events to go to one element while keyboard input goes elsewhere? I may need to clarify the specification to state that there is only a single state of mouse lock global to the user agent. You may be suggesting that the MouseLockable interface be added to only the window and not to all elements? An argument was made that multiple elements may attempt a mouseLock, especially in pages composing / aggregating content. If so, it would be undesirable for an unlockMouse() call on one element to disrupt a lock held by another element. I will update the spec to explain that decision. If you were suggesting something else, I didn't follow you. But if the mouseLock API was only available on the Document object, then everyone would call it there and there would be no interference between aggregated content. As soon as the lock was granted, anyone that had requested it would be notified. / Jonas
Re: Mouse Lock
On Fri, Aug 12, 2011 at 9:53 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Thu, Aug 11, 2011 at 10:14 PM, Robert O'Callahan rob...@ocallahan.org wrote: If your implementation had to warp the mouse cursor on Windows to get accurate delta information, the mouse position in the existing mouse events would no longer be very meaningful and a new event type seemed more logical. But assuming Klaas is right, we no longer need to worry about this. It seems we can unconditionally add delta information to existing mouse events. So I withdraw that comment. I suspect that, while locked, we still don't actually want to expose the various x and y properties for the mouse. I agree with Vincent that the *other* mouseevent properties are all useful, though, and that the delta properties are really useful in non-mouselock situations. We should just zero all the position information. Even if we can switch all OSes to a delta mode, the position will be arbitrary and meaningless. This seems easier than making a new type of mouse event that exposes all of normal mouse events except the position, and ensuring that the two stay in sync when we add new info. If we expose delta information in all mouse events, which seems like it could be a good idea, then what is the usecase for the success callback for mouselock? I was under the impression that that was so that the page could start treating mousemove events differently, but if all mousemove events have deltas, then that won't be needed, no? / Jonas
Re: Mouse Lock
On Fri, Aug 12, 2011 at 1:19 PM, Jonas Sicking jo...@sicking.cc wrote: On Fri, Aug 12, 2011 at 9:53 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Thu, Aug 11, 2011 at 10:14 PM, Robert O'Callahan rob...@ocallahan.org wrote: If your implementation had to warp the mouse cursor on Windows to get accurate delta information, the mouse position in the existing mouse events would no longer be very meaningful and a new event type seemed more logical. But assuming Klaas is right, we no longer need to worry about this. It seems we can unconditionally add delta information to existing mouse events. So I withdraw that comment. I suspect that, while locked, we still don't actually want to expose the various x and y properties for the mouse. I agree with Vincent that the *other* mouseevent properties are all useful, though, and that the delta properties are really useful in non-mouselock situations. We should just zero all the position information. Even if we can switch all OSes to a delta mode, the position will be arbitrary and meaningless. This seems easier than making a new type of mouse event that exposes all of normal mouse events except the position, and ensuring that the two stay in sync when we add new info. If we expose delta information in all mouse events, which seems like it could be a good idea, then what is the usecase for the success callback for mouselock? I was under the impression that that was so that the page could start treating mousemove events differently, but if all mousemove events have deltas, then that won't be needed, no? No, it's still definitely needed. You can't do an FPS with non-locked deltas; the user will end up moving their mouse off the screen. The use-cases for delta-without-mouselock are pretty separate from those for delta-within-mouselock. ~TJ
Re: Mouse Lock
On Fri, Aug 12, 2011 at 1:54 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Fri, Aug 12, 2011 at 1:19 PM, Jonas Sicking jo...@sicking.cc wrote: On Fri, Aug 12, 2011 at 9:53 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Thu, Aug 11, 2011 at 10:14 PM, Robert O'Callahan rob...@ocallahan.org wrote: If your implementation had to warp the mouse cursor on Windows to get accurate delta information, the mouse position in the existing mouse events would no longer be very meaningful and a new event type seemed more logical. But assuming Klaas is right, we no longer need to worry about this. It seems we can unconditionally add delta information to existing mouse events. So I withdraw that comment. I suspect that, while locked, we still don't actually want to expose the various x and y properties for the mouse. I agree with Vincent that the *other* mouseevent properties are all useful, though, and that the delta properties are really useful in non-mouselock situations. We should just zero all the position information. Even if we can switch all OSes to a delta mode, the position will be arbitrary and meaningless. This seems easier than making a new type of mouse event that exposes all of normal mouse events except the position, and ensuring that the two stay in sync when we add new info. If we expose delta information in all mouse events, which seems like it could be a good idea, then what is the usecase for the success callback for mouselock? I was under the impression that that was so that the page could start treating mousemove events differently, but if all mousemove events have deltas, then that won't be needed, no? No, it's still definitely needed. You can't do an FPS with non-locked deltas; the user will end up moving their mouse off the screen. The use-cases for delta-without-mouselock are pretty separate from those for delta-within-mouselock. Sure, I wasn't saying that mouselock wasn't needed. I was asking what the use case for the 'success' callback to the mouseLock was. / Jonas
Re: Mouse Lock
On Fri, Aug 12, 2011 at 2:06 PM, Jonas Sicking jo...@sicking.cc wrote: On Fri, Aug 12, 2011 at 1:54 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Fri, Aug 12, 2011 at 1:19 PM, Jonas Sicking jo...@sicking.cc wrote: If we expose delta information in all mouse events, which seems like it could be a good idea, then what is the usecase for the success callback for mouselock? I was under the impression that that was so that the page could start treating mousemove events differently, but if all mousemove events have deltas, then that won't be needed, no? No, it's still definitely needed. You can't do an FPS with non-locked deltas; the user will end up moving their mouse off the screen. The use-cases for delta-without-mouselock are pretty separate from those for delta-within-mouselock. Sure, I wasn't saying that mouselock wasn't needed. I was asking what the use case for the 'success' callback to the mouseLock was. So you can tell that you're locked. Without that assurance, you can't do a game using WASD+mouselook, because if the mouse isn't locked the user will end up scrolling out of your active area. You'll have to make sure the mouse is actually locked *somehow*; might as well be via a success callback. ~TJ
Re: Mouse Lock
On Fri, Aug 12, 2011 at 2:12 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Fri, Aug 12, 2011 at 2:06 PM, Jonas Sicking jo...@sicking.cc wrote: On Fri, Aug 12, 2011 at 1:54 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Fri, Aug 12, 2011 at 1:19 PM, Jonas Sicking jo...@sicking.cc wrote: If we expose delta information in all mouse events, which seems like it could be a good idea, then what is the usecase for the success callback for mouselock? I was under the impression that that was so that the page could start treating mousemove events differently, but if all mousemove events have deltas, then that won't be needed, no? No, it's still definitely needed. You can't do an FPS with non-locked deltas; the user will end up moving their mouse off the screen. The use-cases for delta-without-mouselock are pretty separate from those for delta-within-mouselock. Sure, I wasn't saying that mouselock wasn't needed. I was asking what the use case for the 'success' callback to the mouseLock was. So you can tell that you're locked. Without that assurance, you can't do a game using WASD+mouselook, because if the mouse isn't locked the user will end up scrolling out of your active area. You'll have to make sure the mouse is actually locked *somehow*; might as well be via a success callback. And if the user doesn't approve the lock you do what? Not let them play your game? / Jonas
Re: Mouse Lock
On Fri, Aug 12, 2011 at 3:14 PM, Jonas Sicking jo...@sicking.cc wrote: And if the user doesn't approve the lock you do what? Not let them play your game? Maybe. It really is impossible to play a game with a control scheme based on WASD+mouselook if you can't get a lock. Alternately, you can switch to a different (less good) control scheme if you can't acquire a lock. This is not acceptable behavior for the default case, though. WASD+mouselook is a very entrenched and natural control scheme; anything else will be much harder to use. The difference isn't always quite this extreme; sometimes the experience is only moderately degraded, instead of majorly, if you can't get a lock. For example, edge scrolling in StarCraft, Civilization, or similar games is really useful. You could play the game without it, it would just be annoying. (The gamedev would have to implement some other way to do scrolling, either a modifier key + arrows, or dedicating the drag action to scrolling, or similar.) ~TJ
Re: Mouse Lock
On Fri, Aug 12, 2011 at 7:50 PM, Vincent Scheib sch...@google.com wrote: BTW, draft spec currently states, When mouse lock is enabled clientX, clientY, screenX, and screenY must hold constant values as if the mouse were located at the center of the mouse lock target element I chose this to pick something concrete, but not all zeros which may be used as magic values in JS code, etc. I'm not strong in this opinion. Some things that come to mind: - Delta movement may be received separately from cursor movement. For example, an implementation may use WM_MOUSEMOVE to generate mousemove events, and WM_INPUT to generate mouse delta events, as Klaas pointed out. These window messages aren't defined relative to each other; a WM_INPUT event may or may not be followed by a WM_MOUSEMOVE event. A mouse may generate WM_INPUT events at 1kHz, but WM_MOUSEMOVE events may only be generated at 60 or 120Hz. This means that you'd either end up generating far more mousemove events, eg. three events with delta information followed by a single event with cursor information (possibly placing unwanted stress on unrelated mousemove event handlers--this is a particular problem with Klaas's model, see below); or discarding information, which also seems undesirable. Using a separate event avoids this issue. - The deltas may not be comparable to corresponding cursor movement. For example, mouse cursors often have acceleration and snap-to-axis filters applied. I think these filters aren't applied to the data received by WM_INPUT. Is this a good thing or not? It's probably a good thing for an FPS, to avoid snap-to-axis cursor filters from affecting the viewport. It's probably a bad thing for dragging Google Maps around. Having the map drag at a different rate than the cursor normally moves may be strange. (Using deltas for dragging is useful: you can continually drag the map, without having to reposition the mouse cursor every time you reach the edge of the screen.) - If mousemove events when locked put clientX, etc. at some arbitrary position, are mouseover events generated, as if the cursor had actually moved there? If deltas do end up packed into mousemove events, I'd suggest that these properties not change while locked: continue reporting the last-generated position, as if the unseen cursor is stationary while locked. When the mouse is unlocked, the cursor is back where it was originally; no mouseover events are generated at all, as the cursor (distinct from the mouse) never moved. - Klaas suggests an entirely different approach. Instead of having a separate delta mode, mouse deltas would always be available. Mouse lock would be achieved by locking the mouse within a rectangle (a separate feature entirely from mouse deltas), and hiding the cursor with CSS. There's some appeal to having delta information always available, without forcing the cursor to be hidden and locked to use them. It makes the dragging (Google Maps) use case simpler: no actual locking is needed, as you don't care if the mouse cursor stays within the window--so this use case wouldn't be burdened by any security mechanisms required by locking. The first two points above still apply here; using a separate mousedelta event would still make sense in this model. -- Glenn Maynard
Re: Mouse Lock
You actually can get mouse delta info in windows using raw WM_INPUT data see: http://msdn.microsoft.com/en-us/library/ee418864(VS.85).aspx. This is also the only way to take advantage of 400dpi mice, which is useful for FPS games. As for mouse locking isn't that a completely distinct feature from getting mouse delta information? For example in full-screen mode (and only using one screen) there is no need for mouse lock when you always can get mouse delta (because the mouse can't leave the screen). The only problem that remains is when in multi-screen and/or non-full-screen-mode, the mouse cursor can go outside the game viewport. This is something that could be solved by mouse lock. But wouldn't it be better (as previously suggested in this thread) to implement walling of the mouse cursor by limiting the mouse cursor to the bounds of a targeted element. This is both useful for FPS-type games and RTS's. FPS's can hide the mouse cursor using CSS and don't have to worry about mouse events being fired outside the element or window. RTS-games just could use the available mouse cursor without needing to hide it (they wouldn't even need the mouse delta that way). It appears to me that mouse lock is just a workaround/hack that originated from the assumption that you can't get mouse delta info in Windows. Isn't it always better to first make an ideal design before looking at platform limitations that possibly could undermine that design? So to summarize all the above, design to separate features: 1. an event for getting mouse delta 2. an API for walling the mouse to an element. Klaas Heidstra A few comments: Is there a need to provide mouse-locking on a per-element basis? It seems to me it would be enough for mouse-locking to be per-DOM-window (or per-DOM-document) and deliver events to the focused element. This simplifies the model a little bit by not having to define new state for the mouse-locked element. Or is there a need for mouse-lock motion events to go to one element while keyboard input goes elsewhere? As was suggested earlier in this thread, I think given we're not displaying the normal mouse cursor and in fact warping the system mouse cursor to the center of the screen in some implementations, we shouldn't deliver normal mouse-move events. Instead, while mouselock is active, we should deliver a new kind of mouse motion event, which carries the delta properties. If you do that, then hopefully you don't need a failure or success callback. Your app should just be able to handle both kinds of mouse motion events. I'm not really sure how touch events fit into this. Unlike mouse events, touch events always correspond to a position on the screen, so the delta information isn't as useful. (Or do some platforms detect touches outside the screen?) Maybe the only thing you need to do for touch events is to capture them to the focused element. In many of your use cases, it's OK to automatically release the mouse-lock on mouse-up. If you automatically release on mouse-up, the security issues are far less serious. So I think it would be a good idea to allow applications to accept that behavior via the API. A lot of this would be much simpler if we could somehow get mouse delta information from all platforms (Windows!) without having to warp the cursor :-(. Has research definitively ruled out achieving that by any combination of hacks? Rob -- If we claim to be without sin, we deceive ourselves and the truth is not in us. If we confess our sins, he is faithful and just and will forgive us our sins and purify us from all unrighteousness. If we claim we have not sinned, we make him out to be a liar and his word is not in us. [1 John 1:8-10]
Re: Mouse Lock
Re Rob: Is there a need to provide mouse-locking on a per-element basis? It seems to me it would be enough for mouse-locking to be per-DOM-window (or per-DOM-document) and deliver events to the focused element. This simplifies the model a little bit by not having to define new state for the mouse-locked element. Or is there a need for mouse-lock motion events to go to one element while keyboard input goes elsewhere? I may need to clarify the specification to state that there is only a single state of mouse lock global to the user agent. You may be suggesting that the MouseLockable interface be added to only the window and not to all elements? An argument was made that multiple elements may attempt a mouseLock, especially in pages composing / aggregating content. If so, it would be undesirable for an unlockMouse() call on one element to disrupt a lock held by another element. I will update the spec to explain that decision. If you were suggesting something else, I didn't follow you. As was suggested earlier in this thread, I think given we're not displaying the normal mouse cursor and in fact warping the system mouse cursor to the center of the screen in some implementations, we shouldn't deliver normal mouse-move events. Instead, while mouselock is active, we should deliver a new kind of mouse motion event, which carries the delta properties. If you do that, then hopefully you don't need a failure or success callback. Your app should just be able to handle both kinds of mouse motion events. The argument is that mouse handling code can be simplified by always handling the same MouseEvent structures. The six modifier key and button state members and event target are still desired. When not under lock the movement members are still useful - and are tedious recreate by adding last position and edge case handling in mouseleave/enter. The success or failure events are desired to be once per call to lock, for appropriate application response. Triggering that response logic on every mouse event, or detecting the edge of that signal seems more complicated. The callbacks are also low cost, they can be inline anonymous functions or references.. they don't take special event handler registration. What am I missing in the value of having new event types? I'm not really sure how touch events fit into this. Unlike mouse events, touch events always correspond to a position on the screen, so the delta information isn't as useful. (Or do some platforms detect touches outside the screen?) Maybe the only thing you need to do for touch events is to capture them to the focused element. The motivation for touch events is to make capturing to a specific element easy, including any other window content out of an apps control (e.g. if embedded via iframe) and or any user agent UI that disables its interaction when in mouse lock to offer a better app experience. In many of your use cases, it's OK to automatically release the mouse-lock on mouse-up. If you automatically release on mouse-up, the security issues are far less serious. So I think it would be a good idea to allow applications to accept that behavior via the API. You mean you'd prefer the API have an option specified in lockMouse(...) to cause the mouse to be automatically unlocked when a mouse button is let up if the lock occurred during an event for that button being pressed? Under current API draft an application specifies that with a call to unlockMouse at a button up event. I'm hesitant to add it, since it seems relevant only for the lower priority use cases. The killer use case here is first person controls or full mouse control applications. The mouse capture API satisfies most of the use cases you're referring to, with the only limitation being loss of movement data when the cursor hits a screen edge. I don't think we should complicate this API for those nitch use cases when the functionality is available to them in this API without great effort. A lot of this would be much simpler if we could somehow get mouse delta information from all platforms (Windows!) without having to warp the cursor :-(. Has research definitively ruled out achieving that by any combination of hacks? Rob Deltas alone don't satisfy the key use cases. We must prevent errant clicks. Re: Klaas Heidstra You actually can get mouse delta info in windows using raw WM_INPUT data see: http://msdn.microsoft.com/en-us/library/ee418864(VS.85).aspx. This is also the only way to take advantage of 400dpi mice, which is useful for FPS games. Thanks for implementation tip! As for mouse locking isn't that a completely distinct feature from getting mouse delta information? For example in full-screen mode (and only using one screen) there is no need for mouse lock when you always can get mouse delta (because the mouse can't leave the screen). It is the combination of mouse capture, hidden cursor, deltas being provided with no limits of screen
Re: Mouse Lock
On Fri, Aug 12, 2011 at 2:27 PM, Vincent Scheib sch...@google.com wrote: Re Rob: Is there a need to provide mouse-locking on a per-element basis? It seems to me it would be enough for mouse-locking to be per-DOM-window (or per-DOM-document) and deliver events to the focused element. This simplifies the model a little bit by not having to define new state for the mouse-locked element. Or is there a need for mouse-lock motion events to go to one element while keyboard input goes elsewhere? I may need to clarify the specification to state that there is only a single state of mouse lock global to the user agent. You may be suggesting that the MouseLockable interface be added to only the window and not to all elements? An argument was made that multiple elements may attempt a mouseLock, especially in pages composing / aggregating content. If so, it would be undesirable for an unlockMouse() call on one element to disrupt a lock held by another element. I will update the spec to explain that decision. If you were suggesting something else, I didn't follow you. I'm asking whether we can reuse the existing notion of the currently focused element and send mouse events to that element while the mouse is locked. As was suggested earlier in this thread, I think given we're not displaying the normal mouse cursor and in fact warping the system mouse cursor to the center of the screen in some implementations, we shouldn't deliver normal mouse-move events. Instead, while mouselock is active, we should deliver a new kind of mouse motion event, which carries the delta properties. If you do that, then hopefully you don't need a failure or success callback. Your app should just be able to handle both kinds of mouse motion events. The argument is that mouse handling code can be simplified by always handling the same MouseEvent structures. The six modifier key and button state members and event target are still desired. When not under lock the movement members are still useful - and are tedious recreate by adding last position and edge case handling in mouseleave/enter. The success or failure events are desired to be once per call to lock, for appropriate application response. Triggering that response logic on every mouse event, or detecting the edge of that signal seems more complicated. The callbacks are also low cost, they can be inline anonymous functions or references.. they don't take special event handler registration. What am I missing in the value of having new event types? If your implementation had to warp the mouse cursor on Windows to get accurate delta information, the mouse position in the existing mouse events would no longer be very meaningful and a new event type seemed more logical. But assuming Klaas is right, we no longer need to worry about this. It seems we can unconditionally add delta information to existing mouse events. So I withdraw that comment. I'm not really sure how touch events fit into this. Unlike mouse events, touch events always correspond to a position on the screen, so the delta information isn't as useful. (Or do some platforms detect touches outside the screen?) Maybe the only thing you need to do for touch events is to capture them to the focused element. The motivation for touch events is to make capturing to a specific element easy, including any other window content out of an apps control (e.g. if embedded via iframe) and or any user agent UI that disables its interaction when in mouse lock to offer a better app experience. OK. In many of your use cases, it's OK to automatically release the mouse-lock on mouse-up. If you automatically release on mouse-up, the security issues are far less serious. So I think it would be a good idea to allow applications to accept that behavior via the API. You mean you'd prefer the API have an option specified in lockMouse(...) to cause the mouse to be automatically unlocked when a mouse button is let up if the lock occurred during an event for that button being pressed? Under current API draft an application specifies that with a call to unlockMouse at a button up event. I'm hesitant to add it, since it seems relevant only for the lower priority use cases. The killer use case here is first person controls or full mouse control applications. The mouse capture API satisfies most of the use cases you're referring to, with the only limitation being loss of movement data when the cursor hits a screen edge. And if Klaas is right, we don't even lose that. Right now I'm thinking about full-screen apps. It seems to me that if we add delta information to all mouse events, then full-screen apps don't need anything else from your MouseLock API. Correct? If so, I think it might be a good idea to make the delta information API a separate proposal, and then adjust the use-cases for MouseLock to exclude cases that only make sense for full-screen apps, and exclude cases that can be satisfied with
Re: Mouse Lock
A few comments: Is there a need to provide mouse-locking on a per-element basis? It seems to me it would be enough for mouse-locking to be per-DOM-window (or per-DOM-document) and deliver events to the focused element. This simplifies the model a little bit by not having to define new state for the mouse-locked element. Or is there a need for mouse-lock motion events to go to one element while keyboard input goes elsewhere? As was suggested earlier in this thread, I think given we're not displaying the normal mouse cursor and in fact warping the system mouse cursor to the center of the screen in some implementations, we shouldn't deliver normal mouse-move events. Instead, while mouselock is active, we should deliver a new kind of mouse motion event, which carries the delta properties. If you do that, then hopefully you don't need a failure or success callback. Your app should just be able to handle both kinds of mouse motion events. I'm not really sure how touch events fit into this. Unlike mouse events, touch events always correspond to a position on the screen, so the delta information isn't as useful. (Or do some platforms detect touches outside the screen?) Maybe the only thing you need to do for touch events is to capture them to the focused element. In many of your use cases, it's OK to automatically release the mouse-lock on mouse-up. If you automatically release on mouse-up, the security issues are far less serious. So I think it would be a good idea to allow applications to accept that behavior via the API. A lot of this would be much simpler if we could somehow get mouse delta information from all platforms (Windows!) without having to warp the cursor :-(. Has research definitively ruled out achieving that by any combination of hacks? Rob -- If we claim to be without sin, we deceive ourselves and the truth is not in us. If we confess our sins, he is faithful and just and will forgive us our sins and purify us from all unrighteousness. If we claim we have not sinned, we make him out to be a liar and his word is not in us. [1 John 1:8-10]
Re: Mouse Lock
On Mon, Jun 27, 2011 at 6:17 PM, Glenn Maynard gl...@zewt.org wrote: On Mon, Jun 27, 2011 at 8:59 PM, Gregg Tavares (wrk) g...@google.comwrote: As far as I know if a game wants to limit movement of the mouse inside a window they just mouselock and display their own mouse pointer. The original is hidden and their pointer logic uses the deltas to move their software mouse pointer. Rendering a cursor manually is a non-option. It invariably results in a laggy mouse cursor, even in native applications. Even a single extra frame of latency makes a mouse painful to use. I beg to differ. Nearly every game that has a mouse renders a mouse cursor manually. At least all the ones I've played. I quick search on youtube for rts gameplay shows this is true. The point is, 1 it's unrelated to mouselock, 2 you can implement it on top of mouselock for now. If that's too slow for your app and there's a huge need something else can be added later. But again, this seems like an unrelated feature. -- Glenn Maynard
Re: Mouse Lock
On Tue, Jun 28, 2011 at 3:54 AM, Gregg Tavares (wrk) g...@google.comwrote: I beg to differ. Nearly every game that has a mouse renders a mouse cursor manually. At least all the ones I've played. I quick search on youtube for rts gameplay shows this is true. I've seen plenty of both; drawing the cursor manually gives a painful, delayed cursor, resulting in a stressful UI. The point is, 1 it's unrelated to mouselock As I said. :) If people want it I'd suggest proposing it separately. -- Glenn Maynard
Re: Mouse Lock
On Fri, Jun 24, 2011 at 10:58 AM, Aryeh Gregor simetrical+...@gmail.comwrote: On Wed, Jun 22, 2011 at 5:20 AM, Simon Pieters sim...@opera.com wrote: On Tue, 21 Jun 2011 00:43:52 +0200, Aryeh Gregor simetrical+...@gmail.com wrote: There's a middle ground here: you can lock the mouse to the window, but not completely. That is, if the user moves the mouse to the edge, it remains inside, but if they move it fast enough it escapes. This is enough to stop the window from accidentally losing focus when you're trying to click on something near the edge of the screen, but it lets you easily get outside the window if you actually want to. IIRC, Wine does this in windowed mode. Of course, it might not be suitable for games that want to hide the cursor, like FPSes, but it might be a possible fallback if the browser doesn't trust the site enough for whatever reason to let it fully lock the mouse. This seems weird. When would you use this middle ground? Would users understand it? Also, as you say, totally inappropriate for FPS games. Well, the time when I noticed it in Wine is when I was running some kind of isometric RPG or strategy game or something, and had to run in windowed mode because it was buggy in fullscreen. In these games you have a map, and you scroll around on the map by moving the mouse to the edge of the screen. You do have a visible mouse cursor, but you don't want it to leave the window because then you have to position it pixel-perfect to get the window to scroll, instead of just slamming it against the side. Of course, you could just trap the mouse for real in this case as well. In practice that might be fine. Also, it occurs to me that the author could always make the cursor transparent if they wanted to confuse the user, and the user might not think to move it quicker to get it out even if they could see it (although it's an intuitive thing to try). So it might not be a security advantage at all relative to actually locking the cursor. But this does highlight the fact that we probably want to support mouse-locking that doesn't hide the cursor also, for this kind of mouse-based scrolling. In that case, though, the coordinates and mouse events should behave just like regular. If the user presses the cursor up against the side of the screen and it visually stops moving, no mousemove events should be fired even if the user does keep moving the actual mouse. The application would then want to check the cursor's location instead of listening for events. As far as I know if a game wants to limit movement of the mouse inside a window they just mouselock and display their own mouse pointer. The original is hidden and their pointer logic uses the deltas to move their software mouse pointer. That's a simpler solution than adding stuff to the API for this specific case.
Re: Mouse Lock
On Mon, Jun 27, 2011 at 8:59 PM, Gregg Tavares (wrk) g...@google.comwrote: As far as I know if a game wants to limit movement of the mouse inside a window they just mouselock and display their own mouse pointer. The original is hidden and their pointer logic uses the deltas to move their software mouse pointer. Rendering a cursor manually is a non-option. It invariably results in a laggy mouse cursor, even in native applications. Even a single extra frame of latency makes a mouse painful to use. But again, this seems like an unrelated feature. -- Glenn Maynard
Re: Mouse Lock
I recently saw what appeared to be the AG group complaining that the html WG didn't care to specify Accessibility bits even though W3 policy requires considering both internationalization and accessibility. I know that we like to innovate and let everyone else backfill the missing pieces later, but perhaps that isn't a wonderful approach. On the topic of devices without keyboards, I picked up a MeeGo 1.2 tablet in May. It has no keyboard, and no useful edges. It doesn't have a right click feature, and it's perfectly reasonable for a web application to demand to detect gestures and long presses. It has one real button: Power - which turns the device completely off, after which it takes an incredibly long time to boot. Not to mention that a user might not appreciate losing any other unsaved data. It does have rotation detection (which barely works), again, something which web applications are free to want to handle themselves. It also has single sensor which is incredibly quirky and easy to misplace or overlook which barely functions for task switching. You could claim that it's sufficient as a way out, but it makes me uncomfortable. Plus there's no way for a UI to explain to the user how to press it; it isn't usefully labeled. On 6/23/11, Charles Pritchard ch...@jumis.com wrote: timeless, I agree, it'd be nice to know. I'd really like to see this put toward the AG (Accessibility Guidelines) people, as they're the ones who follow this kind of things. It's absolutely the case that a user needs an ability to escape from the mouse lock context, and to have one which lies outside of any processing handled by the untrusted scripting environment. The *AG practices are always the first to document these needs. Requiring keyboard for something that locks a mouse is not acceptable; a user must be able to release their pointer device without requiring a keyboard. The same applies for keyboard locks. Assistive technologies (AT) have their place in this, and can give some leeway to the user agent (UA), as long as the UA provides enough semantics to allow the AT to handle this kind of work. Again, these issues are always brought up by AG documents. I'll give you two examples for pointer device escape: 1) Right click/context menu; always works. 2) Click and hold; X number of seconds could pop up a context menu. 3) extreme coordinate and hold; x seconds on an extreme coordinate pops up a menu. On the extreme-coordinate; an input device which may only signal changes in x/y positioning may still be used to activate UA menus. Developers are by all practical purposes, required by existing standards and technologies, to leave space on the edges of the screen. Though this is mostly related to safe areas on televisions, the principle applies beyond that, and is de facto required by existing full screen semantics [wherein the top extremes create a dialog to escape]. So, by luck of history; extreme coordinates are encouraged and in practice, required as a means of escaping mouse lock. There are certainly cases where extreme coordinates could be useful to an application. Those corner cases will have to be thought about, by those implementing such apps. All the best, -Charles On 6/23/2011 6:29 PM, timeless wrote: And what if the device in question is just a touchscreen with no keyboard, mouse or hardware buttons? On 6/20/11, Tab Atkins Jr.jackalm...@gmail.com wrote: On Mon, Jun 20, 2011 at 3:03 PM, Olli Pettayolli.pet...@helsinki.fi wrote: On 06/21/2011 12:25 AM, Tab Atkins Jr. wrote: The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. And how would user unlock when some evil sites locks the mouse? Could you give some concrete example about It's probably also useful to instruct the user how to release the lock. I'm assuming that the browser reserves some logical key (like Esc) for releasing things like this, and communicates this in the overlay message. -- Sent from my mobile device
Re: Mouse Lock
And what if the device in question is just a touchscreen with no keyboard, mouse or hardware buttons? From the draft spec: Touch devices may also choose to reserve a portion of the touch interface for an unlock gesture. Mouse lock seems irrelevant on a touchscreen... I've added clarification to the draft spec in the use case section, Touch screen device input All the application use cases are relevant on touch screen devices as well. A user should be permitted to make large gestures that are interpreted as movement deltas, without concern over the absolute positions of the touch points. Other UI elements from the user agent and system should be suppressed while under mouse lock. Absolute position of touch points should always be available, unlike mouse input where the concept of the cursor is removed. On a touch screen the absolute positions will always be relevant. Devices with no keyboard must provide escape from lock via either hard buttons or reserved screen space for an escape gesture.
Re: Mouse Lock
On Fri, Jun 24, 2011 at 12:00 PM, Vincent Scheib sch...@google.com wrote: I've added clarification to the draft spec in the use case section, Touch screen device input All the application use cases are relevant on touch screen devices as well. A user should be permitted to make large gestures that are interpreted as movement deltas, without concern over the absolute positions of the touch points. Other UI elements from the user agent and system should be suppressed while under mouse lock. Absolute position of touch points should always be available, unlike mouse input where the concept of the cursor is removed. On a touch screen the absolute positions will always be relevant. I don't believe it makes sense to have a mouse lock mode on a touchscreen which causes touching UI elements to instead be sent as mouse events to the window. That would be very confusing. Touching in-window and then dragging out-of-window on a touchscreen is useful, but you should be able to do that with regular mouse (or touch) events already. It can be used in fullscreen mode, where there are no UI elements, but in that case it's not necessary--you can just use mouse events directly. The main reason for mouse lock--the issue of the mouse being moved beyond the edge of the screen--doesn't apply to touchscreens (you can't touch outside the screen). -- Glenn Maynard
Re: Mouse Lock
On Wed, Jun 22, 2011 at 5:20 AM, Simon Pieters sim...@opera.com wrote: On Tue, 21 Jun 2011 00:43:52 +0200, Aryeh Gregor simetrical+...@gmail.com wrote: There's a middle ground here: you can lock the mouse to the window, but not completely. That is, if the user moves the mouse to the edge, it remains inside, but if they move it fast enough it escapes. This is enough to stop the window from accidentally losing focus when you're trying to click on something near the edge of the screen, but it lets you easily get outside the window if you actually want to. IIRC, Wine does this in windowed mode. Of course, it might not be suitable for games that want to hide the cursor, like FPSes, but it might be a possible fallback if the browser doesn't trust the site enough for whatever reason to let it fully lock the mouse. This seems weird. When would you use this middle ground? Would users understand it? Also, as you say, totally inappropriate for FPS games. Well, the time when I noticed it in Wine is when I was running some kind of isometric RPG or strategy game or something, and had to run in windowed mode because it was buggy in fullscreen. In these games you have a map, and you scroll around on the map by moving the mouse to the edge of the screen. You do have a visible mouse cursor, but you don't want it to leave the window because then you have to position it pixel-perfect to get the window to scroll, instead of just slamming it against the side. Of course, you could just trap the mouse for real in this case as well. In practice that might be fine. Also, it occurs to me that the author could always make the cursor transparent if they wanted to confuse the user, and the user might not think to move it quicker to get it out even if they could see it (although it's an intuitive thing to try). So it might not be a security advantage at all relative to actually locking the cursor. But this does highlight the fact that we probably want to support mouse-locking that doesn't hide the cursor also, for this kind of mouse-based scrolling. In that case, though, the coordinates and mouse events should behave just like regular. If the user presses the cursor up against the side of the screen and it visually stops moving, no mousemove events should be fired even if the user does keep moving the actual mouse. The application would then want to check the cursor's location instead of listening for events.
Re: Mouse Lock
And what if the device in question is just a touchscreen with no keyboard, mouse or hardware buttons? On 6/20/11, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jun 20, 2011 at 3:03 PM, Olli Pettay olli.pet...@helsinki.fi wrote: On 06/21/2011 12:25 AM, Tab Atkins Jr. wrote: The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. And how would user unlock when some evil sites locks the mouse? Could you give some concrete example about It's probably also useful to instruct the user how to release the lock. I'm assuming that the browser reserves some logical key (like Esc) for releasing things like this, and communicates this in the overlay message. ~TJ -- Sent from my mobile device
Re: Mouse Lock
timeless, I agree, it'd be nice to know. I'd really like to see this put toward the AG (Accessibility Guidelines) people, as they're the ones who follow this kind of things. It's absolutely the case that a user needs an ability to escape from the mouse lock context, and to have one which lies outside of any processing handled by the untrusted scripting environment. The *AG practices are always the first to document these needs. Requiring keyboard for something that locks a mouse is not acceptable; a user must be able to release their pointer device without requiring a keyboard. The same applies for keyboard locks. Assistive technologies (AT) have their place in this, and can give some leeway to the user agent (UA), as long as the UA provides enough semantics to allow the AT to handle this kind of work. Again, these issues are always brought up by AG documents. I'll give you two examples for pointer device escape: 1) Right click/context menu; always works. 2) Click and hold; X number of seconds could pop up a context menu. 3) extreme coordinate and hold; x seconds on an extreme coordinate pops up a menu. On the extreme-coordinate; an input device which may only signal changes in x/y positioning may still be used to activate UA menus. Developers are by all practical purposes, required by existing standards and technologies, to leave space on the edges of the screen. Though this is mostly related to safe areas on televisions, the principle applies beyond that, and is de facto required by existing full screen semantics [wherein the top extremes create a dialog to escape]. So, by luck of history; extreme coordinates are encouraged and in practice, required as a means of escaping mouse lock. There are certainly cases where extreme coordinates could be useful to an application. Those corner cases will have to be thought about, by those implementing such apps. All the best, -Charles On 6/23/2011 6:29 PM, timeless wrote: And what if the device in question is just a touchscreen with no keyboard, mouse or hardware buttons? On 6/20/11, Tab Atkins Jr.jackalm...@gmail.com wrote: On Mon, Jun 20, 2011 at 3:03 PM, Olli Pettayolli.pet...@helsinki.fi wrote: On 06/21/2011 12:25 AM, Tab Atkins Jr. wrote: The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. And how would user unlock when some evil sites locks the mouse? Could you give some concrete example about It's probably also useful to instruct the user how to release the lock. I'm assuming that the browser reserves some logical key (like Esc) for releasing things like this, and communicates this in the overlay message.
Re: Mouse Lock
On Thu, Jun 23, 2011 at 10:10 PM, Charles Pritchard ch...@jumis.com wrote: There are certainly cases where extreme coordinates could be useful to an application. Those corner cases will have to be thought about, by those implementing such apps. Moving the cursor to the top of the screen doesn't make sense when there's no cursor. The mouse no longer has a position onscreen; mouse lock essentially turns it into a delta-based input device. (FWIW, I've always found the top-edge mechanic used by browsers to exit fullscreen to be a bad case of best we could think of. It interferes badly with a ton of common UI mechanisms: menus, toolbars, tabs, address bars, and so on.) 2) Click and hold; X number of seconds could pop up a context menu. Hold escape for 3 seconds would probably work well, with a fade-in keep holding escape to exit fullscreen overlay while holding, so the user knows something's happening. I try to avoid do-something-and-wait interfaces, but it's reasonable for an escape mechanism. This also avoids eating the escape key entirely, so it's still available to applications, though of course browsers could choose a different key. And what if the device in question is just a touchscreen with no keyboard, mouse or hardware buttons? Mouse lock seems irrelevant on a touchscreen... -- Glenn Maynard
Re: Mouse Lock
On Tue, 21 Jun 2011 00:43:52 +0200, Aryeh Gregor simetrical+...@gmail.com wrote: On Mon, Jun 20, 2011 at 1:48 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: A model which I suggested privately, and which I believe others have suggested publicly, is this: 1. While fullscreen is enabled, you can lock the mouse to the fullscreened element without a prompt or persistent message. A temporary message may still be shown. The lock is automatically released if the user exits fullscreen. 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. 3. Otherwise, any attempt to lock the mouse triggers a permissions prompt, and while the lock is active a persistent message is shown. There's a middle ground here: you can lock the mouse to the window, but not completely. That is, if the user moves the mouse to the edge, it remains inside, but if they move it fast enough it escapes. This is enough to stop the window from accidentally losing focus when you're trying to click on something near the edge of the screen, but it lets you easily get outside the window if you actually want to. IIRC, Wine does this in windowed mode. Of course, it might not be suitable for games that want to hide the cursor, like FPSes, but it might be a possible fallback if the browser doesn't trust the site enough for whatever reason to let it fully lock the mouse. This seems weird. When would you use this middle ground? Would users understand it? Also, as you say, totally inappropriate for FPS games. -- Simon Pieters Opera Software
Re: Mouse Lock
On Wed, Jun 22, 2011 at 2:54 AM, Glenn Maynard gl...@zewt.org wrote: Unrelated, another detail: if most implementations are going to need to warp the mouse cursor to do this, the other mouse event coordinates should always be 0 (or null). Otherwise, implementations on platforms which don't need to warp the cursor may still fill these in, causing incompatibilities. Events like mouseover should probably be suppressed, too. At that point, it's probably cleaner to stop firing *all* mouse movement events entirely, as if the mouse isn't moving, and to use a separate mousedelta event when locked which only has deltaX and deltaY. I had this thought initially, but didn't pursue it. Now that you bring it up again, though, I think I agree. ~TJ
Re: Mouse Lock
On Mon, Jun 20, 2011 at 10:48 AM, Tab Atkins Jr. jackalm...@gmail.comwrote: 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. Does this mean that website is forced to release the lock before mouse-up happens? Or did you just mean that a website can start the lock during a click? If it's latter, I have a problem with that. I know many people who select text on a page just to read them. They obviously don't expect their cursors being tracked/locked by a website. Also, once my mouse is locked, how do I free it? - Ryosuke
Re: Mouse Lock
On Wed, Jun 22, 2011 at 8:02 AM, Ryosuke Niwa rn...@webkit.org wrote: On Mon, Jun 20, 2011 at 10:48 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. Does this mean that website is forced to release the lock before mouse-up happens? Or did you just mean that a website can start the lock during a click? The latter; the former is useful, but in a different way (it's for letting people drag something, like a scrollbar, without having to stay precisely on the element). If it's latter, I have a problem with that. I know many people who select text on a page just to read them. They obviously don't expect their cursors being tracked/locked by a website. We can't do anything about cursors being tracked - that's allowed already by the existing mouse events. I don't expect authors to lock the mouse arbitrarily with this ability unless they're being malicious. The intent is to allow non-fullscreen games to lock the mouse when the user clicks the Start Game button or similar. (For the malicious authors, see below.) Also, once my mouse is locked, how do I free it? That was covered in the paragraph you quoted, though cursorily. If the mouse is locked in this way, the browser should show a persistent message (either in its chrome or as an overlay) saying something like Your mouse cursor is being hidden by the webpage. Press Esc to show the cursor.. This shouldn't be too annoying for the games case, but should allow users, even clueless ones, to know when a site is being malicious and how to fix it. Once they get their cursor back, they can just leave that page. ~TJ
Re: Mouse Lock
On Wed, Jun 22, 2011 at 8:27 AM, Ryosuke Niwa rn...@webkit.org wrote: On Wed, Jun 22, 2011 at 8:17 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: Also, once my mouse is locked, how do I free it? That was covered in the paragraph you quoted, though cursorily. If the mouse is locked in this way, the browser should show a persistent message (either in its chrome or as an overlay) saying something like Your mouse cursor is being hidden by the webpage. Press Esc to show the cursor.. This shouldn't be too annoying for the games case, but should allow users, even clueless ones, to know when a site is being malicious and how to fix it. Once they get their cursor back, they can just leave that page. This might just be a UI problem but I can assure you many users won't be able to find such a message or won't be able to understand what it means. e.g. I know quite few people who don't know what mouse cursor or Esc is. I have trapped your mouse cursor in this box: [picture of mouse cursor]. ...that would actually be pretty funny. Someone should do that. ~TJ
Re: Mouse Lock
On 06/22/2011 06:17 PM, Tab Atkins Jr. wrote: On Wed, Jun 22, 2011 at 8:02 AM, Ryosuke Niwarn...@webkit.org wrote: On Mon, Jun 20, 2011 at 10:48 AM, Tab Atkins Jr.jackalm...@gmail.com wrote: 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. Does this mean that website is forced to release the lock before mouse-up happens? Or did you just mean that a website can start the lock during a click? The latter; the former is useful, but in a different way (it's for letting people drag something, like a scrollbar, without having to stay precisely on the element). If it's latter, I have a problem with that. I know many people who select text on a page just to read them. They obviously don't expect their cursors being tracked/locked by a website. We can't do anything about cursors being tracked - that's allowed already by the existing mouse events. How? If mouse is not pressed and you move mouse outside the browser window, the mouse sure can't be tracked. I don't expect authors to lock the mouse arbitrarily with this ability unless they're being malicious. And we need to protect user from malicious web apps. The intent is to allow non-fullscreen games to lock the mouse when the user clicks the Start Game button or similar. (For the malicious authors, see below.) Also, once my mouse is locked, how do I free it? That was covered in the paragraph you quoted, though cursorily. If the mouse is locked in this way, the browser should show a persistent message (either in its chrome or as an overlay) saying something like Your mouse cursor is being hidden by the webpage. Press Esc to show the cursor.. This shouldn't be too annoying for the games case, but should allow users, even clueless ones, to know when a site is being malicious and how to fix it. Once they get their cursor back, they can just leave that page. So we would end up web apps where after clicking anything in the page user needs to use keyboard to get out of the application? Doesn't sound too user friendly. IMO, user really should be informed before locking the mouse. -Olli
Re: Mouse Lock
John Villar and I have produced a draft specification: https://docs.google.com/document/d/1uV4uDVIe9-8XdVndW8nNGWBfqn9ieeop-5TRfScOG_o/edit?hl=en_USauthkey=CM-dw7QG Some additional notes included in that draft not yet discussed here: - .deltaX conflicts with WheelEvent, I've used .movementX/Y instead unless we feel DOM L3 should be modified. - I've omitted the call of mouselocklost at a page close or ‘onbeforeunload’, as I can not determine a use case that benefits from it. - I've omitted the suggestion of cancelLock on a bubbling MouseEvent - as it is not needed, .target.unlockMouse() can be called.
Re: Mouse Lock
A range of security methods have been discussed. Please read the thread in
detail if this summary is too succinct:
The Security concern is that of the user agent hiding the mouse and not
letting it be used normally due to malicious code on a web site. Thus, user
agents must address this issue. No other security issue has been raised.
A User agent has a large number of options to gauge user intent, and may use
a combination of these techniques to avoid user frustration: prompt user
before lock, is page full screen, require user gesture, avoid immediate
relock, only lock on a forground page/tab in focus, per-domain permissions,
installed application permissions, persistent instructional message to user
on how to unlock, ESC key (or others) always unlocks.
On Sat, Jun 18, 2011 at 11:38 PM, Adam Barth w...@adambarth.com wrote:
I'm sorry that I didn't follow the earlier thread. What is the
security model for mouse lock? (Please feel free to point me to a
message in the archive if this has already been discussed.)
Thanks,
Adam
On Thu, Jun 16, 2011 at 3:21 PM, Vincent Scheib sch...@google.com wrote:
[Building on the Mouse Capture for Canvas
thread:
http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/thread.html#msg437
]
I'm working on an implementation of mouse lock in Chrome, and would
appreciate collaboration on refinement of the spec proposal. Hopefully
webapps is willing to pick up this spec work? I'm up for helping write
the
draft.
Some updates from Sirisian's Comment 12 on the w3 bug
(http://www.w3.org/Bugs/Public/show_bug.cgi?id=9557#c12):
- We shouldn't need events for success/failure to obtain lock, those can
be
callbacks similar to geolocation API:
(http://dev.w3.org/geo/api/spec-source.html#geolocation_interface).
My short summary is then:
- 2 new methods on an element to enter and exit mouse lock. Two callbacks
on
the entering call provide notification of success or failure.
- Mousemove event gains .deltaX .deltaY members, always valid, not just
during mouse lock.
- Elements have an event to detect when mouseLock is lost.
Example
x = document.getElementById(x);
x.addEventListener(mousemove, mouseMove);
x.addEventListener(mouseLockLost, mouseLockLost);
x.lockMouse(
function() { console.log(Locked.); },
function() { console.log(Lock rejected.); } );
function mouseMove(e) { console.log(e.deltaX + , + e.deltaY); }
function mouseLockLost(e) { console.log(Lost lock.); }
Re: Mouse Lock
So it sounds like we don't have a security model but we're hoping UA
implementors can dream one up by combining enough heuristics.
Adam
On Mon, Jun 20, 2011 at 9:07 AM, Vincent Scheib sch...@google.com wrote:
A range of security methods have been discussed. Please read the thread in
detail if this summary is too succinct:
The Security concern is that of the user agent hiding the mouse and not
letting it be used normally due to malicious code on a web site. Thus, user
agents must address this issue. No other security issue has been raised.
A User agent has a large number of options to gauge user intent, and may use
a combination of these techniques to avoid user frustration: prompt user
before lock, is page full screen, require user gesture, avoid immediate
relock, only lock on a forground page/tab in focus, per-domain permissions,
installed application permissions, persistent instructional message to user
on how to unlock, ESC key (or others) always unlocks.
On Sat, Jun 18, 2011 at 11:38 PM, Adam Barth w...@adambarth.com wrote:
I'm sorry that I didn't follow the earlier thread. What is the
security model for mouse lock? (Please feel free to point me to a
message in the archive if this has already been discussed.)
Thanks,
Adam
On Thu, Jun 16, 2011 at 3:21 PM, Vincent Scheib sch...@google.com wrote:
[Building on the Mouse Capture for Canvas
thread: http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/thread.html#msg437 ]
I'm working on an implementation of mouse lock in Chrome, and would
appreciate collaboration on refinement of the spec proposal. Hopefully
webapps is willing to pick up this spec work? I'm up for helping write
the
draft.
Some updates from Sirisian's Comment 12 on the w3 bug
(http://www.w3.org/Bugs/Public/show_bug.cgi?id=9557#c12):
- We shouldn't need events for success/failure to obtain lock, those can
be
callbacks similar to geolocation API:
(http://dev.w3.org/geo/api/spec-source.html#geolocation_interface).
My short summary is then:
- 2 new methods on an element to enter and exit mouse lock. Two
callbacks on
the entering call provide notification of success or failure.
- Mousemove event gains .deltaX .deltaY members, always valid, not just
during mouse lock.
- Elements have an event to detect when mouseLock is lost.
Example
x = document.getElementById(x);
x.addEventListener(mousemove, mouseMove);
x.addEventListener(mouseLockLost, mouseLockLost);
x.lockMouse(
function() { console.log(Locked.); },
function() { console.log(Lock rejected.); } );
function mouseMove(e) { console.log(e.deltaX + , + e.deltaY); }
function mouseLockLost(e) { console.log(Lost lock.); }
Re: Mouse Lock
On Mon, Jun 20, 2011 at 10:18 AM, Adam Barth w...@adambarth.com wrote: So it sounds like we don't have a security model but we're hoping UA implementors can dream one up by combining enough heuristics. A model which I suggested privately, and which I believe others have suggested publicly, is this: 1. While fullscreen is enabled, you can lock the mouse to the fullscreened element without a prompt or persistent message. A temporary message may still be shown. The lock is automatically released if the user exits fullscreen. 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. 3. Otherwise, any attempt to lock the mouse triggers a permissions prompt, and while the lock is active a persistent message is shown. These wouldn't be normative, of course, because different platforms may have different permissions models, but they seem like a good outline for balancing user safety with author convenience/lack of user annoyance. ~TJ
Re: Mouse Lock
On Mon, Jun 20, 2011 at 10:48 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jun 20, 2011 at 10:18 AM, Adam Barth w...@adambarth.com wrote: So it sounds like we don't have a security model but we're hoping UA implementors can dream one up by combining enough heuristics. A model which I suggested privately, and which I believe others have suggested publicly, is this: 1. While fullscreen is enabled, you can lock the mouse to the fullscreened element without a prompt or persistent message. A temporary message may still be shown. The lock is automatically released if the user exits fullscreen. ^^^ This part sounds solid. 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. ^^^ That also sounds reasonable too. There's some subtly to make sure the message is actually visible to the user, especially in desktop situations where one window can overly another. It's probably also useful to instruct the user how to release the lock. 3. Otherwise, any attempt to lock the mouse triggers a permissions prompt, and while the lock is active a persistent message is shown. ^^^ This part sounds scary. What's the use case for locking the mouse without the user interacting with the page? These wouldn't be normative, of course, because different platforms may have different permissions models, but they seem like a good outline for balancing user safety with author convenience/lack of user annoyance. Sure, it shouldn't be normative, but it's important to think though the security model for features before adding them to the platform. Adam
Re: Mouse Lock
On Mon, Jun 20, 2011 at 11:17 AM, Adam Barth w...@adambarth.com wrote: On Mon, Jun 20, 2011 at 10:48 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jun 20, 2011 at 10:18 AM, Adam Barth w...@adambarth.com wrote: So it sounds like we don't have a security model but we're hoping UA implementors can dream one up by combining enough heuristics. A model which I suggested privately, and which I believe others have suggested publicly, is this: 1. While fullscreen is enabled, you can lock the mouse to the fullscreened element without a prompt or persistent message. A temporary message may still be shown. The lock is automatically released if the user exits fullscreen. ^^^ This part sounds solid. Why do you need to lock the mouse when you're in full-screen mode? The mouse can't go outside the element anyway, right? Or is this to handle the multi-monitor scenario where a fullscreen'ed element might just cover one monitor. 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. ^^^ That also sounds reasonable too. There's some subtly to make sure the message is actually visible to the user, especially in desktop situations where one window can overly another. It's probably also useful to instruct the user how to release the lock. Hmm.. I'm less comfortable with this I think. It's always very easy to get the user to click somewhere on a page, so this effectively means that it's very easy for any page to lock the mouse. / Jonas
Re: Mouse Lock
On Thu, Jun 16, 2011 at 6:21 PM, Vincent Scheib sch...@google.com wrote: - Mousemove event gains .deltaX .deltaY members, always valid, not just during mouse lock. Is this implementable? First-person games typically implement delta mouse movement by hiding the mouse cursor, warping the invisible cursor to the center of the screen when it moves, and monitoring the distance of mouse movement from the center of the screen to calculate deltas. I don't think Windows provides a way to retrieve delta mouse movements that doesn't clip when the mouse reaches the edge of the screen. I'm not sure about other environments. -- Glenn Maynard
Re: Mouse Lock
On 06/20/2011 10:18 PM, Jonas Sicking wrote: On Mon, Jun 20, 2011 at 11:17 AM, Adam Barthw...@adambarth.com wrote: On Mon, Jun 20, 2011 at 10:48 AM, Tab Atkins Jr.jackalm...@gmail.com wrote: On Mon, Jun 20, 2011 at 10:18 AM, Adam Barthw...@adambarth.com wrote: So it sounds like we don't have a security model but we're hoping UA implementors can dream one up by combining enough heuristics. A model which I suggested privately, and which I believe others have suggested publicly, is this: 1. While fullscreen is enabled, you can lock the mouse to the fullscreened element without a prompt or persistent message. A temporary message may still be shown. The lock is automatically released if the user exits fullscreen. ^^^ This part sounds solid. Why do you need to lock the mouse when you're in full-screen mode? The mouse can't go outside the element anyway, right? Or is this to handle the multi-monitor scenario where a fullscreen'ed element might just cover one monitor. 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. ^^^ That also sounds reasonable too. There's some subtly to make sure the message is actually visible to the user, especially in desktop situations where one window can overly another. It's probably also useful to instruct the user how to release the lock. Hmm.. I'm less comfortable with this I think. It's always very easy to get the user to click somewhere on a page, so this effectively means that it's very easy for any page to lock the mouse. Yeah. Mouse could be locked on mousedown, but it should be automatically released on mouseup. That is the way set/releaseCapture works in Firefox. Other cases should need explicit permission from user. -Olli
Re: Mouse Lock
On Mon, Jun 20, 2011 at 12:18 PM, Jonas Sicking jo...@sicking.cc wrote: On Mon, Jun 20, 2011 at 11:17 AM, Adam Barth w...@adambarth.com wrote: On Mon, Jun 20, 2011 at 10:48 AM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jun 20, 2011 at 10:18 AM, Adam Barth w...@adambarth.com wrote: So it sounds like we don't have a security model but we're hoping UA implementors can dream one up by combining enough heuristics. A model which I suggested privately, and which I believe others have suggested publicly, is this: 1. While fullscreen is enabled, you can lock the mouse to the fullscreened element without a prompt or persistent message. A temporary message may still be shown. The lock is automatically released if the user exits fullscreen. ^^^ This part sounds solid. Why do you need to lock the mouse when you're in full-screen mode? The mouse can't go outside the element anyway, right? Or is this to handle the multi-monitor scenario where a fullscreen'ed element might just cover one monitor. Multi-monitor is one reason. Another is that, even in single-monitor fullscreen, some browsers pop up an overlay when the mouse hits the top of the screen. Mouselocking would allow browsers to keep doing that in the normal case (it's useful) while making it not happen during games. 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. ^^^ That also sounds reasonable too. There's some subtly to make sure the message is actually visible to the user, especially in desktop situations where one window can overly another. It's probably also useful to instruct the user how to release the lock. Hmm.. I'm less comfortable with this I think. It's always very easy to get the user to click somewhere on a page, so this effectively means that it's very easy for any page to lock the mouse. Yes, which is why I suggest a persistent message be shown with instructions on how to release the lock. Thus the user is aware that the website is being hostile and knows how to stop it long enough to get away (clicking Back or closing the tab). ~TJ
Re: Mouse Lock
On Mon, Jun 20, 2011 at 1:06 PM, Glenn Maynard gl...@zewt.org wrote: On Thu, Jun 16, 2011 at 6:21 PM, Vincent Scheib sch...@google.com wrote: - Mousemove event gains .deltaX .deltaY members, always valid, not just during mouse lock. Is this implementable? First-person games typically implement delta mouse movement by hiding the mouse cursor, warping the invisible cursor to the center of the screen when it moves, and monitoring the distance of mouse movement from the center of the screen to calculate deltas. I don't think Windows provides a way to retrieve delta mouse movements that doesn't clip when the mouse reaches the edge of the screen. I'm not sure about other environments. In a non-mouselock situation, mouse events stop being fired anyway when the mouse goes outside of the window, so you don't have to worry about the delta information. In a mouselock situation, the browser can do precisely what you describe to keep the mouse from leaving the window. ~TJ
Re: Mouse Lock
On Mon, Jun 20, 2011 at 1:19 PM, Olli Pettay olli.pet...@helsinki.fi wrote: On 06/20/2011 10:18 PM, Jonas Sicking wrote: On Mon, Jun 20, 2011 at 11:17 AM, Adam Barthw...@adambarth.com wrote: On Mon, Jun 20, 2011 at 10:48 AM, Tab Atkins Jr.jackalm...@gmail.com 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. ^^^ That also sounds reasonable too. There's some subtly to make sure the message is actually visible to the user, especially in desktop situations where one window can overly another. It's probably also useful to instruct the user how to release the lock. Hmm.. I'm less comfortable with this I think. It's always very easy to get the user to click somewhere on a page, so this effectively means that it's very easy for any page to lock the mouse. Yeah. Mouse could be locked on mousedown, but it should be automatically released on mouseup. That is the way set/releaseCapture works in Firefox. Other cases should need explicit permission from user. The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. Requiring a permissions prompt in this case would be suboptimal, as it would mean the user needs to click at least two things to actually play the game. (The game itself, and then the permissions prompt.) Releasing on mouseup isn't useful for mouselock. The sorts of things that you can do on a drag like that are captured by the mousecapture concept, which is distinct. ~TJ
Re: Mouse Lock
On Mon, Jun 20, 2011 at 5:21 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: In a non-mouselock situation, mouse events stop being fired anyway when the mouse goes outside of the window, so you don't have to worry about the delta information. Mouse events continue to be fired while you hold a mouse button. The browser window may also be maximized, putting one or more edges of the window against the edge of the screen, which shouldn't cause incorrect deltas. (I think deltas should logically represent mouse movement, not cursor movement.) -- Glenn Maynard
Re: Mouse Lock
On 06/21/2011 12:25 AM, Tab Atkins Jr. wrote: On Mon, Jun 20, 2011 at 1:19 PM, Olli Pettayolli.pet...@helsinki.fi wrote: On 06/20/2011 10:18 PM, Jonas Sicking wrote: On Mon, Jun 20, 2011 at 11:17 AM, Adam Barthw...@adambarth.comwrote: On Mon, Jun 20, 2011 at 10:48 AM, Tab Atkins Jr.jackalm...@gmail.com 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. ^^^ That also sounds reasonable too. There's some subtly to make sure the message is actually visible to the user, especially in desktop situations where one window can overly another. It's probably also useful to instruct the user how to release the lock. Hmm.. I'm less comfortable with this I think. It's always very easy to get the user to click somewhere on a page, so this effectively means that it's very easy for any page to lock the mouse. Yeah. Mouse could be locked on mousedown, but it should be automatically released on mouseup. That is the way set/releaseCapture works in Firefox. Other cases should need explicit permission from user. The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. And how would user unlock when some evil sites locks the mouse? Could you give some concrete example about It's probably also useful to instruct the user how to release the lock. Requiring a permissions prompt in this case would be suboptimal, as it would mean the user needs to click at least two things to actually play the game. (The game itself, and then the permissions prompt.) Well, requiring permission is always suboptimal for the one who is going to use the feature for which the permission is needed. -Olli Releasing on mouseup isn't useful for mouselock. The sorts of things that you can do on a drag like that are captured by the mousecapture concept, which is distinct. ~TJ
Re: Mouse Lock
On Mon, Jun 20, 2011 at 3:03 PM, Olli Pettay olli.pet...@helsinki.fi wrote: On 06/21/2011 12:25 AM, Tab Atkins Jr. wrote: The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. And how would user unlock when some evil sites locks the mouse? Could you give some concrete example about It's probably also useful to instruct the user how to release the lock. I'm assuming that the browser reserves some logical key (like Esc) for releasing things like this, and communicates this in the overlay message. ~TJ
Re: Mouse Lock
On 06/21/2011 01:08 AM, Tab Atkins Jr. wrote: On Mon, Jun 20, 2011 at 3:03 PM, Olli Pettayolli.pet...@helsinki.fi wrote: On 06/21/2011 12:25 AM, Tab Atkins Jr. wrote: The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. And how would user unlock when some evil sites locks the mouse? Could you give some concrete example about It's probably also useful to instruct the user how to release the lock. I'm assuming that the browser reserves some logical key (like Esc) for releasing things like this, and communicates this in the overlay message. And what if the web page moves focus to some browser window, so that ESC is fired there? Or what if the web page moves the window to be outside the screen so that user can't actually see the message how to unlock mouse? -Olli ~TJ
Re: Mouse Lock
On Mon, Jun 20, 2011 at 3:26 PM, Olli Pettay olli.pet...@helsinki.fi wrote: On 06/21/2011 01:08 AM, Tab Atkins Jr. wrote: On Mon, Jun 20, 2011 at 3:03 PM, Olli Pettayolli.pet...@helsinki.fi wrote: On 06/21/2011 12:25 AM, Tab Atkins Jr. wrote: The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. And how would user unlock when some evil sites locks the mouse? Could you give some concrete example about It's probably also useful to instruct the user how to release the lock. I'm assuming that the browser reserves some logical key (like Esc) for releasing things like this, and communicates this in the overlay message. And what if the web page moves focus to some browser window, so that ESC is fired there? Or what if the web page moves the window to be outside the screen so that user can't actually see the message how to unlock mouse? How is a webpage able to do either of those things? ~TJ
Re: Mouse Lock
On Mon, Jun 20, 2011 at 1:48 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: A model which I suggested privately, and which I believe others have suggested publicly, is this: 1. While fullscreen is enabled, you can lock the mouse to the fullscreened element without a prompt or persistent message. A temporary message may still be shown. The lock is automatically released if the user exits fullscreen. 2. During a user-initiated click, you can lock the mouse to the target or an ancestor without a permissions prompt, but with a persistent message, either as an overlay or in the browser's chrome. 3. Otherwise, any attempt to lock the mouse triggers a permissions prompt, and while the lock is active a persistent message is shown. There's a middle ground here: you can lock the mouse to the window, but not completely. That is, if the user moves the mouse to the edge, it remains inside, but if they move it fast enough it escapes. This is enough to stop the window from accidentally losing focus when you're trying to click on something near the edge of the screen, but it lets you easily get outside the window if you actually want to. IIRC, Wine does this in windowed mode. Of course, it might not be suitable for games that want to hide the cursor, like FPSes, but it might be a possible fallback if the browser doesn't trust the site enough for whatever reason to let it fully lock the mouse.
Re: Mouse Lock
On Mon, Jun 20, 2011 at 3:30 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jun 20, 2011 at 3:26 PM, Olli Pettay olli.pet...@helsinki.fi wrote: On 06/21/2011 01:08 AM, Tab Atkins Jr. wrote: On Mon, Jun 20, 2011 at 3:03 PM, Olli Pettayolli.pet...@helsinki.fi wrote: On 06/21/2011 12:25 AM, Tab Atkins Jr. wrote: The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. And how would user unlock when some evil sites locks the mouse? Could you give some concrete example about It's probably also useful to instruct the user how to release the lock. I'm assuming that the browser reserves some logical key (like Esc) for releasing things like this, and communicates this in the overlay message. And what if the web page moves focus to some browser window, so that ESC is fired there? Or what if the web page moves the window to be outside the screen so that user can't actually see the message how to unlock mouse? How is a webpage able to do either of those things? window.focus() Adam
Re: Mouse Lock
On Mon, Jun 20, 2011 at 4:53 PM, Adam Barth w...@adambarth.com wrote: On Mon, Jun 20, 2011 at 3:30 PM, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jun 20, 2011 at 3:26 PM, Olli Pettay olli.pet...@helsinki.fi wrote: On 06/21/2011 01:08 AM, Tab Atkins Jr. wrote: On Mon, Jun 20, 2011 at 3:03 PM, Olli Pettayolli.pet...@helsinki.fi wrote: On 06/21/2011 12:25 AM, Tab Atkins Jr. wrote: The use-case is non-fullscreen games and similar, where you'd prefer to lock the mouse as soon as the user clicks into the game. Minecraft is the first example that pops into my head that works like this - it's windowed, and mouselocks you as soon as you click at it. And how would user unlock when some evil sites locks the mouse? Could you give some concrete example about It's probably also useful to instruct the user how to release the lock. I'm assuming that the browser reserves some logical key (like Esc) for releasing things like this, and communicates this in the overlay message. And what if the web page moves focus to some browser window, so that ESC is fired there? Or what if the web page moves the window to be outside the screen so that user can't actually see the message how to unlock mouse? How is a webpage able to do either of those things? window.focus() Seems like calling window.focus() would cancel mouselock. As I suspect would changing the focus any other way like Alt-Tab, Cmd-Tab, etc. Adam
Re: Mouse Lock
I'm sorry that I didn't follow the earlier thread. What is the
security model for mouse lock? (Please feel free to point me to a
message in the archive if this has already been discussed.)
Thanks,
Adam
On Thu, Jun 16, 2011 at 3:21 PM, Vincent Scheib sch...@google.com wrote:
[Building on the Mouse Capture for Canvas
thread: http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/thread.html#msg437 ]
I'm working on an implementation of mouse lock in Chrome, and would
appreciate collaboration on refinement of the spec proposal. Hopefully
webapps is willing to pick up this spec work? I'm up for helping write the
draft.
Some updates from Sirisian's Comment 12 on the w3 bug
(http://www.w3.org/Bugs/Public/show_bug.cgi?id=9557#c12):
- We shouldn't need events for success/failure to obtain lock, those can be
callbacks similar to geolocation API:
(http://dev.w3.org/geo/api/spec-source.html#geolocation_interface).
My short summary is then:
- 2 new methods on an element to enter and exit mouse lock. Two callbacks on
the entering call provide notification of success or failure.
- Mousemove event gains .deltaX .deltaY members, always valid, not just
during mouse lock.
- Elements have an event to detect when mouseLock is lost.
Example
x = document.getElementById(x);
x.addEventListener(mousemove, mouseMove);
x.addEventListener(mouseLockLost, mouseLockLost);
x.lockMouse(
function() { console.log(Locked.); },
function() { console.log(Lock rejected.); } );
function mouseMove(e) { console.log(e.deltaX + , + e.deltaY); }
function mouseLockLost(e) { console.log(Lost lock.); }
Re: Mouse Lock
On 06/17/2011 01:21 AM, Vincent Scheib wrote: - 2 new methods on an element to enter and exit mouse lock. Two callbacks on the entering call provide notification of success or failure. - Mousemove event gains .deltaX .deltaY members, always valid, not just during mouse lock. I don't understand the need for .deltaX/Y. More comments later. -Olli
Re: Mouse Lock
On Sun, Jun 19, 2011 at 5:10 AM, Olli Pettay olli.pet...@helsinki.fiwrote: On 06/17/2011 01:21 AM, Vincent Scheib wrote: - 2 new methods on an element to enter and exit mouse lock. Two callbacks on the entering call provide notification of success or failure. - Mousemove event gains .deltaX .deltaY members, always valid, not just during mouse lock. I don't understand the need for .deltaX/Y. I'm sure there are lots of other use cases but a typical use case for deltaX and deltaY is camera movement in a first person game. You move the mouse to the left to look left. The current mouseX, mouseY stop when mouse hits the edge of the window/screen. deltaX, deltaY do not stop. More comments later. -Olli
