Re: [webkit-dev] Mouse Lock API
Can I get a first pass technical review of the patch here: https://bugs.webkit.org/show_bug.cgi?id=68468 The main goals are to put a compile / runtime switch in for mouse lock, get the IDL in place, and pass the new movementX/Y members around. I could also use any hint people have for correctly compiling a call with a VoidCallback: .idl: in [optional] VoidCallback successCallback, in [optional] VoidCallback failureCallback*/); .h: void webkitLockMouse(/*PassRefPtrVoidCallback successCallback, PassRefPtrVoidCallback failureCallback*/); Thx ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
21.09.2011, в 12:52, Darin Fisher написал(а): True, although there are still questions about how the user experience will work for non-fullscreen. I think we only feel confident that we have a decent fullscreen solution at this point, and the plan was to restrict the initial implementation to fullscreen mode for that reason. Would it make sense to simplify the API too? Parts of it seem unnecessary if mouse lock only works in full screen - all additions to MouseEvent in particular can likely be dropped. Security considerations are also substantially different. It might be more productive to treat fullscreen mouse lock as a complete API, with only a potential for being some day extended to support windowed mode. - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
On Thu, Sep 22, 2011 at 2:23 PM, Alexey Proskuryakov a...@webkit.org wrote: 21.09.2011, в 12:52, Darin Fisher написал(а): True, although there are still questions about how the user experience will work for non-fullscreen. I think we only feel confident that we have a decent fullscreen solution at this point, and the plan was to restrict the initial implementation to fullscreen mode for that reason. Would it make sense to simplify the API too? Parts of it seem unnecessary if mouse lock only works in full screen - all additions to MouseEvent in particular can likely be dropped. Security considerations are also substantially different. It might be more productive to treat fullscreen mouse lock as a complete API, with only a potential for being some day extended to support windowed mode. - WBR, Alexey Proskuryakov Our next step is to extend this to non-fullscreen, so I think it would be counter-productive to preclude non-fullscreen at this time. -Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
22.09.2011, в 14:25, Darin Fisher написал(а): Our next step is to extend this to non-fullscreen, so I think it would be counter-productive to preclude non-fullscreen at this time. I think that adding an API for non-fullscreen mouse lock would be a detriment to the Web platform. So gating fullscreen API on it seems pretty controversial. - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
On Thu, Sep 22, 2011 at 2:38 PM, Alexey Proskuryakov a...@webkit.org wrote: 22.09.2011, в 14:25, Darin Fisher написал(а): Our next step is to extend this to non-fullscreen, so I think it would be counter-productive to preclude non-fullscreen at this time. I think that adding an API for non-fullscreen mouse lock would be a detriment to the Web platform. So gating fullscreen API on it seems pretty controversial. - WBR, Alexey Proskuryakov Would you mind raising your concerns with the working group? I believe there is a fair bit of interest in providing mouse lock in the non-fullscreen case too. I can imagine it may require some additional UI. It makes sense to me to design the mouse lock API with the idea that it could work in either fullscreen or non-fullscreen mode. Then it just becomes a question of policy and UI as to whether it runs in neither, one or both. -Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
This API seems clearly necessary to implement traditional FPS game controls. On the other hand, I would not want my browser to have this kind of functionality. Arguably the main benefit of browsers is that they cannot do many things, and are thus relatively safe to use, not yet triggering the sort of fear normally associated with installing software on desktop platforms. Interfering with mouse movement would be one of the most dangerous and frustrating things Web pages could possibly do. The specification draft acknowledges the problem of inability to interact with user agent and the rest of the OS in mouse lock mode. However, the suggested mitigations seem fairly weak. Users are not trained to hit Esc anytime their computer stops properly reacting to their actions - I certainly wouldn't think of doing that. A permanent reminder would be extremely annoying, and we cannot expect that it would be read anyway. Fullscreen is one mode where browsers are expected to take more control over user interaction. Perhaps mouse lock could be appropriate in fullscreen. However, regular fullscreen applications do not use a mouse lock, so users are not trained to escape it - one can always move the mouse pointer to top of screen to get back their menu bar. The only related case I can think of is emulators, which release the mouse on certain modifier keys, such as Cmd or Ctrl+Option. So, I'm not sure how a safe UI would work even in this case. I'm curious about the following provision in the spec: Mouse events normally considered user gestures (e.g. mousedown, click) for security purposes (such as when allowing pop-up windows) should not be treated as user gestures when under mouse lock to prevent malicious cross origin click redirecting. My understanding is that when browser is in control, the primary security concern is with making the user believe that they are interacting with a different site, and thus stealing confidential data the user types (especially passwords). Displaying a pop-up window sounds like a very minor issue when malicious code is already in main frame, can draw anything, and can control mouse movement. Is there a specific attack scenario where this limitation helps? - WBR, Alexey Proskuryakov 20.09.2011, в 20:53, Vincent Scheib написал(а): I have proposed Mouse Lock to be adopted by the W3 WebEvents Working Group: http://lists.w3.org/Archives/Public/public-webevents/2011JulSep/0064.html http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/1424.html On Mon, Sep 12, 2011 at 4:50 PM, Vincent Scheib sch...@chromium.org wrote: A Mouse Lock API has been under discussion on the W3 public-webapps list Mouse Lock(1) and earlier Mouse Capture for Canvas(2). The primary goal is to enable use cases such as first person perspective 3D applications. This is done by providing scripted access to mouse movement data while locking the target of mouse events to a single element and removing the cursor from view. I have been working to formalize the discussions into a draft specification: http://goo.gl/9G8pd, and have a work in progress prototype for WebKit | Chrome. I will publish a WIP patch when it is ready for more eyes. It will be developed behind a compile (and runtime) time flag. I'd like to invite any specification discussion to the Mouse Lock thread (1), and WebKit implementation discussion here. Cheers, -Vince -- 1. http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/thread.html#msg960 2. http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/thread.html#msg437 W3 issue: http://www.w3.org/Bugs/Public/show_bug.cgi?id=9557 Firefox issue: https://bugzilla.mozilla.org/show_bug.cgi?id=633602 Chrome issue: http://code.google.com/p/chromium/issues/detail?id=72754 ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
On Wed, Sep 21, 2011 at 10:44 AM, Alexey Proskuryakov a...@webkit.org wrote: This API seems clearly necessary to implement traditional FPS game controls. On the other hand, I would not want my browser to have this kind of functionality. Arguably the main benefit of browsers is that they cannot do many things, and are thus relatively safe to use, not yet triggering the sort of fear normally associated with installing software on desktop platforms. Interfering with mouse movement would be one of the most dangerous and frustrating things Web pages could possibly do. The specification draft acknowledges the problem of inability to interact with user agent and the rest of the OS in mouse lock mode. However, the suggested mitigations seem fairly weak. Users are not trained to hit Esc anytime their computer stops properly reacting to their actions - I certainly wouldn't think of doing that. A permanent reminder would be extremely annoying, and we cannot expect that it would be read anyway. Fullscreen is one mode where browsers are expected to take more control over user interaction. Perhaps mouse lock could be appropriate in fullscreen. However, regular fullscreen applications do not use a mouse lock, so users are not trained to escape it - one can always move the mouse pointer to top of screen to get back their menu bar. Is that a Mac thing? Mousing around in a fullscreen flash app on Linux or Windows 7 certainly doesn't pop up a menu bar when I hit the top. And the way out is always to hit ESC [although there's often a button as well, depending on the application], so I'm not sure what the problem with fullscreen mouse lock would be. The only related case I can think of is emulators, which release the mouse on certain modifier keys, such as Cmd or Ctrl+Option. So, I'm not sure how a safe UI would work even in this case. I'm curious about the following provision in the spec: Mouse events normally considered user gestures (e.g. mousedown, click) for security purposes (such as when allowing pop-up windows) should not be treated as user gestures when under mouse lock to prevent malicious cross origin click redirecting. My understanding is that when browser is in control, the primary security concern is with making the user believe that they are interacting with a different site, and thus stealing confidential data the user types (especially passwords). Displaying a pop-up window sounds like a very minor issue when malicious code is already in main frame, can draw anything, and can control mouse movement. Is there a specific attack scenario where this limitation helps? - WBR, Alexey Proskuryakov 20.09.2011, в 20:53, Vincent Scheib написал(а): I have proposed Mouse Lock to be adopted by the W3 WebEvents Working Group: http://lists.w3.org/Archives/Public/public-webevents/2011JulSep/0064.html http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/1424.html On Mon, Sep 12, 2011 at 4:50 PM, Vincent Scheib sch...@chromium.org wrote: A Mouse Lock API has been under discussion on the W3 public-webapps list Mouse Lock(1) and earlier Mouse Capture for Canvas(2). The primary goal is to enable use cases such as first person perspective 3D applications. This is done by providing scripted access to mouse movement data while locking the target of mouse events to a single element and removing the cursor from view. I have been working to formalize the discussions into a draft specification: http://goo.gl/9G8pd, and have a work in progress prototype for WebKit | Chrome. I will publish a WIP patch when it is ready for more eyes. It will be developed behind a compile (and runtime) time flag. I'd like to invite any specification discussion to the Mouse Lock thread (1), and WebKit implementation discussion here. Cheers, -Vince -- 1. http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/thread.html#msg960 2. http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/thread.html#msg437 W3 issue: http://www.w3.org/Bugs/Public/show_bug.cgi?id=9557 Firefox issue: https://bugzilla.mozilla.org/show_bug.cgi?id=633602 Chrome issue: http://code.google.com/p/chromium/issues/detail?id=72754 ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
21.09.2011, в 10:56, Eric Uhrhane написал(а): one can always move the mouse pointer to top of screen to get back their menu bar. Is that a Mac thing? Yes, this is how fullscreen applications regularly work on OS X. Mousing around in a fullscreen flash app on Linux or Windows 7 certainly doesn't pop up a menu bar when I hit the top. And the way out is always to hit ESC [although there's often a button as well, depending on the application], so I'm not sure what the problem with fullscreen mouse lock would be. I do not have recent knowledge of Linux, but my understanding is that Windows UI is undergoing a major redesign with Metro, so basing APIs largely on how Windows used to work up to version 7 may not be future proof. - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
2011/9/21 Alexey Proskuryakov a...@webkit.org: platforms. Interfering with mouse movement would be one of the most dangerous and frustrating things Web pages could possibly do. While I agree, that doesn't seem like a problem with the spec. As I read the spec, it seems that it leaves enough latitude to allow each browser to do what it feels is best for its users as far as security and annoyance is concerned. If your browser simply prompted or required whitelisting before allowing a page to mouse lock along with directions on how to cancel, it doesn't seem like it would be problematic to me. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
21.09.2011, в 11:21, James Robinson написал(а): one can always move the mouse pointer to top of screen to get back their menu bar. Is that a Mac thing? Yes, this is how fullscreen applications regularly work on OS X. This is not true for several Flash-based fullscreen video players that I've used on my mac. The interaction there is that upon entering fullscreen Flash displays an overlay indicating that hitting ESC will exit fullscreen mode and the video player displays its controls near the bottom of the screen. After these fade away, the video controls appear only when moving the mouse pointer to the bottom ~1/3rd of the screen. Moving the mouse pointer to the top of the screen does nothing. Hitting ESC or clicking on the appropriate button in the video player's controls exits fullscreen mode. This seems entirely reasonable to me, the keyboard control is provided by Flash itself to prevent bad SWFs from taking control of my computer and the SWF itself provides the additional controls that make sense for its domain. You are describing how Flash exits full screen, and the fact that it apparently always has mouse lock in full screen (so you cannot get to browser menu bar by moving mouse pointer up). It's a different behavior from what this API provides. Besides being a security measure for a subtly different feature, Flash behavior is quite annoying, and not very efficient - see e.g. http://www.bunnyhero.org/2008/05/10/scaring-people-with-fullscreen/. To demonstrate the difference with Flash, there is no mouse lock in Safari fullscreen, so you can move the mouse pointer up to get the menu bar. Anyway, I'm not sure if we already agreed that mouse lock is only desirable in full screen. I think that the spec has the goal of enabling it in browser windows. 21.09.2011, в 11:22, Darin Fisher написал(а): basing APIs largely on how Windows used to work up to version 7 may not be future proof. Yes, but 90% of internet users are not familiar with Metro. They are familiar with Windows as it exists today (XP thru 7). More than 90% is an understatement :) Is your implication that Microsoft will be forced to make future Windows UI behave the same as Windows 7 does? Otherwise, it's not clear to me how a reference to old behavior is relevant to being future proof. - WBR, Alexey Proskuryakov ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
On Wed, Sep 21, 2011 at 12:05 PM, Alexey Proskuryakov a...@webkit.org wrote: 21.09.2011, в 11:21, James Robinson написал(а): one can always move the mouse pointer to top of screen to get back their menu bar. Is that a Mac thing? Yes, this is how fullscreen applications regularly work on OS X. This is not true for several Flash-based fullscreen video players that I've used on my mac. The interaction there is that upon entering fullscreen Flash displays an overlay indicating that hitting ESC will exit fullscreen mode and the video player displays its controls near the bottom of the screen. After these fade away, the video controls appear only when moving the mouse pointer to the bottom ~1/3rd of the screen. Moving the mouse pointer to the top of the screen does nothing. Hitting ESC or clicking on the appropriate button in the video player's controls exits fullscreen mode. This seems entirely reasonable to me, the keyboard control is provided by Flash itself to prevent bad SWFs from taking control of my computer and the SWF itself provides the additional controls that make sense for its domain. You are describing how Flash exits full screen, and the fact that it apparently always has mouse lock in full screen (so you cannot get to browser menu bar by moving mouse pointer up). It's a different behavior from what this API provides. Besides being a security measure for a subtly different feature, Flash behavior is quite annoying, and not very efficient - see e.g. http://www.bunnyhero.org/2008/05/10/scaring-people-with-fullscreen/. To demonstrate the difference with Flash, there is no mouse lock in Safari fullscreen, so you can move the mouse pointer up to get the menu bar. Anyway, I'm not sure if we already agreed that mouse lock is only desirable in full screen. I think that the spec has the goal of enabling it in browser windows. 21.09.2011, в 11:22, Darin Fisher написал(а): basing APIs largely on how Windows used to work up to version 7 may not be future proof. Yes, but 90% of internet users are not familiar with Metro. They are familiar with Windows as it exists today (XP thru 7). More than 90% is an understatement :) Is your implication that Microsoft will be forced to make future Windows UI behave the same as Windows 7 does? Otherwise, it's not clear to me how a reference to old behavior is relevant to being future proof. My point was that emulating the behavior that most users are familiar with will likely create a predictable user experience. Most users are familiar with hitting ESC to exit fullscreen, so it seems sensible to piggyback on that. By the way, wouldn't it be better to debate this API in the WebEvents WG? -Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
On Wed, Sep 21, 2011 at 12:05 PM, Alexey Proskuryakov a...@webkit.org wrote: Anyway, I'm not sure if we already agreed that mouse lock is only desirable in full screen. I think that the spec has the goal of enabling it in browser windows. Indeed, this is explicitly a use case we want to allow. It's reasonable to want to play many mouse-lock-requiring games (Quake, Starcraft, etc.) in non-fullscreen mode, e.g. so one can still keep some other critical display open elsewhere on the screen. PK ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
On Wed, Sep 21, 2011 at 12:46 PM, Peter Kasting pkast...@chromium.orgwrote: On Wed, Sep 21, 2011 at 12:05 PM, Alexey Proskuryakov a...@webkit.orgwrote: Anyway, I'm not sure if we already agreed that mouse lock is only desirable in full screen. I think that the spec has the goal of enabling it in browser windows. Indeed, this is explicitly a use case we want to allow. It's reasonable to want to play many mouse-lock-requiring games (Quake, Starcraft, etc.) in non-fullscreen mode, e.g. so one can still keep some other critical display open elsewhere on the screen. PK True, although there are still questions about how the user experience will work for non-fullscreen. I think we only feel confident that we have a decent fullscreen solution at this point, and the plan was to restrict the initial implementation to fullscreen mode for that reason. -Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
Re: Alexey Proskuryakov's comment: I'm curious about the following provision in the spec: Mouse events normally considered user gestures (e.g. mousedown, click) for security purposes (such as when allowing pop-up windows) should not be treated as user gestures when under mouse lock to prevent malicious cross origin click redirecting. My understanding is that when browser is in control, the primary security concern is with making the user believe that they are interacting with a different site, and thus stealing confidential data the user types (especially passwords). Displaying a pop-up window sounds like a very minor issue when malicious code is already in main frame, can draw anything, and can control mouse movement. Is there a specific attack scenario where this limitation helps? The concern I was attempting to address is primarily that of click jacking. E.g. Preventing a malicious top level page from directing unintended clicks to an iframe with a target site (e.g. a bank). Upon reflection, perhaps the correct way to limit this is to require the target element to match the top level origin as well. Then user gestures can only be delivered to the top level origin. Re: Mouse lock can be a nuisance. Yes, it can, but it also is essential for many compelling applications and user interactions. We believe user-agents can find a balance of providing that ability and diminishing it's abuse. The example of Flash's abusable full screen mode is relevant in that I believe the amount of users enjoying the feature vastly outweigh the amount of nuisance caused. We'll seek to find the appropriate ease of use and reduction of nuisance, but in the extreme can fall back onto heavy weight measures such as installing a web-app or prompting for permission first. The API is designed to tolerate these methods (asynchronous success/failure). ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Mouse Lock API
On Wed, Sep 21, 2011 at 1:15 PM, Vincent Scheib sch...@chromium.org wrote: Re: Alexey Proskuryakov's comment: I'm curious about the following provision in the spec: Mouse events normally considered user gestures (e.g. mousedown, click) for security purposes (such as when allowing pop-up windows) should not be treated as user gestures when under mouse lock to prevent malicious cross origin click redirecting. My understanding is that when browser is in control, the primary security concern is with making the user believe that they are interacting with a different site, and thus stealing confidential data the user types (especially passwords). Displaying a pop-up window sounds like a very minor issue when malicious code is already in main frame, can draw anything, and can control mouse movement. Is there a specific attack scenario where this limitation helps? The concern I was attempting to address is primarily that of click jacking. E.g. Preventing a malicious top level page from directing unintended clicks to an iframe with a target site (e.g. a bank). Upon reflection, perhaps the correct way to limit this is to require the target element to match the top level origin as well. Then user gestures can only be delivered to the top level origin. I wouldn't worry too much about that. A site can already mount a clickjacking attack without using mouselock. It's not clear to me that mouselock make the situation any worse than it is already. Adam Re: Mouse lock can be a nuisance. Yes, it can, but it also is essential for many compelling applications and user interactions. We believe user-agents can find a balance of providing that ability and diminishing it's abuse. The example of Flash's abusable full screen mode is relevant in that I believe the amount of users enjoying the feature vastly outweigh the amount of nuisance caused. We'll seek to find the appropriate ease of use and reduction of nuisance, but in the extreme can fall back onto heavy weight measures such as installing a web-app or prompting for permission first. The API is designed to tolerate these methods (asynchronous success/failure). ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] Mouse Lock API
A Mouse Lock API has been under discussion on the W3 public-webapps list Mouse Lock(1) and earlier Mouse Capture for Canvas(2). The primary goal is to enable use cases such as first person perspective 3D applications. This is done by providing scripted access to mouse movement data while locking the target of mouse events to a single element and removing the cursor from view. I have been working to formalize the discussions into a draft specification: http://goo.gl/9G8pd, and have a work in progress prototype for WebKit | Chrome. I will publish a WIP patch when it is ready for more eyes. It will be developed behind a compile (and runtime) time flag. I'd like to invite any specification discussion to the Mouse Lock thread (1), and WebKit implementation discussion here. Cheers, -Vince -- 1. http://lists.w3.org/Archives/Public/public-webapps/2011JulSep/thread.html#msg960 2. http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/thread.html#msg437 W3 issue: http://www.w3.org/Bugs/Public/show_bug.cgi?id=9557 Firefox issue: https://bugzilla.mozilla.org/show_bug.cgi?id=633602 Chrome issue: http://code.google.com/p/chromium/issues/detail?id=72754 ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
