Re: [whatwg] Proposal for a tab visibility API
Hi all, Thanks again for your comments and feedback. It's been a couple of weeks since the last activity on this thread, and I wanted to take the opportunity to do a very high-level summary of the discussion so far and put forward an updated version of the proposal (which I've included at the end of this e-mail). My high-level summary: Although there is *high *level agreement about the need for a similar type of API, there are a number of concerns: 1) *Additional potential for abuse*. My response to this was Although there is some additional opportunity for abuse, I think that it is not particularly large, possible to defend against if necessary, and outweighed by the advantages such an API would provide to legitimate web developers. I'd be interested if those who originally raised this concern agree with this assessment. 2) *Not the right tool for the use cases*. Various other proposals, including an ability to set a timer that only fires when a page is visible or a way to semantically define what visibility specific types of content require, have been proposed to address the provided use cases. So far those solutions are not concretely defined. 3) *Too specific about tab/window/visibility*. I've attempted to update the proposal (pasted below) to be more clear about the specific semantics and how they might differ on different platforms. I'd appreciate your comments. *Updated API proposal* * * document.visibility A read-only property that returns a string, one of the values described in the next section. Developers can use the existence of this property to know that they can rely on the rest of this API, too. Values returned by all conforming implementations * “visible” : the full-size page content is at least partially visible on at least one screen. * “hidden” : the full-size page content is not visible to the user at all. Additional values potentially returned by some implementations in some cases * “prerender” : the page is currently being loaded off-screen and might never be shown to the user. * “cache” : the page is currently “frozen” in a cache and not displayed on screen (e.g. the back-forward cache). * “preview” : the page is currently visible only in a lower-resolution thumbnail. States in the second set are not guaranteed to be returned in all cases where they might otherwise appear to apply--it is left to the discretion of the implementation. Additional return values may be added to this API in the future. It is up to the implementation to interpret what these values mean in the precise context of interface and platform. As an example, a current-generation desktop browser might interpret the values the following way: * “visible” : the tab is focused in its non-minimized window (regardless of the focus state of the containing window). * “hidden” : the tab is backgrounded within its window, or the containing window is minimized. document.isVisible A simple convenience read-only property that returns a boolean. Returns true if document.visibility’s current value is in the set of visibility states considered to be visible. In most implementations, only the “visible” state is considered visible--although some implementations may consider other values to be visible as well (for example, an implementation that makes regular use of nearly-full-size thumbnail previews may consider “preview” to be a visible state). visibilitychange A simple event, fired at the document object immediately after document.visibility transitions between visibility states. The event has a property, fromState, that is set to the value of document.visibility just before it was changed to the current value. Note that visibility has nothing to do with whether the document’s contents have fully loaded or not, which implies that for any given visibility transition event, onload may or may not have already fired. On Mon, Dec 20, 2010 at 3:44 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 12/20/10 5:16 PM, Alex Komoroske wrote: But I think overall the discussion about precisely what a tab means is not central to the core proposal. Is that reasonable? Yep. -Boris
Re: [whatwg] Proposal for a tab visibility API
On Tue, Jan 4, 2011 at 3:50 PM, Glenn Maynard gl...@zewt.org wrote: The earlier condition that I think you mentioned seemed reasonable: never say the page is hidden when it's not, eg. no false positives. It's more harmful to tell a visible page that it's invisible, than to tell an invisible page that it's visible: * “visible” : the full-size page content may be at least partially visible on at least one screen. * “hidden” : the full-size page content is not visible to the user at all. This gives implementations some latitude to decide whether a condition should be treated as hidden, and doesn't require figuring out precisely whether a window is visible in every case. Great point, Glenn. I agree completely. --Alex
Re: [whatwg] Proposal for a tab visibility API
Hi Bjartur, Thanks for your comments. I've replied inline. On Sun, Dec 19, 2010 at 8:11 PM, Bjartur Thorlacius svartma...@gmail.comwrote: On Wed, 15 Dec 2010 19:27:51 -, Alex Komoroske komoro...@chromium.org wrote: Regarding the fact that background tabs aren't necessarily invisible: - On December 8, Boris Zbarsky wrote: There is no such guarantee for background tabs. For example, browsers may show tab previews in various contexts (Panorama in Firefox 4, e.g.). - The point of the API, as proposed, is that page scripts will know when their content is guaranteed to be invisible to the user--that is, the API will not provide a false positive about invisibility. However, the API may provide false negatives about invisibility, for reasons many others on this thread have been pointed out (including different windowing systems, multiple monitors, partial transparency, etc.). The easiest way to achieve this guarantee is to only consider a tab hidden when it is a background tab within* *a window. The window itself, of course, may be on a little-noticed second monitor, partially obscured, etc. I don´t see how that information is useful. Now, you have to define 'window' and 'tab' differently and define a background state of the latter. Do multiple non-backgrounded (attached) tabs in a window need special treatment? If you use the term 'tab' anywhere it _will_ be confused with the UI metaphor, causing confusion with the approach to hierarchical window management. I don't understand what the term 'tab' means to you. To me a tab is a window. I'm not sure that I understand the point of confusion. When I say 'tab', I mean the current UI construct implemented in Firefox, Safari, Chrome, Opera, Internet Explorer, and others. Each window can have one or more tabs, and in curent implementations (with very few exceptions), each window can only have a single visible tab. But as you point out, there are still some edge cases where even a background tab is visible. In this specific example, I think the right answer would be to have an additional visibility value of preview, which, for the purposes of the isVisible property, would be considered a hidden state. There are some cases where a tab would consider a tab preview to be hidden (like the puzzle timer use case) and some cases where it would be considered visible (like the video playing use case). This would allow web developers to decide for themselves how they wanted to respond to that case. Or, one could mark them up semantically. A video player depends on visibility and audibility and an UA should thus not play video unless it fulfill said requirements. There is also the case of an optional linked soundtrack, which won´t prevent visual playback. In case of an interactive game such as a puzzle, it shouldn't even execute while not focused. IMHO, programs should be stalled (think SIGTTOU) while dynamic requirements can't be fulfilled. In theory, we'd just use blocking operations, but they've been deemed to hard for JavaScript. Regarding the additional abuse potential: Implementations of my counterproposal don't even notify scripts about 'tab' visibility changes, and additionally suspend unimportant scripts, rendering current focus-stealing methods useless. I'm not sure that I have seen your counterproposal. Based on your comments in this thread, I presume that it includes an ability for pages to declare what capabilities they require (e.g. an animation requires that it be visible) and then the browser automatically pauses scripts when those required conditions are not met. Is that a proper understanding of your counterproposal? Is there a more comprehensive/detailed version of the proposal that I could read and comment on? Arguably, it could still be useful to throw an event upon suspension. alert could potentially be removed from window prior to launching the event and the scripts given a timeout, before they get suspended forcibly. I don't know the inner workings of JavaScript implementations to realize whether this could be circumvented by cloning new instances of alert, or whether implementations could simply disable access to potentially harmful methods at an higher level. Regarding the video player use case from the initial proposal: - On December 8th, Maciej Stachowiak wrote: This use case can be handled without help from the page. In Safari, video (whether through media elements or plugins) won't start playing when a user opens a tab in the background, until the user switches to that tab. - Although what you describe satisfies the specific use case, it doesn't address the more general use case of animations (either explicit via javascript or via CSS Animations) or content that is not a plugin/video file. I argue that there are two potentially viable solutions: Implementations exploiting more methods á la Safari
Re: [whatwg] Proposal for a tab visibility API
Ah, yes, thank you Boris, I think I understand now. Note that the actual proposal doesn't depend on the existence of a UI construct called tabs that operate like they do on desktop browsers today. I think the better way to think about it is, if the content of the page is partially visible on *any *screen then it should be considered visible. If it's not visible at all, then it can be considered hidden. This will differ on different browsers and different platforms. I used the word tab only for help in explaining the proposal because today the majority of shipping browsers have a consistent notion of tabs. I agree with you that new UI constructs (and platforms, especially mobile) will change what a tab means. But I think overall the discussion about precisely what a tab means is not central to the core proposal. Is that reasonable? --Alex On Mon, Dec 20, 2010 at 7:40 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 12/20/10 10:21 AM, Alex Komoroske wrote: I'm not sure that I understand the point of confusion. When I say 'tab', I mean the current UI construct implemented in Firefox, Safari, Chrome, Opera, Internet Explorer, and others. I think the point of confusion is that you think this UI construct is an important fundamental, whereas others thing it's not. Each window can have one or more tabs, and in curent implementations (with very few exceptions), each window can only have a single visible tab. As you note, there are exceptions. What makes you think that two years from now the now-common case won't be the exception? It would be preferable to define whatever visibility API is defined without reference to tabs; they're a possibly-transient implementation detail. For example, Firefox on mobile has different rendering areas, etc, but they're not surfaced as tabs to the user; the UI looks and acts totally different, last I checked (and is implemented quite differently, iirc). -Boris
Re: [whatwg] Proposal for a tab visibility API
Hi Bjartur, Thanks for your comments. My responses are inline. On Mon, Dec 20, 2010 at 8:56 PM, Bjartur Thorlacius svartma...@gmail.comwrote: On 12/20/10, Alex Komoroske komoro...@chromium.org wrote: Thanks for your comments. I've replied inline. On Sun, Dec 19, 2010 at 8:11 PM, Bjartur Thorlacius svartma...@gmail.comwrote: On Wed, 15 Dec 2010 19:27:51 -, Alex Komoroske komoro...@chromium.org wrote: Regarding the fact that background tabs aren't necessarily invisible: - On December 8, Boris Zbarsky wrote: There is no such guarantee for background tabs. For example, browsers may show tab previews in various contexts (Panorama in Firefox 4, e.g.). - The point of the API, as proposed, is that page scripts will know when their content is guaranteed to be invisible to the user--that is, the API will not provide a false positive about invisibility. However, the API may provide false negatives about invisibility, for reasons many others on this thread have been pointed out (including different windowing systems, multiple monitors, partial transparency, etc.). The easiest way to achieve this guarantee is to only consider a tab hidden when it is a background tab within* *a window. The window itself, of course, may be on a little-noticed second monitor, partially obscured, etc. The obvious way to achieve this is to only consider a tab/window hidden when it's not on-screen, and consider it visible when it's on-screen. What good is being the current tab when the window is off-screen? I don´t see how that information is useful. Now, you have to define 'window' and 'tab' differently and define a background state of the latter. Do multiple non-backgrounded (attached) tabs in a window need special treatment? If you use the term 'tab' anywhere it _will_ be confused with the UI metaphor, causing confusion with the approach to hierarchical window management. I don't understand what the term 'tab' means to you. To me a tab is a window. I'm not sure that I understand the point of confusion. When I say 'tab', I mean the current UI construct implemented in Firefox, Safari, Chrome, Opera, Internet Explorer, and others. Each window can have one or more tabs, and in curent implementations (with very few exceptions), each window can only have a single visible tab. In context of many of the less-widely used browsers the word 'tab' is never used. Each top-level browsing context has an associated window, which may be grouped with other windows and that group may be represented as a tab strip, wherefrom windows may be selected. They may also be tiled, and a tab strip which allows multiple windows to be selected (tabbed desktops), causing them to tile. How does your proposal work wrt tagged windows (as used in wmii, dwm)? I don't understand an UI construct used by WMs is of any interest to applications. AFAIK, X11 only notifies clients about changing visibility, and by extension, partial overlaps. Why would apps rather know whether they're selected in a tab strip (creating an unnecessary requirement for tab strips) than whether they're visible? With the aforementioned WMs (wmii, dwm) windows may be in multiple groups (tags). In wmii, windows are managed hierarchically. This isn't of any concern to anyone but the user. Applications shouldn't care at all. They're given a resizing window of variable visibility to draw on; tabs don't matter. Thanks for the clarification. I attempted to address this in the e-mail I just sent in response to Boris. (In a nutshell: I agree with you about tabs not being a central UI concept, but I think that the concept of tab is not core to the proposal.) Let me know if that e-mail doesn't address these concerns. I'm not sure that I have seen your counterproposal. Based on your comments in this thread, I presume that it includes an ability for pages to declare what capabilities they require (e.g. an animation requires that it be visible) and then the browser automatically pauses scripts when those required conditions are not met. Is that a proper understanding of your counterproposal? Is there a more comprehensive/detailed version of the proposal that I could read and comment on? Yes, your understanding is correct, and no, there is none. On December 8th, Maciej Stachowiak wrote: This use case can be handled without help from the page. In Safari, video (whether through media elements or plugins) won't start playing when a user opens a tab in the background, until the user switches to that tab. - Although what you describe satisfies the specific use case, it doesn't address the more general use case of animations (either explicit via javascript or via CSS Animations) or content that is not a plugin/video file. Modularity is good. I argue that there are two potentially
Re: [whatwg] Proposal for a tab visibility API
Sorry for the delayed reply. I sent a number of responses over the past week, but it just came to my attention that due to some kind of mailing-list snafu, they never actually were sent out. I've attempted to bring all of my replies into this one message. Sorry for the impression that I had abandoned this thread--that was not my intention! Regarding the fact that background tabs aren't necessarily invisible: - On December 8, Boris Zbarsky wrote: There is no such guarantee for background tabs. For example, browsers may show tab previews in various contexts (Panorama in Firefox 4, e.g.). - The point of the API, as proposed, is that page scripts will know when their content is guaranteed to be invisible to the user--that is, the API will not provide a false positive about invisibility. However, the API may provide false negatives about invisibility, for reasons many others on this thread have been pointed out (including different windowing systems, multiple monitors, partial transparency, etc.). The easiest way to achieve this guarantee is to only consider a tab hidden when it is a background tab within* *a window. The window itself, of course, may be on a little-noticed second monitor, partially obscured, etc. But as you point out, there are still some edge cases where even a background tab is visible. In this specific example, I think the right answer would be to have an additional visibility value of preview, which, for the purposes of the isVisible property, would be considered a hidden state. There are some cases where a tab would consider a tab preview to be hidden (like the puzzle timer use case) and some cases where it would be considered visible (like the video playing use case). This would allow web developers to decide for themselves how they wanted to respond to that case. Regarding the additional abuse potential: -- On December 8th, Boris Zbarksy wrote: I'd really appreciate some comment on this. I'm pretty worried about adding features that we then have to start working around people abusing almost immediately... - Although I agree that there is some additional potential for abuse, I don't think it's a particularly large incremental potential. Sites that want to be annoying already have a very large toolbox today. Sites today could easily hook up a script that detects inactivity on a tab (e.g. lack of scrolling or mouse movement) and pops an alert, refocussing the tab. In practice, this is not a common occurrence--users can vote with their address bar and avoid sites that are needlessly annoying. There would be some easy defenses browser implementors could enact if this focus-grabbing did indeed become a problem. For example, code running in response to a visibilitychange event could be forbidden to open an alert (something that would be easy for moderately-savvy developers to circumvent via a setTimeout). Additionally, if a site pops multiple alerts when the tab is hidden, the alert shown to the user could contain an additional option to Prevent this site from grabbing focus in the future that would not allow alerts when the tab is hidden. Although there is some additional opportunity for abuse, I think that it is not particularly large, possible to defend against if necessary, and outweighed by the advantages such an API would provide to legitimate web developers. Regarding the video player use case from the initial proposal: - On December 8th, Maciej Stachowiak wrote: This use case can be handled without help from the page. In Safari, video (whether through media elements or plugins) won't start playing when a user opens a tab in the background, until the user switches to that tab. - Although what you describe satisfies the specific use case, it doesn't address the more general use case of animations (either explicit via javascript or via CSS Animations) or content that is not a plugin/video file. Regarding solving the use cases that cannot be addressed currently: -- On December 8th, Maciej Stachowiak wrote: That leaves the following use cases: * A puzzle game has a timer that keeps track of how long the user has taken to solve the puzzle. It wants to pause the timer when the user has hidden the tab. * A web app that uses polling to fetch dynamic content can pause polling when it knows the page is hidden from the user. * A page wants to detect when it is being prerendered so it can behave appropriately. I am not sure what the third needs exactly, but it seems like first two could be better served with an API that sets a timer which will only fire when the page is visible. That kind of API might be easier to use right, and avoids the need for JS to run when switching tabs, just to cancel and restart timers. - Although that API might be easier to use correctly (I don't know if I'm convinced), note that it would still have the same abuse concerns as the proposed API. A website developer determined to be annoying
[whatwg] Proposal for a tab visibility API
Hi all, On the Chromium team we’ve identified a couple of use cases that we’d like to address with a simple API, and we’d love your feedback. In particular, there is currently no good way for a web page to detect that it is a background tab and is thus completely invisible to the user, although some heuristics do exist (like detecting mousemove events). In the future, there may be cases where such detection is even more important, for example in the prerendering feature ( http://code.google.com/p/chromium/issues/detail?id=61745) that Chromium is currently in the early stages of experimentation with. ==Use cases== * A puzzle game has a timer that keeps track of how long the user has taken to solve the puzzle. It wants to pause the timer when the user has hidden the tab. * A web app that uses polling to fetch dynamic content can pause polling when it knows the page is hidden from the user. * A streaming video site doesn’t want to start the video until the user actually views the tab for the first time (i.e. video shouldn’t start automatically if a user opens the tab in the background). * A page wants to detect when it is being prerendered so it can behave appropriately. * A page wants to detect when it is moving into or out of the back-forward cache. With these use-cases in mind, there are a number of requirements. ==Requirements== * Easy for developers to write scripts that fall back on old behaviors for browsers that do not implement this API * Ability to query the document’s current visibility state * Events fired when the document transitions between visibility states * Ability for browser vendors to add new visibility states in the future ==Strawman API== What follows is a proposed API that fits the requirements. Note that another route would be to attempt a mostly-compatible extension of Mozilla’s existing pageshow and pagehide events, which would not necessarily be perfectly backwards compatible. =document.visibility= A read-only property that returns a string, one of: * “visible” : the tab is focused in its window * “hidden” : the tab is backgrounded within its window * “prerender” : the tab is currently being loaded in an off-screen tab, and may never be shown to the user. * “cache” : the tab is currently in the back-forward-cache. Note that in Mozilla’s current implementation, document.visibility would never actually be “cache” because Javascript cannot execute when in the cache. In the future, the list of possible values may be extended. Of these states in this list, all except “visible” are considered to be hidden. Developers can use the existence of this property to know that they can rely on the rest of this API, too. =document.isVisible= A simple convenience read-only property that returns a boolean. Returns true if document.visibility’s current value is in the set of visibility states considered to be visible (for the first iteration of this API, that would only include the “visible” state). =visibilitychanged= A simple event, fired at the document object immediately after document.visibility transitions between visibility states. The event has a property, fromState, that is set to the value of document.visibility just before it was changed to the current value. Note that visibility has nothing to do with whether the document’s contents have fully loaded or not, which implies that for any given visibility transition event, onload may or may not have already fired. Thoughts or comments are welcome. --Alex Komoroske
