Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On 2015-03-31 23:17, Felix Miata wrote: Roger Hågensen composed on 2015-03-31 21:09 (UTC+0200): ... For Mozilla browsers, you can go to about:config and set media.autoplay.enabled to “falseâ€�. Also, the NoScript browser extension can make media click-to-play by default. I hardly think a lot of users want to follow directions like that. As a programmer/content designer it would make more sense to do so using either attributes or javascript instead of bothering the user. Turning off autoplay is a one time thing, not a big deal like the constant disrespect of sites disregarding instead of embracing user needs. ... I have a hard time seeing rationality in any option other than click-to-play. *My* PC is a tool. *I* should be the one to decide if and when to play, and if or when to stop or pause. What are you suggesting exactly? That all iframes should be click to show? Are you talking about iframe visibility and autopause or autounload? If you are veering off topic then if possible please start a new thread (new subject) in that case. I'll assume you are talking about autopause for a video tag. As a user you open another tab, the video pauses at once (since the autopause attribute is set) since it's no longer visible. Now if you go back to it and click play and then tab away again then you have manually overridden the autopause. There might also be a autopause option on the video UI someplace. My suggestion was for a autopause for iframes, I only noted that it might be of use to video and audio tags as well but I did not really outline how hey should behave as this topic is about iframes rather than video and audio. Starting a new thread to discuss autopause and autounload or video and audio is preferred as that will require more UI discussion than iframes (which has no UI). -- Roger Hågensen, Freelancer, http://skuldwyrm.no/
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On Mon, Mar 30, 2015 at 03:47:26PM -0700, Seth Fowler wrote: I think we should modify the Page Visibility spec to let UA’s take actual visibility of iframes into account when deciding if an iframe is hidden. Right now, the visibility of an iframe is the same as that of the top level browsing context it’s embedded in. Here are the details: http://www.w3.org/TR/page-visibility/ This design doesn’t do much for iframes which may be doing significant work, though. The most obvious example is HTML5 ads. These ads may be performing significant work - computation, network IO, rendering, etc. Some or all of that work is often unnecessary when the ad is outside the viewport. Having an API that would allow those ads to throttle back their work when they’re not visible could have significant positive effects on performance and battery life. I am concerned that if an iframe is on its honor to use the API, the API will have a very small impact, when mandatory resource arbitration seems to be needed. Responsibility for good performance and battery life on my laptop should belong to my UA, not to the web developer. I cannot take for granted the good will of the web developer, and even developers with good intentions may make a mistake or cut corners. It seems to me that the UA should divvy up resources among iframes based on the availability of an *audience* to each one. Invisible pages, occluded iframes, iframes that have scrolled out of the viewport, and so on, definitely shouldn't get a prime share unless the user has made an explicit grant. Give the bulk of the resources to what you could conceivably be looking at. Do you see what I'm getting at? Dave -- David Young dyo...@pobox.comUrbana, IL(217) 721-9981
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On Apr 1, 2015, at 10:35 PM, David Young dyo...@pobox.com wrote: I cannot take for granted the good will of the web developer, and even developers with good intentions may make a mistake or cut corners. Trust me, you’re preaching to the choir on that! It seems to me that the UA should divvy up resources among iframes based on the availability of an *audience* to each one. Invisible pages, occluded iframes, iframes that have scrolled out of the viewport, and so on, definitely shouldn't get a prime share unless the user has made an explicit grant. Give the bulk of the resources to what you could conceivably be looking at. Do you see what I'm getting at? Absolutely. I think all of the UAs have implemented features like that, and are working on more. But making iframes aware that they should throttle themselves is still helpful, because well-behaved iframes can take drastic action to reduce their performance and energy impact - even totally stopping all their processing. UAs always have to worry about breaking existing web content, and it’s often hard to take such drastic action automatically for that reason. I think we’ll get the best results if we take both approaches simultaneously. - Seth
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On Tue, Mar 31, 2015 at 12:47 AM, Seth Fowler s...@mozilla.com wrote: I think we should modify the Page Visibility spec to let UA’s take actual visibility of iframes into account when deciding if an iframe is hidden. Wouldn't it be better to discuss that on public-web-perf? -- https://annevankesteren.nl/
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
Looking at https://developer.mozilla.org/en/docs/Web/HTML/Element/iframe Wouldn't maybe the addition of new attribute to the iframe be the best way? ** autopause If present the client can pause any processing related to the iframe if the iframe is not currently visible. When unpaused a Page Visibility event will be sent to the iframe as if the whole page had changed status from invisible to visible. For visibility events see https://developer.mozilla.org/en-US/docs/Web/Guide/User_experience/Using_the_Page_Visibility_API ** This basically makes it opt in, it changes nothing about the behavior of current iframes. Just an example would be a iframe that can be hidden/unhidden by the user clicking a button, and if the iframe has the autopause attribute then it's state is effectively paused. Once the iframe is unpaued a Page Visibility event is sent and whatever code is running in the frame can then react to this and and resume, as it never got a event indicating the page was made non-visible a programmer should be able to programmatically infer that the iframe was unpaused (it only got the one event instead of two). What type of iframes would benefit from this? Some chats, news feeds, log views, anything that constantly updates or works in the background, but does not need to be updated if not viewable (saves CPU, bandwidth, and server resources) And maybe down the road one could see if a similar autopause can be added to the parent page itself (or not). Maybe a autopause would make sense if added as a attribute to the body (but actually apply to the whole document including any scripts declared in the head). Adding a autopause attribute to a iframe is probably the easiest way to add/deal with this. If nobody ends up using it, it can easily be dropped again too, so there is no immediate downside to this that I can currently think of at least. On 2015-03-31 02:17, Seth Fowler wrote: I do want to clarify one other thing: we’re definitely not yet at the point of implementing this in Gecko, especially not in a release version. We think this functionality is important, and modifying the Page Visibility spec is one way to make it accessible to the web. It’s probably even the nicest way to make it accessible to the web, if it’s feasible. But it’s not certain that it’s web compatible or that everyone agrees this is the best way to go; we’re at the starting point of the process here. I’d be interested to hear any comments that others may have! Thanks, - Seth On Mar 30, 2015, at 3:47 PM, Seth Fowler s...@mozilla.com wrote: I think we should modify the Page Visibility spec to let UA’s take actual visibility of iframes into account when deciding if an iframe is hidden. This design doesn’t do much for iframes which may be doing significant work, though. The most obvious example is HTML5 ads. These ads may be performing significant work - computation, network IO, rendering, etc. Some or all of that work is often unnecessary when the ad is outside the viewport. Having an API that would allow those ads to throttle back their work when they’re not visible could have significant positive effects on performance and battery life. We could get these benefits through a very simple modification of the Page Visibility spec. We should make the visibility of iframes independent of the top-level browsing context, and instead let UA’s take the actual visibility of the iframes into account. If an iframe has been scrolled out of the viewport, has become occluded, or has otherwise been rendered non-visible, we should regard the iframe as hidden and dispatch a visibilitychange event to let the iframe throttle itself. ... - Seth -- Roger Hågensen, Freelancer, http://skuldwyrm.no/
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
autopause looks promising, but I want to ask for more: also add an autounload attribute to allow UAs to unload specific iframes when they are invisible. I ask for this because I'm a contributor of pdf2htmlEX ( https://github.com/coolwanglu/pdf2htmlEX ). Currently pdf2htmlEX can convert each PDF page to one SVG image and embed it in the main HTML via embeds （almost equivalent to iframe here). However, if the PDF file contains many pages, the memory consumption of embeds becomes unacceptable, although most of them are out of the viewport. Because UAs are not allowed to automatically unload invisible nested browsing context, we have to do this in JS (by removing/adding embeds from/to the tree). This is complicated and doesn't work if JS is disabled. If autounload were supported, things would be much simpler. Also I think in some use cases of autopause, autounload is even more suitable becaue it not only save CPU and network, but also save memory, which are equally important to mobile devices. 在 2015年03月31日 14:18, Roger Hågensen 写道: Looking at https://developer.mozilla.org/en/docs/Web/HTML/Element/iframe Wouldn't maybe the addition of new attribute to the iframe be the best way? ** autopause If present the client can pause any processing related to the iframe if the iframe is not currently visible. When unpaused a Page Visibility event will be sent to the iframe as if the whole page had changed status from invisible to visible. For visibility events see https://developer.mozilla.org/en-US/docs/Web/Guide/User_experience/Using_the_Page_Visibility_API ** This basically makes it opt in, it changes nothing about the behavior of current iframes. Just an example would be a iframe that can be hidden/unhidden by the user clicking a button, and if the iframe has the autopause attribute then it's state is effectively paused. Once the iframe is unpaued a Page Visibility event is sent and whatever code is running in the frame can then react to this and and resume, as it never got a event indicating the page was made non-visible a programmer should be able to programmatically infer that the iframe was unpaused (it only got the one event instead of two). What type of iframes would benefit from this? Some chats, news feeds, log views, anything that constantly updates or works in the background, but does not need to be updated if not viewable (saves CPU, bandwidth, and server resources) And maybe down the road one could see if a similar autopause can be added to the parent page itself (or not). Maybe a autopause would make sense if added as a attribute to the body (but actually apply to the whole document including any scripts declared in the head). Adding a autopause attribute to a iframe is probably the easiest way to add/deal with this. If nobody ends up using it, it can easily be dropped again too, so there is no immediate downside to this that I can currently think of at least. On 2015-03-31 02:17, Seth Fowler wrote: I do want to clarify one other thing: we’re definitely not yet at the point of implementing this in Gecko, especially not in a release version. We think this functionality is important, and modifying the Page Visibility spec is one way to make it accessible to the web. It’s probably even the nicest way to make it accessible to the web, if it’s feasible. But it’s not certain that it’s web compatible or that everyone agrees this is the best way to go; we’re at the starting point of the process here. I’d be interested to hear any comments that others may have! Thanks, - Seth On Mar 30, 2015, at 3:47 PM, Seth Fowler s...@mozilla.com wrote: I think we should modify the Page Visibility spec to let UA’s take actual visibility of iframes into account when deciding if an iframe is hidden. This design doesn’t do much for iframes which may be doing significant work, though. The most obvious example is HTML5 ads. These ads may be performing significant work - computation, network IO, rendering, etc. Some or all of that work is often unnecessary when the ad is outside the viewport. Having an API that would allow those ads to throttle back their work when they’re not visible could have significant positive effects on performance and battery life. We could get these benefits through a very simple modification of the Page Visibility spec. We should make the visibility of iframes independent of the top-level browsing context, and instead let UA’s take the actual visibility of the iframes into account. If an iframe has been scrolled out of the viewport, has become occluded, or has otherwise been rendered non-visible, we should regard the iframe as hidden and dispatch a visibilitychange event to let the iframe throttle itself. ... - Seth
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On 2015-03-31 10:16, duanyao wrote: autopause looks promising, but I want to ask for more: also add an autounload attribute to allow UAs to unload specific iframes when they are invisible. This is also a good idea. I also realized that maybe video and audio can benefit from a autopause attribute, so that when the user tabs away or minimizes the browser window the video or audio automatically pause. Currently this is done using javascript and Page Visibility, but with a autopause this would make a plain video or audio tag to do the same, but without javascript. It is also possible that autounload may be of use there as well, for live streams for example and when tabbing back the player have been reset to it starting state. Although in this case autopause and autounload would be mutually exclusive, if both are present (by mistake?) I think autopause should have priority as it's less destructive (to the user) than autounload. -- Roger Hågensen, Freelancer, http://skuldwyrm.no/
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On 2015-03-31 16:09, Boris Zbarsky wrote: On 3/31/15 2:18 AM, Roger Hågensen wrote: What type of iframes would benefit from this? Ads, from a user point of view. Now getting them to opt in to being throttled... -Boris Would not a ad delivery network prefer not to have to push ads out that the user is not seeing at all? If not then they are only wasting bandwidth/CPU/memory on the server, and causing impressions that are wasted on nothing (important for the advertisers since they pay for them). It's not throttling, it's proper use of resources. And while a ad network can not be guaranteed there are eyeballs on the ad, they can at least be assured that the ad is visible. Imagine a clock or counter or similar, personally I'd love to use something similar for a status monitor for a server project, since it shows the live status there is no point in updating it while not visible, and the moment after being made visible again it will be current within a second. No wasted CPU no wasted bandwidth nor server resources. And I already mentioned video and audio (if autopause is taken beyond just iframes). I often open multiple tabs, and then I go through them one by one later. If I end up opening 3-4 videos at the same time I have to stop the other 3 so I do not get a cacophony of 4 videos at once. There is also the issue of quadruple bandwidth load on the server. I often also open anywhere from 2 to 20 tabs with pages in them, what point is there in doing ad rotation or similar in all those if I'm not looking at any of them except tab #19 ? But who knows, you may be right, getting people to op in is a issue, take gzip for example, almost a halving of the bandwidth on average yet there are so many sites out there not making use of it; but that is their choice and loss I guess. -- Roger Hågensen, Freelancer, http://skuldwyrm.no/
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On 2015-03-31 20:55, Nils Dagsson Moskopp wrote: Roger Hågensen rh_wha...@skuldwyrm.no writes: I often open multiple tabs, and then I go through them one by one later. If I end up opening 3-4 videos at the same time I have to stop the other 3 so I do not get a cacophony of 4 videos at once. This is something that can be fixed by the UA: For Mozilla browsers, you can go to about:config and set media.autoplay.enabled to “false”. Also, the NoScript browser extension can make media click-to-play by default. I hardly think a lot of users want to follow directions like that. As a programmer/content designer it would make more sense to do so using either attributes or javascript instead of bothering the user. -- Roger Hågensen, Freelancer, http://skuldwyrm.no/
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
Roger Hågensen rh_wha...@skuldwyrm.no writes: I often open multiple tabs, and then I go through them one by one later. If I end up opening 3-4 videos at the same time I have to stop the other 3 so I do not get a cacophony of 4 videos at once. This is something that can be fixed by the UA: For Mozilla browsers, you can go to about:config and set media.autoplay.enabled to “false”. Also, the NoScript browser extension can make media click-to-play by default. -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On Mar 31, 2015, at 9:42 AM, Roger Hågensen rh_wha...@skuldwyrm.no wrote: Would not a ad delivery network prefer not to have to push ads out that the user is not seeing at all? If not then they are only wasting bandwidth/CPU/memory on the server, and causing impressions that are wasted on nothing (important for the advertisers since they pay for them). It's not throttling, it's proper use of resources. And while a ad network can not be guaranteed there are eyeballs on the ad, they can at least be assured that the ad is visible. There’s a strong motivation to reduce usage of the ad network’s resources. I’m less confident that ad networks will rush to opt in to this feature just to reduce usage of CPU and battery on end users’ machines, and that’s really what the feature is about. That said, “autopause” is an interesting idea! If changing the Page Visibility spec turns out to not be web compatible, a feature like “autopause” seems like a good alternative. And I already mentioned video and audio (if autopause is taken beyond just iframes). I think this would be great, and would definitely solve a common issue on pages that embed lots of media. - Seth
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
Roger Hågensen composed on 2015-03-31 21:09 (UTC+0200): ... For Mozilla browsers, you can go to about:config and set media.autoplay.enabled to âfalseâ. Also, the NoScript browser extension can make media click-to-play by default. I hardly think a lot of users want to follow directions like that. As a programmer/content designer it would make more sense to do so using either attributes or javascript instead of bothering the user. Turning off autoplay is a one time thing, not a big deal like the constant disrespect of sites disregarding instead of embracing user needs. Virtually every site bothers the user from the very moment any new domain is first accessed. Before support for IE6 was abandoned, some sites provided some respect for users by not sizing text in px units. Needing to zoom every new domain accessed to restore text to optimal size is bother. Needing to widen the viewport to accommodate the consequence of zooming so-called responsive pages is bother. Needing to click off unmovable flash needs update and works best with Javascript announcement popup windows is bother. Having to click a toolbar's readable bookmarklet to undo gray text is bother. All that activity just to make a site usable is considerable bother. Bother is a result of an assumption that because a page has been loaded that the visitor is ready right then to watch and listen to its contained media. Bother is a result of an assumption that because a page no longer has visibility, hearing its content is no longer desired. Bother is a result of control that is not in the hands of the user and his browser. I have a hard time seeing rationality in any option other than click-to-play. *My* PC is a tool. *I* should be the one to decide if and when to play, and if or when to stop or pause. -- The wise are known for their understanding, and pleasant words are persuasive. Proverbs 16:21 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/
[whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
I think we should modify the Page Visibility spec to let UA’s take actual visibility of iframes into account when deciding if an iframe is hidden. Right now, the visibility of an iframe is the same as that of the top level browsing context it’s embedded in. Here are the details: http://www.w3.org/TR/page-visibility/ http://www.w3.org/TR/page-visibility/ This design doesn’t do much for iframes which may be doing significant work, though. The most obvious example is HTML5 ads. These ads may be performing significant work - computation, network IO, rendering, etc. Some or all of that work is often unnecessary when the ad is outside the viewport. Having an API that would allow those ads to throttle back their work when they’re not visible could have significant positive effects on performance and battery life. We could get these benefits through a very simple modification of the Page Visibility spec. We should make the visibility of iframes independent of the top-level browsing context, and instead let UA’s take the actual visibility of the iframes into account. If an iframe has been scrolled out of the viewport, has become occluded, or has otherwise been rendered non-visible, we should regard the iframe as hidden and dispatch a visibilitychange event to let the iframe throttle itself. I think it’s worth noting that requestAnimationFrame could cover some of the rendering-related part of this issue, but it’s frequently the case that iframes are performing a good deal of computation and IO that isn’t tied to requestAnimationFrame. Even for the rendering case, the requestAnimationFrame API doesn’t give iframes any way to detect this kind of transition between a state where fast rendering is important and a state where it’s not useful, so in practice extending the Page Visibility spec in this way will often be useful even for iframes which rely mostly on requestAnimationFrame. I think we should view this change as complementary to the benefits that requestAnimationFrame can give us. If there’s willingness to change the spec, this is a change we’d be interested in making in Gecko in the near term. Sound good? Thanks, - Seth
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
A coworker pointed me to this thread on public-web-perf where exactly this proposal has been made before: https://lists.w3.org/Archives/Public/public-web-perf/2014Jan/0047.html https://lists.w3.org/Archives/Public/public-web-perf/2014Jan/0047.html Reading through the posts there has given me an idea of the challenges here, which is what I was hoping for when I sent the original email. It looks like we will need to gather some data about web compatibility before going forward, especially since other specs like the Vibration API reference the Page Visibility spec. I do want to clarify one other thing: we’re definitely not yet at the point of implementing this in Gecko, especially not in a release version. We think this functionality is important, and modifying the Page Visibility spec is one way to make it accessible to the web. It’s probably even the nicest way to make it accessible to the web, if it’s feasible. But it’s not certain that it’s web compatible or that everyone agrees this is the best way to go; we’re at the starting point of the process here. I’d be interested to hear any comments that others may have! Thanks, - Seth On Mar 30, 2015, at 3:47 PM, Seth Fowler s...@mozilla.com wrote: I think we should modify the Page Visibility spec to let UA’s take actual visibility of iframes into account when deciding if an iframe is hidden. Right now, the visibility of an iframe is the same as that of the top level browsing context it’s embedded in. Here are the details: http://www.w3.org/TR/page-visibility/ http://www.w3.org/TR/page-visibility/ This design doesn’t do much for iframes which may be doing significant work, though. The most obvious example is HTML5 ads. These ads may be performing significant work - computation, network IO, rendering, etc. Some or all of that work is often unnecessary when the ad is outside the viewport. Having an API that would allow those ads to throttle back their work when they’re not visible could have significant positive effects on performance and battery life. We could get these benefits through a very simple modification of the Page Visibility spec. We should make the visibility of iframes independent of the top-level browsing context, and instead let UA’s take the actual visibility of the iframes into account. If an iframe has been scrolled out of the viewport, has become occluded, or has otherwise been rendered non-visible, we should regard the iframe as hidden and dispatch a visibilitychange event to let the iframe throttle itself. I think it’s worth noting that requestAnimationFrame could cover some of the rendering-related part of this issue, but it’s frequently the case that iframes are performing a good deal of computation and IO that isn’t tied to requestAnimationFrame. Even for the rendering case, the requestAnimationFrame API doesn’t give iframes any way to detect this kind of transition between a state where fast rendering is important and a state where it’s not useful, so in practice extending the Page Visibility spec in this way will often be useful even for iframes which rely mostly on requestAnimationFrame. I think we should view this change as complementary to the benefits that requestAnimationFrame can give us. If there’s willingness to change the spec, this is a change we’d be interested in making in Gecko in the near term. Sound good? Thanks, - Seth
