Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On Apr 1, 2015, at 10:35 PM, David Young dyo...@pobox.com wrote: I cannot take for granted the good will of the web developer, and even developers with good intentions may make a mistake or cut corners. Trust me, you’re preaching to the choir on that! It seems to me that the UA should divvy up resources among iframes based on the availability of an *audience* to each one. Invisible pages, occluded iframes, iframes that have scrolled out of the viewport, and so on, definitely shouldn't get a prime share unless the user has made an explicit grant. Give the bulk of the resources to what you could conceivably be looking at. Do you see what I'm getting at? Absolutely. I think all of the UAs have implemented features like that, and are working on more. But making iframes aware that they should throttle themselves is still helpful, because well-behaved iframes can take drastic action to reduce their performance and energy impact - even totally stopping all their processing. UAs always have to worry about breaking existing web content, and it’s often hard to take such drastic action automatically for that reason. I think we’ll get the best results if we take both approaches simultaneously. - Seth
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
On Mar 31, 2015, at 9:42 AM, Roger Hågensen rh_wha...@skuldwyrm.no wrote: Would not a ad delivery network prefer not to have to push ads out that the user is not seeing at all? If not then they are only wasting bandwidth/CPU/memory on the server, and causing impressions that are wasted on nothing (important for the advertisers since they pay for them). It's not throttling, it's proper use of resources. And while a ad network can not be guaranteed there are eyeballs on the ad, they can at least be assured that the ad is visible. There’s a strong motivation to reduce usage of the ad network’s resources. I’m less confident that ad networks will rush to opt in to this feature just to reduce usage of CPU and battery on end users’ machines, and that’s really what the feature is about. That said, “autopause” is an interesting idea! If changing the Page Visibility spec turns out to not be web compatible, a feature like “autopause” seems like a good alternative. And I already mentioned video and audio (if autopause is taken beyond just iframes). I think this would be great, and would definitely solve a common issue on pages that embed lots of media. - Seth
[whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
I think we should modify the Page Visibility spec to let UA’s take actual visibility of iframes into account when deciding if an iframe is hidden. Right now, the visibility of an iframe is the same as that of the top level browsing context it’s embedded in. Here are the details: http://www.w3.org/TR/page-visibility/ http://www.w3.org/TR/page-visibility/ This design doesn’t do much for iframes which may be doing significant work, though. The most obvious example is HTML5 ads. These ads may be performing significant work - computation, network IO, rendering, etc. Some or all of that work is often unnecessary when the ad is outside the viewport. Having an API that would allow those ads to throttle back their work when they’re not visible could have significant positive effects on performance and battery life. We could get these benefits through a very simple modification of the Page Visibility spec. We should make the visibility of iframes independent of the top-level browsing context, and instead let UA’s take the actual visibility of the iframes into account. If an iframe has been scrolled out of the viewport, has become occluded, or has otherwise been rendered non-visible, we should regard the iframe as hidden and dispatch a visibilitychange event to let the iframe throttle itself. I think it’s worth noting that requestAnimationFrame could cover some of the rendering-related part of this issue, but it’s frequently the case that iframes are performing a good deal of computation and IO that isn’t tied to requestAnimationFrame. Even for the rendering case, the requestAnimationFrame API doesn’t give iframes any way to detect this kind of transition between a state where fast rendering is important and a state where it’s not useful, so in practice extending the Page Visibility spec in this way will often be useful even for iframes which rely mostly on requestAnimationFrame. I think we should view this change as complementary to the benefits that requestAnimationFrame can give us. If there’s willingness to change the spec, this is a change we’d be interested in making in Gecko in the near term. Sound good? Thanks, - Seth
Re: [whatwg] Modify the Page Visibility spec to let UA's take into account whether iframes are visible on the screen
A coworker pointed me to this thread on public-web-perf where exactly this proposal has been made before: https://lists.w3.org/Archives/Public/public-web-perf/2014Jan/0047.html https://lists.w3.org/Archives/Public/public-web-perf/2014Jan/0047.html Reading through the posts there has given me an idea of the challenges here, which is what I was hoping for when I sent the original email. It looks like we will need to gather some data about web compatibility before going forward, especially since other specs like the Vibration API reference the Page Visibility spec. I do want to clarify one other thing: we’re definitely not yet at the point of implementing this in Gecko, especially not in a release version. We think this functionality is important, and modifying the Page Visibility spec is one way to make it accessible to the web. It’s probably even the nicest way to make it accessible to the web, if it’s feasible. But it’s not certain that it’s web compatible or that everyone agrees this is the best way to go; we’re at the starting point of the process here. I’d be interested to hear any comments that others may have! Thanks, - Seth On Mar 30, 2015, at 3:47 PM, Seth Fowler s...@mozilla.com wrote: I think we should modify the Page Visibility spec to let UA’s take actual visibility of iframes into account when deciding if an iframe is hidden. Right now, the visibility of an iframe is the same as that of the top level browsing context it’s embedded in. Here are the details: http://www.w3.org/TR/page-visibility/ http://www.w3.org/TR/page-visibility/ This design doesn’t do much for iframes which may be doing significant work, though. The most obvious example is HTML5 ads. These ads may be performing significant work - computation, network IO, rendering, etc. Some or all of that work is often unnecessary when the ad is outside the viewport. Having an API that would allow those ads to throttle back their work when they’re not visible could have significant positive effects on performance and battery life. We could get these benefits through a very simple modification of the Page Visibility spec. We should make the visibility of iframes independent of the top-level browsing context, and instead let UA’s take the actual visibility of the iframes into account. If an iframe has been scrolled out of the viewport, has become occluded, or has otherwise been rendered non-visible, we should regard the iframe as hidden and dispatch a visibilitychange event to let the iframe throttle itself. I think it’s worth noting that requestAnimationFrame could cover some of the rendering-related part of this issue, but it’s frequently the case that iframes are performing a good deal of computation and IO that isn’t tied to requestAnimationFrame. Even for the rendering case, the requestAnimationFrame API doesn’t give iframes any way to detect this kind of transition between a state where fast rendering is important and a state where it’s not useful, so in practice extending the Page Visibility spec in this way will often be useful even for iframes which rely mostly on requestAnimationFrame. I think we should view this change as complementary to the benefits that requestAnimationFrame can give us. If there’s willingness to change the spec, this is a change we’d be interested in making in Gecko in the near term. Sound good? Thanks, - Seth
Re: [whatwg] Effect of image-orientation on naturalWidth/Height
On Mar 13, 2015, at 11:56 AM, Tab Atkins Jr. jackalm...@gmail.com wrote:
If it happens at the markup level, it should *definitely* affect the
naturalWidth/Height properties. I don't think that's in question at
all. But nobody's moved on the markup issue, so I haven't removed the
CSS property yet. ^_^
Not to rehash comments that I and others have already made in bug 25508, but I
think specifying whether we honor EXIF orientation on a per-image basis is not
really very interesting.
By far the most desirable outcome, if it’s sufficiently web-compatible, is to
just respect EXIF orientation by default.
If we can’t do that, then I think content authors will mostly just opt in via a
single “img { image-orientation: from-image }” in their CSS. That’s the
simplest opt in solution that’s feasible. It’s also trivial to encapsulate in a
standard CSS library.
I’m opposed to the removal of the CSS property for a markup-based solution, as
that will force content authors to specify “autorotate” on every single img
element on the page. That’s a waste of effort and bandwidth (though admittedly
compression will make the impact there minimal), and it makes it more likely
that content authors will simply forget to do so on some elements.
Encapsulating this solution is also significantly more heavyweight.
Having a DOM-based way to request that EXIF orientation be respected is
desirable, though, so that it can be used with non-HTML uses of images like
canvas.
- Seth
Re: [whatwg] Effect of image-orientation on naturalWidth/Height
The more I think about this, the more I agree with David. It really does make more sense to act like the rotation is part of the image format, because after all it *is*, at least when from-image is used. This approach also gives us a smoother path to eventually respecting EXIF orientation by default. If we did that, we’d want naturalWidth and naturalHeight to take EXIF orientation into account, so planning for that with the behavior of image-orientation makes sense. And FWIW, Safari (which respects EXIF orientation in image documents and by default on mobile) does appear to take EXIF orientation into account for naturalWidth and naturalHeight, so this approach is web compatible. Consider this a second vote for “naturalWidth and naturalHeight should respect image-orientation”. - Seth On Mar 10, 2015, at 10:09 AM, L. David Baron dba...@dbaron.org wrote: On Monday 2015-03-09 16:52 -0700, Tab Atkins Jr. wrote: That's a good question. I suspect that .naturalWidth/Height should return the image's dimensions before applying CSS rotations. This is likely to be surprising, but also probably the correct answer for separation-of-concerns reasons. I wonder whether I need to tweak Images, or Hixie needs tweak img. Hmm. I really think that the mechanism for opting in to honoring EXIF should make the browser act as though the rotation were in the image format. It's a compatibility hack (because implementations were initially shipping without EXIF support, and there may be a dependency on that), but once the developer has opted in, everything should really act like the rotation is part of the image format. -David -- 턞 L. David Baron http://dbaron.org/ 턂 턢 Mozilla https://www.mozilla.org/ 턂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914)
Re: [whatwg] Effect of image-orientation on naturalWidth/Height
On Mar 9, 2015, at 5:06 PM, Garrett Smith dhtmlkitc...@gmail.com wrote: On 3/9/15, Tab Atkins Jr. jackalm...@gmail.com wrote: That's a good question. I suspect that .naturalWidth/Height should return the image's dimensions before applying CSS rotations. I think that that is not what Seth was asking about. IIUC, he asked about EXIF rotation info. When you take a pic in your iPhone, if there is rotation data on it, and if that data is not removed, the image will look rotated in browsers that recognize this header, like Safari. No, Tab is right. The question is about the CSS image-orientation property, which allows (among other things) an image to be rotated according to the orientation specified in its EXIF info. - Seth
[whatwg] Effect of image-orientation on naturalWidth/Height
Hi all! I wanted to get the opinion of this list on how image-orientation and the img element’s naturalWidth and naturalHeight properties should interact. The css-images level 3 spec says: The intrinsic height and width are derived from the rotated rather than the original image dimensions.” The HTML spec says: The IDL attributes naturalWidth and naturalHeight must return the intrinsic width and height of the image, in CSS pixels, if the image is available, or else 0.” On the surface, it seems clear that image-orientation must affect naturalWidth/Height. However, I’m not sure whether this was intended, and I don’t have a strong intuition for whether this is more or less surprising to content authors than having these two features be totally independent. There is certainly a potential performance cost if the two features do interact, since that means that naturalWidth/Height will depend on style information. On the other hand, naturalWidth and naturalHeight would definitely take EXIF orientation into account if we respected it by default, so perhaps they also should when content authors opt in to EXIF orientation support using image-orientation. Let me know what you think. Thanks, - Seth
