RE: solving the CPU usage issue for non-visible pages

2009-10-21 Thread Ennals, Robert
[hoping that quoting works better this time]

On Oct 20, 2009, at 7:41 PM, Maciej Stachowiak wrote:
> On Oct 20, 2009, at 7:13 PM, Ennals, Robert wrote:
[snip]
> > I'd even be tempted to risk breaking existing applications a little bit and 
> > make the
> > *default* behavior for HTML5 pages be that "time stops" when a page is not 
> > visible. 
> > If a programmer has a good reason to run javascript on an invisible page 
> > then they 
> > should have to pass an option to make it clear that they know what they are 
> > doing.
>
> One challenge with this approach is that there's no good way at present to 
> make time stop for a > plugin. I suspect more generally that this approach 
> would cause compatibility bugs.

Yeah. I expect this would break some stuff. It could be useful to think about 
what people might be doing that could be broken, so we can see if there is an 
easy solution.

Are you aware of any common patterns in which a plugin communicates with 
javascript in such a way that the app would stop working if javascript timer 
events stopped firing for a while?

My hunch is that the general problem of "javascript doesn't run for a while" 
(in the absence of plugins) shouldn't break many web sites, since this is 
already what happens when a computer goes to sleep. OTOH there is probably 
something I'm missing.

-Rob




Re: solving the CPU usage issue for non-visible pages

2009-10-21 Thread Brian Kardell
So.. I wound up speaking to Robert offline and in our discussion his
comments became much clearer to me and I think that it's at least
worth documenting in case anyone else misunderstands as I did (even
historically via the archive).

There are really a few proposals here which are sort of only
tangentially related in that they all happen to deal with time and
visibility in the current ways of coding.  I think that it would be a
mistake to assume that they are necessarily inherently related in
designing a new API without at least considering that they might be
different things.  The first has to do with the fact that timers are
currently used for animation which has the effect of wastefully eating
CPU cycles to do things that the OS won't ultimately respect anyway
(i.e. calculate visual updates for something that is non-visible).
What Robert has suggested specifically is that there be an API
specifically "about" animation which allows the user agent to
essentially publish a universal "next frame" kind of event that
animations subscribe to.

There are at least two practical side effects to this - one of which
is the thing being discussed here... That user agents _could_ (if they
choose) optimize this universally with techniques like avoiding the
"paint" part if the window is minimized.  The  second practical
benefit is that the API expresses much more clearly what it is about
and is less shoe-horned into what just happens to be the way we've
gotten it to work with the existing technologies -- no more need to
create timers or set frame rates, worry about interleaving frames, etc
-- that would all be handled quite beautifully by the simplicity of
the API design.

The second - and seemingly only partially related topic is whether the
windows state or visibility should be accessible in script... It is
this second question to which my questions are directed.  I definitely
see a potential utility there, but without well defined answers to
some of those questions - it seems that you could easily create a real
rats nest of complexity, incompatibility and unexpected side-effects.



On Tue, Oct 20, 2009 at 8:48 PM, Brian Kardell  wrote:
> I suppose I should not have used that phrasing... It wasn't really
> accurate and it obscures my point...  My point was that I actually
> wanted it to run in the background... So - does time stop, or just
> rendering?  I think that you have to be very clear.
>
>
>
> On Tue, Oct 20, 2009 at 8:43 PM, Robert O'Callahan  
> wrote:
>> On Wed, Oct 21, 2009 at 4:34 PM, Brian Kardell  wrote:
>>>
>>> For example, I recently the Image Evolution demo from
>>> http://www.canvasdemos.com/2009/07/15/image-evolution/ as a kind of a
>>> performance test and let it run for three days - during which it was
>>> not "visible" 99.999% of the time.  Should processing stop - or just
>>> painting?  Painting wont happen because the OS says it wont right?
>>
>> Depends on the OS, I guess. Performance testing is hard; for good
>> performance testing you need a carefully set up environment. It's OK to
>> require special browser configuration to make it believe that the user is
>> always present and the window is always visible. I don't think we need to
>> avoid Web platform or browser features because they might make performance
>> testing a bit harder.
>>
>> Rob
>> --
>> "He was pierced for our transgressions, he was crushed for our iniquities;
>> the punishment that brought us peace was upon him, and by his wounds we are
>> healed. We all, like sheep, have gone astray, each of us has turned to his
>> own way; and the LORD has laid on him the iniquity of us all." [Isaiah
>> 53:5-6]
>>
>




Re: solving the CPU usage issue for non-visible pages

2009-10-21 Thread Brian Kardell
I like window.hasAttention if you can even vaguely define what it
means... It's pointless to make it so vague that useful things will
work differently in different browsers by accident rather than by
design (for example, it might be ok for mobile devices to work
differently by design, but it would royally stink to have a particular
application (like gmail and wave which I think were mentioned earlier)
work fundamentally differently in different browsers - so that's kind
of what I'm trying to get at:  There really seem to be a few classes
of things, which ones mean that your window "has attention"?

Some things might be tougher than they are worth and probably exceed
the practices of even the best non-web-based solutions... I think that
primarily, relying on an OS level maintenance for things like low
power mode is more rational than requiring each JavaScript programmer
dealing with it individually..

browser windows that are minimized and inactive tabs are certainly a
related class of problem and seem like the simplest one.  Other things
I think are more debatable, but potentially useful... I'm not sure.  I
think that it would be tremendously hard for a programmer to "guess"
some of these things in such a way that they would be easily
predictable for users without some kind of prompting if you're not
careful... For example, I recently the Image Evolution demo from
http://www.canvasdemos.com/2009/07/15/image-evolution/ as a kind of a
performance test and let it run for three days - during which it was
not "visible" 99.999% of the time.  Should processing stop - or just
painting?  Painting wont happen because the OS says it wont right?





On Tue, Oct 20, 2009 at 8:16 PM, Robert O'Callahan  wrote:
> On Wed, Oct 21, 2009 at 3:57 PM, Brian Kardell  wrote:
>>
>> Is it really the visibility of the page that is being queried - or the
>> some kind of state of a window?  Maybe it's a silly bit of semantics,
>> but it seems clearer to me that most of the things discussed here are
>> about a whole window/tab being "minimized" (either to a taskbar or tab
>> or something).  If I have one app open and it is covering a browser
>> window - the browser window is not visible (it's lower in the stacking
>> order).  Likewise, a page is generally "partially" visible
>> (scrollbars) so that seems more confusing than it needs to be too.
>
> There are lots of reasons why the browser might deduce that the user is not
> paying attention to a document, e.g.
> -- the browser window containing the document is minimized
> -- the tab containing the document is hidden
> -- the document is in an IFRAME and scrolled offscreen
> -- the browser window is buried behind other windows on the desktop
> -- the screen is dimmed for power saving
> -- gaze tracking detects that the user is looking somewhere else
> -- ultrasonic pings detect that the user is not there
>
> If we need an API beyond just animation, you might as well call it something
> like window.hasAttention so browsers can cover all of those cases.
>
> Rob
> --
> "He was pierced for our transgressions, he was crushed for our iniquities;
> the punishment that brought us peace was upon him, and by his wounds we are
> healed. We all, like sheep, have gone astray, each of us has turned to his
> own way; and the LORD has laid on him the iniquity of us all." [Isaiah
> 53:5-6]
>




Re: solving the CPU usage issue for non-visible pages

2009-10-21 Thread Brian Kardell
Right... Not to beat the point - but page or window? :)  You said page
again and I'm just trying to get some clarity...


On Tue, Oct 20, 2009 at 8:05 PM, Ennals, Robert  wrote:
> [my last attempt at an inline reply seems to have interacted strangely with 
> Maciej's email client, so I'm going to top-post for the moment until I work 
> out what was going on]
>
> Good point. I don't know what other people are thinking, but when I say 
> "invisible" I'm thinking about pages that have been minimized or are in an 
> invisible tab. Changing the semantics of a page when it is occluded by 
> another page could be confusing.
>
> -Rob
>
>> -Original Message-
>> From: Brian Kardell [mailto:bkard...@gmail.com]
>> Sent: Tuesday, October 20, 2009 7:58 PM
>> To: Maciej Stachowiak
>> Cc: Ennals, Robert; Jonas Sicking; rob...@ocallahan.org; public-
>> weba...@w3.org
>> Subject: Re: solving the CPU usage issue for non-visible pages
>>
>> So... in describing this feature:
>>
>> Is it really the visibility of the page that is being queried - or the
>> some kind of state of a window?  Maybe it's a silly bit of semantics,
>> but it seems clearer to me that most of the things discussed here are
>> about a whole window/tab being "minimized" (either to a taskbar or tab
>> or something).  If I have one app open and it is covering a browser
>> window - the browser window is not visible (it's lower in the stacking
>> order).  Likewise, a page is generally "partially" visible
>> (scrollbars) so that seems more confusing than it needs to be too.
>>
>>
>> On Tue, Oct 20, 2009 at 7:41 PM, Maciej Stachowiak 
>> wrote:
>> >
>> > On Oct 20, 2009, at 7:13 PM, Ennals, Robert wrote:
>> >
>> > One thing I like about the "requestAnimationFrame" approach is that
>> it makes
>> > it easy to do the right thing. If the simplest approach burns CPU
>> cycles,
>> > and programmers have to think a bit harder to avoid doing this, then
>> I
>> > suspect the likely outcome would be that many programmers will take
>> the
>> > shortest path, and not check whether their page is visible.
>> >
>> > It's nice if you are able to re-engineer your animations enough to
>> make use
>> > of it. The other approaches discussed seem easier to bolt on to
>> existing
>> > code.
>> > Note: if you really want to optimize CPU use, then the best thing IMO
>> is to
>> > use CSS Transitions or CSS Animations, that way the browser is fully
>> in
>> > control of the frame rate and in many cases can do most of the work
>> on the
>> > GPU, with no need to execute any script as the animation goes. I
>> think this
>> > has the potential to be more CPU-friendly than
>> the requestAnimationFrame
>> > approach, though obviously it's not applicable in some cases (e.g.
>> canvas
>> > drawing).
>> >
>> > I'd even be tempted to risk breaking existing applications a little
>> bit and
>> > make the *default* behavior for HTML5 pages be that "time stops" when
>> a page
>> > is not visible. If a programmer has a good reason to run javascript
>> on an
>> > invisible page then they should have to pass an option to make it
>> clear that
>> > they know what they are doing.
>> >
>> > One challenge with this approach is that there's no good way at
>> present to
>> > make time stop for a plugin. I suspect more generally that this
>> approach
>> > would cause compatibility bugs.
>> > Regards,
>> > Maciej
>> >
>




Re: solving the CPU usage issue for non-visible pages

2009-10-21 Thread Brian Kardell
So... in describing this feature:

Is it really the visibility of the page that is being queried - or the
some kind of state of a window?  Maybe it's a silly bit of semantics,
but it seems clearer to me that most of the things discussed here are
about a whole window/tab being "minimized" (either to a taskbar or tab
or something).  If I have one app open and it is covering a browser
window - the browser window is not visible (it's lower in the stacking
order).  Likewise, a page is generally "partially" visible
(scrollbars) so that seems more confusing than it needs to be too.


On Tue, Oct 20, 2009 at 7:41 PM, Maciej Stachowiak  wrote:
>
> On Oct 20, 2009, at 7:13 PM, Ennals, Robert wrote:
>
> One thing I like about the "requestAnimationFrame" approach is that it makes
> it easy to do the right thing. If the simplest approach burns CPU cycles,
> and programmers have to think a bit harder to avoid doing this, then I
> suspect the likely outcome would be that many programmers will take the
> shortest path, and not check whether their page is visible.
>
> It's nice if you are able to re-engineer your animations enough to make use
> of it. The other approaches discussed seem easier to bolt on to existing
> code.
> Note: if you really want to optimize CPU use, then the best thing IMO is to
> use CSS Transitions or CSS Animations, that way the browser is fully in
> control of the frame rate and in many cases can do most of the work on the
> GPU, with no need to execute any script as the animation goes. I think this
> has the potential to be more CPU-friendly than the requestAnimationFrame
> approach, though obviously it's not applicable in some cases (e.g. canvas
> drawing).
>
> I'd even be tempted to risk breaking existing applications a little bit and
> make the *default* behavior for HTML5 pages be that "time stops" when a page
> is not visible. If a programmer has a good reason to run javascript on an
> invisible page then they should have to pass an option to make it clear that
> they know what they are doing.
>
> One challenge with this approach is that there's no good way at present to
> make time stop for a plugin. I suspect more generally that this approach
> would cause compatibility bugs.
> Regards,
> Maciej
>




Re: solving the CPU usage issue for non-visible pages

2009-10-21 Thread Brian Kardell
I suppose I should not have used that phrasing... It wasn't really
accurate and it obscures my point...  My point was that I actually
wanted it to run in the background... So - does time stop, or just
rendering?  I think that you have to be very clear.



On Tue, Oct 20, 2009 at 8:43 PM, Robert O'Callahan  wrote:
> On Wed, Oct 21, 2009 at 4:34 PM, Brian Kardell  wrote:
>>
>> For example, I recently the Image Evolution demo from
>> http://www.canvasdemos.com/2009/07/15/image-evolution/ as a kind of a
>> performance test and let it run for three days - during which it was
>> not "visible" 99.999% of the time.  Should processing stop - or just
>> painting?  Painting wont happen because the OS says it wont right?
>
> Depends on the OS, I guess. Performance testing is hard; for good
> performance testing you need a carefully set up environment. It's OK to
> require special browser configuration to make it believe that the user is
> always present and the window is always visible. I don't think we need to
> avoid Web platform or browser features because they might make performance
> testing a bit harder.
>
> Rob
> --
> "He was pierced for our transgressions, he was crushed for our iniquities;
> the punishment that brought us peace was upon him, and by his wounds we are
> healed. We all, like sheep, have gone astray, each of us has turned to his
> own way; and the LORD has laid on him the iniquity of us all." [Isaiah
> 53:5-6]
>




Re: solving the CPU usage issue for non-visible pages

2009-10-21 Thread Thomas Broyer
On Wed, Oct 21, 2009 at 5:16 AM, Robert O'Callahan wrote:
>
> There are lots of reasons why the browser might deduce that the user is not
> paying attention to a document, e.g.
> -- the browser window containing the document is minimized
> -- the tab containing the document is hidden
> -- the document is in an IFRAME and scrolled offscreen
> -- the browser window is buried behind other windows on the desktop
> -- the screen is dimmed for power saving
> -- gaze tracking detects that the user is looking somewhere else
> -- ultrasonic pings detect that the user is not there
>
> If we need an API beyond just animation, you might as well call it something
> like window.hasAttention so browsers can cover all of those cases.

...which looks a lot like some kind of "idleness", as has been already
proposed/discussed at the WHATWG and then here:
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2009-August/022443.html
(later revised at
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2009-September/022984.html
and 
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2009-September/022985.html
)
http://lists.w3.org/Archives/Public/public-webapps/2009JulSep/1103.html

We could add a new idleState value that'd be set (with an
idleStateChange event) as soon as the window (browsing context)
becomes "inactive".


-- 
Thomas Broyer
/tɔ.ma.bʁwa.je/



Re: solving the CPU usage issue for non-visible pages

2009-10-21 Thread Jonas Sicking
On Tue, Oct 20, 2009 at 7:13 PM, Ennals, Robert  wrote:
>> On Tuesday, October 20, 2009 at 5:37 PM, Jonas Sicking  
>> wrote:
>> On Tue, Oct 20, 2009 at 4:44 PM, Robert O'Callahan
>>  wrote:
>> > On Wed, Oct 21, 2009 at 11:59 AM, Ennals, Robert
>> 
>> > wrote:
>> >>
>> >> Should we also consider the case where a web site wants to keep its
>> >> interface up to date with some server state and is using up CPU time
>> and
>> >> network resource to do so?
>> >
>> > You could abuse my proposal to do this, by periodically (as
>> frequently as
>> > you run script now) calling requestAnimationFrame and seeing if you
>> actually
>> > get a paint event. Maybe that's not an ideal solution, though.
>>
>> Yeah, I think having an API that lets you check if the page is
>> visible, as well as an event that lets you know when the visibility
>> changes, would be a good idea. That's better than adding *WhenVisible
>> APIs, like setTimeoutWhenVisible, EventSourceWhenVisible, etc.
>
> One thing I like about the "requestAnimationFrame" approach is that it makes 
> it easy to do the right thing. If the simplest approach burns CPU cycles, and 
> programmers have to think a bit harder to avoid doing this, then I suspect 
> the likely outcome would be that many programmers will take the shortest 
> path, and not check whether their page is visible.
>
> I'd even be tempted to risk breaking existing applications a little bit and 
> make the *default* behavior for HTML5 pages be that "time stops" when a page 
> is not visible. If a programmer has a good reason to run javascript on an 
> invisible page then they should have to pass an option to make it clear that 
> they know what they are doing.
>
> I'm sure there are lots of reasons why this wouldn't work, but you get the 
> idea :-)

Sorry, I think I was unclear.

I still think that requestAnimationFrame is a great feature. It's
useful for many other reasons than as a way to deal with non-visible
pages. The fact that it happens to save cycles for non-visible pages
is just a nice benefit.

It's the other features (visible-only-timeouts, visible-only-network)
that I think would be better addressed using a generic visibility API.

/ Jonas



Re: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Robert O'Callahan
On Wed, Oct 21, 2009 at 4:34 PM, Brian Kardell  wrote:

> For example, I recently the Image Evolution demo from
> http://www.canvasdemos.com/2009/07/15/image-evolution/ as a kind of a
> performance test and let it run for three days - during which it was
> not "visible" 99.999% of the time.  Should processing stop - or just
> painting?  Painting wont happen because the OS says it wont right?
>

Depends on the OS, I guess. Performance testing is hard; for good
performance testing you need a carefully set up environment. It's OK to
require special browser configuration to make it believe that the user is
always present and the window is always visible. I don't think we need to
avoid Web platform or browser features because they might make performance
testing a bit harder.

Rob
-- 
"He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all." [Isaiah
53:5-6]


RE: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Ennals, Robert
Actually, I think Robert O'Callahan just made me change my mind on this one. 
The choice of when to consider something "inactive" should probably be left up 
to the user agent, rather than being part of a specification.

-Rob

> -Original Message-
> From: Brian Kardell [mailto:bkard...@gmail.com]
> Sent: Tuesday, October 20, 2009 8:12 PM
> To: Ennals, Robert
> Cc: Maciej Stachowiak; Jonas Sicking; rob...@ocallahan.org; public-
> weba...@w3.org
> Subject: Re: solving the CPU usage issue for non-visible pages
> 
> Right... Not to beat the point - but page or window? :)  You said page
> again and I'm just trying to get some clarity...
> 
> 
> On Tue, Oct 20, 2009 at 8:05 PM, Ennals, Robert
>  wrote:
> > [my last attempt at an inline reply seems to have interacted
> strangely with Maciej's email client, so I'm going to top-post for the
> moment until I work out what was going on]
> >
> > Good point. I don't know what other people are thinking, but when I
> say "invisible" I'm thinking about pages that have been minimized or
> are in an invisible tab. Changing the semantics of a page when it is
> occluded by another page could be confusing.
> >
> > -Rob
> >
> >> -Original Message-
> >> From: Brian Kardell [mailto:bkard...@gmail.com]
> >> Sent: Tuesday, October 20, 2009 7:58 PM
> >> To: Maciej Stachowiak
> >> Cc: Ennals, Robert; Jonas Sicking; rob...@ocallahan.org; public-
> >> weba...@w3.org
> >> Subject: Re: solving the CPU usage issue for non-visible pages
> >>
> >> So... in describing this feature:
> >>
> >> Is it really the visibility of the page that is being queried - or
> the
> >> some kind of state of a window?  Maybe it's a silly bit of
> semantics,
> >> but it seems clearer to me that most of the things discussed here
> are
> >> about a whole window/tab being "minimized" (either to a taskbar or
> tab
> >> or something).  If I have one app open and it is covering a browser
> >> window - the browser window is not visible (it's lower in the
> stacking
> >> order).  Likewise, a page is generally "partially" visible
> >> (scrollbars) so that seems more confusing than it needs to be too.
> >>
> >>
> >> On Tue, Oct 20, 2009 at 7:41 PM, Maciej Stachowiak 
> >> wrote:
> >> >
> >> > On Oct 20, 2009, at 7:13 PM, Ennals, Robert wrote:
> >> >
> >> > One thing I like about the "requestAnimationFrame" approach is
> that
> >> it makes
> >> > it easy to do the right thing. If the simplest approach burns CPU
> >> cycles,
> >> > and programmers have to think a bit harder to avoid doing this,
> then
> >> I
> >> > suspect the likely outcome would be that many programmers will
> take
> >> the
> >> > shortest path, and not check whether their page is visible.
> >> >
> >> > It's nice if you are able to re-engineer your animations enough to
> >> make use
> >> > of it. The other approaches discussed seem easier to bolt on to
> >> existing
> >> > code.
> >> > Note: if you really want to optimize CPU use, then the best thing
> IMO
> >> is to
> >> > use CSS Transitions or CSS Animations, that way the browser is
> fully
> >> in
> >> > control of the frame rate and in many cases can do most of the
> work
> >> on the
> >> > GPU, with no need to execute any script as the animation goes. I
> >> think this
> >> > has the potential to be more CPU-friendly than
> >> the requestAnimationFrame
> >> > approach, though obviously it's not applicable in some cases (e.g.
> >> canvas
> >> > drawing).
> >> >
> >> > I'd even be tempted to risk breaking existing applications a
> little
> >> bit and
> >> > make the *default* behavior for HTML5 pages be that "time stops"
> when
> >> a page
> >> > is not visible. If a programmer has a good reason to run
> javascript
> >> on an
> >> > invisible page then they should have to pass an option to make it
> >> clear that
> >> > they know what they are doing.
> >> >
> >> > One challenge with this approach is that there's no good way at
> >> present to
> >> > make time stop for a plugin. I suspect more generally that this
> >> approach
> >> > would cause compatibility bugs.
> >> > Regards,
> >> > Maciej
> >> >
> >



Re: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Robert O'Callahan
On Wed, Oct 21, 2009 at 3:57 PM, Brian Kardell  wrote:

> Is it really the visibility of the page that is being queried - or the
> some kind of state of a window?  Maybe it's a silly bit of semantics,
> but it seems clearer to me that most of the things discussed here are
> about a whole window/tab being "minimized" (either to a taskbar or tab
> or something).  If I have one app open and it is covering a browser
> window - the browser window is not visible (it's lower in the stacking
> order).  Likewise, a page is generally "partially" visible
> (scrollbars) so that seems more confusing than it needs to be too.
>

There are lots of reasons why the browser might deduce that the user is not
paying attention to a document, e.g.
-- the browser window containing the document is minimized
-- the tab containing the document is hidden
-- the document is in an IFRAME and scrolled offscreen
-- the browser window is buried behind other windows on the desktop
-- the screen is dimmed for power saving
-- gaze tracking detects that the user is looking somewhere else
-- ultrasonic pings detect that the user is not there

If we need an API beyond just animation, you might as well call it something
like window.hasAttention so browsers can cover all of those cases.

Rob
-- 
"He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all." [Isaiah
53:5-6]


RE: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Ennals, Robert
[my last attempt at an inline reply seems to have interacted strangely with 
Maciej's email client, so I'm going to top-post for the moment until I work out 
what was going on]

Good point. I don't know what other people are thinking, but when I say 
"invisible" I'm thinking about pages that have been minimized or are in an 
invisible tab. Changing the semantics of a page when it is occluded by another 
page could be confusing.

-Rob

> -Original Message-
> From: Brian Kardell [mailto:bkard...@gmail.com]
> Sent: Tuesday, October 20, 2009 7:58 PM
> To: Maciej Stachowiak
> Cc: Ennals, Robert; Jonas Sicking; rob...@ocallahan.org; public-
> weba...@w3.org
> Subject: Re: solving the CPU usage issue for non-visible pages
> 
> So... in describing this feature:
> 
> Is it really the visibility of the page that is being queried - or the
> some kind of state of a window?  Maybe it's a silly bit of semantics,
> but it seems clearer to me that most of the things discussed here are
> about a whole window/tab being "minimized" (either to a taskbar or tab
> or something).  If I have one app open and it is covering a browser
> window - the browser window is not visible (it's lower in the stacking
> order).  Likewise, a page is generally "partially" visible
> (scrollbars) so that seems more confusing than it needs to be too.
> 
> 
> On Tue, Oct 20, 2009 at 7:41 PM, Maciej Stachowiak 
> wrote:
> >
> > On Oct 20, 2009, at 7:13 PM, Ennals, Robert wrote:
> >
> > One thing I like about the "requestAnimationFrame" approach is that
> it makes
> > it easy to do the right thing. If the simplest approach burns CPU
> cycles,
> > and programmers have to think a bit harder to avoid doing this, then
> I
> > suspect the likely outcome would be that many programmers will take
> the
> > shortest path, and not check whether their page is visible.
> >
> > It's nice if you are able to re-engineer your animations enough to
> make use
> > of it. The other approaches discussed seem easier to bolt on to
> existing
> > code.
> > Note: if you really want to optimize CPU use, then the best thing IMO
> is to
> > use CSS Transitions or CSS Animations, that way the browser is fully
> in
> > control of the frame rate and in many cases can do most of the work
> on the
> > GPU, with no need to execute any script as the animation goes. I
> think this
> > has the potential to be more CPU-friendly than
> the requestAnimationFrame
> > approach, though obviously it's not applicable in some cases (e.g.
> canvas
> > drawing).
> >
> > I'd even be tempted to risk breaking existing applications a little
> bit and
> > make the *default* behavior for HTML5 pages be that "time stops" when
> a page
> > is not visible. If a programmer has a good reason to run javascript
> on an
> > invisible page then they should have to pass an option to make it
> clear that
> > they know what they are doing.
> >
> > One challenge with this approach is that there's no good way at
> present to
> > make time stop for a plugin. I suspect more generally that this
> approach
> > would cause compatibility bugs.
> > Regards,
> > Maciej
> >



Re: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Robert O'Callahan
On Wed, Oct 21, 2009 at 3:41 PM, Maciej Stachowiak  wrote:

> On Oct 20, 2009, at 7:13 PM, Ennals, Robert wrote:
>
>
> One thing I like about the "requestAnimationFrame" approach is that it
> makes it easy to do the right thing. If the simplest approach burns CPU
> cycles, and programmers have to think a bit harder to avoid doing this, then
> I suspect the likely outcome would be that many programmers will take the
> shortest path, and not check whether their page is visible.
>
>
> It's nice if you are able to re-engineer your animations enough to make use
> of it. The other approaches discussed seem easier to bolt on to existing
> code.
>

I'd hoped it would be easy to retrofit. Given

var timer = setInterval(doStuff, /*some inappropriately chosen frame
duration*/);
...
clearInterval(timer);

you can just change it to

var cancelled = false;
window.addEventListener("beforePaint",
  function() { doStuff(); if (!cancelled) window.requestAnimationFrame(); },
false);
window.requestAnimationFrame();
...
cancelled = true;

Note: if you really want to optimize CPU use, then the best thing IMO is to
> use CSS Transitions or CSS Animations, that way the browser is fully in
> control of the frame rate and in many cases can do most of the work on the
> GPU, with no need to execute any script as the animation goes. I think this
> has the potential to be more CPU-friendly than the requestAnimationFrame
> approach, though obviously it's not applicable in some cases (e.g. canvas
> drawing).
>

That is absolutely true, but there will always be lots of situations where
declarative animation is insufficiently flexible --- games, physics
simulations, etc. And of course the ease of retrofitting existing
script-based animations is an issue.

Rob
-- 
"He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all." [Isaiah
53:5-6]


Re: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Maciej Stachowiak


On Oct 20, 2009, at 7:13 PM, Ennals, Robert wrote:



One thing I like about the "requestAnimationFrame" approach is that  
it makes it easy to do the right thing. If the simplest approach  
burns CPU cycles, and programmers have to think a bit harder to  
avoid doing this, then I suspect the likely outcome would be that  
many programmers will take the shortest path, and not check whether  
their page is visible.


It's nice if you are able to re-engineer your animations enough to  
make use of it. The other approaches discussed seem easier to bolt on  
to existing code.


Note: if you really want to optimize CPU use, then the best thing IMO  
is to use CSS Transitions or CSS Animations, that way the browser is  
fully in control of the frame rate and in many cases can do most of  
the work on the GPU, with no need to execute any script as the  
animation goes. I think this has the potential to be more CPU-friendly  
than the requestAnimationFrame approach, though obviously it's not  
applicable in some cases (e.g. canvas drawing).




I'd even be tempted to risk breaking existing applications a little  
bit and make the *default* behavior for HTML5 pages be that "time  
stops" when a page is not visible. If a programmer has a good reason  
to run javascript on an invisible page then they should have to pass  
an option to make it clear that they know what they are doing.


One challenge with this approach is that there's no good way at  
present to make time stop for a plugin. I suspect more generally that  
this approach would cause compatibility bugs.


Regards,
Maciej



Re: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Maciej Stachowiak


On Oct 20, 2009, at 5:36 PM, Jonas Sicking wrote:

On Tue, Oct 20, 2009 at 4:44 PM, Robert O'Callahan > wrote:
On Wed, Oct 21, 2009 at 11:59 AM, Ennals, Robert >

wrote:


Should we also consider the case where a web site wants to keep its
interface up to date with some server state and is using up CPU  
time and

network resource to do so?


You could abuse my proposal to do this, by periodically (as  
frequently as
you run script now) calling requestAnimationFrame and seeing if you  
actually

get a paint event. Maybe that's not an ideal solution, though.


Yeah, I think having an API that lets you check if the page is
visible, as well as an event that lets you know when the visibility
changes, would be a good idea. That's better than adding *WhenVisible
APIs, like setTimeoutWhenVisible, EventSourceWhenVisible, etc.


I agree. I like a status flag plus event better than adding  
WhenVisible variants. The Web app can then decide what activity to  
throttle when the page is not visible.


 - Maciej




RE: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Ennals, Robert
> On Tuesday, October 20, 2009 at 5:37 PM, Jonas Sicking  
> wrote:
> On Tue, Oct 20, 2009 at 4:44 PM, Robert O'Callahan
>  wrote:
> > On Wed, Oct 21, 2009 at 11:59 AM, Ennals, Robert
> 
> > wrote:
> >>
> >> Should we also consider the case where a web site wants to keep its
> >> interface up to date with some server state and is using up CPU time
> and
> >> network resource to do so?
> >
> > You could abuse my proposal to do this, by periodically (as
> frequently as
> > you run script now) calling requestAnimationFrame and seeing if you
> actually
> > get a paint event. Maybe that's not an ideal solution, though.
> 
> Yeah, I think having an API that lets you check if the page is
> visible, as well as an event that lets you know when the visibility
> changes, would be a good idea. That's better than adding *WhenVisible
> APIs, like setTimeoutWhenVisible, EventSourceWhenVisible, etc.

One thing I like about the "requestAnimationFrame" approach is that it makes it 
easy to do the right thing. If the simplest approach burns CPU cycles, and 
programmers have to think a bit harder to avoid doing this, then I suspect the 
likely outcome would be that many programmers will take the shortest path, and 
not check whether their page is visible.  

I'd even be tempted to risk breaking existing applications a little bit and 
make the *default* behavior for HTML5 pages be that "time stops" when a page is 
not visible. If a programmer has a good reason to run javascript on an 
invisible page then they should have to pass an option to make it clear that 
they know what they are doing.

I'm sure there are lots of reasons why this wouldn't work, but you get the idea 
:-)


-Rob






Re: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Jonas Sicking
On Tue, Oct 20, 2009 at 4:44 PM, Robert O'Callahan  wrote:
> On Wed, Oct 21, 2009 at 11:59 AM, Ennals, Robert 
> wrote:
>>
>> Should we also consider the case where a web site wants to keep its
>> interface up to date with some server state and is using up CPU time and
>> network resource to do so?
>
> You could abuse my proposal to do this, by periodically (as frequently as
> you run script now) calling requestAnimationFrame and seeing if you actually
> get a paint event. Maybe that's not an ideal solution, though.

Yeah, I think having an API that lets you check if the page is
visible, as well as an event that lets you know when the visibility
changes, would be a good idea. That's better than adding *WhenVisible
APIs, like setTimeoutWhenVisible, EventSourceWhenVisible, etc.

/ Jonas



Re: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Robert O'Callahan
On Wed, Oct 21, 2009 at 11:59 AM, Ennals, Robert wrote:

> Should we also consider the case where a web site wants to keep its
> interface up to date with some server state and is using up CPU time and
> network resource to do so?
>

You could abuse my proposal to do this, by periodically (as frequently as
you run script now) calling requestAnimationFrame and seeing if you actually
get a paint event. Maybe that's not an ideal solution, though.

Rob
-- 
"He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all." [Isaiah
53:5-6]


RE: solving the CPU usage issue for non-visible pages

2009-10-20 Thread Ennals, Robert
This looks like a nice solution to the animation case.

Should we also consider the case where a web site wants to keep its interface 
up to date with some server state and is using up CPU time and network resource 
to do so?

 E.g. I might have Google Wave open in a background window, viewing a document 
that is being rapidly updated by other users. If the window is minimized then I 
can’t see the window, and so there is little point consuming network, CPU, and 
power resources maintaining an interface that I’m not looking at.

This is similar to the animation case in that we have a page that is being 
updated unnecessarily, but it is also sufficiently different that it may want a 
different solution.

I’m thinking perhaps we want some kind of API that basically says:
“deliver me messages from this server event source, but only if I’m visible”


Any thoughts?

-Rob

From: public-webapps-requ...@w3.org [mailto:public-webapps-requ...@w3.org] On 
Behalf Of Robert O'Callahan
Sent: Monday, October 19, 2009 2:52 PM
To: public-webapps@w3.org
Subject: Re: solving the CPU usage issue for non-visible pages

I have a proposal for solving this here:
http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/527d0cedb9b0df7f/57625c94cdf493bf

The gist is very simple:
1) window.requestAnimationFrame(): Signals that an animation is in progress, 
and requests that the browser schedule a repaint of the window for the next 
animation frame, if the window is visible.
2) The browser fires a beforePaint event at the document root element (bubbling 
to document and window) any time it repaints the window. This event contains a 
'displayTime' field containing a Javascript Date object representing the time 
at which this frame is likely to become visible to the user. Animation scripts 
should register an event handler that updates the DOM/CSS state to show that 
point in the animation. If the animation has not ended, the event handler 
should call window.requestAnimationFrame() again to ensure another frame will 
be scheduled in a timely manner.

This gives the browser control over the frame rate, lets it suppress animation 
in non-visible windows, lets multiple animations play in synchrony, and should 
be easy to retrofit into code that's currently based on setInterval. I think it 
can also work well with architectures that do compositing in a dedicated thread 
or process.

Rob
--
"He was pierced for our transgressions, he was crushed for our iniquities; the 
punishment that brought us peace was upon him, and by his wounds we are healed. 
We all, like sheep, have gone astray, each of us has turned to his own way; and 
the LORD has laid on him the iniquity of us all." [Isaiah 53:5-6]


Re: solving the CPU usage issue for non-visible pages

2009-10-19 Thread Robert O'Callahan
I have a proposal for solving this here:
http://groups.google.com/group/mozilla.dev.platform/browse_thread/thread/527d0cedb9b0df7f/57625c94cdf493bf

The gist is very simple:
1) window.requestAnimationFrame(): Signals that an animation is in progress,
and requests that the browser schedule a repaint of the window for the next
animation frame, if the window is visible.
2) The browser fires a beforePaint event at the document root element
(bubbling to document and window) any time it repaints the window. This
event contains a 'displayTime' field containing a Javascript Date object
representing the time at which this frame is likely to become visible to the
user. Animation scripts should register an event handler that updates the
DOM/CSS state to show that point in the animation. If the animation has not
ended, the event handler should call window.requestAnimationFrame() again to
ensure another frame will be scheduled in a timely manner.

This gives the browser control over the frame rate, lets it suppress
animation in non-visible windows, lets multiple animations play in
synchrony, and should be easy to retrofit into code that's currently based
on setInterval. I think it can also work well with architectures that do
compositing in a dedicated thread or process.

Rob
-- 
"He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all." [Isaiah
53:5-6]


Re: solving the CPU usage issue for non-visible pages

2009-10-19 Thread Jeremy Orlow
FYI, the original WhatWG thread:
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-October/thread.html#23625
On Mon, Oct 19, 2009 at 1:51 PM, Gregg Tavares  wrote:

> I posted something about this in the whatwg list and was told to bring it
> here.
>
> Currently, AFAIK, the only way to do animation in HTML5 + JavaScript is
> using setInterval. That's great but it has the problem that even when the
> window is minimized or the page is not the front tab, JavaScript has no way
> to know to stop animating.  So, for a CPU heavy animation using canvas 2d or
> canvas 3d, even a hidden tab uses lots of CPU. Of course the browser does
> not copy the bits from the canvas to the window but JavaScript is still
> drawing hundreds of thousands of pixels to the canvas's internal image
> buffer through canvas commands.
>
> To see an example run this sample in any browser
>
> http://mrdoob.com/projects/chromeexperiments/depth_of_field/
>
> Minimize the window or switch to another tab and notice that it's still
> taking up a bunch of CPU time.
>
> Conversely, look at this flash page.
>
> http://www.alissadean.com/
>
> While it might look simple there is actually a lot of CPU based pixel work
> required to composite the buttons with alpha over the scrolling clouds with
> alpha over the background.
>
> Minimize that window or switch to another tab and unlike HTML5 +
> JavaScript, flash has no problem knowning that it no longer needs to render.
>
> There are probably other possible solutions to this problem but it seems
> like the easiest would be either
>
> *) adding an option to window.setInterval or only callback if the window is
> visible
>
> *) adding window.setIntervalIfVisible (same as the previous option really)
>
> A possibly better solution would be
>
> *) element.setIntervalIfVisible
>
> Which would only call the callback if that particular element is visible.
>
> It seems like this will be come an issue as more and more HMTL5 pages start
> using canvas to do stuff they would have been doing in flash like ads or
> games. Without a solution those ads and games will continue to eat CPU even
> when not visible which will make the user experience very poor.
>
> There may be other solutions. The advantage to this solution is it requires
> almost no changes to logic of current animating applications.
>
> Some have suggested the UA can solve this but I don't see how a UA can know
> when JavaScript should and should not run. For example chat and mail
> applications run using setInterval even when not visible so just stopping
> non-visible pages from running at all is not an option.
>
> Another suggested solution is for pages to default to only be processed
> when visible and requiring the page to somehow notify the UA it needs
> processing even when not-visible. This could break some existing apps but
> they would likely be updated immediately. This solution might lessen the
> probability of resource hogging pages in the  future as
> HTML5+JavaScript+canvas ads, games and other apps become more common since
> the default would be for them not to hog the CPU when not visible.
>
> -gregg
>
>
>
>