Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
On Tue, Nov 18, 2008 at 4:09 AM, timeless [EMAIL PROTECTED] wrote:
On Tue, Nov 18, 2008 at 3:15 AM, Tab Atkins Jr. [EMAIL PROTECTED]
wrote:
Keep in mind what we're dealing with here.
dumb web authors. check.
This isn't some trivial Javascript timer firing off events at 60Hz.
This is a full video
element. If a little i++; if( i%60 == 0) {code} is killing the CPU,
then
that device isn't capable of playing a video either.
if that little code causes the entire page to reflow because the web
author is stupid, then it's a problem.
if i'm really lucky my device might be able to use a special module to
deal w/ video processing without killing my battery life.
while i am praying that JIT will improve my life, it won't save my
battery from stupid web authors and their pages.
where possible we want to be able to optimize, video is something
which can be handled by a specialized processor.
javascript which causes the page to reflow because the author did
something 'strange' which only effects my small device and didn't
appear when he tested it on something else, otoh isn't something i can
specialize. other than turning it off, and if i do that, people
complain my product is broken.
they also say my device sucks when the battery life doesn't match
their expectations.
sadly, customers are valuable.
if the embedders (and antti clearly represents one, and I represent
another) express clearly that they don't want something, it's probably
a good idea to listen.
while it's true that you could write code which checked a counter and
only did work on the nth iteration, it's unlikely that you would. such
counters are also likely to be calibrated for the wrong type of device
meaning you get other problems - perhaps the progress bar doesn't
update often enough on slower devices, and not because the event
wasn't sent often enough but because it was filtered by the page.
On Tue, Nov 18, 2008 at 1:19 AM, Robert O'Callahan [EMAIL PROTECTED]
wrote:
That makes a lot more sense than limiting the timeupdate period.
Firing timeupdate on discontinuous changes and requiring apps
to also use a regular timer to get periodic updates sounds reasonable to
me.
this sounds good to me too.
After reading more feedback, I'm agreeing with you, but note that my
original point wasn't against what you were saying anyway. ^_^ It was just
disputing that a counter-based js loop would kill performance.
Noted, though, that many people will, without thinking, just update on every
tick rather than figuring out what a significant interval to use is, and
that the potential differences in tick speed (4Hz to 66Hz right now, right?
somewhere around there?) mean that you *can't* just run a simple counter and
update every nth tick. You'll still have to do some time comparisons.
~TJ
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
Robert O'Callahan wrote:
On Mon, Nov 17, 2008 at 3:19 PM, Jonas Sicking [EMAIL PROTECTED] wrote:
Ian Hickson wrote:
Video and audio playback is already extremely CPU intensive,
we shouldn't require the UA to burn extra cycles doing
unnecessary work.
I agree. That was exactly the thinking behind the timeupdate
event. It allows the UA to determine how fast to update the UI
without hurting performance. Basically it puts the UA in charge
of the performance critical aspects instead of hoping that the
author will work it out.
Though if all implementations are saying that it has the opposite
effect then clearly we need to look into if something went wrong :)
I don't think firing timeupdate on every frame is *that* bad for us, to
be honest.
It seems like a pretty big waste of resources to have the following
script executing 50 times per second:
function timeupdatehandler(e) {
video = e.target;
completed = video.currentTime / video.duration;
thumb = document.getElementById('thumb' + video.id);
thumb.style.left =
calcScreenPositionUsingOffsetLeftRecursion(thumb.parentNode) +
thumb.parentNode.offsetWidth * completed;
progress = document.getElementById('progress' + video.id);
progress.style.width =
progress.parentNode.offsetWidth * completed;
}
Sure, we can pull it off, but why do it? At it certainly doesn't seem to
archive the goal of the event which apparently is to reduce the amount
of CPU resources used.
/ Jonas
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
On Tue, Nov 18, 2008 at 8:48 AM, Jonas Sicking [EMAIL PROTECTED] wrote:
It seems like a pretty big waste of resources to have the following script
executing 50 times per second:
function timeupdatehandler(e) {
video = e.target;
completed = video.currentTime / video.duration;
thumb = document.getElementById('thumb' + video.id);
thumb.style.left =
calcScreenPositionUsingOffsetLeftRecursion(thumb.parentNode) +
thumb.parentNode.offsetWidth * completed;
progress = document.getElementById('progress' + video.id);
progress.style.width =
progress.parentNode.offsetWidth * completed;
}
Sure, we can pull it off, but why do it? At it certainly doesn't seem to
archive the goal of the event which apparently is to reduce the amount of
CPU resources used.
It ensures that the UI is always perfectly up to date. We can throttle
timeupdate if we get too busy but otherwise, we may as well update the UI as
frequently as the video rendering itself.
Rob
--
He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all. [Isaiah
53:5-6]
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
i would hope that repainting a progress bar that has not moved 50x/second
would not be a normal implementation too. 2x/second seems more realistic (a
300s video with a 600 pixel-wide playbar).
On Mon, Nov 17, 2008 at 1:23 PM, Robert O'Callahan [EMAIL PROTECTED]wrote:
On Tue, Nov 18, 2008 at 8:48 AM, Jonas Sicking [EMAIL PROTECTED] wrote:
It seems like a pretty big waste of resources to have the following script
executing 50 times per second:
function timeupdatehandler(e) {
video = e.target;
completed = video.currentTime / video.duration;
thumb = document.getElementById('thumb' + video.id);
thumb.style.left =
calcScreenPositionUsingOffsetLeftRecursion(thumb.parentNode) +
thumb.parentNode.offsetWidth * completed;
progress = document.getElementById('progress' + video.id);
progress.style.width =
progress.parentNode.offsetWidth * completed;
}
Sure, we can pull it off, but why do it? At it certainly doesn't seem to
archive the goal of the event which apparently is to reduce the amount of
CPU resources used.
It ensures that the UI is always perfectly up to date. We can throttle
timeupdate if we get too busy but otherwise, we may as well update the UI as
frequently as the video rendering itself.
Rob
--
He was pierced for our transgressions, he was crushed for our iniquities;
the punishment that brought us peace was upon him, and by his wounds we are
healed. We all, like sheep, have gone astray, each of us has turned to his
own way; and the LORD has laid on him the iniquity of us all. [Isaiah
53:5-6]
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
Jeremy Doig wrote: i would hope that repainting a progress bar that has not moved 50x/second would not be a normal implementation too. 2x/second seems more realistic (a 300s video with a 600 pixel-wide playbar). Well, pages are most likely going to update the progress bar as often as we fire the event. So if we fire the event 50x/second then that's how often the progress bar will be updated. Sure, some pages might do things like only update on every 10th event, but this seems like a complex future since then content might start relying on that implementations use a specific firing rate. / Jonas
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
no - all I'm saying is that in 48 of the 50x/sec you get called, you can trivially figure out that nothing needs to be done, so return quickly. On Mon, Nov 17, 2008 at 3:05 PM, Jonas Sicking [EMAIL PROTECTED] wrote: Jeremy Doig wrote: i would hope that repainting a progress bar that has not moved 50x/second would not be a normal implementation too. 2x/second seems more realistic (a 300s video with a 600 pixel-wide playbar). Well, pages are most likely going to update the progress bar as often as we fire the event. So if we fire the event 50x/second then that's how often the progress bar will be updated. Sure, some pages might do things like only update on every 10th event, but this seems like a complex future since then content might start relying on that implementations use a specific firing rate. / Jonas
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
On 16.11.2008, at 16:16, Ian Hickson wrote: First of all, I'm not sure I fully understand the problem you are solving here. Could you summarize it? You don't have to fire the event if nobody is listening for it. (Browsers likely already have this implementation for mutation events if nothing else, so this shouldn't be that difficult to implement.) If I want to update a time display once per second (which I think is fairly typical) why should I listen to a flood of events 60 times per second? On some devices those events and the Javascript that needs to run to filter them might consume substantial portion of the available CPU capacity. The idea of the timeupdate event is actually to lighten the load on the user agent while simultaneously making the display more accurate than a timer could be. As far as I see what you write below does not describe anything that would make the display more accurate or lighten the load. But perhaps you have some additional mechanism in mind that you did not document here? With polling, the polling will miss key points, e.g. when the playback loops, which will result in the UI appearing to lag behind the playback. It will also cause higher processing cost while there is no need to send updates, e.g. while seeking or waiting for data, times where you really don't want extra load. The earlier iteration of the spec already fired timeupdate events on all discontinuous changes in time. How would this event flood be an improvement over that? Any reasonable application will have its polling timer running only when the playback is actually in progress. The timer can be enabled and disabled based on changes in playback and ready state. (I have earlier suggested adding a state variable and associated simple event which would change state when continuous playback starts and end, for any reason, to make this easier to use). Few other points: - We cant require time update events to be frame accurate during continuous playback since there is no way to guarantee smooth playback with that requirement. If the events are not accurate they have no benefits over simple polling. - Requiring events on every frame might make some playback optimizations impossible (by requiring constant activation of the web engine thread during playback). - Any media synchronization mechanism should be considered as a separate issue. antti
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
On Tue, Nov 18, 2008 at 12:09 PM, Antti Koivisto [EMAIL PROTECTED] wrote: On 16.11.2008, at 16:16, Ian Hickson wrote: With polling, the polling will miss key points, e.g. when the playback loops, which will result in the UI appearing to lag behind the playback. It will also cause higher processing cost while there is no need to send updates, e.g. while seeking or waiting for data, times where you really don't want extra load. The earlier iteration of the spec already fired timeupdate events on all discontinuous changes in time. How would this event flood be an improvement over that? That makes a lot more sense than limiting the timeupdate period. Firing timeupdate on discontinuous changes and requiring apps to also use a regular timer to get periodic updates sounds reasonable to me. - Requiring events on every frame might make some playback optimizations impossible (by requiring constant activation of the web engine thread during playback). Can you be more specific? Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
Jeremy Doig wrote: On Mon, Nov 17, 2008 at 3:05 PM, Jonas Sicking [EMAIL PROTECTED] wrote: Jeremy Doig wrote: i would hope that repainting a progress bar that has not moved 50x/second would not be a normal implementation too. 2x/second seems more realistic (a 300s video with a 600 pixel-wide playbar). Well, pages are most likely going to update the progress bar as often as we fire the event. So if we fire the event 50x/second then that's how often the progress bar will be updated. Sure, some pages might do things like only update on every 10th event, but this seems like a complex future since then content might start relying on that implementations use a specific firing rate. no - all I'm saying is that in 48 of the 50x/sec you get called, you can trivially figure out that nothing needs to be done, so return quickly. Are they really going to bother since things will 'work' even if they don't (just suck up a lot of CPU speed)? And are they really going to do something more clever than just update on every 10th event? / Jonas
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
Date: Tue, 18 Nov 2008 12:19:06 +1300 From: Robert O'Callahan [EMAIL PROTECTED] On Tue, Nov 18, 2008 at 12:09 PM, Antti Koivisto [EMAIL PROTECTED] wrote: On 16.11.2008, at 16:16, Ian Hickson wrote: snip With polling, the polling will miss key points, e.g. when the playback loops, which will result in the UI appearing to lag behind the playback. It will also cause higher processing cost while there is no need to send updates, e.g. while seeking or waiting for data, times where you really don't want extra load. The earlier iteration of the spec already fired timeupdate events on all discontinuous changes in time. How would this event flood be an improvement over that? That makes a lot more sense than limiting the timeupdate period. Firing timeupdate on discontinuous changes and requiring apps to also use a regular timer to get periodic updates sounds reasonable to me. snip I'd like to see the option to have an timeupdate event sent whenever the time/position has changed by more than a time period from the last one sent, that time period threshold being settable from javascript. Say I have a progress bar 120 pixels long, and a four minute long video clip is playing. I'd like to be able to set that threshold to 2 seconds, so the timeupdate event would be received each time the progress bar actually needs to be updated (i.e. the position has moved by the equivalent of at least one pixel on the progress bar). Making a distinction between progress of the position through the content, and 'real time' passing (as you'd get with a timer) becomes important when you're playing at rates other than 1.0x forward. It makes sense to me to receive more frequent update events when the content is being played at greater than 1.0x speeds (e.g. fast forward, rewind) and less frequently if in a slow-motion mode, or paused. I'm not a fan of timer-based/polling solutions, particularly for mobile devices - it's hard to know at the power management level when it's safe to start clocking down and/or turning stuff off. I guess it all depends how long you want the battery to last ;) -- Paul
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
On Tue, 28 Oct 2008, Charles Iliya Krempeaux wrote: I was looking over the spec and looking to see how I'd go about creating a custom progress bar. (Like you see as part of scrub bars.) To do this I'd imagine that a playing video would have to send out events fast enough so that the progress bar could be updated enough. Perhaps I didn't read the spec carefully enough, but I don't see any such event. On Wed, 29 Oct 2008, Chris Double wrote: You're looking for the 'timeupdate' event. This gets raised whenever the current playback position changes. From the spec section 4.8.10.8: If the time was reached through the usual monotonic increase of the current playback position during normal playback, the user agent must then queue a task to fire a simple event called timeupdate at the element. (In the other cases, such as explicit seeks, relevant events get fired as part of the overall process of changing the current playback position.) Although this is hidden away in the cue ranges section, it happens on normal playback without cue ranges. I've tried to make it less hidden. On Fri, 31 Oct 2008, Eric Carlson wrote: I think requiring a user agent to post events as a movie plays is a mistake. It will mean that *every* page will incur the overhead, even if no scripts are listening for the events. You don't have to fire the event if nobody is listening for it. (Browsers likely already have this implementation for mutation events if nothing else, so this shouldn't be that difficult to implement.) I understand that some pages will need to keep scripted UI in sync with movie playback, but this can be done very easily with a JavaScript timer: The idea of the timeupdate event is actually to lighten the load on the user agent while simultaneously making the display more accurate than a timer could be. With polling, the polling will miss key points, e.g. when the playback loops, which will result in the UI appearing to lag behind the playback. It will also cause higher processing cost while there is no need to send updates, e.g. while seeking or waiting for data, times where you really don't want extra load. Using a timer also allows a script to update at exactly the frequency they need instead of at whatever frequency the UA happens to post events. Using a timer also allows them to go too fast. :-) Video and audio playback is already extremely CPU intensive, we shouldn't require the UA to burn extra cycles doing unnecessary work. I agree. That was exactly the thinking behind the timeupdate event. It allows the UA to determine how fast to update the UI without hurting performance. Basically it puts the UA in charge of the performance critical aspects instead of hoping that the author will work it out. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
Ian Hickson wrote: Video and audio playback is already extremely CPU intensive, we shouldn't require the UA to burn extra cycles doing unnecessary work. I agree. That was exactly the thinking behind the timeupdate event. It allows the UA to determine how fast to update the UI without hurting performance. Basically it puts the UA in charge of the performance critical aspects instead of hoping that the author will work it out. Though if all implementations are saying that it has the opposite effect then clearly we need to look into if something went wrong :) I think one problem is that a timer can be set to fire much less often than the current time changes. For example the UI doesn't need to be updated more than maybe twice a second for most videos (since it won't move more than one pixel in that time), whereas the timerupdate event sounds like it should fire with the framerate of the video, somewhere around 25-30 times per second. / Jonas
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
On Sun, 16 Nov 2008, Jonas Sicking wrote: I think one problem is that a timer can be set to fire much less often than the current time changes. For example the UI doesn't need to be updated more than maybe twice a second for most videos (since it won't move more than one pixel in that time), whereas the timerupdate event sounds like it should fire with the framerate of the video, somewhere around 25-30 times per second. The spec allows the event to be fired at any rate from 4Hz to 66Hz at the moment. I can adjust that range based on implementation experience if that would help. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
On Mon, Nov 17, 2008 at 3:19 PM, Jonas Sicking [EMAIL PROTECTED] wrote: Ian Hickson wrote: Video and audio playback is already extremely CPU intensive, we shouldn't require the UA to burn extra cycles doing unnecessary work. I agree. That was exactly the thinking behind the timeupdate event. It allows the UA to determine how fast to update the UI without hurting performance. Basically it puts the UA in charge of the performance critical aspects instead of hoping that the author will work it out. Though if all implementations are saying that it has the opposite effect then clearly we need to look into if something went wrong :) I don't think firing timeupdate on every frame is *that* bad for us, to be honest. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
Eric Carlson wrote:
Ian -
On Oct 28, 2008, at 10:36 PM, Chris Double wrote:
On Wed, Oct 29, 2008 at 5:22 PM, Charles Iliya Krempeaux
[EMAIL PROTECTED] wrote:
Perhaps I didn't read the spec carefully enough, but I don't see any
such
event.
You're looking for the 'timeupdate' event. This gets raised whenever
the current playback position changes. From the spec section 4.8.10.8:
If the time was reached through the usual monotonic increase of the
current
playback position during normal playback, the user agent must then
queue a task
to fire a simple event called timeupdate at the element. (In the other
cases,
such as explicit seeks, relevant events get fired as part of the overall
process of changing the current playback position.)
Although this is hidden away in the cue ranges section, it happens on
normal playback without cue ranges.
I think requiring a user agent to post events as a movie plays is a
mistake. It will mean that *every* page will incur the overhead, even if
no scripts are listening for the events.
I understand that some pages will need to keep scripted UI in sync
with movie playback, but this can be done very easily with a JavaScript
timer:
document.addEventListener('play', updateTimeDisplay, true);
document.addEventListener('pause', updateTimeDisplay, true);
function updateTimeDisplay()
{
var vid = document.getElementById('video');
if (!vid.paused !vid.ended vid.networkState
HTMLMediaElement.HAVE_FUTURE_DATA)
setTimeout(updateTimeDisplay, 200);
// update display here
}
Using a timer also allows a script to update at exactly the frequency
they need instead of at whatever frequency the UA happens to post events.
Video and audio playback is already extremely CPU intensive, we
shouldn't require the UA to burn extra cycles doing unnecessary work.
I agree. For the normal as-time-passes changes to currentTime I don't
think an event is needed. When currentTime is changed either directly or
though looping (if we end up supporting some type of looping) it makes
sense to dispatch an event.
/ Jonas
[whatwg] Video Element Events? - Use Case: Custom Progress Bar
Hello,
(Sorry if this has come up before and I missed it.)
I was looking over the spec and looking to see how I'd go about creating a
custom progress bar. (Like you see as part of scrub bars.)
To do this I'd imagine that a playing video would have to send out events
fast enough so that the progress bar could be updated enough.
Perhaps I didn't read the spec carefully enough, but I don't see any such
event.
I'd expect to be able to do something like...
videoElem = document.getElementById(aVideoElement);
videoElem.addEventListener( moment,
function()
{
// Handle the event.
// I.e., update the progress bar or something.
//
// I'll need a way to know a what point in time the video is at now.
}
,false);
Note I used the fictitious moment event. (I doesn't have to be called
moment. I just made up something.)
*Is there anything like this already?*
Or do you just set up a timer and poll the currentTime attribute.
I.e., something like...
function updateProgressBar(id)
{
videoElem = document.getElementById(id);
// Update the progress bar using: videoElem.currentTime
}
setTimeout(updateProgressBar(\aVideoElement\), 100);
I suppose the advantage of the polling method is that you can specify how
often you do the update. But it doesn't seems to make for poorer developer
usability.
--
Charles Iliya Krempeaux, B.Sc.
http://ChangeLog.ca/
Re: [whatwg] Video Element Events? - Use Case: Custom Progress Bar
On Wed, Oct 29, 2008 at 5:22 PM, Charles Iliya Krempeaux [EMAIL PROTECTED] wrote: Perhaps I didn't read the spec carefully enough, but I don't see any such event. You're looking for the 'timeupdate' event. This gets raised whenever the current playback position changes. From the spec section 4.8.10.8: If the time was reached through the usual monotonic increase of the current playback position during normal playback, the user agent must then queue a task to fire a simple event called timeupdate at the element. (In the other cases, such as explicit seeks, relevant events get fired as part of the overall process of changing the current playback position.) Although this is hidden away in the cue ranges section, it happens on normal playback without cue ranges. Chris. -- http://www.bluishcoder.co.nz
