Re: [whatwg] HTML5 video seeking

[Ian Hickson]

On Mon, 14 Nov 2011, Aaron Colwell wrote:
> 
> I was looking at the seeking algorithm and had a question about step 10.
> 
># 10. Wait until the user agent has established whether or not the media 
># data for the new playback position is available, and, if it is, until 
># it has decoded enough data to play back that position.
> 
> Does this mean the user agent must resume playback at the exact location 
> specified?

Since you asked this question, the spec has been updated to explicitly 
support, in certain cases, approximate seeking. The answer, therefore, is 
now "yes, unless the approximate-for-speed flag is set". The flag gets set 
by the fastSeek() method, and optionally by native seek bars.


> What if the nearest keyframe is several seconds away? Is the UA expected 
> to decode and toss the frames instead of starting playback at the 
> nearest keyframe?

Yes.


> On desktop machines I don't think this would be a problem, but on mobile 
> devices it might be since the hardware may not be able to decode 
> significantly faster than realtime. What is the intended behavior for 
> such constrained devices?

Same as for desktop.

Having said that:

On Mon, 14 Nov 2011, Ralph Giles wrote:
> 
> Maybe you can muck with the 'media.seekable' TimeRanges object to only 
> show keyframes?

This would indeed be a plausible solution.


On Tue, 15 Nov 2011, Aaron Colwell wrote:
> 
> Yeah I thought about this, but it seems a little hacky and not what the 
> spec intended for seekable.

Well, it's really intended for cases where the format simply has no way to 
seek to the given position, but if you don't want to implement seeking to 
a particular position, then you don't have to, so long as you don't 
include it in seekable.

This is a quality of implementation feature. The idea UA would cache all 
incoming data and implement instantaneous arbitrary seeking and still 
miraculously use no memory, but real implementations have to make 
tradeoffs. The media element feature exposes a number of ways for authors 
to detect those tradeoffs; one of them is "seekable".


On Wed, 16 Nov 2011, Ralph Giles wrote:
> 
> Video splicing/mixing was a use case we wanted to support, and such 
> applications aren't really possible without frame-accurate seeking. 
> Thus, it's better to require it up front and possibly allow applications 
> to relax it later, as with Frank and Philip's 'strict' attribute, than 
> to disallow such applications by leaving this to implementation 
> variance.

Right, exactly. The seekable attribute can be used to indicate that this 
isn't supported, so that the author can detect such implementations, but I 
would indeed not expect authors to like such an implementation.


On Thu, 17 Nov 2011, Philip Jägenstedt wrote:
> On Wed, 16 Nov 2011 19:36:26 +0100, Frank Galligan 
>  wrote:
> > Here is another link where Ian mentions that the seek should be the 
> > exact frame. 
> > http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2008-November/017225.html
> >  
> > This came up from 2 years ago, where I wanted to add an optional 
> > "strict" parameter to the seek call, which would default to false. 
> > Exactly the same as what Philip suggested. I'm all for it.
> > 
> > I think Chrome added frame accurate seeking in version 7.
> 
> I filed http://www.w3.org/Bugs/Public/show_bug.cgi?id=14851 to see if we 
> can make this happen.

This ended up resulting in fastSeek();


On Fri, 13 Jan 2012, Hugh Guiney wrote:
>
> I'm currently creating a custom HTML video player UI. What I'm trying to 
> achieve is a seek bar control�a horizontal slider that allows the user 
> to seek to a specific point in time. Right now the basic functionality 
> of this can be achieved using a range input with a max value set to the 
> video's duration, and which sets the video's currentTime onchange. 
> However, there are two problems with this:
> 
> The first is that I'd like to be able to show the user, before selecting 
> a point in the range, what point they are about to seek to. At present, 
> even if there is an element that exposes currentTime to the user, 
> there's no way to show them what currentTime *will* be until they 
> actually select a point on the range. Although I realize range inputs 
> are meant to be imprecise, it's often helpful to be able to see more 
> precise values one at a time, so that if there is a value they're aiming 
> for, they can find it or get close enough with less trial and error than 
> blind selection.
>
> It has become a ubiquitous UI convention on the Web to provide time code 
> and/or frame previews in a tooltip as the user scans the seek bar (but 
> before seeking). This can be seen for example on YouTube, Vimeo, Hulu, 
> blip.tv, Brightcove, Firefox, and many JS library players. And although 
> the effect is most common in video players, it would be helpful in audio 
> players, or any other situation where a user wants to browse available 
> options within a range without committing to any right 

Re: [whatwg] HTML5 video seeking

[Philip Jägenstedt]

On Wed, 16 Nov 2011 19:36:26 +0100, Frank Galligan   
wrote:



Here is another link where Ian mentions that the seek should be the exact
frame.
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2008-November/017225.html
 This came up from 2 years ago, where I wanted to add an optional  
"strict"
parameter to the seek call, which would default to false. Exactly the  
same

as what Philip suggested. I'm all for it.

I think Chrome added frame accurate seeking in version 7.


I filed http://www.w3.org/Bugs/Public/show_bug.cgi?id=14851 to see if we  
can make this happen.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] HTML5 video seeking

[Ralph Giles]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 15/11/11 10:32 AM, Aaron Colwell wrote:

> Yeah it looks to me like starting at the requested position is the
> only option. I saw the text about media engine triggered seeks, but
> it seems like users would be very surprised to see the seeking &
> seeked events for the seek they requested immediately followed by a
> pair of events to a location they didn't request. I can envision
> the angry bug reports now. ;)

Yeah, it's definitely bending the rules. If you only intended to
support seeking to the nearest keyframe, setting media.seekable would
be an honest way to advertise that, but also violates least surprise.

> Thanks for the response. Looks like we are interpreting the text
> the same way.

Yes, my recollection of the earlier discussion aligns with your
summary and Chris Double's. It's expensive, but what one naively
expects the API to do.

Video splicing/mixing was a use case we wanted to support, and such
applications aren't really possible without frame-accurate seeking.
Thus, it's better to require it up front and possibly allow
applications to relax it later, as with Frank and Philip's 'strict'
attribute, than to disallow such applications by leaving this to
implementation variance.

 -r

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOxDrhAAoJEEcAD3uxRB3vHT4H/0DfMChkHztMwqO0FEkJql3u
BYU8B0ZFxQ/rllU9qdTdu+ioYRvIriFP9UFbeO+sO85Qy5Jaz9u9soKWE8siIHMP
rWWxOVQbbZMbbLcrtgbNreePwyRX6P1fdTTpxjUUnl0g/mVajE+5BohaVwsc/dSK
mr0S53a49od3675dNQQaycLbSAEI8eaVvG5saOyOfN41GK+ctEtnfro7Z0cUZhzZ
A0C3P+/Jr+fKOZPocpJ4RPPWbkzCeO8BZOblRgrHynTpHYs20OTBbvW3TMZuT6Np
hnwVclFwye1hgqYjAR83PezVAz/9rPKsox96+VQc/in7fne2H5drKIMW0ADd+hM=
=r1zI
-END PGP SIGNATURE-

Re: [whatwg] HTML5 video seeking

[Frank Galligan]

Here is another link where Ian mentions that the seek should be the exact
frame.
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2008-November/017225.html
 This came up from 2 years ago, where I wanted to add an optional "strict"
parameter to the seek call, which would default to false. Exactly the same
as what Philip suggested. I'm all for it.

I think Chrome added frame accurate seeking in version 7.

Frank


On Tue, Nov 15, 2011 at 10:37 PM, Chris Double wrote:

> On Wed, Nov 16, 2011 at 7:19 AM, Aaron Colwell 
> wrote:
> > I'm just trying to
> > determine if we are intentionally requiring frame accurate seeking at
> this
> > point or am I just misinterpreting some part of the spec.
>
> My understanding from the discussion at the time was that it was
> intentionally requiring frame accurate seeking and this is what
> Mozilla ended up implementing in Firefox as a result.
>
> Chris.
> --
> http://www.bluishcoder.co.nz
>

Re: [whatwg] HTML5 video seeking

[Chris Double]

On Wed, Nov 16, 2011 at 7:19 AM, Aaron Colwell  wrote:
> I'm just trying to
> determine if we are intentionally requiring frame accurate seeking at this
> point or am I just misinterpreting some part of the spec.

My understanding from the discussion at the time was that it was
intentionally requiring frame accurate seeking and this is what
Mozilla ended up implementing in Firefox as a result.

Chris.
-- 
http://www.bluishcoder.co.nz

Re: [whatwg] HTML5 video seeking

[Aaron Colwell]

Hi Ralph,

On Mon, Nov 14, 2011 at 3:58 PM, Ralph Giles  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 14/11/11 03:49 PM, Aaron Colwell wrote:
>
> > Does this mean the user agent must resume playback at the exact
> > location specified?
>
> Maybe you can muck with the 'media.seekable' TimeRanges object to only
> show keyframes?
>

Yeah I thought about this, but it seems a little hacky and not what the
spec intended for seekable.



> Otherwise, it kind of sounds like you're supposed to start playback at
> the requested position. The final paragraph of that section suggests
> another out: you can reposition the playback head inside the playback
> engine as long as you clip to media.seekable and fire the timeupdate
> and seeked events.
>

Yeah it looks to me like starting at the requested position is the only
option. I saw the text about media engine triggered seeks, but it seems
like users would be very surprised to see the seeking & seeked events for
the seek they requested immediately followed by a pair of events to a
location they didn't request. I can envision the angry bug reports now. ;)

Thanks for the response. Looks like we are interpreting the text the same
way.

Aaron

Re: [whatwg] HTML5 video seeking

[Aaron Colwell]

Hi Philip,

Thanks for the link. It took me a little while to read the whole thread.

I'm not looking to support multiple styles of seeking. I was just noticing
that the spec text does not appear to allow you to seek to a position
different from the exact time specified. Steps 5-7 appear to bound the "new
playback position" based on the start, end, and seekable ranges, but I'm
not seeing any text that allows "new playback position" to be modified
based on other factors like keyframes locations. I'm just trying to
determine if we are intentionally requiring frame accurate seeking at this
point or am I just misinterpreting some part of the spec.

Aaron

On Tue, Nov 15, 2011 at 1:30 AM, Philip Jägenstedt wrote:

> On Tue, 15 Nov 2011 00:49:32 +0100, Aaron Colwell 
> wrote:
>
>  Hi,
>>
>> I was looking at the seeking
>> algorithm> work/multipage/the-video-**element.html#seeking
>> >
>>
>> and
>> had a question about step 10.
>>
>> 10. Wait until the user agent has established whether or not the media
>> data for the new playback position is available, and, if it is, until it
>> has decoded enough data to play back that position.
>>
>>
>> Does this mean the user agent must resume playback at the exact location
>> specified?
>> What if the nearest keyframe is several seconds away?
>> Is the UA expected to decode and toss the frames instead of starting
>> playback at the nearest keyframe?
>>
>> On desktop machines I don't think this would be a problem, but on mobile
>> devices it might be since the hardware may not be able to decode
>> significantly faster than realtime. What is the intended behavior for such
>> constrained devices?
>>
>> Aaron
>>
>
> This was last discussed in the huge thread "HTML5 video: frame accuracy /
> SMPTE" where I made the suggestions [1] to introduce a seek() function to
> allow both frame-accurate and fast seeking. If you like that idea maybe we
> can just spec and implement it?
>
> [1] http://lists.whatwg.org/htdig.**cgi/whatwg-whatwg.org/2011-**
> January/029806.html
>
> --
> Philip Jägenstedt
> Core Developer
> Opera Software
>

Re: [whatwg] HTML5 video seeking

[Philip Jägenstedt]

On Tue, 15 Nov 2011 00:49:32 +0100, Aaron Colwell   
wrote:



Hi,

I was looking at the seeking
algorithm
and
had a question about step 10.

10. Wait until the user agent has established whether or not the media
data for the new playback position is available, and, if it is, until it
has decoded enough data to play back that position.


Does this mean the user agent must resume playback at the exact location
specified?
What if the nearest keyframe is several seconds away?
Is the UA expected to decode and toss the frames instead of starting
playback at the nearest keyframe?

On desktop machines I don't think this would be a problem, but on mobile
devices it might be since the hardware may not be able to decode
significantly faster than realtime. What is the intended behavior for  
such

constrained devices?

Aaron


This was last discussed in the huge thread "HTML5 video: frame accuracy /  
SMPTE" where I made the suggestions [1] to introduce a seek() function to  
allow both frame-accurate and fast seeking. If you like that idea maybe we  
can just spec and implement it?


[1]  
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-January/029806.html


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] HTML5 video seeking

[Simon Pieters]

On Tue, 15 Nov 2011 00:49:32 +0100, Aaron Colwell   
wrote:



Hi,

I was looking at the seeking
algorithm
and
had a question about step 10.

10. Wait until the user agent has established whether or not the media
data for the new playback position is available, and, if it is, until it
has decoded enough data to play back that position.


Does this mean the user agent must resume playback at the exact location
specified?
What if the nearest keyframe is several seconds away?
Is the UA expected to decode and toss the frames instead of starting
playback at the nearest keyframe?

On desktop machines I don't think this would be a problem, but on mobile
devices it might be since the hardware may not be able to decode
significantly faster than realtime. What is the intended behavior for  
such

constrained devices?

Aaron


I believe this is a quality of implementation issue.

--
Simon Pieters
Opera Software

Re: [whatwg] HTML5 video seeking

[Ralph Giles]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 14/11/11 03:49 PM, Aaron Colwell wrote:

> Does this mean the user agent must resume playback at the exact
> location specified?

Maybe you can muck with the 'media.seekable' TimeRanges object to only
show keyframes?

Otherwise, it kind of sounds like you're supposed to start playback at
the requested position. The final paragraph of that section suggests
another out: you can reposition the playback head inside the playback
engine as long as you clip to media.seekable and fire the timeupdate
and seeked events.

 -r

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJOwasKAAoJEEcAD3uxRB3vOqkH/0QFNAiOir+EOgZaAhZmhoub
xr0CThlwEIHUoo6TbDHJmqPRaiKVu9hobkf7DScG2yhjUDaT2vTptF2Wg+lgI2LE
pRdjSUi0hArKrNmC8zCV+zG/82yE0l+RlBXOLjLPKXBo0PDqovXKbOknlnv68/P7
0vyhB9t7L8zLhDCL0BEbuF5OoikpW1Zt9iru+ThbY+bU7RTCFSvE0MmnqMAB3MOx
7HOa2liovoeUotoFVEpDMnD5ZbSxbmifax0CRRdcb9u4m/7HG4EaVoK9GOjxmGzA
zqCtVZ7Yb+hoAPQIXP4tqsrgM9ma/U0LKgYj1lQMjD4whyp96X+iv/vNuxvVRh8=
=tort
-END PGP SIGNATURE-