[whatwg] Closing the WHATWG mailing lists

[Philip Jägenstedt]

Hi all,

This list has not been very active in recent years and maintaining the
mailing list setup as our hosting infrastructure change has become a
bit of a burden. Because of this, I'm proposing that this and the
other (even less active) lists are closed in favor of GitHub issues.
If you have concerns about this, please reply here or on this tracking
issue: https://github.com/whatwg/meta/issues/153

Details follow.

wha...@whatwg.org as well as the "help" and "implementors" lists will
be configured to auto-reply with an explanation, pointing to
https://whatwg.org/mailing-list which is being updated in
https://github.com/whatwg/whatwg.org/pull/269.

lists.whatwg.org will also be redirected to
https://whatwg.org/mailing-list with an effort made to help find the
original link on web.archive.org.

Philip

Re: [whatwg] metadata

[Philip Jägenstedt]

On Mon, Apr 10, 2017 at 8:38 AM Andy Valencia 
wrote:

> What follows is a first pass at addressing a missing ability
> when dealing with Internet streams, usually radio ones.
> Comments and suggestions are quite welcome; as my first
> attempt--ever--at submitting to this group, apologies if
> I've made any mistakes in how I've proceeded.
>
> Thanks,
> Andy Valencia
> Contact: https://vsta.org/contact/andy
>
> 
> Proposal for enhancement to HTML5  element to better support
> metadata provided by streams.
>
> # Background
>
> Browsers have an  element which can directly play streaming
> URL's.  When the stream is an Internet radio station, the stream
> almost certainly offers metadata; typically, the artist and track,
> and often an album cover URL as well.  While modern browsers can
> play these streams, there is no way to access this metadata.
>
> Media elements in modern browsers _do_ have the notion of metadata,
> but in current standards this is limited to static characteristics
> such as duration.  When listening to a radio stream, the metadata will
> change as successive tracks are played.
>
> # Stream Delivery of Dynamic Metadata
>
> A moderately detailed description of current practices in providing
> metadata is provided at:
>
> http://www.smackfu.com/stuff/programming/shoutcast.html
>
> (One detail glossed over in this page is that older streaming servers
> start their GET response with "ICY 200 OK" rather than a standard
> HTTP response.  This technically means the response is HTTP 0.9
> without headers; the headers are present, but must be processed by
> the application and trimmed from the start of the media stream
> in the GET response.  Apple Safari already declines to play 0.9
> media elements within a post-0.9 page, and there are signs that
> Chrome will follow suit.  Thus, accomodating this older mode should
> be considered optional.)
>
> Newer streaming server versions use a proper HTTP response, with
> the documented elements in the HTTP response header.
>
> Because the metadata is encoded in a general attribute=value
> format, a wide variety of metadata may be encoded.  By convention,
> at least the attributes "StreamTitle" and "StreamUrl" will be
> included in the response.  Also by convention, StreamTitle is
> structured as "Artist - Track Name".
>
> # API Considerations
>
> While listening to streams with metadata is a compelling application
> of web technology, it is probably a tiny percentage of the use
> of  elements.  Thus, this proposal describes a mechanism
> which leaves the default browser behavior unchanged.  It also
> gracefully degrades when presented to a non-implementing
> browser, still permitting stream play while in the existing
> non-metadata fashion.
>
> This proposal is designed with the current state of streaming,
> currently depending on the HTTP Icy-MetaData header element.  However,
> this specific detail is abstracted, in recognition that streaming
> technology could evolve in the future.  The intention is that this API
> will remain stable even if the details of metadata request and
> delivery change.
>
> As noted, current HTML5 media elements do have some metadata support.
> This API is also structured to permit these existing elements to
> participate in this API if desired.
>
> #  element changes
>
> These enhancements are activated when the  element has the
> new attribute "dynamicmetadata" present.  Without this attribute,
> no metadata request header is added to the stream HTTP request, and no
> other attributes of this proposal will be acted upon even if present.
>
> If the server response indicates support for dynamic metadata,
> on each metadata update, the  element's attributes are changed
> to reflect the latest received metadata.  For each "Attribute=Value"
> in the metadata update, an attribute "metaAttribute" with value "Value"
> will
> be added to the  element.  Previous metadata attributes which
> are not present in the latest update are left untouched.  The browser
> must verify well-formed identifier names for each Attribute, and
> quietly reject ill-formed names.  It can also apply checks on the
> Value.  (Implementors are reminded that the Value, because it encodes
> information such as names, might contain a wide range of character
> encodings.)
>
> The  element adds the hook "onmetadatachange", connecting to
> a function which is called on each update of metadata.  This
> function is called with an event, which includes the attribute
> "changed" with a value which is a list of Attribute names which
> are present in the update.
>
> # Example code
>
> 
> function new_meta(ev) {
> const changes = ev.changed;
> if (changes.indexOf("StreamTitle") >= 0) {
> const title = player.metaStreamTitle;
> const idx = title.indexOf(" - ");
> if (idx == -1) {
> // "Just a plain old piece of text"
> artist.textContent = title;
> 

Re: [whatwg] VIDEO and pitchAdjustment

[Philip Jägenstedt]

On Wed, Sep 28, 2016 at 5:18 PM, Garrett Smith <dhtmlkitc...@gmail.com> wrote:
> On Wed, Mar 2, 2016 at 2:36 AM, Philip Jägenstedt <phil...@opera.com> wrote:
>> On Wed, Mar 2, 2016 at 5:08 PM, Paul Adenot <pade...@mozilla.com> wrote:
>>>
>>> Gecko uses code that can do arbitrary pitch adjustment, unrelated to 
>>> time-stretching (which is speed change with pitch compensation).
>>
>> That's neat. If you're interested in exposing this as an API on
>> HTMLMediaElement, can you file a spec bug with a proposal of how it
>> might work?
>>
>
> Please post a link to that spec bug.

I don't think a bug was ever filed, but you can file one at
https://github.com/whatwg/html/issues

-- 
Philip Jägenstedt

Re: [whatwg] VIDEO and pitchAdjustment

[Philip Jägenstedt]

On Wed, Mar 2, 2016 at 5:08 PM, Paul Adenot  wrote:
>
> Gecko uses code that can do arbitrary pitch adjustment, unrelated to 
> time-stretching (which is speed change with pitch compensation).

That's neat. If you're interested in exposing this as an API on
HTMLMediaElement, can you file a spec bug with a proposal of how it
might work?

Philip

Re: [whatwg] VIDEO and pitchAdjustment

[Philip Jägenstedt]

On Wed, Mar 2, 2016 at 9:19 AM, Garrett Smith <dhtmlkitc...@gmail.com> wrote:
> On Thu, Nov 12, 2015 at 11:32 AM, Philip Jägenstedt <phil...@opera.com> wrote:
>> On Thu, Nov 12, 2015 at 10:55 AM, Garrett Smith <dhtmlkitc...@gmail.com>
>> wrote:
>>> On 11/12/15, Philip Jägenstedt <phil...@opera.com> wrote:
>>>> On Thu, Nov 12, 2015 at 9:07 AM, Garrett Smith <dhtmlkitc...@gmail.com>
>>>> wrote:
>>>>>
>>>>> On 10/19/15, Philip Jägenstedt <phil...@opera.com> wrote:
>>>>> > On Tue, Sep 1, 2015 at 11:21 AM, Philip Jägenstedt <phil...@opera.com>
>>>>> > wrote:
>>>>> >> On Mon, Aug 31, 2015 at 9:48 PM, Domenic Denicola <d...@domenic.me>
>>>>> >> wrote:
>>>>> >>> From: Eric Carlson [mailto:eric.carl...@apple.com]
>
>> Two things.
>>
>> 1. Do the underlying media frameworks that browsers are using support
>> arbitrary pitch changes, or do they also only have the limited
>> preservesPitch-style API?
>
> Are there any problems getting in the way of pitch adjustment (without
> depending on playbackRate)?

I don't know, that was basically my question too. If the underlying
APIs don't support it, that's a problem that needs to be fixed first.

Philip

Re: [whatwg] Removing mediagroup/MediaController from HTML

[Philip Jägenstedt]

On Tue, Nov 3, 2015 at 4:20 PM, David Singer  wrote:
>
>> On Oct 3, 2015, at 13:39 , Silvia Pfeiffer 
wrote:
>>
>> On Fri, Oct 2, 2015 at 10:27 AM, Domenic Denicola  wrote:
>>> From: whatwg [mailto:whatwg-boun...@lists.whatwg.org] On Behalf Of
>>>
 is removal really the right thing to do, given that we have an
 implementation?
>>>
>>> I agree this is a problematic question. I opened
https://github.com/whatwg/html/issues/209 for the more general issue but am
happy to have the discussion here since that hasn't gotten much replies. Do
check out the examples listed there though. E.g. Blink is in similar
situations with  and HTML imports.
>>>
>>> The web seems to end up with a lot of APIs like this, where the spec
ends up just being documentation for a single-vendor implementation. I
don't really know what to do in these cases. If our goal in writing these
specs is to produce an interoperable web platform, then such features seem
like they shouldn't be part of the platform.
>>
>>
>> There is also a question about the why of the current state: is it
>> just a single-vendor implementation because nobody at the other
>> vendors has gotten around to implementing it or is it because they
>> fundamentally object to implementing it. If there are objections, then
>> it's reasonable to consider removing the feature. Otherwise, it would
>> be premature to remove it IMHO.
>
> Yes.  It wasn’t even our proposal (see <
https://lists.w3.org/Archives/Public/public-html/2011Mar/0436.html>) and we
feel it answers some important cases that we can’t otherwise answer.  Some
insights from others would be welcome.

My main concern with the API, and the reason I unshipped it in Blink, is
the apparent difficulty in implementing it really well, to actually get the
sync sample-accurate at the media framework level. I understand that it's
not really *that* hard in principle, but still enough work that it wasn't
done for Safari, or Chromium. Some frameworks seem to make it fairly
straightforward, I wouldn't be surprised if QuickTime has supported it
since the 90's. When I looked into implementing it using GStreamer for
Presto, it seemed like it was a matter of using a single pipeline with a
single audio output node and thus a single clock. What seems quite tricky
to me is to dynamically split or merge pipelines as media elements join or
leave a media controller. I never really looked much deeper than this,
maybe it's quite doable.

Another more recent concern that I have is that media controller builds on
top of an already quite high-level abstraction that is HTMLMediaElement. In
the interests of a layered and (somewhat) rational platform, I wonder if it
wouldn't be more sensible to crack open HTMLMediaElement, exposing the
primitives all the way down clocks and audio output devices, so that one
could create and connect demuxers and decoders to the same driving clock in
a way that's more similar to how you'd to it using the underlying
frameworks. Of course this isn't even half-baked, and it would be a
many-year project to bring sanity to the media stack (media elements + MSE
+ EME + Web Audio + WebRTC) of the web.

Philip

Re: [whatwg] VIDEO and pitchAdjustment

[Philip Jägenstedt]

On Thu, Nov 12, 2015 at 9:43 AM, Eric Carlson <eric.carl...@apple.com> wrote:
>
>> On Nov 12, 2015, at 9:34 AM, Philip Jägenstedt <phil...@opera.com> wrote:
>>
>> On Thu, Nov 12, 2015 at 9:07 AM, Garrett Smith <dhtmlkitc...@gmail.com 
>> <mailto:dhtmlkitc...@gmail.com>> wrote:
>>>
>>> On 10/19/15, Philip Jägenstedt <phil...@opera.com> wrote:
>>>> On Tue, Sep 1, 2015 at 11:21 AM, Philip Jägenstedt <phil...@opera.com>
>>>> wrote:
>>>>> On Mon, Aug 31, 2015 at 9:48 PM, Domenic Denicola <d...@domenic.me> wrote:
>>>>>> From: Eric Carlson [mailto:eric.carl...@apple.com]
>>>>>>
>>>
>>> [...]
>>>> I've filed a spec issue to make it so:
>>>> https://github.com/whatwg/html/issues/262
>>>>
>>>> If there's any implementor interest in pitch control that goes beyond
>>>> (independently) or that, please file a separate issue.
>>>>
>>>
>>> They won't.
>>>
>>> You can hold the standard of "they need to come here and post up
>>> cogent arguments in favor of feature X", but it ain't gonna happen
>>> that way.
>>>
>>> There just isn't a whole lot of money in music education. How many
>>> music education companies are W3C members?
>>>
>>> Unlike technology companies like Facebook, Google, Nokia, Opera, and
>>> other companies the post here, small music education operations like
>>> artistworks, jammit, licklibrary, etc are more about their domain —
>>> "music" — than they are about technology.
>>>
>>> Major music education websites are still using Flash; their developers
>>> are busy fixing broken links, making the login feature, database, etc
>>> work, etc. Flash is not nice but they apparently were not funded or
>>> motivated enough by the existing HTML5 HTMLMediaElement to use it
>>> instead.
>>>
>>> Control over playbackRate has a greater value than pitch control. But
>>> those sites don't even allow the students to change the playbackRate
>>> because they're still using Flash.
>>>
>>> You won't read posts here about what students have to say about it the
>>> value of having HTML5 vs Flash, or independent control over pitch and
>>> playbackRate.
>>
>> Have you investigated whether you can achieve your use cases using the
>> Web Audio API? If it isn't possible, is there a small addition to Web
>> Audio that would solve the problem?
>>
>> It is unfortunately quite hard to get all browser vendors (or even
>> one) interested in implementing support for something that is only
>> expected to benefit a niche use case, but we should strive to make
>> available the primitives that would it possible to implement yourself.
>>
>   I am actually quite ambivalent about this feature - it is currently broken 
> on OSX and it has never been implemented on iOS, but as far as I can see we 
> haven’t received a single bug report about this.

Ah. I removed webkitPreservesPitched entirely from Blink because it
wasn't implemented on any platforms, maybe you could do the same on
iOS?

More importantly, is it possible to support this on iOS, and is it
likely to bubble to the top of your priorities any time soon if it
were added to the spec? I don't think it's high priority for anyone
working on Chromium, but unless Safari and Firefox are ready to drop
their prefixed APIs entirely it still seems like a good idea to get
what little interop we can by standardizing.

Philip

Re: [whatwg] VIDEO and pitchAdjustment

[Philip Jägenstedt]

On Thu, Nov 12, 2015 at 10:55 AM, Garrett Smith <dhtmlkitc...@gmail.com>
wrote:
> On 11/12/15, Philip Jägenstedt <phil...@opera.com> wrote:
>> On Thu, Nov 12, 2015 at 9:07 AM, Garrett Smith <dhtmlkitc...@gmail.com>
>> wrote:
>>>
>>> On 10/19/15, Philip Jägenstedt <phil...@opera.com> wrote:
>>> > On Tue, Sep 1, 2015 at 11:21 AM, Philip Jägenstedt <phil...@opera.com>
>>> > wrote:
>>> >> On Mon, Aug 31, 2015 at 9:48 PM, Domenic Denicola <d...@domenic.me>
>>> >> wrote:
>>> >>> From: Eric Carlson [mailto:eric.carl...@apple.com]
>>> >>>
>>>
>>> [...]
>>> > I've filed a spec issue to make it so:
>>> > https://github.com/whatwg/html/issues/262
>>> >
>>> > If there's any implementor interest in pitch control that goes beyond
>>> > (independently) or that, please file a separate issue.
>>> >
>>>
>>> They won't.
>>>
>>> You can hold the standard of "they need to come here and post up
>>> cogent arguments in favor of feature X", but it ain't gonna happen
>>> that way.
>>>
>>> There just isn't a whole lot of money in music education. How many
>>> music education companies are W3C members?
>>>
>>> Unlike technology companies like Facebook, Google, Nokia, Opera, and
>>> other companies the post here, small music education operations like
>>> artistworks, jammit, licklibrary, etc are more about their domain —
>>> "music" — than they are about technology.
>>>
>>> Major music education websites are still using Flash; their developers
>>> are busy fixing broken links, making the login feature, database, etc
>>> work, etc. Flash is not nice but they apparently were not funded or
>>> motivated enough by the existing HTML5 HTMLMediaElement to use it
>>> instead.
>>>
>>> Control over playbackRate has a greater value than pitch control. But
>>> those sites don't even allow the students to change the playbackRate
>>> because they're still using Flash.
>>>
>>> You won't read posts here about what students have to say about it the
>>> value of having HTML5 vs Flash, or independent control over pitch and
>>> playbackRate.
>>
>> Have you investigated whether you can achieve your use cases using the
>> Web Audio API? If it isn't possible, is there a small addition to Web
>> Audio that would solve the problem?
>>
>
> https://html.spec.whatwg.org/multipage/embedded-content.html#audiotrack

I don't understand, isn't the AudioTrack API (enabling/disabling individual
audio tracks) separate a separate concern? Or are you saying that there's
no way to get the audio tracks separately into the Web Audio API in order
to process them differently? That is certainly true, and if having that
capability would allow you to solve the problem with the Web Audio API, I
think that seems like a powerful primitive to have.

>> It is unfortunately quite hard to get all browser vendors (or even
>> one) interested in implementing support for something that is only
>> expected to benefit a niche use case, but we should strive to make
>> available the primitives that would it possible to implement yourself.
>>
>
> The situation where you want hear the audio at the desired pitch,
> while simultaneously watching the video at the desired speed, to see
> it performed, and to play along. It doesn't work well when trying to
> play to a video example, such as playing an open G chord, and the
> sound from the video does not match the sound from your own guitar.
>
> What I can do now on websites that use VIDEO, such as YouTube, is
> limited to using the web inspector (console). I click "inspect
> element" to find the VIDEO element, then change its DOM properties for
> playbackRate = .95 and mozPreservesPitch = false. That'll get
> everything down a half-step (although at 95% speed).
>
> For example, watch Andy James play "Harvester of Sorrow" but hear it
> 1/2 step down:
>
> https://www.youtube.com/watch?v=vYZjC_U1VJg
>
> 1) Right click on the video "Inspect Element"
> 2) Right click on the VIDEO in the HTML tab. Choose "Show DOM Properties"
> 3) set playbackRate: 0.95, mozPreservesPitch: 0
>
> Ed Van Halen, Yngwie Malmsteen, Jimi Hendrix, Zakk Wylde, to name a
> few tune down 1/2 step. Others, Black Sabbath, Racer X, have tuned
> down further.
>
> A=440 (standard tuning) is much more common. When you want to learn
> something played tuned down 1/2 step and 

Re: [whatwg] VIDEO and pitchAdjustment

[Philip Jägenstedt]

On Thu, Nov 12, 2015 at 9:07 AM, Garrett Smith <dhtmlkitc...@gmail.com> wrote:
>
> On 10/19/15, Philip Jägenstedt <phil...@opera.com> wrote:
> > On Tue, Sep 1, 2015 at 11:21 AM, Philip Jägenstedt <phil...@opera.com>
> > wrote:
> >> On Mon, Aug 31, 2015 at 9:48 PM, Domenic Denicola <d...@domenic.me> wrote:
> >>> From: Eric Carlson [mailto:eric.carl...@apple.com]
> >>>
>
> [...]
> > I've filed a spec issue to make it so:
> > https://github.com/whatwg/html/issues/262
> >
> > If there's any implementor interest in pitch control that goes beyond
> > (independently) or that, please file a separate issue.
> >
>
> They won't.
>
> You can hold the standard of "they need to come here and post up
> cogent arguments in favor of feature X", but it ain't gonna happen
> that way.
>
> There just isn't a whole lot of money in music education. How many
> music education companies are W3C members?
>
> Unlike technology companies like Facebook, Google, Nokia, Opera, and
> other companies the post here, small music education operations like
> artistworks, jammit, licklibrary, etc are more about their domain —
> "music" — than they are about technology.
>
> Major music education websites are still using Flash; their developers
> are busy fixing broken links, making the login feature, database, etc
> work, etc. Flash is not nice but they apparently were not funded or
> motivated enough by the existing HTML5 HTMLMediaElement to use it
> instead.
>
> Control over playbackRate has a greater value than pitch control. But
> those sites don't even allow the students to change the playbackRate
> because they're still using Flash.
>
> You won't read posts here about what students have to say about it the
> value of having HTML5 vs Flash, or independent control over pitch and
> playbackRate.

Have you investigated whether you can achieve your use cases using the
Web Audio API? If it isn't possible, is there a small addition to Web
Audio that would solve the problem?

It is unfortunately quite hard to get all browser vendors (or even
one) interested in implementing support for something that is only
expected to benefit a niche use case, but we should strive to make
available the primitives that would it possible to implement yourself.

Philip

Re: [whatwg] VIDEO and pitchAdjustment

[Philip Jägenstedt]

On Tue, Sep 1, 2015 at 11:21 AM, Philip Jägenstedt <phil...@opera.com> wrote:
> On Mon, Aug 31, 2015 at 9:48 PM, Domenic Denicola <d...@domenic.me> wrote:
>> From: Eric Carlson [mailto:eric.carl...@apple.com]
>>
>>>   FWIW, Safari supports negative playback rates on the desktop and on iOS.
>>>
>>> ...
>>>
>>>   The crash Garrett noted in Safari 8 is a bug that “only" happens with MSE
>>> content.
>>
>> That's really helpful, thanks. Combined with Edge's keyframes-only support, 
>> it sounds like we should probably leave the spec as it is.
>>
>> Do you have thoughts on a mozPreservesPitch equivalent?
>
> Should we just standardize HTMLMediaElement.preservesPitch, perhaps?
> Note that WebKit also has webkitPreservesPitch, but I removed it from
> Blink because it didn't actually do anything in Chromium.
>
> In both Gecko and WebKit it defaults to true. Is there anything else
> worth knowing before writing the spec for this?

I've filed a spec issue to make it so:
https://github.com/whatwg/html/issues/262

If there's any implementor interest in pitch control that goes beyond
(independently) or that, please file a separate issue.

Philip

Re: [whatwg] W3C Timed Text Working Group proposal for HTMLCue

[Philip Jägenstedt]

Hi Nigel,

I'm trying to spend less time on WebVTT and related questions these
days, but here's my high-level feedback, inline:

On Thu, Sep 10, 2015 at 11:46 PM, Nigel Megitt  wrote:
> Dear WhatWG, Jer, Eric, Silvia, Philip,
>
> W3C Timed Text Working Group kindly requests you to consider the following
> proposal:
>
> -
> What?
> -
> We propose to specify an implementation of the text track cue API where
> the format of the cue data is HTML. The "cue type" is called in the
> following "HTMLCue".
>
> The default onenter() handler in the HTMLCue interface would attach the
> cue data to the target element; conversely the default onexit() handler
> would clear the target element's inner HTML.
>
> -
> Why?
> -
> Different file formats are used for the distribution of subtitles and
> captions in the HTML ecosystem. Currently only WebVTT has a defined Cue
> concept that is implemented by Web Browsers. It would extend the reach
> of accessible content greatly if the text track API could be used by any
> subtitle format.
>
> Options for a solution:
>
> 1) Mapping of other formats to VTTCues
> Although this may be a short-term option a lossless mapping is often not
> feasible and requires considerable knowledge of the source and the
> destination format. It would also need continuous alignment of each of
> the subtitle formats with WebVTT and vice versa.

I agree, extending WebVTT until it is a superset of all other formats
will not end well. Reaching interoperability on WebVTT in its current
states looks like a formidable challenge already.

> 2) Define a cue type for every subtitle format
> Even if these different cue type specifications would exist it is
> unlikely that browsers will support all different cue "specializations".

Yeah, this is also unlikely to yield positive results, and it support
for new formats would be blocked on all browsers implementing a new
FooCue every time.

> 3) Define a generic cue type
> This cue type should be easy to implement by browsers and it should be
> possible to map the semantic to the scope of different existing formats.
>
> We think an HTML cue as a variant of the third option is the best
> solution.
>
> The strength of a  generic  HTML cue type is that, assuming that the way
> to render these cues is clearly defined somewhere, basically any kind of
> subtitle format that can be translated into HTML could be supported, as
> long as the browser, a client side JS   or a server based solution does
> the translation work somewhere. One way to make use of the doc fragment
> is to place the doc fragment as an overlay over the video.
>
> The HTMLCue could be defined as an HTML extension. The HTML5 Spec itself
> does not need to be changed.
>
> It may be worth noting that under the hood some browsers already
> translate WebVTT to HTML and some client side JS solutions translate
> TTML to HTML.

If you mean VTTCue.getCueAsHTML(), this code path is actually not used
for rendering, it's just a JavaScript API. WebVTT cannot be mapped to
plain HTML as it is because it requires special rendering code,
specifically to handle the line stepping and overlap avoidance. (It is
of course possible to use HTML as part of a WebVTT implementation, but
not only plain HTML and CSS.)

That being said, I think there would be great value in working on
primitives that are more low-level than WebVTT on which both WebVTT
and (say) TTML could be implemented. A cue type that simply appends an
arbitrary DocumentFragment to a container isn't sufficient though,
because you can't implement WebVTT like that.

>From the history of this discussion I know that "just use metadata
VTTCues with script-based rendering" isn't the answer you are looking
for, but it would be helpful to make the reasons explicit. I suspect
the important ones are:
 • Wanting to support unmodified in-band (and out-of-band?) TTML tracks.
 • Not wanting to depend on JavaScript.

Are these accurate, and is there anything else? FWIW, I think both of
these points are problematic, because the first necessarily implies
some TTML support in browsers, and the second is at odds with how new
web platform features are being designed today.

Philip

Re: [whatwg] deprecating

[Philip Jägenstedt]

On Wed, Sep 2, 2015 at 1:00 PM, henry.st...@bblfish.net
 wrote:
>
>> On 1 Sep 2015, at 19:56, Ian Hickson  wrote:
>>
>> As far as I can tell, therefore, things here are working exactly as one
>> should expect.
>
> Indeed: they seem to be working as one would expect where one thinking that 
> forces
> that don't like asymetric key cryptography to be widely deployed were trying 
> to
> remove that capability as far as possible. The manner of doing this - by 
> secret
> evidence, and pointers to closed non deployed standards - seems to be very 
> much
> the way of doing of organisations that like to keep things secret and closed.

This is borderline conspiratorial and is really not helpful. The first
message in the blink-dev thread [1] nicely summarizes the motivation.
If you distrust that and think that something more sinister is going
on, fine, but that's no way to have a fruitful discussion.

Lots of things have been removed from specs and implementations
following roughly the same "process", which is that some implementor
realizes that they'd like to remove something, check if other
implementors are on board, and then ask to have the spec changed.

[1] 
https://groups.google.com/a/chromium.org/d/msg/blink-dev/pX5NbX0Xack/kmHsyMGJZAMJ

Philip

Re: [whatwg] VIDEO and pitchAdjustment

[Philip Jägenstedt]

On Mon, Aug 31, 2015 at 9:48 PM, Domenic Denicola  wrote:
> From: Eric Carlson [mailto:eric.carl...@apple.com]
>
>>   FWIW, Safari supports negative playback rates on the desktop and on iOS.
>>
>> ...
>>
>>   The crash Garrett noted in Safari 8 is a bug that “only" happens with MSE
>> content.
>
> That's really helpful, thanks. Combined with Edge's keyframes-only support, 
> it sounds like we should probably leave the spec as it is.
>
> Do you have thoughts on a mozPreservesPitch equivalent?

Should we just standardize HTMLMediaElement.preservesPitch, perhaps?
Note that WebKit also has webkitPreservesPitch, but I removed it from
Blink because it didn't actually do anything in Chromium.

In both Gecko and WebKit it defaults to true. Is there anything else
worth knowing before writing the spec for this?

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Thu, Jul 16, 2015 at 4:32 AM, Robert O'Callahan rob...@ocallahan.org wrote:
 On Thu, Jul 16, 2015 at 1:59 AM, Philip Jägenstedt phil...@opera.com
 wrote:

 On Wed, Jul 15, 2015 at 12:43 AM, Robert O'Callahan
 rob...@ocallahan.org wrote:
  On Tue, Jul 14, 2015 at 11:24 PM, Philip Jägenstedt phil...@opera.com
  wrote:
 
  Basically a I trust the browser to decide and promise not to assume
  anything state? The auto state is already suitable named and
  defined for that, so if implementing auto as anything other than
  try to reach HAVE_ENOUGH_DATA is in fact web compatible, a new state
  for that might make sense.
 
  Practically speaking, though, what would be the policy for auto that
  differs between platforms, in the case of a single video element in a
  page?
 
 
  Our current policy, which is basically HAVE_ENOUGH_DATA on desktop and
  HAVE_METADATA on mobile.

 Does this mean that you don't decode the first frame on mobile? Do all
 formats really have reliable data about the video size without
 attempting to decode a frame?


 We do enough work to make sure that the video size is available. If that
 requires decoding the first frame, we do that.

Makes sense :)

 OK, I take it the most important bit for you is in the title of this
 thread: preload=metadata elements don't necessarily fire canplay.

 I'd be quite happy to see preload=metadata spec'd with a requirement
 to never go beyond readyState HAVE_CURRENT_DATA, except when the
 effective preload state is changed by the autoplay attribute, play()
 or setting currentTime. Spec'ing that concept of effective preload
 would also be great, I assume it's been implemented in multiple
 engines.

 I've added some telemetry to Blink to see how common an explicit
 preload=metadata is compared to no explicit preload, to get some
 kind of idea about the risk.

 Next steps?


 When I started this thread I didn't realize there was no specced way to
 preload to a HAVE_ENOUGH_DATA, and I think that's just as important as the
 Web compatibility questions around preload=metadata --- after all, if we
 spec preload=metadata to limit to HAVE_CURRENT_DATA, we should also offer
 an official way to preload beyond HAVE_CURRENT_DATA. So I'm enthusiastic
 about doing all of the changes I suggested.

OK, so perhaps file two bugs for each of those things and continue
discussion there?

Philip

Re: [whatwg] Proposal: Allow disabling of default scroll restoration behavior

[Philip Jägenstedt]

On Mon, Jul 13, 2015 at 5:56 PM, Majid Valipour maji...@google.com wrote:
 On Mon, Jul 13, 2015 at 10:55 AM Philip Jägenstedt phil...@opera.com
 If the StateOptions interface could be implemented with no internal
 reference back to its owning History object it seems pretty harmless,
 a mere holder of values,

 I think a reference to History may be needed because changing scrollState
 needs to update current entry with the new effective value. So it is not a
 simple value holder.

I somewhat suspected that, and it would make StateOptions a tear-off
from History, and tear-offs are a bit annoying IMHO :)

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Wed, Jul 15, 2015 at 12:43 AM, Robert O'Callahan
rob...@ocallahan.org wrote:
 On Tue, Jul 14, 2015 at 11:24 PM, Philip Jägenstedt phil...@opera.com
 wrote:

 Basically a I trust the browser to decide and promise not to assume
 anything state? The auto state is already suitable named and
 defined for that, so if implementing auto as anything other than
 try to reach HAVE_ENOUGH_DATA is in fact web compatible, a new state
 for that might make sense.

 Practically speaking, though, what would be the policy for auto that
 differs between platforms, in the case of a single video element in a
 page?


 Our current policy, which is basically HAVE_ENOUGH_DATA on desktop and
 HAVE_METADATA on mobile.

Does this mean that you don't decode the first frame on mobile? Do all
formats really have reliable data about the video size without
attempting to decode a frame?

 If the different policies could be made explicit, it might be
 nice to have explicit preload states for them and a way for scripts to
 know the effective preload state.


 Maybe that's overengineering.

 I propose we leave 'auto' as-is for now until we have more data, and first
 do the other work I mentioned.

 Then a possible next step would be to determine how much preloading
 preload=auto must do for Web compatibility (i.e., which readyState it must
 reach), and define preload=auto to have that as the goal readyState, *but*
 we could also say that it preloads as much data as possible given
 browser-specific policy (while not actually advancing the readyState beyond
 the goal readyState until the autoplaying flag is false).

OK, I take it the most important bit for you is in the title of this
thread: preload=metadata elements don't necessarily fire canplay.

I'd be quite happy to see preload=metadata spec'd with a requirement
to never go beyond readyState HAVE_CURRENT_DATA, except when the
effective preload state is changed by the autoplay attribute, play()
or setting currentTime. Spec'ing that concept of effective preload
would also be great, I assume it's been implemented in multiple
engines.

I've added some telemetry to Blink to see how common an explicit
preload=metadata is compared to no explicit preload, to get some
kind of idea about the risk.

Next steps?

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Tue, Jul 7, 2015 at 11:56 AM, Robert O'Callahan rob...@ocallahan.org wrote:
 On Tue, Jul 7, 2015 at 9:41 PM, Philip Jägenstedt phil...@opera.com wrote:

 Unsurprisingly, I like this, as it's basically what Presto did. However, I
 think preload=metadata should reach HAVE_CURRENT_DATA, because you do
 want to decode the first frame. The naming mismatch is weird, but oh well.


 OK.


  The question is, would Chrome(ium) be willing to implement that
  reasonably
  quickly? I'm not optimistic given the canplaythrough debacle...

 Do you have links to previous debacles, so that I can see what the
 constraints are, or have been?


 The constraint just seems to be a lack of interest in fixing interop issues
 in this area:
 http://code.google.com/p/chromium/issues/detail?id=73609

Thanks for the link. It looks like some attempt was made to fix it on
the Chromium side (i.e. in the media framework, not HTMLMediaElement)
and that the effort was abandoned. A simple clamping of readyState in
HTMLMediaElement would presumably be quite simple to implement, and
would reveal if it's web compatible or not quite quickly.

Hixie, do you have the bandwidth to figure out a spec for this at this
point? If not, maybe roc and I could work it out in a spec bug and you
can review?

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Tue, Jul 14, 2015 at 3:15 AM, Robert O'Callahan rob...@ocallahan.org wrote:
 Apart from the problems already discussed here, there is currently no
 specced or interoperably implemented way to set a preload value that
 guarantees HAVE_CURRENT_DATA, HAVE_FUTURE_DATA or HAVE_ENOUGH_DATA will be
 reached ... auto may do one of those things, but it doesn't have to, and
 in fact on mobile Firefox we don't guarantee any of those things.

 So I suggest we revamp preload as follows:
 preload=none: the goal readyState is HAVE_NOTHING.
 preload=loadedmetadata: the goal readyState is HAVE_METADATA.
 preload=metadata: the goal readyState is HAVE_METADATA (or some higher
 value if needed for Web compatbility).
 preload=loadeddata: the goal readyState is HAVE_CURRENT_DATA.
 preload=canplay: the goal readyState is HAVE_FUTURE_DATA.
 preload=canplaythrough: the goal readyState is HAVE_ENOUGH_DATA.
 (Thus the expectation for authors is that if you set preload=X then event
 X will eventually fire.)

 I would spec that while the autoplaying flag is true, the readyState of the
 element is limited to less than or equal to the goal readyState if there is
 one. (Note that the autoplaying flag value does not depend on the presence
 of the autoplay attribute.)

Would it solve the same practical problems if auto were redefined to
mean that the goal readyState is HAVE_ENOUGH_DATA? Is there a reason
it doesn't do this in mobile Firefox?

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Tue, Jul 14, 2015 at 1:08 PM, Robert O'Callahan rob...@ocallahan.org wrote:
 On Tue, Jul 14, 2015 at 10:33 PM, Philip Jägenstedt phil...@opera.com
 wrote:

 I see, is this a policy that's applied even with a single video in a
 page, or is it something like a limit on the total number of players
 that can be created, or limits on how much stuff (decoders) they set
 up internally? Presto actually had a limit on the number of players on
 32-bit systems, as one might otherwise run out of virtual memory with
 just a few hundred video elements...


 There are various kinds of resource limits. The problem with those is that
 they're fragile and make behavior unpredictable.

Yeah, this is tricky, obviously no browser could successfully load a
million video elements up to readyState HAVE_ENOUGH_DATA
simultaneously, but making that fail in the same way cross-browser
seems like a big task.

If the behavior could be made interoperable for when resources are not
a problem, that would be a good start at least. The spec can say that
the browser should at least try to ready HAVE_ENOUGH_DATA for
preload=auto, to rule out artificial limits if any browser has them.

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Tue, Jul 14, 2015 at 11:57 AM, Robert O'Callahan
rob...@ocallahan.org wrote:
 On Tue, Jul 14, 2015 at 8:51 PM, Philip Jägenstedt phil...@opera.com
 wrote:

 Would it solve the same practical problems if auto were redefined to
 mean that the goal readyState is HAVE_ENOUGH_DATA?


 Probably.


 Is there a reason it doesn't do this in mobile Firefox?


 We don't want a page with many video preload elements to collapse. Maybe
 we shouldn't worry about that.

I see, is this a policy that's applied even with a single video in a
page, or is it something like a limit on the total number of players
that can be created, or limits on how much stuff (decoders) they set
up internally? Presto actually had a limit on the number of players on
32-bit systems, as one might otherwise run out of virtual memory with
just a few hundred video elements...

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Tue, Jul 14, 2015 at 1:17 PM, Robert O'Callahan rob...@ocallahan.org wrote:
 On Tue, Jul 14, 2015 at 11:13 PM, Philip Jägenstedt phil...@opera.com
 wrote:

 If the behavior could be made interoperable for when resources are not
 a problem, that would be a good start at least. The spec can say that
 the browser should at least try to ready HAVE_ENOUGH_DATA for
 preload=auto, to rule out artificial limits if any browser has them.


 If that behavior is required for Web compatibility, then we should do it.
 However I also think there's value in having a preload value that preloads
 aggressively on some platforms but not on others.

Basically a I trust the browser to decide and promise not to assume
anything state? The auto state is already suitable named and
defined for that, so if implementing auto as anything other than
try to reach HAVE_ENOUGH_DATA is in fact web compatible, a new state
for that might make sense.

Practically speaking, though, what would be the policy for auto that
differs between platforms, in the case of a single video element in a
page? If the different policies could be made explicit, it might be
nice to have explicit preload states for them and a way for scripts to
know the effective preload state.

Philip

Re: [whatwg] Proposal: Allow disabling of default scroll restoration behavior

[Philip Jägenstedt]

On Mon, Jul 13, 2015 at 4:26 PM, Anne van Kesteren ann...@annevk.nl wrote:
 On Mon, Jul 13, 2015 at 3:58 PM, Majid Valipour maji...@chromium.org wrote:
 It is only used as way to group properties (perhaps similar to
 ValidityState?) and to keep History interface clean and stack-like. If that
 is not valuable enough to introduce a new interface then putting these on
 the History interface is fine.

 I personally prefer flatter structures, but you're correct that
 there's precedent for both and given a 1:1 relationship without setter
 it does seem rather harmless.

FWIW, I also think that just history.scrollRestoration would be fine,
better even. Given the generic name options, any future additions to
it would still have the same names as if they're added directly to the
History interface, I'm guessing.

If the StateOptions interface could be implemented with no internal
reference back to its owning History object it seems pretty harmless,
a mere holder of values, but it'll look pretty weird if no additions
are actually made to it in the future.

Philip

Re: [whatwg] DOM Events Proposal: EventListenerOptions 'mayCancel' for improved scroll performance

[Philip Jägenstedt]

On Thu, Jul 9, 2015 at 3:44 PM, Rick Byers rby...@chromium.org wrote:
 On Thu, Jul 9, 2015 at 9:39 AM, Rick Byers rby...@chromium.org wrote:


 On Thu, Jul 9, 2015 at 9:05 AM, James Ross w3c-20040...@james-ross.co.uk
 wrote:

  Date: Thu, 9 Jul 2015 14:42:07 +0200
  From: phil...@opera.com
 
  I think this looks like a very promising approach. Would there be any
  way to feature detect support for EventListenerOptions as the third
  argument? It seems like a problem that addEventListener(type,
  callback, { mayCancel: false }) would be treated as
  addEventListener(type, callback, true) in existing browsers.


 http://rbyers.github.io/EventListenerOptions/EventListenerOptions.html#extensions-to-the-event-interface
 and
 http://rbyers.github.io/EventListenerOptions/EventListenerOptions.html#examples
 example 2 are intended to cover this.


 Right, thanks.  What I've suggested is kind of weird though.  Olli raises
 a good point that we should discuss general dictionary-member feature
 detection patterns in the abstract before committing to something like this.
 I'd love other opinions / ideas on the bug.


 I should perhaps add that I'd hope developers don't really need to use the
 pattern in example #2 explicitly.  Instead I expect they'd mostly rely on a
 simple polyfill (which I'll take an initial crack at writing once the API
 shape settles down a bit).

I see, the answer is getSupportedListenerOptions(). Seems slightly
peculiar, but would work. Should be on EventTarget if anything,
though.

Re: [whatwg] DOM Events Proposal: EventListenerOptions 'mayCancel' for improved scroll performance

[Philip Jägenstedt]

On Thu, Jul 9, 2015 at 6:40 PM, Boris Zbarsky bzbar...@mit.edu wrote:
 On 7/9/15 8:42 AM, Philip Jägenstedt wrote:

 Would there be any way to feature detect support for EventListenerOptions
 as the third
 argument?


 Yes.  You call addEventListener and pass an object that has getters for the
 properties you care about detecting.  If those getters get invoked, the
 browser knows about those dictionary properties.

Ah, that's cool :)

Philip

Re: [whatwg] DOM Events Proposal: EventListenerOptions 'mayCancel' for improved scroll performance

[Philip Jägenstedt]

On Thu, Jul 9, 2015 at 5:30 PM, Rick Byers rby...@chromium.org wrote:
 On Thu, Jul 9, 2015 at 11:22 AM, Anne van Kesteren ann...@annevk.nl wrote:

 On Thu, Jul 9, 2015 at 5:15 PM, Rick Byers rby...@chromium.org wrote:
  I think there's a big opportunity to substantially improve scroll
  performance on the web in the relatively short term by doing something
  incremental.  I.e. I'm pretty sure I can get major scroll-blocking
  libraries
  like Google Analytics to opt into the pattern proposed here in a
  relatively
  short timeframe.  I'm much less sure I could get them to switch to a
  completely new event API in any sort of reasonable timeframe.

 Either way they need to branch their code, no?


 Yeah, but the way it's defined now it's 2 small lines of extra JS for them:
 http://rbyers.github.io/EventListenerOptions/EventListenerOptions.html#example_2

 We've also already got a lot of support for this incremental approach from
 some IE and Mozilla folks (I've reached out to Safari folks privately, but
 nothing public yet).  I probably should have reached out to this list long
 ago - sorry about that (I was mainly focused from the input scenario side,
 for quite awhile it was looking like we'd prefer a CSS API instead of
 something like this).

 That said, I like the look of your proposal (though not sure what exactly
 filter and marker would do).  If you think it's something that's practical
 accomplish in a reasonable time frame then I'm all for it.  Is it fully
 polyfillable? I just don't think this tiny tweak should be the thing that
 necessitates such a major API change.

  What do you think about an incremental path?  I don't see any
  fundamental
  reason that things need to change drastically.

 Overloading a boolean argument with a dictionary seems bad. And if we
 are to have a new API anyway, we might as well pick the better names.


 We could just add a 4th argument for now (a dictionary or even just a
 boolean) if you think that's better to keep the changes absolutely minimal
 until this larger rework can be done.

How about if we just pick nice and short names with a dictionary as
one of the arguments, but initially limit these new APIs to exactly
the existing behavior plus the mayCancel bit? We can tackle other
nice-to-haves later, unless there's some default behavior of the
existing APIs that would be worth changing at the same time? Speaking
of that, having a new function makes it an option to let mayCancel be
false by default, compat-wise at least.

Philip

Re: [whatwg] DOM Events Proposal: EventListenerOptions 'mayCancel' for improved scroll performance

[Philip Jägenstedt]

On Wed, Jul 8, 2015 at 9:12 PM, Rick Byers rby...@chromium.org wrote:
 [Cross-posted to www-...@w3.org - please let me know if there's a better
 way to account for the DOM spec duality]

 In Chromium we've long worked hard at maximizing  scroll performance, with
 scroll-blocking DOM events (wheel and touchstart in particular) being by
 far the biggest source of scroll jank.

 I've been talking about this problem off-and-on for several years with
 various folks including the Pointer Events Working Group, engineers of
 other browser vendors, and engineers working on popular libraries that are
 a source of such scroll-blocking event handlers (eg. Google Ads and
 Google Analytics).

 I've now written a relatively formal (but still sloppy by W3C standards)
 concrete spec for extending the DOM event model
 http://rbyers.github.io/EventListenerOptions/EventListenerOptions.html to
 address this problem and would like to solicit feedback.  It's probably
 most constructive to discuss specific issues on GitHub
 https://github.com/RByers/EventListenerOptions/issues, but I'd appreciate
 any high-level feedback here by e-mail too.  Please feel free to submit
 pull requests for eg. editorial improvements.

 Once there's a bit more consensus on the API shape, I plan to write a
 polyfill and tests and then begin a prototype implementation in Chromium.
 We have some initial evidence to believe that this (in partnership with a
 few key library authors) can make a substantial improvement to the user
 experience on mobile.  I hope to be able to share more concrete data on the
 real-world performance impact, but there's obviously a chicken and egg
 problem here.

I think this looks like a very promising approach. Would there be any
way to feature detect support for EventListenerOptions as the third
argument? It seems like a problem that addEventListener(type,
callback, { mayCancel: false }) would be treated as
addEventListener(type, callback, true) in existing browsers.

Also, I think that this would make good sense as part of DOM itself,
in particular to get rid of the calls to preventDefault will be
ignored patching of DOM algorithms. https://github.com/whatwg/dom is
open for pull requests, or at least I've done some trivial ones.

Anne, what do you think?

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Tue, Jul 7, 2015 at 4:48 AM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Tue, Jul 7, 2015 at 1:44 AM, Philip Jägenstedt phil...@opera.com
 wrote:

 On Mon, Jul 6, 2015 at 2:19 PM, Robert O'Callahan rob...@ocallahan.org
 wrote:

  On Mon, Jul 6, 2015 at 11:17 PM, Philip Jägenstedt phil...@opera.com
  wrote:
 
  On Wed, Jun 10, 2015 at 6:53 AM, Robert O'Callahan 
 rob...@ocallahan.org
  wrote:
   In Gecko, video preload=metadata doesn't fire canplay. This is
   allowed (encouraged, even) by the spec, since we can efficiently
 satisfy
   preload=metadata by stopping decoding after one frame, and if we
 only
   decode one frame then readyState will not advance beyond
  HAVE_CURRENT_DATA.
   However, this may be counterintuitive for Web developers. Also,
 Chrome
   fires canplay in this situation (although that's probably part of
 the
   Chrome fires canplay and canplaythrough willy-nilly bug). Anyone
 else
   have an opinion on this?
 
  Are you having site compat problems related to this?
 
  There's https://bugzilla.mozilla.org/show_bug.cgi?id=1165203 (the same
  bug as in the other thread), thought that involves load() as well. I
 don't
  know of any sites where this causes problems in the absence of load(),
  though I wouldn't be surprised if they exist. I think it would be a good
  idea to align browser behaviors or the spec.

 Yeah, I agree. In Presto I added some explicit limits to readyState based
 on preload so that even for data: URLs or fully cached resources, it
 wouldn't progress beyond HAVE_CURRENT_DATA. (However, buffered wasn't
 censored, so it could look a bit strange if inspected closely.) Is this
 how
 it works in Gecko too, or is it actually the media framework that makes
 the
 guarantee without any clamping on top of it?


 We don't do any clamping of the state, so given preload=metadata you may
 or may not reach HAVE_CURRENT_DATA or HAVE_FUTURE_DATA (or even
 HAVE_ENOUGH_DATA).

 My previous attempts to nail down the precise behavior or preload haven't
 gone well, as Hixie considers preload as a hint where anything goes. Even
 so, specifying the normal behavior seems like a good idea.

 In the specific case of preload=metadata, what would you like the spec
 to
 say?


 Clamping the readyState would be a good way to get reliable behavior and
 interop while keeping preload useful. We'd spec some kind of goal
 readyState associated with a resource load, and spec that readyState is
 clamped to the goal readyState. For preload=none that's HAVE_NOTHING, for
 preload=metadata that's HAVE_METADATA, for other preload values it would
 be HAVE_ENOUGH_DATA. Method calls like play() or setting autoplay would
 force it to HAVE_ENOUGH_DATA and it could never decrease.


Unsurprisingly, I like this, as it's basically what Presto did. However, I
think preload=metadata should reach HAVE_CURRENT_DATA, because you do
want to decode the first frame. The naming mismatch is weird, but oh well.


 The question is, would Chrome(ium) be willing to implement that reasonably
 quickly? I'm not optimistic given the canplaythrough debacle...


Do you have links to previous debacles, so that I can see what the
constraints are, or have been? Given that Blink's default preload state is
auto that ought to limit the breakage somewhat, but I wouldn't really be
surprised to find sites that use preload=metadata and yet assume that
canplay will fire, or similar.

Whatever kind of mess we're in, it would be nice to aim for
interoperability.

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Wed, Jun 10, 2015 at 6:53 AM, Robert O'Callahan rob...@ocallahan.org wrote:
 In Gecko, video preload=metadata doesn't fire canplay. This is
 allowed (encouraged, even) by the spec, since we can efficiently satisfy
 preload=metadata by stopping decoding after one frame, and if we only
 decode one frame then readyState will not advance beyond HAVE_CURRENT_DATA.
 However, this may be counterintuitive for Web developers. Also, Chrome
 fires canplay in this situation (although that's probably part of the
 Chrome fires canplay and canplaythrough willy-nilly bug). Anyone else
 have an opinion on this?

Are you having site compat problems related to this? Since Chrome's
default preload 'auto', I wouldn't expect a whole lot of dependency on
the precise amount of buffering for 'metadata', but would quite expect
some sites to assume that by default one will HAVE_FUTURE_DATA so that
the canplay event is fired.

As for preload 'metadata' itself, it looks like in Chrome it's driven
by the read requests that come out of the media framework, not so much
an explicit policy to decode only one frame:
https://code.google.com/p/chromium/codesearch#chromium/src/media/blink/buffered_resource_loader.hl=35

I would be unsurprised if there are other issues hidden here, though,
CC Andrew Scherkus who knows the Chromium-side code of this.

Philip

Re: [whatwg] preload=none and HTMLMediaElement.load()

[Philip Jägenstedt]

On Wed, Jun 17, 2015 at 2:55 AM, Brian Birtles bbirt...@mozilla.com wrote:
 The HTMLMediaElement.load()[1] method currently cancels playback of the
 current resource and runs the resource selection algorithm[2] (and in turn
 the resource fetch algorithm[3]). When preload=none, however, the resource
 fetch algorithm doesn't require the UA to load data or progress to
 HAVE_METADATA--instead it can suspend the load until, for example, the user
 goes to play the video.

 In Firefox we've hit compatibility issues where some sites are calling
 load() on preload=none resources and expecting to get a loadedmetadata
 event.[4]

 Apparently, even with preload=none, Chrome and IE progress to
 HAVE_METADATA when load() is called. Presto, apparently, does not.

 I've just landed a patch in Firefox to make a call to load() treat
 preload=none as preload=metadata to match the behavior of Chrome/IE in
 this case.

 (Note that we need to do this not only when preload=none is explicitly
 specified but also when the preload attribute is missing and the UA's
 missing value default for preload is 'none'--as is the case for us on
 mobile.)

 Is it possible to get this reflected in the spec?

 Best regards,

 Brian

 [1]
 https://html.spec.whatwg.org/multipage/embedded-content.html#dom-media-load
 [2]
 https://html.spec.whatwg.org/multipage/embedded-content.html#concept-media-load-algorithm
 [3]
 https://html.spec.whatwg.org/multipage/embedded-content.html#concept-media-load-resource
 [4] https://bugzilla.mozilla.org/show_bug.cgi?id=1165203

Thanks for raising this issue, Brian!

It turns out it is a very deliberate change in WebKit made in 2011:
https://bugs.webkit.org/show_bug.cgi?id=66414

I presume that was motivated by a similar problem in the wild,
although no reference was made to a real site in the bug report.

It's unfortunate that load() means anything more than to run the media
element load algorithm, but it's not crazy that load() actually load
something, so I agree that we should spec this behavior somehow. Can
you file a spec bug at https://whatwg.org/newbug to track this?

The most straightforward way to specify this is probably if load()
sets an internal flag on the media element, and if the the resource
fetch algorithm simply skips the preload=none steps if it is set.
Another path could be to make more explicit that there's an internal
preload state that is influenced not only by the content attribute,
but also by setting the autoplay attribute, calling play(), setting
currentTime, and now, calling load().

Philip

Re: [whatwg] preload=metadata elements don't necessarily fire canplay

[Philip Jägenstedt]

On Mon, Jul 6, 2015 at 2:19 PM, Robert O'Callahan rob...@ocallahan.org
wrote:

 On Mon, Jul 6, 2015 at 11:17 PM, Philip Jägenstedt phil...@opera.com
 wrote:

 On Wed, Jun 10, 2015 at 6:53 AM, Robert O'Callahan rob...@ocallahan.org
 wrote:
  In Gecko, video preload=metadata doesn't fire canplay. This is
  allowed (encouraged, even) by the spec, since we can efficiently satisfy
  preload=metadata by stopping decoding after one frame, and if we only
  decode one frame then readyState will not advance beyond
 HAVE_CURRENT_DATA.
  However, this may be counterintuitive for Web developers. Also, Chrome
  fires canplay in this situation (although that's probably part of the
  Chrome fires canplay and canplaythrough willy-nilly bug). Anyone else
  have an opinion on this?

 Are you having site compat problems related to this?


 There's https://bugzilla.mozilla.org/show_bug.cgi?id=1165203 (the same
 bug as in the other thread), thought that involves load() as well. I don't
 know of any sites where this causes problems in the absence of load(),
 though I wouldn't be surprised if they exist. I think it would be a good
 idea to align browser behaviors or the spec.


Yeah, I agree. In Presto I added some explicit limits to readyState based
on preload so that even for data: URLs or fully cached resources, it
wouldn't progress beyond HAVE_CURRENT_DATA. (However, buffered wasn't
censored, so it could look a bit strange if inspected closely.) Is this how
it works in Gecko too, or is it actually the media framework that makes the
guarantee without any clamping on top of it?

My previous attempts to nail down the precise behavior or preload haven't
gone well, as Hixie considers preload as a hint where anything goes. Even
so, specifying the normal behavior seems like a good idea.

In the specific case of preload=metadata, what would you like the spec to
say?

Philip

[whatwg] Announcing the Media Session Standard

[Philip Jägenstedt]

Hi all,

https://mediasession.spec.whatwg.org/ has been live since April 30, and
we'd like to let everyone know of its existence. I recommend reading the
introduction and
https://github.com/whatwg/mediasession/blob/master/README.md for an
overview of what we're trying to enable.

We're doing a little bit of prototyping and expect to soon send an Intent
to Implement on blink-dev. Please be aware that the spec doesn't try to
solve all of the use cases at this point and many details are still
undecided, but please do create issues for anything and everything that
seems wrong.

Philip

Re: [whatwg] Effect of image-orientation on naturalWidth/Height

[Philip Jägenstedt]

For video the rotation is applied to videoWidth and videoHeight, at least
in Chromium/Blink. A video with rotation metadata is thus indistinguishable
from one where the frame themselves are rotated.

If there's any hope that doing the same for img could be Web compatible,
and Safari's behavior makes that seem likely, that seems like a pretty good
outcome.

Philip

On Fri, Mar 13, 2015 at 6:14 AM, Seth Fowler s...@mozilla.com wrote:

 The more I think about this, the more I agree with David. It really does
 make more sense to act like the rotation is part of the image format,
 because after all it *is*, at least when from-image is used.

 This approach also gives us a smoother path to eventually respecting EXIF
 orientation by default. If we did that, we’d want naturalWidth and
 naturalHeight to take EXIF orientation into account, so planning for that
 with the behavior of image-orientation makes sense.

 And FWIW, Safari (which respects EXIF orientation in image documents and
 by default on mobile) does appear to take EXIF orientation into account for
 naturalWidth and naturalHeight, so this approach is web compatible.

 Consider this a second vote for “naturalWidth and naturalHeight should
 respect image-orientation”.

 - Seth

  On Mar 10, 2015, at 10:09 AM, L. David Baron dba...@dbaron.org wrote:
 
  On Monday 2015-03-09 16:52 -0700, Tab Atkins Jr. wrote:
  That's a good question.  I suspect that .naturalWidth/Height should
  return the image's dimensions before applying CSS rotations.  This is
  likely to be surprising, but also probably the correct answer for
  separation-of-concerns reasons.
 
  I wonder whether I need to tweak Images, or Hixie needs tweak img.
 Hmm.
 
  I really think that the mechanism for opting in to honoring EXIF
  should make the browser act as though the rotation were in the image
  format.
 
  It's a compatibility hack (because implementations were initially
  shipping without EXIF support, and there may be a dependency on
  that), but once the developer has opted in, everything should really
  act like the rotation is part of the image format.
 
  -David
 
  --
  턞   L. David Baron http://dbaron.org/   턂
  턢   Mozilla  https://www.mozilla.org/   턂
  Before I built a wall I'd ask to know
  What I was walling in or walling out,
  And to whom I was like to give offense.
- Robert Frost, Mending Wall (1914)

Re: [whatwg] How to expose caption tracks without TextTrackCues

[Philip Jägenstedt]

On Sun, Oct 26, 2014 at 8:28 AM, Silvia Pfeiffer
silviapfeiff...@gmail.com wrote:

 On Thu, Oct 23, 2014 at 2:01 AM, Philip Jägenstedt phil...@opera.com wrote:
  On Sun, Oct 12, 2014 at 11:45 AM, Silvia Pfeiffer
  silviapfeiff...@gmail.com wrote:
 
  Hi all,
 
  In the Inband Text Tracks Community Group we've recently had a
  discussion about a proposal by HbbTV. I'd like to bring it up here to
  get some opinions on how to resolve the issue.
 
  (The discussion thread is at
  http://lists.w3.org/Archives/Public/public-inbandtracks/2014Sep/0008.html
  , but let me summarize it here, because it's a bit spread out.)
 
  The proposed use case is as follows:
  * there are MPEG-2 files that have an audio, a video and several caption 
  tracks
  * the caption tracks are not in WebVTT format but in formats that
  existing Digital TV receivers are already capable of decoding and
  displaying (e.g. CEA708, DVB-T, DVB-S, TTML)
  * there is no intention to standardize a TextTrackCue format for those
  other formats (statements are: there are too many formats to deal
  with, a set-top-box won't need access to cues)
 
  The request was to expose such caption tracks as textTracks:
  interface HTMLMediaElement : HTMLElement {
  ...
readonly attribute TextTrackList textTracks;
  ...
  }
 
  Then, the TextTrack interface would list them as a kind=captions,
  but without any cues, since they're not exposed. This then allows
  turning the caption tracks on/off via JavaScript. However, for
  JavaScript it is indistinguishable from a text track that has no
  captions. So the suggestion was to introduce a new kind=UARendered.
 
 
  My suggestion was to instead treat such tracks as burnt-in video
  tracks (by combination with the main video track):
  interface HTMLMediaElement : HTMLElement {
  ...
 
  readonly attribute VideoTrackList videoTracks;
  ...
  }
 
  Using the VideoTrack interface it would list them as a kind=captions
  and would thus also be able to be activated by JavaScript. The
  downside would that if you have N video tracks and m caption tracks in
  the media file, you'd have to expose NxM videoTracks in the interface.
 
 
  So, given this, should we introduce a kind=UARendered or expose such
  tracks a videoTracks or is there another solution that we're
  overlooking?
 
  VideoTrackList can have at most one video track selected at a time, so
  representing this as a VideoTrack would require some additional
  tweaking to the model.

 The captions video track is one that has video and captions rendered
 together, so you only need the one video track active. If you want to
 turn off captions, you merely activate a different video track which
 is one without captions.

 There is no change to the model necessary - in fact, it fits perfectly
 to what the spec is currently describing without any change.

Ah, right! Unless I'm misunderstanding again, your suggestion is to
expose extra video tracks with kind captions or subtitles, requiring
no spec change at all. That sounds good to me.

Philip

Re: [whatwg] How to expose caption tracks without TextTrackCues

[Philip Jägenstedt]

On Wed, Oct 22, 2014 at 5:33 PM, Bob Lund b.l...@cablelabs.com wrote:


 On 10/22/14, 9:01 AM, Philip Jägenstedt phil...@opera.com wrote:

On Sun, Oct 12, 2014 at 11:45 AM, Silvia Pfeiffer
silviapfeiff...@gmail.com wrote:

 Hi all,

 In the Inband Text Tracks Community Group we've recently had a
 discussion about a proposal by HbbTV. I'd like to bring it up here to
 get some opinions on how to resolve the issue.

 (The discussion thread is at

http://lists.w3.org/Archives/Public/public-inbandtracks/2014Sep/0008.html
 , but let me summarize it here, because it's a bit spread out.)

 The proposed use case is as follows:
 * there are MPEG-2 files that have an audio, a video and several
caption tracks
 * the caption tracks are not in WebVTT format but in formats that
 existing Digital TV receivers are already capable of decoding and
 displaying (e.g. CEA708, DVB-T, DVB-S, TTML)
 * there is no intention to standardize a TextTrackCue format for those
 other formats (statements are: there are too many formats to deal
 with, a set-top-box won't need access to cues)

 The request was to expose such caption tracks as textTracks:
 interface HTMLMediaElement : HTMLElement {
 ...
   readonly attribute TextTrackList textTracks;
 ...
 }

 Then, the TextTrack interface would list them as a kind=captions,
 but without any cues, since they're not exposed. This then allows
 turning the caption tracks on/off via JavaScript. However, for
 JavaScript it is indistinguishable from a text track that has no
 captions. So the suggestion was to introduce a new kind=UARendered.


 My suggestion was to instead treat such tracks as burnt-in video
 tracks (by combination with the main video track):
 interface HTMLMediaElement : HTMLElement {
 ...

 readonly attribute VideoTrackList videoTracks;
 ...
 }

 Using the VideoTrack interface it would list them as a kind=captions
 and would thus also be able to be activated by JavaScript. The
 downside would that if you have N video tracks and m caption tracks in
 the media file, you'd have to expose NxM videoTracks in the interface.


 So, given this, should we introduce a kind=UARendered or expose such
 tracks a videoTracks or is there another solution that we're
 overlooking?

VideoTrackList can have at most one video track selected at a time, so
representing this as a VideoTrack would require some additional
tweaking to the model.

A separate text track kind seems better, but wouldn't it still be
useful to distinguish between captions and subtitles even if the
underlying data is unavailable?

 This issue was clarified here [1]. TextTrack.mode would be set
 ³uarendered². TextTrack.kind would still reflect ³captions² or ³subtitles².

 [1]
 http://lists.w3.org/Archives/Public/public-whatwg-archive/2014Oct/0154.html

Oops, I missed that.

I was under the impression that the ability for scripts to detect this
situation was the motivation for a spec change. If there are multiple
tracks most likely all but one will be disabled initially, which
would be indistinguishable from a disabled track with no cues. Since
TextTrack.mode is mutable, even when it is initially uarendered,
scripts would have to remember that before disabling the track, which
seems a bit inconvenient.

P.S. Your mails have an encoding problem resulting in superscript
numbers instead of quotes.

Philip

Re: [whatwg] How to expose caption tracks without TextTrackCues

[Philip Jägenstedt]

On Sun, Oct 12, 2014 at 11:45 AM, Silvia Pfeiffer
silviapfeiff...@gmail.com wrote:

 Hi all,

 In the Inband Text Tracks Community Group we've recently had a
 discussion about a proposal by HbbTV. I'd like to bring it up here to
 get some opinions on how to resolve the issue.

 (The discussion thread is at
 http://lists.w3.org/Archives/Public/public-inbandtracks/2014Sep/0008.html
 , but let me summarize it here, because it's a bit spread out.)

 The proposed use case is as follows:
 * there are MPEG-2 files that have an audio, a video and several caption 
 tracks
 * the caption tracks are not in WebVTT format but in formats that
 existing Digital TV receivers are already capable of decoding and
 displaying (e.g. CEA708, DVB-T, DVB-S, TTML)
 * there is no intention to standardize a TextTrackCue format for those
 other formats (statements are: there are too many formats to deal
 with, a set-top-box won't need access to cues)

 The request was to expose such caption tracks as textTracks:
 interface HTMLMediaElement : HTMLElement {
 ...
   readonly attribute TextTrackList textTracks;
 ...
 }

 Then, the TextTrack interface would list them as a kind=captions,
 but without any cues, since they're not exposed. This then allows
 turning the caption tracks on/off via JavaScript. However, for
 JavaScript it is indistinguishable from a text track that has no
 captions. So the suggestion was to introduce a new kind=UARendered.


 My suggestion was to instead treat such tracks as burnt-in video
 tracks (by combination with the main video track):
 interface HTMLMediaElement : HTMLElement {
 ...

 readonly attribute VideoTrackList videoTracks;
 ...
 }

 Using the VideoTrack interface it would list them as a kind=captions
 and would thus also be able to be activated by JavaScript. The
 downside would that if you have N video tracks and m caption tracks in
 the media file, you'd have to expose NxM videoTracks in the interface.


 So, given this, should we introduce a kind=UARendered or expose such
 tracks a videoTracks or is there another solution that we're
 overlooking?

VideoTrackList can have at most one video track selected at a time, so
representing this as a VideoTrack would require some additional
tweaking to the model.

A separate text track kind seems better, but wouldn't it still be
useful to distinguish between captions and subtitles even if the
underlying data is unavailable?

Philip

Re: [whatwg] Memory management problem of video elements

[Philip Jägenstedt]

On Tue, Aug 19, 2014 at 3:54 PM, duanyao duan...@ustc.edu wrote:
 于 2014年08月19日 20:23, Philip Jägenstedt 写道:

 On Tue, Aug 19, 2014 at 11:56 AM, duanyao duan...@ustc.edu wrote:

 If the media element object keeps track of its current playing url and
 current position (this requires little memory), and the media file is
 seekable, then
 the media is always resumable. UA can drop any other associated memory of
 the media element, and users will not notice any difference except a
 small
 delay
 when they resume playing.

 That small delay is a problem, at least when it comes to audio
 elements used for sound effects. For video elements, there's the
 additional problem that getting back to the same state will require
 decoding video from the previous keyframe, which could take several
 seconds of CPU time.

 Of course, anything is better than crashing, but tearing down a media
 pipeline and recreating it in the exact same state is quite difficult,
 which is probably why nobody has tried it, AFAIK.

 UA can pre-create the media pipeline according to some hints, e.g. the video
 element is becoming visible,
 so that the delay may be minimized.

 There is a load() method on media element, can it be extended to instruct
 the UA to recreate
 the media pipeline? Thus script can reduce the delay if it knows the media
 is about to be played.

load() resets all state and starts resource selection anew, so without
a way of detecting when a media element has destroyed its media
pipeline to save memory, calling load() can in the worst case increase
the time until play.

 Audios usually eat much less memory, so UAs may have a different strategy
 for them.

 Many native media players can save playing position on exit, and resume the
 playing from that position on the next run.
 Most users are satisfied with such feature. Is recovering to exact same
 state important to some web applications?

I don't know what is required for site compat, but ideally destroying
and recreating a pipeline should get you back to the exact same
currentTime and continue playback at the correct video frame and audio
sample. It could be done.

 I'm not familiar with game programing. Are sound effects small audio files
 that are usually
 played as a whole? Then it should be safe to recreate the pipeline.

There's also a trick called audio sprites where you put all sound
effects into a single file with some silence in between and then seek
to the appropriate offset.

Philip

Re: [whatwg] Memory management problem of video elements

[Philip Jägenstedt]

On Wed, Aug 20, 2014 at 12:04 PM, duanyao duan...@ustc.edu wrote:
 于 2014年08月20日 15:52, Philip Jägenstedt 写道:

 On Tue, Aug 19, 2014 at 3:54 PM, duanyao duan...@ustc.edu wrote:

 I'm not familiar with game programing. Are sound effects small audio
 files
 that are usually
 played as a whole? Then it should be safe to recreate the pipeline.

 There's also a trick called audio sprites where you put all sound
 effects into a single file with some silence in between and then seek
 to the appropriate offset.

 I think if UA can get and set currentTime property accurately, it should be
 able to recreate the pipeline
 with the same accuracy. What are the main factors limiting the accuracy?

I don't know, but would guess that not all media frameworks can seek
to an exact audio sample but only to the beginning of a video frame or
an audio frame, in which case currentTime would be slightly off. One
could just lie about currentTime until playback continues, though.

Philip

Re: [whatwg] Memory management problem of video elements

[Philip Jägenstedt]

On Tue, Aug 19, 2014 at 9:12 AM, duanyao duan...@ustc.edu wrote:
 Hi,

 Recently I have investigated memory usage of HTML video element in
 several desktop browsers (firefox and chrome on windows and linux, and
 IE 11), and have found some disappointing results:

 1. A video element in a playable state consumes significant amount of
 memory. For each playing or paused or preload=auto video element, the
 memory usage
 is up to 30~80MB; for those with preload=metadata, memory usage is
 6~13MB; for those with preload=none, memory usage is not notable. Above
 numbers are measured with 720p to 1080p H.264 videos, and videos in
 lower resolutions use less memory.

 2. For a page having multiple video elements, memory usage is scaled up
 linearly. So a page with tens of videos can exhaust the memory space of
 a 32bit browser. In my tests, such a page may crash the browser or
 freeze a low memory system.

 3. Even if a video element is or becomes invisible, either by being out
 of viewport, having display:none style, or being removed from the active
 DOM tree (but not released),
 almost same amount of memory is still occupied.

 4. The methods to reduce memory occupied by video elements requires
 script, and the element must be modified. For example, remove and
 release the element.

 Although this looks like a implementors' problem, not a spec's problem,
 but I think the current spec is encouraging implementors to push the
 responsibility of memory management of media elements to authors, which
 is very bad. See the section 4.8.14.18
 (http://www.whatwg.org/specs/web-apps/current-work/multipage/embedded-content.html#best-practices-for-authors-using-media-elements):

4.8.14.18 Best practices for authors using media elements
it is a good practice to release resources held by media elements when
 they are done playing, either by being very careful about removing all
 references to the element and allowing it to be garbage collected, or,
 even better, by removing the element's src attribute and any source
 element descendants, and invoking the element's load() method.

 Why this is BAD in my opinion?

 1. It requires script. What if the UA doesn't support or disables script
 (email reader, epub reader, etc), or the script is simply failed to
 download? What if users insert many video elements to a page hosted by a
 site that is not aware of this problem (so no video management script
 available)? Users' browsers may be crashed, or systems may be freezed,
 with no obvious reason.

 2. It is hard to make the script correct. Authors can't simply depend on
 done playing, because users may simply pause a video in the middle and
 start playing another one, and then resume the first one. So authors
 have to determine which video is out of viewport, and remove its src,
 and record its currentTime; when it comes back to viewport, set src and
 seek to previous currentTime. This is quite complicated. For WYSIWYG
 html editors based on browsers, this is even more complicated because of
 the interaction with undo manager.

 3. Browsers are at a much better position to make memory management
 correct. Browsers should be able to save most of the memory of an
 invisible video by only keep its state (or with a current frame), and
 limit the total amount of memory used by media elements.

 So I think the spec should remove section 4.8.14.1, and instead stresses
 the the responsibility of UA to memory management of media elements.

What concrete advice should the spec give to UAs on memory management?
If a script creates a thousand media elements and seeks those to a
thousand different offsets, what is a browser to do? It looks like a
game preparing a lot of sound effects with the expectation that they
will be ready to go, so which ones should be thrown out?

A media element in an active document never gets into a state where it
could never start playing again, so I don't know what to do other than
trying to use less memory per media element. Have you filed bugs at
the browsers that crash or freeze the system?

Regardless of what the UA does, section 4.8.14.1 is still good advice
when the script knows that a resource won't be needed but the browser
cannot. Example: a sound effect is played for last time in a game as
the last secret in the level is found.

Philip

Re: [whatwg] Memory management problem of video elements

[Philip Jägenstedt]

On Tue, Aug 19, 2014 at 11:56 AM, duanyao duan...@ustc.edu wrote:
 于 2014年08月19日 16:00, Philip Jägenstedt 写道:

 On Tue, Aug 19, 2014 at 9:12 AM, duanyao duan...@ustc.edu wrote:

 Hi,

 Recently I have investigated memory usage of HTML video element in
 several desktop browsers (firefox and chrome on windows and linux, and
 IE 11), and have found some disappointing results:

 1. A video element in a playable state consumes significant amount of
 memory. For each playing or paused or preload=auto video element, the
 memory usage
 is up to 30~80MB; for those with preload=metadata, memory usage is
 6~13MB; for those with preload=none, memory usage is not notable. Above
 numbers are measured with 720p to 1080p H.264 videos, and videos in
 lower resolutions use less memory.

 2. For a page having multiple video elements, memory usage is scaled up
 linearly. So a page with tens of videos can exhaust the memory space of
 a 32bit browser. In my tests, such a page may crash the browser or
 freeze a low memory system.

 3. Even if a video element is or becomes invisible, either by being out
 of viewport, having display:none style, or being removed from the active
 DOM tree (but not released),
 almost same amount of memory is still occupied.

 4. The methods to reduce memory occupied by video elements requires
 script, and the element must be modified. For example, remove and
 release the element.

 Although this looks like a implementors' problem, not a spec's problem,
 but I think the current spec is encouraging implementors to push the
 responsibility of memory management of media elements to authors, which
 is very bad. See the section 4.8.14.18

 (http://www.whatwg.org/specs/web-apps/current-work/multipage/embedded-content.html#best-practices-for-authors-using-media-elements):

 4.8.14.18 Best practices for authors using media elements
 it is a good practice to release resources held by media elements when

 they are done playing, either by being very careful about removing all
 references to the element and allowing it to be garbage collected, or,
 even better, by removing the element's src attribute and any source
 element descendants, and invoking the element's load() method.

 Why this is BAD in my opinion?

 1. It requires script. What if the UA doesn't support or disables script
 (email reader, epub reader, etc), or the script is simply failed to
 download? What if users insert many video elements to a page hosted by a
 site that is not aware of this problem (so no video management script
 available)? Users' browsers may be crashed, or systems may be freezed,
 with no obvious reason.

 2. It is hard to make the script correct. Authors can't simply depend on
 done playing, because users may simply pause a video in the middle and
 start playing another one, and then resume the first one. So authors
 have to determine which video is out of viewport, and remove its src,
 and record its currentTime; when it comes back to viewport, set src and
 seek to previous currentTime. This is quite complicated. For WYSIWYG
 html editors based on browsers, this is even more complicated because of
 the interaction with undo manager.

 3. Browsers are at a much better position to make memory management
 correct. Browsers should be able to save most of the memory of an
 invisible video by only keep its state (or with a current frame), and
 limit the total amount of memory used by media elements.

 So I think the spec should remove section 4.8.14.1, and instead stresses
 the the responsibility of UA to memory management of media elements.

 What concrete advice should the spec give to UAs on memory management?
 If a script creates a thousand media elements and seeks those to a
 thousand different offsets, what is a browser to do? It looks like a
 game preparing a lot of sound effects with the expectation that they
 will be ready to go, so which ones should be thrown out?


 UA can limit the number of simultaneously playing medias according to
 available memory or user preference,
 and fire error events on media elements if the limit is hit. We may need
 another error code, currently some UAs fire MEDIA_ERR_DECODE,
 which is misleading.

Opera 12.16 using Presto had such a limit to avoid address space
exhaustion on 32-bit machines, limiting the number of concurrent media
pipelines to 200. However, when the limit was reached it just acted as
if the network was stalling while waiting for an existing pipeline to
be destroyed.

It wasn't a great model, but if multiple browsers (want to) impose
limits like this, maybe a way for script to tell the difference would
be useful.

 If the thousand media elements are just sought, not playing, UA can seek
 them one by one, and drop cached frames afterwards, only keep current
 frames;
 if memory is even more limited, the current frames can also be dropped.

 For a html based slideshows or textbooks, it is quite possible to have tens
 of videos in one html file.

 For audio elements, I think

Re: [whatwg] Media sink device selection on audio/video

[Philip Jägenstedt]

On Tue, Jan 14, 2014 at 5:51 AM, Ami Fischman fisch...@chromium.org wrote:
 On Sun, Jan 12, 2014 at 8:47 PM, Philip Jägenstedt phil...@opera.com
 wrote:
 [...]

 Do you mean to make this per-origin as well? (It will require storing
 that information per-origin forever, or until some invisible timeout.)
 That seems about as restrictive as one could make it, but is the API
 going to actually be useful in this state?


 Sure, why not?
 Canonical use-case: webrtc-using video-chat webapp allows the user to select
 between speakers, wired headset, or bluetooth headset for its audio output;
 the webapp already has gUM permission so now it can use getMediaDevices() to
 enumerate output sinks and use the proposal here to route the remote audio
 feed to the desired device.

OK, so a site that has been given access to any device using
getUserMedia can then enumerate all devices using getMediaDevices? I
interpreted devices to which the user has already granted access
through getUserMedia in the most restrictive (per-origin, per-device)
way possible...

Philip

Re: [whatwg] Media sink device selection on audio/video

[Philip Jägenstedt]

Perhaps :) I'm not on that list, but am fine leaving this to people
who have stronger opinions than me on the issue.

On Tue, Jan 14, 2014 at 6:36 AM, Ami Fischman fisch...@chromium.org wrote:
 Hmm; I wasn't thinking in terms of per-device, only per-origin and
 per-browser/machine.
 Seems like a conversation for public-media-capture?


 On Mon, Jan 13, 2014 at 3:30 PM, Philip Jägenstedt phil...@opera.com
 wrote:

 On Tue, Jan 14, 2014 at 5:51 AM, Ami Fischman fisch...@chromium.org
 wrote:
  On Sun, Jan 12, 2014 at 8:47 PM, Philip Jägenstedt phil...@opera.com
  wrote:
  [...]
 
  Do you mean to make this per-origin as well? (It will require storing
  that information per-origin forever, or until some invisible timeout.)
  That seems about as restrictive as one could make it, but is the API
  going to actually be useful in this state?
 
 
  Sure, why not?
  Canonical use-case: webrtc-using video-chat webapp allows the user to
  select
  between speakers, wired headset, or bluetooth headset for its audio
  output;
  the webapp already has gUM permission so now it can use
  getMediaDevices() to
  enumerate output sinks and use the proposal here to route the remote
  audio
  feed to the desired device.

 OK, so a site that has been given access to any device using
 getUserMedia can then enumerate all devices using getMediaDevices? I
 interpreted devices to which the user has already granted access
 through getUserMedia in the most restrictive (per-origin, per-device)
 way possible...

 Philip

Re: [whatwg] Media sink device selection on audio/video

[Philip Jägenstedt]

On Thu, Dec 19, 2013 at 2:45 PM, Ami Fischman fisch...@chromium.org wrote:
 On Wed, Dec 18, 2013 at 8:38 PM, Ian Hickson i...@hixie.ch wrote:

 On Tue, 17 Dec 2013, Ami Fischman wrote:
  Recently https://www.w3.org/Bugs/Public/show_bug.cgi?id=23263
  Navigator acquired the ability to enumerate media output devices (in
  addition to input devices):
 
 http://dev.w3.org/2011/webrtc/editor/getusermedia.html#enumerating-devices
 What's the privacy story for this API?


 I don't follow public-media-capture but the spec above says:

 The method must only return information that the script is authorized to
 access (TODO expand authorized).

 A narrow reading of that authorized would be devices to which the user
 has already granted access through getUserMedia though I don't want to put
 words in that group's mouth.

Do you mean to make this per-origin as well? (It will require storing
that information per-origin forever, or until some invisible timeout.)
That seems about as restrictive as one could make it, but is the API
going to actually be useful in this state?

Philip

[whatwg] Specify Location.ancestorOrigins

[Philip Jägenstedt]

https://bugs.webkit.org/show_bug.cgi?id=83493

It looks like this fell off the radar, because I can't find it in any
spec. The reason I noticed is that it's implemented using
DOMStringList in Blink, an interface which I also can't find a spec
for...

Philip

Re: [whatwg] Specify Location.ancestorOrigins

[Philip Jägenstedt]

Thanks Jonas and Ian, I'll follow those two bugs!

On Thu, Dec 12, 2013 at 11:50 PM, Ian Hickson i...@hixie.ch wrote:
 On Thu, 12 Dec 2013, Philip Jägenstedt wrote:

 https://bugs.webkit.org/show_bug.cgi?id=83493

 It looks like this fell off the radar, because I can't find it in any
 spec. The reason I noticed is that it's implemented using
 DOMStringList in Blink, an interface which I also can't find a spec
 for...

 The feature request bug for this is:

https://www.w3.org/Bugs/Public/show_bug.cgi?id=22699

 --
 Ian Hickson   U+1047E)\._.,--,'``.fL
 http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
 Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

[whatwg] Specify getSVGDocument

[Philip Jägenstedt]

http://software.hixie.ch/utilities/js/live-dom-viewer/?saved=2643

In my testing, getSVGDocument is supported on embed, frame,
iframe and object in Firefox Nightly, IE11 Preview, Safari 7.0 and
Opera 12.16, the only exception being frame.getSVGDocument in Firefox.

I don't know if this API is actually useful, but since it's already
implemented everywhere it should probably be spec'd.

Philip

Re: [whatwg] Specify getSVGDocument

[Philip Jägenstedt]

On Mon, Nov 18, 2013 at 8:02 PM, Boris Zbarsky bzbar...@mit.edu wrote:
 On 11/18/13 1:37 PM, Philip Jägenstedt wrote:

 http://software.hixie.ch/utilities/js/live-dom-viewer/?saved=2643

 In my testing, getSVGDocument is supported on embed, frame,
 iframe and object in Firefox Nightly, IE11 Preview, Safari 7.0 and
 Opera 12.16, the only exception being frame.getSVGDocument in Firefox.

 I don't know if this API is actually useful, but since it's already
 implemented everywhere it should probably be spec'd.


 SVG2 is trying to spec it, afaict, as basically an alias for
 .contentDocument.  It _would_ make more sense to do that in HTML, I agree.

Indeed, since it's actually on the HTML*Element interfaces. SVG2 can
do partial interfaces, of course, but that makes it a bit harder to
actually find.

Philip

Re: [whatwg] Should onfoo event handler properties be on Element or HTMLElement?

[Philip Jägenstedt]

On Wed, Oct 9, 2013 at 12:02 AM, Ian Hickson i...@hixie.ch wrote:
 On Tue, 8 Oct 2013, Simon Pieters wrote:

 I think it would be bad to have an IDL attribute without a working
 content attribute for a given element. That's just confusing.

 Yeah, that's the main reason I wouldn't put this on Element if it was up
 to me. It seems weird to say to everyone around the world that we're
 basically stomping on any content attribute starting with on. It's one
 thing to take id, and even maybe ok to take class (though that starts
 getting a bit iffy, it's easy to come up with examples where the attribute
 class means something different than we do in HTML), but I'm sure
 there's all kinds of vocabularies that have an onclick attribute with
 _very_ different semantics than ours. For example, I'm sure there's many
 XML-based UI languages that have onclick= handlers but I doubt that
 they all use JavaScript and I really doubt that they all have the crazy
 semantics that HTML has:

OK, I hadn't considered that moving this to Element would imply the
content attributes being reflected for all namespaces. Even though
Blink has the IDL attributes on Element, it looks like the reflection
with the content attributes only works for HTMLElement and SVGElement.
In other words, whatever Blink does doesn't look like something
worth standardizing.

Erik, does going with SVGElement implements GlobalEventHandlers
(like Gecko) seems like a way forward for the SVG spec?

Philip

Re: [whatwg] Should onfoo event handler properties be on Element or HTMLElement?

[Philip Jägenstedt]

On Wed, Oct 9, 2013 at 1:29 PM, Erik Dahlstrom e...@opera.com wrote:
 On Wed, 09 Oct 2013 10:53:29 +0200, Philip Jägenstedt phil...@opera.com
 wrote:

 ...

 OK, I hadn't considered that moving this to Element would imply the
 content attributes being reflected for all namespaces. Even though
 Blink has the IDL attributes on Element, it looks like the reflection
 with the content attributes only works for HTMLElement and SVGElement.
 In other words, whatever Blink does doesn't look like something
 worth standardizing.

 Erik, does going with SVGElement implements GlobalEventHandlers
 (like Gecko) seems like a way forward for the SVG spec?

 Philip


 Given the reasons brought up in this thread I should think that the WG would
 agree that SVGElement implements GlobalEventHandlers is an acceptable way
 forward.

OK, that's what I'll try to get implemented in Blink. Thanks all!

Philip

Re: [whatwg] Should onfoo event handler properties be on Element or HTMLElement?

[Philip Jägenstedt]

On Tue, Oct 8, 2013 at 5:26 PM, Anne van Kesteren ann...@annevk.nl wrote:
 On Tue, Oct 8, 2013 at 4:26 PM, Erik Dahlstrom e...@opera.com wrote:
 On Tue, 08 Oct 2013 15:49:48 +0200, Philip Jägenstedt phil...@opera.com
 wrote:
 1. Do what Gecko does, even though there's no SVG spec requiring it yet.
 2. Just say Element implements GlobalEventHandlers.

 I would prefer option 2, since we already have a unified Document
 implements GlobalEventHandlers.

 Either option is probably fine from a web compat standpoint, but I'd prefer
 option 2 too.

 Have we had success yet making id= and class= super-global? Not
 sure we should extend the experiment prematurely.

I don't know anything about the move of those attributes, has it been
attempted in spec and implementation and run into trouble?

Philip

Re: [whatwg] Should onfoo event handler properties be on Element or HTMLElement?

[Philip Jägenstedt]

On Tue, Oct 8, 2013 at 8:21 PM, Boris Zbarsky bzbar...@mit.edu wrote:
 On 10/8/13 9:49 AM, Philip Jägenstedt wrote:

 Gecko has HTMLElement implements GlobalEventHandlers in its IDL,[1]
 but somewhat surprisingly also SVGElement implements
 GlobalEventHandlers.[2]


 Note that in Gecko SVG elements also have event handler content attributes
 and whatnot.

 What's your proposed handling of event handler content attributes?  And note
 that the handling of these is not quite identical for HTML and SVG last I
 checked.  :(

Is there a spec for how the event handler content attributes (with
reflection) work in SVG? In any event I haven't intended any changes
here, whatever is already implemented sounds good. If implementations
disagree, then getting agreement and a spec for it seems orthogonal to
which interface the IDL attributes sit on.

Philip

Re: [whatwg] Should video controls generate click events?

[Philip Jägenstedt]

On Wed, Sep 11, 2013 at 10:48 AM, Simon Pieters sim...@opera.com wrote:
 On Wed, 11 Sep 2013 01:35:00 +0200, Ian Hickson i...@hixie.ch wrote:

 On Wed, 21 Aug 2013, Robert O'Callahan wrote:


 I think you basically have to assume that if you specify controls then
 the controls may accept clicks anywhere in the video element. There's
 nothing in the spec to say that the controls must be restricted to a bar
 of a certain height at the bottom of the element.


 True, but there _is_ something now that says that if the browser considers
 it the user interacting with a control, that there shouldn't be events
 sent to the page. It's either a control (no events), or an activation
 behaviour (click events, can be canceled by preventDefault()).


 Saying that Firefox's click anywhere to play isn't a control but rather
 activation behavior for the element makes sense. If other browsers want to
 implement that behavior, it would be good if the spec called out this
 difference.


 Should we make this an explicit activation behaviour for the video
 element if it has a controls= attribute?


 That might be good so that the behavior is consistent between browsers.
 However, I think it should be conditional on whether the controls are
 visible rather than whether the controls attribute is present.

What does visible mean here, is it to handle cases where the UA
force-enables the controls without adding the controls attribute to
the DOM?

Philip

Re: [whatwg] Should video controls generate click events?

[Philip Jägenstedt]

On Wed, Sep 11, 2013 at 11:18 AM, Simon Pieters sim...@opera.com wrote:
 On Wed, 11 Sep 2013 10:51:44 +0200, Philip Jägenstedt phil...@opera.com
 wrote:

 That might be good so that the behavior is consistent between browsers.
 However, I think it should be conditional on whether the controls are
 visible rather than whether the controls attribute is present.


 What does visible mean here, is it to handle cases where the UA
 force-enables the controls without adding the controls attribute to
 the DOM?


 Yes.

 http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#expose-a-user-interface-to-the-user

Right, connecting it to the expose a user interface to the user
concept seems best.

Philip

Re: [whatwg] Should video controls generate click events?

[Philip Jägenstedt]

On Wed, Aug 21, 2013 at 12:57 AM, Bob Lund b.l...@cablelabs.com wrote:

 What about a Web page that uses JS to control pause/play/etc based on
 external messages, say from a WebSocket? The sender in this case acts as a
 remote control.

That would involve neither video controls nor click/touch events, so
it won't be affected by the spec change.

Philip

Re: [whatwg] Should video controls generate click events?

[Philip Jägenstedt]

On Wed, Aug 21, 2013 at 3:11 AM, Peter Occil pocci...@gmail.com wrote:

 On Tue, Aug 20, 2013 at 3:28 PM, Silvia Pfeiffer
 silviapfeiff...@gmail.com wrote:

 IMHO, the example that Philip provided in http://people.opera.com/~**
 philipj/click.html http://people.opera.com/~philipj/click.html is not a
 realistic example of something a JS dev would do.


 I'm afraid this example doesn't work well in Firefox and Google Chrome.  It
 affects not only the video itself but also the browser-provided controls,
 and in Firefox it seems to interfere with those controls.

Indeed, the demo was precisely to show how the native controls become
unusable when the scripts on the page haven't taken them into
consideration.

Philip

Re: [whatwg] Should video controls generate click events?

[Philip Jägenstedt]

On Tue, Aug 6, 2013 at 11:30 PM, Ian Hickson i...@hixie.ch wrote:
 On Thu, 27 Jun 2013, Philip Jägenstedt wrote:

 In a discussion about a click to play/pause feature for Opera on
 Android, the issue of click event handlers came up.[1] The problem is
 that pages can do things like this:

 v.onclick = function() {
  if (v.paused) {
v.play();
  } else {
v.pause();
  }
  // no preventDefault()
 }

 I created a demo [2] and it is indeed the case that this makes video
 controls unusable in both Presto and Chromium based browsers. Simon
 Pieters has brought this up before, but the spec wasn't changed at that
 point.[3]

 While my demo may be on the hypothetical side, we do want users to be
 able to bring up the native controls via a context menu and be able to
 use them regardless of what the page does in its event handlers. So, I
 request that the spec be explicit that interacting with the video
 controls does not cause the normal script-visible events to be fired.

 [1] https://codereview.chromium.org/17391015
 [2] http://people.opera.com/~philipj/click.html
 [3] http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-June/031916.html
 (search for As with the post Simon cites above)

 I've made the spec say this is a valid (and recommended) implemenation
 strategy.

The change http://html5.org/r/8134 looks good to me, thanks!

I filed a bug for Blink in
https://code.google.com/p/chromium/issues/detail?id=269454

--
Philip Jägenstedt

[whatwg] Should video controls generate click events?

[Philip Jägenstedt]

In a discussion about a click to play/pause feature for Opera on  
Android, the issue of click event handlers came up.[1] The problem is that  
pages can do things like this:


v.onclick = function() {
  if (v.paused) {
v.play();
  } else {
v.pause();
  }
  // no preventDefault()
}

I created a demo [2] and it is indeed the case that this makes video  
controls unusable in both Presto and Chromium based browsers. Simon  
Pieters has brought this up before, but the spec wasn't changed at that  
point.[3]


While my demo may be on the hypothetical side, we do want users to be able  
to bring up the native controls via a context menu and be able to use them  
regardless of what the page does in its event handlers. So, I request that  
the spec be explicit that interacting with the video controls does not  
cause the normal script-visible events to be fired.


[1] https://codereview.chromium.org/17391015
[2] http://people.opera.com/~philipj/click.html
[3]  
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-June/031916.html  
(search for As with the post Simon cites above)


--
Philip Jägenstedt
Opera Software

Re: [whatwg] Feature Request: Media Elements as Targets for Links

[Philip Jägenstedt]

On Tue, 18 Dec 2012 22:25:19 +0100, Nils Dagsson Moskopp  
n...@dieweltistgarnichtso.net wrote:



Nils Dagsson Moskopp n...@dieweltistgarnichtso.net schrieb am Tue, 18
Dec 2012 17:01:40 +0100:


[…]

I therefore prefer a declarative way of linking to timed resources
embedded in their original context. Working on a polyfill right now.


http://daten.dieweltistgarnichtso.net/src/media-fragments-html-polyfill/
I am delighted how useful it is. Is there any implementor interest?


Redefining/extending what the fragment component does for HTML is somewhat  
risky, so it really comes down to what exactly the processing should be.


What should a browser do with a URL ending with #foot=10 if there is an  
element on the page with id=foot=10? What about #foobar if there is an  
element with id=foo? I would be surprised if treating #foo the same as  
#foo were Web compatible...


The part of the spec that would be affected is  
http://www.whatwg.org/specs/web-apps/current-work/multipage/history.html#scroll-to-fragid,  
can you give more details about the changes you'd make?


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] [mimesniff] Review requested on MIME Sniffing Standard

[Philip Jägenstedt]

On Tue, 13 Nov 2012 01:12:53 +0100, Ian Hickson i...@hixie.ch wrote:


On Mon, 12 Nov 2012, Gordon P. Hemsley wrote:


In that case, I need to know which you think you might want for HTML and
which you know you won't. (I don't know of any other specs reliant on
mimesniff.)


We may one day need a set of rules to sniff for a media resource (e.g.
audio/wave vs video/webm), but whether we'll need this is as yet unclear
(some browser vendors want to sniff, others don't).


Opera has implemented the mimesniff spec and is using it for media  
resources, so I'd appreciate if this stayed in the spec even if not all  
browser vendors want to do the same.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] TextTrack and addCue

[Philip Jägenstedt]

On Wed, 26 Sep 2012 21:41:34 +0200, Cyril Concolato  
cyril.concol...@telecom-paristech.fr wrote:



Hi all,

Has it been considered adding another method to add cues to a track?  
Something like addCues(DOMString text) where the text is not only one  
cue (like in the TextTrackCue ctor) but where the text would be multiple  
cues as written in a WebVTT file? My use case is that I'm getting WebVTT  
content from a server using XHR and I'd like to add that easily to the  
track, without having to parse them myself. Ideally, I would like to  
push the WebVTT content to the track a bit like the Media Source  
Extension API. Another option (less ideal) would be to have a  
WebVTTParser object with a parseFromString method returning a  
TextTrackList (a bit like the DOMParser object). Each cue could be added  
one by one.


Comments?

Cyril


You can parse WebVTT using a dummy video+track element and then move the  
cues to the actual track, why is this less than ideal?


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] metadata attribute for media

[Philip Jägenstedt]

On Fri, 21 Sep 2012 01:32:19 +0200, Ralph Giles gi...@mozilla.com wrote:


Back in June, I proposed[1] a new attribute to get metadata tag data
out of media resources.

I've done an experimental implementation of this which is now in the
Firefox Aurora (alpha) channel[2] and Nightly development builds.

The method is media.mozGetMetadata() and it returns a new object each
time, whose properties are key:value pairs representing the metadata
tags from the file. Aurora supports this for Ogg Vorbis (.ogg) streams.
Nightly supports Opus (.opus) files as well.

Right now the method returns whatever set of tags happen to be in the
media stream, at least if they can be interpreted as unicode text. There
are several things I'd like to fix about this. Media formats generally
have rules about how to map various tags to a standard vocabulary. Right
now, web content has to implement those rules, but the user-agent is in
a better position to do so.

Also, a number of Mozilla (C++) developers I've talked to prefer a fixed
schema for metadata, rather than the more dynamic approach.

So, I'd like to do a new method which returns an object with a fixed
schema we can describe in idl, representing standard fields derived from
the dublic core tag set. The raw metadata would be available under the
old method for advanced applications.

Some metadata is also per-track, rather than per-stream, so it makes
sense to have these calls available on track objects as well as media
objects.

The most essential tags are dc:creator and dc:title. That covers almost
all player use cases. For per-track metadata, the language is perhaps
also useful for selection purposes. Are there any other tags folks would
like to see in the initial proposal?


I'm skeptical that all that we want from ID3v2 or common VorbisComment  
tags can be mapped to Dublin Core, it seems better to define mappings  
directly from the underlying format to the WebIDL interface. As an  
example, the MusicBrainz schema has both a track artist (e.g. Queen feat.  
George Michael) and an album artist (e.g. Queen) and two kinds of  
titles (track title and recording title) which doesn't seem to benefit  
from being passed through dc:creator and dc:title.


Given the open-endedness of metadata contained in actual media resources,  
I'm personally a bit skeptical that there's something we could add to the  
Web platform that would be better than just letting authors pass that  
metadata out-of-band using any representation they like, but what use  
cases are you trying to cover here?


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] TextTrack and addCue

[Philip Jägenstedt]

On Thu, 27 Sep 2012 10:35:26 +0200, Cyril Concolato  
cyril.concol...@telecom-paristech.fr wrote:



Hi Philip,

Le 9/27/2012 10:30 AM, Philip Jägenstedt a écrit :
On Wed, 26 Sep 2012 21:41:34 +0200, Cyril Concolato  
cyril.concol...@telecom-paristech.fr wrote:



Hi all,

Has it been considered adding another method to add cues to a track?  
Something like addCues(DOMString text) where the text is not only one  
cue (like in the TextTrackCue ctor) but where the text would be  
multiple cues as written in a WebVTT file? My use case is that I'm  
getting WebVTT content from a server using XHR and I'd like to add  
that easily to the track, without having to parse them myself.  
Ideally, I would like to push the WebVTT content to the track a bit  
like the Media Source Extension API. Another option (less ideal) would  
be to have a WebVTTParser object with a parseFromString method  
returning a TextTrackList (a bit like the DOMParser object). Each cue  
could be added one by one.


Comments?

Cyril


You can parse WebVTT using a dummy video+track element and then move  
the cues to the actual track, why is this less than ideal?
I did not think about that option. It could do the trick. Thank you. By  
less than ideal, I meant that if I can push content to a buffer and let  
the browser parse the text and add the cues to the track, it seems  
easier/more elegant than having either to parse in JS or to create dummy  
elements and move cues around.


Yeah, creating that temporary document is one step more than is needed to  
parse HTML by setting innerHTML. If constructing text tracks  
piece-by-piece in this fashion is something that a lot of people will need  
to do, I think a utility method could be considered. It certainly wouldn't  
be hard to implement, it's just a bigger API that the Web will have to  
support forever :)


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] alt and title attribute exception

[Philip Jägenstedt]

On Tue, 31 Jul 2012 14:03:02 +0200, Steve Faulkner  
faulkner.st...@gmail.com wrote:



title has differing semantics to alt. In situations where alt it not
present on an img but title is, in webkit based browsers the title
attribute content is displayed on mouse hover and is also displayed in
place of the image when images are disabled or not available. This
implementation appears to contradict the must requirement in the spec.

User agents must not present the contents of the alt attribute in the  
same

way as content of the title attribute.



As there is no way visual distinction between title content being  
displayed

and of alt content in this case.


To be very clear, you agree with the spec, think that WebKit is wrong and  
would not offer any applause if Opera were to use the title attribute to  
replace images when images are disabled and there is no alt attribute?



you wrote:

but I'm confused -- is falling back to title a Good Thing that people  
want
browsers to implement, or is it just a quirk that some legacy browser  
had?


Given that there is a semantic distinction in the spec between what alt
content is and what title content is and a swathe of normative
requirements/advice based on this distinction it would appear unwise to
promote the use of title as fallback without providing normative
requirements on provision of a method to distinguish between the two.

*Note:* in terms of the accessible name calculation for an img element,  
if

the image does not have aria-label or an aria-labelledby or an alt
attribute, but does have a title attribute, then the title attribute is
used as the accessible name. From an accessibility API perspective, no
distinction is indicated as to the source of the accessible name (apart
from in the Mac AX API).

The last point is another reason why making the title attribute on images
(without alt) conforming is that the semantics, for all users, are
ambiguous.


Yes, the semantic difference is clear. What I wanted to add to this  
discussion is confirmation that the title attribute is inaccessibly to  
mobile browser users and likely to remain so. I don't know what  
conclusions to draw or what the spec should say, but to me it seems unwise  
to use the title attribute at all...


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] alt and title attribute exception

[Philip Jägenstedt]

On Wed, 01 Aug 2012 14:56:23 +0200, Henri Sivonen hsivo...@iki.fi wrote:

On Tue, Jul 31, 2012 at 12:18 PM, Philip Jägenstedt phil...@opera.com  
wrote:
When this was last discussed in the HTML WG (January 2012) I opened a  
bug

(MOBILE-275) for Opera Mobile to expose the title attribute in our
long-click menu, arguing that one could not enjoy XKCD without it. I  
meant
to report back to the HTML WG but forgot, so here it is. Unfortunately,  
the

bug was rejected... quoting the project management:

Sure it is nice to have, but noone else has it so we will not put our
effort into this


Firefox for Android (at least on the Nightly channel) displays the
content of the title attribute on XKCD comics (up to a length limit
which can often be too limiting) upon tap and hold:
http://hsivonen.iki.fi/screen/xkcd-firefox-for-android.png

Not to suggest that XKCD's title usage is OK but just to correct the
noone else bit.


Thanks for pointing this out, either we did poor research or this is a new  
feature. In any case, I'll forward this information to our internal bug.  
Opera's context menus are a bit smaller than that, so I don't think adding  
a few paragraphs of text at the top of them would work.


it seems unwise to recommend using the title attribute to convey  
important information.


Indeed. In addition to image considerations, I think
http://www.whatwg.org/specs/web-apps/current-work/#footnotes is bad
advice.


Yeah, that looks like a pretty bad idea, even for sighted users on desktop  
browsers, unless you also add span[title] { border-bottom: 1px dotted  
black; } or similar to your CSS to make it more discoverable. Removing  
that advice seems like a good idea.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] alt and title attribute exception

[Philip Jägenstedt]

On Fri, 27 Jul 2012 11:34:03 +0200, Steve Faulkner
faulkner.st...@gmail.com wrote:


Hi all,

The spec currently allows img without alt if the title attribute is  
present


This is problematic for a number of reasons:

1. One of the functions of alt as implemented is that the text is  
displayed

when images are disabled or not available . I ran some tests a while
back[1] and found that while webkit based browsers display title  
attribute

content if images are disabled or not available, IE, Firefox and Opera do
not. I did a quick recheck and focund the implementations have not  
changed

in the 2.5 years since I ran those tests.

2. title attribute content is commonly displayed as a tooltip that  
appears

when a user moves their mouse over an element (in this case an img) It is
long running issue (14 years or so) that tooltips and thus title  
attribute

content is not displayed for keyboard only users. Browsers vendors are
fully aware of the issue, but as yet there have not yet been moves to fix
the issue*


When this was last discussed in the HTML WG (January 2012) I opened a bug  
(MOBILE-275) for Opera Mobile to expose the title attribute in our  
long-click menu, arguing that one could not enjoy XKCD without it. I meant  
to report back to the HTML WG but forgot, so here it is. Unfortunately,  
the bug was rejected... quoting the project management:


Sure it is nice to have, but noone else has it so we will not put our  
effort into this


Also, we had concerns about where this belongs, UI-wise. The context menu  
is a bit too prominent to display it in, in my opinion. I would like to  
have this, as xkcd isn't the only web comic I read that uses it, but I  
can't think of a way of doing it that doesn't add bloat.


AFAICT there's also no way to read the alt attribute on Opera Mobile. I  
don't know what conclusions to draw, but if the situation is the same on  
other mobile browsers and they are also unwilling to change, it seems  
unwise to recommend using the title attribute to convey important  
information. Of course, it would be equally unwise to use any other new or  
existing attribute unless mobile browsers expose them in some way.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] alt and title attribute exception

[Philip Jägenstedt]

On Tue, 31 Jul 2012 11:48:52 +0200, Chaals McCathieNevile w...@chaals.com  
wrote:



On Tue, 31 Jul 2012 11:18:37 +0200, Philip Jägenstedt phil...@opera.com
wrote:


AFAICT there's also no way to read the alt attribute on Opera Mobile.


You mean title, right?

(alt can be read by turning of image loading - although the famous bug  
http://a11ybugs.org/bug-3.php (I know a few versions of it in Opera's  
BTS) doesn't help).


I really did mean the alt attribute, I didn't try turning images off, just  
long-clicking... oops.


I suppose that if mobile browsers fix Bug 3 *and* fall back to the title  
attribute in the absence of an alt attribute then it would be OK to use  
title instead of alt, but I'm confused -- is falling back to title a Good  
Thing that people want browsers to implement, or is it just a quirk that  
some legacy browser had?


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] video element not ready for prime time

[Philip Jägenstedt]

On Thu, 07 Jun 2012 04:06:10 +0200, Kit Grose k...@iqmultimedia.com.au  
wrote:



On 06/06/2012, at 7:44 AM, Ian Hickson wrote:

On Fri, 13 Jan 2012, Kit Grose wrote:


I'd argue that while we did receive in WebM a common codec it does  
not

enjoy the sort of universal adoption required to be able to mandate its
support in the spec, so on that logic, I think establishing a
declarative fallback mechanism is probably required to prevent a
situation where you cannot write a robust HTML5 page with video and
without resorting to JS.


I don't think it's time to give up yet, but maybe I'm overly  
optimistic...



Is there any reason why it wouldn't be prudent to render the content of  
the video element as HTML if the video cannot be played, given that  
fallback content in the video element is already supported for legacy  
browsers in the spec:


Content may be provided inside the video element. User agents should  
not show this content to the user; it is intended for older Web  
browsers which do not support video, so that legacy video plugins can  
be tried, or to show text to the users of these older browsers  
informing them of how to access the video contents.


How are legacy UAs without video support appreciably different from a  
UA with restrictive or limited video support? Surely the author's  
intent in either case is delivering the content in a different way or  
delivering appropriate alternate content.


Even if we eventually get a common codec (which I—perhaps overly  
pessimistically—feel is unlikely), authors are already using the video  
element without supplying whatever that eventual codec will be, and  
users are suffering without reasonable fallbacks.


As it stands, it's almost better (and certainly easier) for authors to  
default to Flash video and use the existing, declarative fallback  
behaviour of the object element to use a video element instead. That  
can't be in the best interest of the open web.


This was discussed in the thread HTML 5 video tag questions in 2009:

http://lists.w3.org/Archives/Public/public-whatwg-archive/2009Jul/thread.html#msg278

The resource selection algorithm never fails - it simply waits for another  
source to be inserted - so the the hard part is when to show the fallback  
content. At the time I was very skeptical of switching back and forth  
between fallback and trying to load a video as more source elements are  
added, and I still am.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] So if media-queries aren't for determining the media to be used what are they for?

[Philip Jägenstedt]

On Wed, 16 May 2012 03:50:21 +0200, Silvia Pfeiffer  
silviapfeiff...@gmail.com wrote:


On Wed, May 16, 2012 at 9:20 AM, Odin Hørthe Omdal odi...@opera.com  
wrote:



I'm not sure. What do you think? As far as I've seen, you're highly
knowledgeable about video. Why do we have mediaqueries on video  
element?

Do we have a use case page?


Hehe, thanks. :-) But media queries were in video before I arrived,
so I missed the whole discussion around it and how it got there. Some
of the browsers that implement support for it should speak up.


We have supported the media attribute since the first stable release with  
video (Opera 10.50) and Git blames me. However, I must say that I just  
added it because it was in the spec and don't remember any discussion  
about it. Some digging shows that it was added to the spec in  
http://html5.org/r/724, but I don't know why.


As far as I can tell, it's mostly useless and I've never heard of it  
actually being used. The correct way to handle different screen sizes and  
bandwidth constraints is adaptive streaming. Simon Pieters has suggested  
simply dropping the attribute and I would not object to that.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] source media attribute behavior, static or dynamic ?

[Philip Jägenstedt]

On Thu, 10 May 2012 23:03:15 +0200, Paul Adenot  
padenot+wha...@mozilla.com wrote:


Currently implementing the media attribute of the source element in  
Gecko, we are unsure about what the specification requires us to do  
there.


Hixie has already replied, I just want to note that this is exactly what  
has been implemented in Opera -- the media attribute is only considered at  
the point in the resource selection algorithm that the spec requires.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] [media] played member expected behavior.

[Philip Jägenstedt]

On Thu, 26 Apr 2012 23:11:37 +0200, Ian Hickson i...@hixie.ch wrote:


On Thu, 26 Apr 2012, Paul Adenot wrote:


The played member [1] description of the media element states :

 The played attribute must return a new static normalized TimeRanges
 object that represents the ranges of the media resource, if any, that
 the user agent has so far rendered, at the time the attribute is
 evaluated.

Currently implementing this member in Gecko, we are wondering the exact
meaning of the 'rendered' term. If one seek in a video to a location
while being in a paused state, the user agent 'renders' the frame at
that location, since it is displayed on the screen. No audio (if any) is
rendered, though.

In that case, should we create an empty range starting and ending at the
time that was seeked to ? That means creating multiple empty ranges if
multiple seeks occur while being paused. Does the 'rendering' term
implies that playback should occur ? This description need clarification
to specify the exact behavior to adopt.

Semantically, the name of the member itself ('played') seem to imply
playback.


I think playback probably is the most useful. The use case here was
updating a playback scrub bar with colour for where the user has already
watched the video, and zero-length entries aren't useful for that.

I've made a note of this e-mail to fix the spec, but in the absence of
further information or opinions from anyone else, I'd go with playback as
you suggest.


We haven't implemented this yet, but I agree that must be played, since an  
empty range would violate the definition  
http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#normalized-timeranges-object.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] Encoding: big5 and big5-hkscs

[Philip Jägenstedt]

On Mon, 09 Apr 2012 03:08:20 +0200, Øistein E. Andersen li...@coq.no  
wrote:



On 8 Apr 2012, at 18:03, Philip Jägenstedt wrote:

On Sat, 07 Apr 2012 16:04:55 +0200, Øistein E. Andersen li...@coq.no  
wrote:



Suggested change:  map C6CD to U+5E7A.


These are the existing mappings:

C6CD =
opera-hk: U+2F33 ⼳
firefox: U+5E7A 幺
chrome: U+F6DD 
firefox-hk: U+5E7A 幺
opera: U+2F33 ⼳
chrome-hk: U+2F33 ⼳
internetexplorer: U+F6DD 
hkscs-2008: U+2F33 ⼳

At least on the Web, this isn't a question of HK vs non-HK mappings.  
Other than Firefox, which (de-facto) specs or implementations use  
U+5E7A?


I have now had a closer look at my notes  
(http://coq.no/character-tables/chinese-traditional/en). My argument  
for U+5E7A goes as follows:


Of the 214 Kangxi radicals, 186 appear (as normal Han character) in CNS  
11643 Planes 1 or 2, whereas 25 appear in Plane 3 and 3 are missing  
altogether.  Big5 only covers Planes 1 and 2, which means that 28 Kangxi  
radicals (which may be rare in running text, but are nevertheless  
important) are missing.  The E-Ten 1 extension encodes 25 of the missing  
radicals in the range C6BF--C6D7.  Unlike CNS 11643 and Unicode, Big5  
does not encode radicals twice (as radicals and normal characters).   
This means that Big5 with the E-Ten 1 extension contains 211 of the 214  
Kangxi radicals, all mapped to normal Han characters, and no codepoints  
mapped to Unicode Kangxi Radicals in the range U+2F00--U+2FD5.


In summary:  although E-Ten 1 was not defined in terms of Unicode, it is  
clear that the 25 radicals were all meant to map to normal Han  
characters, not to the special radical characters found in CNS 11643 and  
Unicode.


Enter HKSCS.  20 of the E-Ten 1 Kangxi radical mappings (along with the  
rest of E-Ten 1 and E-Ten 2, or almost) are adopted, but the remaining 5  
are instead given new codepoints elsewhere.  Whatever the reason be, 4  
of the 5 unused E-Ten positions are simply left undefined in the HKSCS  
standard, which is not much of a problem for a unified HK/non-HK Big5  
encoding.  Unfortunately, the position C6CD was not left undefined, but  
instead mapped to U+2F33 (⼳), the Unicode Kangxi Radical version of  
U+5E7A (幺), thus introducing not only the only Unicode Kangxi Radical  
into the HKSCS standard, but also a Unicode mapping that is incompatible  
with previous Big5 versions.  I wish I knew why.


Possibly, one could argue that U+2F33 normalizes (NFKC) to U+5E7A, but  
it's not the only hanzi in HKSCS-2008 that normalizes into something  
else:


8BC3 = U+2F878  = U+5C6E 屮
8BF8 = U+F907 龜 = U+9F9C 龜
8EFD = U+2F994 璉 = U+82B3 芳
8FA8 = U+2F9B2 零 = U+456B 䕫
8FF0 = U+2F9D4 倫 = U+8CAB 貫
C6CD = U+2F33 ⼳ = U+5E7A 幺
957A = U+2F9BC 寮 = U+8728 蜨
9874 = U+2F825  = U+52C7 勇
9AC8 = U+2F83B  = U+5406 吆
9C52 = U+2F8CD  = U+6649 晉
A047 = U+2F840  = U+54A2 咢
FC48 = U+2F894  = U+5F22 弢
FC77 = U+2F8A6  = U+6148 慈


The other pairs all contain characters that look slightly different,  
whereas U+5E7A and U+2F33 look the same (and, I believe, are supposed to  
look the same), the only difference being that the former is a normal  
Han character whereas the latter carries the additional semantics of  
being a Kangxi radical.


That the characters in the above list look slightly different is really a  
font issue, they are canonically equivalent in Unicode and therefore the  
same, AFAICT.



I'm not sure what the conclusion is...


I am not entirely sure either.  It seems clear that the mapping from  
C6CD to U+2F33 makes no sense for non-HKSCS Big5 (which does not encode  
U+5E7A anywhere else), but it does not seem to make much sense for  
Big5-HKSCS either, which suggests that I might be missing something.


U+2F33 is indeed the only Kangxi Radical (2F00-2FDF) mapped by HKSCS-2008  
and I agree that it's weird. However, unless U+2F33 causes problems on  
real-world pages, I'm not really comfortable with fixing bugs in  
HKSCS-2008, at least not based only on agreement by two Northern Europeans  
like us... If users or implementors from Hong Kong or Taiwan also speak up  
for U+5E7A, then I will not object. I posted  
http://lists.w3.org/Archives/Public/public-html-ig-zh/2012Apr/0001.html  
a few days ago seeking such feedback, but so far no one has commented on  
this specific issue.


On Fri Apr 6 06:42:26 PDT 2012, Philip Jägenstedt philipj at  
opera.com wrote:



Also, a single mapping fails the Big5-contra[di]ction test:

F9FE =
opera-hk: U+FFED ￭
firefox: U+2593 ▓
chrome: U+2593 ▓
firefox-hk: U+2593 ▓
opera: U+2593 ▓
chrome-hk: U+FFED ￭
internetexplorer: U+2593 ▓
hkscs-2008: U+FFED ￭

I'd say that we should go with U+FFED here, since that's what the  
[HKSCS-2008] spec

says and it's visually close anyway.


Given that the goal is to define a unified Big5 (non-HK) and  
Big5-HKSCS encoding and that this seems to be a case of the HK  
standard going against everything and everyone else, perhaps more  
weight should be given to existing specifications

Re: [whatwg] Encoding: big5 and big5-hkscs

[Philip Jägenstedt]

On Sat, 07 Apr 2012 16:04:55 +0200, Øistein E. Andersen li...@coq.no  
wrote:


On Fri Apr 6 14:03:22 PDT 2012, Philip Jägenstedt philipj at opera.com  
wrote:



So, http://people.opera.com/philipj/2012/04/06/big5-foolip.txt is the
mapping I suggest, with 18594 defined mappings and 1188 U+FFFD.


(Second byte 0xA1 appears as 0x7F in the mapping file.)


Oops, I blame Anne's get_bytes() which had an off-by-one error. This is  
the corrected version:


def get_bytes(index):
row = 0xFE-0xA1 + RANGE + 1
lead = (index // row) + 0x81
cell = index % row
trail = (cell + 0xA1 - RANGE) if cell = RANGE else cell + 0x40
return (lead, trail)

I've also updated big5-foolip.txt with this fix.

Your table is very similar to my idea of an ideal Big5-HKSCS decoder,  
which follows Unihan, official Big5-HKSCS mappings and real  
implementations closely.  My version is documented in detail at  
http://coq.no/character-tables/chinese-traditional/en and summarised  
at http://coq.no/character-tables/charset5/en.


Only 2 mappings differ, viz, C6CD (U+2F33 v. U+5E7A) and F9FE (U+FFED v.  
U+2593), which is quite reassuring given that we have worked  
independently and used somewhat different approaches.  The two divergent  
mappings and related issues are discussed below.


Yes, it's very encouraging indeed that we got this close independently and  
with different methods!



[in summary:]
C6CF = U+5EF4 廴 [v.] U+2F35 ⼵
C6D3 = U+65E0 无 [v.] U+2F46 ⽆
C6D5 = U+7676 癶 [v.] U+2F68 ⽨
C6D7 = U+96B6 隶 [v.] U+2FAA ⾪
C6DE = U+3003 〃[v.] U+F6EE [PUA]
C6DF = U+4EDD 仝 [v.]  U+F6EF [PUA]


To this list can be added:

C6CD = U+5E7A v. U+2F33

These seven characters are all part of the E-Ten 1 extension [1] to  
Big5, which is included in all implementations of Big5 (with or without  
HK extensions) that I have come across in browsers .  The official  
Big5-HKSCS table includes the E-Ten 1 extension as well, but the seven  
characters listed above appear elsewhere in the HKSCS extensions and are  
handled specially to avoid encoding the same character more than once.


	[1] http://coq.no/character-tables/eten1.pdf   
http://coq.no/character-tables/eten1.js


What is the source for the mappings in eten1.pdf? I assume that E-Ten was  
originally just some Big5 fonts with no defined mappings to Unicode?


Five are Kangxi radicals and encoded twice in Unicode (once as radicals,  
once as normal Han characters).  The official Big5-HKSCS table maps C6CD  
to the Unicode Kangxi radical U+2F33 and does not list the remaining  
four codepoints at all.  U+2F33 is the only Unicode Kangxi radical  
included in the official Big5-HKSCS table.  As you have noticed, some  
Big5-HKSCS implementations follow this idea for the remaining four as  
well.  It seems better to follow non-HK Big5 implementations here and  
map all five to normal Unicode Han characters.


For the last two characters, U+3003 and U+4EDD, there is only one  
possible Unicode mapping, so duplicates are impossible to avoid (without  
using PUA characters).  The official Big5-HKSCS table does not map C6DE  
and C6DF to anything.


Suggested change:  map C6CD to U+5E7A.


These are the existing mappings:

C6CD =
opera-hk: U+2F33 ⼳
firefox: U+5E7A 幺
chrome: U+F6DD 
firefox-hk: U+5E7A 幺
opera: U+2F33 ⼳
chrome-hk: U+2F33 ⼳
internetexplorer: U+F6DD 
hkscs-2008: U+2F33 ⼳

At least on the Web, this isn't a question of HK vs non-HK mappings. Other  
than Firefox, which (de-facto) specs or implementations use U+5E7A?


Possibly, one could argue that U+2F33 normalizes (NFKC) to U+5E7A, but  
it's not the only hanzi in HKSCS-2008 that normalizes into something else:


8BC3 = U+2F878  = U+5C6E 屮
8BF8 = U+F907 龜 = U+9F9C 龜
8EFD = U+2F994 璉 = U+82B3 芳
8FA8 = U+2F9B2 零 = U+456B 䕫
8FF0 = U+2F9D4 倫 = U+8CAB 貫
C6CD = U+2F33 ⼳ = U+5E7A 幺
957A = U+2F9BC 寮 = U+8728 蜨
9874 = U+2F825  = U+52C7 勇
9AC8 = U+2F83B  = U+5406 吆
9C52 = U+2F8CD  = U+6649 晉
A047 = U+2F840  = U+54A2 咢
FC48 = U+2F894  = U+5F22 弢
FC77 = U+2F8A6  = U+6148 慈

I'm not sure what the conclusion is...

On Fri Apr 6 06:42:26 PDT 2012, Philip Jägenstedt philipj at opera.com  
wrote:



Also, a single mapping fails the Big5-contra[di]ction test:

F9FE =
opera-hk: U+FFED ￭
firefox: U+2593 ▓
chrome: U+2593 ▓
firefox-hk: U+2593 ▓
opera: U+2593 ▓
chrome-hk: U+FFED ￭
internetexplorer: U+2593 ▓
hkscs-2008: U+FFED ￭

I'd say that we should go with U+FFED here, since that's what the  
[HKSCS-2008] spec

says and it's visually close anyway.


Given that the goal is to define a unified Big5 (non-HK) and Big5-HKSCS  
encoding and that this seems to be a case of the HK standard going  
against everything and everyone else, perhaps more weight should be  
given to existing specifications and (non-HK-specific) implementations.


Suggested change:  map F9FE to U+2593


This is the only mapping where IE maps something other than PUA or ?  
that my mapping doesn't agree on, so I don't object to changing it. Still,  
it would be very

Re: [whatwg] Encoding: big5 and big5-hkscs

[Philip Jägenstedt]

 of these are UTF-8 in a JavaScript comment:

/* DJY left 250x250, 已建立 2010/11/18 */



photo.pchome.com.tw/wen657476/045/
0xA1: []
0xA3: []
0xC6-0xC8: [('0xc6', '0xe4'), ('0xc6', '0xe4'), ('0xc7', '0xae')]


\xc6\xe4 is in a script encoded as GBK:

{Icon:'/s12/w/e/wen657476/book45/p121059706328s.jpg', PK:121059706328,  
Title:'DataSet[ 20079-卡其.jpg ]', Desc:'DataSet[ 20079-卡其.jpg ]'}


\xc7\xae is a link to http://photo.pchome.com.tw/wen657476/119307520020  
encoded as GBK:


a href=/wen657476/119307520020k005-罗马钱夹.jpg(1)/a

罗马钱夹 means roman wallet, which is exactly what is being sold.



www.eye.hk/bbs/zboard.php?category=2id=eyeglasses_collestablespage=1page_num=999sn=offss=onsc=onkeyword=select_arrange=headnumdesc=asc
0xA1: []
0xA3: []
0xC6-0xC8: [('0xc7', '0xd7'), ('0xc8', '0xd5'), ('0xc7', '0xd7'),  
('0xc8',

'0xd5'), ('0xc7', '0xd7'), ('0xc8', '0xd5'), ('0xc8', '0xd5'), ('0xc8',
'0xd5'), ('0xc8', '0xd5'), ('0xc8', '0xd5'), ('0xc8', '0xd5'), ('0xc8',
'0xd5'), ('0xc8', '0xd5'), ('0xc8', '0xd5'), ('0xc8', '0xd5'), ('0xc8',
'0xd5'), ('0xc8', '0xd5'), ('0xc8', '0xd5'), ('0xc8', '0xd5'), ('0xc6',
'0xb0'), ('0xc6', '0xe4')]


This page is mixed Big5 and GBK, nothing could save it.



www.izincan.com/board/novelsys.php?arid=65987
0xA1: []
0xA3: []
0xC6-0xC8: [('0xc8', '0xeb')]


Comments in the JavaScript code at the end of the document are in GBK.



tvcity.tvb.com/drama/wasabi_mon_amour/story/002.html
0xA1: [('0x8f', '0x58'), ('0x92', '0xe5'), ('0x8b', '0x95'), ('0x88',
'0xe5'), ('0x8d', '0x80'), ('0x83', '0x3c'), ('0x8b', '0xe8'), ('0x88',
'0x8a'), ('0x8b', '0xe8'), ('0x98', '0xe6'), ('0x92', '0xe5'), ('0x8b',
'0x95'), ('0x81', '0x9e'), ('0x9f', '0xe6'), ('0x81', '0x93'), ('0x8f',
'0xb8'), ('0x87', '0xe6'), ('0x96', '0x99'), ('0x8d', '0xe5'), ('0x8b',
'0x99'), ('0x9d', '0xe6'), ('0x8a', '0xe6'), ('0x88', '0xb2'), ('0x88',
'0xe7'), ('0x9f', '0xa5'), ('0x89', '0x8d'), ('0x9b', '0xe8'), ('0x81',
'0x98'), ('0x91', '0x8a'), ('0x91', '0xe5'), ('0x91', '0x3c'), ('0x8f',
'0xe9')]
0xA3: [('0xa3', '0xe5')]
0xC6-0xC8: [('0xc7', '0x55')]


The top part of the page is in Big5-HKSCS while the site navigation at the  
bottom is in UTF-8.




www.china-holiday.com/big5/big5train/skbzhwsy3.asp?zrxx=ccxssfcc=北京南cx=全部
0xA1: [('0x97', '0xe4'), ('0x83', '0xa8'), ('0x97', '0xe4'), ('0x83',
'0xa8')]
0xA3: []
0xC6-0xC8: []


北京南 and 全部 are encoded as UTF-8 and end up in the title...



www.iis.sinica.edu.tw/page/library/TechReport/tr2002/threebone02.html
0xA1: [('0x87', '0xe8'), ('0x93', '0xe5'), ('0xa0', '0xb1'), ('0x8a',
'0x3c')]
0xA3: []
0xC6-0xC8: []


This page page is mislabeled; it's actually encoded in UTF-8.



bbs.rc-evo.com/viewthread.php?tid=73138page=1authorid=2487
0xA1: []
0xA3: []
0xC6-0xC8: [('0xc6', '0xbc'), ('0xc6', '0xbc'), ('0xc6', '0xbc')]


The page must have changed, I can't find \xc6\xbc



rumotan.com/guan/modules/tinyd3/
0xA1: [('0x90', '0xe8'), ('0x83', '0xe6'), ('0x90', '0xe8'), ('0x8f',
'0xb4'), ('0x9d', '0xe8'), ('0x8f', '0xb4'), ('0x8f', '0xb4'), ('0x9c',
'0x8b'), ('0x95', '0xab'), ('0x90', '0xe8'), ('0x95', '0xab'), ('0x8f',
'0xb4'), ('0x9d', '0xe8'), ('0x8f', '0xb4'), ('0x8f', '0xb4'), ('0x81',
'0xa3'), ('0x9d', '0xe8'), ('0x8f', '0xb4'), ('0x8f', '0xaf'), ('0x9d',
'0xe8'), ('0x8f', '0xb4'), ('0x90', '0xe5'), ('0x9c', '0x8b'), ('0x90',
'0xe8'), ('0x99', '0x2c'), ('0x82', '0xe6'), ('0x8f', '0x90'), ('0x9b',
'0xe8'), ('0x97', '0x9d'), ('0x93', '0xe5'), ('0x90', '0xe8'), ('0x94',
'0xb6'), ('0x8f', '0xe8'), ('0x88', '0x87'), ('0x95', '0xe8'), ('0x82',
'0xe5'), ('0x8f', '0xb4'), ('0x8c', '0x31'), ('0x94', '0x9f'), ('0x97',
'0xe6'), ('0x90', '0xe4'), ('0x8c', '0xe5'), ('0x9b', '0xe5'), ('0x8c',
'0xe8'), ('0x99', '0x9f'), ('0x83', '0xe7'), ('0x95', '0xab'), ('0x8b',
'0xe4'), ('0x82', '0xe6'), ('0x9b', '0xbe'), ('0x9a', '0xe6'), ('0x9c',
'0x8b'), ('0x8e', '0xe8'), ('0x94', '0xe6'), ('0x9c', '0x83'), ('0x86',
'0xe4'), ('0x81', '0xe8'), ('0x81', '0xb7'), ('0x82', '0x22'), ('0x92',
'0xe5'), ('0x82', '0xe8'), ('0x97', '0x9d'), ('0x93', '0xe7'), ('0x90',
'0xe8'), ('0x99', '0x2c')]
0xA3: [('0xa3', '0xe4')]
0xC6-0xC8: []


There's a chunk of UTF-8 in meta, so I looked no further.

--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] Encoding: big5 and big5-hkscs

[Philip Jägenstedt]

On Fri, 06 Apr 2012 12:54:53 +0200, Philip Jägenstedt phil...@opera.com  
wrote:


As a starting point for the spec, I suggest taking the intersection of  
opera-hk, firefox-hk and chrome-hk.


I've written a script in https://gitorious.org/whatwg/big5 to generate  
the mapping that I think makes sense. This is the logic used:


1. If all 3 *-hk mappings agree, use that.
2. If 2 of the *-hk mappings agree on something that is not in the PUA and  
not U+FFFD, use that.
3. If HKSCS-2008 [1] defines a mapping, verify that at least 1 *-hk  
mapping agrees and use that.


Finally, check that the resulting spec does not use the PUA, U+FFFD or  
contradicts a Big5 mapping that everybody agrees on.


This yields a mapping for 18583 of 19782 combinations, which I propose as  
a starting point. To this I would add these 4 mappings from HKSCS-2008,  
which uses multiple code points to represent what was previously a single  
code point in the PUA in some browsers:


8862 = U+00CA,U+0304 Ê̄
8864 = U+00CA,U+030C Ê̌
88A3 = U+00EA,U+0304 ê̄
88A5 = U+00EA,U+030C ê̌

Also, a single mapping fails the Big5-contraction test:

F9FE =
opera-hk: U+FFED ￭
firefox: U+2593 ▓
chrome: U+2593 ▓
firefox-hk: U+2593 ▓
opera: U+2593 ▓
chrome-hk: U+FFED ￭
internetexplorer: U+2593 ▓
hkscs-2008: U+FFED ￭

I'd say that we should go with U+FFED here, since that's what the spec  
says and it's visually close anyway.


These are the ranges that need more investigation.

8140-817F, 81A2-81FE, 8240-827F, 82A2-82FE, 8340-837F, 83A2-83FE,  
8440-847F, 84A2-84FE, 8540-857F, 85A2-85FE, 8640-867F, 86A2-86FE, 8766,  
87E0-87FE, 8862, 8864, 88A3, 88A5, 88AB-88FE, 8942, 8944-8945, 894A-894B,  
89A7-89AA, 89AF, 89B3-89B4, 89C0, 89C4, 8A42, 8A63, 8A75, 8AAB, 8AB1,  
8ABA, 8AC8, 8ACD, 8ADD-8ADE, 8AF5, 8B54, 8BDD, 8BFE, 8CA6, 8CC6-8CC8,  
8CCD, 8CE5, 8D41, 9B61, 9EAC, 9EC4, 9EF4, 9F4E, 9FAD, 9FB1, 9FC0, 9FC8,  
9FDA, 9FE6, 9FEA, 9FEF, A054, A057, A05A, A062, A072, A0A5, A0AD, A0AF,  
A0D3, A0E1, A3E2-A3FE, C6CF, C6D3, C6D5, C6D7, C6DE-C6DF, C8A5-C8CC,  
C8F2-C8F4


They all map to U+FFFD in opera-hk and mostly to PUA points in other  
mappings. A lot of them should probably be U+FFFD, but not all of them. Is  
someone (Simon?) able to do a search for existing content labeled as Big5  
or Big5-HKSCS that uses any of these bytes?


[1]  
http://www.ogcio.gov.hk/en/business/tech_promotion/ccli/download_area/mapping_table_2008.htm


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] Encoding: big5 and big5-hkscs

[Philip Jägenstedt]

On Fri, 06 Apr 2012 15:42:26 +0200, Philip Jägenstedt phil...@opera.com  
wrote:



These are the ranges that need more investigation.


Sorry for the monologue, but investigate I did. These are the interesting  
ones:


C6CF =
opera-hk: U+FFFD �
firefox: U+5EF4 廴
chrome: U+F6DF 
firefox-hk: U+5EF4 廴
opera: U+2F35 ⼵
chrome-hk: U+FFFD �
internetexplorer: U+F6DF 

C6D3 =
opera-hk: U+FFFD �
firefox: U+65E0 无
chrome: U+F6E3 
firefox-hk: U+65E0 无
opera: U+2F46 ⽆
chrome-hk: U+FFFD �
internetexplorer: U+F6E3 

C6D5 =
opera-hk: U+FFFD �
firefox: U+7676 癶
chrome: U+F6E5 
firefox-hk: U+7676 癶
opera: U+2F68 ⽨
chrome-hk: U+FFFD �
internetexplorer: U+F6E5 

C6D7 =
opera-hk: U+FFFD �
firefox: U+96B6 隶
chrome: U+F6E7 
firefox-hk: U+96B6 隶
opera: U+2FAA ⾪
chrome-hk: U+FFFD �
internetexplorer: U+F6E7 

C6DE =
opera-hk: U+FFFD �
firefox: U+3003 〃
chrome: U+F6EE 
firefox-hk: U+3003 〃
opera: U+3003 〃
chrome-hk: U+FFFD �
internetexplorer: U+F6EE 

C6DF =
opera-hk: U+FFFD �
firefox: U+4EDD 仝
chrome: U+F6EF 
firefox-hk: U+4EDD 仝
opera: U+4EDD 仝
chrome-hk: U+FFFD �
internetexplorer: U+F6EF 

The first 4 were Opera using a compatibility code point instead of the  
canonical one. The final 2 are PUA vs proper, at least they render the  
same on my computer. In all 6 cases, firefox and firefox-hk are correct.


I manually added the above 6 mappings and the 4 multi-code point mappings  
from HKSCS-2008 to https://gitorious.org/whatwg/big5.


There are 29 mappings to U+003F (?) in IE that no other browser has. The  
remaining mappings are to PUA or U+FFFD in all browsers, which appears to  
simply be an artifact of the way the mapping is done internally. Mapping  
these to U+FFFD unless anyone finds pages using these byte sequences seems  
the only sane option.


So, http://people.opera.com/philipj/2012/04/06/big5-foolip.txt is the  
mapping I suggest, with 18594 defined mappings and 1188 U+FFFD.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] [media] startOffsetTime, also add startTime?

[Philip Jägenstedt]

On Wed, 04 Apr 2012 20:56:03 +0200, Ian Hickson i...@hixie.ch wrote:


On Wed, 4 Apr 2012, Philip Jägenstedt wrote:

 
  In current Opera and Firefox the timeline is always normalized to
  start at 0, so the time that corresponds to 0 in the original
  timeline would be at a negative currentTime.

 I still don't really understand what you mean by start here.

What I mean with normalized to start at 0 is that when playing the
whole resource


Now I don't understand what you mean by whole resource. :-)

If a TV station starts broadcasting in October, and you join the stream  
in

February, the whole resource is many months long. Where's your zero?

(I guess it's academic if you plan on changing; I'm just curious.)


Oh, whole resource is a bit of a misnomer, I really mean everything  
that we received. We don't make that distinction internally, since over  
HTTP there's no way to tell the difference between a streaming resource  
and a static resource served by a server that doesn't support byte ranges.  
The timeline is shifted by the timestamp of the first audio or video  
sample in the stream, such that currentTime is initially 0. The plan is to  
remove that adjustment.



The spec changes around explicit timelines and static/streaming
resources are a big improvement, thanks! However, it now talks about
both explicit timeline and explicit timings in a way that makes me
uncertain about Ogg. Ogg (at least without skeleton) is just a stream of
timestamped packets, so the timeline simply spans the timestamp of the
first packet to the timestamp of the last packet. WebM is similar in the
streaming case in that timestamps the don't start at 0. Clarification of
whether or not explicit timestamps (Ogg, WebM) implies an explicit
timeline would be welcome. I assume that's the intention, which I also
agree with. (Perhaps saying explicit frame durations instead of
explicit timings would also help.)


I've tried to change this.


Excellent! For anyone still reading, the new text is:

If the media resource somehow specifies an explicit timeline whose origin  
is not negative (i.e. gives each frame a specific time offset and gives  
the first frame a zero or positive offset), then the media timeline should  
be that timeline.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] [media] startOffsetTime, also add startTime?

[Philip Jägenstedt]

On Wed, 04 Apr 2012 02:25:01 +0200, Robert O'Callahan  
rob...@ocallahan.org wrote:


On Tue, Apr 3, 2012 at 9:28 PM, Philip Jägenstedt phil...@opera.com  
wrote:



AFAIK, no browser supports any format for video that does not have
timestamps.



I have patches being reviewed that add support for using MediaStreams as
video sources. These do not have timestamps.


OK, how do they work? Does the current language of the spec solve a  
problem for them?


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] [media] startOffsetTime, also add startTime?

[Philip Jägenstedt]

On Tue, 03 Apr 2012 19:13:12 +0200, Ian Hickson i...@hixie.ch wrote:


On Tue, 3 Apr 2012, Philip Jägenstedt wrote:

 
  It could also do with a good example. The spec says:
 
  If the media resource specifies an explicit start time and date,
  then that time and date should be considered the zero point in the
  media timeline; the timeline offset will be the time and date,
  exposed using the startOffsetTime attribute.
 
  I interpret this as [...] currentTime=-initialTime (unless media
  fragments are used) in the Opera/Firefox definition of currentTime.

 Not sure what this means.

In current Opera and Firefox the timeline is always normalized to start
at 0, so the time that corresponds to 0 in the original timeline would
be at a negative currentTime.


I still don't really understand what you mean by start here.

The idea is that all the times are unsigned, though. So if there's any  
way

to seek to one of these times that are before what you're calling the
start, then yeah, it'll be a mess, because the naive approach of simply
drawing a seek bar from 0 to duration (rather than seekable.start(0) to
duration) will fail.


What I mean with normalized to start at 0 is that when playing the whole  
resource, currentTime will start at 0 and end at duration. (This was not  
really a deliberate choice in Opera, it's just what GStreamer does and I  
never thought about it until this issue came up.)



We will have to change this at the same time as implementing startDate,
since otherwise everything will be a mess...


So long as startDate gives the Date at media timeline's 0 point, it
doesn't really matter exactly what the media timeline is.


Yeah, I guess we could shift startDate by same (undetectable) offset, but  
if we're going to spend efforts shifting things around we might as well  
shift currentTime into alignment with the spec :)



Finally, what about initialTime? [...]
 
  Yes, but why do we need to expose that in the DOM API, what is the
  use case?

 Allows controllers to trivially implement UI to jump back to where the
 stream started, while still showing the full seekable range.

Unless I'm missing something, initialTime is just the initial value of
currentTime, so this is already easy.


Only if the controller is around when the video is created. Don't forget
that one of the design principles of this API is that you should be able
to hook up a controller at any time and have it be able to provide a
fully-fledged controller.


Right, I keep forgetting that...


Also, if media fragments are not used, just setting currentTime=0 will
clamp and seek to the earliest position. However, I've never actually
seen such UI for video, do you have a real world example? It seems to
me like this is a 1% use case that is already easy to solve and that
it's not worth adding an API to go from easy to trivial.


Yeah, that's probably fair. I've removed initialTime.


Thanks!

[snip]

The spec changes around explicit timelines and static/streaming resources  
are a big improvement, thanks! However, it now talks about both explicit  
timeline and explicit timings in a way that makes me uncertain about  
Ogg. Ogg (at least without skeleton) is just a stream of timestamped  
packets, so the timeline simply spans the timestamp of the first packet to  
the timestamp of the last packet. WebM is similar in the streaming case in  
that timestamps the don't start at 0. Clarification of whether or not  
explicit timestamps (Ogg, WebM) implies an explicit timeline would be  
welcome. I assume that's the intention, which I also agree with. (Perhaps  
saying explicit frame durations instead of explicit timings would also  
help.)


Finally, a typo: no explicit timings ofd any kind

--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] [media] startOffsetTime, also add startTime?

[Philip Jägenstedt]

Thanks for the spec changes, startDate is now in a state where I'd be  
happy to implement it! More comments inline:


On Tue, 03 Apr 2012 02:21:43 +0200, Ian Hickson i...@hixie.ch wrote:


On Fri, 9 Mar 2012, Philip Jägenstedt wrote:

On Thu, 08 Mar 2012 19:16:40 +0100, Ian Hickson i...@hixie.ch wrote:
 On Thu, 8 Mar 2012, Philip Jägenstedt wrote:



I really don't know what startOffsetTime is intended for. AFAICT it's a
piece of metadata that you could just as well provide out-of-band, but
for convenience it is exposed via the DOM API. I think it could be handy
to have and would like to implement it, but I don't understand if it's
any different from other metadata like producer or location of a video.


The startOffsetTime is useful for controllers who want to display a
controller with real times, e.g. like TiVo's DVR UI, even when the
underlying media resource has some more or less arbitrary timeline.

e.g. if a TV station starts broadcasting on some Friday at 2pm, that  
would

be its zero time for its timeline, but eight months later, a user joining
that stream doesn't care that the stream is 21 megaseconds old -- they
just want to see 14:20 as the time that corresponds to what was streaming
at 2:20pm.


This makes sense, and the new spec example makes it clearer.


It could also do with a good example. The spec says:

If the media resource specifies an explicit start time and date, then
that time and date should be considered the zero point in the media
timeline; the timeline offset will be the time and date, exposed using
the startOffsetTime attribute.

I interpret this as a date at currentTime=0 in the spec's definition of
currentTime


Right.



and currentTime=-initialTime (unless media fragments are used) in the
Opera/Firefox definition of currentTime.


Not sure what this means.


In current Opera and Firefox the timeline is always normalized to start at  
0, so the time that corresponds to 0 in the original timeline would be at  
a negative currentTime. We will have to change this at the same time as  
implementing startDate, since otherwise everything will be a mess...



  Finally, what about initialTime? It can be set to a non-zero value
  at two points in the spec:
 
  Establish the media timeline for the purposes of the current
  playback position, the earliest possible position, and the initial
  playback position, based on the media data.
 
  If either the media resource or the address of the current media
  resource indicate a particular start time, then set the initial
  playback position to that time and
 
  Does any format expose something like this in-band? I don't know of
  any that do and how to implement this, so the only thing that
  remains is exposing the start time of media fragments. This seems
  rather useless to me, so unless someone has already implemented
  initialTime and explain what it means, I suggest dropping it from
  the spec.

 The address of the current media resource can indicate a particular
 start time if you implement media fragments.

Yes, but why do we need to expose that in the DOM API, what is the use
case?


Allows controllers to trivially implement UI to jump back to where the
stream started, while still showing the full seekable range.


Unless I'm missing something, initialTime is just the initial value of  
currentTime, so this is already easy. Also, if media fragments are not  
used, just setting currentTime=0 will clamp and seek to the earliest  
position. However, I've never actually seen such UI for video, do you  
have a real world example? It seems to me like this is a 1% use case that  
is already easy to solve and that it's not worth adding an API to go from  
easy to trivial.





On Tue, 13 Mar 2012, Philip Jägenstedt wrote:


In the absence of an explicit timeline, the zero time on the media
timeline should correspond to the first frame of the media resource. For
static audio and video files this is generally trivial. For streaming
resources, if the user agent will be able to seek to an earlier point
than the first frame originally provided by the server, then the zero
time should correspond to the earliest seekable time of the media
resource; otherwise, it should correspond to the first frame received
from the server (the point in the media resource at which the user agent
began receiving the stream).

There are multiple problems here, and I think it's responsible for some
of the confusion.

* What is an explicit timeline? For example, does an Ogg stream that
starts with a non-zero timestamp have an explicit timeline?


If there's a timestamp in the resource, then yes, it has an explicit
timeline. That seems self-evident, but if you can think of a way that I
could clarify this, I would be happy to do so.

An example of a video resource without an explicit timeline would be
a multipart/x-replace JPEG stream. There, the time between the frames is
determined by the server's transmission rate, and the data itself has no
timing information

Re: [whatwg] Encoding: big5 and big5-hkscs

[Philip Jägenstedt]

On Wed, 28 Mar 2012 15:36:35 +0200, Anne van Kesteren ann...@opera.com  
wrote:


On Wed, 28 Mar 2012 12:18:41 +0200, Anne van Kesteren ann...@opera.com  
wrote:
I'm not sure what to do with big5 and big5-hkscs. After generating all  
possible byte sequences (lead bytes 0x81 to 0xFE, trail bytes 0x40 to  
0x7E and 0xA1 to 0xFE) and getting the code points for those in various  
browsers there does not seem to be that much interoperability.


http://html5.org/temp/big5.json has all the code points for Internet  
Explorer (internetexplorer, same for big5 and hkscs), Firefox  
(firefox and firefox-hk), Opera (opera and opera-hk), and  
Chrome (chrome and chrome-hk). internetexplorer and chrome are  
quite close, the rest is a little further apart.


Some help as to how best to proceed would be appreciated.


To give some more context, IE treats big5 and big5-hkscs identical. Out  
of the total 19782 code points, 6217 of them map to the Private Use Area  
(PUA) in IE. Chrome does the same for big5, but has a different mapping  
for big5-hkscs. To deal with HKSCS Microsoft brought out this patch:  
http://www.microsoft.com/hk/hkscs/ Basically people living in the Hong  
Kong area are expected to have that installed and therefore the PUA code  
points map to different glyphs. I'm not sure what the situation is like  
on Mac or Linux, but given the market share statistics I saw the market  
is pretty heavenly dominated by Microsoft.


Gecko seems to use a combination of things as documented in  
https://bugzilla.mozilla.org/show_bug.cgi?id=310299 though it is unclear  
how successful that approach is.


There are also various threads online such as  
http://www.google.com/support/forum/p/Chrome/thread?tid=466c210af3fb6d08  
that seem to indicate pages in the Hong Kong area are not using the  
big5-hkscs label and therefore rely on what IE and Chrome do for big5  
and rely on users having the compatible fonts.


Making big5 and big5-hkscs aliases sounds like a good idea, on the  
assumption that big5-hkscs is a pure extension of Big5.


To make this more concrete, here are a few fairly common characters that I  
think are in big5-hkscs but not in big5, their unicode point and byte  
representation in big5-hkscs when converted using Python:


啫 U+556B '\x94\xdc'
嗰 U+55F0 '\x9d\xf5'
嘅 U+5605 '\x9d\xef'

I'm not sure how to use big5.json, so perhaps you can tell me what these  
map to in various browsers? If they're all the same, examples of byte  
sequences that don't would be interesting.


It seems fairly obvious that the most sane solution would be to just use a  
more correct mapping that doesn't involve the PUA, but:


1. What is the compatible subset of all browsers?

2. Does that subset include anything mapping to the PUA?

3. Do Hong Kong or Taiwan sites depend on charCodeAt returning values in  
the PUA?


4. Would hacks be needed on the font-loading side if browsers started  
using a more correct mapping?


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] [media] startOffsetTime, also add startTime?

[Philip Jägenstedt]

On Fri, 09 Mar 2012 15:40:26 +0100, Philip Jägenstedt phil...@opera.com  
wrote:



let me first try to summarize what I think the spec says:

* currentTime need not start at 0, for streams it will typically  
represent for how long the server has been serving a stream.


* duration is not the duration, it is the last timestamp of a resource.

* startOffsetTime is the date at time 0, it's not an offset. It has  
nothing to do with syncing live streams.


* initialTime is the first timestamp of the stream or the start time of  
a media fragment URL, if one is used.


* For chained streams, the 2nd and subsequent clips have their timelines  
normalized and appended to the first clips timeline.


I think this is mostly correct, but Odin pointed out to me this section of  
the spec:


In the absence of an explicit timeline, the zero time on the media  
timeline should correspond to the first frame of the media resource. For  
static audio and video files this is generally trivial. For streaming  
resources, if the user agent will be able to seek to an earlier point than  
the first frame originally provided by the server, then the zero time  
should correspond to the earliest seekable time of the media resource;  
otherwise, it should correspond to the first frame received from the  
server (the point in the media resource at which the user agent began  
receiving the stream).


There are multiple problems here, and I think it's responsible for some of  
the confusion.


* What is an explicit timeline? For example, does an Ogg stream that  
starts with a non-zero timestamp have an explicit timeline?


* Does For streaming resources ... apply only in the absence of an  
explicit timeline, or in general? In other words, what's the scope of In  
the absence of an explicit timeline?


* Why does the spec differentiate between static and streaming resources  
at all? This is not a distinction Opera makes internally, the only mode  
switch we have depends on whether or not a resource is seekable, which  
for HTTP means support for byte-range requests. A static resource can be  
served by a server without support for byte-range requests such that the  
size and duration are known up front, and I certainly wouldn't call that  
streaming.


These definitions can be tweaked/clarified in one of two ways:

1. currentTime always reflects the underlying timestamps, such that a  
resource can start playing at a non-zero offset and seekable.start(0)  
could be non-zero even for a fully seekable resource. This is what the  
spec already says, modulo the streaming resources weirdness.


2. Always normalize the timeline to start at 0 and end at duration.

I think that the BBC blog post is favoring option 2, and while that's  
closest to our implementation I don't feel strongly about it. A benefit of  
option 1 is that currentTime=300 represents the same thing on all clients,  
which should solve the syncing problem without involving any kinds of  
dates.


To sum up, here's the spec changes I still think should be made:

* Make it pedantically clear which of the above two options is correct,  
preferably with a pretty figure of a timeline with all the values clearly  
marked out.


* Rename startOffsetTime to make it clear that it represents the date at  
currentTime=0 and document that it's intended primarily for display. I  
wouldn't object to just dropping it until we expose other kinds of  
metadata like producer/location, but don't care deeply.


* Drop initialTime.

--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] [media] startOffsetTime, also add startTime?

[Philip Jägenstedt]

On Thu, 08 Mar 2012 19:16:40 +0100, Ian Hickson i...@hixie.ch wrote:



(Oops, sorry. Missed these e-mails in my earlier reply.)

On Thu, 8 Mar 2012, Philip Jägenstedt wrote:

On Wed, 07 Mar 2012 11:56:42 +0100, Odin Hørthe Omdal
odi...@opera.com wrote:

 startOffsetTime seem to leave people confused, I often have to explain
 it, and yesterday I read the spec[5] and old emails and got confused
 myself. It hasn't been implemented after almost 2 years.

We (Opera) have wanted to implement this for a long time, but it has
been stalled by the fact that the spec is confusing to the point that we
haven't been able to agree on what it's actually trying to say. Let's
fix that.


I'm happy to make it clearer, but it seems clear to me. What are your
interpretations, so that I can explicitly rule out in the spec the ones
that are not intended?


Excellent, see below.


I agree that it would be useful to expose the constant by which
timestamps are adjusted


Time stamps should not be adjusted.



to guarantee that that currentTime starts at 0 and ends at duration.


That is not what the spec requires.



I think that both a name like startTime (or initialTime) would suggest
that it is the initial value of currentTime, which it is not.


initialTime is the initial value of currentTime.



I suggest the property offsetTime, defined as the stream time in seconds
which currentTime and duration are relative to.


I don't understand what this means. The currentTime is relative to the
media timeline, which is UA-defined and should be based on the media
timeline.


The BBC wrote a blog post [1] about how currentTime varies between Firefox  
and Chrome. Opera does the same as Firefox here. You're right, however,  
that the way media timeline doesn't make any guarantee that currentTime  
starts at 0 or that duration is the duration. I think that the  
implementations predate the media timeline concept, and I agree with the  
BBC blog post that the Opera/Firefox behavior is better. Controls written  
assuming that currentTime goes from 0 to duration won't break and duration  
will actually mean duration.



In practice it would often be understood as the time since the server
began streaming and would be useful to sync live streams with
out-of-band content simply by letting the out-of-band content be
relative to the start of the stream.


That should be zero. I can change that to a must if you like; it's
a should because in some cases (e.g. MJPEG) you don't know what the
media timeline is or how to interpret it, so there's no way to do it.


Which should are you referring to here?


No round-trip with Date representations should be necessary in the
common case.


The startOffsetTime attribute is intended for display, no? Why would you
round-trip with it?



As hinted above, I don't think that startOffsetTime should really be the
first choice for trying to sync live streams.


Indeed.


I really don't know what startOffsetTime is intended for. AFAICT it's a  
piece of metadata that you could just as well provide out-of-band, but for  
convenience it is exposed via the DOM API. I think it could be handy to  
have and would like to implement it, but I don't understand if it's any  
different from other metadata like producer or location of a video.



However, knowing the date of a video is still useful, potentially even
for the streaming case, so we do want to expose the DateUTC field from
WebM. However, startOffsetTime is a bad name for it, since it's not
using the same unit as currentTime. I suggest offsetDate, to go with
offsetTime.


I don't mind renaming startOffsetTime if people think that would help. I
don't think offsetDate is any clearer though.

How about mediaTimelineOriginDate?


Simply originDate or startDate, perhaps? It could also do with a good  
example. The spec says:


If the media resource specifies an explicit start time and date, then  
that time and date should be considered the zero point in the media  
timeline; the timeline offset will be the time and date, exposed using the  
startOffsetTime attribute.


I interpret this as a date at currentTime=0 in the spec's definition of  
currentTime, and currentTime=-initialTime (unless media fragments are  
used) in the Opera/Firefox definition of currentTime. However, there's a  
weird spec example which can lead one into thinking otherwise:


The startOffsetTime attribute would return a Date object with a time  
corresponding to 2010-03-20 23:15:00 UTC. However, if a different user  
agent connected five minutes later, it would (presumably) receive  
fragments covering timestamps 2010-03-20 23:20:00 UTC to 2010-03-21  
00:05:00 UTC and 2010-02-12 14:25:00 UTC to 2010-02-12 14:35:00 UTC, and  
would expose this with a media timeline starting at 0s and extending to  
3,300s (fifty five minutes).


This seems like a rather atypical streaming scenario. It would be a lot  
nicer if the single example of startOffsetTime was for the common scenario  
where each

Re: [whatwg] [media] startOffsetTime, also add startTime?

[Philip Jägenstedt]

, potentially even for the streaming case, so we do  
want to expose the DateUTC field from WebM. However, startOffsetTime is a  
bad name for it, since it's not using the same unit as currentTime. I  
suggest offsetDate, to go with offsetTime.


Finally, what about initialTime? It can be set to a non-zero value at two  
points in the spec:


Establish the media timeline for the purposes of the current playback  
position, the earliest possible position, and the initial playback  
position, based on the media data.


If either the media resource or the address of the current media resource  
indicate a particular start time, then set the initial playback position  
to that time and


Does any format expose something like this in-band? I don't know of any  
that do and how to implement this, so the only thing that remains is  
exposing the start time of media fragments. This seems rather useless to  
me, so unless someone has already implemented initialTime and explain what  
it means, I suggest dropping it from the spec.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] [media] startOffsetTime, also add startTime?

[Philip Jägenstedt]

On Thu, 08 Mar 2012 11:11:06 +0100, Philip Jägenstedt phil...@opera.com  
wrote:


As hinted above, I don't think that startOffsetTime should really be the  
first choice for trying to sync live streams. However, knowing the date  
of a video is still useful, potentially even for the streaming case, so  
we do want to expose the DateUTC field from WebM. However,  
startOffsetTime is a bad name for it, since it's not using the same unit  
as currentTime. I suggest offsetDate, to go with offsetTime.


We discussed this some more internally, specifically if the date is an  
offset at all and if so what it is relative to. In WebM, the DateUTC field  
is defined as Date of the origin of timecode (value 0), i.e. production  
date. [1] Exposing this directly would mean that it is the date at  
currentTime=-offsetTime, an origin time that you can't actually seek to in  
the streaming case.


We discussed the concatenation of two clips and how to represent the date.  
At least chained WebM and chained Ogg should be able to represent this.


To reduce the possibility for confusion about what date is represented and  
to allow the recording date to be preserved in editing, how about exposing  
currentDate instead?


[1] http://www.webmproject.org/code/specs/container/

--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] video element not ready for prime time

[Philip Jägenstedt]

On Fri, 13 Jan 2012 01:53:23 +0100, Aryeh Gregor a...@aryeh.name wrote:


On Thu, Jan 12, 2012 at 12:46 PM, Francis Boumphrey
boumphre...@gmail.com wrote:

e.g. video src='myvideo.mp4' controls

and my user agent does not support the format, all I get (in my  
versions of

Opera and Firefox) is a blank screen. No message (as I would get with
embed) and as far as I can see there is no way for me as an author to
know that the video is not being played so I cannot code a 'write  
around'.


Boris answered the rest of your questions, but here's a way to detect  
errors:


data:text/html,!doctype html
video src='nonexistent.mp4' controls onerror='alert(error)'/video

If you use source, the error event is fired at the source element
instead and doesn't bubble (why not?), so you have to put onerror on
the source element instead.


Or use a capturing event listener.


I don't know why UAs don't provide their own error messages, though.
They provide error icons for failed image loads.


I agree that we should do something, but it would be limited to video  
controls as audio doesn't have a rendering area at all and for video  
without controls it could interfere with the scripted interface. For  
audio controls you could always make the controls look dead of course,  
to make it more clear that there was a problem.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] Microdata feedback

[Philip Jägenstedt]

On Thu, 08 Dec 2011 22:04:41 +0100, Ian Hickson i...@hixie.ch wrote:


I changed the spec as you suggest.


Thanks!

--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] WebVTT feedback (and some other video feedback that snuck in)

[Philip Jägenstedt]

On Fri, 02 Dec 2011 19:07:14 +0100, Glenn Maynard gl...@zewt.org wrote:


On Fri, Dec 2, 2011 at 4:03 AM, David Singer sin...@apple.com wrote:


I guess I may be confused.

I see two ways of rendering VTT (at least):

1) the UA interprets only VTT, and chooses 'suitable' fonts etc.



Without language metadata, this is impossible to do reliably for CJK  
text.


We're going to be doing the same script detection heuristics that we do on  
web pages. Differentiating between simplified Chinese, traditional Chinese  
and Japanese isn't particularly hard.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] WebVTT feedback (and some other video feedback that snuck in)

[Philip Jägenstedt]

On Sat, 03 Dec 2011 17:47:15 +0100, Glenn Maynard gl...@zewt.org wrote:

On Sat, Dec 3, 2011 at 11:38 AM, Philip Jägenstedt  
phil...@opera.comwrote:


We're going to be doing the same script detection heuristics that we do  
on
web pages. Differentiating between simplified Chinese, traditional  
Chinese

and Japanese isn't particularly hard.



That's fine as a fallback, just as with character set guessing, but a
fallback is what it should be.


AFAICT the srclang attribute takes precedence, so it already is the  
fallback.


It's not quite clear to what is a concrete suggestion and what is  
speculation in this discussion, but it seems to me that there's no  
pressing need to have language on any finer granularity than we already  
have (file-wide).


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] WHATWG on Google+

[Philip Jägenstedt]

On Tue, 22 Nov 2011 01:40:10 +0100, Ian Hickson i...@hixie.ch wrote:


On Mon, 21 Nov 2011, Boris Zbarsky wrote:


 Another option is for someone (possibly me) to create a system whereby
 people can subscribe to specific portions of the specification, and
 for a tool to detect when a diff affects that portion and e-mail them.
 I'm not exactly sure how to make that work, but if it's something
 people are interested in, we could figure something out.

This could be interesting if new sections are added rarely enough


I'm poking around at this. I'm not sure sections are the best way to
organise this, because self-contained features are often in various parts
of the spec, or end up sprouting new sections unexpectedly. So I'm  
looking

at annotating the spec source with specific topics, e.g. WebSockets or
Navigation or whatnot.

If people could e-mail me the lists of topics they would be interested in
being e-mailed diffs for, it would give me a good idea of what coarseness
would be helpful here, and thus whether this is a realistic idea.


I'd be interested in following any changes to the video, audio,  
source and track elements, as well as the WebVTT rendering rules. If  
you want to do it manually, categories video and track would  
probably suffice, if we consider WebVTT part of track.


As an aside, I once made an attempt at writing tools for tracking changes  
on a per-section basis in and have dumped that in  
https://gitorious.org/whatwg/diff-sections. It used the outlining  
algorithm to split things into sections and write each section to a  
separate file. It's rather over-engineered in the use of a complicated git  
history and I never got to the part that would actually create a feed of  
relevant changes, but some of the ideas might be worth keeping.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] HTML5 video seeking

[Philip Jägenstedt]

On Wed, 16 Nov 2011 19:36:26 +0100, Frank Galligan fgalli...@google.com  
wrote:



Here is another link where Ian mentions that the seek should be the exact
frame.
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2008-November/017225.html
 This came up from 2 years ago, where I wanted to add an optional  
strict
parameter to the seek call, which would default to false. Exactly the  
same

as what Philip suggested. I'm all for it.

I think Chrome added frame accurate seeking in version 7.


I filed http://www.w3.org/Bugs/Public/show_bug.cgi?id=14851 to see if we  
can make this happen.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] HTML5 video seeking

[Philip Jägenstedt]

On Tue, 15 Nov 2011 00:49:32 +0100, Aaron Colwell acolw...@chromium.org  
wrote:



Hi,

I was looking at the seeking
algorithmhttp://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#seeking
and
had a question about step 10.

10. Wait until the user agent has established whether or not the media
data for the new playback position is available, and, if it is, until it
has decoded enough data to play back that position.


Does this mean the user agent must resume playback at the exact location
specified?
What if the nearest keyframe is several seconds away?
Is the UA expected to decode and toss the frames instead of starting
playback at the nearest keyframe?

On desktop machines I don't think this would be a problem, but on mobile
devices it might be since the hardware may not be able to decode
significantly faster than realtime. What is the intended behavior for  
such

constrained devices?

Aaron


This was last discussed in the huge thread HTML5 video: frame accuracy /  
SMPTE where I made the suggestions [1] to introduce a seek() function to  
allow both frame-accurate and fast seeking. If you like that idea maybe we  
can just spec and implement it?


[1]  
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-January/029806.html


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] DOMTokenList methods would be more useful with a space separated token list

[Philip Jägenstedt]

On Fri, 28 Oct 2011 12:58:01 +0200, David Håsäther hasat...@gmail.com  
wrote:



It would be more useful if the DOMTokenList methods (contains, add,
remove, toggle) would take a space separated list of tokens. This is
the behavior most DOM libraries have when dealing with class names.

So, when setting two classes, instead of

  ele.classList.add(hey);
  ele.classList.add(ho);

you could do

ele.classList.add(hey ho);

Currently, the latter results in an INVALID_CHARACTER_ERR.

The behavior of all methods should be obvious I guess.


+1, but what should toggle return?

toggle(a) returns true if a was added, which is equivalent to  
!contains(a) on the initial string or contains(a) on the final string.  
For toggle(a b), !contains(a b) on the initial string may not be the  
same as contains(a b) on the final string, so which should it be? Put  
simpler, should it return true if any token is added or if all tokens are  
added?


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] SRT research: timestamps

[Philip Jägenstedt]

On Thu, 06 Oct 2011 07:36:00 +0200, Silvia Pfeiffer  
silviapfeiff...@gmail.com wrote:



On Thu, Oct 6, 2011 at 10:51 AM, Ralph Giles gi...@mozilla.com wrote:

On 05/10/11 04:36 PM, Glenn Maynard wrote:

If the files don't work in VTT in any major implementation, then  
probably
not many.  It's the fault of overly-lenient parsers that these things  
happen

in the first place.


A point Philip Jägenstedt has made is that it's sufficiently tedious to
verify correct subtitle playback that authors are unlikely to do so with
any vigilance. Therefore the better trade-off is to make the parser
forgiving, rather than inflict the occasional missing cue on viewers.


That's a slippery slope to go down on. If they cannot see the
consequence, they assume it's legal. It's not like we are totally
screwing up the display - there's only one mis-authored cue missing.
If we accept one type of mis-authoring, where do you stop with
accepting weirdness? How can you make compatible implementations if
everyone decides for themselves what weirdness that is not in the spec
they accept?

I'd rather we have strict parsing and recover from brokenness. It's
the job of validators to identify broken cues. We should teach authors
to use validators before they decide that their files are ok.

As for some of the more dominant mis-authorings: we can accept them as
correct authoring, but then they have to be made part of the
specification and legalized.


To clarify, I have certainly never suggested that implementation do  
anything other than follow the spec to the letter. I *have* suggested that  
the parsing spec be more tolerant of certain errors, but looking at the  
extremely low error rates in our sample I have to conclude that either (1)  
the data is biased or (2) most of these errors are not common enough that  
they need to be handled.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] SRT research: timestamps

[Philip Jägenstedt]

On Thu, 06 Oct 2011 05:46:15 +0200, Glenn Maynard gl...@zewt.org wrote:


On Wed, Oct 5, 2011 at 7:51 PM, Ralph Giles gi...@mozilla.com wrote:


A point Philip Jägenstedt has made is that it's sufficiently tedious to
verify correct subtitle playback that authors are unlikely to do so with
any vigilance. Therefore the better trade-off is to make the parser
forgiving, rather than inflict the occasional missing cue on viewers.



How can you even time subtitles without ever looking at them?


http://www.w3.org/Bugs/Public/show_bug.cgi?id=13943#c15

I just picked a start time by watching the video and made it 2 seconds  
long. If our data is correct, this particular error only happens in  
0.00035% of cues, which suggests that I am an exceptionally untalented  
author of SRT-like cues.


--
Philip Jägenstedt
Core Developer
Opera Software

[whatwg] track / WebVTT issues

[Philip Jägenstedt]

Implementors of track / WebVTT from several browser vendors (Opera,  
Mozilla, Google, Apple) met at the Open Video Conference recently. There  
was a session on video accessibility,[1] a bunch of new bugs were filed  
[2] and there was much rejoicing.


There were a few issues that weren't concrete enough to file bugs on, but  
which I think are still worthwhile discussing further:


== Comments ==

If you look at the source of the spec, you'll find comments as a v2  
feature request:


COMMENT --
this is a comment, bla bla

I do not think this would be very useful. As a one-line comment at the top  
of the file (for authorship, etc) it is rather verbose and ugly, while for  
commenting out cues you would have to comment out each cue individually.  
It also doesn't work inside cues, where something like ! comment  is  
what would be backwards compatible with the current parser. If comments  
are left for v2, the above is what it'll be, because of compatibility  
constraints. If anyone is less than impressed with that, now would be the  
time to suggest an alternative and have it spec'd.


== Scrolling captions ==

The WebVTT layout algorithm tries to not move cues around once they've  
been displayed and to never obscure other cues. This means that for cues  
that overlap in time, the rendering will often be out of order, with the  
earliest cue at the bottom. This is quite contrary to the (mainly US?)  
style of (live) scrolling captions, where cues are always in order and  
scroll to bring new captions into view. (I am not suggesting any specific  
change.)


== Scaling up and down ==

Scaling the font size with the video will not be optimal for either small  
screens (text will be too small) or very large screens (text will be too  
big). Do we change the default rendering in some way, or do we let users  
override the font size? If users can override it, do we care that this may  
break the intended layout of the author?


== Strict vs forgiving parsing ==

The parser is fairly strict in some regards:

 * double id line discards entire cue  
(http://www.w3.org/Bugs/Public/show_bug.cgi?id=13943)

 * must use exactly 2 digits for minutes and seconds
 * minutes and seconds must be 60
 * must use . as the decimal separator
 * must use exactly 3 decimal digits
 * stray  consumes the rest of the cue text

A small percentage of cues (or cue text) will be dropped because of these  
constraints and this is not very likely to be noticed unless the entire  
video+captions are watched. Possible remedies:


 * make the parser more forgiving where it does not conflict with  
extensibility

 * make browsers complain a lot in the error console
 * point and laugh at those who failed to use a (non-existent) validator

== Chapter end time ==

In most systems chapters are really chapter markers, a point in time. A  
chapter implicitly ends when the next begins. For nested chapters this  
isn't so, as the end time is used to determine nesting. Do we expect that  
UIs for chapter navigation make the end time visible in some fashion (e.g.  
highlighting the chapter on the timeline) or that when a chapter it is  
chosen, it will pause at the end time?


== -- next ==

A suggestion that was brought up when discussing chapters. When one simply  
wants the chapter to end when the next starts, it's a bit of a hassle to  
always include the end time. Some additional complexity in the parser  
could allow for this:


00:00.000 -- next
Chapter 1

01:00.000 -- next
Intermezzo

02:00.000 -- next
Last Chapter

Cues would be created with endTime = Infinity, and be modified to the  
startTime of the following cue (in source order) if there is a following  
cue. This would IMO be quite neat, but is the use case strong enough?


[1] http://openvideoconference.org/standards-for-video-accessibility/
[2] http://wiki.whatwg.org/wiki/WebVTT

--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] video preload implementation feedback

[Philip Jägenstedt]

On Thu, 18 Aug 2011 04:50:17 +0200, Chris Pearce cpea...@mozilla.com  
wrote:



I implemented preload support in Firefox.


On 18/08/2011 3:44 a.m., Philip Jägenstedt wrote:
I'd very much like to see feedback from other implementors. Are you  
happy with treating autoplay and preload as just hints as in [4] or  
do you think that we should specify them in greater detail?


I think autoplay should not be treated as a hint, else it's can't be  
relied upon to work, and thus would be completely useless.


Preload is a less critical; if it's not supported users will just end up  
loading more data, which isn't too bad. Clients that care more about  
bandwidth will probably be more likely to support it.


I think that too much variation in how preload is implemented is also  
likely to give compat problems. In  
http://www.w3.org/Bugs/Public/show_bug.cgi?id=12596#c7 I have an example  
of what might break when pages inevitably assume that preload=none  
causes the loadedmetadata event to not be fired.



== Resetting internal preload state ==

Due to the above, it's necessary to reset the internal preload state at  
some point. Otherwise, a script like this wouldn't work:


function setSource(url) {
  var v = document.querySelector('video');
  v.src = url;
  v.preload = none;
  v.onplay = function() { v.preload = autoplay; };


Did you mean |v.preload = auto;| here instead? or |v.autoplay =  
true;|? It seems in this case the onplay handler would only happen if  
the user pressed play on the controls or from script, so the preload  
action be promoted to auto anyway since the resource is playing. I guess  
that's what you're getting at with your point about preload internal  
state promotion?


Oops, it should be v.preload = auto. Indeed this example is more  
complicated than necessary. Promotion of the internal state due to user  
interaction also needs to be reset in the resource selection algorithm, so  
any time that a HTMLMediaElement is reused this is a problem.



}

If a previous resource was playing and preload was set to autoplay by  
script, then we still want preload=none to apply to the new resource.  
To solve this, we are resetting the internal preload state as part of  
the resource selection algorithm, right before step 5 to fire the  
loadstart event. There are various other places one could do this, but  
we think it is important to do it in the async section so that the  
order of setting .src and .preload does not matter.


Currently we update the internal preload action whenever the value of  
the preload state changes, and we check it's not preload=none before  
kicking off a new resource load (resource fetch algorithm step 2) and we  
check it again when we reach loadedmetadata and suspend the load if it's  
preload=metadata.


I think the preload=metadata case is implied by the spec, but having it  
explicitly stated wouldn't hurt.


It sounds like we're applying preload=none at exactly the same point,  
that's good!


If you only allow the internal state to increase, don't you need to reset  
it at some point as well? Or is it impossible in your implementation to  
use preload=auto on one load and preload=metadata on the next due to  
this?



== video preload=none ==

It's not possible to specify exactly how much preload=metadata and  
preload=auto buffers and when, but this is possible for  
preload=none. This is what we do:


After step 1 of the source selection algorithm, if preload==none, set  
networkState to IDLE, fire a suspend event and set the  
delaying-the-load-event flag to false.


This is actually specified now in step 2 of the resource fetch algorithm  
(this must have been added after we implemented @preload).


Right, this is the partially accepted resolution of  
http://www.w3.org/Bugs/Public/show_bug.cgi?id=12596


Doing this at step 1 of the resource selection algorithm means that if  
you're loading from child source elements, and none of them have a  
supported type, then you won't report the error until something calls  
play() or load() explicitly.


Oops, I meant resource fetch algorithm and what I have implemented is  
exactly what the spec says, except that it's unconditional and the  
implementation-defined event is playing or seeking.


Thanks for your feedback!

--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] video preload implementation feedback

[Philip Jägenstedt]

On Thu, 18 Aug 2011 05:36:42 +0200, Bjartur Thorlacius  
svartma...@gmail.com wrote:



Þann mið 17.ágú 2011 15:44, skrifaði Philip Jägenstedt:

I'd very much like to see feedback from other implementors. Are you
happy with treating autoplay and preload as just hints as in [4] or do
you think that we should specify them in greater detail? (This does not
preclude having user preferences to override the standardized defaults.)

If an UA may ignore them, than them being honored can't be relied upon -  
rendering the default configuration a question of implementation (i.e.  
providing sane defaults for the user agent's target user base) but not  
of interoperability.


This is true, but as long as a few big browsers implement e.g.  
preload=none in a somewhat compatible way, it's hard to imagine page  
authors not coming to depend on that behavior so that it becomes required  
for web compat. It would be interesting to know if there are  
counter-examples, any script-visible behavior that is allowed to vary  
greatly between implementations without causing scripts to break.


--
Philip Jägenstedt
Core Developer
Opera Software

[whatwg] video preload implementation feedback

[Philip Jägenstedt]

Greetings,

Opera's implementation [1] of video preload and the testsuite [2][3] has  
been available for about a month now. While implementing this, we found  
and reported a number of issues. [4-10] In general, it is my opinion that  
video preload is under-defined, giving implementors more freedom than is  
necessary or useful. We have done our best to implement and test something  
that we feel makes sense. I'm sure that other browsers will do things  
slightly different and that standardization will be necessary for web  
compat down the line.


I'd very much like to see feedback from other implementors. Are you happy  
with treating autoplay and preload as just hints as in [4] or do you  
think that we should specify them in greater detail? (This does not  
preclude having user preferences to override the standardized defaults.)


Specific issues that don't have open bugs:

== Internal preload state ==

We maintain an internal preload state that is not always in sync with the  
content attribute. This internal state is what actually determines which  
buffering mode is used. For example, if the content attribute is none  
but the user starts playing, then the internal state is promoted to  
auto. Since the existence and behavior of this internal state is  
script-detectable, I think it would make sense to specify it. Any of the  
following promotes the internal state to auto:


1. The presence of autoplay forces it to auto
2. Calling play() by script or native controls
3. Setting currentTime by script or native controls

All of these are unconditional and can only be undone by resetting the  
internal preload state, see below.


== Dynamically changing preload ==

It makes no sense for a script to change preload=auto to preload=none.  
Going from preload=auto to preload=metadata isn't nonsensical, but  
supporting it would allow authors to toggle it continuously to work around  
buggy buffering behavior. I'd much rather that buffering problems be fixed  
in the browser, so I don't want to support this. Consequently, we only  
allow the internal preload states to increase, not decrease. I understand  
that Mozilla has done the same. Unless there are strong reasons not do, I  
think this should be spec'd.


== Resetting internal preload state ==

Due to the above, it's necessary to reset the internal preload state at  
some point. Otherwise, a script like this wouldn't work:


function setSource(url) {
  var v = document.querySelector('video');
  v.src = url;
  v.preload = none;
  v.onplay = function() { v.preload = autoplay; };
}

If a previous resource was playing and preload was set to autoplay by  
script, then we still want preload=none to apply to the new resource. To  
solve this, we are resetting the internal preload state as part of the  
resource selection algorithm, right before step 5 to fire the loadstart  
event. There are various other places one could do this, but we think it  
is important to do it in the async section so that the order of setting  
.src and .preload does not matter.


== video preload=none ==

It's not possible to specify exactly how much preload=metadata and  
preload=auto buffers and when, but this is possible for preload=none.  
This is what we do:


After step 1 of the source selection algorithm, if preload==none, set  
networkState to IDLE, fire a suspend event and set the  
delaying-the-load-event flag to false.


[1] http://my.opera.com/desktopteam/blog/2011/07/14/javascript-on-a-diet
[2] http://w3c-test.org/html/tests/submission/Opera/preload/overview.html
[3] http://w3c-test.org/html/tests/submission/Opera/preload/runner.php
[4] http://www.w3.org/Bugs/Public/show_bug.cgi?id=11602
[5] http://www.w3.org/Bugs/Public/show_bug.cgi?id=11981
[6] http://www.w3.org/Bugs/Public/show_bug.cgi?id=12175
[7] http://www.w3.org/Bugs/Public/show_bug.cgi?id=12267
[8] http://www.w3.org/Bugs/Public/show_bug.cgi?id=12595
[9] http://www.w3.org/Bugs/Public/show_bug.cgi?id=12596
[10] http://www.w3.org/Bugs/Public/show_bug.cgi?id=12608

--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] Microdata - Handling the case where a string is upgraded to an object

[Philip Jägenstedt]

On Mon, 18 Jul 2011 22:01:37 +0200, Tab Atkins Jr. jackalm...@gmail.com  
wrote:



On Mon, Jul 18, 2011 at 4:20 AM, Philip Jägenstedt phil...@opera.com



As for the solution, are you suggesting that .itemValue return a special
object which is like HTMLElement in all regards except for how it
toString()s?


Yes.


Currently, it's spec'd as returning the element itself. This isn't  
terribly useful, at least I've just checked e.itemScope and then accessed  
e.properties directly rather than going through e.itemValue.properties.  
Given this, a simpler fix would be to let .itemValue act like .textContent  
when an itemscope attribute is present.


Still, I'm not sure if it's a good idea. It makes the Microdata model kind  
of odd if a property is both an item and has a fallback text  
representation. It will also mask the fact that a text property has been  
upgraded to an item, somewhat decreasing the chance that the consuming  
code will be updated.


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] Microdata - Handling the case where a string is upgraded to an object

[Philip Jägenstedt]

On Thu, 14 Jul 2011 20:49:44 +0200, Tab Atkins Jr. jackalm...@gmail.com  
wrote:



Some IRC discussion this morning concerned the scenario where an API
starts by exposing a property as a string, but later wants to change
it to be a complex object.

This appears to be a reasonably common scenario.  For example, a
vocabulary with a name property may start with it being a string,
and then later change to an object exposing firstname/lastname/etc
properties.  A vocabulary for a music library may start by having
track as a string, then later expanding it to expose the track
title, the individual artist, the running time, etc.

In a very similar vein, the CSSOM is currently defined to always
return property values as strings.  We want to instead return complex
objects that expose useful information and interfaces specialized on
the value's type, however.  For compat reasons, we have to use an
entirely different accessor in order to expose this type of thing.

It seems that this may be a useful problem to solve in Microdata.  We
can expose either an attribute or a privileged property name for the
object's name/title/string representation.  Then, when using the
.items accessor, objects can be returned with a custom .toString that
returns that value, so they can be used as strings in legacy code.

Thoughts?


There is no items IDL attribute, do you mean getItems() or .itemValue  
perhaps?


I take it the problem is with code like this:

div itemscope itemtype=personspan itemprop=nameFoo  
Barsson/span/div

script
var p = document.getItems(person)[0];
alert(p.properties.namedItem(name)[0].itemValue);
/script

If the HTML changes to

div itemscope itemtype=personspan itemprop=name itemscopespan  
itemprop=givenNameFoo/span span  
itemprop=familyNameBarsson/span/span/div


then the script would be alerting [object HTMLElement] instead of Foo  
Barsson.


I'm not sure why this would be a problem. If someone changes the page,  
then can't they adjust the script to match? Is it extensions and libraries  
that you're worried about?


As for the solution, are you suggesting that .itemValue return a special  
object which is like HTMLElement in all regards except for how it  
toString()s?


--
Philip Jägenstedt
Core Developer
Opera Software

Re: [whatwg] Microdata feedback

[Philip Jägenstedt]

On Tue, 12 Jul 2011 09:41:18 +0200, Henri Sivonen hsivo...@iki.fi wrote:


On Thu, 2011-07-07 at 22:33 +, Ian Hickson wrote:

The JSON algorithm now ends the crawl when it hits a loop, and replaces
the offending duplicate item with the string ERROR.

The RDF algorithm preserves the loops, since doing so is possible with
RDF. Turns out the algorithm almost did this already, looks like it was  
an

oversight.


It seems to me that this approach creates an incentive for people who
want to do RDFesque things to publish deliberately non-conforming
microdata content that works the way they want for RDF-based consumers
but breaks for non-RDF consumers. If such content abounds and non-RDF
consumers are forced to support loopiness but extending the JSON
conversion algorithm in ad hoc ways, part of the benefit of microdata
over RDFa (treeness) is destroyed and the benefit of being well-defined
would be destroyed, too, for non-RDF consumption cases.


I don't have a strong opinion, but note that even before this change the  
algorithm produced a non-tree for the Avenue Q example [1] where the  
adr property is shared between two items using itemref. (In JSON, it is  
flattened.) If we want to ensure that RDF consumers don't depend on  
non-treeness, then this should change as well.


[1]  
http://www.whatwg.org/specs/web-apps/current-work/multipage/microdata.html#examples-4


--
Philip Jägenstedt
Core Developer
Opera Software