On Wed, 09 Feb 2011 03:57:37 +0100, Silvia Pfeiffer
<[email protected]> wrote:
A. Feedback on the WebVTT format
1. Introduce file-wide metadata
WebVTT requires a structure to add header-style metadata. We are here
talking about lists of name-value pairs as typically in use for header
information. The metadata can be optional, but we need a defined means
of adding them.
Required attributes in WebVTT files should be the main language in use
and the kind of data found in the WebVTT file - information that is
currently provided in the <track> element by the @srclang and @kind
attributes. These are necessary to allow the files to be interpreted
correctly by non-browser applications, for transcoding or to determine
if a file was created as a caption file or something else, in
particular the @kind=metadata. @srclang also sets the base
directionality for BiDi calculations.
Are there non-browsers that use the language for font-selection or
bidi? Is
auto-detection not likely to give a better user experience? Are there
any
other use cases for knowing the language of the captions *after* they've
been opened?
I can't see a different way to let non-browser applications know what
font to choose, even how to provide the user with a menu of available
caption tracks for a video, or to set the base directionality for
BiDi. Also, language auto-detection is a huge burden to put onto
non-browser applications. Having a readable language tag at the
beginning of the file is useful to quickly figure it all out.
The language set in <track> would certainly overrule what is in the
file. Also, the last language attribute in the header would probably
win.
I guess it would also be ok to have language and kind optional -
different applications may then default to interpreting WebVTT files
differently, such as by default English and Captions - or English and
Descriptions, but that's probably acceptable from context.
Given that most existing subtitle formats don't have any language
metadata, I'm a bit skeptical. However, if implementors of non-browser
players want to implement WebVTT and ask for this I won't stand in the way
(not that I could if I wanted to). For simplicity, I'd prefer the language
metadata from the file to not have any effect on browsers though, even if
no language is given on <track>.
Why do non-browser players need to know the kind? All kinds are
processed in
the same way except metadata, and there's no reason to use metadata
tracks
with external players.
Maybe I have a different view of what applications will make use of
WebVTT files than most. My thinking is that there will also be uses
for metadata tracks in external applications. Aside from this, there
will be authoring applications and players, yes, but there will also
be automated processing tools. So, to know what type of content is
inside a file without having to look at more than the file's headers
is really important.
For both of these cases, putting some magic strings inside comments that
are ignored by browsers sounds like it would be sufficient. Name-value
metadata that is ignored by browsers would be fine as well.
Further metadata fields that are typically used by authors to keep
specific authoring information or usage hints are necessary, too. As
examples of current use see the format of MPlayer mpsub’s header
metadata [2], EBU STL’s General Subtitle Information block [3], and
even CEA-608’s Extended Data Service with its StartDate, Station,
Program, Category and TVRating information [4]. Rather than specifying
a specific subset of potential fields we recommend to just have the
means to provide name-value pairs and leave it to the negotiation
between the author and the publisher which fields they expect of each
other.
This approach has worked very well with Vorbis Comments, probably mostly
because all interesting fields have been pre-defined in
http://www.xiph.org/vorbis/doc/v-comment.html
For a web format though, wouldn't some kind of wiki registry be good to
avoid total mayhem, especially if there are some predefined fields? (Not
having file-wide metadata would also avoid such mayhem.)
It might be good to define a base set - the Vorbis Comments one or the
ID3 ones could be appropriate. Even the old Dublin Core set (the first
ones, not the current chaos) could be good. I could also analyse the
sets used in current typical caption formats and propose a superset of
those.
While I think you're right with suggesting a predefined set of fields,
I am mostly keen right now to agree on the general format of the
fields and how we need to parse them rather than what they actually
are.
So, I would suggest we allow lines of "name=value" under the WEBVTT
magic string. A blank line defines the end of the header section and
the beginning of the cues. Would be simple enough to parse, right?
Sure, it's already handled by the current parsing spec, since it ignores
everything up to the first blank line.
4. Cue formatting requirements
In analysing the available cue formatting functionality, we have found
that some features are missing. Most of these features can be added
through using CSS on cues that have received a <b>, <i>, <c> or <v>
marker. The following features are core to traditional TV and exist in
EBU STL and CEA-608/708 captions. Support of these will be a core
requirement for browsers as well as non-browser applications and it
makes sense to add these to WebVTT rather than relying on external CSS
which cannot be used for non-browser captions:
The unstated requirement here seems to be that WebVTT needs to work as
an
interchange format for various TV captioning formats even in user agents
without any support for CSS (or JavaScript). I'm trying to not make a
straw
man argument, but if want an interchange format, we should pick TTML,
which
is explicitly designed to be just that and doesn't depend on CSS.
Is it not enough that a lossy conversion can be made from various
formats
into WebVTT+CSS(+JavaScript)? If not, the "Web" in "WebVTT" is highly
misleading...
We're trying to avoid the need for multiple transcodings and are
trying to achieve something like the following pipeline:
broadcast captions -> transcode to WebVTT -> show in browser ->
transcode to broadcast devices -> show
If we have to plug TTML into this pipeline, too, it will be much
slower and we would need to additionally define a mapping from TTML to
WebVTT and back.
I'm sure with SMPTE-TT around we will end up seeing things like
broadcast->TTML->WebVTT->browser, but even then we don't want WebVTT
to be a lossy format.
I can only disagree. Trying to make WebVTT into an interchange format will
inevitably turn it into a highly presentational format with lots of legacy
baggage. I can certainly see the use cases for an interchange format, but
I don't think it's worth the added complexity. I'd prefer an approach
where any format quirks that can't be mapped to WebVTT are expressed using
<c.foo> and if it turns out lots of people want the feature, we can add it
to a future revision.
* underline: EBU STL, CEA-608 and CEA-708 support underlining of
characters. The underline character is also particularly important for
some Asian languages. Please make it possible to provide text
underlines without the use of CSS in WebVTT.
Which Asian languages? If it's just the Chinese
<http://en.wikipedia.org/wiki/Proper_name_mark>, then I don't think that
needs <u> or similar. In my experience, use of the Chinese proper name
mark
is in fact extremely rare in Chinese captions, at least in movies and TV
series from the mainland and Taiwan. It would be best to use e.g.
我來自<c.pnm>中國</c> to make it easy to change the style between
single/double/wavy/no underline.
OK. So if we need underlined text, it will need to be
<c.underline>..</c> and CSS underline? I guess in a Web context
underline text is usually a hyperlink so it makes sense to discourage
<u> for the Web. But is that also an argument for
captions/subtitles/descriptions? What is the argument against using
<u> in captions?
I don't really have an argument against it, I just questioned that it is
important for Asian languages in particular. Adding <u> would be really
simple, it's just a question of why. I've seldom seen underlining in
captions, so it's not clear to me how it's usually used.
* font face: CEA-708 provides a choice of eight font tags: undefined,
monospaced serif, proportional serif, monospaced sans serif,
proportional sans serif, casual, cursive, small capital. These fonts
should be available for WebVTT as well. Is this the case?
Does the choice of font ever carry any semantic meaning? Isn't it a good
thing that captions can't specify their own fonts, so that it's easy to
pick
a style that's suitable for the embedding site?
The choice of fonts for captions has traditionally been a key to
providing quality captions. Some fonts are more readable than others.
So, captioning handbooks have traditionally prescribed the best fonts
to use for captioning to explicitly point out those that are easily
readable. After having checked with the handbooks that are available
to me it seems sans serif and proportional are the preferred ones, so
I do wonder why CEA-708 provides this choice of fonts. You are right
though that it makes more sense to provide semantic meaning and then
style through css. At minimum <c.cursive> etc would be possible with
an appropriate choice of font through styling, again using the class
span element to solve this.
Coming at it from a devices background, it's actually all a matter of
pre-defined choices. They're not going to package a large number of
fonts with every device, so it's good if all devices support a basic
subset that can be relied on to exist cross-device. We're increasingly
going to have to consider such requirements, too, because we will see
Web browsers run on devices with restricted capabilities, not just the
browser on a computer where you can install missing fonts.
Personally, I think we're going to see more and more devices running full
browsers with webfonts support, and that this isn't going to be a big
problem.
I guess what we are discovering is that we can define the general
format of WebVTT for the Web, but that there may be an additional need
to provide minimum implementation needs (a "profile" if you want - as
much as I hate this word). This seems to apply to the file-wide
metadata fields, to some specific standard classes (underline, blink),
to the set of colors supported and to the set of fonts supported. I
don't think these are issues that browsers need to worry about, and
therefore are probably beyond what we need to specify here for WebVTT.
But there probably needs to be a group to do this eventually.
The way that all standalone players I've used work is allowing the user to
pick a font and size which is used for all captions. I know I quite
appreciate this and wouldn't be thrilled if the captions file forced a
font I didn't like. Of course we could make it such that the author picks
a font and the user can override that. If it's purely presentational I'd
kind of prefer to keep it out of WebVTT, though.
5. Markup changes
* Suggest dropping “-->”: In the context of HTML, “-->” is an end
comment marker. It may confuse Web developers and parsers if such a
sign is used as a separator. For example, some translation tools
expect HTML or XML-based interchange formats and interpret the “>” as
part of a tag. Also, common caption convention often uses “>” to
represent speaker identification. Thus it is more difficult to write a
filter which correctly escapes “-->” but retains “>” for speaker ID.
Trying to use an HTML or XML parser to make any sense of WebVTT is
going to
fail horrendously in any case, so if anything I think it's good that
they
fail early. Also, a translation tool that has no concept of WebVTT is
going
to make a mess of various magic strings used in the file format too.
Since the “-->” characters serve no obvious purpose, it should be
possible to safely replace them by a blank that separates start and
end time, thus making the format denser and removing annoying parsing
issues. (Or alternatively use a the npt-range spec of RTSP for time
ranges, which uses “-” as a separator.).
No strong opinion, but I think a non-blank separator is more
aesthetically
pleasing.
Maybe just a dash "-" then, which can also remove the extra blanks?
Maybe :)
* Duration specification: WebVTT time stamps are always absolute time
stamps calculated in relation to the base time of synchronisation with
the media resource. While this is simple to deal with for machines, it
is much easier for hand-created captions to deal with relative time
stamps for cue end times and for the timestamp markers within cues.
Cue start times should continue to stay absolute time stamps.
Timestamp markers within cues should be relative to the cue start
time. Cue end times should be possible to be specified either as
absolute or relative timestamps. The relative time stamps could be
specified through a prefix of “+” in front of a “ss.mmm” second and
millisecond specification. These are not only simpler to read and
author, but are also more compact and therefore create smaller files.
An example document with relative timestamps is:
==
WEBVTT
Language=en
Kind=Subtitle
00:00:15.000 +2.950
At the left we can see...
00:00:18.160 +1.920
At the right we can see the...
00:00:20.110 +1.850
...the <+0.400>head-<+0.800>snarlers
==
I rather like it, although it might be confusing if "-" means "to
absolute
time" and "+" means "to relative time". That the intra-cue timings are
relative but the timing lines are absolute has bugged me a bit, so if
the
distinction was more obvious just from the syntax, that'd be great!
With "-" you are referring to replacing "-->" with "-" to arrive at
things like:
15.000-17.950
At the left we can see...
as compared to:
15.000+2.950
At the left we can see...
Yes, that's what I meant.
I actually think they read fairly given that people are used to the
double meaning of "-": to mean both "from ... to" and "minus".
But we could use a different character for "absolute time" if you
prefer, e.g. "/".
15.000/17.950
At the left we can see...
I find this fairly readable, too.
Either would work for me. As I mentioned, the room for improvement here
isn't only the syntax of the timing line, but also to make it obvious that
cue timestamps like <00:01.000> are relative. Using + for relative
timestamps is potentially confusing too, as one might think that many
consecutive <+00:01.000> are cumulative, rather than all being 1 second
from the start time of the cue.
6. Format identifier
We are happy to see the introduction of the magic file identifier for
WebVTT which will make it easier to identify the file format. We do
not believe the “FILE” part of the string is necessary.
I agree, mostly because it's ugly. While we're bikeshedding, "WebSRT" is
prettier than "WEBSRT".
"WebVTT" rather than "WebSRT"? ;-)
Yes :)
7. Comments
we recommend the introduction of comments.
I agree and think it needs to happen before WebVTT starts to get
implemented
and used on the web. In other words: now.
Agreed. I'm happy for the previously suggested "//" at the line start
to be comments, or, for that matter, "#" or ";" or any other special
character. I would prefer not to use "/*" since it implies a "*/" is
required to end the comment. Similarly we should avoid "<!--" and
"-->" or anything else that requires a special comment end mark and
more than one or two characters.
I'd quite like to have block comments, so I think the best options are:
1. // and /* */ like JavaScript
2. <!-- --> like HTML/XML
I think that the main difficulty is actually not picking a syntax, but
deciding how it works in the parser. Unlike HTML, I don't think we want
the comments to show up in the "DOM", since that would only work for
intra-cue comments. Ideally it would be preprocessor-ish, but yet the
magic bytes ("WEBVTT FILE") should be checked first as otherwise
identifying WebVTT would require implementing its preprocessor steps :/
8. Line wrapping
CEA-708 captions support automatic line wrapping in a more
sophisticated way than WebVTT -- see
http://en.wikipedia.org/wiki/CEA-708#Word_wrap.
In our experience with YouTube we have found that in certain
situations this type of automatic line wrapping is very useful.
Captions that were authored for display in a full-screen video may
contain too many words to be displayed fully within the actual video
presentation (note that mobile / desktop / internet TV devices may
each have a different amount of space available, and embedded videos
may be of arbitrary sizes). Furthermore, user-selected fonts or font
sizes may be larger than expected, especially for viewers who need
larger print.
WebVTT as currently specified wraps text at the edge of their
containing blocks, regardless of the value of the 'white-space'
property, even if doing so requires splitting a word where there is no
line breaking opportunity. This will tend to create poor quality
captions. For languages where it makes sense, line wrapping should
only be possible at carriage return, space, or hyphen characters, but
not on characters. (Note that CEA-708 also contains
non-breaking space and non-breaking transparent space characters to
help control wrapping.)However, this algorithm will not necessarily
work for all languages.
We therefore suggest that a better solution for line wrapping would be
to use the existing line wrapping algorithms of browsers, which are
presumably already language-sensitive.
[Note: the YouTube line wrapping algorithm goes even further by
splitting single caption cues into multiple cues if there is too much
text to reasonably fit within the area. YouTube then adjusts the times
of these caption cues so they appear sequentially. Perhaps this could
be mentioned as another option for server-side tools.]
Yeah, with SRT people are manually line-wrapping when authoring the
captions
and often enough the end result is that you get something rendered:
- Who could have guessed that not all fonts are the same
size?
- That's news to me, so I get four lines of text where I
wanted two!
I'm inclined to say that we should normalize all whitespace during
parsing
and not have explicit line breaks at all. If people really want two
lines,
they should use two cues. In practice, I don't know how well that would
fare, though. What other solutions are there?
I don't think I would go that far. The concern has mostly been with
the line wrapping of lines that are too long and the possibility of
splitting words that way. The particular concern was with this
paragraph:
"Text runs must be wrapped at the edge of their containing blocks,
regardless of the value of the 'white-space' property, even if doing
so requires splitting a word where there is no line breaking
opportunity."
see
http://www.whatwg.org/specs/web-apps/current-work/multipage/rendering.html#timed-text-tracks-0
So we want to avoid splitting mid-word and we suggest introducing the
ability to have non-breaking spaces.
I think splitting in the middle of words would only happen for words that
are longer than the whole line.
There's still plenty of room for improvements in line wrapping, though. It
seems to me that the main reason that people line wrap captions manually
is to avoid getting two lines of very different length, as that looks
quite unbalanced. There's no way to make that happen with CSS, and AFAIK
it's not done by the WebVTT rendering spec either.
B. Feedback on the <track> element
1. Pop-on/paint-on/roll-up support
Three different types of captions are common on TV: pop-on, roll-up
and paint-on. Captions according to CEA-608/708 need to support
captions of all three of these types. We believe they are already
supported in WebVTT, but see a need to re-confirm.
The underlying use case here is live captioning, right? Just copying the
styling used on broadcast TV seems like it wouldn't be enough, you also
need
the ability to erase typos, right? Are there any existing captioning
formats
that handle live captioning well from which one could draw inspiration?
Yes, CEA-608/609 do these things and we have analysed them for these
features. They have control characters for backspace (only within
row), delete to end of row, erase displayed memory and erase
non-displayed memory. Further there is the concept of a cursor and
there are means to move the cursor to other screen locations.
I don't think we really need the concept of a cursor or display memory
and we don't need backspace and delete etc. because we have the
concept of mutableTimedTrack. So, a live captioning application can
always remove an existing TimedTrackCue and replace it with a new one
where the errors are fixed. At Google we came to the conclusion that
this was sufficient and therefore did not see a need to request
features for this type of application.
However, the three types of captions are actually not just used in
live captioning, but they are three different captioning styles that
could all be created by live or "canned" captions. We think they can
be supported, so this is good news.
If you have to rely on scripting to fix errors, I think you could rely on
scripts for achieving whatever other effects you need as well, in
particular the scrolling style.
2. Duplicate track
The HTML spec specifies that it is not allowed to have two tracks that
provide the same kind of data for the same language (potentially
empty) and for the same label (potentially empty). However, we need
clarification on what happens if there is a duplicate track, ie: does
the most recent one win or the first one or will both be made
available in the UI and JavaScript? The spec only states that the
combination of {kind, type, label} must be unique. It doesn't say what
happens if they are not.
In <http://whatwg.org/html#sourcing-out-of-band-text-tracks> all track
are
added to the list of text tracks, even duplicates.
In other words, it's just a requirement for validators, not user agents.
OK, so a browser still has to deal with "duplicate tracks" as though
they were not duplicates?
Yes, that's how I interpret the spec.
4. Addressing individual cues through CSS
As far as we understand, you can currently address all cues through
::cue and you can address a cue part through ::cue-part(<voice> ||
<part> || <position> || <future-compatibility>). However, if we
understand correctly, it doesn’t seem to be possible to address an
individual cue through CSS, even though cues have individual
identifiers. This is either an oversight or a misunderstanding on our
parts. Can you please clarify how it is possible to address an
individual cue through CSS?
Since I've been arguing against the id's in WebVTT, I'm curious about
the
use case here. Isn't using a unique class good enough?
This links in with the discussion above on CSS styling and classes.
Rather than define classes of cue settings and reference them from the
cues, this allows them to be applied to individual cues in style
sheets. I thought the whole reason of cue identifiers was to have this
addressing functionality, so this would just close the loop.
For example:
Style sheet of the Web page:
<style>
video track#t1 ::cue(cue10) {
text-decoration: blink;
}
</style>
The Web page (extract):
<video src="video.webm" controls>
<track id="t1" label="captions" kind="captions" srclang="en-US"
src="cap1.vtt"/>
</video>
The caption file cap1.vtt:
WEBVTT
Language=en-US
Kind=Captions
cue1
0.000-5.000
blab blah
cue10
40.000-60.000
ALERT: Your basement is flooding - evacuate!
Cue10 is addressed through CSS and turned into a blinking text without
a need to change the markup at all.
My point was that you could just as well do this:
0.000-5.000
<c.cue1>blab blah</c>
In my view of things, id's in HTML are primarily for addressing via
#fragments and as hooks for scripts, for styling class is quite
sufficient, so I'm thinking it would be for WebVTT as well.
5. Ability to move captions out of the way
Our experience with automated caption creation and positioning on
YouTube indicates that it is almost impossible to always place the
captions out of the way of where a user may be interested to look at.
We therefore allow users to dynamically move the caption rendering
area to a different viewport position to reveal what is underneath. We
recommend such drag-and-drop functionality also be made available for
TimedTrack captions on the Web, especially when no specific
positioning information is provided.
This would indeed be rather nice, but wouldn't it interfere with text
selection? Detaching the captions into a floating, draggable window via
the
context menu would be a theoretically possible solution, but that's
getting
rather far ahead of ourselves before we have basic captioning support.
On YouTube you can only move them within the video viewport. You
should try it - it's really awesome actually.
When you say "interfere with text selection" are you suggesting that
the text of captions/subtitles should be able to be cut and pasted? I
wonder what copyright holders think about that.
Being able to select the captions just like any other text is a great
thing that I wouldn't want to disable. It's very useful if you want to
pause and look up the definition of a word or to report a typo in the
captions without having to retype the whole text.
Premium Captions can be protected using the same tricks that are used to
prevent Premium DOM Text Nodes from being copied.
--
Philip Jägenstedt
Core Developer
Opera Software
