[whatwg] JSON Playlist Format
Hi WHATWG, We've been getting some traffic on our forums from users wondering about JSON playlist formats. Specifically, because we aren't supporting XML playlists in our player in HTML5 mode (setting the Access-Control-Allow-Origin headers would be impossible for most of our users), they're looking for a JavaScript-friendly way to encapsulate their playlists. A very cursory search turned up JSPF (http://wiki.xiph.org/JSPF_Draft), but it appears to be in draft. Are y'all aware of any other formats, either in draft or that have been released? If no, any idea why no one's done it already? Best, Zach -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution
Re: [whatwg] JSON Playlist Format
The use case I'm thinking of is a blogging service that allows a publisher to upload static files, but they cannot host executable code like a proxy. They could include a script tag as part of their content and use a third party JSONP (http://en.wikipedia.org/wiki/JSONP) proxy to load their playlist. Additionally, in older browsers (IE 6 7), I believe that Access-Control-Allow-Origin headers aren't respected, so you'd never be able to load XML cross domain, but you can load script tags. Best, Zach -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution On Thu, May 5, 2011 at 12:08 PM, Nils Dagsson Moskopp n...@dieweltistgarnichtso.net wrote: Zachary Ozer z...@longtailvideo.com schrieb am Thu, 5 May 2011 11:17:42 -0400: We've been getting some traffic on our forums from users wondering about JSON playlist formats. Specifically, because we aren't supporting XML playlists in our player in HTML5 mode (setting the Access-Control-Allow-Origin headers would be impossible for most of our users), they're looking for a JavaScript-friendly way to encapsulate their playlists. Which would also need CORS measures, or am I mistaken here? A very cursory search turned up JSPF (http://wiki.xiph.org/JSPF_Draft), but it appears to be in draft. Are y'all aware of any other formats, either in draft or that have been released? If no, any idea why no one's done it already? Probably because fallback to a proxy is way easier. Anything blocking the use of XSPF + pass-through proxies? Cheers, -- Nils Dagsson Moskopp // erlehmann http://dieweltistgarnichtso.net
Re: [whatwg] Limiting the amount of downloaded but not watched video
On Thu, Jan 20, 2011 at 3:14 AM, Philip Jägenstedt phil...@opera.comwrote: * effective state can only increase to higher states, never go from e.g. metadata to none (it makes no sense) What if my bandwidth situation improves (moving from 3g to WiFi, for example)? At that point, perhaps I should go from auto to invoked? On Thu, Jan 20, 2011 at 4:19 AM, Glenn Maynard gl...@zewt.org wrote: I think that pausing shouldn't affect read-ahead buffering behavior. I'd suggest another preload value, preload=buffer, sitting between metadata and auto. In addition to everything loaded by metadata, it also fills the read-ahead buffer (whether the video is playing or not). Would the read-ahead buffer length be a configurable property? == New Proposal == Based on the feedback we've gotten, I'd like to propose the following: * Adding an additional preload state between metadata and auto (I'll call it state3, but we should name it invoked or buffer) * Adding the downloadTargetBuffer property, which can be updated at any time === Use cases === Starting with preload = none / meta * Once the user has indicated they want to play a video, the browser will go into state3 and buffer at least downloadTargetBuffer seconds of video before playing * Once in state3, the browser will attempt to maintain a buffer of approximately downloadTargetBuffer seconds, regardless of state * If the browser cannot maintain a buffer of length downloadTargetBuffer, it will set preload = auto * If preload is later set back to state3, the browser will: If the buffer = downloadTargetBuffer seconds: - The browser will play through the current buffer surplus until it has approximately downloadTargetBuffer seconds in the buffer - Once the surplus has been used, the browser will attempt to maintain a buffer of approximately downloadTargetBuffer seconds - If the browser cannot maintain a buffer of length downloadTargetBuffer, it will set preload = auto If the buffer downloadTargetBuffer seconds: - The browser will set preload = auto Starting with preload = state3 * On page load, the browser will go into state3 and buffer at least downloadTargetBuffer seconds of video before playing * Once the user has indicated they want to play a video, the browser will attempt to maintain a buffer of approximately downloadTargetBuffer seconds, regardless of state * If the browser cannot maintain a buffer of length downloadTargetBuffer, it will set preload = auto * If preload is later set back to state3, the browser will: If the buffer = downloadTargetBuffer seconds: - The browser will play through the current buffer surplus until it has approximately downloadTargetBuffer seconds in the buffer - Once the surplus has been used, the browser will attempt to maintain a buffer of approximately downloadTargetBuffer seconds - If the browser cannot maintain a buffer of length downloadTargetBuffer, it will set preload = auto If the buffer downloadTargetBuffer seconds: - The browser will set preload = auto Starting with preload = auto * On page load, the browser will go into auto and buffer as much video as possible * If preload is later set to state3, the browser will: If the buffer = downloadTargetBuffer seconds: - The browser will play through the current buffer surplus until it has approximately downloadTargetBuffer seconds in the buffer - Once the surplus has been used, the browser will attempt to maintain a buffer of approximately downloadTargetBuffer seconds - If the browser cannot maintain a buffer of length downloadTargetBuffer, it will set preload = auto If the buffer downloadTargetBuffer seconds: - The browser will set preload = auto Best, Zach -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution
Re: [whatwg] Limiting the amount of downloaded but not watched video
We never make any promises about when we'll get something into an official release, but I think we'd start playing around with it in our development version once a reference implementation was publicly available. -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution On Thu, Jan 20, 2011 at 7:03 PM, Roger Hågensen resca...@emsai.net wrote: On 2011-01-20 19:16, Zachary Ozer wrote: == New Proposal == I like this. It seems you laid out everything to ensure a balanced buffer, kinda like a moving window buffer which I pointed out earlier. So as far as I can see, your proposal looks pretty solid, unless there are any implementation snafus. (looks at the Chrome, Safari, Opera, Firefox guys in the list (hmm, where's the IE guys?)) I really like the way you described the state3, and I think that would be my personal preference for playback myself. I assume JW player would be very quick at supporting/using it? -- Roger Rescator Hågensen. Freelancer - http://www.EmSai.net/
Re: [whatwg] Limiting the amount of downloaded but not watched video
Two ideas just struck me: == Network API calls == What if, instead of trying to solve this problem, we leave it up to the publishers. The current behavior would be unchanged, but we could add explicit bandwidth management API calls, ie startBuffer() and stopBuffer(). This would let developers / site publishers control how much to buffer and when. == User Behavior == We might also consider leaning on users a bit to tell us what they want. For example, I think people are pretty used to hitting play and then pause to buffer until the end of the video. What if we just used our bandwidth heuristics while in the play state, and buffered blindly when a pause occurs less than X seconds into a video? I won't argue that this is a wonderful solution (or a habit we should encourage), but I figured I'd throw a random idea out there… Best, Zach -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution On Wed, Jan 19, 2011 at 8:22 AM, Philip Jägenstedt phil...@opera.com wrote: On Wed, 19 Jan 2011 10:42:01 +0100, Aryeh Gregor simetrical+...@gmail.com wrote: On Mon, Jan 17, 2011 at 6:41 PM, Jeroen Wijering jer...@longtailvideo.com wrote: We are getting some questions from JW Player users that HTML5 video is quite wasteful on bandwidth for longer videos (think 10min+). This because browsers download the entire movie once playback starts, regardless of whether a user pauses the player. If throttling is used, it seems very conservative, which means a lot of unwatched video is in the buffer when a user unloads a video. A lot of people are complaining about this right now, and it needs to be addressed, but nothing I've seen makes it clear that it's a spec issue rather than QoI. I.e., will this still be a problem once all browsers reliably implement preload= and iron our their video fetch behavior a bit more? Are non-Mozilla browsers working on preload=? I'm working on video preload right now, so have had reason to think a lot about this recently. At first sight exposing something like downloadBufferTarget seems quite reasonable, but things are complicated, as usual... == Preload States == The 3 preload states imply 3 simple buffering strategies: none: don't touch the network at all preload: buffer as little as possible while still reaching readyState HAVE_METADATA auto: buffer as fast and much as possible However, the state we're discussing is when the user has begun playing the video. The spec doesn't talk about it, but I call it: invoked: buffer as little as possible without readyState dropping below HAVE_FUTURE_DATA (in other words: being able to play from currentTime to duration at playbackRate without waiting for the network) == Bandwidth Management == If the available bandwidth exceeds the bandwidth of the resource, some kind of throttling must eventually be used. There are mainly 2 options for doing this: 1. Throttle at the TCP level by not reading data from the socket (not at all to suspend, or at a controlled rate to buffer ahead) 2. Use HTTP byte ranges, making many smaller requests with any kind of throttling at the TCP level After much deliberation, I've tentatively concluded that option 2 is best on balance, for the following reasons: * Browsers already have the infrastructure to do byte range requests for video * More requests gives the server more opportunities to do load balancing * Servers may limit the number of concurrent requests, and the long-lived requests caused by TCP throttling add to the total * Using HTTP byte range requests, connections don't live longer than necessary and it's possible to interleave fetching of many resources from the same server * Smooth reverse playback (playbackRate 0) is only possible using byte range requests (or waiting for the whole resource to buffer, which could be impossible with a limited cache size) * HTTP pipelining should ensure that a new connection doesn't need to be established if the requests are not too far apart (Since there is some overhead with each HTTP request, one must make sure that they are not unreasonably small.) When HTTP byte ranges are used to achieve bandwidth management, it's hard to talk about a single downloadBufferTarget that is the number of seconds buffered ahead. Rather, there might be an upper and lower limit within which the browser tries to stay, so that each request can be of a reasonable size. Neither an author-provided minumum or maximum value can be followed particularly closely, but could possibly be taken as a hint of some sort. == User Expectations == The above buffering strategies are still not enough, because users seem to expect that in a low-bandwidth situation, the video will keep buffering until they can watch it through to the end. These seem to be the options for solving the problem: * Make sites that want
Re: [whatwg] Limiting the amount of downloaded but not watched video
I've heard from some people that they're a bit lost, so I wanted to take a moment to summarize. We have two competing interests here: * Viewers want a smooth playback experience regardless of their bandwidth or device. Some viewers may also want to limit the amount they download because they're paying for bandwidth. Additionally, devices may have memory limitations in terms of how much they're able to buffer. * Content providers are worried about bandwidth costs. While they want a great experience for viewers, a lot of people click play and then watch just a small fraction of their video. Currently, there's no way to stop / limit the browser from buffering - once you hit play, you start downloading and don't stop until the resource is completely loaded. This is largely the same as Flash, save the fact that some browsers don't respect the preload attribute. (Side note: I also haven't found a browser that stops loading the resource even if you destroy the video tag.) There have been a few suggestions for how to deal with this, but most have revolved around using downloadBufferTarget - a settable property that determines how much video to buffer ahead in seconds. Originally, it was suggested that the content producers should have control over this, but most seem to favor the client retaining some control since they are the most likely to be in low bandwidth situations. (Publishers who want strict bandwidth control could use a more advanced server and communication layer ala YouTube). The simplest enhancement would be to honor the downloadBufferTarget only when readyState=HAVE_ENOUGH_DATA and playback is paused, as this would imply that there is not a low bandwidth situation. As an enhancement to this, the browser could always respect the downloadBufferTarget until the buffer underruns (networkState=NETWORK_LOADING and readyState=HAVE_CURRENT_DATA). At this point, the browser could either: * Ignore downloadBufferTarget and load as fast as possible * Double the size of downloadBufferTarget As a further enhancement, the browser could store these values per site so that they are not recalculated on each playback. Finally, if there is a playback with no underruns, the browser would reduce downloadBufferTarget by some factor to ensure that it is not over buffering. Separately, there has been some discussion about how much buffer needs to be retained / when the buffer should be cleared. (I think this should be moved off to a separate discussion.) == Personally, I really like the idea of allowing the content provider to specify a downloadBufferTarget, but allowing the browser to override this based on historical data / current network conditions. I'm not sure how much work each of the proposed solutions would be, I think that respecting downloadBufferTarget until the buffer underruns and then downloading as fast as possible would be fairly straight forward, and a big improvement on what's available today. Best, Zach -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution On Tue, Jan 18, 2011 at 1:40 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/18/11 6:09 AM, Glenn Maynard wrote: I'm confused--how is the required buffer size a function of the length of the video? Once the buffer is large enough to smooth out network fluctuations, either you have the bandwidth to stream the video or you don't; the length of the video doesn't enter into it. The point is that many users _don't_ have enough bandwidth to stream the video. At that point, the size of the buffer that puts you in HAVE_ENOUGH_DATA depends on the length of the video. -Boris
Re: [whatwg] Limiting the amount of downloaded but not watched video
On Tue, Jan 18, 2011 at 6:46 PM, Boris Zbarsky bzbar...@mit.edu wrote: On 1/18/11 12:11 PM, Zachary Ozer wrote: (Side note: I also haven't found a browser that stops loading the resource even if you destroy the video tag.) destroy in what sense? You verified in a debugger that it had been garbage collected? I'm doing document.body.removeChild. Is there a better way to do it? Best, Zach
Re: [whatwg] Limiting the amount of downloaded but not watched video
What no one has mentioned so far is that the real issue isn't the network utilization or the memory capacity of the devices, it's bandwidth cost. The big issue for publishers is that they're incurring higher costs when using the video tag, which is a disincentive for adoption. Since there are situations where both the publisher and the user are potentially incurring bandwidth costs (or have other limitations), we could allow the publisher to specify downloadBufferTarget and the user to specify a setting in the browser's config. The browser would then actually buffer min(user setting, downloadBufferTarget). At that point there would probably need to be another read-only property that specified what value the browser is currently using as it's buffer length, but maybe the getter for downloadBufferTarget is sufficient. Best, Zach -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution On Mon, Jan 17, 2011 at 6:19 PM, Roger Hågensen resca...@emsai.net wrote: On 2011-01-17 18:36, Markus Ernst wrote: Am 17.01.2011 17:41 schrieb Jeroen Wijering: We are getting some questions from JW Player users that HTML5 video is quite wasteful on bandwidth for longer videos (think 10min+). This because browsers download the entire movie once playback starts, regardless of whether a user pauses the player. If throttling is used, it seems very conservative, which means a lot of unwatched video is in the buffer when a user unloads a video. Could this be done at the user side, e.g. with some browser setting? Or even by a stop downloading control in the player? An intuitive user control would be separate stop and pause buttons, as we know them from tape and CD players. Pause would then behave as it does now, while stop would cancel downloading. I think that's the right way to do it, this should be in the hands of the user and exposed as a preference in the browsers. Although exposing (read only?) the user's preferred buffer setting to the HTML App/Plugin etc. would be a benefit I guess as the desired buffering could be communicated back to the streaming server for example for a better bandwidth utilization. -- Roger Rescator Hågensen. Freelancer - http://www.EmSai.net/
Re: [whatwg] Limiting the amount of downloaded but not watched video
On Mon, Jan 17, 2011 at 9:08 PM, Chris Pearce ch...@pearce.org.nz wrote: On 18/01/2011 8:05 a.m., Zachary Ozer wrote: What no one has mentioned so far is that the real issue isn't the network utilization or the memory capacity of the devices, it's bandwidth cost. The big issue for publishers is that they're incurring higher costs when using thevideo tag, which is a disincentive for adoption. I assume you're comparing to the bandwidth usage of flash? Does flash allow developers to control how the media is downloaded on the client? What mechanisms does it provide? Maybe we can do something similar? There are a bunch: * backBufferLength : Number - [read-only] The number of seconds of previously displayed data that currently cached for rewinding and playback. * backBufferTime : Number - Specifies how much previously displayed data Flash Player tries to cache for rewinding and playback, in seconds. * bufferLength : Number - [read-only] The number of seconds of data currently in the buffer. * bufferTime : Number - Specifies how long to buffer messages before starting to display the stream. * bufferTimeMax : Number - Specifies a maximum buffer length for live streaming content, in seconds. * bytesLoaded : uint - [read-only] The number of bytes of data that have been loaded into the application. * bytesTotal : uint - [read-only] The total size in bytes of the file being loaded into the application. See http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/net/NetStream.html Can this not be implemented on the server side? If you know the media has an average playback rate of X KB/s, can the server reliably throttle its transmission at 1.5X? I suppose it could be implemented server side, but it's not currently part of the HTTP spec (so far as I'm aware). That would *really* hinder adoption.
Re: [whatwg] Html 5 video element's poster attribute
On Wed, Sep 22, 2010 at 12:14 AM, Shiv Kumar sku...@exposureroom.com wrote: Now as far as how to fix this is concerned, I've repeatedly provided use cases for why the load() may not work (won't satisfy one of the use cases I've cited) as well as why the load() method shouldn't be used (because of all the side effects it produces). Totally agree - calling load() has a *ton* of side effects besides just showing the poster and it doesn't seem appropriate, except for (perhaps) the case of playback complete / stopped. While I have your attention, what are your thoughts and what you call The Golden Rule and Html 5 video players? Different discussion (content protection). More than happy to take it up in another thread. -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution
Re: [whatwg] Html 5 video element's poster attribute
Shiv - Why is simply removing the poster attribute unacceptable? -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution On Tue, Sep 21, 2010 at 10:45 AM, Shiv Kumar sku...@exposureroom.com wrote: Benjamin, To add to my earlier reply... I proposed that there be a method such as setPostervisibility(bool) to implement this feature while Silvia is proposing this feature be implemented via the load() method. So it then makes sense that she give good reason for and against, just as I have. Shiv http://exposureroom.com
Re: [whatwg] Html 5 video element's poster attribute
On Tue, Sep 21, 2010 at 1:53 PM, Shiv Kumar sku...@exposureroom.com wrote:
Zachary,
Why is simply removing the poster attribute unacceptable?
Gosh, that's pretty radical isn't it? I've thought about it and if the poster
is not usable then yes, I'd rather not have it at all and resort to the
implementations (by websites) currently in place.
Sorry to be unclear - I should have specified that I meant removing
the poster attribute for a specific video tag, vis-a-vis
'videoElement.removeAttribute(''poster)'.
On Tue, Sep 21, 2010 at 2:17 PM, Shiv Kumar sku...@exposureroom.com wrote:
Benjamin,
Ok, I see your point. So yes, I'm asking for a better implementation of an
existing feature.
So far I've made two proposals as regards the video element's poster.
1. The poster should stay visible until the video is played, rather than
disappear as soon as the first frame is loaded. In addition, the poster
should not show during buffering or any operation during video playback or
switching video streams in mid step.
2. We need direct control over the poster.
In principal, both seem to have been accepted. The ongoing debate is about
how to implement, #2 above. So far there have been two proposals
1. Enhance the existing load() method to do this.
2. provide a spate method (for example setPostervisibility(bool)) to do
this.
My reasoning (mind you I've provided many use cases to support this) is
that:
1. Currently, the poster is treated as a stepchild of the video element in
that there is *no direct control* over the poster.
2. I also see the poster separate from the video in that it's only
functionality is show/hide (or fade in/out). Attaching it to
logic/processing sequences associated to loading and initializing the video
stream (which has many side effects) is not appropriate.
3. Being able to turn on/off a poster *without side effects* is very
important.
Using the current API, we've implemented the alternative you've
described, namely removing the poster attribute, placing another image
above video (of the same size), and swapping z-indexes as needed.
The other benefit to this is that it allows to to appropriately
stretch / size the poster image.
As you point out, the video and poster are stepchildren - related in
that it's clear that certain properties should change in lockstep, but
necessarily clear *how* they should change. Assume for a moment that
browsers allowed separate CSS configurations for the video and poster.
What happens when I change the size of the video? Would it update
both? Just the active one? Additionally - size is the obvious case.
What about opacity? Stretching (which should be fixed in CSS at some
point - http://dev.w3.org/csswg/css3-images/#object-fit0)?
My point is that it gets very complicated very quickly, and it's not
clear what to do. So we go to use cases: concrete examples that
demonstrate why a certain functionality is needed / useful and cannot
be replicated otherwise.
As an analogy, consider the audio track for a video. We also allow
audio levels to be controlled separately from the video via volume.
This is because there is no other way to adjust the volume without
changing the audio file itself.
However, it's possible to show / hide the poster either by hiding the
video tag (and adding an image) or by removing the poster attribute
from the tag itself. You could even dial back the opacity of the video
tag (analogous to lowering the volume of the video).
For example, your issue got me thinking that there should be an API
for telling the browser to render whatever's inside of a video tag
while the video isn't play (with the controls on top) since my gut
tells me that there are use cases for this. However, I'm really
ambivalent, since there are many other ways to achieve the same effect
(mostly), so why not just use those? Until I have a really compelling
use case, I can't really justify proposing that it go into the spec.
As other people have said, it would be really useful if you could
provide a *compelling* use case, one where the other proposed
solutions wouldn't do.
Best,
Zach
--
Zachary Ozer
Developer, LongTail Video
w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 •
f: 212.656.1335
JW Player | Bits on the Run | AdSolution
Re: [whatwg] Html 5 video element's poster attribute
Part of the issue is that I'm not sure.
Now that the poster (rather than the first frame) should appear for
Webkit browsers, and stretching should be handled by CSS (for both the
video and poster), I'm rather satisfied.
While I can think of times when one *might* want it to also appear (on
buffer, on pause, etc), I'm not sure it's used in practice. Is there a
different instance where it would be usable?
And for this, I refer you to Einstein, who said (approximately), Make
things as simple as possible, but not simpler. [1]
Best,
Zach
[1] : http://en.wikiquote.org/wiki/Albert_Einstein
--
Zachary Ozer
Developer, LongTail Video
w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 •
f: 212.656.1335
JW Player | Bits on the Run | AdSolution
On Tue, Sep 21, 2010 at 3:45 PM, Shiv Kumar sku...@exposureroom.com wrote:
Hi Zach,
Sure pretty much anything is possible. In fact everything in the Html 5 spec
is already doable is it not? If it's not JavaScript that comes to the rescue
it's Flash player or a combination of the two.
I understand completely your point of view on the other stuff you described
and believe me I don't think for a moment that defining a spec is easy or the
job of implementers is easy.
But I do believe the purpose of this spec (Html 5) is to make what people
have been doing over the years using JavaScript and Flash player) easier and
more approachable.
Who was it that said, If it's worth doing, then it's worth doing right?
Since there *is* a poster feature to the video element then I'd like to see
it implemented such that it if useful. Or as I stated earlier, I'd rather see
it no there. And honestly, what is compelling to some, is not compelling
enough for others. People are using this so there is a use case.
Would you support taking the poster attribute out of the spec?
Or are you saying just leave it broken because fixing it a really complicated.
I'm not sure where you stand on the issue of providing a method to turn
on/off the poster or modify the spec on the load() method to include verbiage
about how the implementations should treat the poster attribute.
Shiv
http://exposureroom.com
-Original Message-
From: whatwg-boun...@lists.whatwg.org
[mailto:whatwg-boun...@lists.whatwg.org] On Behalf Of Zachary Ozer
Sent: Tuesday, September 21, 2010 3:15 PM
To: Shiv Kumar
Cc: Silvia Pfeiffer; WHATWG; Benjamin Hawkes-Lewis
Subject: Re: [whatwg] Html 5 video element's poster attribute
On Tue, Sep 21, 2010 at 1:53 PM, Shiv Kumar sku...@exposureroom.com wrote:
Zachary,
Why is simply removing the poster attribute unacceptable?
Gosh, that's pretty radical isn't it? I've thought about it and if the
poster is not usable then yes, I'd rather not have it at all and resort to
the implementations (by websites) currently in place.
Sorry to be unclear - I should have specified that I meant removing
the poster attribute for a specific video tag, vis-a-vis
'videoElement.removeAttribute(''poster)'.
On Tue, Sep 21, 2010 at 2:17 PM, Shiv Kumar sku...@exposureroom.com wrote:
Benjamin,
Ok, I see your point. So yes, I'm asking for a better implementation of an
existing feature.
So far I've made two proposals as regards the video element's poster.
1. The poster should stay visible until the video is played, rather than
disappear as soon as the first frame is loaded. In addition, the poster
should not show during buffering or any operation during video playback or
switching video streams in mid step.
2. We need direct control over the poster.
In principal, both seem to have been accepted. The ongoing debate is about
how to implement, #2 above. So far there have been two proposals
1. Enhance the existing load() method to do this.
2. provide a spate method (for example setPostervisibility(bool)) to do
this.
My reasoning (mind you I've provided many use cases to support this) is
that:
1. Currently, the poster is treated as a stepchild of the video element in
that there is *no direct control* over the poster.
2. I also see the poster separate from the video in that it's only
functionality is show/hide (or fade in/out). Attaching it to
logic/processing sequences associated to loading and initializing the video
stream (which has many side effects) is not appropriate.
3. Being able to turn on/off a poster *without side effects* is very
important.
Using the current API, we've implemented the alternative you've
described, namely removing the poster attribute, placing another image
above video (of the same size), and swapping z-indexes as needed.
The other benefit to this is that it allows to to appropriately
stretch / size the poster image.
As you point out, the video and poster are stepchildren - related in
that it's clear that certain properties should change in lockstep, but
necessarily clear *how* they should change. Assume for a moment that
browsers allowed separate CSS configurations
Re: [whatwg] Html 5 video element's poster attribute
I'd like to weight in quickly on this based on feedback from our users on this (they have a lot). * Webkit's original implementation (show the first frame once it's available) is requested by a lot of people. What they don't realize is that the first frame is black for 99% of videos, so they end up using the poster anyway. We've resisted this because of limitations in Flash (you have to start loading the video, then call play() and pause() on the first frame), but I'd say it's still a good idea to display the first frame if there's no poster set and preload is set to auto (or add another preload level for poster). * Don't show the poster when the video buffers - just pause the video and give some visual indication that you're buffering. * We've never had anyone request different poster images for begin / pause / end. People generally just want it to appear at the beginning and end, and they want the same image. If someone wants to change it, allow them to set the poster attribute via JavaScript. * Don't clear the poster on load(). A lot of people get confused by this. It might make sense to clear it in the setter for video src, but I could see this confusing people as well. * I'm not sure how reset() would work. Would you reset the list of source too? Best, Zach -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution On Mon, Sep 20, 2010 at 10:04 AM, Eric Carlson eric.carl...@apple.com wrote: On Sep 19, 2010, at 3:17 PM, Silvia Pfeiffer wrote: Not quite: this is an implementation decision that Webkit-based browsers made. Neither Opera nor Firefox work that way (haven't checked IE yet). I agree that this implementation of poster frames is practically useless and it really annoys me as a user. I've been considering registering a bug on Webkit. On Sep 19, 2010, at 5:50 PM, Aryeh Gregor wrote: On Sun, Sep 19, 2010 at 4:53 PM, Shiv Kumar sku...@exposureroom.com wrote: The poster frame should remain visible until the video is played. I agree with Silvia, this should be required by the spec. The alternative is clearly wrong. Someone should also file a bug with WebKit to ask them to change. Someone might want to try a WebKit nightly build before filing a bug. This was changed in r64884. A poster is displayed until there is a movie frame to display and playback begins or the current time is changed. eric
Re: [whatwg] Video with MIME type application/octet-stream
On Wed, Sep 1, 2010 at 10:51 AM, Adrian Sutton adrian.sut...@ephox.com wrote: Given that there is a very limited set of video formats that are supported anyway, wouldn't it be reasonable to just identify or define the standard file extensions then work with server vendors to update their standard file extension to mime type definitions to include that. While adoption and upgrading to the new versions would obviously take time, that applies to the video tag itself anyway and is just a temporary source of pain. At first glance, my eyes almost popped out of my sockets when I saw this suggestion. Using the file extension?! He must be mad! Then I remembered that our Flash player *has* to use file extension since the MIME type isn't available in Flash. Turns out that file extension is a pretty good indicator, but it doesn't work for custom server configurations where videos don't have extensions, ala YouTube. For that reason, we allow users to override whatever we detect with a type configuration parameter. Ultimately, the question is, What are we trying to accomplish? I think we're trying to make it easy for content creators to guarantee that their content is available to all viewers regardless of their browser. If that's the case, I'd actually suggest that the browsers *strictly* follow the MIME type, with the source type as a override, and eliminating all sniffing (assuming that the file container format contains the codec meta-data). If a publisher notices that their video isn't working, they can either update their server's MIME type mapping, or just hard code the type in the HTML. Neither is that time consuming / difficult. Moreover, as Adrian suggested, it's probably quite easy to get the big HTTP servers (Apache, IIS, nginx, lighttpd) to add the new extensions (if they haven't already), so this would gradually become less and less of an issue. Best, Zach -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution
Re: [whatwg] Video with MIME type application/octet-stream
On Wed, Sep 1, 2010 at 12:29 PM, Eric Carlson eric.carl...@apple.com wrote: Hard coding the type is only possible if the element uses a source element, @type isn't allowed on audio or video. Why isn't type allowed for video and audio? I know it doesn't strictly make sense (since the tag doesn't have a type per-se), but perhaps it could be an alias for the current item's type, much in the same way src is the current source. It isn't hard to update a server if you control it, but it can be *very* difficult and time consuming if you don't (as is the case with most web developers, I assume). Correct - but being able to manually specify type should be fine for those situations, since that can be written into the HTML itself. Really? Your company specializes in web video and flv files have been around for years, but your own server still isn't configured for it: Thanks for the heads up on this. However, I think this reemphasizes my original point: The Flash platform *isn't* strict about MIME types, so we've never bothered to do anything about it.
Re: [whatwg] HTML5 video source dimensions and bitrate
On Thu, Aug 12, 2010 at 10:03 PM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: As far as I am aware, the adaptive HTTP streaming approaches work with an ordinary HTTP server such as Apache, so do not need anything special on the server. It's more about authoring the right set of resources. The publisher has to create a set of video copies at different bandwidth and maybe even resolutions carefully such that switching between them can happen at specific points. Then he puts them on the server together with a manifest file that links to these resources and states what they provide and when switching can happen. So, wile authoring is challenging, no new server software is used and it also wouldn't listen to any information that the client would want to send. However, the big challenge is to support the switching between resources on the client. And the client needs a gather as much information as possible about the quality of the playback to make the switching decision. Switching is then simply a different HTTP request. It would be nice if we had such switching functionality available for HTML5 video. [It's Friday, so this is a bit more lighthearted than usual] Completely correct. I was thinking of FMS (RTMP with dynamic streaming) and in a momentary lapse of reason [1], wrongly believed that the server would automatically switch bitrates when the client hit a certain threshold of dropped frames. It's at least somewhat ironic given that one of my favorite function names of all time is the Flash 10 Netstream's play2() [2], which is the flux capacitor of Flash bitrate switching: it makes dynamic streaming possible [3]. It would still be nice if the video made dropped frame information available, but that's probably not in the cards. Microsoft Smooth Streaming also uses a SMIL variant. Is yours compatible? Currently no, but perhaps in the future. Yes, that sounds like a sensible approach. It's also what Apple do when they use Live Streaming: they put the m3u8 file into the @src element. It would be nice if that was all we would need to enable adaptive HTTP streaming. It might be good to standardise on a baseline file format for the manifest though - unless we want to support all types of files that people will come up with to do adaptive HTTP streaming. Agreed. Right now we only support MRSS, but SMIL and m3u8 would all work as well. Given that SMIL is a W3C format, it seems to be the logical choice, no? [1] http://en.wikipedia.org/wiki/A_Momentary_Lapse_of_Reason [2] http://help.adobe.com/en_US/AS3LCR/Flash_10.0/flash/net/NetStream.html [3] http://www.imdb.com/title/tt0088763/quotes?qt0416300
Re: [whatwg] HTML5 video source dimensions and bitrate
On Tue, Aug 10, 2010 at 8:26 PM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: It's good to have thought this through. I agree, this isn't a workable solution, since the whole bandwidth switching logic is introduced into the browsers, when in fact it should happen in the media framework with information from the network stack without a need for the Web page to even be aware of this. After all, it's really different versions of the same resource that we are dealing with. That's also what happens in adaptive HTTP streaming solutions of Apple, MS, Adobe and MPEG-4. Absolutely - though these types of solutions are generally out of the reach of the average publisher. Even in that case, the server would (probably) want information about dropped frames from the client. So, to generalise this, I agree there should be a solution such as what Chris Double is suggesting with an additional resource describing what files can be switched between and then JavaScript doing the switching. This could eventually also be introduced into HTML5, if it's done with the same additional resource format for all video formats. In this case, would you still require an additional attribute? We do this currently with SMIL files, which have all the necessary information. In that case the only thing I could imagine be necessary is some sort of flag to indicate that the file listed as the src isn't the actual file to be played, but rather a list of resources.
Re: [whatwg] HTML5 video source dimensions and bitrate
On Mon, Aug 9, 2010 at 8:05 PM, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: Have you looked at the @media attribute? I would suggest to put them there. As that's not currently specified in media queries, you might want to demonstrate how it could work and make a proposal to extend the media queries. I thought about suggesting something like bitrate a bandwidth to media queries. The syntax would look something like: video width=720 source src=video-480.mp4 media=(width:480px) and (min-bitrate:2mbps) source src=video-720.mp4 media=(width:720px) and (min-bitrate:4mbps) source src=video-1080.mp4 media=(width:1080px) and (min-bitrate:20mbps) /video The problem is that the user agent would have to then know what bitrate can be supported between you and that source. User agents don't currently keep track of this information (so far as I'm aware), and even if they did, it's highly variable based on your location, the website you're visiting, etc. Additionally, while media queries do help content producers specify which resource would be selected for a device under ideal circumstances, it doesn't really allow for degradation. However, if you look at what's out there today, it's pretty common to allow users to switch to a degraded experience as needed. I can think of a few solutions that might make sense here: * Allow source sub-elements / groupings and provide a call to switch between elements of a group. Ideally, this would look something like: video width=720px sourcegroup type=video/mp4 media=screen and (device-aspect-ratio: 16/9) source src=video-1080.mp4 meta=width:1920px;height:1080px;bitrate:20mbps; source src=video-720.mp4 meta=width:1280px;height:720px;bitrate:4mbps; /sourcegroup sourcegroup type=video/mp4 media=screen and (device-aspect-ratio: 4/3) source src=video-480.mp4 meta=width:640px;height:480px;bitrate:2mbps; /sourcegroup sourcegroup type=video/ogv source src=video-480.ogv meta=width:640px;height:480px;bitrate:2mbps; source src=video-720.ogv meta=width:1280px;height:720px;bitrate:4mbps; source src=video-1080.ogv meta=width:1920px;height:1080px;bitrate:20mbps; /sourcegroup /video * Have user agents keep track of information about dropped frames. Publishers could then list multiple valid sources in order of rank. When the dropped frame rate increased, either the user agent or a bit of JavaScript could remove that source and re-run the resource selection algorithm. On Mon, Aug 9, 2010 at 8:23 PM, Chris Double chris.dou...@double.co.nz wrote: Since your player is already JavaScript is having a JS object holding the URL, dimensions and bitrate not an option? This is probably what we'll end up doing for now, however, it would be nice to be able to parse this data out of the video tag. How are you working out the current playback rate to decide when to switch to a different bitrate version? Is having an attribute of the media element that contains this information useful? It would be useful for the initial selection. See http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v4/12208/video-delivery-rtmp-streaming#dynamicstreaming. Best, Zach -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution
Re: [whatwg] HTML5 video source dimensions and bitrate
On Tue, Aug 10, 2010 at 12:44 PM, Tab Atkins Jr. jackalm...@gmail.com wrote:
It *appears* that the minimum you're asking for is some way to include
the bitrate/etc. information in the source element so that you can
switch between them in js, right? You're not trying to add something
that will cause changed browser behavior?
Correct - we're just trying to do some JS switching.
In that case, your solution already exists - use the data-*
attributes. source src=foo data-jw-bitrate=2 works just fine. In
existing browsers you can access the attribute with
elem.getAttribute('data-jw-bitrate'), and in future browsers it'll be
elem.data['jw-bitrate'].
Wow - I had no idea this was valid. This is exactly what we were
looking for - thanks!
[whatwg] HTML5 video source dimensions and bitrate
Hello! I'm working on some bandwidth switching logic for the JW Player for HTML5. The way that we've implemented our bandwidth switching logic relies on knowing both the dimensions of the video as well as it's bitrate (for more info on the logic, see our dynamic streaming documentation [1]). MRSS supports this quite nicely, as their media content elements allow for width, height, and bitrate, per our example [2]. As a result, I was wondering if there's some way to attach metadata like the dimensions and bitrate to source elements of HTMLMediaElements in the DOM. I realize that we could wait for metadata to arrive for each source, but it would be nice if it was possible to make an intelligent choice before attempting to play all of them. Best, Zach [1] : http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v4/12208/video-delivery-rtmp-streaming#dynamicstreaming [2] : http://developer.longtailvideo.com/player/testing/files/bitrates.xml -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution
Re: [whatwg] Proposal: @srctype or @type on iframe
It think that offering some hinting makes sense. If the goal is to try and move away from plugins, giving a hint would allow browsers to display appropriate UI elements before the request is completed (in the case of PDF, something like the embedded Scribd controls). -- Zachary Ozer Developer, LongTail Video w: longtailvideo.com • e: z...@longtailvideo.com • p: 212.244.0140 • f: 212.656.1335 JW Player | Bits on the Run | AdSolution On Tue, Jul 13, 2010 at 7:37 AM, Marques Johansson marq...@displague.com wrote: In one of the video related threads someone from Opera said that browsers have a rough time trying to figure out how to intelligently handle video content fetches. I think that a type suggestion, before the Content-type could help a browser use a more optimal fetch behavior. If the type of a resource is predefined then the browser can do a partial request to fetch the meta data of the alleged file format and in the case of a PDF perhaps the first few pages of content, leaving the rest to be loaded on demand as the content is scrolled / paginated. I don't know if this is really possible with the structure of PDF documents but there are certainly some media types for which this could be applied. I would think 'size' may be a partner attribute for hinting fetch behavior. The browser doesn't want to have to make more than one request unless there is good reason to do so. If the entire PDF is only one page long then a meta data prefetch and continued fetch would be wasteful. However if the filesize was 26mb then a meta data prefetch could indicate where partial fetches should be made to acquire the 1mb of content that will actually need to be displayed to the user immediately. If type is a reasonable attribute to add to all of these elements I would think size should also be considered. It wouldn't have to be accurate, Content-length would take priority, but this attribute would serve for hinting purposes. Perhaps some alternative could be found with a DOM solution. On Tue, Jul 13, 2010 at 3:45 AM, Gordon P. Hemsley gphems...@gmail.com wrote: On Tue, Jul 13, 2010 at 3:26 AM, Boris Zbarsky bzbar...@mit.edu wrote: On 7/12/10 11:31 PM, Gordon P. Hemsley wrote: The particular use case that prompted me to think about this is including a PDF via iframe. In Firefox (last I checked), one is required to install a separate add-on in order to support in-browser display of PDF files on Mac OS X, since there is no native or integrated Adobe Reader support available. I'm pretty sure you can install the Adobe Reader plug-in on Mac if you want to. Perhaps now, but that wasn't always the case—at least not for Firefox. I admit that my experience is somewhat outdated. Installing the third-party PDF viewer add-on is one of the first things I did, in a set it and forget it kind of way. (Plus, I'm still on Tiger.) But, again, the PDF example was just one possible use case. I'm sure there are plenty of other file types that cause similar situations, including the TIFF issue that I mentioned. Without the add-on, the user will be prompted to download the PDF file Which is exactly what would happen for a type=application/pdf iframe, no? Silently not showing the content doesn't seem acceptable. -Boris Well, the idea is to have the browser operate more intelligently than that. The page in the iframe is (by definition) not the primary document that the user is trying to load, so it shouldn't have the power steal the user's attention immediately upon page load. It would be very disorienting, and would likely cause the user to lose their train of thought. I was thinking more along the lines of Flashblock does or what happens when the window in an iframe can't load: The content would be replaced somehow by a message and a button/link to allow the user to manually download the contents of the iframe, if they so choose. It shouldn't make that decision for the user, as it's not the user's fault that their browser does not support the format of some ancillary document. At least, that's how I see it. Gordon -- Gordon P. Hemsley m...@gphemsley.org http://gphemsley.org/ • http://gphemsley.org/blog/ http://sasha.sourceforge.net/ • http://www.yoursasha.com/
