Re: [whatwg] What is the purpose of timeupdate?
On Thu, 05 Nov 2009 21:11:15 +0100, Andrew Scherkus scher...@chromium.org wrote: On Thu, Nov 5, 2009 at 6:10 AM, Brian Campbell brian.p.campb...@dartmouth.edu wrote: On Nov 5, 2009, at 1:17 AM, Andrew Scherkus wrote: On Fri, Oct 30, 2009 at 10:18 PM, Brian Campbell brian.p.campb...@dartmouth.edu wrote: As a multimedia developer, I am wondering about the purpose of the timeupdate event on media elements. On first glance, it would appear that this event would be useful for synchronizing animations, bullets, captions, UI, and the like. The spec specifies a rate of 4 to 66 Hz for these events. The high end of this (30 or more Hz) is pretty reasonable for displaying things in sync with the video. The low end, however, 4 Hz, is far too slow for most types of synchronization; everything feels laggy at this frequency. From my testing on a two year old MacBook Pro, Firefox is giving me about 25 timeupdate events per second, while Safari and Chrome are giving me the bare minimum, of 4 timeupdate events per second. At 4 timeupdate events per second, it isn't all that useful. I can replace it with setInterval, at whatever rate I want, query the time, and get the synchronization I need, but that makes the timeupdate event seem to be redundant. At 25 timeupdate events per second, it is reasonably useful, and can be used to synchronize various things to the video. So, I'm wondering if there's a purpose for the timeupdate event that I'm missing. If it is intended for any sort of synchronization with the video, I think it should be improved to give better guarantees on the interval between updates, or just dropped from the spec; it's not useful enough in its current form. To improve it, the maximum interval between updates could be reduced to about 40 ms, or perhaps the interval could be made settable so the author could control how often they want to get the event. -- Brian I believe it's a convenience over using setTimeout/setInterval + polling to determine whether playback has progressed ie., for rendering your own playback progress bar. I've also seen it been used as a signal to copy frames into canvas on Firefox, however if timeupdate frequency != fps of video you either miss frames or paint too much. I don't think timeupdate today is very useful for doing anything beyond a progress bar or other simple synchronized animations. Right. That's what I figured the point is; I just wanted to check to make sure I wasn't missing something. As implemented by Safari and Chrome (which is the minimum rate allowed by the spec), it's not really useful for that purpose, as 4 updates per second makes any sort of synchronization feel jerky and laggy. If it were done at the frame rate of the video, or perhaps with a minimum of 25 frames per second, it would be much more useful. Even at a minimum of 15 frames per second, you would still be able to get some sorts of useful synchronization, though animations synchronized wouldn't feel as smooth as they could. Would something like video firing events for every frame rendered help you out? This would help also fix the canvas over/under painting issue and improve synchronization. Yes, this would be considerably better than what is currently specced. -- Brian I'll see if we can do something for WebKit based browsers, because today it literally is hardcoded to 250ms for all ports. http://trac.webkit.org/browser/trunk/WebCore/html/HTMLMediaElement.cpp#L1254 Maybe we'll end up firing events based on frame updates for video, and something arbitrary for audio (as it is today). Brian, since Firefox is doing what you proposed -- can you think of any other issues with its current implementation? What about for audio files? Thanks, Andrew We've considered firing it for each frame, but there is one problem. If people expect that it fires once per frame they will probably write scripts which do frame-based animations by moving things n pixels per frame or similar. Some animations are just easier to do this way, so there's no reason to think that people won't do it. This will break horribly if a browser is ever forced to drop a frame, which is going to happen on slower machines. In balance this may or may not be a risk worth taking. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Microdata feedback
On Wed, 14 Oct 2009 13:53:46 +0200, Ian Hickson i...@hixie.ch wrote: On Fri, 21 Aug 2009, Philip Jägenstedt wrote: Shouldn't namedItem [6] be namedItems? Code like .namedItem().item(0) would be quite confusing. [6] http://www.whatwg.org/specs/web-apps/current-work/multipage/infrastructure.html#dom-htmlpropertycollection-nameditem I don't understand what this is referring to. I was incorrectly under the impressions that .namedItem on other collections always returned a single element and arguing that since HTMLPropertyCollection.namedItem always returns a PropertyNodeList namedItems in plural would make more sense. Now I see that some other namedItem methods aren't as simple as I'd thought, so I'm not sure what to make of it. Is there a reason why HTMLPropertyCollection.namedItem unlike some other collections' .namedItem don't return an element if there is only 1 element in the collection at the time the method is called? Perhaps this is legacy quirks that we don't want to replicate? On Tue, 25 Aug 2009, Philip Jägenstedt wrote: There's something like an inverse relationship between simplicity of the syntax and complexity of the resulting markup, the best balance point isn't clear (to me at least). Perhaps option 3 is better, never allowing item+itemprop on the same element. That would preclude being able to make trees. Given that flat items like vcard/vevent are likely to be the most common use case I think we should optimize for that. Child items can be created by using a predefined item property: itemprop=com.example.childtype item. The value of that property would then be the first item in tree-order (or all items in the subtree, not sure). This way, items would have better copy-paste resilience as the whole item element could be made into a top-level item simply by moving it, without meddling with the itemprop. That sounds kinda confusing... More confusing than item+itemprop on the same element? In many cases the property value is the contained text, having it be the contained item node(s) doesn't seem much stranger. Based on the studies Google did, I'm not convinced that people will find the nesting that complicated. IMHO the proposal above is more confusing, too. I'm not sure this is solving a problem that needs solving. If the parent-item (com.example.blog) doesn't know what the child-items are, it would simply use itemprop=item. I don't understand this at all. This was an attempt to have anonymous sub-items. Re-thinking this, perhaps a better solution would be to have each item behave in much the same way that the document itself does. That is, simply add items in the subtree without using itemprop and access them with .getItems(itemType) on the outer item. How would you do things like agent in the vEvent vocabulary? Comparing the current model with a DOM tree, it seems odd in that a property could be an item. It would be like an element attribute being another element: outer foo=inner//. That kind of thing could just as well be outerfooinner//foo/outer, outerinner type=foo//outer or even outerinner//outer if the relationship between the elements is clear just from the fact that they have a parent-child relationship (usually the case). Microdata's datamodel is more similar to JSON's than XML's. It's only in the case where both itemprop and item have a type that an extra level of nesting will be needed and I expect that to be the exception. Changing the model to something more DOM-tree-like is probably going to be easier to understand for many web developers. I dunno. People didn't seem to have much trouble getting it once we used itemscope= rather than just item=. People understand the JSON datamodel pretty well, why would this be different? After http://blog.whatwg.org/usability-testing-html5, the recent syntax changes, the improved DOM API and the passage of time I'm not very worried about the things I was worrying about above. If there's any specific point that seems valid after another review I'll send separate feedback on it. Thanks for all the other fixes! -- Philip Jägenstedt Opera Software
Re: [whatwg] Restarting the media element resource fetch algorithm after load event
On Wed, 14 Oct 2009 12:08:19 +0200, Ian Hickson i...@hixie.ch wrote: On Fri, 9 Oct 2009, Philip Jägenstedt wrote: Since we're going to contradict the progress events spec anyway, I would suggest dropping all 'loadend' events. They're just not very useful. I've left it in the other cases, since, well, Progress Events says to. But I'd be happy to drop loadend in all cases (including other Progress Events cases) if that makes sense. On Wed, 14 Oct 2009, Robert O'Callahan wrote: We have the same issue, unsurprisingly. Currently our progress events are not very useful because we report the download position (although we do suppress progress events while we read metadata from the end of the file). According to the spec, however, progress events should report the *amount* of data downloaded, not the position within the resource. That's a bit more useful, although it's still unclear what to do when data is discarded from the cache and then re-downloaded. Ok, I've switched them back to regular Event events, though using the same event names. We added loadend just to comply with Progress Events. Now that we fire simple events instead, please drop loadend again as it serves no purpose at all. I doubt any browser has yet shipped an implementation firing loadend, correct me if I'm wrong. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Restarting the media element resource fetch algorithm after load event
On Wed, 14 Oct 2009 12:58:17 +0200, Anne van Kesteren ann...@opera.com wrote: On Wed, 14 Oct 2009 12:51:09 +0200, Philip Jägenstedt phil...@opera.com wrote: We added loadend just to comply with Progress Events. Now that we fire simple events instead, please drop loadend again as it serves no purpose at all. I doubt any browser has yet shipped an implementation firing loadend, correct me if I'm wrong. The loadend event is dispatched for XMLHttpRequest and that might be in Gecko I suppose. I'm not particularly attached to it though. I just added it for consistency with the media elements and the progress events specification. I'm only talking about dropping it for media elements. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Restarting the media element resource fetch algorithm after load event
On Thu, 08 Oct 2009 23:08:32 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Fri, Oct 9, 2009 at 1:32 AM, Philip Jägenstedt phil...@opera.com wrote: The spec notes that Some resources, e.g. streaming Web radio, can never reach the NETWORK_LOADED state. In my understanding, you mustn't go to NETWORK_LOADED if you can't guarantee that the resource will remain in cache. Browsers with clever caching or small caches simply won't send a load event most of the time. Right, HTML5 allows Gecko to simply never enter the NETWORK_LOADED state and send a load event. That would be the simplest thing for us to do, and I think the best in terms of letting us do intelligent caching. But I'd prefer not to do it alone. No objection to dropping the event and implementing as such here. Aesthetically, however, I think it would be strange to not have the load event. If you mean because of the analogy with the load events of other elements, Aesthetics is not a serious argument. More importantly, the progress events spec [1] requires that exactly one of error/abort/load be fired followed by loadend. Dropping load and loadend would be a willful violations of that spec. In my opinion, the progress events spec should be the one to change. [1] http://www.w3.org/TR/progress-events/ -- Philip Jägenstedt Opera Software
Re: [whatwg] Restarting the media element resource fetch algorithm after load event
On Fri, 09 Oct 2009 08:42:12 +0200, Simon Pieters sim...@opera.com wrote: On Thu, 08 Oct 2009 23:19:28 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Fri, Oct 9, 2009 at 6:42 AM, Eric Carlson eric.carl...@apple.com wrote: I am not worried about the aesthetics of not having the event. I am somewhat concerned about existing content that uses it (including many of the WebKit layout tests :-( ), but I think we will be better off in the long run if we get rid of the event and network state now. Me too. I'm game if you are! So I propose: 1) Remove the NETWORK_LOADED state and load and loadend events from media elements. 'loadend' also fires after 'abort' and (when using the src attribute) 'error'. Should that stay as is, or also be removed? Since we're going to contradict the progress events spec anyway, I would suggest dropping all 'loadend' events. They're just not very useful. The resource fetch algorithm simply never transitions from step 2 to step 3. 2) Modify Gecko and Webkit accordingly. If we do part 2, which I think is already permitted by the spec, then authors will stop depending on load whether or not we get consensus for altering the spec. Rob -- Philip Jägenstedt Opera Software
Re: [whatwg] Restarting the media element resource fetch algorithm after load event
On Thu, 08 Oct 2009 22:53:45 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Fri, Oct 9, 2009 at 7:31 AM, Philip Jägenstedt phil...@opera.com wrote: I wouldn't be particularly opposed to dropping the load event, unless there's a use case for the guarantee that the resource won't be evicted from cache as long as the page is loaded. If there is a need for that, then I think we should add some kind of pinning API, although I'm not sure how it would avoid the problem of greedy authors pinning resources unnecessarily. But we'd have to consider the actual use cases. I'd say that the autobuffer attribute is exactly that API, which gives both author control and browser liberty to ignore it when necessary. -- Philip Jägenstedt Opera Software
Re: [whatwg] Restarting the media element resource fetch algorithm after load event
On Thu, 08 Oct 2009 12:10:01 +0200, Robert O'Callahan rob...@ocallahan.org wrote: http://www.whatwg.org/specs/web-apps/current-work/#loading-the-media-resource In the resource fetch algorithm, after we reach the NETWORK_LOADED state in step 3 which indicates that all the data we need to play the resource is now available locally, we end the resource fetch algorithm. However, in Gecko we have a media cache which might discard blocks of media data after we've reached the NETWORK_LOADED state (to make room for data for other loading resources). This means we might have to start fetching the resource again later. The spec does not seem to allow for this. Do we need to change our behavior, or does the spec need to change to accommodate our behavior? I'd prefer not to change our behavior since I think to follow the spec we'd need to pin the entire resource permanently in the cache after we reached NETWORK_LOADED, which could be highly suboptimal in some situations. The spec notes that Some resources, e.g. streaming Web radio, can never reach the NETWORK_LOADED state. In my understanding, you mustn't go to NETWORK_LOADED if you can't guarantee that the resource will remain in cache. Browsers with clever caching or small caches simply won't send a load event most of the time. Another issue is that it's not completely clear to me what is meant by While the user agent might still need network access to obtain parts of the media resourcehttp://www.whatwg.org/specs/web-apps/current-work/#media-resource... What if there is data in the resource that we don't need in order to play through normally, but which might be needed in some special situations (e.g., enabling subtitles, or seeking using an index), and we optimize to not load that data unless/until we need it? In that case would we never reach NETWORK_LOADED? As I understand it, NETWORK_LOADED means that all bytes of the resource have been loaded, regardless of whether they will be used or not. Are there any formats that would actually allow not downloading parts of the resource in a meaningful way? Subtitles and indexes are too small to bother, and multiplexed audio/video tracks can hardly be skipped without zillions of HTTP Range requests. It seems to me that kind of thing would have to be done either with a server side media fragment request (using the 'track' dimension) or with an external audio/video track somehow synced to the master track (much like external subtitles). In general NETWORK_LOADED and the load event seem rather useless and dangerous IMHO. If you're playing a resource that doesn't fit in your cache then you'll certainly never reach NETWORK_LOADED, and since authors can't know the cache size they can never rely on load firing. And if you allow the cache discarding behavior I described above, authors can't rely on data actually being present locally even after load has fired. I suspect many authors will make invalid assumptions about load being sure to fire and about what load means if it does fire. Does anyone have any use cases that load actually solves? I agree, sites that depend on the load event sites will likely break randomly for file sizes that usually barely fit into the cache of the browser they were tested with. If browsers are conservative with bandwidth and only send the load event when it's true, I think we will have less of a problem however. Note that the load event isn't strictly needed, waiting for a progress event with loaded==total would achieve the same thing. Aesthetically, however, I think it would be strange to not have the load event. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Restarting the media element resource fetch algorithm after load event
On Thu, 08 Oct 2009 19:42:14 +0200, Eric Carlson eric.carl...@apple.com wrote: On Oct 8, 2009, at 5:32 AM, Philip Jägenstedt wrote: On Thu, 08 Oct 2009 12:10:01 +0200, Robert O'Callahan rob...@ocallahan.org wrote: Another issue is that it's not completely clear to me what is meant by While the user agent might still need network access to obtain parts of the media resourcehttp://www.whatwg.org/specs/web-apps/current-work/#media-resource ... What if there is data in the resource that we don't need in order to play through normally, but which might be needed in some special situations (e.g., enabling subtitles, or seeking using an index), and we optimize to not load that data unless/until we need it? In that case would we never reach NETWORK_LOADED? As I understand it, NETWORK_LOADED means that all bytes of the resource have been loaded, regardless of whether they will be used or not. Are there any formats that would actually allow not downloading parts of the resource in a meaningful way? Yes. A disabled track in an MPEG-4 or QuickTime file is not rendered so the data is not used when presenting the movie. Media data for an enabled but invisible video track (eg. size 0x0, or not within the visible region) or an enabled but muted audio track isn't technically needed for the presentation either. The question was rather of such tracks can be multiplexed in a way that make it possible to skip them efficiently. As per below, probably yes. Subtitles and indexes are too small to bother, and multiplexed audio/ video tracks can hardly be skipped without zillions of HTTP Range requests. It seems to me that kind of thing would have to be done either with a server side media fragment request (using the 'track' dimension) or with an external audio/video track somehow synced to the master track (much like external subtitles). I don't agree that this is necessarily best done on a server. Some file formats include tables with the location of every sample, so a media engine that uses range requests anyway can easily read just the data needed. It might be wise for such an engine to optimize the size of chunks read from the server, but that is an implementation detail. Also remember that multiplexed is a relative term, different chunking/interleaving schemes make sense for different media types and use cases so not all multiplexed files interleave data in small chunks. Right, if the chunks are large enough (at least 100s of KB I suppose) it might be worthwhile using range requests. In general NETWORK_LOADED and the load event seem rather useless and dangerous IMHO. If you're playing a resource that doesn't fit in your cache then you'll certainly never reach NETWORK_LOADED, and since authors can't know the cache size they can never rely on load firing. And if you allow the cache discarding behavior I described above, authors can't rely on data actually being present locally even after load has fired. If data can be evicted from the cache you can never reach NETWORK_LOADED because Network connectivity could be lost without affecting the media playback. I suspect many authors will make invalid assumptions about load being sure to fire and about what load means if it does fire. Does anyone have any use cases that load actually solves? I also agree that the 'load' event and the NETWORK_LOADED state are not terribly useful and will likely cause a great deal of confusion for developers. We have have seen a number of cases where experienced web developers have used the 'load' event when they should have used the 'canplaythough', and I fear that this will be a common mistake. I agree, sites that depend on the load event sites will likely break randomly for file sizes that usually barely fit into the cache of the browser they were tested with. If browsers are conservative with bandwidth and only send the load event when it's true, I think we will have less of a problem however. I don't agree that it will be any less of a problem if browsers are conservative, users will still not *ever* be able to depend on the 'load' event firing (except perhaps for local files). Precisely, if browsers don't fire 'load' until the last data is loaded when playback is closing in on completion, people can't accidentally use it instead of 'canplay'. Still, for local files, very small files or in combination with autobuffer the problem will still be the same. Note that the load event isn't strictly needed, waiting for a progress event with loaded==total would achieve the same thing. Actually, a progress event with loaded==total tells you even less than the 'load' event because it doesn't guarantee that the data won't be evicted from the cache. Good point. Aesthetically, however, I think it would be strange to not have the load event. I am not worried about the aesthetics of not having the event. I am somewhat concerned about existing content that uses
Re: [whatwg] cloneNode and HTML elements
On Thu, 10 Sep 2009 07:26:06 +0200, Robert O'Callahan rob...@ocallahan.org wrote: If you call cloneNode on a media element, the state of the resulting media element seems unspecified. Should it be playing the same media resource at the same current time as the original? Similar questions arise when you clone form elements; is the state that's not visible in the DOM cloned? Who should be responsible for defining this? Rob In my opinion, expecting the cloned MediaElement to retain all of its state is more work to implement and not really worth it. I don't know where it should be defined, but think the best behavior would be to just clone the content attributes. In other words, you'd get a media element in state NETWORK_EMPTY/HAVE_NOTHING which would start resource selection when you insert it into a document. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] cloneNode and HTML elements
On Thu, 10 Sep 2009 11:15:06 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Thu, Sep 10, 2009 at 7:41 PM, Maciej Stachowiak m...@apple.com wrote: On Sep 9, 2009, at 10:26 PM, Robert O'Callahan wrote: If you call cloneNode on a media element, the state of the resulting media element seems unspecified. Should it be playing the same media resource at the same current time as the original? Similar questions arise when you clone form elements; is the state that's not visible in the DOM cloned? Who should be responsible for defining this? Does cloneNode require copying any state besides an element's qualified name, attributes and DOM children? The definition in DOM3Core doesn't say anything else should be copied. Is form control state (such as set of items selected in a list multiple) copied? Reference for cloneNode: http://www.w3.org/TR/DOM-Level-3-Core/core.html#ID-3A0ED0A4 My assumption based on the spec is that no element-specific internal state should be copied, the cloning should only be DOM-wise. It's not obvious to me that DOM 3 Core's silence means nothing else is copied, since non-DOM state is outside its scope. I wonder if authors would be surprised if the non-DOM state is not copied. I would suggest only copying state when it is needed for web compatibility or when there's a compelling use case. In the case of HTMLMediaElement the complexity of setting up a second decoding pipeline and trying to get it into the same state as the first must also be taken into account. A true clone may simply not be possible in all implementations. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] createImageData should take unsigned long
On Fri, 04 Sep 2009 02:44:25 +0200, Oliver Hunt oli...@apple.com wrote: On Sep 3, 2009, at 4:50 PM, Robert O'Callahan wrote: On Fri, Sep 4, 2009 at 4:48 AM, Oliver Hunt oli...@apple.com wrote: On Sep 3, 2009, at 4:54 AM, Ian Hickson wrote: Yeah, that seems likely, since none of you implemented the higher-DPI ImageData in your first versions. :-( WebKit's implementation has always worked with high dpi backing stores and follows the spec accordingly. Under what circumstances do you use more than one device pixel per CSS pixel? Does it require the user to turn on UI scaling on Mac? Regardless, I bet that most people using Webkit to write scripts using getImageData still get it wrong, because they have normal screens. Implementing high-res backing store in more browsers won't solve this problem, not until the average developer has a high-dpi screen. But I repeat myself. Indeed -- i was merely commenting that there was actually a correct implementation -- i am still of the opinion that exposing image data was a bad thing and that a filtering API would have been superior. Oh well, the past is the past and we must now live with it. :-/ How can one use this implementation? In my tests getImageData(0,0,w,h) simply returns a wxh ImageData object. It would be interesting to enable this non-1:1 backing store to see if sites break or not. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] createImageData should take unsigned long
On Thu, 03 Sep 2009 13:54:03 +0200, Ian Hickson i...@hixie.ch wrote: On Mon, 31 Aug 2009, Philip J�genstedt wrote: On Mon, 31 Aug 2009 08:08:05 +0200, Ian Hickson i...@hixie.ch wrote: On Mon, 24 Aug 2009, Philip J�genstedt wrote: As far as I can see there's no good reason why createImageData should take a float as input rather than unsigned long. Having it as float creates the odd situation where (0.1, 0.1) gives a 1x1 ImageData while (10.1, 10.1) gives a 10x10 or 11x11 depening on if you ceil or round the input (not defined). Unless there's a compelling reason to allow something like (0.1, 0.1) I suggest changing the type and leaving the float-unsigned conversion to WebIDL. Twenty years from now, when we're using 960dpi screens, 1 CSS pixel might well map to ten device pixels reliably, such that people will want sub-CSS-pixel-level accuracy in their calls to createImageData(). I get the impression this has all been discussed before. Still, it seems unlikely that any browser will ever be able to switch to anything but a 1:1 CSS pixel:device pixel ratio, as that would break all existing pages assuming that getImageData(0, 0, 100, 100) returns a 100x100 bitmap (because assuming that is much easier, unless you read the spec carefully you're unlikely to know it could ever be any different). Yeah, that seems likely, since none of you implemented the higher-DPI ImageData in your first versions. :-( I expect we'll introduce a new API that actually works once there is a browser vendor actually interested in supporting higher-DPI canvases. I wasn't involved then, but I can only presume that there was no perceived benefit of high-DPI ImageData since you can get high-quality rendering just as well with techniques that don't rely on the canvas being higher resolution than the display device. In any event, judging by existing implementations, the behavior of createImageData(w, h) isn't as clear as it needs to be: http://software.hixie.ch/utilities/js/live-dom-viewer/saved/223 Firefox: log: ctx.createImageData(-1.1,1) = [Exception... log: ctx.createImageData(-1,1) = [Exception... log: ctx.createImageData(-0.1,1) = [Exception... log: ctx.createImageData(0,1) = [Exception... log: ctx.createImageData(0.1,1) = [Exception... log: ctx.createImageData(1,1) = 1x1 log: ctx.createImageData(1.1,1) = 1x1 Safari/Chrome: log: ctx.createImageData(-1.1,1) = 1x1 log: ctx.createImageData(-1,1) = 1x1 log: ctx.createImageData(-0.1,1) = 1x1 log: ctx.createImageData(0,1) = 1x1 log: ctx.createImageData(0.1,1) = 1x1 log: ctx.createImageData(1,1) = 1x1 log: ctx.createImageData(1.1,1) = 2x1 My interpretation of the spec: log: ctx.createImageData(-1.1,1) = 1x1 log: ctx.createImageData(-1,1) = 1x1 log: ctx.createImageData(-0.1,1) = 1x1 log: ctx.createImageData(0,1) = INDEX_SIZE_ERR log: ctx.createImageData(0.1,1) = 1x1 log: ctx.createImageData(1,1) = 1x1 log: ctx.createImageData(1.1,1) = 1x1 The spec doesn't say what size the ImageData objects should be in these cases. Your interpretation is correct insofar as the exception is concerned, though. If the spec doesn't say to round rather than ceil, we're bound to have subtle compatibility bugs on this. The spec says it doesn't matter so long as you're consistent. On Mon, 31 Aug 2009, Robert O'Callahan wrote: Still, it seems unlikely that any browser will ever be able to switch to anything but a 1:1 CSS pixel:device pixel ratio, as that would break all existing pages assuming that getImageData(0, 0, 100, 100) returns a 100x100 bitmap (because assuming that is much easier, unless you read the spec carefully you're unlikely to know it could ever be any different). I agree, but Ian doesn't. It's not so much that I disagree so much as that there is no point fixing this now, since whatever new API we introduce today will just end up broken in the exact some way as the existing API. The fix is not to introduce a new API that can handle high-DPI ImageData, but rather to make the spec reflect the reality that high-DPI ImageData implementations won't be possible with this version of the API. That would include, among other things, specifying that getImageData(0, 0, w, h) returns a wxh ImageData object, removing createImageData(ImageData) and making createImageData(w, h) take unsigned long and return a wxh ImageData object. In other words, aligning with what implementations already do (and will continue to do for compatibility reasons). -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] createImageData should take unsigned long
On Mon, 31 Aug 2009 08:08:05 +0200, Ian Hickson i...@hixie.ch wrote: On Mon, 24 Aug 2009, Philip Jägenstedt wrote: As far as I can see there's no good reason why createImageData should take a float as input rather than unsigned long. Having it as float creates the odd situation where (0.1, 0.1) gives a 1x1 ImageData while (10.1, 10.1) gives a 10x10 or 11x11 depening on if you ceil or round the input (not defined). Unless there's a compelling reason to allow something like (0.1, 0.1) I suggest changing the type and leaving the float-unsigned conversion to WebIDL. Twenty years from now, when we're using 960dpi screens, 1 CSS pixel might well map to ten device pixels reliably, such that people will want sub-CSS-pixel-level accuracy in their calls to createImageData(). I get the impression this has all been discussed before. Still, it seems unlikely that any browser will ever be able to switch to anything but a 1:1 CSS pixel:device pixel ratio, as that would break all existing pages assuming that getImageData(0, 0, 100, 100) returns a 100x100 bitmap (because assuming that is much easier, unless you read the spec carefully you're unlikely to know it could ever be any different). I don't doubt that high DPI screens will happen, but when it does browsers are more likely to provide an extra flag like getImageData(..., useDevicePixels) or another opt-in method in order to stay compatible with existing content. Another option for the page author is simply creating a 1000x1000 canvas and setting its CSS width/height to 100x100 (assuming the CSS pixel:device pixel ratio can be found via script). In any event, judging by existing implementations, the behavior of createImageData(w, h) isn't as clear as it needs to be: http://software.hixie.ch/utilities/js/live-dom-viewer/saved/223 Firefox: log: ctx.createImageData(-1.1,1) = [Exception... log: ctx.createImageData(-1,1) = [Exception... log: ctx.createImageData(-0.1,1) = [Exception... log: ctx.createImageData(0,1) = [Exception... log: ctx.createImageData(0.1,1) = [Exception... log: ctx.createImageData(1,1) = 1x1 log: ctx.createImageData(1.1,1) = 1x1 Safari/Chrome: log: ctx.createImageData(-1.1,1) = 1x1 log: ctx.createImageData(-1,1) = 1x1 log: ctx.createImageData(-0.1,1) = 1x1 log: ctx.createImageData(0,1) = 1x1 log: ctx.createImageData(0.1,1) = 1x1 log: ctx.createImageData(1,1) = 1x1 log: ctx.createImageData(1.1,1) = 2x1 My interpretation of the spec: log: ctx.createImageData(-1.1,1) = 1x1 log: ctx.createImageData(-1,1) = 1x1 log: ctx.createImageData(-0.1,1) = 1x1 log: ctx.createImageData(0,1) = INDEX_SIZE_ERR log: ctx.createImageData(0.1,1) = 1x1 log: ctx.createImageData(1,1) = 1x1 log: ctx.createImageData(1.1,1) = 1x1 If the spec doesn't say to round rather than ceil, we're bound to have subtle compatibility bugs on this. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Microdata
On Tue, 25 Aug 2009 00:29:06 +0200, Ian Hickson i...@hixie.ch wrote: On Mon, 24 Aug 2009, Philip Jägenstedt wrote: I've found two related things that are a bit problematic. First, because itemprops are only associated with ancestor item elements or via the subject attribute, it's always necessary to find or create a separate element for the item. This leads to more convoluted markup for small items, so it would be nice if the first item and itemprop could be on the same element when it makes sense: p item=vevent itemprop=description Concert at span itemprop=dtstart19:00/span at span itemprop=locationthe beach/span. /p rather than p item=vevent span itemprop=description Concert at span itemprop=dtstart19:00/span at span itemprop=locationthe beach/span. /span /p As specced now, having itemprop= and item= on the same element implies that the value of the property is an item rooted at this element. Not supporting the above was intentional, to keep the mental model of the markup very simple, rather than having shortcuts. (RDFa has lots of shortcuts and it ended up being very difficult to keep the mental model straight.) There's something like an inverse relationship between simplicity of the syntax and complexity of the resulting markup, the best balance point isn't clear (to me at least). Perhaps option 3 is better, never allowing item+itemprop on the same element. Second, because composite items can only be made by adding item and itemprop to the same element, the embedded item has to know that it has a parent and what itemprop it should use to describe itself. James gave the example of something like planet where each article could be a com.example.blog item and within each article there could be any arbitrary author-supplied microdata [1]. I also feel that the item+itemprop syntax for composite items is one of the least intuitive parts of the current spec. It's easy to get confused about what the type of the item vs the itemprop should be and which item the itemprop actually belongs to. Fair points. Given that flat items like vcard/vevent are likely to be the most common use case I think we should optimize for that. Child items can be created by using a predefined item property: itemprop=com.example.childtype item. Ok... The value of that property would then be the first item in tree-order (or all items in the subtree, not sure). This way, items would have better copy-paste resilience as the whole item element could be made into a top-level item simply by moving it, without meddling with the itemprop. That sounds kinda confusing... More confusing than item+itemprop on the same element? In many cases the property value is the contained text, having it be the contained item node(s) doesn't seem much stranger. If the parent-item (com.example.blog) doesn't know what the child-items are, it would simply use itemprop=item. I don't understand this at all. This was an attempt to have anonymous sub-items. Re-thinking this, perhaps a better solution would be to have each item behave in much the same way that the document itself does. That is, simply add items in the subtree without using itemprop and access them with .getItems(itemType) on the outer item. Comparing the current model with a DOM tree, it seems odd in the a property could be an item. It would be like an element attribute being another element: outer foo=inner//. That kind of thing could just as well be outerfooinner//foo/outer, outerinner type=foo//outer or even outerinner//outer if the relationship between the elements is clear just from the fact that they have a parent-child relationship (usually the case). All examples of nested items in the spec are on the form p itemprop=subtype item These would be replaced with p item=subtype It's only in the case where both itemprop and item have a type that an extra level of nesting will be needed and I expect that to be the exception. Changing the model to something more DOM-tree-like is probably going to be easier to understand for many web developers. It would also fix the problem in my other mail where it's a bit tricky to determine via the DOM API whether a property is a string or an item. When on the topic of the DOM API, document.getItems(outer)[0].getItems(inner)[0] would be so much clearer than what we currently have. Example: p item=vcard itemprop=n item My name is span itemprop=given-namePhilip/span span itemprop=family-nameJägenstedt/span. /p I don't understand what this maps to at all. The same as p item=vcard span itemprop=n item My name is span itemprop=given-namePhilip/span span itemprop=family-nameJägenstedt/span. /span /p Unless I've misunderstood the n in vcard (there's no example in the spec). But let's move on. I'll admit that my examples are a bit simple, but the main point in my opinion is to make item+itemprop less confusing. There are basically only 3
Re: [whatwg] Microdata
On Tue, 25 Aug 2009 09:43:58 +0200, Philip Jägenstedt phil...@opera.com wrote: On Tue, 25 Aug 2009 00:29:06 +0200, Ian Hickson i...@hixie.ch wrote: On Mon, 24 Aug 2009, Philip Jägenstedt wrote: I've found two related things that are a bit problematic. First, because itemprops are only associated with ancestor item elements or via the subject attribute, it's always necessary to find or create a separate element for the item. This leads to more convoluted markup for small items, so it would be nice if the first item and itemprop could be on the same element when it makes sense: p item=vevent itemprop=description Concert at span itemprop=dtstart19:00/span at span itemprop=locationthe beach/span. /p rather than p item=vevent span itemprop=description Concert at span itemprop=dtstart19:00/span at span itemprop=locationthe beach/span. /span /p As specced now, having itemprop= and item= on the same element implies that the value of the property is an item rooted at this element. Not supporting the above was intentional, to keep the mental model of the markup very simple, rather than having shortcuts. (RDFa has lots of shortcuts and it ended up being very difficult to keep the mental model straight.) There's something like an inverse relationship between simplicity of the syntax and complexity of the resulting markup, the best balance point isn't clear (to me at least). Perhaps option 3 is better, never allowing item+itemprop on the same element. Second, because composite items can only be made by adding item and itemprop to the same element, the embedded item has to know that it has a parent and what itemprop it should use to describe itself. James gave the example of something like planet where each article could be a com.example.blog item and within each article there could be any arbitrary author-supplied microdata [1]. I also feel that the item+itemprop syntax for composite items is one of the least intuitive parts of the current spec. It's easy to get confused about what the type of the item vs the itemprop should be and which item the itemprop actually belongs to. Fair points. Given that flat items like vcard/vevent are likely to be the most common use case I think we should optimize for that. Child items can be created by using a predefined item property: itemprop=com.example.childtype item. Ok... The value of that property would then be the first item in tree-order (or all items in the subtree, not sure). This way, items would have better copy-paste resilience as the whole item element could be made into a top-level item simply by moving it, without meddling with the itemprop. That sounds kinda confusing... More confusing than item+itemprop on the same element? In many cases the property value is the contained text, having it be the contained item node(s) doesn't seem much stranger. If the parent-item (com.example.blog) doesn't know what the child-items are, it would simply use itemprop=item. I don't understand this at all. This was an attempt to have anonymous sub-items. Re-thinking this, perhaps a better solution would be to have each item behave in much the same way that the document itself does. That is, simply add items in the subtree without using itemprop and access them with .getItems(itemType) on the outer item. Comparing the current model with a DOM tree, it seems odd in the a property could be an item. It would be like an element attribute being another element: outer foo=inner//. That kind of thing could just as well be outerfooinner//foo/outer, outerinner type=foo//outer or even outerinner//outer if the relationship between the elements is clear just from the fact that they have a parent-child relationship (usually the case). All examples of nested items in the spec are on the form p itemprop=subtype item These would be replaced with p item=subtype It's only in the case where both itemprop and item have a type that an extra level of nesting will be needed and I expect that to be the exception. Changing the model to something more DOM-tree-like is probably going to be easier to understand for many web developers. It would also fix the problem in my other mail where it's a bit tricky to determine via the DOM API whether a property is a string or an item. When on the topic of the DOM API, document.getItems(outer)[0].getItems(inner)[0] would be so much clearer than what we currently have. Example: p item=vcard itemprop=n item My name is span itemprop=given-namePhilip/span span itemprop=family-nameJägenstedt/span. /p I don't understand what this maps to at all. The same as p item=vcard span itemprop=n item My name is span itemprop=given-namePhilip/span span itemprop=family-nameJägenstedt/span. /span /p Unless I've misunderstood the n in vcard (there's no example in the spec). But let's move on. I'll admit that my examples are a bit simple
[whatwg] createImageData should take unsigned long
As far as I can see there's no good reason why createImageData should take a float as input rather than unsigned long. Having it as float creates the odd situation where (0.1, 0.1) gives a 1x1 ImageData while (10.1, 10.1) gives a 10x10 or 11x11 depening on if you ceil or round the input (not defined). Unless there's a compelling reason to allow something like (0.1, 0.1) I suggest changing the type and leaving the float-unsigned conversion to WebIDL. -- Philip Jägenstedt Opera Software
Re: [whatwg] Microdata
On Sat, 22 Aug 2009 23:51:48 +0200, Ian Hickson i...@hixie.ch wrote: Based on some of the feedback on Microdata recently, e.g.: http://www.jenitennison.com/blog/node/124 ...and a number of e-mails sent to this list and the W3C lists, I am going to try some tweaks to the Microdata syntax. Google has kindly offered to provide usability testing resources so that we can try a variety of different syntaxes and see which one is easiest for authors to understand. If anyone has any concrete syntax ideas that they would like me to consider, please let me know. There's a (pretty low) limit to how many syntaxes we can perform usability tests on, though, so I won't be able to test every idea. I've found two related things that are a bit problematic. First, because itemprops are only associated with ancestor item elements or via the subject attribute, it's always necessary to find or create a separate element for the item. This leads to more convoluted markup for small items, so it would be nice if the first item and itemprop could be on the same element when it makes sense: p item=vevent itemprop=description Concert at span itemprop=dtstart19:00/span at span itemprop=locationthe beach/span. /p rather than p item=vevent span itemprop=description Concert at span itemprop=dtstart19:00/span at span itemprop=locationthe beach/span. /span /p Second, because composite items can only be made by adding item and itemprop to the same element, the embedded item has to know that it has a parent and what itemprop it should use to describe itself. James gave the example of something like planet where each article could be a com.example.blog item and within each article there could be any arbitrary author-supplied microdata [1]. I also feel that the item+itemprop syntax for composite items is one of the least intuitive parts of the current spec. It's easy to get confused about what the type of the item vs the itemprop should be and which item the itemprop actually belongs to. Given that flat items like vcard/vevent are likely to be the most common use case I think we should optimize for that. Child items can be created by using a predefined item property: itemprop=com.example.childtype item. The value of that property would then be the first item in tree-order (or all items in the subtree, not sure). This way, items would have better copy-paste resilience as the whole item element could be made into a top-level item simply by moving it, without meddling with the itemprop. If the parent-item (com.example.blog) doesn't know what the child-items are, it would simply use itemprop=item. Example: p item=vcard itemprop=n item My name is span itemprop=given-namePhilip/span span itemprop=family-nameJägenstedt/span. /p I'll admit that my examples are a bit simple, but the main point in my opinion is to make item+itemprop less confusing. There are basically only 3 options: 1. for compositing items (like now) 2. as shorthand on the top-level item (my suggestion) 3. disallow I'd primarily like for 1 and 2 to be tested, but 3 is a real option too. [1] http://krijnhoetmer.nl/irc-logs/whatwg/20090824#l-375 -- Philip Jägenstedt Opera Software
[whatwg] Microdata DOM API
Hi,
There are already two demos of converting Microdata to other formats which
I found quite useful [1]. I've taken a closer look at the Microdata DOM
API and hacked up a somewhat working JavaScript implementation of it [2].
A few issues came up in the process:
To avoid total confusion I'll use item-property and DOM-property to
disambiguate.
The spec says that properties can also themselves be groups of name-value
pairs, but this isn't exposed in a very convenient way in the DOM API.
The 'properties' DOM-property is a HTMLPropertyCollection of all
associated elements. Discovering if the item-property value is a plain
string or an item seems to require item.hasAttribute('item'), which seems
out of place when everything else has been so neatly reflected. (Just
checking item.item won't work if the item attribute is empty.) Also, the
'contents' DOM-property is always the item-property value except in the
case where the item-property is another item -- in that case it is
something random like .href or .textContent depending on the element type.
I think it would be better if the DOM-property were simply called 'value'
(the spec does talk about name-value pairs after all) and corresponded
more exactly to 'property value' [3]. Elements that have no 'property
names' [4] should return null and otherwise elements with an 'item'
attribute should return itself, although I don't think it should be
writable in that case. One might also/otherwise consider adding a
valueType DOM-property which could be 'string', 'item' or something
similar.
One example [5] uses document.items[item].names but document.items isn't
defined anywhere. I assume this is an oversight and that it is equivalent
to document.getItems() Further, names is a member of
HTMLPropertyCollection, so document.items[item].properties.names is
probably intended instead of document.items[item].names. Assuming this the
example actually produces the output it claims to.
Shouldn't namedItem [6] be namedItems? Code like .namedItem().item(0)
would be quite confusing. Also, RadioNodeList should be PropertyNodeList.
I think many will wonder why item and itemprop can't be given on a single
element for compactness:
span item=org.example.fruit itemprop=org.example.nameApple/spans
and span item=org.example.fruit
itemprop=org.example.nameOrange/spans don't compare well.
Allowing this would complicate the definition of 'corresponding item' [7],
but I think that might be acceptable. I suggest either allowing it or
adding a note explaining why it isn't allowed and giving examples of
alternative techniques.
[1] http://philip.html5.org/demos/microdata/demo.html
http://james.html5.org/microdata/
[2]
http://gitorious.org/microdatajs/microdatajs/blobs/0032bac85ceaf4fd2a6379b357a225f74c89d61f/microdata.js
[3]
http://www.whatwg.org/specs/web-apps/current-work/multipage/microdata.html#concept-property-value
[4]
http://www.whatwg.org/specs/web-apps/current-work/multipage/microdata.html#property-names
[5]
http://www.whatwg.org/specs/web-apps/current-work/multipage/microdata.html#using-the-microdata-dom-api
[6]
http://www.whatwg.org/specs/web-apps/current-work/multipage/infrastructure.html#dom-htmlpropertycollection-nameditem
[7]
http://www.whatwg.org/specs/web-apps/current-work/multipage/microdata.html#concept-item-corresponding
--
Philip Jägenstedt
Doing Microdata just for fun, not for Opera Software.
Re: [whatwg] Remove addCueRange/removeCueRanges
On Fri, 14 Aug 2009 10:28:14 +0200, Dr. Markus Walther walt...@svox.com wrote: Hi, The .start/.end properties were dropped in favor of media fragments, which the Media Fragments Working Group is producing a spec for. Who decided this? Has this decision been made public on this list? It happened in November 2008: http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2008-November/017157.html This is the specific spec change: http://html5.org/tools/web-apps-tracker?from=2400to=2401context=10 It will be something like http://www.example.com/movie.mov#t=12.33,21.16 var audioObject = new Audio(); audioObject.src ='data:audio/x-wav;base64,UklGRiIAAABXQVZFZm10IBABAAEAIlYAAESsAAACABAAZGF0Yf7///8A'; // play entire audio audioObject.play(); // play (0.54328,0.72636) media fragment ? See http://www.w3.org/2008/01/media-fragments-wg.html and http://www.w3.org/2008/WebVideo/Fragments/wiki/Syntax#Examples Did you look at these yourself? I couldn't find something that approaches a spec of comparable quality to WHATWG in these pages. Implementation (should) always comes before spec and there is no implementation yet. Given the simplicity of the most basic syntax, I am not at all worried by the lack of a proper spec yet. Is there any provision for the dynamic case, where you want to change the media fragment after it has been loaded, with zero server interaction, and working for data URIs as well? You can change the fragment with scripts at any time you want followed by load()/play(). data URI's with fragments are perfectly valid as far as I can see. The rest is up to proper implementation (caching). Actually, out of curiousity: could gapless concatenation of several audio objects be added as well, e.g. audioObject1.append(audioObject2) or even audioObject.join([audioObject1,audioObject2,...,audioObjectN) There has been much discussion about audio canvas API:s and I trust this could fit into that scope. As the 'inventor' of the term, I am of course familiar with the discussion - here I was merely adding an item to the wishlist. View source at http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#video and search for v2 and you'll find some of these ideas. Could these be lifted from hidden HTML comments to something with better visibility somehow? I agree that would be nice, perhaps on http://wiki.whatwg.org/ ? -- Philip Jägenstedt Opera Software
Re: [whatwg] Remove addCueRange/removeCueRanges
On Fri, 14 Aug 2009 10:42:50 +0200, Dr. Markus Walther walt...@svox.com wrote: Silvia, 2009/8/13 Dr. Markus Walther walt...@svox.com: please note that with cue ranges removed, the last HTML 5 method to perform audio subinterval selection is gone. Not quite. You can always use the video.currentTime property in a javascript to directly jump to a time offset in a video. And in your javascript you can check this property until it arrives at your determined end time. So, there is a way to do this even now. How can polling approach that somehow monitors currentTime meet any halfway-decent accuracy requirements? E.g. to be accurate to 1-3 samples at 22050 Hz sampling frequency? I doubt your approach could fulfill this. To my mind, the current turn of events suggests simply to allow start/end attributes back into the WHATWG spec, eased by the fact that there were already browser implementations of it. Highly accurate looping or fragment selection is something that requires a lot of work on the implementation side. I don't have a particular aversion to .start/.end, but think it would be wise to wait for implementor experience before (re)introducing any of start/end, addCueRange/removeCueRanges or a timed text API. Whatever the spec says before that is likely to be wrong. -- Philip Jägenstedt Opera Software
Re: [whatwg] Remove addCueRange/removeCueRanges
On Fri, 14 Aug 2009 12:48:59 +0200, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: On Fri, Aug 14, 2009 at 7:54 PM, Dr. Markus Waltherwalt...@svox.com wrote: Silvia Pfeiffer wrote: 2009/8/14 Dr. Markus Walther walt...@svox.com: Hi, The .start/.end properties were dropped in favor of media fragments, which the Media Fragments Working Group is producing a spec for. Who decided this? Has this decision been made public on this list? It will be something like http://www.example.com/movie.mov#t=12.33,21.16 var audioObject = new Audio(); audioObject.src ='data:audio/x-wav;base64,UklGRiIAAABXQVZFZm10IBABAAEAIlYAAESsAAACABAAZGF0Yf7///8A'; // play entire audio audioObject.play(); // play (0.54328,0.72636) media fragment ? Not in this way. In fact, the new way will be much much simpler and does not require javascript. With the code snippet given I was pointing out that it is not obvious (to me at least) how the proposed media fragment solution covers data URIs. If it is not meant to cover them, it is limited in a way that the solution it seeks to replace is not. I see no reason why they should not be applicable to data URIs when it is obvious that the data URI is a media file. This has not yet been discussed, but would be an obvious use case. BTW: Did the start and end attribute implementations that you refer to cover the data scheme, too? To my knowledge/experience, there is nothing special about data URIs. Any differences in how they work with the DOM interface or media fragments are more than likely implementation bugs. -- Philip Jägenstedt Opera Software
[whatwg] Remove addCueRange/removeCueRanges
Hi, We would like to request that addCueRange/removeCueRanges be dropped from the spec before going into Last Call. We are not satisfied with it and want to see it replaced with a solution that includes (scriptless) timed text (a.k.a captions/subtitles). I don't think that this will be finished in time for Last Call however, because we need implementor experience to write a good spec. However, we have no intention of implementing both cue ranges and its replacement, so it is better if the spec doesn't provide any solution for now. I have been briefly in contact with other browser vendors and while I cannot speak for them here, I hope those that agree will chime in if necessary. -- Philip Jägenstedt Opera Software
Re: [whatwg] Remove addCueRange/removeCueRanges
On Thu, 13 Aug 2009 14:34:55 +0200, Dr. Markus Walther walt...@svox.com wrote: Hi, please note that with cue ranges removed, the last HTML 5 method to perform audio subinterval selection is gone. AFAIK, when dropping support for 'start' and 'end' attributes it was noted on this list that cue ranges would provide a replacement to dynamically select, say, a 3-second range from a 1-hour audio source. So, if cue ranges will indeed be dropped, could browser vendors and standards people consider putting 'start' and 'end' back in, just like Safari had it for a while (albeit buggy)? The .start/.end properties were dropped in favor of media fragments, which the Media Fragments Working Group is producing a spec for. It will be something like http://www.example.com/movie.mov#t=12.33,21.16 See http://www.w3.org/2008/01/media-fragments-wg.html and http://www.w3.org/2008/WebVideo/Fragments/wiki/Syntax#Examples Actually, out of curiousity: could gapless concatenation of several audio objects be added as well, e.g. audioObject1.append(audioObject2) or even audioObject.join([audioObject1,audioObject2,...,audioObjectN) There has been much discussion about audio canvas API:s and I trust this could fit into that scope. View source at http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#video and search for v2 and you'll find some of these ideas. -- Philip Jägenstedt Opera Software
Re: [whatwg] Alt attribute for video and audio
On Wed, 12 Aug 2009 12:52:38 +0200, Remco remc...@gmail.com wrote: On Wed, Aug 12, 2009 at 10:57 AM, Philip Jägenstedtphil...@opera.com wrote: Before suggesting any changes to the source element, make sure you have read http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#concept-media-load-algorithm Put simply, the handling of source is already quite complex, overloading it with completely different meanings is not a good idea. video won't handle text/html as a source, but if you want different media files for different audiences I suggest experimenting with source media. source media doesn't do anything useful for my case. It can't load textual data. Also, if the resources are unavailable, there will be nothing to see, since all resources are off-page. It also doesn't work for iframe, object, embed or img. Is it really the idea that the only way you're going to have alternative textual content, is to Build It Yourself? You have to abuse details or a hidden div with some Javascript to build a construction that has alternative content in case the video/audio/iframe/object/embed is not available or desirable. If you want it to be semantically accessible, you even have to build another layer on top of that, in the form of ARIA attributes. No, in the long term we want native captions/subtitle support in the browsers. See http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2009-July/021658.html and maybe http://lists.w3.org/Archives/Public/public-html/2009Aug/0439.html Nobody will do that. Even the source solution is harder, maybe too hard, to use than the alt= solution. It requires authors to create additional elements or pages to house the alternative content. Since accessibility is often an afterthought, about the most an author will be willing to do, is filling in an alt attribute. What do you suggest a browser do with the alt attribute? The resource selection algorithm never ends until a suitable source is found, so when should the alt text be displayed? By requiring anything at all, browsers can't do things like display a box with a direct download link, suggestion to install a specific codec, etc. If nothing at all is required of user agents for the alt attribute, then I have no opinion (but then I expect no one would use it either). -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Alt attribute for video and audio
in a previous email. I believe that was accomplished with some kind of ARIA attributes, correct me if I'm wrong. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Progress on video accessibility
On Fri, 31 Jul 2009 05:36:40 +0200, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: I am curious about comments to this proposal and suggestions for improvement. I have not yet developed an improved specification, but instead have collected feedback at https://wiki.mozilla.org/Accessibility/Experiment1_feedback#Thoughts_.2F_Feedback I have already given feedback on this (see above link) and am supportive of the general approach taken with this. I think the core functionality is pretty basic -- external subtitles/captions are displayed together with the video. The points that seem most important to me to work out at this point are: * DOM interfaces. These should be the same regardless of source (external/internal) and format. I also believe these ranges have event listeners, so as to replace the cue ranges interface. * Security. What restrictions should apply for cross-origin loading? * Complexity. There is no limit to the complexity one could argue for (bouncing ball multi-color karaoke with fan translations/annotations anyone?). We should accept that some use cases will require creative use of scripts/SVG/etc and not even try to solve them up-front. Draw a line and stick to it. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Audio synthesis
On Wed, 22 Jul 2009 04:38:08 +0200, Patrick Mueller pmue...@muellerware.org wrote: Robert O'Callahan wrote: On Wed, Jul 22, 2009 at 1:36 AM, Patrick Mueller pmue...@muellerware.orgwrote: I've just started playing a bit with audio. One thing I noticed with both FF 3.5 and WebKit nightlies is that usage of the loop attribute set to true does not provide seamless looping. ie, there is a significant pause between when the audio clip end is reached and when the clip starts again. That might be related to the fact that we haven't implemented 'loop' yet. Woops, caught me there. It's exciting to have programmatic audio available AT ALL, was forgetting some of the details. But since you mention it, any idea whether the Moz developers plan on implementing seamless looping or not? I'm curious whether anyone cares, and what folks interpretation of looping means. The spec should be made more specific, one way or another (must/should/may implement seamless looping). Or even provide some other mechanism to allow for seamless looping if the current APIs are insufficient, and it's deemed important. The spec simply says If the media element has a loop attribute specified, then seek to the earliest possible position of the media resource and abort these steps. This would give seamless looping if an implementation makes an effort to make it so. I certainly don't think there should be an option to have seamless looping or not, each implementation will simply have to do its best. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] autobuffer on new Audio objects
On Mon, 20 Jul 2009 07:06:32 +0200, Nils Dagsson Moskopp nils-dagsson-mosk...@dieweltistgarnichtso.net wrote: Am Montag, den 20.07.2009, 05:46 +0100 schrieb David Wilson: It's easy to see how some naively implemented JS audio widget could fetch 200mb over an expensive 3G connection, simply by navigating to some site in a background tab (say, by creating an array of elements to represent their playlist - something I'd have thought was perfectly valid behaviour). I second that motion, not only as owner of a smartphone, but also as someone with webspace that has a volume cap. Automagic audio element buffering could deter web authors from dynamically putting more than one element on a page, thus reserving javascript playlist widgets to those who can afford more bandwith on an order of magnitude (!). I believe the burden of writing another line should solely be on those who want autobuffering, to prevent unneccessary bandwith consumption. Cheers Regardless of whether the spec mandates that the autobuffering attribute be set for new Audio(), browsers that want to be competitive on mobile will have to be conservative with bandwidth. autobuffering is only a hint and doesn't need to be obeyed where it doesn't make sense (in fact can't be obeyed because the disk cache is too small). -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Vorbis in audio
On Thu, 16 Jul 2009 13:26:18 +0200, David Gerard dger...@gmail.com wrote: Let's put it another way. Of the browser vendors here: * who supports Vorbis as a baseline codec for audio? Opera would support Vorbis as an audio baseline. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Thoughts on video accessibility
-weight packaging code. So, yes, it is totally possible to have the browsers create a binary video file that has the subtitles encapsulated that were previously only accessible as referenced text files behind a separate URL. The only issue I see is the baseline codec issue: every browser that wants to support multiple media formats has to implement this multiplexing and text encoding for every media encapsulation format differently, which is annoying and increases complexity. It's however generally a small amount of complexity compared to the complexity created by having to support multiple codecs. I disagree, remuxing files would be much more of an implementation burden than supporting multiple codecs, at least if a format-agnostic media framework is used (be that internal or external to the browser). Remuxing would require you to support/create parts of the media framework that you otherwise aren't using, i.e. parsers, muxers, file writers and plugging of these together (which unlike decoding isn't automatic in any framework I've seen). Anything is doable of course, but I think this is really something that is best done server-side using specialized tools. Here is my example again: video src=http://example.com/video.ogv; controls text category=CC lang=en type=text/x-srt src=caption.srt/text text category=SUB lang=de type=application/ttaf+xml src=german.dfxp/text text category=SUB lang=jp type=application/smil src=japanese.smil/text text category=SUB lang=fr type=text/x-srt src=translation_webservice/fr/caption.srt/text /video Here's a counterproposal: video src=http://example.com/video.ogv; subtitles=http://example.com/caption.srt; controls /video Subtitle files are created to enable users to choose the text in the language that they speak to be displayed. With a simple addition like what you are proposing, I don't think such a choice is possible. Or do you have a proposal on how to choose the adequate language file? Also, the attributes on the proposed text element of course serve a purpose: * the category attribute is meant to provide a default for styling the text track, * the language attribute is meant to provide a means to build a menu to choose the adequate subtitle file from, * the type attribute is meant to both identify the mime type of the format and the character set used in the file. The character set question is actually a really difficult problem to get right, because srt files are created in an appropriate character set for the language, but there is no means to store in a srt file what character set was used in its creation. That's a really bad situation to be in for the Web server, who can then only take an educated guess. By giving the ability to the HTML author to specify the charset of the srt file with the link, this can be solved. BTW: my latest experiments with subtitles have even a few more attributes. I am not ready to publish that yet, but should be within a week or so and will be glad to have a further discussion then. I think this would be fine, on the long term. I don't think the existing implementations of video are at a point yet where it makes sense to define this yet, though. I think we have to start discussing it and doing experiments. I think video is getting stable enough to move forward. I'm expecting a period of discussion and experimentation with time-aligned text both in-band and out-of-band, so it's good to get started on this rather sooner than later. It would be interesting to hear back from the browser vendors about how easily the subtitles could be kept with the video in a way that survives reuse in other contexts. I think that in the case of external subtitles the browser could simply save it alongside with the video. It is my experience that is media players have much more robust support for external subtitles (like SRT) than for internal subtitles, so this is my preferred option (plus it's easier). Incidentally, I'd be interested in such information about H.264. I wonder how easy it will be for example with QuickTime or mp4 to encapsulate srt on-the-fly inside a browser. Regards, Silvia. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] HTML 5 video tag questions
On Mon, 13 Jul 2009 23:32:44 +0200, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jul 13, 2009 at 4:01 PM, Philip Jägenstedtphil...@opera.com wrote: I thought you meant video source fallback fallback content here /source /video I would prefer if fallback content and source elements weren't mixed on the same level, but maybe that's just me. Eh, fallback content for non-video browsers is already there, so it's nothing new. (You could also just put @fallback on the second source and drop the third source entirely for the same effect.) Once the source element is iterated by the resource selection algorithm and starts loading, the fallback attribute or the whole source element might be removed. You would have to set a flag at some point to remember the fallback state and unset it... when? What is the effect of video source fallback source src=cant.play.ogg fallback here /video ? It would hit the first source, note that there's no @src, see that there is @fallback, and immediately drop to showing fallback content. Is fallback irrevocable or could the same video element go back to showing video under some circumstances? It should be revocable in the exact same circumstances that a video element might start playing from one source, and then switch to another. From my reading of the algorithm I don't think this is possible, so fallback would be irrevocable. Right, anything but irrevocable fallback would make this even stranger than object fallback. Let's not go there. Does audio also have fallback content? It uses source and the same source-selection algorithm (and the same fallback situation in the case of non-audio browsers), so yes. This is why I suggested videosource src=cant.play.oggnew-fallback-elementOoops!/new-fallback-element/video I still think the use of this is questionable though. I'm not sure why you think the usecase is questionable. It seems pretty clear - it'd be nice to have a non-script way of showing content (specifically, alternate video players) when the browser supports video but can't play any of the provided sources. Yes, handling the (minor) use case video support + no supported source + no script would be nice, but only if it doesn't make an even bigger mess of the resource selection algorithm and doesn't allow switching back and forth between video and fallback. From what I can tell of the resource selection algorithm, fallback should be irrevocable, just like a successful source load is. The resource selection algorithm appears to need fairly trivial changes to accommodate this. Step 4, when parsing @src from video, insert a new substep between substeps 2 and 3. Pull the first sentence from the current substep 3 into this new substep, and say something like If the element has a fallback attribute, abort this algorithm immediately and display the fallback content. When parsing See below for why we can't put the fallback attribute on the video element. source elements, change step 3 so that whenever any of those conditions are met, if the source has @fallback the algorithm aborts immediately and shows the fallback content, otherwise it fires the error event and jumps to the search loop step as normal. This would only cause fallback to be used when the source element is rejected up front, not if it is a candidate for the resource fetch algorithm that fails to fetch/play. So you also need to check @fallback before step 6. However, by then the source element may very well be removed from the DOM, so I you'd have to assign @fallback to a variable earlier in the algorithm and check that. This approach is implementable in my guesstimation, but I'd really want to know more about the history of object fallback issues. Should fallback content be treated as if it were display:none, or are there nastier hacks involved? Alternately, we could just put @fallback always on the video element directly, rather than on source. That would preserve the first part of what I said, but now we have a step between substeps 9 and 10 that checks for @fallback on the video. If it finds it, it immediately aborts and shows the fallback content, otherwise substep 10 proceeds normally (waiting forever). That would cause the fallback to be triggered unconditionally during parsing: video fallback source src=foo type=video/x-unsupported !-- steps 8-10 run here -- source src=bar type=video/x-supported /video There's nothing in the resource selection algorithm that's special-cased for static markup by hooking into the parser, which would be required to make it work (but then only for static markup). Putting @fallback on the video element just won't work. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] HTML 5 video tag questions
On Mon, 13 Jul 2009 23:28:41 +0200, Jonas Sicking jo...@sicking.cc wrote:
On Mon, Jul 13, 2009 at 5:01 AM, Philip Jägenstedtphil...@opera.com
wrote:
The design you describe is what object tried to do, and it proved
to be
extremely problematic in practice -- and that was without another
built-in
fallback mechanism to complicate matters.
While object has had a very poor implementation story, I don't think
this was a big reason for that.
Robert O'Callahan, Boris Zbarsky and other gecko layout folks can
answer this better, but at least in gecko I don't think this part of
object was particularly hard to implement correctly once we actually
tried.
Has any browser vendor argued against displaying the fallback due to
high implementation burden?
Implementation probably isn't the biggest burden here, specifying sane
behavior is. For example:
If fallback content is displayed after the last source element has
failed,
what should happen when a new source element is inserted? Switching
back to
video mode would make it even more special than object fallback and
if you
don't you'll just get stuck with fallback and have effectively broken
the
possibility of inserting source elements via scripts.
Something like videosource
src=cant.play.oggnew-fallback-elementOoops!/new-fallback-element/video
is what you'd need to make the resource selection algorithm deal with
fallback in a sane way when scripts are disabled, but this is too much
of a
corner case to justify the complexity in my opinion.
I think fallback contents is simply defined as the contents of the
video, minus any source elements (which wouldn't render anyway).
No need for new-fallback-element.
If I may nit-pick, the spec explicitly says this content is not fallback
content. But your suggestion is to change is, so very well...
The simplest solution would be to display the fallback when there
aren't any source (or @src) elements being either displayed or
waiting to load. I think spec-wise the simplest thing would be to
display fallback when
The networkState of the element is NETWORK_NO_SOURCE.
This way even incremental rendering of the fallback contents would
work fine. The only case that's weird is markup like:
video
lots and lots of fallback here
source src=...
/video
There is a risk that content would be displayed, and then switch to
displaying video. This doesn't seem like a big problem as it seems
more likely that people will put the source first. And if someone
doesn't the effects aren't very bad.
Alternatively we could use the following rules:
1. The networkState of the element is NETWORK_NO_SOURCE.
*and*
2. The video element is fully parsed. I.e. it has been removed from
the stack of open elements.
This would mean that incremental rendering doesn't work. This doesn't
seem any worse than what we have now when fallback is never displayed.
Though I think it'd work fine always display fallback whenever the
networkState is NETWORK_NO_SOURCE.
/ Jonas
Anything that can cause the element to switch back and forth between
displaying fallback and video is a no-go, that would cause a race
condition for if plugins/images in the fallback content. If they have
event handlers the results will get ugly fast:
video
!-- network lag --
source
!-- network lag --
source
!-- network lag --
img src=foo onload=alert('how many times will you see this message?')
/video
--
Philip Jägenstedt
Core Developer
Opera Software
Re: [whatwg] HTML 5 video tag questions
On Tue, 14 Jul 2009 11:46:08 +0200, Boris Zbarsky bzbar...@mit.edu wrote:
Philip Jägenstedt wrote:
Anything that can cause the element to switch back and forth between
displaying fallback and video is a no-go, that would cause a race
condition for if plugins/images in the fallback content. If they have
event handlers the results will get ugly fast:
video
!-- network lag --
source
!-- network lag --
source
!-- network lag --
img src=foo onload=alert('how many times will you see this
message?')
/video
The answer to the question in the alert is once. This is true in the
current fallback model, and would presumably be true in the new fallback
model.
For the current model, note that all the text says is should not show
this content to the user. While this is not defined anywhere, it
doesn't seem to indicate that the content's DOM should not exist, for
example. In Gecko, at least, the image in your example will be loaded
and hence its onload will fire.
True, but you could easily have other things in the fallback content (like
mouse listeners) that are suddenly being triggered even though the
fallback content isn't going to be used. No switching back and forth,
please.
I've discussed the object fallback a bit with a knowledgeable colleague
who confirmed that we've had lots of problems with it. Uncertain state of
the fallback content from a scripts point of view is one issue. What to do
with nested objects (when to load/unload them) is another.
If we allow fallback people will certainly begin using nested video,
unless we want to another hack to prevent that.
Ian: can you make nested video elements non-conforming so that
validators will flag it? This would be helpful regardless of the fallback
discussion.
--
Philip Jägenstedt
Core Developer
Opera Software
[whatwg] Error handling for MIME type parsing (canPlayType)
While implementing canPlayType I've found that Firefox/Safari/Chrome (and now Opera) all have different error handling in parsing the MIME types. RFC 2045[1] gives the BNF form, but it appears that no browser gives much weight to this. Sample of quirks: (Ignore no vs here, it's not relevant) Firefox: AUDIO/X-WAV: no audio/x-wav codecs: maybe audio/x-wav; codecs=: probably audio/x-wav; codecs=,: no Safari: AUDIO/X-WAV: no audio/x-wav codecs: no audio/x-wav; codecs=: probably audio/x-wav; codecs=,: maybe Opera internal: AUDIO/X-WAV: audio/x-wav codecs: audio/x-wav; codecs=: maybe audio/x-wav; codecs=,: maybe Chrome ignores codecs, so I can't get meaningful results. I believe the correct answers are: AUDIO/X-WAV: same as for audio/x-wav (by RFC 2045, but we could ignore it because it looks ugly) audio/x-wav codecs: the same as audio/x-unknown-type (i.e. no for Firefox) audio/x-wav; codecs=: same as audio/x-wav (unless we want this to mean no codecs, in which case no would be more appropriate) audio/x-wav; codecs=,: same as audio/x-wav (i.e. ignore broken codecs parameter) Has there been any work done on defining error handling for Content-Type parsing or the like? It wouldn't be fun to get cross-browser bugs as soon as someone forgets a semicolon... -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Serving up Theora video in the real world
On Mon, 13 Jul 2009 00:00:20 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Sun, Jul 12, 2009 at 10:34 AM, Philip Jägenstedt phil...@opera.comwrote: Not that I except this discussion to go anywhere, but out of curiosity I checked how Firefox/Safari/Chrome actually implement canPlayType: http://wiki.whatwg.org/wiki/Video_type_parameters#Browser_Support Firefox is conservative and honest (except maybe for audio/wav; codecs=0, what could you do with the RIFF DATA chunk?) Wave codec 0 means unknown codec, so we return maybe, but we should actually return no since in practice we refuse to play a file with 0 in the format field. We'll fix that. Safari gets maybe/probably backwards compared to what the spec suggests. Chrome seems to ignore the codecs parameter, claiming probably even for bogus codecs. Authors obviously can't trust the distinction between maybe and probably to any extent. Please file bugs against those browsers --- especially Chrome, since they haven't shipped video support yet. It's not hard to implement this right, these issues reflect sloppy development more than a fundamental problem IMHO. I agree that this isn't hard (even when using platform media frameworks), I'm only arguing that it isn't useful. However, both Mozilla and Apple representatives think differently, so I will not bother with this issue any more and simply implement it per spec as written. Also, I've reported bugs on Safari and Chrome (I think, neither give confirmation that the report has been sent successfully!) -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] HTML 5 video tag questions
On Mon, 13 Jul 2009 11:14:21 +0200, Jonas Sicking jo...@sicking.cc wrote: On Sun, Jul 12, 2009 at 11:20 PM, Ian Hicksoni...@hixie.ch wrote: On Fri, 10 Jul 2009, Jonas Sicking wrote: Hmm.. is that good? What if you want to use an object (to use flash or java) or a img as fallback? Then you do it with script. The design is based around the assumption that we will eventually find a common codec so that fallback won't ever be needed in supporting UAs. I agree that the current design makes sense once there is a common codec supported across all browsers. However currently it seems like we might not reach that point until after all major browsers support video. What would be the downside of displaying the fallback contents if none of the videos can be displayed due to unsupported codecs? When would you fall back? For example, while parsing, would you fall back in between the video element being parsed and the first source element being parsed? You could display the fallback once you've reached the /video and not found an acceptable source. Technically it'd be when you pop the video element off the stack of open elements. I don't even think it'd be hard to pull down all sources and check that none of them are supported before displaying the fallback if types aren't specified on the source element. It would have to be part of the resource selection algorithm. Since that waits for new source elements indefinitely, when exactly would you decide to switch to fallback content? Bad solutions include special-casing static markup and/or (falsely) assuming that scripts will not insert more source elements at some point. If fallback content is defined simply as the content of the video element I also can't figure out any other solutions. The design you describe is what object tried to do, and it proved to be extremely problematic in practice -- and that was without another built-in fallback mechanism to complicate matters. While object has had a very poor implementation story, I don't think this was a big reason for that. Robert O'Callahan, Boris Zbarsky and other gecko layout folks can answer this better, but at least in gecko I don't think this part of object was particularly hard to implement correctly once we actually tried. Has any browser vendor argued against displaying the fallback due to high implementation burden? Implementation probably isn't the biggest burden here, specifying sane behavior is. For example: If fallback content is displayed after the last source element has failed, what should happen when a new source element is inserted? Switching back to video mode would make it even more special than object fallback and if you don't you'll just get stuck with fallback and have effectively broken the possibility of inserting source elements via scripts. Something like videosource src=cant.play.oggnew-fallback-elementOoops!/new-fallback-element/video is what you'd need to make the resource selection algorithm deal with fallback in a sane way when scripts are disabled, but this is too much of a corner case to justify the complexity in my opinion. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] HTML 5 video tag questions
On Mon, 13 Jul 2009 20:38:11 +0200, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jul 13, 2009 at 1:33 PM, Boris Zbarskybzbar...@mit.edu wrote: Philip Jägenstedt wrote: It would have to be part of the resource selection algorithm. Since that waits for new source elements indefinitely, when exactly would you decide to switch to fallback content? Bad solutions include special-casing static markup and/or (falsely) assuming that scripts will not insert more source elements at some point. If fallback content is defined simply as the content of the video element I also can't figure out any other solutions. A source that says use the content? -Boris Ie inserting source fallback or source contents. If both @src and @fallback are specified on a source, it is treated like a source srcsource fallback; that is, it first tries the @src attribute, and if that doesn't work, then it goes to the fallback content. That would require the parser to inspect an attribute to determine if the element is a void element (one that does not have a closing tag) or not, which I've been told is not very nice. Are there any other such cases? This is why I suggested videosource src=cant.play.oggnew-fallback-elementOoops!/new-fallback-element/video I still think the use of this is questionable though. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] HTML 5 video tag questions
On Mon, 13 Jul 2009 21:28:45 +0200, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jul 13, 2009 at 2:15 PM, Philip Jägenstedtphil...@opera.com wrote: On Mon, 13 Jul 2009 20:38:11 +0200, Tab Atkins Jr. jackalm...@gmail.com wrote: On Mon, Jul 13, 2009 at 1:33 PM, Boris Zbarskybzbar...@mit.edu wrote: Philip Jägenstedt wrote: It would have to be part of the resource selection algorithm. Since that waits for new source elements indefinitely, when exactly would you decide to switch to fallback content? Bad solutions include special-casing static markup and/or (falsely) assuming that scripts will not insert more source elements at some point. If fallback content is defined simply as the content of the video element I also can't figure out any other solutions. A source that says use the content? -Boris Ie inserting source fallback or source contents. If both @src and @fallback are specified on a source, it is treated like a source srcsource fallback; that is, it first tries the @src attribute, and if that doesn't work, then it goes to the fallback content. That would require the parser to inspect an attribute to determine if the element is a void element (one that does not have a closing tag) or not, which I've been told is not very nice. Are there any other such cases? Hm? I'm not sure what you mean here. It would work like this: video source src=foo source src=bar source fallback pI'm some fallback content!/p /video You'll see the p if the browser doesn't support video, or if the resource selection algorithm hits that third source and hasn't found anything it can play yet. It wouldn't change whether the source is a void element or not. I thought you meant video source fallback fallback content here /source /video I would prefer if fallback content and source elements weren't mixed on the same level, but maybe that's just me. (You could also just put @fallback on the second source and drop the third source entirely for the same effect.) Once the source element is iterated by the resource selection algorithm and starts loading, the fallback attribute or the whole source element might be removed. You would have to set a flag at some point to remember the fallback state and unset it... when? What is the effect of video source fallback source src=cant.play.ogg fallback here /video ? Is fallback irrevocable or could the same video element go back to showing video under some circumstances? Does audio also have fallback content? This is why I suggested videosource src=cant.play.oggnew-fallback-elementOoops!/new-fallback-element/video I still think the use of this is questionable though. I'm not sure why you think the usecase is questionable. It seems pretty clear - it'd be nice to have a non-script way of showing content (specifically, alternate video players) when the browser supports video but can't play any of the provided sources. Yes, handling the (minor) use case video support + no supported source + no script would be nice, but only if it doesn't make an even bigger mess of the resource selection algorithm and doesn't allow switching back and forth between video and fallback. The browsers that have already shipped with video don't support fallback but they are the ones that are the most likely to need it (assuming that format support will get better over time). It would be nice to hear what they think about all of this. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Serving up Theora video in the real world
On Sat, 11 Jul 2009 03:35:22 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Sat, Jul 11, 2009 at 9:37 AM, Philip Jagenstedt phil...@opera.comwrote: the point is simply that calling canPlayType without out a codecs list or with specific codecs, you can learn exactly what is supported and not out of the container formats and codecs you are interested in, without the need for the strange probably/maybe/ API. I think it would be somewhat counterintuitive for canPlayType(video/ogg) to return true, but canPlayType(video/ogg; codecs=dirac) to return false. Well I disagree of course, because having canPlayType(video/ogg) mean anything else than can I demux Ogg streams is pointless. Quoting myself from http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2008-November/017212.html (replies from Ian) On Thu, 13 Nov 2008, Philip Jägenstedt wrote: When asking about application/ogg, this could mean 2 things: 1. can I demux Ogg streams? 2. can I demux Ogg streams and decode unknown codecs? It's the second (and thus the answer can only ever be maybe or no). [snip] Unless the codecs parameter is to be made mandatory I think that spec should explicitly make it such that the question asked is 1. In either case we will end up there because 2 is not a meaningful question anduser agents will make untruthful answers in attempts to stay compatiblewith unknown and future content (which might be supported by installingnew codecs in the media framework without upgrading the browser). Currently the spec says we should interpret canPlayType(video/ogg) as can I demux Ogg streams and decode unknown codecs?, which is a pointless question. If we seriously believe that people need the level of control provided by the 3-state answer, just let them make several queries to ask the precise questions they want to ask. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Serving up Theora video in the real world
On Sat, 11 Jul 2009 03:31:50 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Sat, Jul 11, 2009 at 5:23 AM, Aryeh Gregor simetrical+...@gmail.comsimetrical%2b...@gmail.com wrote: Maybe you'd try testing all the video types you support, and if one is maybe while another is probably you'd go with probably? Right. Or you might have plugin-based fallback you can use if you get maybe. Other authors with no plugin-based fallback would go ahead if they get maybe. Programmers expect binary logic, not ternary (look at the complaints about SQL's NULL). I agree. In the past I argued for two boolean functions. Anyway, it's a little late now to change canPlayType more than we just did. There is already deployed content checking for probably/maybe. Not really, the spec isn't stable yet and all deployed content is still experimental. We still have a chance to fix broken APIs (or live with them for 100 years). -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Serving up Theora video in the real world
On Sat, 11 Jul 2009 13:15:14 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Sat, Jul 11, 2009 at 9:38 PM, Philip Jägenstedt phil...@opera.comwrote: Well I disagree of course, because having canPlayType(video/ogg) mean anything else than can I demux Ogg streams is pointless. So you want canPlayType to mean one thing when provided a type without codecs, and another thing when provided a type with codecs. I don't think that's a good idea. Yes, I'm saying that when codecs are provided true means probably and otherwise it means maybe, because the distinction is pointless. Anyway, it's too late. If you care passionately about this you should have reopened this discussion months ago, not now that two browsers have just shipped support for the API in the spec. Ian, if you agree that it's too late to change you needn't reply to any of my other points, I won't push the issue further. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Serving up Theora video in the real world
On Sat, 11 Jul 2009 14:38:02 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Sat, Jul 11, 2009 at 11:51 PM, Philip Jägenstedt phil...@opera.comwrote: Yes, I'm saying that when codecs are provided true means probably and otherwise it means maybe, because the distinction is pointless. IIRC some browsers using system media frameworks don't know what codecs they support, so they still need to be able to answer maybe when codecs are provided; you still need a three-valued result. Opera is one such browser (when using GStreamer), but even if we return maybe for video/ogg; codecs=uncertain-codec, what could be done with the information? The only way of knowing is trying to play it, which is what the resource selection algorithm would do. That's true of both maybe and probably, neither are actual guarantees. Is there any use case for distinguishing them? This is primarily a disagreement about what kind of API is more aesthetically pleasing, either way exposes the same (useful) information. I still think it would confuse authors if you return true for canPlayType(T) and false for canPlayType(U) where U is a subset of T. video/ogg; codecs=vorbis (U) is not a subset of video/ogg (T) as the codecs in T is the empty set, not the set of all codecs. Or did you have some other values of U and T in mind? -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] slideml
On Sat, 11 Jul 2009 16:57:23 +0200, narendra sisodiya narendra.sisod...@gmail.com wrote: Hi All, http://slideml.bitflux.ch/specification/slideml_1.0/ ( old ) I am looking for any standard for slideshow !! do anybody have any knowledge of any recent development for this task. It will be difficult to implement the .odp (open standard presentation) for web !! Perhaps you should have a look at http://www.opera.com/browser/tutorials/operashow/ Everything you need is already in HTML/CSS, it's just a question of browser support. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Serving up Theora video in the real world
On Sat, 11 Jul 2009 21:15:47 +0200, Maciej Stachowiak m...@apple.com wrote: On Jul 11, 2009, at 7:56 AM, Ian Fette (イアンフェッティ) wrote: 2009/7/11 Robert O'Callahan rob...@ocallahan.org On Sat, Jul 11, 2009 at 9:38 PM, Philip Jägenstedt phil...@opera.com wrote: Well I disagree of course, because having canPlayType(video/ogg) mean anything else than can I demux Ogg streams is pointless. So you want canPlayType to mean one thing when provided a type without codecs, and another thing when provided a type with codecs. I don't think that's a good idea. Anyway, it's too late. If you care passionately about this you should have reopened this discussion months ago, not now that two browsers have just shipped support for the API in the spec. Disagree -- the whole point of candidate rec (which the spec is driving towards) is to find out how implementable the spec is -- not just from the browser side, but from a web author side as well. If a feature turns out to not be implementable / usable in practice, that is certainly valid feedback at this stage. (Not to say it wouldn't be better to have had this conversation earlier, but I definitely don't think that the ship has sailed on this, and in practice some things you only find out once it's implemented and you can actually try using it.) At this point, I think video should only be changed incompatibly if it is discovered to be literally unimplementable (unlikely now that it's been implemented twice) or unusable for its desired use cases, and an incompatible break is the only fix. But I don't think the issue at hand is that serious - we're just talking about potential refinement. The design we have now for canPlayType is not my favorite, but I can't really say it's broken. Not that I except this discussion to go anywhere, but out of curiosity I checked how Firefox/Safari/Chrome actually implement canPlayType: http://wiki.whatwg.org/wiki/Video_type_parameters#Browser_Support Firefox is conservative and honest (except maybe for audio/wav; codecs=0, what could you do with the RIFF DATA chunk?) Safari gets maybe/probably backwards compared to what the spec suggests. Chrome seems to ignore the codecs parameter, claiming probably even for bogus codecs. Authors obviously can't trust the distinction between maybe and probably to any extent. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Codecs for audio and video
(if the buffer has anything, start playing it). I believe the buffered attribute can be used by the ECMAScript loop to detect how much data is buffered, and whether it should continue decoding or take other actions. The buffered audio API should be handled by the media API in a way similar to streaming Web radio. There should be an origin-clean flag, for future use. One might theoretically add audio into a currently playing stream. (regardless of toDataURL support). Does this sound reasonable? What I'm requesting is an append-only raw audio buffer, and an origin-clean flag (similar to canvas) to be added to the audio tag, if not the Media element interface, for future use. The audio buffer plays immediately, if any data is available in it. It sounds like fun to implement and fun to play with, although none of the details really matter at this point (and I shouldn't have asked). When (if) a first browser implements something like this, that API will likely become the standard (unless it's not useful and all other browsers ignore it). In v2, we would discuss Vlad's getAudioSampleData proposal, native channel mixing (mix two audio streams, for whatever reason), and other effects that allow the more complex audio editor use case. For now, let's just consider an audio player to support arbitrary audio codecs and address accessibility for the visually impaired. We need Audio.appendBuffer, Audio.createBufferArray and an AudioBufferArray interface of some sort, and I think it's good to go. The naming and arguments still need to be worked out. I'd enthusiastically support such an interface in Java, Flash and .Net/Active X plugins. For the legacy/IE crowd. -Charles -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Codecs for audio and video
On Wed, 01 Jul 2009 19:01:02 +0200, Philip Jägenstedt phil...@opera.com wrote: On Wed, 01 Jul 2009 18:29:17 +0200, Peter Kasting pkast...@google.com wrote: On Wed, Jul 1, 2009 at 2:41 AM, Anne van Kesteren ann...@opera.com wrote: On Tue, 30 Jun 2009 21:39:05 +0200, Peter Kasting pkast...@google.com wrote: There is no other reason to put a codec in the spec -- the primary reason to spec a behavior (to document vendor consensus) does not apply. Some vendors agreed, and some objected violently is not consensus. The vendor consensus line of argument seems like a very dangerous slippery slope. It would mean that whenever a vendor refuses to implement something it has to be taken out of the specification. I.e. giving a single vendor veto power over the documentation of the Web Platform. Not good at all in my opinion. I am merely echoing Hixie; from his original email in this thread: At the end of the day, the browser vendors have a very effective absolute veto on anything in the browser specs, You mean they have the power to derail a spec? They have the power to not implement the spec, turning the spec from a useful description of implementations into a work of fiction. That's something I would have considered before the advent of Mozilla Firefox. Mozilla also has the power of veto here. For example, if we required that the browsers implement H.264, and Mozilla did not, then the spec would be just as equally fictional as it would be if today we required Theora. My sole goal was to try and point out that the situation with codecs is not equivalent to past cases where vendors merely _hadn't implemented_ part of the spec; in this case vendors have _actively refused_ to implement support for various codecs (Apple with Theora and Mozilla(/Opera?) with H.264). PK That is correct, we consider H.264 to be incompatible with the open web platform due to its patent licensing. For the time being we will support Ogg Vorbis/Theora, which is the best option patent-wise and neck-in-neck with the competition in the quality-per-bit section (especially with recent encoder improvements). We would love to see it as the baseline for HTML5, but in the absense of that hope that the web community will push it hard enough so that it becomes the de-facto standard. A private email has correctly pointed out that neck-in-neck is exaggerating Theora's quality when compared to a state of the art H.264 encoder. It's worth pointing out though that in the immediate future Theora/Vorbis will also be competing against FLV (H.263/MP3), where it compares favorably (especially for audio). Some relevant reading: http://web.mit.edu/xiphmont/Public/theora/demo7.html http://people.xiph.org/~greg/video/ytcompare/comparison.html http://www-cs-faculty.stanford.edu/~nick/theora-soccer/ Previously, I have watched the video's from Greg's ~499kbit/sec comparison of H.264 (as encoded by YouTube) and Theora and it is clear that Theora has more block artifacts and noise overall. However, in the H.264 version when the bunny hole is zoomed in the bunny flicks with a period with about 1 sec, similar to the effect of a keyframe in a dark scene (I haven't inspected if these were actually keyframes). I'd argue that for this clip, which you prefer depends on how you like your artifacts and if you're watching in fullscreen or not (block artifacts are more noticeable when magnified while a keyframe flick is noticeable also at 1:1 magnification). However, this comparison was only made to counter a claim by Chris DiBona and doesn't show that Theora is neck-in-neck with H.264 in the general case. It does however show that it is neck-in-neck with what YouTube produces. I haven't watched the video's of the soccer comparison, but assume that the conclusions are roughly correct. H.264 is a more modern codec than Theora and will outperform it if the best encoders from both sides are used, I don't want to imply anything else. Still, Theora is certainly good enough at this point that the deciding factor is no longer about technical quality, but about finding a common baseline that everyone can support. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Codecs for audio and video
On Wed, 01 Jul 2009 18:29:17 +0200, Peter Kasting pkast...@google.com wrote: On Wed, Jul 1, 2009 at 2:41 AM, Anne van Kesteren ann...@opera.com wrote: On Tue, 30 Jun 2009 21:39:05 +0200, Peter Kasting pkast...@google.com wrote: There is no other reason to put a codec in the spec -- the primary reason to spec a behavior (to document vendor consensus) does not apply. Some vendors agreed, and some objected violently is not consensus. The vendor consensus line of argument seems like a very dangerous slippery slope. It would mean that whenever a vendor refuses to implement something it has to be taken out of the specification. I.e. giving a single vendor veto power over the documentation of the Web Platform. Not good at all in my opinion. I am merely echoing Hixie; from his original email in this thread: At the end of the day, the browser vendors have a very effective absolute veto on anything in the browser specs, You mean they have the power to derail a spec? They have the power to not implement the spec, turning the spec from a useful description of implementations into a work of fiction. That's something I would have considered before the advent of Mozilla Firefox. Mozilla also has the power of veto here. For example, if we required that the browsers implement H.264, and Mozilla did not, then the spec would be just as equally fictional as it would be if today we required Theora. My sole goal was to try and point out that the situation with codecs is not equivalent to past cases where vendors merely _hadn't implemented_ part of the spec; in this case vendors have _actively refused_ to implement support for various codecs (Apple with Theora and Mozilla(/Opera?) with H.264). PK That is correct, we consider H.264 to be incompatible with the open web platform due to its patent licensing. For the time being we will support Ogg Vorbis/Theora, which is the best option patent-wise and neck-in-neck with the competition in the quality-per-bit section (especially with recent encoder improvements). We would love to see it as the baseline for HTML5, but in the absense of that hope that the web community will push it hard enough so that it becomes the de-facto standard. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Frame advance feature for a paused VIDEO
On Tue, 26 May 2009 01:26:38 +0200, Robert O'Callahan rob...@ocallahan.org wrote: On Tue, May 26, 2009 at 11:25 AM, Robert O'Callahan rob...@ocallahan.orgwrote: I don't think there is a standard way to expose the frame rate. We might even want something more general than the frame rate, since conceivably you could have a video format where the interval between frames is variable. It's more than conceivable, actually --- chained Oggs can have the frame rate varying between segments. So if you're at the last frame of one segment the time till the next frame can be different from the time since the previous frame. Rob Indeed, I don't suggest adding an API for exposing the frame rate, I'm just saying that if you know the frame rate by some external means then you can just set currentTime. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] Frame advance feature for a paused VIDEO
On Thu, 21 May 2009 06:35:39 +0200, Biju bijumaill...@gmail.com wrote: I dont see a way to do frame advance feature for a paused VIDEO. Is there a way to achieve that ? As well as frame backward also. Thanks Biju If you pause the video and set currentPosition it should advance to that frame. As long as you know the frame rate you're good to go. All in theory of course, implementations may not be all the way there yet. -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] single step v.play();
On Sat, 18 Apr 2009 10:21:44 +0200, Biju bijumaill...@gmail.com wrote:
if a video element is already playing/loaded video from URL
http://mysite.com/a.ogg
and if we want play another file http://mysite.com/b.ogg
we should do following JS code
v = $('v1');
v.src = http://mysite.com/b.ogg;;
v.load();
v.play();
Why cant it be as simple as
v = $('v1');
v.play(http://mysite.com/b.ogg;);
Similarly for load
v = $('v1');
v.load(http://mysite.com/b.ogg;);
TIA
Since static markup uses the src attribute it needs to be supported via
the DOM, so adding a parameter to load/play would only mean that there are
several ways to do the same thing. I'm not sure replacing an already
playing media resource is an important enough use case to make such a
change to an already complex spec.
--
Philip Jägenstedt
Opera Software
Re: [whatwg] single step v.play();
On Mon, 20 Apr 2009 10:14:14 +0200, Biju bijumaill...@gmail.com wrote:
On Mon, Apr 20, 2009 at 9:37 AM, Philip Jägenstedt phil...@opera.com
wrote:
Since static markup uses the src attribute it needs to be supported via
the
DOM, so adding a parameter to load/play would only mean that there are
several ways to do the same thing. I'm not sure replacing an already
playing
media resource is an important enough use case to make such a change to
an
already complex spec.
I did not mean to remove option for assigning to .src property. Like
v = $('v1');
v.src = http://mysite.com/b.ogg;;
and then at a later time user may do
v.play();
if he want to play it.
Asked for two signature for
v.play() andv.load()
one with the URL as parameter, and other without parameter
That was clear, I'm just saying that it complicates the spec for an
untypical use case (changing the media resource of an already playing
element). Given the already ridiculous complexity of media element's
loading behavior I'm not sure adding more equivalent syntax will help
matters.
As well as an implicit call to v.load() in v.play()
The spec used to have an implicit call to load() from play(), but that has
been replaced with invoke the media element's resource selection
algorithm.
This will make web developers work easier. ie, the js code will become
1/3 most time for the same operation.
Thanks
--
Philip Jägenstedt
Opera Software
Re: [whatwg] Start position of media resources
On Tue, 07 Apr 2009 06:11:51 +0200, Chris Double chris.dou...@double.co.nz wrote: On Tue, Apr 7, 2009 at 3:40 AM, Eric Carlson eric.carl...@apple.com wrote: Media time values are expressed in normal play time (NPT), the absolute position relative to the beginning of the presentation. I don't see mention of this in the spec which is one of the reasons I raised the question. Have I missed it? If not I'd like to see the spec clarified here. Chris. Indeed clarification is needed. In my opinion time 0 should correspond to the beginning of the media resource, no matter what the timeline of the container format says. This means that currentTime doesn't jump when playback begins and never goes beyond duration. Taking Ogg as an example, there's no requirement that the granulepos start at zero nor does a non-zero granulepos imply any special semantics such as the beginning of the file has been chopped off. A tool like oggz-chop might retain the original granulepos of each packet or it could just as well adjust the stream to start at granulepos 0. Neither is more correct than the other, so I'd prefer we not try to infer anything from it, especially since such low-level timing information might be hidden deep inside the platform media framework (if it normalizes the time like XiphQT presumably does). Perhaps we would like to have some way to express where a resource is positioned on a timeline external to the resource itself, but could SMIL do this perhaps? I suppose that an UA could parse the media fragment in the URL and adjust the timeline accordingly, but I'm not sure if it's a great idea... -- Philip Jägenstedt Opera Software
Re: [whatwg] Start position of media resources
On Tue, 07 Apr 2009 10:26:15 +0200, Silvia Pfeiffer silviapfeiff...@gmail.com wrote: On Tue, Apr 7, 2009 at 5:12 PM, Philip Jägenstedt phil...@opera.com wrote: On Tue, 07 Apr 2009 06:11:51 +0200, Chris Double chris.dou...@double.co.nz wrote: On Tue, Apr 7, 2009 at 3:40 AM, Eric Carlson eric.carl...@apple.com wrote: Media time values are expressed in normal play time (NPT), the absolute position relative to the beginning of the presentation. I don't see mention of this in the spec which is one of the reasons I raised the question. Have I missed it? If not I'd like to see the spec clarified here. Chris. Indeed clarification is needed. In my opinion time 0 should correspond to the beginning of the media resource, no matter what the timeline of the container format says. This means that currentTime doesn't jump when playback begins and never goes beyond duration. I humbly disagree. If a media file explicitly knows at what time offset it starts, the timeline needs to represent that, otherwise there are contradictory user experiences. If the media resource really does explicitly define an offset then I might agree that it makes sense to adjust the timeline. However, for plain Ogg or any other container format that just happens to have a non-zero timestamp at the beginning of the file I think we should definitely align them to zero. You can get such files easily by cutting streams and it would be confusing if the timeline was relative to the original file. As an example, in MPEG the PTS (Presentation Time Stamp) can start at non-zero, be discontinuos and wrap around 0 so normalization is necessary. I'm not sure if anyone disagrees, but it would be a very bad idea to infer any semantics from the container time stamps in the absense of some explicit mapping like Ogg Skeleton. Not generally requiring low-level inspection of the container format time stamps is important to us as that would virtually require that the UA itself to demux and inspect the time stamps of different container formats. If a platform media framework is used, time is normalized in some way, likely differently depending on framework and plugins used. For example, take a video that is a subpart of a larger video and has been delivered through a media fragment URI (http://www.w3.org/2008/WebVideo/Fragments/WD-media-fragments-reqs/). When a user watches both, the fragment and the full resource, and both start at 0, he/she will assume they are different resources, when in fact one is just a fragment of the other. Worse still: if he/she tries to send a URI with a link to a specific time offset in the video to a friend, he/she will come up with diverging URIs based on whether he/she watched the fragment or the full resource. Representing the wrong timeline for a media fragment will only cause confusion and inconsistencies. OK, I agree with this. Taking Ogg as an example, there's no requirement that the granulepos start at zero nor does a non-zero granulepos imply any special semantics such as the beginning of the file has been chopped off. A tool like oggz-chop might retain the original granulepos of each packet or it could just as well adjust the stream to start at granulepos 0. Neither is more correct than the other, so I'd prefer we not try to infer anything from it, especially since such low-level timing information might be hidden deep inside the platform media framework (if it normalizes the time like XiphQT presumably does). For Ogg and the definition of Skeleton (http://wiki.xiph.org/index.php/Ogg_Skeleton), both the original basetime of the beginning of the file and the presentation time of the chopped off part are recorded, so it actually does imply special semantics. The most consistent behavior in my opinion would be to report duration as the duration of the whole resource and for buffered/played/seekable to only return ranges within the range indicated by the media fragment. Exposing the start and stop position of the fragment via the DOM seems like overkill to me at this point. -- Philip Jägenstedt Opera Software
Re: [whatwg] video and acceleration
On Sat, 28 Mar 2009 05:57:35 +0100, Benjamin M. Schwartz bmsch...@fas.harvard.edu wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear What, Short: video won't work on slow devices. Help! Long: The video tag has great potential to be useful on low-powered computers and computing devices, where current internet video streaming solutions (such as Adobe's Flash) are too computationally expensive. My personal experience is with OLPC XO-1*, on which Flash (and Gnash) are terribly slow for any purpose, but Theora+Vorbis playback is quite smooth at reasonable resolutions and bitrates. The video standard allows arbitrary manipulations of the video stream within the HTML renderer. To permit this, the initial implementations (such as the one in Firefox 3.5) will perform all video decoding operations on the CPU, including the tremendously expensive YUV-RGB conversion and scaling. This is viable only for moderate resolutions and extremely fast processors. Recognizing this, the Firefox developers expect that the decoding process will eventually be accelerated. However, an accelerated implementation of the video spec inevitably requires a 3D GPU, in order to permit transparent video, blended overlays, and arbitrary rotations. Pure software playback of video looks like a slideshow on the XO, or any device with similar CPU power, achieving 1 or 2 fps. However, these devices typically have a 2D graphics chip that provides video overlay acceleration: 1-bit alpha, YUV-RGB, and simple scaling, all in special-purpose hardware.** Using the overlay (via XVideo on Linux) allows smooth, full-speed playback. THE QUESTION: What is the recommended way to handle the video tag on such hardware? There are two obvious solutions: 0. Implement the spec, and just let it be really slow. 1. Attempt to approximate the correct behavior, given the limitations of the hardware. Make the video appear where it's supposed to appear, and use the 1-bit alpha (dithered?) to blend static items over it. Ignore transparency of the video. Ignore rotations, etc. 2. Ignore the HTML context. Show the video in manners more suitable to the user (e.g. full-screen or in an independent resizable window). Which is preferable? Is it worth specifying a preferred behavior? In the typical case a simple hardware overlay correctly positioned could be used, but there will always be a need for a software fallback when rotation, filters, etc are used. Like Robert O'Callahan said, a user agent would need to detect when it is safe to use hardware acceleration and use it only then. If there is something that could be changed in the spec to make things a bit easier for user agents it might be an overlay attribute, just like SVG has: http://www.w3.org/TR/SVGTiny12/multimedia.html#compositingBehaviorAttribute I'm not convinced such an attribute would help, just pointing it out here... -- Philip Jägenstedt Opera Software
Re: [whatwg] Video playback quality metric
On Mon, 09 Feb 2009 22:13:09 +0100, Jeremy Doig jerem...@google.com wrote: Measuring the rate at which the playback buffer is filling/emptying gives a fair indication of network goodput, but there does not appear to be a way to measure just how well the client is playing the video itself. If I have a wimpy machine behind a fat network connection, you may flood me with HD that I just can't play very well. The cpu or video card may just not be able to render the video well.Exposing a metric (eg: Dropped Frame count, rendered frame rate) would allow sites to dynamically adjust the video which is being sent to a client [eg: switch the url to a differently encoded file] and thereby optimize the playback experience. Anyone else think this would be good to have ? While I think this kind of thing might be useful, I would be careful about requiring any kind of detailed metrics like dropped frames or effective frame rate to be exposed via DOM, as getting this information reliably over different devices, platforms and media frameworks would be quite difficult. How about an event which the user agent can optionally fire to indicate that it cannot play at the requested rate due to processing/memory constraints (rather than network)? This would (I think) provide the same functionality but put less burden on implementors. There is already a placeholder for non-fatal errors in the spec, perhaps this could be included with that in some fashion? -- Philip Jägenstedt Opera Software
Re: [whatwg] Adding and removing media source elements
On Tue, 03 Feb 2009 05:44:07 +0100, Ian Hickson i...@hixie.ch wrote:
On Tue, 3 Feb 2009, Chris Pearce wrote:
(2) Why don't we invoke load() whenever a media element's src attribute
or source children are changed, regardless of networkState? That way
changes to the media's src/source other than the first change would have
the same effect as first change, i.e. they'd have an immediate effect,
causing load() to be invoked.
Doing this would cause the first file to be downloaded multiple times in
a
row, leading to excessive network usage.
Surely this can't be the only reason? User agents are free to
speculatively keep the current source loading when src/source changes and
to stop loading it only if the current media resource does change. That,
and caching, should be enough.
I have always imagined that the reason for the conditioned load() is to
not interrupt playback by fiddling with the DOM or doing something like
v.src=v.src (although I'm quite sure that doesn't count as changing the
attribute). However, now I can't convince myself that this makes any
sense, since surely if you change src/source you actually do want to
change the effective source (and load() is scheduled to run after the
current script, so there's no risk of it being run too early).
Related, since load() is async it depends on timing whether or not
video id=v/video
script
v = document.getElementById('v');
v.src = 'test';
/script
causes the source 'test' to be loaded, as the network state may not be
NETWORK_EMPTY when the src attribute is set. The same goes for adding
source child elements of course. Yes, this is the same issue as
http://lists.w3.org/Archives/Public/public-html/2009Jan/0103.html and
would be resolved by calling load() unconditionally.
--
Philip Jägenstedt
Opera Software
Re: [whatwg] Media elements delaying the load event.
On Thu, 2008-12-18 at 14:41 +1300, Robert O'Callahan wrote: Currently HTML5 does not require media elements to delay the load event in any way. We certainly don't want to delay the load event until the entire resource has finished loading (as we do for images), but I think it would make sense to delay the load event until media elements that are loading have reached the HAVE_CURRENT_DATA state (or detected an error). This ensures intrinsic sizes are available so the layout can stabilize, it ensures that scripts can access metadata, and it ensures that the video is displaying something (if it's going to). Surely the load event should not be sent until the whole resource is loaded, otherwise it would clash rather severely with how progress events are defined. Is there some problem with holding the load event until the time when the resource is actually loaded? -- Philip Jägenstedt Opera Software
Re: [whatwg] video tag: pixel aspect ratio
On Mon, 2008-12-01 at 12:39 +, Ian Hickson wrote: On Mon, 1 Dec 2008, Philip Jgenstedt wrote: Now that the pixelratio override is gone, shouldn't the influence of pixel aspect ratio on the layout be removed also? It is, isn't it? What did I leave in? Video content should be rendered inside the element's playback area such that the video content is shown centered in the playback area at the largest possible size that fits completely within it, with the video content's aspect ratio being preserved. Thus, if the aspect ratio of the playback area does not match the aspect ratio of the video, the video will be shown letterboxed or pillarboxed. Areas of the element's playback area that do not contain the video represent nothing. I would prefer if the default were to stretch to fit if both width/height are given, just like for img. Letterboxing/pillarboxing should be the special case, is the idea that we should have the equivalent of SVG:s preserveAspectRatio either via CSS or HTML? We definitely don't want to stretch the video. One of the important use cases if having a video playback area and then playing videos with different aspect ratios in that playback area. It should all just work. I'm having a hard time seeing how this is a use case for video and not for img. If one wants the intrinsic width/height to be used, then simply don't set width/height. Otherwise, just setting just one of width/height or using CSS max-width/max-height should do the trick. This is strange: video src=circle.mpg width=400 height=400 !-- circle -- video src=circle.mpg width=400 height=300 !-- pillarbox -- img src=circle.jpg width=400 height=400 !-- circle -- img src=circle.jpg width=400 height=300 !-- oval -- I think it would be much more consistent if these elements behaved in the same way. As it stands, after the pixelratio was removed, there is no way to display a circle as an oval or vice versa, which means it is both impossible to fix an incorrect aspect ratio or to stretch the video for some other purpose. So, what is the difference between images and moving images? -- Philip Jägenstedt Opera Software
Re: [whatwg] video tag: pixel aspect ratio
On Mon, 2008-12-01 at 18:19 +0100, Lachlan Hunt wrote: Philip Jägenstedt wrote: On Mon, 2008-12-01 at 12:39 +, Ian Hickson wrote: We definitely don't want to stretch the video. One of the important use cases if having a video playback area and then playing videos with different aspect ratios in that playback area. It should all just work. I'm having a hard time seeing how this is a use case for video and not for img. If one wants the intrinsic width/height to be used, then simply don't set width/height. Otherwise, just setting just one of width/height or using CSS max-width/max-height should do the trick. This is strange: video src=circle.mpg width=400 height=400 !-- circle -- video src=circle.mpg width=400 height=300 !-- pillarbox -- This is effectively how YouTube behaves now with their recent change to a widescreen player. It would look terrible if 4:3 videos were stretched to fill the 16:9 viewport, instead of just using black bars on the side. Even before when they were still using a 4:3 player, widescreen videos were rendered as letterbox. The strange part isn't that the video is pillarboxed, but that it's impossible to not do it and that it's inconsistent with img. img src=circle.jpg width=400 height=400 !-- circle -- img src=circle.jpg width=400 height=300 !-- oval -- I think it would be much more consistent if these elements behaved in the same way. What is the use case for wanting a video to be stretched? The use case for stretching moving images (video) is exactly the same as for stretching animations (img src=animation.gif) or static images (img src=static.jpg). While I think that fixing incorrect aspect ratio is the most probable use, consistency and generality is the real issue here. With an image of size 800x600 it's possible to display at any size, even those which don't match the aspect 4:3. With moving images (video) we can't influence it at all. The previous solution was the pixelratio attribute, but I have 2 other possible solutions: video src=4x3.mpg width=1920 height=1080 keepaspect The video would be pillarboxed. video src=4x3.mpg width=1920 height=1080 The video would be stretched and have incorrect aspect, just like img src=4x3.jpg width=1920 height=1080 This way it's easy to stretch or to keep aspect, whichever you want. Given the number of people who watch 4:3 video stretched to a 16:9 display without even noticing/caring that the aspect ratio is wrong, I don't think we can trust that the aspect ratio of all videos is always going to be correctly encoded and simple say that this should be fixed by reencoding the video -- that's not even an option if the source file is not available and reencoding is good for quality. But to reiterate, this is mainly about generality and consistency. Are there any other suggestions? Could this perhaps be done with CSS so that the same thing could be done with img? It is in fact rather difficult to scale something to fit inside a box using only CSS right now. It would be rather sad to see img and video be so inconsistent when -- Philip Jägenstedt Opera Software
Re: [whatwg] video tag: pixel aspect ratio
On Mon, 2008-12-01 at 23:07 +0100, Lachlan Hunt wrote: Given the number of people who watch 4:3 video stretched to a 16:9 display without even noticing/caring that the aspect ratio is wrong, While I'm aware that there are such people, largely because they don't know how to configure their TV's properly or because their TV's are broken by design, there are many others who do notice the difference (including myself). From my experience, stretching a 4:3 picture to fill a 16:9 screen is enough to make people on the screen look weird and out of proportion. It's less noticeable with cartoons, but in general, it's very noticeable and annoying. We should inflict such annoyances upon end users if it avoidable; at least not by default. The point wasn't that it is OK for UA:s to distort video because there are people who don't notice, but that some people do/will encode video with the wrong aspect ratio without noticing/caring. The general consensus seems to be that this isn't worth the effort fixing. But in any event, if a CSS solution is in fact possible that would be much better than any video-specific solution. I don't think CSS transforms can do it though, unless someone can give an example? -- Philip Jägenstedt Opera Software
Re: [whatwg] media elements: Relative seeking
Relative seeking is always possible: currentTime+=10. Proportional seeking is possible as long as the duration is known and finite. I would avoid as far as possible any use of byte positions as this information isn't exactly trivial to get given all the buffering and asynchronous decoding involved in typical media frameworks. Philip On Tue, 2008-11-25 at 08:58 +0100, Maik Merten wrote: Dave Singer schrieb: I don't think you mean 'relative' here, which I would take to be go forward 10 seconds, but 'proportional', please go to 60% of the way through. Right, proportional for sure is the correct word for what I had in mind. Thanks. IF we are to do this, I would have thought it would be by adding units to the where to seek to argument: * go to this time in NPT (normal play time, which runs from 0 to media duration) * go to this SMPTE time code * go by this relative distance in NPT times * go to this proportional time * go to this proportional byte distance * go by this relative byte distance Hmmm... I'm in favor of of making implementations more simple (and thus more robust in this case), so I think inflating the number of ways of how one can seek may be going into the wrong direction. Note that proportional distances are not well defined for some streams (e.g. indefinite ones). Okay, this use case basically rules out just seeking by values between zero and one. Even with indefinite streams the user may want to e.g. jump to second 20 of the stream, which won't work with the proportional seeking I asked for. We'd have to define what bytes are counted and what aren't, especially if a URL offers a set of sub-streams only some of which a client would normally choose to have sent to it for playing. Oh my, that's true and it isn't pretty. I'm currently slamming a subset of the HTML5 media API onto a Java applet (to offer a fallback for browsers without audio/video). http://people.xiph.org/~maikmerten/demos/bigbuckbunny-applet-javascript.html Estimating the duration *does* work - poorly in this case, but still. Currently this applet uses the byte-position of the last byte fetched from the server, which isn't the correct *playback* byte-position due to not accounting for the bytes in the buffer (which, in this case, is 4 megabytes big - so this is severe). I assume that once this is corrected a reasonable-enough (for a status bar position slider) estimate may actually be possible, meaning things could just stay the way they are. Maik -- Philip Jägenstedt Opera Software
Re: [whatwg] video tag: pixel aspect ratio
On Mon, 2008-11-17 at 22:28 -0800, Sander van Zoest wrote: Depending on how you want to accomplish you can do that with an enum that defines how to handle the case: 1) do nothing. 2) disproportionately adjust 3) stretch follow by letter-,pillar-,windowbox appropriately. 4) complete fill the screen and ignore overscan (ala pan scan or zoom) The spec says 3, but if stretch behaviour is wanted I guess one could consider doing something like SVG:s preserveAspectRatio attribute[1] Oh, and if the video also needs cropping to get the correct aperture, couldn't this be done with CSS perhaps? Wow. How would you know how much to crop for each video clip? Wouldn't specifying the enum above do the trick? I don't know, is aperture encoded in the media file itself or is it implicit in certain types of video formats that aperture cropping (or whatever the technical term is) should be applied? If you should always crop e.g. 8 px on left and right then a CSS container with some overflow tricks would certainly work. If on the other hand it's given in the media file and it's likely to be incorrectly set then perhaps we do need an override for it (although it's likely that such cropping is performed by the video decoder and not a subsequent crop filter so it might not be so trivial to fix in all media frameworks). [1] http://www.w3.org/TR/SVG/coords.html#PreserveAspectRatioAttribute -- Philip Jägenstedt Opera Software
Re: [whatwg] video tag: pixel aspect ratio
I should point out that the pixelratio attribute isn't only for authors, it's also useful when the media framework used doesn't recognize the (pixel) aspect ratio even when it's correctly set. From reading the mplayer man page I see that AVI files can have aspect ratio set in the OpenDML vprp header, but I doubt e.g. DirectShow supports this (I could be wrong though). I don't feel very strongly about the attribute either way, but given that video is scaled to fit inside its element box with aspect preserved and not simply stretched then an incorrectly parsed/guessed aspect ratio would make a big difference. This seems very similar to the width/height attributes of an image and that usually isn't considered an ugly hack. If the pixelratio attribute is removed then I would suggest also removing the involvement of aspect ratio over the size of the element. By the way, the pixel-aspect-ratio on video caps in the GStreamer framework has precisely the same meaning as this attribute, overriding it on a video sink also has an effect similar to what is suggested in the HTML5 spec. In other words, it's not so outlanding from a media framework's point of view. Oh, and if the video also needs cropping to get the correct aperture, couldn't this be done with CSS perhaps? Philip On Mon, 2008-11-17 at 14:02 -0800, Peter Kasting wrote: On Mon, Nov 17, 2008 at 1:58 PM, Pierre-Olivier Latour [EMAIL PROTECTED] wrote: 1) I don't remember any major media system I've dealt with so far having an explicit pixel aspect ratio override API, 2) on the web, neither QT plug-in nor Flash have it, 3) in the case of this spec, the way it's defined makes it behave incorrectly 4) it's not straightforward to use (see very first reply above) 5) there's no _actual_ data that proves it's necessary (shouldn't the software or video web site fix the videos upfront?) Based on this, it seems to me this attribute should not be in the spec by default, and we should switch the burden of the proof to people who want it (rather than it being on people who don't want it as it seems to be the case today), and finally wait to see 1) if there's a real need for a solution here and 2) if the best solution is indeed a pixel aspect ratio override. I agree. The more this attribute is discussed, the more this is a hack that no one should actually use starts to sound like we shouldn't put this in the spec to begin with. The potential for problems seems greater than the upside from authors correctly using this to do emergency-overrides of particular videos whose sources they don't control. PK -- Philip Jägenstedt Opera Software
Re: [whatwg] Can AUDIO/VIDEO element have a balance attribute
While I don't strongly object to the suggestion as such, there are 2 things I want to point out. 1. It is not clear (to me at least) what balance means for anything by mono/stereo audio. What is the expected behaviour for 5.1 audio? 2. Balance is just one kind of audio filter effect. The more filter features are added the tougher the requirements on the user agent and the media framework backend they use will be. Is there a use case for having the same LARGE media file played back with different balance settings? All the obvious use cases would be short sound effects where creating duplicate versions of the files might be a better trade-off than introducing more complexity in the API. Philip On Sun, 2008-11-16 at 03:59 -0400, Biju [EMAIL PROTECTED] wrote: On Sun, Nov 16, 2008 at 12:50 AM, Nils Dagsson Moskopp [EMAIL PROTECTED] wrote: Not to be rude, but can you desribe a use case that is not some kind of game ? It can be educational or game sites for kids. Like http://www.starfall.com/ http://pbskids.org/ Some where at these sites I saw a flash tool to build music. So having control on balance can enhance a music composing training page.. One of thing a blind person cant do is using mouse on computer. Because mouse need feed back through eye to use it. (unlike old technologies like Digitizer Stylus, Light Pen etc) I have a little ambitious idea, We can help blind person to use mouse if we have ability to control balance and volume. ie, * 1 audio source and balance can give left and right position on screen. * 2 audio source, one to indicate top and other for bottom along with balance and volume control can give 2D positioning on screen. And 3 audio source, third one for depth can give 3D positioning. Which solves the zooming effect on google map. -- Philip Jägenstedt Opera Software
Re: [whatwg] Reworking the media load() algorithm so that it tries more than one video
The changes look like improvements to me, but I have some questions. 1. Is there any particular reason why step 7 generate the list of potential media resources is done in the synchronous part of the algorithm? Would it for example be an error to consider playing sources added in a loadstart event handler? As it stands one would have to actually make a list of candidate URLS and pixel ratio overrides up front. I don't really object, I just want to confirm that this is intentional. 2. Did you remove canPlayType? It was added in http://html5.org/tools/web-apps-tracker?from=2331to=2332 and now it seems to be gone, but I can't find the revision in which that happened. 3. MEDIA_ERR_DECODE isn't just for corrupted data. If source type=application/ogg is given and the browser doesn't know it cannot render it but it turns out that it couldn't decode any of the content (perhaps it was Ogg FLAC), isn't MEDIA_ERR_DECODE precisely what should happen? In other words, the word corrupted ought to be unrecognized, not decodable or something which doesn't imply that the media resource itself is at fault. Philip On Fri, 2008-11-14 at 06:51 +, Ian Hickson wrote: On Thu, 16 Oct 2008, Robert O'Callahan wrote: I am proposing changing the spec to make the load() not select a resource. Instead, load() would immediately return, and the user agent would then asynchronously try each resource that it might otherwise use in turn, firing a single loadstart event at the start, lots of progress events, and then a loadedmetadata or error event depending on how things go. That sounds good. I've now done this. This is a pretty big change. I'm sure I've screwed something up in the process. Implementors! Please let me know if you see anything newly broken with loading resources! Thanks. :-) Cheers, -- Philip Jägenstedt Opera Software
Re: [whatwg] Scripted querying of video capabilities
I'm also a bit concerned about how to interpret the yes, no and maybe return values. The truthful answer is going to be maybe for all but the obviously unsupport (application/x-ms-dos-executable) and the more trivial formats (audio/wav). When asking about application/ogg, this could mean 2 things: 1. can I demux Ogg 2. can I demux Ogg and decode an unknown codec If to answer yes the browser must be certain that it can play back all content which could have the requested mime type then the answer must always be maybe for container formats, e.g. video/avi, video/x-msvideo, application/ogg, video/mp4, video/mpeg and so on. Only if the codecs parameter is used is it ever possible to anwer yes It's hardly ever possible to answer no either, unless the underlying media framework explicitly doesn't support a When faced with something unrecognized the safest thing is to say maybe. To sum up, the answer is always maybe. This is nice for implementors who don't need to write a lot of code, but it's not useful. To fix this, I would suggest the following: If the mime type is just a container format then the browser may answer yes if it knows that it supports at least some content in that format (again, it's difficult without On Thu, 2008-11-13 at 10:52 -0800, Jeremy Doig wrote: did this thread go anywhere ? i'm concerned about the maybe case - looks way too much like: http://en.wikipedia.org/wiki/DShow#Codec_hell also - when you probe for mime type, do you mean the entire type parameter (including the codecs string) ? for example, there are too many cases where just passing video/mp4 would be insufficient. (fragmented index support ? base/main/high profile ? paff ? cabac ?) source src=video.mp4 type=video/mp4; codecs=quot;avc1.42E01E, mp4a.40.2quot; On Wed, Oct 15, 2008 at 11:14 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: On Oct 15, 2008, at 1:44 AM, Ian Hickson wrote: On Tue, 14 Oct 2008, Robert O'Callahan wrote: On Tue, Oct 14, 2008 at 12:13 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: While the underlying media frameworks can't necessarily answer, if I give you a file with this MIME type, can you play it?, they can at least give a yes/no/maybe answer, which can still be quite helpful, since the UA will know it does not need to check some media streams at all. I agree. If the API lets us answer maybe, there is not much need or temptation to lie, and we can still return information that could be useful to scripts. I have added window.navigator.canPlayType(mimeType). It returns 1, 0, or -1 to represent positive, neutral, and negative responses. This API would be tempting to treat as a boolean but would of course do completely the wrong thing. I think it would be better to either ensure that the positive and neutral responses are both values that JS would treat as true (for instance make the values true, maybe and false), or else make all of the return values something self-descriptive and symbolic (for instance the strings yes, maybe and no). I think 1, 0, -1 are neither clear nor likely to be in any way beneficial for perforamnce. Regards, Maciej -- Philip Jägenstedt Opera Software
Re: [whatwg] Same-origin checking for media elements
I don't quite see why one should betray the size of media data and not other data. Surely the existance and size of media files on an intranet could be sensitive information too, so a more general solution is needed in my opinion. Am I misunderstanding what is being suggested? Philip On Wed, 2008-11-12 at 23:58 +1300, Robert O'Callahan wrote: On Wed, Nov 12, 2008 at 11:19 PM, Jonas Sicking [EMAIL PROTECTED] wrote: An additional, though rather minor problem, is that implementations will have to delay the loadstart event until it has confirmed that the targeted file is in fact a real video file, and has confirmed that with relatively high level of confidence. Otherwise the size of random HTML files can be measured using the video element. We can fire loadstart normally and return 0 in the loaded attribute and unknown in the total attribute (however that should be encoded). (We have to be able to return unknown since there can be situations, e.g. live streams, where we'll never know the total size.) -- Philip Jägenstedt Opera Software
[whatwg] HTMLMediaElement: static TimeRanges/ByteRanges, potentially playing definition
More random issues related to HTMLMediaElement http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-media-buffered The buffered attribute must return a static normalized TimeRanges object The phrasing for the other ranges attributes is similar, but what does static mean in this context. I'm guessing that it means static in the sense that the same object is returned each time. As a consequence you can't keep a copy of an old ranges object to compare it with a new one later. This is all very fine, I just want to verify that it's correct. http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#potentially-playing A media element is said to be potentially playing when its paused attribute is false, the readyState attribute is either HAVE_FUTURE_DATA or HAVE_FUTURE_DATA, the element has not ended playback, playback has not stopped due to errors, and the element has not paused for user interaction. There's a typo here with two HAVE_FUTURE_DATA. I'm not sure what the intention was, but it seems to me that readyState shouldn't be involved here. Consider: new Audio(foo.wav).play(); Media elements that are potentially playing while not in a Document must not play any video, but should play any audio component. Media elements must not stop playing just because all references to them have been removed... Immediately after the above call has returned, the readyState of the media element is HAVE_NOTHING unless the resource happened to be cached. The definition of potentially playing must be such that this object isn't immediately garbage collected. Suggested definition: A media element is said to be potentially playing when its paused attribute is false, the element has not ended playback, playback has not stopped due to errors. In other words, remove the readyState and paused for user interaction stuff (surely a media element could potentially keep playing after such user interaction, otherwise there would be no need to ask). (Also, I'm still wondering about the issues in my previous mail.) -- Philip Jägenstedt Opera Software
Re: [whatwg] HTMLMediaElement: static TimeRanges/ByteRanges
On Thu, 2008-11-06 at 16:55 -0500, Jim Jewett wrote: Philip Jägenstedt wrote: http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-media-buffered The buffered attribute must return a static normalized TimeRanges object ... what does static mean in this context. I'm guessing that it means static in the sense that the same object is returned each time. I would have assumed exactly the opposite -- that static meant it wasn't live, and would not be updated as the buffering state and playing continued. Actually, this does makes more sense. Otherwise ranges might be merged as loading progresses with confusing results and exceptions. In any event, since there were different interpretations it's probably best to clarify this in the spec. -- Philip Jägenstedt Opera Software
[whatwg] HTMLMediaElement: pixelratio, unsigned int, timeupdate
Some assorted questions related to http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-source-pixelratio How should a pixelratio = 0.0 be treated? I would suggest that the DOM attribute actually return the negative value but that it is explicitly ignored in step 5 of http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#pick-a-media-resource What should be done if a reflected unsigned integer DOM attribute is set to a negative number? INDEX_SIZE_ERR, ignored or something else? I'm quite sure it's well defined, just not in http://www.whatwg.org/specs/web-apps/current-work/multipage/infrastructure.html#reflect In the section about cue ranges (can't seem to find an #anchor nearby): If the time was reached through the usual monotonic increase of the current playback position during normal playback, the user agent must then queue a task to fire a simple event called timeupdate at the element. How often should the timeupdate event be fired? Without a requirement for this, anything from every ms to never would seem to be acceptable. I agree with Eric Carlson and Jonas Sickling that this event isn't particularly useful and would rather see it removed from the spec. -- Philip Jägenstedt Opera Software
[whatwg] video implicitly invoke the load() method
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#load There are a number of conditions for media elements where the user agent must implicitly invoke the load() method on the media element as soon as all other scripts have finished executing: * a source element is inserted into a media element that is already in a document and whose networkState is in the EMPTY state * the src attribute of a media element that is already in a document and whose networkState is in the EMPTY state is added, changed, or removed * a media element whose networkState has the value EMPTY is inserted into a document Now consider this markup video src=foo/video with this script dynamically changing the sources video = document.getElementById(video); video.removeAttribute(src); // implicit delayed load source = document.createElement(source); source.src = bar; source.media = projection; video.appendChild(source); // implicit delayed load source = document.createElement(source); source.src = baz; source.media = screen; video.appendChild(source); // implicit delayed load video.play() // i want to play now! play() will call load() before returning, so after this has finished there shouldn't be any need to load() any more. However, there is no less than 3 calls to load() waiting. Playback will begin, but will stop as soon as the first of those delayed loads are invoked. I believe that the spec actually may allow the N implicit loads to be treated as 1, but at the very least there must be that 1, which is still too many in this case. I doubt many authors will realize all the things which may cause an implicit load, so I think the spec needs to allow for this to be made simpler (from the script author's point of view). More importantly, there's absolutely no need to implicitly call load once its been explicitly called once. I would suggest the following solution: If when the time comes to invoke the load() method it has already been invoked (directly or via another implicit invocation triggered by the same script) then don't call it again. Comments? I'm not sure exactly how to change the spec, but if we agree that this needs fixing we can come back to that. -- Philip Jägenstedt Opera Software
Re: [whatwg] Video : Slow motion, fast forward effects
I suggest that the spec allows raising the NOT_SUPPORTED_ERR exception in response to any playback rate which it cannot provide for the current configuration. With a netcast you couldn't support any playback rate except 1.0 without first buffering all the data you want to play at a faster rate, so changing the playback rate doesn't make sense. Throwing NOT_SUPPORTED_ERR must be better than just ignoring it, but the question is if script authors will remember to check for exceptions when setting the attribute... Philip On Thu, 2008-08-07 at 10:28 +0200, Maik Merten wrote: Chris Double schrieb: Given that codecs are often highly optimized for forward playback the cost in memory can be excessive to go the other way. Could there be a possibility to say 'reverse playback not supported'? This may be needed anyway, given that with streaming media you can't reverse playback, even if the codec would allow it easily (which is not the case for MPEG, VC-1, Theora, Dirac, your favorite codec here). An interesting case would also be how to handle playback speeds smaller than 1x in the streaming case, given that you cannot possibly have an infinite buffer of input data. Streaming mostly forces a playback speed of +1x in all cases. Maik -- Philip Jägenstedt Opera Software
Re: [whatwg] VIDEO tag height/width best fit
Video is automatically scaled to fit inside its box while maintaining the correct aspect ratio. Philip On Mon, 2008-08-04 at 00:19 -0400, Biju [EMAIL PROTECTED] wrote: I apologize if I did not understood http://www.whatwg.org/specs/web-apps/current-work/#video I see a situation where we want to specify the height and width of the BOX in which a video need to be displayed. And while writing HTML code we dont know the height/width of the video. But when playing we want the video to be displayed with maximum size which can fit in the BOX without loosing the aspect ratio. After reading the video tag specification I dont see a way to achieve this with out javascript code. We should have one more attribute to tell how to fit the image/video == fit attribute values === * center - if it is small keep at center * stretch - stretch/skew height and width to fit * stretch-height - stretch/skew height only to fit, adjust width to maintain aspect ratio. * stretch-width - stretch/skew width only to fit, adjust height to maintain aspect ratio. * stretch-best- stretch/skew height or width to maximum so that other will fit in and maintain aspect ratio. May be we can also have a tile option This problem also exist for img tag. -- Philip Jägenstedt Opera Software
Re: [whatwg] re-thinking cue ranges
On Tue, 2008-07-22 at 09:58 +, Ian Hickson wrote: On Wed, 9 Jul 2008, Dave Singer wrote: On Sat, 12 Jul 2008, Philip Jgenstedt wrote: Like Dave, I am not terribly enthusiastic about the current cue ranges spec, which strikes me adding a fair amount of complexity and yet doesn't solve the basic use case in straightforward manner. What are the use cases you think are basic? It's unclear to me what isn't being solved. Here's one use case, a slide deck: The most obvious use case in my mind is displaying captions/subtitles. I agree that proper events make a lot of sense here instead of callbacks. We could use some new event -- CueEvent maybe -- which would actually include the start and stop times and a reference to the target HTMLMediaElement. I might suggest a modified addCueRange which takes a data argument which is also passed along in the event object. Does the identifier argument address this sufficiently? Yes, it makes sense and should eliminate the need for closures in most cases. -- Philip Jägenstedt Opera Software
Re: [whatwg] HTMLMediaElement: more issues and ambiguities
On Tue, 2008-07-22 at 21:58 +, Ian Hickson wrote: On Thu, 10 Jul 2008, Philip Jgenstedt wrote: I understand what the intention is, but think the terminology is confusing: [snip] I've tried to make the spec generally much more explicit about this. Please let me know if the spec makes more sense now. I can definitely still change the terminology if you think it is still confusing. Thanks, that looks much better. -- Philip Jägenstedt Opera Software
Re: [whatwg] re-thinking cue ranges
On Tue, 2008-07-22 at 22:00 +, Ian Hickson wrote: On Tue, 22 Jul 2008, Philip Jgenstedt wrote: What are the use cases you think are basic? It's unclear to me what isn't being solved. Here's one use case, a slide deck: The most obvious use case in my mind is displaying captions/subtitles. I'd much, much ratio subtitles were done by the user agent natively based on captions actually included in the video data. We shouldn't rely on authors to provide accessibility features. Given how unreliable embedded subtitles tend to be in desktop media players (at least in my experience) I think it's very likely someone will write an JavaScript SRT parser library to use with this API rather than hoping that the embedded subtitles can be reliably detected in all different combinations of media frameworks and browsers. I guess standardizing on an embedded caption/subtitle format might be possible after we actually have decided on baseline codecs though... Having said that, changing the code I gave in my last e-mail to support captions is pretty trivial -- simply add an exit callback that empties the current subtitles display. The rest is basically the same. Indeed, I expect that some would even abuse the id parameter to pass the caption text directly, although that isn't very elegant. -- Philip Jägenstedt Opera Software
Re: [whatwg] re-thinking cue ranges
that they can have an ID and make an event * adding a cookie/rangeID to the current API would help... * adding an attribute to source called annotations which could point at a variety of types, including at an XML file (to be defined) which contains meta-data, cue-range definitions etc., as if they were part of the media, would help move this out of the HTML5 but still provide a uniform interface... example source src=myMovie.mp4 annotations=myMovie-tags.xml / then if the annotations should be got from the media resource itself, the notation source src=myMovie.mp4 annotations=myMovie.mp4 / could be used, and source src=myMovie.mp4 / would be equivalent. we could even use source src=myMovie.mp4 annotations= / to explicitly defeat the retrieval of annotations. (Such an annotations href might also help with associating metadata with media resources, particularly when the same metadata should be associated with a set of sources that differ in bitrate, codec, etc.). -- Philip Jägenstedt Opera Software
Re: [whatwg] HTMLMediaElement: more issues and ambiguities
Thanks for addressing all of my questions, there is only one issue below which I think deserves a second round. On Wed, 2008-07-09 at 00:05 +, Ian Hickson wrote: On Thu, 3 Jul 2008, Philip Jägenstedt wrote: http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#adjusted If the video's pixel ratio override's is none, then the video's intrinsic width is the width given by the resource itself. If the video has a pixel ratio override other than none, then the intrinsic width of the video is the width given by the resource itself, divided by the pixel ratio given by the resource itself, multiplied by the video's pixel ratio override. This is a pixel ratio *override*, suggest changing it to: If the video's pixel ratio override's is none, then the video's intrinsic width is the width given by the resource itself multiplied by the pixel ratio given by the resource itself. If the video has a pixel ratio override other than none, then the intrinsic width of the video is the width given by the resource itself multiplied by the video's pixel ratio override. The idea is that if you set the override to a 1:2 ratio, then each pixel of video data will be rendered 1:2. So you first have to normalise the width, getting rid of the influence of the official pixel ratio. No? I understand what the intention is, but think the terminology is confusing: - width given by the resource itself - height given by the resource itself - pixel ratio given by the resource itself I had thought that these 3 were actually orthogonal, such that the pixel ratio does not affect the width or the height. Instead, it seems that width/height is supposed to be aspect ratio pre-multiplied. This doesn't sit well with my intuition of what the self-given width/height of video is supposed to mean. If you open video in common media players the dimensions or width/height will be the physical width/height, not aspect-corrected width/height. Unless my intuition is severely broken, I think other will also assume what I have assumed. So again, I suggest that the paragraph be changed: If the video's pixel ratio override's is none, then the video's intrinsic width is the width given by the resource itself multiplied by the pixel ratio given by the resource itself. If the video has a pixel ratio override other than none, then the intrinsic width of the video is the width given by the resource itself multiplied by the video's pixel ratio override. Yes, it implicitly redefines width/height given by the resource itself to mean what they sound like. Strictly this really only matters to implementors, but less confusing language is good. Incidentally it is more in line with how it would actually be implemented -- one would hardly waste floating point calculations by multiplying by pixel ratio, dividing by pixel ratio and then multiplying it by override pixel ratio. -- Philip Jägenstedt Opera Software
[whatwg] HTMLMediaElement: more issues and ambiguities
and insert into another without it breaking, but there are other consequences too. Media elements must not stop playing just because all references to them have been removed can be taken to mean that media should keep playing even when the user navigates away from the containing document, which doesn't seem very nice as there would be no way to stop it. If the media element's ownerDocument stops being an active document, then the playback will stop until the document is active again. Why? If someone wants to listen to music in the background, why should we stop them? It seems to me that this should be up to implementors to decide, possibly stopping background media elements if they are consuming too much resources, etc. This note is a consequence of When a media element is actively playing and its owner Document is an active document, its current playback position must increase monotonically at playbackRate units of media time per unit time of wall clock time. I suggest simply removing its owner Document is an active document, possibly adding that user agents may or may not stop media elements which are playing the background (per the user setting, possibly). http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#voidcallback As discussed with Hixie on IRC, this is actually the WebIDL way of describing a function with no parameters or return value, so a note that user agents *shouldn't* accept objects with handleEvent methods would be in order. http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#volume Just to reiterate the request that the default volume be 1.0. OK, that's all folks! -- Philip Jägenstedt Opera Software
[whatwg] HTMLMediaElement load and autoplay
Keeping video discussion on the whatwg list as Firefox and Apple implementors seem to be listening here... I have some issues with the load method and when it is invoked. http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#load As far as I can see, there are 6 distinct ways that the method can be invoked: Immediate: - User invoked the DOM load() method - In the play() method, if the media element's networkState attribute has the value EMPTY - In the pause() method, if the media element's networkState attribute has the value EMPTY As soon as all other scripts have finished executing: - If a media element whose networkState has the value EMPTY is inserted into a document - If the src attribute of a media element that is already in a document and whose networkState is in the EMPTY state is added, changed, or removed - If a source element is inserted into a media element that is already in a document and whose networkState is in the EMPTY state It is not clear if the user agent should automatically invoke the load method on media elements which are in the document's static markup. Is it supposed to be implied by media element ... inserted into a document? As far as I can tell, there is no other way the autoplay mechanism could work. If automatic loading is the intention, at what time is it supposed to happen? I suggest: as soon as the DOM is in a consistent state, after the DOMContentLoaded event is fired (where supported). The reason being of course that media content isn't DOM content as such and DOMContentLoaded shouldn't have to wait longer than necessary. I think this is the best option, as it is consistent -- load is always invoked automatically irrespective of if the media element was in the markup or inserted via DOM. If user agents don't want to waste time downloading video that isn't visible (e.g. due to CSS display:none) they can block the download (step 14 of the load algorithm) until it becomes visible. In this situation, does the spec allow firing the stalled event immediately instead of waiting 3 seconds? Since faking it is allowed, script authors should be aware that loading media (at least video) in the background isn't going to work. I'm not sure if this might be a problem, but if it is I expect we could cater to that need by making an explicit load() to cause the user agent to rush to CAN_PLAY as fast as possible without fake-stalling. Any ideas on this? This might be nit-picking, but consistency across browsers would be nice if possible. -- Philip Jägenstedt Opera Software
Re: [whatwg] HTMLMediaElement load and autoplay
On Wed, 2008-07-02 at 15:53 +, Ian Hickson wrote: On Wed, 2 Jul 2008, Philip Jgenstedt wrote: It is not clear if the user agent should automatically invoke the load method on media elements which are in the document's static markup. Is it supposed to be implied by media element ... inserted into a document? Yes. The parser can be thought of as just another script manipulating the DOM; when it inserts the video element, start downloading video. (And if that didn't work, then try again when the first source is inserted, and then the second one, and so forth). (It's fine to batch inserts too.) OK, so in that case I suppose the requirement is to call load() sometime during the parsing phase. This probably implies that the loadstart event must be fired on all media elements before the load event is fired on the owner document. As a side note, we certainly can't want to wait for all media elements to fire the load before firing it on the parent document event as that may take several minutes or never happen at all -- breaking half of the web that relies on this behavior. If user agents don't want to waste time downloading video that isn't visible (e.g. due to CSS display:none) they can block the download (step 14 of the load algorithm) until it becomes visible. In this situation, does the spec allow firing the stalled event immediately instead of waiting 3 seconds? Sure. The 3 seconds is just an about. 0 seconds is about 3 secods. :-) Would you like the spec to be more explicit about this? That's not really necessary in my opinion. Since faking it is allowed, script authors should be aware that loading media (at least video) in the background isn't going to work. I'm not sure if this might be a problem, but if it is I expect we could cater to that need by making an explicit load() to cause the user agent to rush to CAN_PLAY as fast as possible without fake-stalling. Any ideas on this? This might be nit-picking, but consistency across browsers would be nice if possible. The script can tell if the download is being blocked by checking bufferingThrottled and bufferingRate. I'm not sure we want a way to override the blocking since it is supposed to represent user intent. Indeed, but it wouldn't be a spec violation to value automatic loads and script-invoked loads slightly different internally. That said, it's not a priority and I will reraise the question if it appears to be a real need in the future. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' -- Philip Jägenstedt Opera Software
Re: [whatwg] Consider changing the default volume for audio and video to be 1.0 instead of 0.5
I support this change, assuming that volume 1.0 corresponds to digital 0 dB. This way, no volume adjustment is needed by default, which seems like a sensible... default. On Thu, 2008-06-26 at 18:27 -0700, Adele Peterson wrote: Hi all, The spec currently says that the default volume level is 0.5. In WebKit's initial implementation, we have received feedback that the audio seems much too soft. If a user has already adjusted their device's volume to their liking, then I think 1.0 should correspond to that level, and that should be the default. This will enable the browser to have similar volume levels to other applications on the system, and will respect the system volume. Thanks, Adele -- Philip Jägenstedt Opera Software
Re: [whatwg] video background color (Was: Interpretation of video poster attribute)
Safari already uses a transparent background by default and to me that
doesn't seem like a bad idea -- it may be best to hide small 1px
letterboxes due to rounding errors in aspect ratio calculation etc.
Setting background-color:transparent to override the default black is
probably less known to most authors and transparent background is also
more in line with most other HTML elements.
I would suggest eventually specifying this behavior in the rendering
section, unless someone feels that default black letterboxes is very
important.
// Philip
On Fri, 2008-06-13 at 21:02 +, Ian Hickson wrote:
On Fri, 13 Jun 2008, Philip Jgenstedt wrote:
The issue with the poster attribute is resolved, but one comment made me
remember something I've wondered about:
On Fri, 2008-06-13 at 09:26 +, Ian Hickson wrote:
It's not impossible; first black would render 300x150, then the poster
The spec says: Areas of the element's playback area that do not contain
the video represent nothing.
What does this mean? Black is customary for video, but leaving the
region transparent (thus falling back to css background color) is
another option. Which is better?
It's transparent, but I intended to have the following rule in the style
sheet:
video { background: black; }
...so that it looks black unless the author restyles it. Does that make
sense?
--
Ian Hickson U+1047E)\._.,--,'``.fL
http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
--
Philip Jägenstedt
Opera Software
Re: [whatwg] Javascript API to query supported codecs for video and audio
It seems to me that it's a good idea to wait with this until we know more about what will happen with baseline codecs etc. Implementation-wise it might be less than trivial to return an exhaustive list of all supported mime-types if the underlying framework doesn't use the concept of mime-types, but can say when given a few bytes of the file whether it supports it or not. Allowing JavaScript to second-guess this doesn't seem great On Wed, 2008-06-18 at 12:18 +0200, [EMAIL PROTECTED] wrote: On Wed, 2008-06-18 at 12:03 +0200, Anne van Kesteren wrote: Why is that needed? The elements provide a way to link to multiple codecs of which the user agent will then make a choice. i do not intend to provide multiple codecs since that would require multiple backend implementations for playing files form an offset, encoding files in multiple codecs on the server, more disk space etc, instead i would only use the video tag if the codec i use is supported and fall back to other means via object / java / flash or whatever to playback the video or indicate that the user has to install a qt/dshow/gstreamer plugin. in an ideal world i could use video like i can use img now and be done with it, but since this is not the case we need tools to make the best out of video, not knowing what the browser supports and just hoping that it could work is not an option. j -- Philip Jägenstedt Opera Software
Re: [whatwg] Javascript API to query supported codecs for video and audio
Sorry, my reply was cut short. Again: It seems to me that it's a good idea to wait with this until we know more about what will happen with baseline codecs etc. Implementation-wise it might be less than trivial to return an exhaustive list of all supported mime-types if the underlying framework doesn't use the concept of mime-types, but can say when given a few bytes of the file whether it supports it or not. Allowing JavaScript to second-guess this seems like a potential source of incompatibility. Isn't it sufficient to look for MEDIA_ERR_DECODE and add fallback content when that happens? Philip On Wed, 2008-06-18 at 17:34 +0700, Philip Jägenstedt wrote: It seems to me that it's a good idea to wait with this until we know more about what will happen with baseline codecs etc. Implementation-wise it might be less than trivial to return an exhaustive list of all supported mime-types if the underlying framework doesn't use the concept of mime-types, but can say when given a few bytes of the file whether it supports it or not. Allowing JavaScript to second-guess this doesn't seem great On Wed, 2008-06-18 at 12:18 +0200, [EMAIL PROTECTED] wrote: On Wed, 2008-06-18 at 12:03 +0200, Anne van Kesteren wrote: Why is that needed? The elements provide a way to link to multiple codecs of which the user agent will then make a choice. i do not intend to provide multiple codecs since that would require multiple backend implementations for playing files form an offset, encoding files in multiple codecs on the server, more disk space etc, instead i would only use the video tag if the codec i use is supported and fall back to other means via object / java / flash or whatever to playback the video or indicate that the user has to install a qt/dshow/gstreamer plugin. in an ideal world i could use video like i can use img now and be done with it, but since this is not the case we need tools to make the best out of video, not knowing what the browser supports and just hoping that it could work is not an option. j -- Philip Jägenstedt Opera Software
[whatwg] video background color (Was: Interpretation of video poster attribute)
The issue with the poster attribute is resolved, but one comment made me remember something I've wondered about: On Fri, 2008-06-13 at 09:26 +, Ian Hickson wrote: It's not impossible; first black would render 300x150, then the poster The spec says: Areas of the element's playback area that do not contain the video represent nothing. What does this mean? Black is customary for video, but leaving the region transparent (thus falling back to css background color) is another option. Which is better? -- Philip Jägenstedt Opera Software
Re: [whatwg] Interpretation of video poster attribute
On Thu, 2008-06-12 at 07:15 +, Ian Hickson wrote: So, to summarise, an img represents its src=, and a video represents its poster=. So use the same mechanism (stretching the image to fit the box dimensions) for both. Fair enough. This isn't currently defined (even without a poster, as far as I can tell), but my intention would be to not make the poster affect the intrinsic dimensions, and for the default, in the absence of video data, is 300x150. The problem with scaling to the poster's size is that it would make the veo resize twice e (300x150 - poster - video) instead of just once (300x150 blank, then poster - video). If poster is to video what src is to img, then surely the video element should take the size of the poster image if no width/height is given? This is certainly my preference, as width/height would otherwise effectively be mandatory when poster is used, which would in turn require the poster to be of the same size as the video unless one is to be scaled. That the element may be resized twice is not a problem, at least not implementation-wise. As some container formats allow changing the video frame size mid-stream (notably chained Ogg streams, so called sequential multiplexing) the UA could possibly resize the video element an unlimited number of times, so handling that is necessary anyway. On that note, perhaps an event to notify JavaScript UIs of such changes would be useful/necessary. Perhaps reusing resize event would be the best, other names could be videochange or sizechange. Should aspect ratio correction be done for poster images? What if different videos have different aspect ratios? It would be bad for the poster frame to be rendered at different ratios as the UA tries each video in turn. Good point, assuming 1:1 pixel ratio for poster images makes more sense, even if it means tools for automatically generating poster images will have to perform aspect ratio correction. HTTP 4xx and 5xx errors (and equivalents in other protocols) must (may?) cause the user agent to ignore the poster attribute. Well what else are they going to do? HTTP already requires this as far as I can tell. It appears that Safari shows a little X symbol instead, but perhaps specifying this behavior is not necessary. We will likely ignore the poster attribute entirely instead. I'm not sure a fake button (which wouldn't actually work anyway) is a natural interpretation of poster frame. :-) As long as it's mentioned somewhere there can be no misunderstanding. -- Philip Jägenstedt Opera Software
Re: [whatwg] Change request: rules for pixel ratio
I see that I made a rather serious typo in my original message. I meant to ask Should this be taken to mean that the pixelratio attribute is defaulted to 1.0 even for media resources that *DO* self-describe their pixel ratio? Still, my meaning seems to have gotten across and the new phrasing is much better. Philip On Thu, 2008-06-12 at 09:18 +, Ian Hickson wrote: On Tue, 10 Jun 2008, Philip Jgenstedt wrote: http://www.w3.org/html/wg/html5/#pixelratio Incidentally, you may prefer this version of the spec: http://whatwg.org/html5 It has a style sheet optimised for this specification instead of using the W3C style sheet. (It's otherwise identical.) The pixelratio attribute allows the author to specify the pixel ratio of anamorphic media resources that do not self-describe their pixel ratio. [...] The default value, if the attribute is omitted or cannot be parsed, is 1.0. This seems more than strange. Should this be taken to mean that the pixelratio attribute is defaulted to 1.0 even for media resources that DO NOT self-describe their pixel ratio? Yes, but that doesn't mean what you think it means. (I've clarified this in the spec.) The only way the default is used is in deciding what number to return for pixelRatio in the DOM when the content attribute is missing. If the content attribute is omitted, then the user agent doesn't adjust the intrinsic width of the video at all; the intrinsic dimensions and the intrinsic pixel ratio of the video are honoured. It is also worth noting that when the pixel ratio is 1.0 it will make more sense for the user agent to adjust the height than the width if no width/height attributes have been set. This could be pointed out in the rendering section eventually. I've allowed this explicitly now. On Tue, 10 Jun 2008, Ralph Giles wrote: Does this mean the implementation must retrieve the self-described pixel aspect ratio from the stream and supply it as part of the DOM? On Tue, 10 Jun 2008, Charles wrote: Offhand, I can't think of any use cases where it'd be useful to expose pixel aspect ratio. On Wed, 11 Jun 2008, Philip Jgenstedt wrote: Exposing pixel ratio as readonly float videoPixelRatio in HTMLVideoElement might be a good idea. If the media resource doesn't self-describe its pixel ratio (many formats don't) it should be assumed to be 1.0. I'm not convinced it is necessary as content authors who want to be very explicit will probably set the pixelratio manually. Still, it is trivial to expose and might make life easier for those who want this information for low-level control of the video element size via the DOM. I don't see a good reason to expose it, so I haven't added a way to do that. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.' -- Philip Jägenstedt Opera Software
Re: [whatwg] Interpretation of video poster attribute
On Thu, 2008-06-12 at 09:56 +, Ian Hickson wrote: The problem with scaling to the poster's size is that it would make the veo resize twice e (300x150 - poster - video) instead of just once (300x150 blank, then poster - video). If poster is to video what src is to img, then surely the video element should take the size of the poster image if no width/height is given? This is certainly my preference, as width/height would otherwise effectively be mandatory when poster is used, which would in turn require the poster to be of the same size as the video unless one is to be scaled. Why? The posted would be displayed in the default 300x150 box. video poster=image_of_unknown_dimension src=video_of_unknown_but_same_dimension/video This is a probable and reasonable scenario, but currently it's impossible to use a poster image without knowing its dimensions. That the element may be resized twice is not a problem, at least not implementation-wise. That it's resized at all is a terrible problem; that it woul resize twice would be a disaster, UI-wise. Since the poster image will only be displayed until the video starts playing, the options when width/height is not given are: 1. the poster image is displayed at size 300x150 for several seconds while the video is loading, after which the video element takes the size of the video 2. the poster image is displayed at its native size for several seconds while the video is loading, after which the video element takes the size of the video (which will often be the same size) Since a resize is possible in both cases (but less likely in case 2) both are equally problematic UI-wise, except that the image will actually be displayed with its correct dimensions in case 2. This is just to provide sane defaults for authors who trust the browser to do the right things in absence of width/height. Safari already uses the intrinsic dimensions of the poster image and then resizes to the intrinsic dimensions of the video, which is exactly the behavior we want to implement. -- Philip Jägenstedt Opera Software
Re: [whatwg] Change request: rules for pixel ratio
On Tue, 2008-06-10 at 10:48 -0700, Ralph Giles wrote: Does this mean the implementation must retrieve the self-described pixel aspect ratio from the stream and supply it as part of the DOM? Yes, the implementation would read the pixel ratio from the video stream. In libtheora this is aspect_numerator/aspect_denominator of the th_info struct (yes, this is pixel ratio, not frame aspect ratio). This would all be handled by some multimedia framework anyway, so most browsers won't have to worry too much about such details. Exposing pixel ratio as readonly float videoPixelRatio in HTMLVideoElement might be a good idea. If the media resource doesn't self-describe its pixel ratio (many formats don't) it should be assumed to be 1.0. I'm not convinced it is necessary as content authors who want to be very explicit will probably set the pixelratio manually. Still, it is trivial to expose and might make life easier for those who want this information for low-level control of the video element size via the DOM. -- Philip Jägenstedt Opera Software
[whatwg] HTMLMediaElement buffered/bufferedBytes
Hi, I'm currently implementing more of audio and video (in Opera) and will probably have quite a lot of questions/comments during the coming months. If this is not the best place for such discussion, please point out where I need to be. Today's issue: The name of the buffered/bufferedBytes attributes imply that these ranges are buffered on disk or in memory, so that they will not have to be re-downloaded. However, the description reads the ranges of the media resource, if any, that the user agent has downloaded, at the time the attribute is evaluated. This is not the same things, as we will not be able to buffer large files on memory-limited devices. Instead, we might only buffer 1 MB of data around the current playback point, or some other scheme. I would suggest that buffered/bufferedBytes be taken to mean exactly what they sound like by changing the description to something like: (differences marked *like this*) The buffered attribute must return a static normalized TimeRanges object that represents the ranges of the media resource, if any, that the user agent has *buffered*, at the time the attribute is evaluated. Note: Typically this will be a single range anchored at the zero point, but if, e.g. the user agent uses HTTP range requests in response to seeking, then there could be multiple ranges. *There is no guarantee that all buffered ranges will remain buffered, due to storage/memory constraints or other reasons.* The intention is that only ranges which are actually internally buffered should be exposed in the buffered/bufferedBytes ranges, whereas the current phrasing implies that all ranges which have at some point been downloaded/buffered should be included. Admittedly, this makes the attributes useless for determining how much of the resource has been downloaded, but if this is needed I might suggest the attributes downloaded/downloadedBytes instead. The usefulness of the buffered attribute (in my current interpretation) is not obvious to me at all, I would appreciate some use cases if possible. -- Philip Jägenstedt Opera Software
Re: [whatwg] Interpretation of video poster attribute
Hi again, Reading http://lists.w3.org/Archives/Public/public-html/2007Oct/0108.html it is clear that the intention of the poster attribute is to represent a still image from the video. This should probably be made explicit in the spec with something in the style of: The poster image typically represents a still frame of video. User agents must (may?) take this into account by applying the same rules as for rendering video (aspect ratio correction, scaling, centering). HTTP 4xx and 5xx errors (and equivalents in other protocols) must (may?) cause the user agent to ignore the poster attribute. This needs to be clear as there are two quite natural interpretations of what kind of image the user agent can show while no video data is available. The first is an unscaled, centered click to play button, while the second is a scaled, centered and aspect ratio corrected still frame from the video. Philip On Tue, 2008-06-03 at 10:00 -0500, Tab Atkins Jr. wrote: On Tue, Jun 3, 2008 at 7:36 AM, Philip Jägenstedt [EMAIL PROTECTED] wrote: Hi! I'm a bit puzzled about how to interpret the poster attribute on HTMLVideoElement: The poster attribute gives the address of an image file that the user agent can show while no video data is available. The attribute, if present, must contain a URI (or IRI). Is the intention that this image should be stretched to the size of the video element, or that it should be centered in the frame? If the width and height attributes are not given, should the video element initially be given the size of the poster image, or should the user agent wait until it has the dimensions of the video (thereby making the poster useless)? In short, what is the intended use of poster? -- Philip Jägenstedt Just for similar-implementation-ideas, flvplayer simply aligns the poster image to the upper-left of the object element, with no scaling at all. If width and height are not given, it simply doesn't display at all. Unless there's already some alternate intent, I suggest video scale to the poster's size if no explicit size is given. ~TJ
[whatwg] Interpretation of video poster attribute
Hi! I'm a bit puzzled about how to interpret the poster attribute on HTMLVideoElement: The poster attribute gives the address of an image file that the user agent can show while no video data is available. The attribute, if present, must contain a URI (or IRI). Is the intention that this image should be stretched to the size of the video element, or that it should be centered in the frame? If the width and height attributes are not given, should the video element initially be given the size of the poster image, or should the user agent wait until it has the dimensions of the video (thereby making the poster useless)? In short, what is the intended use of poster? -- Philip Jägenstedt
