Re: [whatwg] Scripted querying of video capabilities
FWIW, in nightly Gecko builds we have an implementation of canPlayType per spec. It parses the codecs parameter of the provided MIME type, so we can answer no, maybe or probably depending on what's in the type. E.g.: http://mxr.mozilla.org/mozilla-central/source/content/media/video/test/can_play_type_ogg.js Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] Scripted querying of video capabilities
On Fri, 14 Nov 2008, Silvia Pfeiffer wrote: On a little bit of a side not, may I point out that there is an updated RFC for Ogg media types at http://www.ietf.org/rfc/rfc5334.txt and it explicitly includes the codecs parameter with standard values for the current ones supported by Ogg. It also recommends using video/ogg for video files, so that would change your example to: video source type=video/ogg/ source type=video/ogg; codecs=dirac,flac/ /video I know it's a minor issue, but I'd like us at HTML5 to get it right and to recommend the correct mime types to web developers. Please let me know if there are any problems in the HTML5 spec here. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Scripted querying of video capabilities
On Mon, Nov 17, 2008 at 12:49 PM, Ian Hickson [EMAIL PROTECTED] wrote: On Fri, 14 Nov 2008, Silvia Pfeiffer wrote: On a little bit of a side not, may I point out that there is an updated RFC for Ogg media types at http://www.ietf.org/rfc/rfc5334.txt and it explicitly includes the codecs parameter with standard values for the current ones supported by Ogg. It also recommends using video/ogg for video files, so that would change your example to: video source type=video/ogg/ source type=video/ogg; codecs=dirac,flac/ /video I know it's a minor issue, but I'd like us at HTML5 to get it right and to recommend the correct mime types to web developers. Please let me know if there are any problems in the HTML5 spec here. The spec is fine. I just wanted to point out the correct usage on the mailing list, since there was a misleading example in the thread. Just being picky. Thanks, Silvia.
Re: [whatwg] Scripted querying of video capabilities
On Thu, 16 Oct 2008, Robert O'Callahan wrote: I have added window.navigator.canPlayType(mimeType). It returns 1, 0, or -1 to represent positive, neutral, and negative responses. navigator.canPlayType could be confusing since authors might think it includes media playable via plugins or other browser functionality. Making it a method on video/audio would be more obvious and, I think, no less useful in practice. Ok. Done. On Thu, 16 Oct 2008, Maciej Stachowiak wrote: This API would be tempting to treat as a boolean but would of course do completely the wrong thing. I think it would be better to either ensure that the positive and neutral responses are both values that JS would treat as true (for instance make the values true, maybe and false), or else make all of the return values something self-descriptive and symbolic (for instance the strings yes, maybe and no). I think 1, 0, -1 are neither clear nor likely to be in any way beneficial for perforamnce. I've made the method return strings. On Fri, 14 Nov 2008, Robert O'Callahan wrote: I think it would be a lot better if there were two methods returning booleans, say, canPlayType and cannotPlayType. I think it would be misleading to return false from cannotPlayType(). On Thu, 13 Nov 2008, Jeremy Doig wrote: did this thread go anywhere ?i'm concerned about the maybe case - looks way too much like: http://en.wikipedia.org/wiki/DShow#Codec_hell Well we have to have a maybe case, since the UA might not know the answer. also - when you probe for mime type, do you mean the entire type parameter (including the codecs string) ? for example, there are too many cases where just passing video/mp4 would be insufficient. (fragmented index support ? base/main/high profile ? paff ? cabac ?) Clarified this. On Thu, 13 Nov 2008, Philip J�genstedt wrote: I'm also a bit concerned about how to interpret the yes, no and maybe return values. The truthful answer is going to be maybe for all but the obviously unsupporter (application/x-ms-dos-executable) and the more trivial formats (audio/wav). Indeed (except for audio/wave). When asking about application/ogg, this could mean 2 things: 1. can I demux Ogg streams? 2. can I demux Ogg streams and decode unknown codecs? It's the second (and thus the answer can only ever be maybe or no). The answer to question 1 is reasonably simple to answer in most cases, even if the underlying media frame doesn't support querying for mime types a mapping between mime type and supported decoders isn't impossible. The answer to question 2, on the other hand, is always no or maybe. Right. Unless the codecs parameter is to be made mandatory I think that spec should explicitly make it such that the question asked is 1. In either case we will end up there because 2 is not a meaningful question and user agents will make untruthful answers in attempts to stay compatible with unknown and future content (which might be supported by installing new codecs in the media framework without upgrading the browser). Now, if the codec parameter is used then the user agent may answer yes and no in a way that actually makes some sense. The spec now encourages codecs= be provided. I also think that this should be explicitly related to the type attribute of the source element. One should be able to use canPlayType to predict with 100% accuracy which source the user agent will use when it encounters. Done. On Fri, 14 Nov 2008, Robert O'Callahan wrote: The browser can only answer no or maybe in this case. That seems fine to me. It lets the author know whether the browser can demux Ogg streams, while staying consistent with the behaviour when codec information is given in the MIME type. I don't understand what the incentives are for browsers to lie. If we start off telling the truth, then authors will have to write scripts that either accept maybe or, better still, provide the codec information, or their scripts won't work. I don't see how that would change over time. Authors should be strongly encouraged to give the codec parameters. Agreed. On Thu, 13 Nov 2008, Dave Singer wrote: Pitching in here, I think it's OK if we want to go to a two-state answer -- but those answers are No and Maybe, not No and Yes. There are, after all, vanishingly small numbers of mime types where I can be 'completely' (within reason) confident of a 'yes' answer. On the other hand, given a mime type I simply don't recognize, or a codec I know I don't have, I can be confident of a No. Documenting it as providing a No/Maybe indication, and that how positive the 'Maybe' is depends to a large extent on how precise the supplied mime type is, would be fine by me, I think. I've left it as a three-state method, but encouraged implementors to not return a positive answer unless they're sure. -- Ian Hickson U+1047E
Re: [whatwg] Scripted querying of video capabilities
did this thread go anywhere ?i'm concerned about the maybe case - looks way too much like: http://en.wikipedia.org/wiki/DShow#Codec_hell also - when you probe for mime type, do you mean the entire type parameter (including the codecs string) ? for example, there are too many cases where just passing video/mp4 would be insufficient. (fragmented index support ? base/main/high profile ? paff ? cabac ?) source src=video.mp4 type=video/mp4; codecs=quot;avc1.42E01E, mp4a.40.2quot; On Wed, Oct 15, 2008 at 11:14 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: On Oct 15, 2008, at 1:44 AM, Ian Hickson wrote: On Tue, 14 Oct 2008, Robert O'Callahan wrote: On Tue, Oct 14, 2008 at 12:13 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: While the underlying media frameworks can't necessarily answer, if I give you a file with this MIME type, can you play it?, they can at least give a yes/no/maybe answer, which can still be quite helpful, since the UA will know it does not need to check some media streams at all. I agree. If the API lets us answer maybe, there is not much need or temptation to lie, and we can still return information that could be useful to scripts. I have added window.navigator.canPlayType(mimeType). It returns 1, 0, or -1 to represent positive, neutral, and negative responses. This API would be tempting to treat as a boolean but would of course do completely the wrong thing. I think it would be better to either ensure that the positive and neutral responses are both values that JS would treat as true (for instance make the values true, maybe and false), or else make all of the return values something self-descriptive and symbolic (for instance the strings yes, maybe and no). I think 1, 0, -1 are neither clear nor likely to be in any way beneficial for perforamnce. Regards, Maciej
Re: [whatwg] Scripted querying of video capabilities
On Nov 13, 2008, at 10:52 AM, Jeremy Doig wrote: did this thread go anywhere ? See http://www.whatwg.org/specs/web-apps/current-work/multipage/browsers.html#dom-navigator-canplaytype . i'm concerned about the maybe case - looks way too much like: http://en.wikipedia.org/wiki/DShow#Codec_hell also - when you probe for mime type, do you mean the entire type parameter (including the codecs string) ? for example, there are too many cases where just passing video/mp4 would be insufficient. (fragmented index support ? base/main/high profile ? paff ? cabac ?) source src=video.mp4 type=video/mp4; codecs=quot;avc1.42E01E, mp4a.40.2quot; My interpretation is that it does, and the vagueness of many MIME types is the reason for the maybe case. eric On Wed, Oct 15, 2008 at 11:14 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: On Oct 15, 2008, at 1:44 AM, Ian Hickson wrote: On Tue, 14 Oct 2008, Robert O'Callahan wrote: On Tue, Oct 14, 2008 at 12:13 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: While the underlying media frameworks can't necessarily answer, if I give you a file with this MIME type, can you play it?, they can at least give a yes/no/maybe answer, which can still be quite helpful, since the UA will know it does not need to check some media streams at all. I agree. If the API lets us answer maybe, there is not much need or temptation to lie, and we can still return information that could be useful to scripts. I have added window.navigator.canPlayType(mimeType). It returns 1, 0, or -1 to represent positive, neutral, and negative responses. This API would be tempting to treat as a boolean but would of course do completely the wrong thing. I think it would be better to either ensure that the positive and neutral responses are both values that JS would treat as true (for instance make the values true, maybe and false), or else make all of the return values something self- descriptive and symbolic (for instance the strings yes, maybe and no). I think 1, 0, -1 are neither clear nor likely to be in any way beneficial for perforamnce. Regards, Maciej
Re: [whatwg] Scripted querying of video capabilities
I'm also a bit concerned about how to interpret the yes, no and maybe return values. The truthful answer is going to be maybe for all but the obviously unsupport (application/x-ms-dos-executable) and the more trivial formats (audio/wav). When asking about application/ogg, this could mean 2 things: 1. can I demux Ogg 2. can I demux Ogg and decode an unknown codec If to answer yes the browser must be certain that it can play back all content which could have the requested mime type then the answer must always be maybe for container formats, e.g. video/avi, video/x-msvideo, application/ogg, video/mp4, video/mpeg and so on. Only if the codecs parameter is used is it ever possible to anwer yes It's hardly ever possible to answer no either, unless the underlying media framework explicitly doesn't support a When faced with something unrecognized the safest thing is to say maybe. To sum up, the answer is always maybe. This is nice for implementors who don't need to write a lot of code, but it's not useful. To fix this, I would suggest the following: If the mime type is just a container format then the browser may answer yes if it knows that it supports at least some content in that format (again, it's difficult without On Thu, 2008-11-13 at 10:52 -0800, Jeremy Doig wrote: did this thread go anywhere ? i'm concerned about the maybe case - looks way too much like: http://en.wikipedia.org/wiki/DShow#Codec_hell also - when you probe for mime type, do you mean the entire type parameter (including the codecs string) ? for example, there are too many cases where just passing video/mp4 would be insufficient. (fragmented index support ? base/main/high profile ? paff ? cabac ?) source src=video.mp4 type=video/mp4; codecs=quot;avc1.42E01E, mp4a.40.2quot; On Wed, Oct 15, 2008 at 11:14 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: On Oct 15, 2008, at 1:44 AM, Ian Hickson wrote: On Tue, 14 Oct 2008, Robert O'Callahan wrote: On Tue, Oct 14, 2008 at 12:13 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: While the underlying media frameworks can't necessarily answer, if I give you a file with this MIME type, can you play it?, they can at least give a yes/no/maybe answer, which can still be quite helpful, since the UA will know it does not need to check some media streams at all. I agree. If the API lets us answer maybe, there is not much need or temptation to lie, and we can still return information that could be useful to scripts. I have added window.navigator.canPlayType(mimeType). It returns 1, 0, or -1 to represent positive, neutral, and negative responses. This API would be tempting to treat as a boolean but would of course do completely the wrong thing. I think it would be better to either ensure that the positive and neutral responses are both values that JS would treat as true (for instance make the values true, maybe and false), or else make all of the return values something self-descriptive and symbolic (for instance the strings yes, maybe and no). I think 1, 0, -1 are neither clear nor likely to be in any way beneficial for perforamnce. Regards, Maciej -- Philip Jägenstedt Opera Software
Re: [whatwg] Scripted querying of video capabilities
On Fri, Nov 14, 2008 at 8:19 AM, Eric Carlson [EMAIL PROTECTED]wrote: See http://www.whatwg.org/specs/web-apps/current-work/multipage/browsers.html#dom-navigator-canplaytype . There was widespread dissatisfaction with the form of the API. I think it would be a lot better if there were two methods returning booleans, say, canPlayType and cannotPlayType. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] Scripted querying of video capabilities
On Fri, Nov 14, 2008 at 8:38 AM, Philip Jägenstedt [EMAIL PROTECTED]wrote: I'm also a bit concerned about how to interpret the yes, no and maybe return values. The truthful answer is going to be maybe for all but the obviously unsupporter (application/x-ms-dos-executable) and the more trivial formats (audio/wav). Even WAV is extensible. When asking about application/ogg, this could mean 2 things: 1. can I demux Ogg streams? 2. can I demux Ogg streams and decode unknown codecs? It means can I play this stream, given that the only thing I know about it is that it's an Ogg stream? Which I guess is what you meant by option 2. The browser can only answer no or maybe in this case. That seems fine to me. It lets the author know whether the browser can demux Ogg streams, while staying consistent with the behaviour when codec information is given in the MIME type. Unless the codecs parameter is to be made mandatory I think that spec should explicitly make it such that the question asked is 1. In either case we will end up there because 2 is not a meaningful question and user agents will make untruthful answers in attempts to stay compatible with unknown and future content (which might be supported by installing new codecs in the media framework without upgrading the browser). I don't understand what the incentives are for browsers to lie. If we start off telling the truth, then authors will have to write scripts that either accept maybe or, better still, provide the codec information, or their scripts won't work. I don't see how that would change over time. Now, if the codec parameter is used then the user agent may answer yes and no in a way that actually makes some sense. Yes. Authors should be strongly encouraged to give the codec parameters. I also think that this should be explicitly related to the type attribute of the source element. One should be able to use canPlayType to predict with 100% accuracy which source the user agent will use when it encounters. video source type=application/ogg/ source type=application/ogg codecs=dirac,flac/ /video If this isn't specified then one can expect lots of web app authors being forced to write scripts to work around the different behaviour of canPlayType and the pick a source algorithm in different browsers. I agree. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] Scripted querying of video capabilities
On Fri, Nov 14, 2008 at 6:38 AM, Philip Jägenstedt [EMAIL PROTECTED] wrote: Now, if the codec parameter is used then the user agent may answer yes and no in a way that actually makes some sense. I also think that this should be explicitly related to the type attribute of the source element. One should be able to use canPlayType to predict with 100% accuracy which source the user agent will use when it encounters. video source type=application/ogg/ source type=application/ogg codecs=dirac,flac/ /video On a little bit of a side not, may I point out that there is an updated RFC for Ogg media types at http://www.ietf.org/rfc/rfc5334.txt and it explicitly includes the codecs parameter with standard values for the current ones supported by Ogg. It also recommends using video/ogg for video files, so that would change your example to: video source type=video/ogg/ source type=video/ogg; codecs=dirac,flac/ /video I know it's a minor issue, but I'd like us at HTML5 to get it right and to recommend the correct mime types to web developers. Cheers, Silvia.
Re: [whatwg] Scripted querying of video capabilities
Pitching in here, I think it's OK if we want to go to a two-state answer -- but those answers are No and Maybe, not No and Yes. There are, after all, vanishingly small numbers of mime types where I can be 'completely' (within reason) confident of a 'yes' answer. On the other hand, given a mime type I simply don't recognize, or a codec I know I don't have, I can be confident of a No. Documenting it as providing a No/Maybe indication, and that how positive the 'Maybe' is depends to a large extent on how precise the supplied mime type is, would be fine by me, I think. -- David Singer Multimedia Standards, Apple Inc.
Re: [whatwg] Scripted querying of video capabilities
On Oct 15, 2008, at 1:44 AM, Ian Hickson wrote: On Tue, 14 Oct 2008, Robert O'Callahan wrote: On Tue, Oct 14, 2008 at 12:13 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: While the underlying media frameworks can't necessarily answer, if I give you a file with this MIME type, can you play it?, they can at least give a yes/no/maybe answer, which can still be quite helpful, since the UA will know it does not need to check some media streams at all. I agree. If the API lets us answer maybe, there is not much need or temptation to lie, and we can still return information that could be useful to scripts. I have added window.navigator.canPlayType(mimeType). It returns 1, 0, or -1 to represent positive, neutral, and negative responses. This API would be tempting to treat as a boolean but would of course do completely the wrong thing. I think it would be better to either ensure that the positive and neutral responses are both values that JS would treat as true (for instance make the values true, maybe and false), or else make all of the return values something self- descriptive and symbolic (for instance the strings yes, maybe and no). I think 1, 0, -1 are neither clear nor likely to be in any way beneficial for perforamnce. Regards, Maciej
Re: [whatwg] Scripted querying of video capabilities
On Tue, 14 Oct 2008, Robert O'Callahan wrote: On Tue, Oct 14, 2008 at 12:13 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: While the underlying media frameworks can't necessarily answer, if I give you a file with this MIME type, can you play it?, they can at least give a yes/no/maybe answer, which can still be quite helpful, since the UA will know it does not need to check some media streams at all. I agree. If the API lets us answer maybe, there is not much need or temptation to lie, and we can still return information that could be useful to scripts. I have added window.navigator.canPlayType(mimeType). It returns 1, 0, or -1 to represent positive, neutral, and negative responses. On Mon, 13 Oct 2008, Maciej Stachowiak wrote: Also, should we make .load() asynchronous as far as selecting a media file goes? Right now, a media file has to be instantaneously and synchronously selected, so the UA can't try each one in turn. I believe the original design intent was to make selection completely static and not not based on a fallback-like approach of trying each in turn. Let's assume IE implements video at some point but does not support Ogg or MPEG-4 out of the box. Then an author wishing to provide 3 media streams could still likely get the job done, since browsers could likely determine whether or not particular basic content types are supported. It might be hard to get more complex cases right though. On Tue, 14 Oct 2008, Robert O'Callahan wrote: IMHO load() should be asynchronous and should try each in turn. That would dramatically increase the power of fallback and doesn't seem to cost too much. Chris Double points out that there are edge cases where some streams in a container are supported, but others aren't; for example a framework might support only Ogg Vorbis but not Theora, and you try to play a Vorbis+Theora file, and you get no error from the framework, but you might have preferred to fall back. However, if the author provided the right MIME information, the UA could know that there is supposed to be video, and detect that there isn't any video coming out, and fall back. On Tue, 14 Oct 2008, Robert O'Callahan wrote: We definitely have to change that! Having load() synchronous would be a disaster. On Tue, 14 Oct 2008, Eric Carlson wrote: I agree! A truly synchronous load() will result in a terrible user experience as a UA may need to download a significant amount of data before it knows if a file is playable. Right now the spec picks a resource *without checking with the server*, statically, and then the method returns, and then asynchronously it checks to see if the resource is usable. I am proposing changing the spec to make the load() not select a resource. Instead, load() would immediately return, and the user agent would then asynchronously try each resource that it might otherwise use in turn, firing a single loadstart event at the start, lots of progress events, and then a loadedmetadata or error event depending on how things go. On Tue, 14 Oct 2008, Dave Singer wrote: This brings up another point, which is, is the type attribute on source actually useful? Should we remove that and just have browsers probe the video subsystem for each resource? We can always add the attribute back later if it becomes useful again, but I'd rather not have something that isn't used by browsers, since then it'll be used wrong by authors, making it useless forever. I think it is. Without it, the UA would have to open each source and get its type via HTTP content-type, which is a lot slower, wouldn't it? Is it clear that if the type is supplied in the markup, the UA can (and probably will) use it for content selection, at least at the coarse level, thus avoiding opening connections for files it knows (from the type) it cannot support. If we make the load() asynchronous and allow probing from the client (testing each potentially supported resource in turn), should we make the type optional, then? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Scripted querying of video capabilities
That assignment makes the intuitive code if(canPlayType(_)) incorrect. Cannot should be 0, and enumeration symbols should be provided. Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Ian Hickson Sent: Wednesday, October 15, 2008 10:44 AM To: Maciej Stachowiak; Robert O'Callahan; Dave Singer; Nils Dagsson Moskopp; Eric Carlson Cc: WHATWG List Subject: Re: [whatwg] Scripted querying of video capabilities On Tue, 14 Oct 2008, Robert O'Callahan wrote: On Tue, Oct 14, 2008 at 12:13 PM, Maciej Stachowiak [EMAIL PROTECTED] wrote: While the underlying media frameworks can't necessarily answer, if I give you a file with this MIME type, can you play it?, they can at least give a yes/no/maybe answer, which can still be quite helpful, since the UA will know it does not need to check some media streams at all. I agree. If the API lets us answer maybe, there is not much need or temptation to lie, and we can still return information that could be useful to scripts. I have added window.navigator.canPlayType(mimeType). It returns 1, 0, or -1 to represent positive, neutral, and negative responses.
Re: [whatwg] Scripted querying of video capabilities
Am Dienstag, den 14.10.2008, 14:51 +0900 schrieb Dave Singer: At 7:40 +0200 14/10/08, Nils Dagsson Moskopp wrote: No, I don't think authors can expect that the UA will automatically suggest codec installs. It doesn't know the basis, and maintaining a reliable database of mimetype-download-site mappings would be a pain. Really ? Hasn't Firefox such a mapping ? Also. at least on Ubuntu, multiple Gstreamer-based Programs prompt me to install stuff if codecs are missing. Somehow I fear it's object all over again. Nils
Re: [whatwg] Scripted querying of video capabilities
On Thu, 7 Aug 2008, Tim Starling wrote: Would it be possible to add methods or properties to HTMLMediaElement to support scripted determination of client codec capabilities? The answer, based on replies quoted below from browser vendors, appears to be no, sadly. I agree that it would be a really useful thing to have. On Thu, 7 Aug 2008, Robert O'Callahan wrote: That would be nice to have. Unfortunately DirectShow and Quicktime do not seem to expose the ability to enumerate supported codecs, so it might be hard to implement for some browsers. As things stand, you can use source elements to offer different formats, and you can try to play a stream and use script to detect if it fails to play. On Thu, 7 Aug 2008, Dave Singer wrote: In general, the source fallbacks are also a way to 'probe' this, albeit in a very different way. I'm not sure you can always get a definitive answer to the question if I gave you a file with this (extended) MIME type, could you play it? and I am fairly sure that asking the implementation to enumerate all the types it could support would be hard. It is sad that we can't provide an API such as the requested one. This brings up another point, which is, is the type attribute on source actually useful? Should we remove that and just have browsers probe the video subsystem for each resource? We can always add the attribute back later if it becomes useful again, but I'd rather not have something that isn't used by browsers, since then it'll be used wrong by authors, making it useless forever. Also, should we make .load() asynchronous as far as selecting a media file goes? Right now, a media file has to be instantaneously and synchronously selected, so the UA can't try each one in turn. On Fri, 8 Aug 2008, Henri Sivonen wrote: Does what HTML5 says now match the framework APIs? The MIME codecs parameter seems to confuse both WebKit and Minefield, for instance: http://hsivonen.iki.fi/test/moz/video-selection/source-mp4-ogg.html vs. http://hsivonen.iki.fi/test/moz/video-selection/source-mp4-ogg-params.html This is a bad sign; what should we do to fix this? On Thu, 7 Aug 2008, Tim Starling wrote: DirectShow and QuickTime can add those interfaces at a later date. When the backends develop this capability, there should be a standard way to go the next step and expose it to JavaScript. Otherwise every implementor will develop their own interface. As soon as implementors want to expose this, I would be more than happy to specify how it should work. We shouldn't specify it too soon, though, because otherwise UAs are wont to implement the API and just guess or lie, making the API useless -- and when there's a more reliable option, we'll have to invent a new API anyway. On Wed, 13 Aug 2008, Chris Double wrote: I don't believe this to be the case. See my previous message about this. There's one specific instance of it not working as far as I know: video src=foo.ogg objectfallback for Ogg playback using plugin/object /video A browser that supports video but not Ogg will not use the fallback object. Instead it will just give an error when loading the foo.ogg file. If some way of having this case work is supplied then a media sniffing API is possibly not needed. Tim, can you confirm that? If you can get the user to download a binary platform-specific NPAPI plugin for Ogg support, why not get the user to download a binary platform-specific video backend plugin for Ogg support? On Tue, 12 Aug 2008, James Ide wrote: In this case I believe it would be possible to listen to the `error' event and dynamically insert fallback content if the video element has an error state of MEDIA_ERR_DECODE. However, this fails when using source elements as the resource-selecting algorithm just terminates if all of the media sources are of unsupported types. It throws an exception in that case too, and the networkState stays EMPTY. So this case can be detected. Would you prefer an error condition? More complications arise when we consider cases where only a subset of the codecs in the resource are supported and different events are fired and error codes set. I agree that there should be some type of fallback mechanism in place, perhaps akin to how object deals with unsupported data. Currently source fills in some of the gaps by providing fallback support to a degree, but I feel that there should be a more comprehensive solution that encourages authors to use video, with or without source, but also provides a safety net by allowing currently existing fallback solutions to function as expected. Intuitively, I would expect video with an unsupported codec to behave as if the video element itself were unsupported, with perhaps a small note on how to acquire the codec(s) required to play the media, if the UA can determine that much and is configured to display such messages. The idea
Re: [whatwg] Scripted querying of video capabilities
On Oct 13, 2008, at 1:06 PM, Ian Hickson wrote: On Thu, 7 Aug 2008, Tim Starling wrote: Would it be possible to add methods or properties to HTMLMediaElement to support scripted determination of client codec capabilities? The answer, based on replies quoted below from browser vendors, appears to be no, sadly. I agree that it would be a really useful thing to have. On Thu, 7 Aug 2008, Robert O'Callahan wrote: That would be nice to have. Unfortunately DirectShow and Quicktime do not seem to expose the ability to enumerate supported codecs, so it might be hard to implement for some browsers. As things stand, you can use source elements to offer different formats, and you can try to play a stream and use script to detect if it fails to play. On Thu, 7 Aug 2008, Dave Singer wrote: In general, the source fallbacks are also a way to 'probe' this, albeit in a very different way. I'm not sure you can always get a definitive answer to the question if I gave you a file with this (extended) MIME type, could you play it? and I am fairly sure that asking the implementation to enumerate all the types it could support would be hard. It is sad that we can't provide an API such as the requested one. This brings up another point, which is, is the type attribute on source actually useful? Should we remove that and just have browsers probe the video subsystem for each resource? We can always add the attribute back later if it becomes useful again, but I'd rather not have something that isn't used by browsers, since then it'll be used wrong by authors, making it useless forever. While the underlying media frameworks can't necessarily answer, if I give you a file with this MIME type, can you play it?, they can at least give a yes/no/maybe answer, which can still be quite helpful, since the UA will know it does not need to check some media streams at all. Also, should we make .load() asynchronous as far as selecting a media file goes? Right now, a media file has to be instantaneously and synchronously selected, so the UA can't try each one in turn. I believe the original design intent was to make selection completely static and not not based on a fallback-like approach of trying each in turn. Let's assume IE implements video at some point but does not support Ogg or MPEG-4 out of the box. Then an author wishing to provide 3 media streams could still likely get the job done, since browsers could likely determine whether or not particular basic content types are supported. It might be hard to get more complex cases right though. Regards, Maciej
Re: [whatwg] Scripted querying of video capabilities
Am Dienstag, den 14.10.2008, 14:21 +0900 schrieb Dave Singer: At 20:06 + 13/10/08, Ian Hickson wrote: On Thu, 7 Aug 2008, Dave Singer wrote: In general, the source fallbacks are also a way to 'probe' this, albeit in a very different way. I'm not sure you can always get a definitive answer to the question if I gave you a file with this (extended) MIME type, could you play it? and I am fairly sure that asking the implementation to enumerate all the types it could support would be hard. It is sad that we can't provide an API such as the requested one. I agree, as it is useful for 'portal pages', where you can prompt the user you need to download and install X to view movies on this site. What's a portal page - wouldn't it be the job of the Browser / Media Framework to prompt for codec installs ? This brings up another point, which is, is the type attribute on source actually useful? Should we remove that and just have browsers probe the video subsystem for each resource? We can always add the attribute back later if it becomes useful again, but I'd rather not have something that isn't used by browsers, since then it'll be used wrong by authors, making it useless forever. How exactly could browsers use it wrong ? Cheers, Nils
Re: [whatwg] Scripted querying of video capabilities
At 7:40 +0200 14/10/08, Nils Dagsson Moskopp wrote: What's a portal page - wouldn't it be the job of the Browser / Media Framework to prompt for codec installs ? They are used today; it's a page with a 'published URL' through which people normally gain access to the site. You can check there whether the UA is suitable for the rest of the site, help teh user find and install anything they need, and so on. No, I don't think authors can expect that the UA will automatically suggest codec installs. It doesn't know the basis, and maintaining a reliable database of mimetype-download-site mappings would be a pain. -- David Singer Apple/QuickTime
Re: [whatwg] Scripted querying of video capabilities
timeless wrote: yes, i'd expect it to just work. however i'd also expect the apis not to work correctly. which means we'd probably be stuck with a case where we either lie and say ogg isn't supported (because we have no way to figure out if it's supported), which means it wouldn't work. or we'd lie and say it is supported (which means that if someone sniffs for ogg first, they'd send you ogg only and you would get nothing). people should just provide source tags for all formats they have in the order in which they'd like to send them. let the user agents figure it out. We only have one format: Theora. We can play it with video, applet or object. The challenge is to determine which is best. please don't encourage sniffing. footnote: if someone's annoying/evil and only provides one source, then yes, bad things will probably happen. There is only one format that is feasible for web deployment and patent-free, and that is Theora. Are you saying now that it's evil to support patent-free codecs? Could i put in a plea for browsers to consider flagging this site isn't nice, it probably won't work if you visit it with another device, you should complain to the content provider. if possible, useragents should be encouraged to encourage their users to demand more availability of content in more formats :). So not only do you want to derail my attempt to bring Theora content to users, you want to turn those users into amateur MPEG-LA license salesmen? -- Tim Starling
Re: [whatwg] Scripted querying of video capabilities
On Aug 14, 2008, at 11:14, timeless wrote: We'd probably be forced to lie and claim every codec imaginable. [including ogg (or rather vorbis, theora, speex, ...), as these are all imaginable] On Thu, Aug 14, 2008 at 12:41 PM, Henri Sivonen [EMAIL PROTECTED] wrote: Would the situation be any different for the source element fallback? yes. people wouldn't try to ask us questions we can't answer. instead they'd be giving gecko lots of data and it'd figure it out. If MicroB.next ships without Gecko's built-in liboggplay video back end no comment about plans for releases ;-) but ships with the GStreamer back end and a third party provides an Ogg plugin for GStreamer, as a user I'd want video on Wikipedia to Just Work if the third-party GStreamer Ogg plugin is installed. yes, i'd expect it to just work. however i'd also expect the apis not to work correctly. which means we'd probably be stuck with a case where we either lie and say ogg isn't supported (because we have no way to figure out if it's supported), which means it wouldn't work. or we'd lie and say it is supported (which means that if someone sniffs for ogg first, they'd send you ogg only and you would get nothing). people should just provide source tags for all formats they have in the order in which they'd like to send them. let the user agents figure it out. please don't encourage sniffing. footnote: if someone's annoying/evil and only provides one source, then yes, bad things will probably happen. Could i put in a plea for browsers to consider flagging this site isn't nice, it probably won't work if you visit it with another device, you should complain to the content provider. if possible, useragents should be encouraged to encourage their users to demand more availability of content in more formats :).
Re: [whatwg] Scripted querying of video capabilities
On Aug 20, 2008, at 12:39, timeless wrote: On Thu, Aug 14, 2008 at 12:41 PM, Henri Sivonen [EMAIL PROTECTED] wrote: Would the situation be any different for the source element fallback? yes. people wouldn't try to ask us questions we can't answer. instead they'd be giving gecko lots of data and it'd figure it out. [...] but ships with the GStreamer back end and a third party provides an Ogg plugin for GStreamer, as a user I'd want video on Wikipedia to Just Work if the third-party GStreamer Ogg plugin is installed. yes, i'd expect it to just work. however i'd also expect the apis not to work correctly. which means we'd probably be stuck with a case where we either lie and say ogg isn't supported (because we have no way to figure out if it's supported), which means it wouldn't work. or we'd lie and say it is supported (which means that if someone sniffs for ogg first, they'd send you ogg only and you would get nothing). Wouldn't it follow that dispatch based on the attributes on the source element wouldn't work, either? Doesn't GStreamer have any working API for querying what capabilities it has installed? people should just provide source tags for all formats they have in the order in which they'd like to send them. let the user agents figure it out. Do you mean trying to download each video, giving it to GStreaming and seeing if an error code comes back? -- Henri Sivonen [EMAIL PROTECTED] http://hsivonen.iki.fi/
Re: [whatwg] Scripted querying of video capabilities
Only the user that actually encounters a Web site deficiency should report it to the creator/owner (assuming they provided a reverse link). Otherwise such a report should be ignored as a supposition. Why should browser vendors bother that some pages do not display correctly in other browsers? This is a validator's job, and a validator is an authoring tool. That would mean supporting your competitor, wouldn't it? Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of timeless Sent: Wednesday, August 20, 2008 11:40 AM To: WHATWG List Subject: Re: [whatwg] Scripted querying of video capabilities footnote: if someone's annoying/evil and only provides one source, then yes, bad things will probably happen. Could i put in a plea for browsers to consider flagging this site isn't nice, it probably won't work if you visit it with another device, you should complain to the content provider. if possible, useragents should be encouraged to encourage their users to demand more availability of content in more formats :).
Re: [whatwg] Scripted querying of video capabilities
On Wed, Aug 20, 2008 at 1:52 PM, Kristof Zelechovski [EMAIL PROTECTED] wrote: Only the user that actually encounters a Web site deficiency should report it to the creator/owner (assuming they provided a reverse link). Otherwise such a report should be ignored as a supposition. mass complaints work better. Why should browser vendors bother that some pages do not display correctly in other browsers? for the good of the web. This is a validator's job, and a validator is an authoring tool. i highly doubt this will work. That would mean supporting your competitor, wouldn't it? can't we all get along and work for a better web? but yes, it would mean helping your own engine on another profile which might not support the same features.
Re: [whatwg] Scripted querying of video capabilities
No, no, and no. Author's POV: Mass complaints about _supposedly_ incompatible Web content from incompetent end users would only cause me, as the author, to file a complaint with the browser vendor. The browser should not pretend it is omniscient and it can teach everyone around. Vendor's POV: Browser vendors can and should agree on the basic constructs and provide the relevant publisher's documentation for the good of the Web; advertising the competitors' products would be an unreasonable requirement. User's POV: I am unwilling to help my browser vendor get the page that works for me display correctly in another product I do not intend to use. The warning is obtrusive, it warns about something immaterial and it bears a slight resemblance to a chain letter. Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of timeless Sent: Wednesday, August 20, 2008 2:29 PM To: WHATWG List Subject: Re: [whatwg] Scripted querying of video capabilities On Wed, Aug 20, 2008 at 1:52 PM, Kristof Zelechovski [EMAIL PROTECTED] wrote: Only the user that actually encounters a Web site deficiency should report it to the creator/owner (assuming they provided a reverse link). Otherwise such a report should be ignored as a supposition. mass complaints work better. Why should browser vendors bother that some pages do not display correctly in other browsers? for the good of the web. This is a validator's job, and a validator is an authoring tool. i highly doubt this will work. That would mean supporting your competitor, wouldn't it? can't we all get along and work for a better web? but yes, it would mean helping your own engine on another profile which might not support the same features.
Re: [whatwg] Scripted querying of video capabilities
On Wed, Aug 20, 2008 at 9:53 PM, Henri Sivonen [EMAIL PROTECTED] wrote: Do you mean trying to download each video, giving it to GStreaming and seeing if an error code comes back? That might be what we have to do, yes. But at least that can be done asynchronously. You couldn't implement a synchronous JS query API that way of course. Hmm, I suppose you could implement a synchronous API by making a collection of videos, one in each known format, and on startup trying to play them all, and recording which ones were successful. But ugly doesn't even begin to describe that solution. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] Scripted querying of video capabilities
On Fri, Aug 8, 2008 at 10:13 PM, Tim Starling [EMAIL PROTECTED] wrote: Or have I been shot down already? I'd like to shoot you down. javascript should not be required to play media. sniffing apps are historically bad, and shouldn't be encouraged. there should be no harm in using multiple sources if there is, please explain what it is so that we can see if that needs to be fixed. I'm working on a product which delivers flash 9.0.31 and we have someone who complained that their browser was told that they needed flash 9.0.47. While it's pretty, it's mostly annoying. I wish they didn't do the sniffing and just made things work.I suspect if they didn't sniff, things would just work. A lot of the browsers I deal with run w/o js for various reasons. (I wonder what the plan is for video in html-mail [thunderbird, etc.]) It sounds very clearly that this is not something which browser vendors could usefully implement (and I know that my browser team would be forced to do its own work here, and from a historical review, i know that we did a bad job of advertising our mime types for our media plugin, which means this thing would be bad too). We'd probably be forced to lie and claim every codec imaginable.
Re: [whatwg] Scripted querying of video capabilities
On Aug 14, 2008, at 11:14, timeless wrote: We'd probably be forced to lie and claim every codec imaginable. Would the situation be any different for the source element fallback? If MicroB.next ships without Gecko's built-in liboggplay video back end but ships with the GStreamer back end and a third party provides an Ogg plugin for GStreamer, as a user I'd want video on Wikipedia to Just Work if the third-party GStreamer Ogg plugin is installed. -- Henri Sivonen [EMAIL PROTECTED] http://hsivonen.iki.fi/
Re: [whatwg] Scripted querying of video capabilities
The spec mandates both video element support + ogg theora support. No, that's incorrect. -- Charles
Re: [whatwg] Scripted querying of video capabilities
All right, in that case I give up. It is plainly insane. The VIDEO element is for displaying movies, not for displaying funny messages that the user should install codec XYZ. If it cannot display the movie, it should display the fallback content provided. If the author wants the user to install the codec, she can put it into the fallback content explicitly. The downside is that the poster frame has to be dropped even if it can be displayed. Besides, there is no API to test whether the browser supports image/targa either. If we allow video support API, why not image support API as well? IMHO, Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris Double Sent: Wednesday, August 13, 2008 12:46 AM To: Kristof Zelechovski Cc: WHATWG List; Tim Starling Subject: Re: [whatwg] Scripted querying of video capabilities On Wed, Aug 13, 2008 at 3:35 AM, Kristof Zelechovski [EMAIL PROTECTED] wrote: Falling back to another method of displaying media is possible without a dedicated media API. In this particular case, you can have a video element with an ogg source and an object running Cortado to display it. I don't believe this to be the case. See my previous message about this. There's one specific instance of it not working as far as I know: video src=foo.ogg objectfallback for Ogg playback using plugin/object /video A browser that supports video but not Ogg will not use the fallback object. Instead it will just give an error when loading the foo.ogg file. If some way of having this case work is supplied then a media sniffing API is possibly not needed. Tim, can you confirm that? Chris. -- http://www.bluishcoder.co.nz
Re: [whatwg] Scripted querying of video capabilities
What is the advantage of using JavaScript to determine a viable embedding method over using alternative streams and fallback content that can include the OBJECT element where appropriate? Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tim Starling Sent: Thursday, August 07, 2008 1:23 PM To: WHATWG List Subject: Re: [whatwg] Scripted querying of video capabilities The reason this is needed, as opposed to using multiple source tags, is because inevitably, some clients will support certain formats via object (or in our special case, applet) and not via video. Querying the browser's video capabilities allows JS to decide what sort of embedding method to use. It's an essential migration feature. -- Tim Starling
Re: [whatwg] Scripted querying of video capabilities
On Tue, Aug 12, 2008 at 7:47 PM, Kristof Zelechovski [EMAIL PROTECTED] wrote: What is the advantage of using JavaScript to determine a viable embedding method over using alternative streams and fallback content that can include the OBJECT element where appropriate? video src=foo.ogg fallback content /video On a browser that doesn't support video this will use the fallback content (OBJECT, etc) to instantiate something that can play the Ogg file. On a browser that supports video and Ogg it will play the video. On a browser that supports video but not Ogg, how do you then instantiate a fallback that can play the Ogg file. Without JavaScript and without providing an alternative source re-encoded in a different format. Chris. -- http://www.bluishcoder.co.nz
Re: [whatwg] Scripted querying of video capabilities
On Wed, Aug 13, 2008 at 3:35 AM, Kristof Zelechovski [EMAIL PROTECTED] wrote: Falling back to another method of displaying media is possible without a dedicated media API. In this particular case, you can have a video element with an ogg source and an object running Cortado to display it. I don't believe this to be the case. See my previous message about this. There's one specific instance of it not working as far as I know: video src=foo.ogg objectfallback for Ogg playback using plugin/object /video A browser that supports video but not Ogg will not use the fallback object. Instead it will just give an error when loading the foo.ogg file. If some way of having this case work is supplied then a media sniffing API is possibly not needed. Tim, can you confirm that? Chris. -- http://www.bluishcoder.co.nz
Re: [whatwg] Scripted querying of video capabilities
On Tue, Aug 12, 2008 at 3:46 PM, Chris Double [EMAIL PROTECTED]wrote: A browser that supports video but not Ogg will not use the fallback object. Instead it will just give an error when loading the foo.ogg file. In this case I believe it would be possible to listen to the `error' event and dynamically insert fallback content if the video element has an error state of MEDIA_ERR_DECODE. However, this fails when using source elements as the resource-selecting algorithm just terminates if all of the media sources are of unsupported types. More complications arise when we consider cases where only a subset of the codecs in the resource are supported and different events are fired and error codes set. I agree that there should be some type of fallback mechanism in place, perhaps akin to how object deals with unsupported data. Currently source fills in some of the gaps by providing fallback support to a degree, but I feel that there should be a more comprehensive solution that encourages authors to use video, with or without source, but also provides a safety net by allowing currently existing fallback solutions to function as expected. Intuitively, I would expect video with an unsupported codec to behave as if the video element itself were unsupported, with perhaps a small note on how to acquire the codec(s) required to play the media, if the UA can determine that much and is configured to display such messages. Just some thoughts, - James
Re: [whatwg] Scripted querying of video capabilities
James Ide wrote: On Tue, Aug 12, 2008 at 3:46 PM, Chris Double [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: A browser that supports video but not Ogg will not use the fallback object. Instead it will just give an error when loading the foo.ogg file. In this case I believe it would be possible to listen to the `error' event and dynamically insert fallback content if the video element has an error state of MEDIA_ERR_DECODE. However, this fails when using source elements as the resource-selecting algorithm just terminates if all of the media sources are of unsupported types. More complications arise when we consider cases where only a subset of the codecs in the resource are supported and different events are fired and error codes set. Even if it did work, you couldn't support a user interface in the Wikipedia style, where Cortado is used by default, and the video element is offered as a non-default option to the user if it is supported. You'd have to play the video just to find out. Sorry Chris, but currently Cortado and QuickTime are the only players with seek/progress bars, so it made sense to put them first. -- Tim Starling
Re: [whatwg] Scripted querying of video capabilities
On Sat, Aug 9, 2008 at 7:13 AM, Tim Starling [EMAIL PROTECTED] wrote: Where do I go from here with this? Should I make a concrete proposal? A sample implementation? Get that Mozilla bug fixed? Or have I been shot down already? I think it is important for web sites to be able to detect in some way what a browser supports. Lot's of sites currently do this (Wikimedia obviously, but there's also Metavid, the numerous 'video blogging' scripts, etc). While they could all switch overnight to only supporting video I doubt that suggesting that solution is going to result in people looking favourably on migrating towards it. Can the functionality provided by HTML 5 video do what is needed for these sorts of detection scripts? If not, I'd be interested in seeing a proposal of something that would work. Chris. -- http://www.bluishcoder.co.nz
Re: [whatwg] Scripted querying of video capabilities
Jeremy Doig wrote: how would this work (say) for different avc profile levels and features (eg: PAFF support) ? I don't think that's our problem. The details of determining a type name for a given file should be in another standard, it should not be in HTML 5. All HTML 5 has to do is delegate responsibility for type assignment. would we require video creators to know the specific capabilities of every fourCC target ? Video creators aren't *required* to do anything. They can just embed their videos in HTML with no specified type and test it on a few platforms to see if it works. The point of the interface (from my point of view) is to assist website designers when they wish to promulgate a new or unpopular video format. Presumably, the website designer will know the new video format very well, otherwise they wouldn't bother. -- Tim Starling
Re: [whatwg] Scripted querying of video capabilities
Dave Singer wrote: I think this is a good idea, but one rapidly runs into the problems talked about in the 'bucket' RFC, notably that there is not a universal language for naming codecs (4ccs etc). But it's proved useful in the past. In general, the source fallbacks are also a way to 'probe' this, albeit in a very different way. I'm not sure you can always get a definitive answer to the question if I gave you a file with this (extended) MIME type, could you play it? and I am fairly sure that asking the implementation to enumerate all the types it could support would be hard. If you can't get an answer to that question, then how will the browsers implement source fallbacks? What I'm suggesting is basically the same, except without the requirement to download a video file. All the issues with type names are also present with source. -- Tim Starling
Re: [whatwg] Scripted querying of video capabilities
Robert O'Callahan wrote:
That would be nice to have. Unfortunately DirectShow and Quicktime do
not seem to expose the ability to enumerate supported codecs, so it
might be hard to implement for some browsers.
As things stand, you can use source elements to offer different
formats, and you can try to play a stream and use script to detect if
it fails to play.
DirectShow and QuickTime can add those interfaces at a later date. When
the backends develop this capability, there should be a standard way to
go the next step and expose it to JavaScript. Otherwise every
implementor will develop their own interface.
In fact, QuickTime already has enough query capabilities for Wikimedia's
purposes. Here is part of the JavaScript code I wrote to embed QuickTime:
// Detect XiphQT (may throw)
var xiphQtVersion = false, done = false;
try {
xiphQtVersion = videoElt.GetComponentVersion('imdc','XiTh', 'Xiph');
done = true;
} catch ( e ) {}
if ( done ) {
window.clearInterval( this_.qtTimers[params.id] );
if ( !xiphQtVersion || xiphQtVersion == '0.0' ) {
// ... show error message no XiphQt ...
}
// Disable autoplay on back button
this_.setParam( videoElt, 'AUTOPLAY', 'False' );
}
It works very well, but it's the sort of thing that I'd rather not have
to implement a dozen times for a dozen incompatible scriptable interfaces.
--
Tim Starling
Wikimedia Foundation
Re: [whatwg] Scripted querying of video capabilities
On Aug 7, 2008, at 09:53, Tim Starling wrote:
xiphQtVersion = videoElt.GetComponentVersion('imdc','XiTh',
'Xiph');
This kind of FourCC use is exactly the kind of thing I meant earlier
when I asked if the MIME stuff is really the best match for frameworks.
--
Henri Sivonen
[EMAIL PROTECTED]
http://hsivonen.iki.fi/
Re: [whatwg] Scripted querying of video capabilities
Henri Sivonen wrote:
On Aug 7, 2008, at 09:53, Tim Starling wrote:
xiphQtVersion = videoElt.GetComponentVersion('imdc','XiTh',
'Xiph');
This kind of FourCC use is exactly the kind of thing I meant earlier
when I asked if the MIME stuff is really the best match for frameworks.
FourCC and MIME are not mutually exclusive, see RFC 2361.
-- Tim Starling
Re: [whatwg] Scripted querying of video capabilities
Mikko Rantalainen wrote:
Tim Starling wrote:
Henri Sivonen wrote:
On Aug 7, 2008, at 09:53, Tim Starling wrote:
xiphQtVersion = videoElt.GetComponentVersion('imdc','XiTh',
'Xiph');
This kind of FourCC use is exactly the kind of thing I meant earlier
when I asked if the MIME stuff is really the best match for frameworks.
FourCC and MIME are not mutually exclusive, see RFC 2361.
RFC 2361 doesn't seem to provide a method for describing both video
codec and audio codec for a resource. The same parameter (codec) is
used for both audio and video codec information but I cannot understand
how a resource that contains both video and audio should be labeled.
RFC 4281 seems to provide a parameter codecs that is a comma separated
list of used codecs. The detail required (MUST) for the codecs
parameter seems quite high and I'm afraid that this parameter will not
be correctly used in real world, unfortunately. In theory, this would
seem a good way to provide the information needed for a resource.
Well, I was thinking of an interface which would list the codecs, rather
than the overall file types. But it could be done either way. By file type:
if ( video.supportsType( 'application/ogg;codecs=vorbis,theora' ) ) {
...
}
By codec/container:
if ( 'application/ogg' in video.supportedTypes
'video/theora' in video.supportedTypes
'audio/vorbis' in video.supportedTypes )
{
...
}
The first one looks easier in this application, and allows for multiple
backends with different allowed combinations of container and codec. The
second one allows for complete enumeration of client capabilities.
Obviously you couldn't provide an interface to enumerate every possible
combination of container, audio codec and video codec.
The reason this is needed, as opposed to using multiple source tags,
is because inevitably, some clients will support certain formats via
object (or in our special case, applet) and not via video.
Querying the browser's video capabilities allows JS to decide what
sort of embedding method to use. It's an essential migration feature.
-- Tim Starling
Re: [whatwg] Scripted querying of video capabilities
how would this work (say) for different avc profile levels and features (eg:
PAFF support) ?would we require video creators to know the specific
capabilities of every fourCC target ?
On Thu, Aug 7, 2008 at 4:23 AM, Tim Starling [EMAIL PROTECTED]wrote:
Mikko Rantalainen wrote:
Tim Starling wrote:
Henri Sivonen wrote:
On Aug 7, 2008, at 09:53, Tim Starling wrote:
xiphQtVersion = videoElt.GetComponentVersion('imdc','XiTh',
'Xiph');
This kind of FourCC use is exactly the kind of thing I meant earlier
when I asked if the MIME stuff is really the best match for frameworks.
FourCC and MIME are not mutually exclusive, see RFC 2361.
RFC 2361 doesn't seem to provide a method for describing both video
codec and audio codec for a resource. The same parameter (codec) is
used for both audio and video codec information but I cannot understand
how a resource that contains both video and audio should be labeled.
RFC 4281 seems to provide a parameter codecs that is a comma separated
list of used codecs. The detail required (MUST) for the codecs
parameter seems quite high and I'm afraid that this parameter will not
be correctly used in real world, unfortunately. In theory, this would
seem a good way to provide the information needed for a resource.
Well, I was thinking of an interface which would list the codecs, rather
than the overall file types. But it could be done either way. By file type:
if ( video.supportsType( 'application/ogg;codecs=vorbis,theora' ) ) {
...
}
By codec/container:
if ( 'application/ogg' in video.supportedTypes
'video/theora' in video.supportedTypes
'audio/vorbis' in video.supportedTypes )
{
...
}
The first one looks easier in this application, and allows for multiple
backends with different allowed combinations of container and codec. The
second one allows for complete enumeration of client capabilities.
Obviously you couldn't provide an interface to enumerate every possible
combination of container, audio codec and video codec.
The reason this is needed, as opposed to using multiple source tags,
is because inevitably, some clients will support certain formats via
object (or in our special case, applet) and not via video.
Querying the browser's video capabilities allows JS to decide what
sort of embedding method to use. It's an essential migration feature.
-- Tim Starling
Re: [whatwg] Scripted querying of video capabilities
I think this is a good idea, but one rapidly runs into the problems talked about in the 'bucket' RFC, notably that there is not a universal language for naming codecs (4ccs etc). But it's proved useful in the past. In general, the source fallbacks are also a way to 'probe' this, albeit in a very different way. I'm not sure you can always get a definitive answer to the question if I gave you a file with this (extended) MIME type, could you play it? and I am fairly sure that asking the implementation to enumerate all the types it could support would be hard. But this kind of query is really useful on a 'portal' page -- entry points to sites. A script can detect that you need to download oh, say, Theora, in order to see the videos in the rest of the site, and shows a frame or page that would help you load it. :-) -- David Singer Apple/QuickTime
Re: [whatwg] Scripted querying of video capabilities
On Thu, Aug 7, 2008 at 6:53 PM, Tim Starling [EMAIL PROTECTED]wrote: DirectShow and QuickTime can add those interfaces at a later date. When the backends develop this capability, there should be a standard way to go the next step and expose it to JavaScript. Otherwise every implementor will develop their own interface. Sure. But the interface has to be able to say I don't know. And then we're relying on authors to do the right thing if they get that answer ... which is always a worry. In fact, QuickTime already has enough query capabilities for Wikimedia's purposes. Probing for a particular component is a different problem from listing all supported codecs, which is what I thought you were asking for. I suppose we could hardcode a list of well-known components along with the codecs they support and then probe for them to estimate what codecs are supported. But, yuck. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] Scripted querying of video capabilities
That would be nice to have. Unfortunately DirectShow and Quicktime do not seem to expose the ability to enumerate supported codecs, so it might be hard to implement for some browsers. As things stand, you can use source elements to offer different formats, and you can try to play a stream and use script to detect if it fails to play. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
