[whatwg] createObjectURL(stream) protocol issue
Would it be possible to give the associated URL for a mediastream to have its own protocol, for example mediastream:, instead of the proposed blob:? window . URL . createObjectURL(stream) Mints a Blob URL to refer to the given MediaStream. This would tremendously help the implementation. Thanks in advance, Tommy -- Tommy Widenflycht, Senior Software Engineer Google Sweden AB, Kungsbron 2, SE-11122 Stockholm, Sweden Org. nr. 556656-6880 And yes, I have to include the above in every outgoing email according to EU law.
Re: [whatwg] Proposal for a MediaSource API that allows sending media data to a HTMLMediaElement
Hi Aaron, I think it would be good if the API recognized the fact that the media data may becoming from several different original files/streams (e.g. different bitrates) as the player adapts to network or other conditions. The different files may have different initialization information (Info and Tracks in WebM, Movie Box in mp4 etc.), which could be provided either in the first append call for each stream or with a separate API call. But subsequently you need to know which initialization information is relevant for each appended block. An integer streamId in the append call would be sufficient - the absolute value has no meaning - it would just associate data from the same stream across calls. The alternatives are: (a) to require that all streams have the same or compatible initialization information or (b) to pass the initialization information every time you change streams (a) has the disadvantage of constraining encoding, and making adding new streams more dependent on the details of how the existing streams were encoded/packaged (b) is ok, except that it is nice for the player to know this data is from the same stream you were playing a while ago - it can re-use some previously established state - rather than every stream change being 'out of the blue'. A separate comment is that practically we have found it very useful for the media player to know the maximum resolution, frame rate and codec level/profile that will be used, which may be different from the resolution and codec/level/profile of the first stream. ...Mark On Jul 11, 2011, at 11:42 AM, Aaron Colwell wrote: Hi, Based on comments in the File API Streaming Blobshttp://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-January/029973.html thread and my Extending HTML 5 video for adaptive streaminghttp://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-June/032277.html thread, I decided on taking a stab at writing a MediaSource API spechttp://html5-mediasource-api.googlecode.com/svn/trunk/draft-spec/mediasource-draft-spec.html for streaming data to a media tag. Please take a look at the spechttp://html5-mediasource-api.googlecode.com/svn/trunk/draft-spec/mediasource-draft-spec.htmland provide some feedback. I've tried to start with the simplest thing that would work and hope to expand from there if need be. For now, I'm intentionally not trying to solve the generic streaming file case because I believe there might be media specific requirements around handling seeking especially if we intend to support non-packetized media streams like WAV. If the feedback is generally positive on this approach, I'll start working on patches for WebKit Chrome so people can experiment with an actual implementation. Thanks, Aaron
Re: [whatwg] File API Streaming Blobs
Comments inline...
On Wed, Aug 10, 2011 at 2:05 PM, Charles Pritchard ch...@jumis.com wrote:
On 8/9/2011 9:38 AM, Aaron Colwell wrote:
FYI I'm working on an experimental extension to Chromium to allow media
data to be streamed into a media element via JavaScript. Here is the draft
spechttp://html5-mediasource-api.googlecode.com/svn/tags/0.2/draft-spec/mediasource-draft-spec.html
and
pending WebKit patch https://bugs.webkit.org/show_bug.cgi?id=64731 related
to this work. I have simple WebM VOD playback w/ seeking working where all
media data is fetched via XHR.
It's nice to see this patch.
Thanks. Hopefully I can get it landed soon so people can start playing with
it in Chrome Dev Channel builds.
I'm hoping to see streamed array buffers in XHR, though fetching in chunks
can work,
given the relatively small overhead of HTTP headers vs Video content.
Eventually I'd like to see streamed array buffers in XHR. For now I'm just
using range requests and allowing the JavaScript code determine how large
the ranges should be to control overhead.
The WHAWG specs have a Media Stream example which uses URL createObjectURL:
navigator.getUserMedia('video user', gotStream, noStream);
function gotStream(stream) {
video.src = URL.createObjectURL(stream);
http://www.whatwg.org/specs/web-apps/current-work/complete/video-conferencing-and-peer-to-peer-communication.html#dom-mediastream
The WHATWG spec seems closer to (mediaElement.createStream()).append()
semantics.
There was a previous discussion about this on WHATWG. There was concern
about providing compressed data to a MediaStream object since they are
basically format agnostic right now.
Both WHATWG and the draft spec agree on src=uri;
The benefit of src=uri is that it allows you to leverage all the existing
state transition and behavior defined in the spec.
File API has to toURL semantics on objects, simlar to the draft spec, for
getting filesystem:// uris.
My understanding: The draft spec is simpler, intended only to be used by
HTMLMediaElement
and only by one element at a time, without introducing a new object. In the
long
run, it may make sense to create a media stream object, consistent with the
WHATWG direction.
The draft spec was intended to be as simple as possible. Attaching this
functionality to HTMLMediaElement instead of
creating a MediaStream came out of discussions on whatwg
herehttp://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-July/032283.html
and
herehttp://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2011-July/032384.html.
I'm definitely open to revisiting this, but I got
the feeling that people wanted to see a more concrete implementation first.
I also like having this functionality part of
HTMLMediaElement because then I only have to deal with the HTMLMediaElement
during seeking instead of having to coordinate behavior
between the MediaStream the HTMLMediaElement.
On another note, Mozilla Labs has some experiments on recording video from
canvas (as well as general webcam, etc):
https://mozillalabs.com/rainbow/
https://github.com/mozilla/rainbow
https://github.com/mozilla/rainbow/blob/master/content/example_canvas.html
I'll take a look at this.
Aaron
Re: [whatwg] createObjectURL(stream) protocol issue
On 8/11/11 5:13 AM, Tommy Widenflycht (ᛏᚮᛘᛘᚤ) wrote: Would it be possible to give the associated URL for a mediastream to have its own protocol, for example mediastream:, instead of the proposed blob:? window . URL . createObjectURL(stream) Mints a Blob URL to refer to the given MediaStream. This would tremendously help the implementation. It's hard to mint new protocols. Can you be a bit more specific? Why would it help the implementation? -- A* Thanks in advance, Tommy
Re: [whatwg] createObjectURL(stream) protocol issue
On Thu, 11 Aug 2011, Tommy Widenflycht (�~[~O�~Z��~[~X�~[~X�~Z�) wrote: Would it be possible to give the associated URL for a mediastream to have its own protocol, for example mediastream:, instead of the proposed blob:? window . URL . createObjectURL(stream) Mints a Blob URL to refer to the given MediaStream. This would tremendously help the implementation. Could you elaborate? In what sense does it impact the implementation? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] File API Streaming Blobs
Þann mán 8.ágú 2011 20:31, skrifaði Simon Heckmann: Well, not directly an answer to your question, but the use case I had in mind is the following: A large encrypted video (e.g. HD movie with 2GB) file is stored using the File API, I then want to decrypt this file and start playing with only a minor delay. I do not want to decrypt the entire file before it can be viewed. As long as such as use case gets covered I am fine with everything. Has the decryption to happen above the File API?
Re: [whatwg] relationship between Document and HTMLDocument
Partial interface [1] was added for the 12 July 2011 – LCWD. It was designed to replace Supplemental [2]. I think the beginning of it was in a thread on public-script-coord [3]. [1] http://www.w3.org/TR/WebIDL/#dfn-partial-interface [2] http://www.w3.org/TR/2010/WD-WebIDL-20101021/#es-extended-attributes [3] http://lists.w3.org/Archives/Public/public-script-coord/2010OctDec/0084.html On 8/9/11, David Flanagan dflana...@mozilla.com wrote: On 8/9/11 1:58 PM, Ian Hickson wrote: On Tue, 9 Aug 2011, David Flanagan wrote: Possibly. I think an alternative is to make the HTML spec just add all the members to Document, and then define window.HTMLDocument as returning the Document interface object. This would make instanceof and monkeypatching work as today. So you'd declare HTMLDocument with the [NoInterfaceObject] extended attribute and then add attribute HTMLDocument to the Window interface? That would have the same effect, but what I had in mind was actually to change the HTML spec to not define an HTMLDocument interface, instead renaming it to Document and adding the 'partial' WebIDL modifier. We'd also have to do this for SVGDocument and other document objects; before doing this it would be good to see if it's something that is generally agreeable to everyone. Is the partial keyword a brand-new feature of WebIDL? I didn't see them discussed on public-script-coord at all... A partial interface sounds like it would work to me. That changes HTMLDocument from non-enumerable to enumerable, but that seems unlikely to be a compatibility issue. That works for me, I think. Could you elaborate on this? I'm not sure what you mean exactly. The HTMLDocument interface object is current (at least in FF, and per the WebIDL spec) non-enumerable. It doesn't show up in for/in loops on the window. If the HTML spec were to add an attribute to the Window object to define the HTMLDocument property, WebIDL would make that property enumerable. It would also change from a data property to an accessor property. I'm not arguing that these changes would be a problem, just noting them. The much bigger change, of course, is that HTMLDocument would be === Document. -- Sent from my mobile device
Re: [whatwg] Fixing undo on the Web - UndoManager and Transaction
On 11-08-05 1:01 PM, Ryosuke Niwa wrote: On Fri, Aug 5, 2011 at 9:57 AM, Jonas Sickingjo...@sicking.cc wrote: Why treat documentElement specially here? Just make the documentElement *not* have a undoManager by default and have it just use it's ancestor's, just like all other elements. The document is an ancestor of the documentElement after all. Anne had a different idea to make documentElement have the root undoManager to avoid adding a new property to Document. I personally like your approach but I don't know enough about Web IDLs to give a call. Anyone else have opinions on this? I think the confusion is arising because you chose to attach undoManager to elements, not nodes. Note that document _is_ a node in the DOM, but it's not an element. I think we should just modify the spec to attach undoManager to nodes. Once we have that, we don't need to treat documentElement specially at all, it just looks at its parent (the document node) and gets the undoManager from there. The only downside is that we should explicitly prohibit some node types from having an undoManager where it doesn't make sense (such as text nodes, comment nodes, etc.). We can enumerate them explicitly and say that accessing the undoManager on these types of nodes will throw. Does that make sense? Ehsan
Re: [whatwg] Fixing undo on the Web - UndoManager and Transaction
On Thu, Aug 11, 2011 at 2:53 PM, Ehsan Akhgari eh...@mozilla.com wrote: I think the confusion is arising because you chose to attach undoManager to elements, not nodes. Note that document _is_ a node in the DOM, but it's not an element. I think we should just modify the spec to attach undoManager to nodes. Once we have that, we don't need to treat documentElement specially at all, it just looks at its parent (the document node) and gets the undoManager from there. Makes sense. The only downside is that we should explicitly prohibit some node types from having an undoManager where it doesn't make sense (such as text nodes, comment nodes, etc.). We can enumerate them explicitly and say that accessing the undoManager on these types of nodes will throw. Alternatively, can we say that only Element and Document are allowed to have it? - Ryosuke
Re: [whatwg] Fixing undo on the Web - UndoManager and Transaction
On Thu, Aug 11, 2011 at 2:56 PM, Ryosuke Niwa rn...@webkit.org wrote: On Thu, Aug 11, 2011 at 2:53 PM, Ehsan Akhgari eh...@mozilla.com wrote: I think the confusion is arising because you chose to attach undoManager to elements, not nodes. Note that document _is_ a node in the DOM, but it's not an element. I think we should just modify the spec to attach undoManager to nodes. Once we have that, we don't need to treat documentElement specially at all, it just looks at its parent (the document node) and gets the undoManager from there. Makes sense. The only downside is that we should explicitly prohibit some node types from having an undoManager where it doesn't make sense (such as text nodes, comment nodes, etc.). We can enumerate them explicitly and say that accessing the undoManager on these types of nodes will throw. Alternatively, can we say that only Element and Document are allowed to have it? Yup, that's what I think we should do. Though it'll sort of fall out naturally since UndoManagers are by default only available on Documents, and the only way you can enable it on other Nodes is through attributes, which only exist on Elements. But mentioning it explicitly too is a good idea. / Jonas
