This is a continuation of discussion about how to deal with media
elements playing media from MediaStreams when Pause is hit (and possibly
also seek, etc). See bug 774771
https://bugzilla.mozilla.org/show_bug.cgi?id=774771
Use cases:
1) realtime mediastream (from getUserMedia() or PeerConnection) as input
Perhaps using MediaStream Processing to modify the media before
playback
2) non-realtime mediastream (from captureStreamUntilEnded/etc) as input
For example, done as <video> -> MediaStream -> MediaStream
Processing to filter/effects -> <video>
For #1, on Pause you want to throw away all data and resume when
unpaused. Technically, there's some advantage to throwing away at the
output (since that means no delay to restart), but if user-scale delays
aren't possible in the MediaStream graph (via a MediaStream Processing
graph doing 'ducking' for example), then the dropping of data can occur
at the output, the input to MediaStreams, or in the code feeding the
MediaStream. If there is significant delay, you really don't want to
show second-old video for a while when you resume and then jump, and
you're prefer not to wait a second to resume - but that might be ok.
I know we had discussion on this; we need to decide on the right
way/place to handle it and do it. (Item 2 below *might* make this moot.)
For #2, you could react the same way, but use cases like "Playback video
with effects" would very much want the visible element's video controls
to control the source video element - pause would pause the source,
resume would resume it, seek would seek it, etc. Pause is the most
simple case; the others would require more complex proxying of controls.
If we define events on MediaStreams/Tracks that bubble up the graph,
then a video element could bubble up a Pause, Seek, etc request to the
source, which could react appropriately (realtime elements would just
start throwing away data; video_elem.createStreamUntilEnded() nodes
might just apply those commands to the video element. Note that Seek in
particular might be problematic unless the element can push *down* the
graph updates to the current time and start/end times, so the playback
element knows what to show in the UI.
There are some interesting cases to work out with composite and
fanned-out MediaStreamTracks/MediaStreams...
Maybe this is all too complex, or maybe it can be all dumped into the
application layer JS (though I don't think the app has that level of
control/interaction with the media element UI). Or we can decide we
don't care, and if you use MediaStreams at all you get no media element
UI for this type of thing - all UI must be in the app. (This may
confuse users, since they don't know how you internally plumbed your
page and streams.)
Comments?
Randell
_______________________________________________
dev-media mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-media
