On Tue, Mar 3, 2015, at 03:18 AM, Robert O'Callahan wrote: > Are these are features we care about, or will care about? Can we address > them in simpler/better ways than with the MediaStream blocking concept? > Or > should we just ignore them, take out the blocking support, and add it > back > in if/when we need to?
I think those use cases represent a small subset of what the MSG is used for, or could be used for (in the future). There seem to be a gap between what the MSG was designed for, and what we are using it for today. We effectively use the MSG as a generic routing mechanism for real-time media streams. It seems important for Gecko to perform very well at taking real-time input data, processing it, and outputing it, as fast as possible (as in, wall clock time from input to output, and overall CPU time). In this context, input streams are, in real life, very unlikely to be unable to deliver data, and not delivering data is certainly not the common case, so it should not be something that is expensive to compute (or computed at all). I think it can be handled just outside the graph without much problems and less complexity: - Microphone are going to come in sync with audio in the rather near future (once we do full-duplex audio streams), and the graph is driven by the audio stream, so it cannot under-run. - HTMLMediaElement that are paused or are buffering should just insert under-run frames: frames that are rendered as silence when rendering is needed, but can be detected as not being part of the media. For video, we just keep the last frame. - Sinks that receive under-run frames decide what they want to do with them (for an AudioDestinationNode, that would be "play silence", for a MediaRecorder, that would be "ignore"). If there has been Web Audio API processing in between the source and the sink, this under-run information is lost, and I believe there is little that can be done. - PeerConnections can stretch audio and do all sorts of magic if audio packets are dropped, worst case, they insert just enough silence frame. We don't want to add under-run frames here, normal silence frames should do, I think. Additionally, blocking is (part of) what makes the MSG complicated to read and reason about: for example, getting the current time for a stream is a non-straightforward operation where it should be a subtraction. Pragmatically, I think we should look into removing "blocking" from the MSG, to move toward a piece of code that does one thing fast, easier to read. Paul. _______________________________________________ dev-media mailing list [email protected] https://lists.mozilla.org/listinfo/dev-media
