Here some draft/braindump about side data - Side data is optional information stored inside container format and codec bitstream.
- It is stored per Frame/Packet since it varies. - Side data is stored as non structured data currently (as in a bytestream field) so all you know is its size, its type and nothing else. - Ideally demuxers fill Packets - The decoders forward what is in the Packets and/or in the codec bitstream so that Frames have it. - The encoder layer should take what is present in a Frame, encoded in the codec bitstream if possible and forward as Packet side data otherwise (or as well). - The muxer would take the information and store in the container if possible. Open questions: - since the side-data is forwarded as serialized and near-opaque how to "decode" and "encode" it? - side data and pure-data streams are quite similar. - some kind of side data could be forwarded in some ways, e.g. we could just forward the SEI as-is or decode it and have the s3d information out of it and ignore other information such as CC. - we assumed side data to be tiny, what to do when the assumption fails? - imagine some avoptions to enable/disable the whole behaviour or part of it. Personal Interest: I'd like to store-and-forward realtime time markers and forward them all over the chain. Ideally the source uses the NUT network/realtime extension to create them, they are forwarded down the encoding pipe till my extended flv muxer that then encodes them using the almost-standard adobe way to deliver the information and then send it over. The receiver then can compare the realtime time marker with the current time and know which is the whole chain delay. Vittorio's Interest: You have s3d information stored in SEI or in mkv, you want to forward it no matter which is the source and store it back no matter which is the container and codec. Bonus point if the application can actually read and understand them. Would be nice if we could flesh out something nice and put it in libav10 IMHO. lu _______________________________________________ libav-devel mailing list [email protected] https://lists.libav.org/mailman/listinfo/libav-devel
