On Thu, Nov 10, 2011 at 04:17:55PM -0700, Brad Huber wrote: > I am planning to write some code to allow blitting of gstreamer buffers into > osg textures. Eg for playing a video file or a network stream on to OSG > primitives. I did a preliminary search and did not see anyone having gone > down this path before.
OpenSceneGraph has basic support for using ffmpeg to render video into textures and hand the audio out to the application. I've fixed some issues with the ffmpeg plugin myself, but others remain. If you haven't already looked at osg::AudioStream and osg::ImageStream you'll want to see what you can do to match the API to allow programs to have the same interface for the different back end renders. I see the current ImageStream and AudioStream API as just for basic playback and as such it isn't all that feature rich, and I don't think it needs to be. I would have looked at it myself, but I haven't had time for it, but I think the most important addition is audio/video synchronization. The video has a reference time, but not the audio. It also needs to give an indication how much audio is left to read from the buffer. The buffer length might be redundant if you know what video frame the audio was supposed to play with. I actually got myself into a bind playing a live multicast rtsp stream with audio and video where I apparently wasn't reading audio fast enough causing the video to freeze, because the ffmpeg audio buffers must have filled up and it would stop decoding video. I could do without the audio in that case so I changed the code to just discard audio fast enough to prevent that problem. The other issue is the only way to get an audio stream is from the ImageStream, and to use the ImageStream with ffmpeg you have to have video. That means we couldn't playback a stream that only had audio, so we had to put some image frames in just to get the audio back out. -- David Fries <[email protected]> PGP pub CB1EE8F0 http://fries.net/~david/ _______________________________________________ osg-users mailing list [email protected] http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
