Hello ffmpeg gurus! I'd like to start a music-visualization-like project and I could really use some hints about the best approach and architecture to take.
Users should be able to dynamically submit audio-video files to the application. In near-real time, the application will decode some or all of the input files audio and video data and adaptatively produce a single output audio/video stream to be displayed live and saved to file. The compositor needs access to all input audio and video data with some amount of look-ahead room to make compositing decisions on the fly. To achieve the required performance, I need to minimize the number of video frame copies and conversions, including conversion to hardware textures if achievable. I'm eager to hear any technical/architectural advice you may give. At this point, my main question is about the choice between options A and B below: [A] "ffmpeg tool based". I've grown quite comfortable with the ffmpeg tool itself, including setting up encoder params and writing filter graphs. Would it make sense to start ffmpeg child processes and tap into them? If so, is there an existing filter that allows me to hook some sort of custom logic (in this case, exporting audio/video frames for another process to consume on the fly) or do I need to write a full blown new filter? [B] "libavxxx based". If not relying on the ffmpeg tool, the alternative is to rely on libavxxx directly. This scares me a bit in terms of complexity as I feel I'd be missing out on tons of built-in abstraction and corner cases taken care of by the ffmpeg tool, not to mention crash protection. On the plus side, there would be no filters or hooks to write, and no cross-process boundary to deal with. Ideally I'll use C++/Qt6 or TS/node for this project. Obviously I'll use plain C/libavxxx if I need to write a libav filter or to patch ffmpeg. Thanks a ton! Cyril _______________________________________________ ffmpeg-user mailing list [email protected] https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email [email protected] with subject "unsubscribe".
