On 6 mrt. 2014, at 14:26, Brion Vibber <[email protected]> wrote: > > Spent some time tonight on audio/video sync; it's now working both with > native Web Audio API and with the Flash audio shim. > > Check out a video with someone talking, and marvel at the synchronization! > https://brionv.com/misc/ogv.js/demo/#file=Sneak_Preview_-_Wikipedia_VisualEditor.webm > (Hi Trevor!)
Wow, really impressive brion. ! > I also snuck in a size selection override, so you can pick the larger 480p > or original files (note that webm originals don't play as only a Theora > decoder is included so far), or down to the tiny 160p, to see the quality > and CPU usage differences. > > If there's no objection to use of a Flash shim, I'll keep doing some > experiments in that direction and see if I can rig up a fully-Flash version > that works in old versions of Internet Explorer as well. (The main ogv.js > needs no Flash as long as needed APIs are available, and even runs on iOS > though it's a bit slow for videos.) > Well, even if there are objections, it might be useful to make sure that it becomes a more suitable solution for other parties as well. And more interested parties would mean those might want to invest in the codebase. Should be a win DJ >> >> Audio-only files run great on iOS 7 devices. The 160p video transcodes we >> experimentally enabled recently run *great* on a shiny 64-bit iPhone 5s, >> but are still slightly too slow on older models. >> >> >> The Flash audio shim for IE is a very simple ActionScript3 program which >> accepts audio samples from the host page and outputs them -- no proprietary >> or patented codecs are in use. It builds to a .swf with the open-source >> Apache Flex SDK, so no proprietary software is needed to create or update >> it. >> >> I'm also doing some preliminary research on a fully Flash version, using >> the Crossbridge compiler[3] for the C codec libraries. Assuming it performs >> about as well as the JS does on modern browsers, this should give us a >> fallback for old versions of IE to supplement or replace the Cortado Java >> player... Before I go too far down that rabbit hole though I'd like to get >> peoples' opinions on using Flash fallbacks to serve browsers with open >> formats. >> >> As long as the scripts are open source and we're building them with an >> open source toolchain, and the entire purpose is to be a shim for missing >> browser feature support, does anyone have an objection? >> >> [3] https://github.com/adobe-flash/crossbridge >> >> -- brion >> >> >> On Mon, Oct 7, 2013 at 9:01 AM, Brion Vibber <[email protected]>wrote: >> >>> TL;DR SUMMARY: check out this short, silent, black & white video: >>> https://brionv.com/misc/ogv.js/demo/ -- anybody interested in a side >>> project on in-browser audio/video decoding fallback? >>> >>> >>> One of my pet peeves is that we don't have audio/video playback on many >>> systems, including default Windows and Mac desktops and non-Android mobile >>> devices, which don't ship with Theora or WebM video decoding. >>> >>> The technically simplest way to handle this is to transcode videos into >>> H.264 (.mp4 files) which is well supported by the troublesome browsers. >>> Unfortunately there are concerns about the patent licensing, which has held >>> us up from deploying any H.264 output options though all the software is >>> ready to go... >>> >>> While I still hope we'll get that resolved eventually, there is an >>> alternative -- client-side software decoding. >>> >>> >>> We have used the 'Cortado <http://www.theora.org/cortado/>' Java applet >>> to do fallback software decoding in the browser for a few years, but Java >>> applets are aggressively being deprecated on today's web: >>> >>> * no Java applets at all on major mobile browsers >>> * Java usually requires a manual install on desktop >>> * Java applets disabled by default for security on major desktop browsers >>> >>> Luckily, JavaScript engines have gotten *really fast* in the last few >>> years, and performance is getting well in line with what Java applets can >>> do. >>> >>> >>> As an experiment, I've built Xiph's ogg, vorbis, and theora C libraries >>> cross-compiled to JavaScript using >>> emscripten<https://github.com/kripken/emscripten>and written a wrapper that >>> decodes Theora video from an .ogv stream and >>> draws the frames into a <canvas> element: >>> >>> * demo: https://brionv.com/misc/ogv.js/demo/ >>> * code: https://github.com/brion/ogv.js >>> * blog & some details: >>> https://brionv.com/log/2013/10/06/ogv-js-proof-of-concept/ >>> >>> It's just a proof of concept -- the colorspace conversion is incomplete >>> so it's grayscale, there's no audio or proper framerate sync, and it >>> doesn't really stream data properly. But I'm pleased it works so far! >>> (Currently it breaks in IE, but I think I can fix that at least for 10/11, >>> possibly for 9. Probably not for 6/7/8.) >>> >>> Performance on iOS devices isn't great, but is better with lower >>> resolution files :) On desktop it's screaming fast for moderate >>> resolutions, and could probably supplement or replace Cortado with further >>> development. >>> >>> Is anyone interested in helping out or picking up the project to move it >>> towards proper playback? If not, it'll be one of my weekend "fun" projects >>> I occasionally tinker with off the clock. :) >>> >>> -- brion >>> >> >> > _______________________________________________ > Wikitech-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Multimedia mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/multimedia
