Hi David, If the LPCNet codec can run without additional dependencies, that'd be great! We have the opportunity to train it for specific speakers when we scrape content from educational sites on our server. The only issue is the size of the model.
Unfortunately we don't have much expertise on c or codec optimization. We would though be happy to maintain an Android package of the library in a form that would be easy for developers to use, and the documentation to go along with that. For what it's worth, we also have a fair few test devices that we could use to see how it performs. We'll send out an email once we have cleaned up and published the Android library, and look forward to the new LPCNet codec. Many thanks! -Mike On 24/02/2019 08:08, David Rowe wrote: > Hi Mike, > > Unfortunately the masking model work didn't lead to a viable Codec 2 mode. > > I think a LPCNet codec might suit your application well, I'll have a > release in the next few weeks for you to play with (around 2000 bits/s). > > Jean-Marc Valin (the author) has ported the code to C and to run on > general purpose CPUs, and we have some done some optimisation for > ARM-NEON. It's real time on a modern smart phone, and has scope for > further optimisation (help wanted here!) > > You don't need any special libraries, and it doesn't (really) need > training for specific speakers, although you could re-train if you > wanted to. > > Cheers, > David > > On 23/02/19 19:40, Mike Dawson wrote: >> Hi Codec2 list, >> >> I'm working together with Samih, looking at shrinking Khan Academy and >> other educational content for our offline library app. I've been trying >> to figure out optimal codec2 encoding / decoding parameters. >> >> We know who the speaker is in each clip. As far as I can understand, the >> best approach for us to achieve optimal results with a fixed speaker >> set, with having access to the original would be using the masking model >> outlined here: https://www.rowetel.com/?p=4454. Is this masking model >> per speaker, or per clip? >> >> I haven't managed to get the masking model running yet, but I made a >> basic script ( >> https://gist.github.com/mikedawson/1d66a1d35bd1538b2a9950246ef061a2 ) to >> generate comparison tables using a basket of clips and different >> parameter combinations. The audio from 4 Khan Academy clips with >> different codec2 settings is here: >> >> https://www.ustadmobile.com/files/codec2/out/ >> >> Using VP9 compression, the video in a 3.5 min clip can be shrunk to just >> under 100kB. If we used 2.4kbps codec2 for the audio, we could get the >> audio to around 70kB. As there are around 15,000 videos (only in >> English), codec2 could save a huge amount of space and bandwidth. That >> makes it around 60-70% smaller than the smallest 'mobile friendly' mp4 >> version from Khan Academy. >> >> On the LPCNet topic: this is definitely interesting, but will need >> further investigation. The examples from the masking model sounded >> pretty good. One obstacle I can see is the size of the training file. >> The app has to work offline and we have to keep the app size itself as >> small as possible. Perhaps with a limited speaker set, and no need to >> work on untrained files, this would not be so bad. We would also need to >> get the model to work with Tensorflow lite. Finally, in many places >> where low bandwidth and device space is an issue, the phones themselves >> often have limited capacity (Android 4.4 is still very much alive). >> >> Any further suggestion on what would be the current recommended / >> optimal approach for a fixed set of speakers would be much appreciated! >> We're very excited about the potential of this to make this education >> content more accessible. >> >> Thanks! >> >> -Mike >> >> CEO >> Ustad Mobile >> >> Email: m...@ustadmobile.com >> Web: www.ustadmobile.com >> Twitter: @ustadmobile >> Facebook: www.facebook.com/Ustad.Mobile >> >> >> _______________________________________________ >> Freetel-codec2 mailing list >> Freetel-codec2@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/freetel-codec2 >> > > _______________________________________________ > Freetel-codec2 mailing list > Freetel-codec2@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/freetel-codec2 -- Mike Dawson CEO Ustad Mobile Email: m...@ustadmobile.com Phone: +971 555 99 7043 Web: www.ustadmobile.com Twitter: @ustadmobile Facebook: www.facebook.com/Ustad.Mobile _______________________________________________ Freetel-codec2 mailing list Freetel-codec2@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freetel-codec2
