I can get waveforms... but say "a" takes 1 second to speak. I get different waveforms over that 1 second... so I'm not matching against a single waveform, but many waveforms in succession. This seems like a tricky thing to match against.
What might be a good approach to matching values over a certain amount of time? Is AS3 fast enough to sync quick enough? I imagine it would need to check for all vowels every frame matching values in waveforms over a certain amount of time. Eric On Fri, Jun 4, 2010 at 8:56 AM, Eric E. Dolecki <edole...@gmail.com> wrote: > I've started implementing some code this morning in the hopes to match the > vowel "a" this morning. Of course there are several intonations for this > depending on the word it's located in, but if I can get a match on a naked > "a" I may be on to something. Like you said, I have a higher chance of > success since the voice is software generated and not from random people's > speech patterns. > > If I don't get something today I'm going to bail on the engine in the hopes > of finding something useful some other time. This isn't a critical feature > for me as I have the jaw moving with precision and the effect comes across. > Mouth shapes would be the icing on the cake. > > Eric > > > On Fri, Jun 4, 2010 at 8:34 AM, Karim Beyrouti <ka...@kurst.co.uk> wrote: > >> Yeh - not sure this will help >> >> however - a (very talented) colleague of mine worked on a simple speech >> recognition software for mobile - it was built to recognise about 20 >> commands with 90% success rate. >> >> His approach (in my simplistic terms) was: >> >> 1) get recordings / audio samples of the commands (in your case vowels - >> it should be easier as it's generated so you wont have to compare against >> too many/different intonations ) - >> 2) create / store a graph of the audio commands ( this used FFT (s) - to >> abstract and simplify, the pattern of the commands - the result was a square >> voice print graph ) >> 3) The stored patterns/voiceprints were then compared against the users >> voice recording. >> >> The trickiest part of this whole business were the Fast Fourier Transforms >> - these things get very complicated, and confuse the life out of me. Anyway, >> hopefully this >> will help you - seems like it might be the best approach. if you do crack >> it - you will end up with a simple voice recognition system. Which would be >> a brilliant and useful thing bit of code to >> have... >> >> hope this was of any use.. >> >> - karim >> >> On 4 Jun 2010, at 01:23, Karl DeSaulniers wrote: >> >> > I would try using that to figure out a way of maping the sounds and then >> translate that to your project. You are able to see the wave forms in >> soundbooth? Haven't used it. If so, can you run your cursor over it at any >> point to get the readings? Might be a little trivial, but may yeild a >> pattern that you can utilize. >> > >> > JAT >> > >> > Karl >> > >> > Sent from losPhone >> > >> > On Jun 3, 2010, at 6:18 PM, "Eric E. Dolecki" <edole...@gmail.com> >> wrote: >> > >> >> SoundBooth >> >> >> >> On Thu, Jun 3, 2010 at 6:39 PM, Karl DeSaulniers <k...@designdrumm.com >> >wrote: >> >> >> >>> Do you have SoundEdit? Or the like? >> >>> >> >>> >> >>> Karl >> >>> >> >>> >> >>> >> >>> On Jun 3, 2010, at 5:09 PM, Eric E. Dolecki wrote: >> >>> >> >>> I think I might make waveform bitmaps and then try and compare against >> the >> >>>> current waveform (block EQ) - and if it's a close match, then fire >> off >> >>>> specific vowel events. If that works, I could do consonants too. If >> this >> >>>> works, I'll do jumping jacks and shots of Jack. >> >>>> >> >>>> So how would I compare two bitmaps to see if a waveform ( >> >>>> On Thu, Jun 3, 2010 at 5:18 PM, Karl DeSaulniers < >> k...@designdrumm.com >> >>>>> wrote: >> >>>> >> >>>> If you need any of these files or can't find them, lmk and I can send >> off >> >>>>> list. >> >>>>> >> >>>>> Best, >> >>>>> >> >>>>> Karl >> >>>>> >> >>>>> >> >>>>> >> >>>>> On Jun 3, 2010, at 3:37 PM, Karl DeSaulniers wrote: >> >>>>> >> >>>>> Don't know if this will help, but have you looked into >> WaveAnalyzer.as >> >>>>> or >> >>>>> >> >>>>>> Flash MX - Audio: Sound completion event (The source files for this >> can >> >>>>>> be >> >>>>>> found in the Flash MX/Samples folder.) >> >>>>>> They both let you control the sound. I am thinking this will point >> you >> >>>>>> in >> >>>>>> a good direction. Its AS2 though. >> >>>>>> >> >>>>>> HTH, >> >>>>>> >> >>>>>> Karl >> >>>>>> >> >>>>>> >> >>>>>> On Jun 3, 2010, at 2:42 PM, Eric E. Dolecki wrote: >> >>>>>> >> >>>>>> Ya - I have the data for both things, but they extend over time and >> are >> >>>>>> >> >>>>>>> difficult to compare. It's the boiling down the signatures into >> >>>>>>> something >> >>>>>>> simple and being able to read the playing audio looking for the >> match >> >>>>>>> (or >> >>>>>>> near match). I thought about using bitmap data and trying to match >> up >> >>>>>>> waveforms, etc. but I don't know enough about it to pull that off. >> It >> >>>>>>> seems >> >>>>>>> like a hack in a way, but if it worked, who cares I suppose. >> >>>>>>> >> >>>>>>> On Thu, Jun 3, 2010 at 3:31 PM, Juan Pablo Califano < >> >>>>>>> califa010.flashcod...@gmail.com> wrote: >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>>>>>> I'm not Henrik, but I've done some lip-synch stuff for Disney. >> We >> >>>>>>>> did >> >>>>>>>> it pretty much the way Eric described--we just used amplitude. >> It's >> >>>>>>>> not as accurate as Disney would demand on a film, but it's ok in >> the >> >>>>>>>> kids' game market. >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>>>>>> I see, amplitudes could be just good enough for some stuff. >> >>>>>>>> >> >>>>>>>> Although the "speed" and the intensitiy of the speech could give >> >>>>>>>> misleading >> >>>>>>>> results, I think. I'm under the impression that you should >> somehow try >> >>>>>>>> to >> >>>>>>>> compare the shape of the waves (somehow simplifiy your input to >> some >> >>>>>>>> value >> >>>>>>>> of sets of values that are easier to compare, possibly in a "time >> >>>>>>>> window") >> >>>>>>>> and compare it in some meaningful way to precalculated samples to >> find >> >>>>>>>> a >> >>>>>>>> matching pattern. That's the part I have no clue about! >> >>>>>>>> >> >>>>>>>> Cheers >> >>>>>>>> Juan Pablo Califano >> >>>>>>>> >> >>>>>>>> 2010/6/3 Kerry Thompson <al...@cyberiantiger.biz> >> >>>>>>>> >> >>>>>>>> Juan Pablo Califano wrote: >> >>>>>>>> >> >>>>>>>>> >> >>>>>>>>> Wow. That was really uncalled for. >> >>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> That was my reaction, too. I didn't see Eric as >> complaining--just >> >>>>>>>>> asking. Maybe Henrik was just having a bad day. >> >>>>>>>>> >> >>>>>>>>> For me, the hard part, which you seem to imply is rather simple >> >>>>>>>>> here, >> >>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> is >> >>>>>>>>> >> >>>>>>>> >> >>>>>>>> *matching+ the input audio against said profiles. Admitedly, I >> don't >> >>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> know >> >>>>>>>>> >> >>>>>>>> >> >>>>>>>> anything about digital signal processing and audio programming in >> >>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> general, >> >>>>>>>>> >> >>>>>>>>> but "matching" sounds a bit vague. Perhaps you could enlighten >> us, I >> >>>>>>>>>> >> >>>>>>>>>> you >> >>>>>>>>> >> >>>>>>>> >> >>>>>>>> feel like. >> >>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> I'm not Henrik, but I've done some lip-synch stuff for Disney. >> We did >> >>>>>>>>> it pretty much the way Eric described--we just used amplitude. >> It's >> >>>>>>>>> not as accurate as Disney would demand on a film, but it's ok in >> the >> >>>>>>>>> kids' game market. >> >>>>>>>>> >> >>>>>>>>> Doing something more accurate would probably involve at least 6 >> mouth >> >>>>>>>>> positions, and if you're doing it in real time, you'd have to do >> a >> >>>>>>>>> reverse FFT. It can be done--there was a really good commercial >> >>>>>>>>> lip-synch program that generated Action Script to control mouth >> >>>>>>>>> positions. I don't know if it's still around--that was 5 years >> ago, >> >>>>>>>>> and it was pretty expensive (about $2,500 for one seat, I >> think). It >> >>>>>>>>> may even have been a Director Xtra that worked with a Flash >> Sprite, >> >>>>>>>>> but let's not talk about Director :-P >> >>>>>>>>> >> >>>>>>>>> Cordially, >> >>>>>>>>> >> >>>>>>>>> Kerry Thompson >> >>>>>>>>> _______________________________________________ >> >>>>>>>>> Flashcoders mailing list >> >>>>>>>>> Flashcoders@chattyfig.figleaf.com >> >>>>>>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> >>>>>>>>> >> >>>>>>>>> _______________________________________________ >> >>>>>>>>> >> >>>>>>>> Flashcoders mailing list >> >>>>>>>> Flashcoders@chattyfig.figleaf.com >> >>>>>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> >>>>>>>> >> >>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> -- >> >>>>>>> http://ericd.net >> >>>>>>> Interactive design and development >> >>>>>>> _______________________________________________ >> >>>>>>> Flashcoders mailing list >> >>>>>>> Flashcoders@chattyfig.figleaf.com >> >>>>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> >>>>>>> >> >>>>>>> >> >>>>>> Karl DeSaulniers >> >>>>>> Design Drumm >> >>>>>> http://designdrumm.com >> >>>>>> >> >>>>>> _______________________________________________ >> >>>>>> Flashcoders mailing list >> >>>>>> Flashcoders@chattyfig.figleaf.com >> >>>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> >>>>>> >> >>>>>> >> >>>>> Karl DeSaulniers >> >>>>> Design Drumm >> >>>>> http://designdrumm.com >> >>>>> >> >>>>> _______________________________________________ >> >>>>> Flashcoders mailing list >> >>>>> Flashcoders@chattyfig.figleaf.com >> >>>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> >>>>> >> >>>>> >> >>>> >> >>>> >> >>>> -- >> >>>> http://ericd.net >> >>>> Interactive design and development >> >>>> _______________________________________________ >> >>>> Flashcoders mailing list >> >>>> Flashcoders@chattyfig.figleaf.com >> >>>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> >>>> >> >>> >> >>> Karl DeSaulniers >> >>> Design Drumm >> >>> http://designdrumm.com >> >>> >> >>> _______________________________________________ >> >>> Flashcoders mailing list >> >>> Flashcoders@chattyfig.figleaf.com >> >>> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> >>> >> >> >> >> >> >> >> >> -- >> >> http://ericd.net >> >> Interactive design and development >> >> _______________________________________________ >> >> Flashcoders mailing list >> >> Flashcoders@chattyfig.figleaf.com >> >> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> > _______________________________________________ >> > Flashcoders mailing list >> > Flashcoders@chattyfig.figleaf.com >> > http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> >> >> _______________________________________________ >> Flashcoders mailing list >> Flashcoders@chattyfig.figleaf.com >> http://chattyfig.figleaf.com/mailman/listinfo/flashcoders >> > > > > -- > http://ericd.net > Interactive design and development > -- http://ericd.net Interactive design and development _______________________________________________ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
