Re: Voice Comparison in LiveCode?

[Paul Dupuis via use-livecode]

I have had an interest in getting a speech-to-text component created for
LiveCode, either as an old fashion External or, these days as a LC8/9 Widget

My skill at grabbing Sphinx and building it and figuring out how to wrap
and LCB interface around it is poor enough that it would take me a long
time to do.

Since this has come up, I'd like to pose two questions to the list:

1) is there someone out there with the expertise to create an LC8/9
Widget that wrapped Sphinx who would like to take this on?
AND
2) Are there others out there interested in having such a widget that
might co-fund this with me?

And I suppose a 3rd question to LiveCode, would this widget be a
candidate for the Feature Request crowd funding thing LiveCode has done
in the past?

Paul Dupuis
Researchware

On 12/28/2017 7:40 PM, Sean Cole (Pi) via use-livecode wrote:
> Hi Peter,
>
> I'm going to recommend to you this code source from CMUSphinx. Head for the
> main Github source and look at the Android build (link also below). Because
> this is in Java, you should be able to write something using the FFS in
> LCB. Ignore all the Gradle stuff. Just load in
> the pocketsphinx-android/src/main/java/)edu/cmu/pocketsphinx files and bind
> to the 'SpeechRecognizer' class and then the various handlers. Also check
> out the Sphinx4 project, particularly the allPhone stuff (
> https://github.com/cmusphinx/sphinx4/tree/master/sphinx4-core/src/main/java/edu/cmu/sphinx/linguist/allphone
> )
>
> https://cmusphinx.github.io/wiki/phonemerecognition/
>
> https://github.com/cmusphinx/pocketsphinx-android/
>
> Sean Cole
> *Pi Digital Productions Ltd*
>
>
>
>>> i'm developing an app for cheap Android tablets (e.g. Amazon Fire 7in)
>> that allows a user to practice speaking a set of words.  The app plays a
>> sample of a word and the user then tries to say the same word.  So far the
>> app can play sample words and capture the user's attempts for the same
>> words.  The sample words and user attempts are uncompressed WAV files.
>>> I'm trying to find the code to do the comparison of 2 WAV files.
>> Ideally, the code will be in the following formats (best first):
>>> 1. LiveCode
>>> 2. Pseudocode
>>> 3. Other code (Python, Java, C++ etc.)
>>> 4. Academic papers
>>>
>>> I'm considering 2 general methods:
>>>
>>> a. Compare 2 voice clips directly
>>> b. Convert 2 voice clips to text (using voice-to-text) and then compare
>> the words in text format
>>> Note that Ali Lloyd from the LiveCode team has developed various things
>> to help.  However I've hit problems as follows:
>>> a. Ali has wrapped a standard Android sound library that compares 2 WAV
>> files and gives a percentage match. However the comparison is either far
>> too forgiving or far too strict, i.e. highly unreliable.
>>> b. Ali has wrapped a standard Android voice-to-text library which works
>> well with the devices he's tried it on.  However, the Amazon tablets do not
>> support this Android library!
>>> Given the two developments from Ali both relied on preformed blackbox
>> code (Android Java libraries), i may have to implement a comparison
>> algorithm from scratch. A solution that's completely in LiveCode would have
>> several benefits:
>>> i. it may work!
>>> ii. it may work cross-platform
>>> iii. it may be understandable!
>>>
>>> General reading around this subject produces recommendations such as
>> using FFTs (Fast Fourier Transforms), MFCCs (Mel Frequency Cepstral
>> Coefficient), etc. but I can't find anything that gives an end-to-end
>> method, from sound in to comparative score out!
>>> Any help with this would be gratefully received!
>>>
>>> Peter
>>> --
>>> Peter Reid
>>> Loughborough, UK
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Voice Comparison in LiveCode?

[Sean Cole (Pi) via use-livecode]

Hi Peter,

I'm going to recommend to you this code source from CMUSphinx. Head for the
main Github source and look at the Android build (link also below). Because
this is in Java, you should be able to write something using the FFS in
LCB. Ignore all the Gradle stuff. Just load in
the pocketsphinx-android/src/main/java/)edu/cmu/pocketsphinx files and bind
to the 'SpeechRecognizer' class and then the various handlers. Also check
out the Sphinx4 project, particularly the allPhone stuff (
https://github.com/cmusphinx/sphinx4/tree/master/sphinx4-core/src/main/java/edu/cmu/sphinx/linguist/allphone
)

https://cmusphinx.github.io/wiki/phonemerecognition/

https://github.com/cmusphinx/pocketsphinx-android/

Sean Cole
*Pi Digital Productions Ltd*



> > i'm developing an app for cheap Android tablets (e.g. Amazon Fire 7in)
> that allows a user to practice speaking a set of words.  The app plays a
> sample of a word and the user then tries to say the same word.  So far the
> app can play sample words and capture the user's attempts for the same
> words.  The sample words and user attempts are uncompressed WAV files.
> >
> > I'm trying to find the code to do the comparison of 2 WAV files.
> Ideally, the code will be in the following formats (best first):
> >
> > 1. LiveCode
> > 2. Pseudocode
> > 3. Other code (Python, Java, C++ etc.)
> > 4. Academic papers
> >
> > I'm considering 2 general methods:
> >
> > a. Compare 2 voice clips directly
> > b. Convert 2 voice clips to text (using voice-to-text) and then compare
> the words in text format
> >
> > Note that Ali Lloyd from the LiveCode team has developed various things
> to help.  However I've hit problems as follows:
> >
> > a. Ali has wrapped a standard Android sound library that compares 2 WAV
> files and gives a percentage match. However the comparison is either far
> too forgiving or far too strict, i.e. highly unreliable.
> >
> > b. Ali has wrapped a standard Android voice-to-text library which works
> well with the devices he's tried it on.  However, the Amazon tablets do not
> support this Android library!
> >
> > Given the two developments from Ali both relied on preformed blackbox
> code (Android Java libraries), i may have to implement a comparison
> algorithm from scratch. A solution that's completely in LiveCode would have
> several benefits:
> >
> > i. it may work!
> > ii. it may work cross-platform
> > iii. it may be understandable!
> >
> > General reading around this subject produces recommendations such as
> using FFTs (Fast Fourier Transforms), MFCCs (Mel Frequency Cepstral
> Coefficient), etc. but I can't find anything that gives an end-to-end
> method, from sound in to comparative score out!
> >
> > Any help with this would be gratefully received!
> >
> > Peter
> > --
> > Peter Reid
> > Loughborough, UK
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Voice Comparison in LiveCode?

[Dave Kilroy via use-livecode]

Hi Peter

Yep what you are trying to do is indeed not easy if you want it to work on your 
friend’s Amazon tablet

Are you not tempted to use ‘standard’ voice-to-text on Android or iOS as per 
your method b and then buying your friend a non-crippled droid tablet?

If the answer is “no i want to stretch the envelope” then I like Phil’s 
stackoverflow link the best :)
https://stackoverflow.com/questions/17010516/how-to-detect-how-similar-a-speech-recording-is-to-another-speech-recording
 


Kind regards

Dave



> i'm developing an app for cheap Android tablets (e.g. Amazon Fire 7in) that 
> allows a user to practice speaking a set of words.  The app plays a sample of 
> a word and the user then tries to say the same word.  So far the app can play 
> sample words and capture the user's attempts for the same words.  The sample 
> words and user attempts are uncompressed WAV files. 
> 
> I'm trying to find the code to do the comparison of 2 WAV files.  Ideally, 
> the code will be in the following formats (best first): 
> 
> 1. LiveCode 
> 2. Pseudocode 
> 3. Other code (Python, Java, C++ etc.) 
> 4. Academic papers 
> 
> I'm considering 2 general methods: 
> 
> a. Compare 2 voice clips directly 
> b. Convert 2 voice clips to text (using voice-to-text) and then compare the 
> words in text format 
> 
> Note that Ali Lloyd from the LiveCode team has developed various things to 
> help.  However I've hit problems as follows: 
> 
> a. Ali has wrapped a standard Android sound library that compares 2 WAV files 
> and gives a percentage match. However the comparison is either far too 
> forgiving or far too strict, i.e. highly unreliable. 
> 
> b. Ali has wrapped a standard Android voice-to-text library which works well 
> with the devices he's tried it on.  However, the Amazon tablets do not 
> support this Android library! 
> 
> Given the two developments from Ali both relied on preformed blackbox code 
> (Android Java libraries), i may have to implement a comparison algorithm from 
> scratch. A solution that's completely in LiveCode would have several 
> benefits: 
> 
> i. it may work! 
> ii. it may work cross-platform 
> iii. it may be understandable! 
> 
> General reading around this subject produces recommendations such as using 
> FFTs (Fast Fourier Transforms), MFCCs (Mel Frequency Cepstral Coefficient), 
> etc. but I can't find anything that gives an end-to-end method, from sound in 
> to comparative score out! 
> 
> Any help with this would be gratefully received! 
> 
> Peter 
> -- 
> Peter Reid 
> Loughborough, UK 
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Voice Comparison in LiveCode?

[Marc Siskin via use-livecode]

Peter,

While I don’t have direct experience with these programs, there are several 
Open-Source code sets that may be appropriate to your use.

They are located at: http://www.speech.cs.cmu.edu/.  The Sphinx software set 
might be the most useful.

Marc Siskin

On Dec 27, 2017, at 10:04 AM, Rick Harrison via use-livecode 
> wrote:

Hi Peter,

What you are trying to do is extremely complex.

Each person’s voice print is unique to that person.
The waveform of one person pronouncing a
spoken phrase correctly could be quite different
from how someone else’s waveform looks when
spoken correctly, due to overtone harmonics,
voice pitch, the speed at which the user is
speaking etc.

Think about dictation software, and about how
it can still be like having a stupid secretary
which misses 5% or more of the words spoken.

It is a voice recognition problem which requires
a lot of “fuzzy logic” to get it right.  Companies
have spent millions of dollars, and tens of
thousands of hours developing these tools.

You need to find a codebase for this that
has already been developed, and that hopefully
is either open-source, (good luck with that one),
or you will have to license it from some company
for a steep price.  It will most probably also be
a large program, will require a lot of CPU
resources, and memory to run on a device.

Good luck, and let us know if you find a good
solution!

Rick


On Dec 27, 2017, at 7:16 AM, Peter Reid via use-livecode 
> wrote:

i'm developing an app for cheap Android tablets (e.g. Amazon Fire 7in) that 
allows a user to practice speaking a set of words.  The app plays a sample of a 
word and the user then tries to say the same word.  So far the app can play 
sample words and capture the user's attempts for the same words.  The sample 
words and user attempts are uncompressed WAV files.

I'm trying to find the code to do the comparison of 2 WAV files.  Ideally, the 
code will be in the following formats (best first):

1. LiveCode
2. Pseudocode
3. Other code (Python, Java, C++ etc.)
4. Academic papers

I'm considering 2 general methods:

a. Compare 2 voice clips directly
b. Convert 2 voice clips to text (using voice-to-text) and then compare the 
words in text format

Note that Ali Lloyd from the LiveCode team has developed various things to 
help.  However I've hit problems as follows:

a. Ali has wrapped a standard Android sound library that compares 2 WAV files 
and gives a percentage match. However the comparison is either far too 
forgiving or far too strict, i.e. highly unreliable.

b. Ali has wrapped a standard Android voice-to-text library which works well 
with the devices he's tried it on.  However, the Amazon tablets do not support 
this Android library!

Given the two developments from Ali both relied on preformed blackbox code 
(Android Java libraries), i may have to implement a comparison algorithm from 
scratch. A solution that's completely in LiveCode would have several benefits:

i. it may work!
ii. it may work cross-platform
iii. it may be understandable!

General reading around this subject produces recommendations such as using FFTs 
(Fast Fourier Transforms), MFCCs (Mel Frequency Cepstral Coefficient), etc. but 
I can't find anything that gives an end-to-end method, from sound in to 
comparative score out!

Any help with this would be gratefully received!

Peter
--
Peter Reid
Loughborough, UK


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

---
Marc Siskin
Manager, Modern Language Resource Center
Carnegie Mellon University
msis...@andrew.cmu.edu



___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Voice Comparison in LiveCode?

[Phil Davis via use-livecode]

Hi Peter,

Here is some additional input:

https://stackoverflow.com/questions/17010516/how-to-detect-how-similar-a-speech-recording-is-to-another-speech-recording

HTH -
Phil Davis


On 12/27/17 4:16 AM, Peter Reid via use-livecode wrote:

i'm developing an app for cheap Android tablets (e.g. Amazon Fire 7in) that 
allows a user to practice speaking a set of words.  The app plays a sample of a 
word and the user then tries to say the same word.  So far the app can play 
sample words and capture the user's attempts for the same words.  The sample 
words and user attempts are uncompressed WAV files.

I'm trying to find the code to do the comparison of 2 WAV files.  Ideally, the 
code will be in the following formats (best first):

1. LiveCode
2. Pseudocode
3. Other code (Python, Java, C++ etc.)
4. Academic papers

I'm considering 2 general methods:

a. Compare 2 voice clips directly
b. Convert 2 voice clips to text (using voice-to-text) and then compare the 
words in text format

Note that Ali Lloyd from the LiveCode team has developed various things to 
help.  However I've hit problems as follows:

a. Ali has wrapped a standard Android sound library that compares 2 WAV files 
and gives a percentage match. However the comparison is either far too 
forgiving or far too strict, i.e. highly unreliable.

b. Ali has wrapped a standard Android voice-to-text library which works well 
with the devices he's tried it on.  However, the Amazon tablets do not support 
this Android library!

Given the two developments from Ali both relied on preformed blackbox code 
(Android Java libraries), i may have to implement a comparison algorithm from 
scratch. A solution that's completely in LiveCode would have several benefits:

i. it may work!
ii. it may work cross-platform
iii. it may be understandable!

General reading around this subject produces recommendations such as using FFTs 
(Fast Fourier Transforms), MFCCs (Mel Frequency Cepstral Coefficient), etc. but 
I can't find anything that gives an end-to-end method, from sound in to 
comparative score out!

Any help with this would be gratefully received!

Peter
--
Peter Reid
Loughborough, UK


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode



--
Phil Davis


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Voice Comparison in LiveCode?

[Rick Harrison via use-livecode]

Hi Peter,

What you are trying to do is extremely complex.

Each person’s voice print is unique to that person.
The waveform of one person pronouncing a
spoken phrase correctly could be quite different
from how someone else’s waveform looks when
spoken correctly, due to overtone harmonics,
voice pitch, the speed at which the user is
speaking etc.  

Think about dictation software, and about how 
it can still be like having a stupid secretary
which misses 5% or more of the words spoken.

It is a voice recognition problem which requires
a lot of “fuzzy logic” to get it right.  Companies
have spent millions of dollars, and tens of
thousands of hours developing these tools.

You need to find a codebase for this that
has already been developed, and that hopefully
is either open-source, (good luck with that one),
or you will have to license it from some company
for a steep price.  It will most probably also be 
a large program, will require a lot of CPU
resources, and memory to run on a device.

Good luck, and let us know if you find a good
solution!

Rick


> On Dec 27, 2017, at 7:16 AM, Peter Reid via use-livecode 
>  wrote:
> 
> i'm developing an app for cheap Android tablets (e.g. Amazon Fire 7in) that 
> allows a user to practice speaking a set of words.  The app plays a sample of 
> a word and the user then tries to say the same word.  So far the app can play 
> sample words and capture the user's attempts for the same words.  The sample 
> words and user attempts are uncompressed WAV files.
> 
> I'm trying to find the code to do the comparison of 2 WAV files.  Ideally, 
> the code will be in the following formats (best first):
> 
> 1. LiveCode
> 2. Pseudocode
> 3. Other code (Python, Java, C++ etc.)
> 4. Academic papers
> 
> I'm considering 2 general methods:
> 
> a. Compare 2 voice clips directly
> b. Convert 2 voice clips to text (using voice-to-text) and then compare the 
> words in text format
> 
> Note that Ali Lloyd from the LiveCode team has developed various things to 
> help.  However I've hit problems as follows:
> 
> a. Ali has wrapped a standard Android sound library that compares 2 WAV files 
> and gives a percentage match. However the comparison is either far too 
> forgiving or far too strict, i.e. highly unreliable.
> 
> b. Ali has wrapped a standard Android voice-to-text library which works well 
> with the devices he's tried it on.  However, the Amazon tablets do not 
> support this Android library!
> 
> Given the two developments from Ali both relied on preformed blackbox code 
> (Android Java libraries), i may have to implement a comparison algorithm from 
> scratch. A solution that's completely in LiveCode would have several benefits:
> 
> i. it may work!
> ii. it may work cross-platform
> iii. it may be understandable!
> 
> General reading around this subject produces recommendations such as using 
> FFTs (Fast Fourier Transforms), MFCCs (Mel Frequency Cepstral Coefficient), 
> etc. but I can't find anything that gives an end-to-end method, from sound in 
> to comparative score out!
> 
> Any help with this would be gratefully received!
> 
> Peter
> --
> Peter Reid
> Loughborough, UK
> 
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Voice Comparison in LiveCode?

[Richmond Mathewson via use-livecode]

Not sure about that:

http://www.webring.org/l/rd?ring=runtimerevoluti1;id=15;url=http%3A%2F%2Fmarksmith%2Eon%2Drev%2Ecom%2Frevstuff%2F

Richmond.

On 27/12/17 3:16 pm, Paul Dupuis via use-livecode wrote:

On 12/27/2017 7:16 AM, Peter Reid via use-livecode wrote:

Any help with this would be gratefully received!

Peter
--
Peter Reid
Loughborough, UK


I am not sure how much help this may be but Mark Smith (email on the
stack is m...@maseurope.net) create a stack called "AudioWaveform 0.94"
ages ago that displays the waveform of an uncompressed AIF or WAV file -
all written in LiveCode.

I am not sure where to find the sample stack itself - possibly in the
Sample Stack library or a Google search?!? I have a copy I downloaded a
long time ago I could email you off list. I just tried running it under
LC 8.1.x and there were script errors as it was created in a pretty old
version of LiveCode - pre LC5.x, possibly pre LC 4.x

Paul Dupuis
Researchware

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Voice Comparison in LiveCode?

[Paul Dupuis via use-livecode]

On 12/27/2017 7:16 AM, Peter Reid via use-livecode wrote:
> Any help with this would be gratefully received!
>
> Peter
> --
> Peter Reid
> Loughborough, UK
>

I am not sure how much help this may be but Mark Smith (email on the
stack is m...@maseurope.net) create a stack called "AudioWaveform 0.94"
ages ago that displays the waveform of an uncompressed AIF or WAV file -
all written in LiveCode.

I am not sure where to find the sample stack itself - possibly in the
Sample Stack library or a Google search?!? I have a copy I downloaded a
long time ago I could email you off list. I just tried running it under
LC 8.1.x and there were script errors as it was created in a pretty old
version of LiveCode - pre LC5.x, possibly pre LC 4.x

Paul Dupuis
Researchware

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Voice Comparison in LiveCode?

[Richmond Mathewson via use-livecode]

Oh, sorry: I thought you were going to convert the sound files into text 
files rather than

use a voice-to-text application.

I wonder how accurate voice-to-text transcription is, and how you can 
squeeze that into your app

for Android tablets.

Richmond.

On 27/12/17 2:49 pm, Richmond Mathewson wrote:

I'd probably do the text comparison.

Although, having said that, as a LInguist, I wonder what value that 
would have
as it would ONLY show that the two sound files were different, it 
would not,
for the sake of argument, show HOW the second one differed from the 
first one

phonetically.

Richmond.

On 27/12/17 2:16 pm, Peter Reid via use-livecode wrote:

i'm developing an app for cheap Android tablets (e.g. Amazon Fire 7in) that 
allows a user to practice speaking a set of words.  The app plays a sample of a 
word and the user then tries to say the same word.  So far the app can play 
sample words and capture the user's attempts for the same words.  The sample 
words and user attempts are uncompressed WAV files.

I'm trying to find the code to do the comparison of 2 WAV files.  Ideally, the 
code will be in the following formats (best first):

1. LiveCode
2. Pseudocode
3. Other code (Python, Java, C++ etc.)
4. Academic papers

I'm considering 2 general methods:

a. Compare 2 voice clips directly
b. Convert 2 voice clips to text (using voice-to-text) and then compare the 
words in text format

Note that Ali Lloyd from the LiveCode team has developed various things to 
help.  However I've hit problems as follows:

a. Ali has wrapped a standard Android sound library that compares 2 WAV files 
and gives a percentage match. However the comparison is either far too 
forgiving or far too strict, i.e. highly unreliable.

b. Ali has wrapped a standard Android voice-to-text library which works well 
with the devices he's tried it on.  However, the Amazon tablets do not support 
this Android library!

Given the two developments from Ali both relied on preformed blackbox code 
(Android Java libraries), i may have to implement a comparison algorithm from 
scratch. A solution that's completely in LiveCode would have several benefits:

i. it may work!
ii. it may work cross-platform
iii. it may be understandable!

General reading around this subject produces recommendations such as using FFTs 
(Fast Fourier Transforms), MFCCs (Mel Frequency Cepstral Coefficient), etc. but 
I can't find anything that gives an end-to-end method, from sound in to 
comparative score out!

Any help with this would be gratefully received!

Peter
--
Peter Reid
Loughborough, UK


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode




___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: Voice Comparison in LiveCode?

[Richmond Mathewson via use-livecode]

I'd probably do the text comparison.

Although, having said that, as a LInguist, I wonder what value that 
would have

as it would ONLY show that the two sound files were different, it would not,
for the sake of argument, show HOW the second one differed from the 
first one

phonetically.

Richmond.

On 27/12/17 2:16 pm, Peter Reid via use-livecode wrote:

i'm developing an app for cheap Android tablets (e.g. Amazon Fire 7in) that 
allows a user to practice speaking a set of words.  The app plays a sample of a 
word and the user then tries to say the same word.  So far the app can play 
sample words and capture the user's attempts for the same words.  The sample 
words and user attempts are uncompressed WAV files.

I'm trying to find the code to do the comparison of 2 WAV files.  Ideally, the 
code will be in the following formats (best first):

1. LiveCode
2. Pseudocode
3. Other code (Python, Java, C++ etc.)
4. Academic papers

I'm considering 2 general methods:

a. Compare 2 voice clips directly
b. Convert 2 voice clips to text (using voice-to-text) and then compare the 
words in text format

Note that Ali Lloyd from the LiveCode team has developed various things to 
help.  However I've hit problems as follows:

a. Ali has wrapped a standard Android sound library that compares 2 WAV files 
and gives a percentage match. However the comparison is either far too 
forgiving or far too strict, i.e. highly unreliable.

b. Ali has wrapped a standard Android voice-to-text library which works well 
with the devices he's tried it on.  However, the Amazon tablets do not support 
this Android library!

Given the two developments from Ali both relied on preformed blackbox code 
(Android Java libraries), i may have to implement a comparison algorithm from 
scratch. A solution that's completely in LiveCode would have several benefits:

i. it may work!
ii. it may work cross-platform
iii. it may be understandable!

General reading around this subject produces recommendations such as using FFTs 
(Fast Fourier Transforms), MFCCs (Mel Frequency Cepstral Coefficient), etc. but 
I can't find anything that gives an end-to-end method, from sound in to 
comparative score out!

Any help with this would be gratefully received!

Peter
--
Peter Reid
Loughborough, UK


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode