Hi JM,
I ran your code on my Fedora 16 box and I ran into a segfault but after
looking at the code and noticing a few things you did differently than me I
narrowed down the issue to the asr_partial_result and asr_result functions.
Specifically, the segfault was coming when trying to set the value for
"uttid" with gststruct.set_value("uttid", uttid);. I changed the parameter
type for uttid in the method definition to string instead of GLib.Value and
bingo it worked. Then I added my language model and dictionary to the code
in the init_gst method and PocketSphinx is now printing out my limited
language definition when I speak it! That was a huge help to me! Thanks a
lot. Are you the original author of the tutorial? It would be great if we
could add this Vala port to the article so that people can take advantage
of it easily. Do you have the contacts to get that done or should I try?
I'm attaching the code with my small changes, including 1)adding the
language model and dictionary 2)changing the parameter types for uttid and
3) my revised pipeline definition which shows how I got gstreamer to
recognize my usb microphone using the device keyword. If you have access to
edit the article then I think it is worth mentioning the lmtool
<http://www.speech.cs.cmu.edu/tools/lmtool-new.html>from CMU which
makes it really easy to define a language model and
dictionary and include them with your code.
Regards
Brian
On Thu, Mar 22, 2012 at 6:39 PM, JM <[email protected]> wrote:
> Hello Brian
> I tried porting that python example to vala (attached file) and it
> doesn't work here (Fails at runtime during asr.set_property("configured",
> true) ).
> Your version crashes at the same point here.
>
> I'm on ubuntu 11.10
>
> I tested the pipeline with
> gst-launch fakesrc ! audioconvert ! audioresample ! vader name=vad
> auto-threshold=true ! pocketsphinx name=asr ! fakesink
> and it fails with 'SIGSEGV accessing address 0x2c' as soon as it is
> (automatically)set to PAUSED.
> Funny thing is that the python example seems to work. Just had a look at
> the console. I didn't do further tests because my computer lacks a
> microphone.
> Your code looks fine, so far. On my machine I just don't get to the point
> where you have the issues.
> Regards
>
>
> Am Mittwoch, den 21.03.2012, 22:01 -0400 schrieb Brian Duffy:
> > Just speaking the words that are defined in my language model and
> > dictionary. I think my problem is more severe than the words I am
> choosing
> > to speak. Besides, if PocketSphinx segfaults just because you say hmmm
> then
> > I am wasting my time.
> >
> > On Wed, Mar 21, 2012 at 6:40 PM, JM <[email protected]> wrote:
> >
> > > What exactly are you speaking into the microphone?
> > > Saying "null" or "hmm" may lead to segfauls ;)
> > >
> > >
> > > Am Mittwoch, den 21.03.2012, 14:38 -0400 schrieb Brian Duffy:
> > > > Sorry, forgot to mention that the segfault happens when I speak into
> the
> > > > microphone; so my pipeline and asr are working well enough to pick up
> > > > utterances from my microphone.
> > > >
> > > >
> > > >
> > > > ---------- Forwarded message ----------
> > > > From: Brian Duffy <[email protected]>
> > > > Date: Wed, Mar 21, 2012 at 2:30 PM
> > > > Subject: Porting "Using PocketSphinx with GStreamer and python" to
> Vala
> > > > To: vala-list <[email protected]>,
> > > > [email protected]
> > > >
> > > >
> > > > Hello all, I am trying to port
> > > > <http://goog_1501870203>this<
> > > http://cmusphinx.sourceforge.net/wiki/gstreamer>example
> > > > "Using PocketSphinx with GStreamer and Python" to Vala. I have an
> > > > example compiling but I am getting some warnings that I don't
> understand
> > > > and a segmentation fault when running. Also, I am not sure what data
> > > types
> > > > I should be using for the "text" and "uttid" params in the asrResult
> ,
> > > > asrPartialResult, partialResult and finalResult methods. My
> environment
> > > is
> > > > Fedora 16. PocketSphinx .7, latest gstreamer that comes with Fedora
> 16. I
> > > > am using Clutter (1.8) for the main loop because my application UI
> will
> > > > ultimately be written in Clutter. Any help would be greatly
> appreciated.
> > > > Here are my compile warnings and code .... thanks.
> [brian@duffybr2foodie]$
> > > > valac -X -O2 -g --pkg clutter-1.0 --pkg gstreamer-0.10 --pkg
> > > > gstreamer-interfaces-0.10 --pkg clutter-gst-1.0 cmu.vala
> > > > cmu.vala:25.24-25.48: warning: unhandled error `GLib.Error' pipeline
> =
> > > > (Pipeline)Gst.parse_launch(pipeDef); ^^^^^^^^^^^^^^^^^^^^^^^^^
> > > > cmu.vala:11.2-11.26: warning: field `Sphinx.vader' never used private
> > > > Gst.Element vader; ^^^^^^^^^^^^^^^^^^^^^^^^^ cmu.vala:112.6-112.26:
> > > > warning: local variable `sphinx' declared but never used var sphinx
> = new
> > > > Sphinx(); ^^^^^^^^^^^^^^^^^^^^^
> > > /home/brian/BitBucketRepos/foodie/cmu.vala:
> > > > In function ‘sphinx_asrPartialResult’:
> > > > /home/brian/BitBucketRepos/foodie/cmu.vala:51:2: warning: passing
> > > argument
> > > > 1 of ‘gst_structure_set_value’ discards ‘const’ qualifier from
> pointer
> > > > target type [enabled by default]
> > > > /usr/include/gstreamer-0.10/gst/gststructure.h:117:25: note: expected
> > > > ‘struct GstStructure *’ but argument is of type ‘const struct
> > > GstStructure
> > > > *’ /home/brian/BitBucketRepos/foodie/cmu.vala:52:2: warning: passing
> > > > argument 1 of ‘gst_structure_set_value’ discards ‘const’ qualifier
> from
> > > > pointer target type [enabled by default]
> > > > /usr/include/gstreamer-0.10/gst/gststructure.h:117:25: note: expected
> > > > ‘struct GstStructure *’ but argument is of type ‘const struct
> > > GstStructure
> > > > *’ /home/brian/BitBucketRepos/foodie/cmu.vala:54:2: warning: passing
> > > > argument 1 of ‘_gst_structure_copy0’ discards ‘const’ qualifier from
> > > > pointer target type [enabled by default] cmu.vala.c:287:17: note:
> > > expected
> > > > ‘gpointer’ but argument is of type ‘const struct GstStructure *’
> > > > /home/brian/BitBucketRepos/foodie/cmu.vala: In function
> > > ‘sphinx_asrResult’:
> > > > /home/brian/BitBucketRepos/foodie/cmu.vala:65:2: warning: passing
> > > argument
> > > > 1 of ‘gst_structure_set_value’ discards ‘const’ qualifier from
> pointer
> > > > target type [enabled by default]
> > > > /usr/include/gstreamer-0.10/gst/gststructure.h:117:25: note: expected
> > > > ‘struct GstStructure *’ but argument is of type ‘const struct
> > > GstStructure
> > > > *’ /home/brian/BitBucketRepos/foodie/cmu.vala:66:2: warning: passing
> > > > argument 1 of ‘gst_structure_set_value’ discards ‘const’ qualifier
> from
> > > > pointer target type [enabled by default]
> > > > /usr/include/gstreamer-0.10/gst/gststructure.h:117:25: note: expected
> > > > ‘struct GstStructure *’ but argument is of type ‘const struct
> > > GstStructure
> > > > *’ /home/brian/BitBucketRepos/foodie/cmu.vala:68:2: warning: passing
> > > > argument 1 of ‘_gst_structure_copy0’ discards ‘const’ qualifier from
> > > > pointer target type [enabled by default] cmu.vala.c:287:17: note:
> > > expected
> > > > ‘gpointer’ but argument is of type ‘const struct GstStructure *’
> > > > /home/brian/BitBucketRepos/foodie/cmu.vala: In function
> > > > ‘sphinx_app_message’:
> /home/brian/BitBucketRepos/foodie/cmu.vala:77:2:
> > > > warning: passing argument 1 of ‘_gst_structure_copy0’ discards
> ‘const’
> > > > qualifier from pointer target type [enabled by default]
> > > cmu.vala.c:287:17:
> > > > note: expected ‘gpointer’ but argument is of type ‘const struct
> > > > GstStructure *’ cmu.vala.c: In function ‘_vala_main’:
> > > > /home/brian/BitBucketRepos/foodie/cmu.vala:107:15: warning: ignoring
> > > return
> > > > value of ‘clutter_init’, declared with attribute warn_unused_result
> > > > [-Wunused-result] Compilation succeeded - 3 warning(s)
> > > >
> > > >
> > > >
> > > >
> > > > [code]
> > > > using Clutter;
> > > > using ClutterGst;
> > > > using Gst;
> > > >
> > > >
> > > > class Sphinx : GLib.Object {
> > > >
> > > >
> > > > // experimenting with gstreamer pocketsphinx
> > > > private Gst.Pipeline pipeline;
> > > > private Gst.Element vader;
> > > > dynamic Gst.Element asr;
> > > > private Gst.Bus bus;
> > > > private Gst.Structure msg;
> > > > private Gst.Structure my_struct;
> > > >
> > > > public Sphinx() {
> > > >
> > > >
> > > > string a = """pulsesrc
> > > > device="alsa_input.usb-AKM_AK5370-00-AK5370.analog-mono" !
> audioconvert
> > > > !""";
> > > > string b = """ audioresample ! vader name=vad auto-threshold=true !
> > > > pocketsphinx name=asr ! fakesink""";
> > > >
> > > >
> > > > string pipeDef = a + b;
> > > > pipeline = (Pipeline)Gst.parse_launch(pipeDef);
> > > >
> > > > asr = pipeline.get_by_name("asr");
> > > >
> > > > asr.set_property("lm", "4038.lm");
> > > > asr.set_property("dict", "4038.dic");
> > > >
> > > > asr.partial_result.connect(asrPartialResult);
> > > > asr.result.connect(asrResult);
> > > >
> > > > asr.set_property("configured", true);
> > > >
> > > > bus = pipeline.get_bus();
> > > > bus.add_signal_watch();
> > > > bus.message.connect(app_message);
> > > >
> > > > pipeline.set_state(Gst.State.PLAYING);
> > > >
> > > > }
> > > >
> > > >
> > > > private void asrPartialResult(Gst.Element asr, GLib.Value text,
> > > GLib.Value
> > > > uttid) {
> > > > my_struct = new Gst.Structure.empty("partial_result");
> > > >
> > > > my_struct.set_value("hyp", text);
> > > > my_struct.set_value("uttid", uttid);
> > > >
> > > > asr.post_message(new Gst.Message.application(asr,
> my_struct));
> > > > }
> > > >
> > > >
> > > > private void asrResult(Gst.Element asr, GLib.Value text, GLib.Value
> > > uttid) {
> > > >
> > > > stdout.printf("%s", "inside asrResult");
> > > > my_struct = new Gst.Structure.empty("result");
> > > >
> > > > my_struct.set_value("hyp", text);
> > > > my_struct.set_value("uttid", uttid);
> > > >
> > > > asr.post_message(new Gst.Message.application(asr,
> my_struct));
> > > > }
> > > >
> > > >
> > > > private void app_message(Gst.Bus bus, Gst.Message p_msg) {
> > > >
> > > > //stdout.printf("%s", "inside app_message");
> > > > msg = p_msg.get_structure();
> > > > string msgType = msg.get_name();
> > > >
> > > > if (msgType == "partial_result") {
> > > > partialResult(msg["hyp"], msg["uttid"]);
> > > > }else if (msgType == "result") {
> > > > finalResult(msg["hyp"], msg["uttid"]);
> > > > pipeline.set_state(Gst.State.PAUSED);
> > > > }
> > > > }
> > > >
> > > > private void partialResult(GLib.Value hyp, GLib.Value uttid) {
> > > >
> > > > stdout.printf("%s", (string)hyp);
> > > > }
> > > >
> > > > private void finalResult(GLib.Value hyp, GLib.Value uttid) {
> > > > stdout.printf("%s", (string)hyp);
> > > > }
> > > >
> > > >
> > > > }
> > > >
> > > >
> > > > int main (string[] args) {
> > > >
> > > > Clutter.init(ref args);
> > > > Gst.init(ref args);
> > > >
> > > > //ClutterGst.init(ref args);
> > > >
> > > > var sphinx = new Sphinx();
> > > >
> > > > Clutter.main();
> > > >
> > > > return 0;
> > > > }
> > > > [/code]
> > > > --
> > > > Duff
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > vala-list mailing list
> > > > [email protected]
> > > > http://mail.gnome.org/mailman/listinfo/vala-list
> > >
> > >
> > > _______________________________________________
> > > vala-list mailing list
> > > [email protected]
> > > http://mail.gnome.org/mailman/listinfo/vala-list
> > >
> >
> >
> >
>
>
>
--
Duff
//# Copyright (c) 2008 Carnegie Mellon University.
//#
//# You may modify and redistribute this file under the same terms as
//# the CMU Sphinx system. See
//# http://cmusphinx.sourceforge.net/html/LICENSE for more information.
// valac --pkg gstreamer-0.10 --pkg gtk+-2.0 shpinx_livedemo.vala
using Gtk;
using Gst;
public class DemoApp : GLib.Object {
private Gtk.Window window;
private Gtk.TextBuffer textbuf;
private dynamic Gst.Element asr;
private dynamic Gst.Pipeline pipeline;
private Gst.Element vader;
private Gtk.TextView text;
private Gtk.ToggleButton button;
////GStreamer/PocketSphinx Demo Application//
public DemoApp() {
////Initialize a DemoApp object//
this.init_gui();
this.init_gst();
}
private void init_gui() {
////Initialize the GUI components//
this.window = new Gtk.Window();
this.window.delete_event.connect( () => { Gtk.main_quit(); return false; });
this.window.set_default_size(400,200);
this.window.set_border_width(10);
var vbox = new Gtk.VBox(false, 0);
this.textbuf = new Gtk.TextBuffer(null);
text = new Gtk.TextView.with_buffer(this.textbuf);
text.set_wrap_mode(WrapMode.WORD);
vbox.pack_start(text, true, true, 0);
button = new Gtk.ToggleButton.with_label("Speak");
button.clicked.connect(this.button_clicked);
vbox.pack_start(button, false, false, 5);
this.window.add(vbox);
this.window.show_all();
}
private void init_gst() {
////Initialize the speech components//
string a = """pulsesrc device="alsa_input.usb-AKM_AK5370-00-AK5370.analog-mono" ! audioconvert !""";
string b = """ audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! fakesink""";
string pipeDef = a + b;
try {
//this.pipeline = (Gst.Pipeline) Gst.parse_launch("gconfaudiosrc ! audioconvert ! audioresample ! vader name=vad auto-threshold=true ! pocketsphinx name=asr ! fakesink");
this.pipeline = (Gst.Pipeline) Gst.parse_launch(pipeDef);
}
catch(Error e) {
print("%s\n", e.message);
}
this.asr = this.pipeline.get_by_name("asr");
this.asr.partial_result.connect(this.asr_partial_result);
this.asr.result.connect(this.asr_result);
this.asr.set_property("lm", "4038.lm");
this.asr.set_property("dict", "4038.dic");
this.asr.set_property("configured", true);
var bus = this.pipeline.get_bus();
bus.add_signal_watch();
bus.message.connect(this.application_message);
// bus.message.connect(this.application_message);
this.pipeline.set_state(Gst.State.PAUSED);
}
private void asr_partial_result(Gst.Element sender, string text, string uttid) {
//Forward partial result signals on the bus to the main thread.//
var gststruct = new Gst.Structure.empty("partial_result");
gststruct.set_value("hyp", text);
gststruct.set_value("uttid", uttid);
asr.post_message(new Gst.Message.application(this.asr, gststruct));
}
private void asr_result(Gst.Element sender, string text, string uttid) {
//Forward result signals on the bus to the main thread.//
var gststruct = new Gst.Structure.empty("result");
gststruct.set_value("hyp", text);
gststruct.set_value("uttid", uttid);
asr.post_message(new Gst.Message.application(this.asr, gststruct));
}
private void application_message(Gst.Bus bus, Gst.Message msg) {
//Receive application messages from the bus.//
if(msg.type != Gst.MessageType.APPLICATION)
return;
if(msg.get_structure() == null)
return;
string msgtype = msg.get_structure().get_name();
if(msgtype == "partial_result") {
GLib.Value hy = msg.get_structure().get_value("hyp");
GLib.Value ut = msg.get_structure().get_value("uttid");
this.partial_result(hy, ut);
}
else if(msgtype == "result") {
GLib.Value hy = msg.get_structure().get_value("hyp");
GLib.Value ut = msg.get_structure().get_value("uttid");
this.final_result(hy, ut);
this.pipeline.set_state(Gst.State.PAUSED);
this.button.set_active(false);
}
}
private void partial_result(GLib.Value hyp, GLib.Value uttid) {
//Delete any previous selection, insert text and select it.//
// All this stuff appears as one single action
this.textbuf.begin_user_action();
this.textbuf.delete_selection(true, this.text.get_editable());
this.textbuf.insert_at_cursor((string)hyp, ((string)hyp).length);
var ins = this.textbuf.get_insert();
Gtk.TextIter iter;
this.textbuf.get_iter_at_mark(out iter, ins);
// var iter = this.textbuf.get_iter_at_mark(ins);
iter.backward_chars(((string)hyp).length);
this.textbuf.move_mark(ins, iter);
this.textbuf.end_user_action();
}
private void final_result(GLib.Value hyp, GLib.Value uttid) {
//Insert the final result.//
// All this stuff appears as one single action
this.textbuf.begin_user_action();
this.textbuf.delete_selection(true, this.text.get_editable());
this.textbuf.insert_at_cursor(((string)hyp), ((string)hyp).length);
this.textbuf.end_user_action();
}
private void button_clicked(Gtk.Widget sender) {
//Handle button presses.//
if(((ToggleButton)sender).get_active()) {
((ToggleButton)sender).set_label("Stop");
this.pipeline.set_state(Gst.State.PLAYING);
}
else {
((ToggleButton)sender).set_label("Speak");
vader = this.pipeline.get_by_name("vad");
vader.set_property("silent", true);
}
}
}
void main(string[] args) {
Gtk.init(ref args);
Gst.init(ref args);
var app = new DemoApp();
Gtk.main();
}
_______________________________________________
vala-list mailing list
[email protected]
http://mail.gnome.org/mailman/listinfo/vala-list
