Re: [Moses-support] Ems: interpolating LM using IrstLM
Hi Mauro, IRSTLM provides a special tool for that. Here you can find more information about how to interpolate LMs using IRSTLM http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=LM_interpolatio n Daniel Von: moses-support-boun...@mit.edu [mailto:moses-support-boun...@mit.edu] Im Auftrag von Philipp Koehn Gesendet: 02 August 2012 00:35 An: Mauro Zanotti Cc: moses-support@mit.edu Betreff: Re: [Moses-support] Ems: interpolating LM using IrstLM Hi, yes, the current implementation relies on SRILM. But maybe someone from IRST can explain how to interpolate their models. -phi On Wed, Aug 1, 2012 at 3:37 PM, Mauro Zanotti mau.zano...@gmail.com wrote: Dear all, I trained 2 LM in EMS module, how can I interpolate them using irstlm instead of srilm? interpolate-lm.perl works only with srilm? Thank you in advance Mauro ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Ems: interpolating LM using IrstLM
Hi Mauro, Alongside the documentation pointed out by Daniel (which is the official IRSTLM documentation), you would need a few more things in order to interpolate LMs using IRSTLM... The interpolate-lm script would create a config file (lets say interp.wt.final) of the following format LMINTERPOLATION 2 0.439053 full path to your LM-1 0.560947 full path to your LM-2 However, IRSTLM does not allow you to create a single interpolated LM as SRILM does... Instead you can directly use the interpolated LMs in your Moses.ini by passing the final config file directly. [lmodel-file] 1 0 5 /home/mt/l_models/interp.wt.final But to have this functionality, you should have a relatively new Moses build and IRSTLM version 5.70.04... For further reference you could look at http://comments.gmane.org/gmane.comp.nlp.moses.user/6341 Hope this helps.. Thanks and Regards, Pratyush On Thu, Aug 2, 2012 at 9:16 AM, Daniel Schaut danielsh...@hotmail.comwrote: Hi Mauro, ** ** IRSTLM provides a special tool for that. Here you can find more information about how to interpolate LMs using IRSTLM http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=LM_interpolation ** ** Daniel ** ** *Von:* moses-support-boun...@mit.edu [mailto:moses-support-boun...@mit.edu] *Im Auftrag von *Philipp Koehn *Gesendet:* 02 August 2012 00:35 *An:* Mauro Zanotti *Cc:* moses-support@mit.edu *Betreff:* Re: [Moses-support] Ems: interpolating LM using IrstLM ** ** Hi, yes, the current implementation relies on SRILM. But maybe someone from IRST can explain how to interpolate their models. -phi On Wed, Aug 1, 2012 at 3:37 PM, Mauro Zanotti mau.zano...@gmail.com wrote: Dear all, ** ** I trained 2 LM in EMS module, how can I interpolate them using irstlm instead of srilm? interpolate-lm.perl works only with srilm? ** ** Thank you in advance Mauro ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ** ** ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Ems: interpolating LM using IrstLM
Thank you Pratyush and thank you all, I installed new version of Irstlm and Moses, but when I try to interpolate the LMs I get the following error: Reading /opt/tools/moses/scripts/ems/example/data-iw/lm-interpol/lmlist.init... Wrong input format. For my lmlist.init i took your example... LMINTERPOLATION 2 0.439053 /opt/tools/moses/scripts/ems/example/ex9watris/lm/Auto+Euro.lm.2 0.560947 /opt/tools/moses/scripts/ems/example/ex9watris/lm/LMEN.lm.1 Do you know how I can solve the problem? Thank you in advance Mauro On Thu, Aug 2, 2012 at 11:28 AM, Pratyush Banerjee pbaner...@computing.dcu.ie wrote: Hi Mauro, Alongside the documentation pointed out by Daniel (which is the official IRSTLM documentation), you would need a few more things in order to interpolate LMs using IRSTLM... The interpolate-lm script would create a config file (lets say interp.wt.final) of the following format LMINTERPOLATION 2 0.439053 full path to your LM-1 0.560947 full path to your LM-2 However, IRSTLM does not allow you to create a single interpolated LM as SRILM does... Instead you can directly use the interpolated LMs in your Moses.ini by passing the final config file directly. [lmodel-file] 1 0 5 /home/mt/l_models/interp.wt.final But to have this functionality, you should have a relatively new Moses build and IRSTLM version 5.70.04... For further reference you could look at http://comments.gmane.org/gmane.comp.nlp.moses.user/6341 Hope this helps.. Thanks and Regards, Pratyush On Thu, Aug 2, 2012 at 9:16 AM, Daniel Schaut danielsh...@hotmail.comwrote: Hi Mauro, ** ** IRSTLM provides a special tool for that. Here you can find more information about how to interpolate LMs using IRSTLM http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=LM_interpolation ** ** Daniel ** ** *Von:* moses-support-boun...@mit.edu [mailto: moses-support-boun...@mit.edu] *Im Auftrag von *Philipp Koehn *Gesendet:* 02 August 2012 00:35 *An:* Mauro Zanotti *Cc:* moses-support@mit.edu *Betreff:* Re: [Moses-support] Ems: interpolating LM using IrstLM ** ** Hi, yes, the current implementation relies on SRILM. But maybe someone from IRST can explain how to interpolate their models. -phi On Wed, Aug 1, 2012 at 3:37 PM, Mauro Zanotti mau.zano...@gmail.com wrote: Dear all, ** ** I trained 2 LM in EMS module, how can I interpolate them using irstlm instead of srilm? interpolate-lm.perl works only with srilm? ** ** Thank you in advance Mauro ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ** ** ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Ems: interpolating LM using IrstLM
Hi Mauro, When you try to interpolate the LMs using the interpolate-LM script in IRSTLM, the initial file should not have the 'LMINTERPOLATION' flag... Only the final version should have it... So initially your init file should look like: 2 0.5 /opt/tools/moses/scripts/ems/example/ex9watris/lm/Auto+Euro.lm.2 0.5 /opt/tools/moses/scripts/ems/example/ex9watris/lm/LMEN.lm.1 Then once you run the interpolation as specified in http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=LM_interpolation you should end up with the final configuration file which looks like: 2 0.439053 /opt/tools/moses/scripts/ems/example/ex9watris/lm/Auto+Euro.lm.2 0.560947 /opt/tools/moses/scripts/ems/example/ex9watris/lm/LMEN.lm.1 NB: The weights here are simply dummy weights, in your case these could be completely different depending on your models and data... Now You manually need to Add the LMINTERPOLATION flag in the first line before 2. and then pass it on to Moses Decoder Ideally this flag should have been set by the lm-interpolate script itself but it is a bug which IRSTLM promised to fix in next releases... Thanks and Regards, Pratyush On Thu, Aug 2, 2012 at 2:43 PM, Mauro Zanotti mau.zano...@gmail.com wrote: Thank you Pratyush and thank you all, I installed new version of Irstlm and Moses, but when I try to interpolate the LMs I get the following error: Reading /opt/tools/moses/scripts/ems/example/data-iw/lm-interpol/lmlist.init... Wrong input format. For my lmlist.init i took your example... LMINTERPOLATION 2 0.439053 /opt/tools/moses/scripts/ems/example/ex9watris/lm/Auto+Euro.lm.2 0.560947 /opt/tools/moses/scripts/ems/example/ex9watris/lm/LMEN.lm.1 Do you know how I can solve the problem? Thank you in advance Mauro On Thu, Aug 2, 2012 at 11:28 AM, Pratyush Banerjee pbaner...@computing.dcu.ie wrote: Hi Mauro, Alongside the documentation pointed out by Daniel (which is the official IRSTLM documentation), you would need a few more things in order to interpolate LMs using IRSTLM... The interpolate-lm script would create a config file (lets say interp.wt.final) of the following format LMINTERPOLATION 2 0.439053 full path to your LM-1 0.560947 full path to your LM-2 However, IRSTLM does not allow you to create a single interpolated LM as SRILM does... Instead you can directly use the interpolated LMs in your Moses.ini by passing the final config file directly. [lmodel-file] 1 0 5 /home/mt/l_models/interp.wt.final But to have this functionality, you should have a relatively new Moses build and IRSTLM version 5.70.04... For further reference you could look at http://comments.gmane.org/gmane.comp.nlp.moses.user/6341 Hope this helps.. Thanks and Regards, Pratyush On Thu, Aug 2, 2012 at 9:16 AM, Daniel Schaut danielsh...@hotmail.comwrote: Hi Mauro, ** ** IRSTLM provides a special tool for that. Here you can find more information about how to interpolate LMs using IRSTLM http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=LM_interpolation ** ** Daniel ** ** *Von:* moses-support-boun...@mit.edu [mailto: moses-support-boun...@mit.edu] *Im Auftrag von *Philipp Koehn *Gesendet:* 02 August 2012 00:35 *An:* Mauro Zanotti *Cc:* moses-support@mit.edu *Betreff:* Re: [Moses-support] Ems: interpolating LM using IrstLM ** ** Hi, yes, the current implementation relies on SRILM. But maybe someone from IRST can explain how to interpolate their models. -phi On Wed, Aug 1, 2012 at 3:37 PM, Mauro Zanotti mau.zano...@gmail.com wrote: Dear all, ** ** I trained 2 LM in EMS module, how can I interpolate them using irstlm instead of srilm? interpolate-lm.perl works only with srilm? ** ** Thank you in advance Mauro ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ** ** ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Ems: interpolating LM using IrstLM
Dear Mauro, could you please send me the command line you use? Probably, there is a mismatch in the versions. I would check immediately. Which version of IRSTLM and MOSES did you actually installed? Meanwhile could you try to edit the configuration file and remove the keyword LMINTERPOLATION Nicola (IRSTLM developer staff) PS: I would suggest you to send IRSTLM-specific questions to the mailing of IRSTLM user-irs...@list.fbk.eumailto:user-irs...@list.fbk.eu On Aug 2, 2012, at 3:43 PM, Mauro Zanotti wrote: Thank you Pratyush and thank you all, I installed new version of Irstlm and Moses, but when I try to interpolate the LMs I get the following error: Reading /opt/tools/moses/scripts/ems/example/data-iw/lm-interpol/lmlist.init... Wrong input format. For my lmlist.init i took your example... LMINTERPOLATION 2 0.439053 /opt/tools/moses/scripts/ems/example/ex9watris/lm/Auto+Euro.lm.2 0.560947 /opt/tools/moses/scripts/ems/example/ex9watris/lm/LMEN.lm.1 Do you know how I can solve the problem? Thank you in advance Mauro On Thu, Aug 2, 2012 at 11:28 AM, Pratyush Banerjee pbaner...@computing.dcu.iemailto:pbaner...@computing.dcu.ie wrote: Hi Mauro, Alongside the documentation pointed out by Daniel (which is the official IRSTLM documentation), you would need a few more things in order to interpolate LMs using IRSTLM... The interpolate-lm script would create a config file (lets say interp.wt.final) of the following format LMINTERPOLATION 2 0.439053 full path to your LM-1 0.560947 full path to your LM-2 However, IRSTLM does not allow you to create a single interpolated LM as SRILM does... Instead you can directly use the interpolated LMs in your Moses.ini by passing the final config file directly. [lmodel-file] 1 0 5 /home/mt/l_models/interp.wt.final But to have this functionality, you should have a relatively new Moses build and IRSTLM version 5.70.04... For further reference you could look at http://comments.gmane.org/gmane.comp.nlp.moses.user/6341 Hope this helps.. Thanks and Regards, Pratyush On Thu, Aug 2, 2012 at 9:16 AM, Daniel Schaut danielsh...@hotmail.commailto:danielsh...@hotmail.com wrote: Hi Mauro, IRSTLM provides a special tool for that. Here you can find more information about how to interpolate LMs using IRSTLM http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=LM_interpolation Daniel Von: moses-support-boun...@mit.edumailto:moses-support-boun...@mit.edu [mailto:moses-support-boun...@mit.edumailto:moses-support-boun...@mit.edu] Im Auftrag von Philipp Koehn Gesendet: 02 August 2012 00:35 An: Mauro Zanotti Cc: moses-support@mit.edumailto:moses-support@mit.edu Betreff: Re: [Moses-support] Ems: interpolating LM using IrstLM Hi, yes, the current implementation relies on SRILM. But maybe someone from IRST can explain how to interpolate their models. -phi On Wed, Aug 1, 2012 at 3:37 PM, Mauro Zanotti mau.zano...@gmail.commailto:mau.zano...@gmail.com wrote: Dear all, I trained 2 LM in EMS module, how can I interpolate them using irstlm instead of srilm? interpolate-lm.perl works only with srilm? Thank you in advance Mauro ___ Moses-support mailing list Moses-support@mit.edumailto:Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edumailto:Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Ems: interpolating LM using IrstLM
Thank you Pratyush and Nicola, I removed the keyword LMINTERPOLATION and all seems to work fine. Thank you Mauro On Thu, Aug 2, 2012 at 4:24 PM, Nicola Bertoldi berto...@fbk.eu wrote: Dear Mauro, could you please send me the command line you use? Probably, there is a mismatch in the versions. I would check immediately. Which version of IRSTLM and MOSES did you actually installed? Meanwhile could you try to edit the configuration file and remove the keyword LMINTERPOLATION Nicola (IRSTLM developer staff) PS: I would suggest you to send IRSTLM-specific questions to the mailing of IRSTLM user-irs...@list.fbk.eumailto:user-irs...@list.fbk.eu On Aug 2, 2012, at 3:43 PM, Mauro Zanotti wrote: Thank you Pratyush and thank you all, I installed new version of Irstlm and Moses, but when I try to interpolate the LMs I get the following error: Reading /opt/tools/moses/scripts/ems/example/data-iw/lm-interpol/lmlist.init... Wrong input format. For my lmlist.init i took your example... LMINTERPOLATION 2 0.439053 /opt/tools/moses/scripts/ems/example/ex9watris/lm/Auto+Euro.lm.2 0.560947 /opt/tools/moses/scripts/ems/example/ex9watris/lm/LMEN.lm.1 Do you know how I can solve the problem? Thank you in advance Mauro On Thu, Aug 2, 2012 at 11:28 AM, Pratyush Banerjee pbaner...@computing.dcu.iemailto:pbaner...@computing.dcu.ie wrote: Hi Mauro, Alongside the documentation pointed out by Daniel (which is the official IRSTLM documentation), you would need a few more things in order to interpolate LMs using IRSTLM... The interpolate-lm script would create a config file (lets say interp.wt.final) of the following format LMINTERPOLATION 2 0.439053 full path to your LM-1 0.560947 full path to your LM-2 However, IRSTLM does not allow you to create a single interpolated LM as SRILM does... Instead you can directly use the interpolated LMs in your Moses.ini by passing the final config file directly. [lmodel-file] 1 0 5 /home/mt/l_models/interp.wt.final But to have this functionality, you should have a relatively new Moses build and IRSTLM version 5.70.04... For further reference you could look at http://comments.gmane.org/gmane.comp.nlp.moses.user/6341 Hope this helps.. Thanks and Regards, Pratyush On Thu, Aug 2, 2012 at 9:16 AM, Daniel Schaut danielsh...@hotmail.com mailto:danielsh...@hotmail.com wrote: Hi Mauro, IRSTLM provides a special tool for that. Here you can find more information about how to interpolate LMs using IRSTLM http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=LM_interpolation Daniel Von: moses-support-boun...@mit.edumailto:moses-support-boun...@mit.edu [mailto:moses-support-boun...@mit.edumailto:moses-support-boun...@mit.edu] Im Auftrag von Philipp Koehn Gesendet: 02 August 2012 00:35 An: Mauro Zanotti Cc: moses-support@mit.edumailto:moses-support@mit.edu Betreff: Re: [Moses-support] Ems: interpolating LM using IrstLM Hi, yes, the current implementation relies on SRILM. But maybe someone from IRST can explain how to interpolate their models. -phi On Wed, Aug 1, 2012 at 3:37 PM, Mauro Zanotti mau.zano...@gmail.com mailto:mau.zano...@gmail.com wrote: Dear all, I trained 2 LM in EMS module, how can I interpolate them using irstlm instead of srilm? interpolate-lm.perl works only with srilm? Thank you in advance Mauro ___ Moses-support mailing list Moses-support@mit.edumailto:Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edumailto:Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
Re: [Moses-support] Ems: interpolating LM using IrstLM
Hi, yes, the current implementation relies on SRILM. But maybe someone from IRST can explain how to interpolate their models. -phi On Wed, Aug 1, 2012 at 3:37 PM, Mauro Zanotti mau.zano...@gmail.com wrote: Dear all, I trained 2 LM in EMS module, how can I interpolate them using irstlm instead of srilm? interpolate-lm.perl works only with srilm? Thank you in advance Mauro ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support ___ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support
