Hi Tom, Thanks for clarifying the meaning of verbosity from point 3. I haven't checked for any intrinsic differences in network path yet, but I didn't specify anything for either of mosesserver or moses other than default settings. I must clarify that I ran moses binary with regular I/O and not with -xml-input. The input length seems to be tied to number of words here than the number of characters. Tried a dummy input with about 4K chars on 4 words and that goes through. Also, by serial mode, I mean running mosesserver in single-threaded mode using the --serial option.
Barry, I know it is weird to give such unnaturally long inputs. This came up when I was testing out the limits of the system under which it would break. I do have mosesserver running on a fairly powerful machine (16 core, 16GB RAM) and running out of memory wasn't the issue here (that seems to happen somewhere near 10K word inputs on single-threaded). Single threaded moses translated the 430 word sentence in about 40 seconds. Breaking the large inputs into smaller chunks is definitely a course of action now, but that came up only after the crash which is weird in the first place. It is more of an academic interest into why this could be happening. Thanks, Nikhil On Fri, Oct 26, 2012 at 11:40 PM, <[email protected]> wrote: > Send Moses-support mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.mit.edu/mailman/listinfo/moses-support > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Moses-support digest..." > > > Today's Topics: > > 1. error during testing (Irene Huang) > 2. Multi-threaded mosesserver crashes with large inputs > (Nikhil Bojja) > 3. Re: Multi-threaded mosesserver crashes with large inputs > (Tom Hoar) > 4. Re: Multi-threaded mosesserver crashes with large inputs > ( Barry Haddow ) > 5. Re: Multi-threaded mosesserver crashes with large inputs > (Tom Hoar) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 26 Oct 2012 12:59:03 -0400 > From: Irene Huang <[email protected]> > Subject: [Moses-support] error during testing > To: [email protected] > Message-ID: > <CALMTtZk08KB= > [email protected]> > Content-Type: text/plain; charset="iso-8859-1" > > Hi, I have trained and tuned the model, now I am using > > > > ~/mosesdecoder/scripts/training/filter-model-given-input.plfiltered-newstest2011 > mert-work/moses.ini ~/corpus/ > newstest2011.true.fr \ > -Binarizer ~/mosesdecoder/bin/processPhraseTable > > to filter the phrase table. > > But the output is empty. And the errors are 1. segmentation fault 2. error: > empty lexicalized reordering file > > So does this mean it's out of memory error? > > Thanks > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20121026/d08cc702/attachment-0001.htm > > ------------------------------ > > Message: 2 > Date: Fri, 26 Oct 2012 12:25:58 -0700 > From: Nikhil Bojja <[email protected]> > Subject: [Moses-support] Multi-threaded mosesserver crashes with large > inputs > To: [email protected] > Message-ID: > <CAFtC8ijJJ=OMdDRyi20js+eYd+yQp+viXkmqCGn2cE= > [email protected]> > Content-Type: text/plain; charset="iso-8859-1" > > Hi all, > > I have a multi-threaded Mosesserver running which seems to crash when it > gets input sentences of size > 430 words. (~2000-2300 chars). Point 3 in > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc26 does seem to > indicate this behavior and I'm assuming verbosity equals long inputs. Also > this seems to be tied to number of words in input than anything else. > > Also, this seems to happen with both KenLM, and IRSTLM (LM's trained with > IRSTLM and run with KenLM ie). Heard from a friend that it happens with > SRILM as well. Moses binary or Mosesserver running in serial mode dont have > these issues though. xmlrpc doesn't seem to be the culprit either, as > serial mode works fine. Wondering if this was fixed in any of the recent > builds or if anyone has insights on how this could be fixed. > > Thank you, > Nikhil > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20121026/1971ff4f/attachment-0001.htm > > ------------------------------ > > Message: 3 > Date: Sat, 27 Oct 2012 06:55:33 +0700 > From: Tom Hoar <[email protected]> > Subject: Re: [Moses-support] Multi-threaded mosesserver crashes with > large inputs > To: <[email protected]> > Message-ID: > <[email protected]> > Content-Type: text/plain; charset="utf-8" > > > > Point 3 you indicate, "increasing the verbosity of moses" refers to > running Moses with the -threads option with the -verbose option set at a > higher number. The -verbose option controls the Moses log output. This > has nothing to do with the number of tokens in the source input. > > I've > never heard of a max limit on the source input token length. If this > only happens with mosesserver and not the moses binary with -xml-input, > have you looked at other differences between the two, for example, the > network path? Does your network have a buffer length of only 2,000 bytes > that would truncate input to mosesserver? > > On 2012-10-27 02:25, Nikhil > Bojja wrote: > > > Hi all, > > I have a multi-threaded Mosesserver running > which seems to crash when it gets input sentences of size > 430 words. > (~2000-2300 chars). Point 3 in > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc26 [1] does > seem to indicate this behavior and I'm assuming verbosity equals long > inputs. Also this seems to be tied to number of words in input than > anything else. > > Also, this seems to happen with both KenLM, and IRSTLM > (LM's trained with IRSTLM and run with KenLM ie). Heard from a friend > that it happens with SRILM as well. Moses binary or Mosesserver running > in serial mode dont have these issues though. xmlrpc doesn't seem to be > the culprit either, as serial mode works fine. Wondering if this was > fixed in any of the recent builds or if anyone has insights on how this > could be fixed. > > Thank you, > > Nikhil > > > Links: > ------ > [1] > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc26 > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20121027/b71ff7c0/attachment-0001.htm > > ------------------------------ > > Message: 4 > Date: Sat, 27 Oct 2012 07:27:25 +0100 > From: " Barry Haddow " <[email protected]> > Subject: Re: [Moses-support] Multi-threaded mosesserver crashes with > large inputs > To: " Nikhil Bojja " <[email protected]>, [email protected] > Message-ID: <[email protected]> > Content-Type: text/plain; charset="utf-8" > > Hi Nikhil > > Using such long sentences will increase the search space, which could lead > to Moses running out of memory. Is it possible to split the sentences into > smaller chunks? > > Cheers - Barry > > Sent from my ZX81 > > ----- Reply message ----- > From: "Nikhil Bojja" <[email protected]> > Date: Fri, Oct 26, 2012 20:25 > Subject: [Moses-support] Multi-threaded mosesserver crashes with large > inputs > To: <[email protected]> > > Hi all, > > I have a multi-threaded Mosesserver running which seems to crash when it > gets input sentences of size > 430 words. (~2000-2300 chars). Point 3 in > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc26 does seem to > indicate this behavior and I'm assuming verbosity equals long inputs. Also > this seems to be tied to number of words in input than anything else. > > Also, this seems to happen with both KenLM, and IRSTLM (LM's trained with > IRSTLM and run with KenLM ie). Heard from a friend that it happens with > SRILM as well. Moses binary or Mosesserver running in serial mode dont have > these issues though. xmlrpc doesn't seem to be the culprit either, as > serial mode works fine. Wondering if this was fixed in any of the recent > builds or if anyone has insights on how this could be fixed. > > Thank you, > Nikhil > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20121027/b572fd3f/attachment-0001.htm > -------------- next part -------------- > An embedded and charset-unspecified text was scrubbed... > Name: not available > Url: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20121027/b572fd3f/attachment-0001.bat > > ------------------------------ > > Message: 5 > Date: Sat, 27 Oct 2012 13:39:56 +0700 > From: Tom Hoar <[email protected]> > Subject: Re: [Moses-support] Multi-threaded mosesserver crashes with > large inputs > To: <[email protected]> > Message-ID: > <[email protected]> > Content-Type: text/plain; charset="utf-8" > > > > Barry, > > The long sentences were odd to me, too. Nonetheless, would > the search space account for his observation, "Moses binary or > Mosesserver running in serial mode dont have these issues"? > > I took > "serial mode" to mean mosesserver with -threads 0 or simply using moses > binary, and using the long sentence. Nikhil, can you confirm? > > On > 2012-10-27 13:27, Barry Haddow wrote: > > > Hi Nikhil > > > > Using such long > sentences will increase the search space, which could lead to Moses > running out of memory. Is it possible to split the sentences into > smaller chunks? > > > > Cheers - Barry > > > > Sent from my ZX81 > > > > ----- > Reply message ----- > > From: "Nikhil Bojja" > > Date: Fri, Oct 26, 2012 > 20:25 > > Subject: [Moses-support] Multi-threaded mosesserver crashes with > large inputs > > To: > > > > Hi all, > > > > I have a multi-threaded > Mosesserver running which seems to crash when it > > gets input sentences > of size > 430 words. (~2000-2300 chars). Point 3 in > > > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc26 [1] does > seem to > > indicate this behavior and I'm assuming verbosity equals long > inputs. Also > > this seems to be tied to number of words in input than > anything else. > > > > Also, this seems to happen with both KenLM, and > IRSTLM (LM's trained with > > IRSTLM and run with KenLM ie). Heard from a > friend that it happens with > > SRILM as well. Moses binary or Mosesserver > running in serial mode dont have > > these issues though. xmlrpc doesn't > seem to be the culprit either, as > > serial mode works fine. Wondering if > this was fixed in any of the recent > > builds or if anyone has insights > on how this could be fixed. > > > > Thank you, > > Nikhil > > > > Links: > ------ > [1] > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc26 > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.mit.edu/mailman/private/moses-support/attachments/20121027/8c71c3bf/attachment.htm > > ------------------------------ > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > End of Moses-support Digest, Vol 72, Issue 53 > ********************************************* >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
