Queries are too fine-grained. The application i.e. Moses takes care of threads. Regarding the command-line query program, it isn't multi-threaded but probably should be. That said, people who care about performance should be using the library instead of the command-line program to avoid string processing overhead.
Kenneth On 04/28/2016 02:34 AM, koormoosh wrote: > Hi, Thanks - Is this happening else where, i.e. in the query step? > > On Thu, Apr 28, 2016 at 2:01 AM, <[email protected] > <mailto:[email protected]>> wrote: > > Send Moses-support mailing list submissions to > [email protected] <mailto:[email protected]> > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.mit.edu/mailman/listinfo/moses-support > or, via email, send a message with subject or body 'help' to > [email protected] > <mailto:[email protected]> > > You can reach the person managing the list at > [email protected] <mailto:[email protected]> > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Moses-support digest..." > > > Today's Topics: > > 1. Call for Participation: IEEE DIPDMWC2016 Moscow, Russia > (Sandra Evans) > 2. kenlm multithreading (koormoosh) > 3. Re: kenlm multithreading (Kenneth Heafield) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 27 Apr 2016 18:03:34 +0800 > From: Sandra Evans <[email protected] > <mailto:[email protected]>> > Subject: [Moses-support] Call for Participation: IEEE DIPDMWC2016 > Moscow, Russia > To: [email protected] <mailto:[email protected]> > Message-ID: > > <caejuhkvizroxu4v0xy4r0uwfhm_qfzrfsi_rncl0dknuj4t...@mail.gmail.com > > <mailto:caejuhkvizroxu4v0xy4r0uwfhm_qfzrfsi_rncl0dknuj4t...@mail.gmail.com>> > Content-Type: text/plain; charset="utf-8" > > 2016 Third International Conference on Digital Information > Processing, Data > Mining, and Wireless Communications (DIPDMWC) > > National Research Nuclear University MEPhI (Moscow Engineering Physics > Institute), Moscow, Russia > > July 06-08, 2016 > > http://goo.gl/6HTGyi > > =================================================================================================================== > > The conference welcomes papers on the following (but not limited to) > research topics: > > - Adaptive Signal Processing > - Parallel Programming & Processing > - Artificial Intelligence > - Expert Systems > - Image Processing > - Information Security and Cryptography > - Modulation, Coding, and Channel Analysis > - Multimedia Signal Processing > - Bioinformatics & Biomedical Imaging > - Biomedical Signal Processing > - Natural Language Processing > - Neural Networks and Genetic Algorithms > - Computer-Aided Surgery > - Data Compression and Watermarking > - Data Mining Techniques > - Ethics of Data Mining > - Risk Management and Analysis > - Data Classification and Clustering > - Abnormally and Outlier Detection > - Feature Extraction and Data Reduction > - Multi-Task Learning > - Optimization Techniques > - Data Cleaning and Processing > - Text and Web Mining > - Bluetooth and Personal Area Networks > - Wireless System Architecture > - Mobile Management in Wireless Networks > - Mobile Database Access and Design > - IP Multimedia Sub-Systems > - Key Management Protocols > - Mobile/ Wireless Network Modeling and Simulation > - Mobile / Wireless Network Planning > - Wireless Network Standard and Protocols > - Digital Right Management and Multimedia Protection > and many more... > > > =================================================================================================================== > Important Dates > > Submission Dates Open from now until June 6, 2016 > Notification of Acceptance 2-4 weeks from the submission date > Camera Ready Submission June 26, 2016 > Registration Deadline June 26, 2016 > Conference Dates July 6-8, 2016 > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > http://mailman.mit.edu/mailman/private/moses-support/attachments/20160427/6bb03766/attachment-0001.html > > ------------------------------ > > Message: 2 > Date: Thu, 28 Apr 2016 00:25:28 +1000 > From: koormoosh <[email protected] <mailto:[email protected]>> > Subject: [Moses-support] kenlm multithreading > To: [email protected] <mailto:[email protected]> > Message-ID: > > <can3_cdg_n4vs1_suojpnaysqr06wfc09ix4szbndf1yhpxk...@mail.gmail.com > > <mailto:can3_cdg_n4vs1_suojpnaysqr06wfc09ix4szbndf1yhpxk...@mail.gmail.com>> > Content-Type: text/plain; charset="utf-8" > > Hello, > > Out of curiosity, does anyone know if multithreading is enabled by > default > during the execution of lmplz of kenlm? I noticed the lmplz cpu > usage goes > up and down around 300% (sometimes). I didn't pass any parameter to > enable > multithreading in lmplz and couldn't find any parameter on the > website so I > assume it's done internally and by default. Does anyone know when it > decides to do multithreading? > > regards, > -k > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > http://mailman.mit.edu/mailman/private/moses-support/attachments/20160427/334b0cd7/attachment-0001.html > > ------------------------------ > > Message: 3 > Date: Wed, 27 Apr 2016 15:54:43 +0100 > From: Kenneth Heafield <[email protected] > <mailto:[email protected]>> > Subject: Re: [Moses-support] kenlm multithreading > To: [email protected] <mailto:[email protected]> > Message-ID: <[email protected] > <mailto:[email protected]>> > Content-Type: text/plain; charset=windows-1252 > > Yes, it uses threads when it wants to. There is no option to turn > threads off (and no code path that would do so). One has limited > control using block size and counts. Ideally it would be more parallel. > > Kenneth > > On 04/27/2016 03:25 PM, koormoosh wrote: > > Hello, > > > > Out of curiosity, does anyone know if multithreading is enabled by > > default during the execution of lmplz of kenlm? I noticed the > lmplz cpu > > usage goes up and down around 300% (sometimes). I didn't pass any > > parameter to enable multithreading in lmplz and couldn't find any > > parameter on the website so I assume it's done internally and by > > default. Does anyone know when it decides to do multithreading? > > > > regards, > > -k > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] <mailto:[email protected]> > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > ------------------------------ > > _______________________________________________ > Moses-support mailing list > [email protected] <mailto:[email protected]> > http://mailman.mit.edu/mailman/listinfo/moses-support > > > End of Moses-support Digest, Vol 114, Issue 58 > ********************************************** > > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
