Thanks Tom - I've committed your changes. On Wednesday 18 January 2012 08:42:30 [email protected] wrote: > Herve, I agree that unbuffered output is fine, but others might have > need for it. So, I added the -b option to both and I've been using for a > couple of weeks. I've attached them here because I don't have github > access. > > The tokenizer.perl default is still buffered and the new -b option > disables Perl buffering. The detokenizer removes the %|++; line, which > changes the script's default to buffered. The new -b option disables > Perl buffering. The usage help prompt is also updated with the changes. > > Maybe someone can update the repository. > > Thanks, > Tom > > On Mon, 09 Jan 2012 12:59:30 +0000, Herve Saint-Amand > > <[email protected]> wrote: > > Hi, > > > > I added that $|++; a while ago so that a script could fork off > > detokenizer.perl, keep it running as a child process, feed it input > > line > > by line and synchronously read back the corresponding output. > > > > As far as I can see it's OK to just always unbuffer output. I don't > > think it can break anything, but honestly I don't know what effect it > > has on I/O performance. For what it's worth I always add that line to > > every Perl script I write. > > > > I'm cool with the command-line switch too. > > > > Herve > > > > On 07/01/12 11:58, Tom Hoar wrote: > >> I just found that detokenizer.perl has a variant of the proposal > >> below > >> on lines 8 & 9. > >> > >> # This added by Herve Saint-Amand for compatibility with > >> translate.cgi > >> $|++; > >> > >> For consistency between the two scripts, I propose adding a > >> command-line > >> option to both that disables buffering if the user wishes, and a > >> line in > >> the usage help. > >> > >> while (@ARGV) { > >> $_ = shift; > >> /^-b$/ && ($| = 1, next); > >> > >> if ($HELP) { > >> print " -b ... disable Perl buffering.\n"; > >> > >> On Thu, 05 Jan 2012 20:28:01 +0700, Tom Hoar > >> > >> <[email protected]> wrote: > >>> The current tokenizer.perl and detokenizer.perl script buffer the > >>> print output. Adding this line to the beginning of each script: > >>> > >>> $| = 1; > >>> > >>> disables buffering and forces Perl to flush each line. Can this be > >>> added? > >>> > >>> Thanks, > >>> Tom > >> > >> _______________________________________________ > >> Moses-support mailing list > >> [email protected] > >> http://mailman.mit.edu/mailman/listinfo/moses-support > -- Barry Haddow University of Edinburgh +44 (0) 131 651 3173
-- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
