Is the code at the bottom of OutputFileStream.cpp correct? The gzip_compressor is not being properly flushed in my build. I tried change it to this->reset(), that didn't help.
On Wed, Jul 4, 2012 at 10:33 AM, Kenneth Heafield <[email protected]> wrote: > Hi, > > It looks like > > set< string > targetLabelCollection, sourceLabelCollection; > map< string, int > targetTopLabelCollection, sourceTopLabelCollection; > > are modified by multiple threads without any form of locking. > > Kenneth > > On 07/04/2012 10:12 AM, Alexander Fraser wrote: >> In the current version built against boost 1 50, with #undef >> WITH_THREADS I am getting invalid gzipped output using this option: >> >> cat source.00000 target.00000 align.00000 >> 1 >> 1 >> 0-0 >> >> /home/fraser/src/master/bin/extract-rules target.00000 source.00000 >> align.00000 extract.00000 --GlueGrammar glue-grammar.1 --MaxSpan 10 >> --GZOutput >> >> zcat extract.00000.gz >> 1 [X] ||| 1 [X] ||| 0-0 ||| 1 ||| >> >> gzip: extract.00000.gz: invalid compressed data--length error >> >> BTW, you are writing the glue-grammar to the same filename >> (model/glue-grammar.1) in all threads, this is likely to cause >> problems. It would be better to specify this option only for the first >> run. >> >> I am using the multithreaded GNU sort, I'll try the options you >> specified, thanks. >> >> Cheers, Alex >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
