I also use a wrapper relying on the fact that moses immediately produces one line for each line (it's a socket-fifo wrapper, to avoid the initialization time and because I didn't compile the server version). Until now, it has worked well (on Ubuntu). And anyway, Ivan said that its script works without problem on Unix, so a flush problem seems unlikely to me. Maybe a \r\n problem ?
Ivan, when you say it stops after 4/38 sentences, are these numbers fixed no matter the input ? Can you try repeating a single simple sentence 100 times ? 2010/5/4 Philip Williams <[email protected]>: > Hi Ivan, > > it looks like the main loop's relying on moses producing exactly one line on > stdout for every line it receives on stdin. If at some point moses doesn't > produce a line of output then the program will deadlock waiting to read a > line that never comes. Could that be happening? Possibly moses' output > isn't getting flushed after every line. I wouldn't've thought so but I'd > have to check the code to be sure... > > Phil > > > On Tuesday, May 04, 2010, at 12:08PM, "Ivan Uemlianin" > <[email protected]> wrote: >>_______________________________________________ >>Moses-support mailing list >>[email protected] >>http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > > Dear All > > I have this little python script to run quick tests on moses models (see > below). The same script works on unix and windows, given the right > paths. I'm also using the same basic idea as a basis for the moses > thread in a translation GUI. > > Essentially, the MosesCMD class starts up moses in a subprocess, and > subsequent calls to translate() send data in on the subprocess' stdin > and receive translations on its stdout. nb: I'm not using communicate > as that kills the process after reading its output. > > I wrote recently (5th April) that on Windows this script freezes after > five sentences, but on unix it's OK. Turns out on unix -- macOSX at least > -- it freezes after 38 sentences. > > Automatically killing the subprocess and starting a new one after every > three > sentences works sometimes, but sometimes (eg when it's powering the > moses thread in a GUI) actually makes things worse. > > The moses decoder I'm using on windows I compiled under cygwin. Both > decoders are using the same irstlm language models. > > Should the moses decoder be able to run indefinitely? Can anyone help me > work out what's going wrong here? > > With thanks and best wishes > > Ivan > > Here's the script: > > #! /usr/bin/env python > > import subprocess, time > > class MosesCMD(object): > def __init__(self): > self.cmd_path = '' > self.model_path = '' > self.recaser_model_path = '' > self.proc = None > > def start(self): > cmd = "%s -f %s" % (self.cmd_path, self.model_path) > if self.recaser_model_path: > cmd = "%s | %s -v 0 -f %s -dl 1" \ > % (cmd, self.cmd_path, self.recaser_model_path) > self.proc = subprocess.Popen(cmd, shell=True, > stdin=subprocess.PIPE, > stdout=subprocess.PIPE, > stderr=subprocess.PIPE) > > def translate(self, line): > line = '%s\n' % line.strip() > self.proc.stdin.write(line) > return self.proc.stdout.readline().capitalize() > > def close(self): > self.proc.kill() > > def main(): > mc = MosesCMD() > mc.cmd_path = "/path/to/mosesdecoder/moses-cmd/src/moses" > mc.model_path = "/path/to/model/moses.ini" > mc.recaser_model_path = "/path/to/recase/moses.ini" > mc.start() > > time.sleep(1) > print 'Ready.' > > cy = '*' > while cy: > cy = raw_input('cymraeg> ') > en = mc.translate(cy) > print 'saesneg> %s\n' % en > mc.close() > > if __name__ == "__main__": > main() > > > -- > ******************************** > Ivan Uemlianin > > Canolfan Bedwyr > Safle'r Normal Site > Prifysgol Bangor University > BANGOR > Gwynedd > LL57 2PZ > > [email protected] > http://www.bangor.ac.uk/~cbs007/ > ******************************** > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
