Hi Phil, Raphael Thanks for your comments.
For the tests I send moses the same very short sentence every time. The output is exactly the same every time. It's as if something somewhere "fills up" (please excuse my jargon). @Phil: you wrote >> Possibly moses' output >> isn't getting flushed after every line. Can I make it do that explicitly? Flushing the stdout doesn't work. More command-line settings for moses? @Raphael: you wrote > I also use a wrapper relying on the fact that moses immediately > produces one line for each line (it's a socket-fifo wrapper, ... I've actually got this running across a socket too (i.e. receiving translation requests and sending translations). The whole thing is working quite well as a lightweight translation server (until you hit the 38th sentence). Do you run moses differently? Do you use something other than subprocess? At the moment I'm thinking either: - kill and restart the moses process every 20 translations; - find a way to flush the moses process' stdout; - see if pexpect can help Thanks and best wishes Ivan On 4 May 2010, at 21:11, Raphael Payen <[email protected]> wrote: > I also use a wrapper relying on the fact that moses immediately > produces one line for each line (it's a socket-fifo wrapper, to avoid > the initialization time and because I didn't compile the server > version). Until now, it has worked well (on Ubuntu). > And anyway, Ivan said that its script works without problem on Unix, > so a flush problem seems unlikely to me. > Maybe a \r\n problem ? > > Ivan, when you say it stops after 4/38 sentences, are these numbers > fixed no matter the input ? Can you try repeating a single simple > sentence 100 times ? > > 2010/5/4 Philip Williams <[email protected]>: >> Hi Ivan, >> >> it looks like the main loop's relying on moses producing exactly >> one line on >> stdout for every line it receives on stdin. If at some point moses >> doesn't >> produce a line of output then the program will deadlock waiting to >> read a >> line that never comes. Could that be happening? Possibly moses' >> output >> isn't getting flushed after every line. I wouldn't've thought so >> but I'd >> have to check the code to be sure... >> >> Phil >> >> >> On Tuesday, May 04, 2010, at 12:08PM, "Ivan Uemlianin" >> <[email protected]> wrote: >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >>> >> >> Dear All >> >> I have this little python script to run quick tests on moses models >> (see >> below). The same script works on unix and windows, given the right >> paths. I'm also using the same basic idea as a basis for the moses >> thread in a translation GUI. >> >> Essentially, the MosesCMD class starts up moses in a subprocess, and >> subsequent calls to translate() send data in on the subprocess' stdin >> and receive translations on its stdout. nb: I'm not using >> communicate >> as that kills the process after reading its output. >> >> I wrote recently (5th April) that on Windows this script freezes >> after >> five sentences, but on unix it's OK. Turns out on unix -- macOSX >> at least >> -- it freezes after 38 sentences. >> >> Automatically killing the subprocess and starting a new one after >> every >> three >> sentences works sometimes, but sometimes (eg when it's powering the >> moses thread in a GUI) actually makes things worse. >> >> The moses decoder I'm using on windows I compiled under cygwin. Both >> decoders are using the same irstlm language models. >> >> Should the moses decoder be able to run indefinitely? Can anyone >> help me >> work out what's going wrong here? >> >> With thanks and best wishes >> >> Ivan >> >> Here's the script: >> >> #! /usr/bin/env python >> >> import subprocess, time >> >> class MosesCMD(object): >> def __init__(self): >> self.cmd_path = '' >> self.model_path = '' >> self.recaser_model_path = '' >> self.proc = None >> >> def start(self): >> cmd = "%s -f %s" % (self.cmd_path, self.model_path) >> if self.recaser_model_path: >> cmd = "%s | %s -v 0 -f %s -dl 1" \ >> % (cmd, self.cmd_path, self.recaser_model_path) >> self.proc = subprocess.Popen(cmd, shell=True, >> stdin=subprocess.PIPE, >> stdout=subprocess.PIPE, >> stderr=subprocess.PIPE) >> >> def translate(self, line): >> line = '%s\n' % line.strip() >> self.proc.stdin.write(line) >> return self.proc.stdout.readline().capitalize() >> >> def close(self): >> self.proc.kill() >> >> def main(): >> mc = MosesCMD() >> mc.cmd_path = "/path/to/mosesdecoder/moses-cmd/src/moses" >> mc.model_path = "/path/to/model/moses.ini" >> mc.recaser_model_path = "/path/to/recase/moses.ini" >> mc.start() >> >> time.sleep(1) >> print 'Ready.' >> >> cy = '*' >> while cy: >> cy = raw_input('cymraeg> ') >> en = mc.translate(cy) >> print 'saesneg> %s\n' % en >> mc.close() >> >> if __name__ == "__main__": >> main() >> >> >> -- >> ******************************** >> Ivan Uemlianin >> >> Canolfan Bedwyr >> Safle'r Normal Site >> Prifysgol Bangor University >> BANGOR >> Gwynedd >> LL57 2PZ >> >> [email protected] >> http://www.bangor.ac.uk/~cbs007/ >> ******************************** >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
