Hi Marcin Thanks for the fast fix. I'm running it now with "-T $PWD". The only odd thing is that when I run "ls -l /proc/<PID>/fd", I don't see any temporary files.
I'll let you know how things go, cheers - Barry On 22/01/13 20:20, Marcin Junczys-Dowmunt wrote: > Hi again, > I just pushed a fix for that. You can now specify for instance > > processPhraseTableMin ... -T $PWD/tmp > > and it should create all temporary files in the current folder. This > uses Kenneth's FMakeTemp. BTW: it might be useful if that function could > check whether the prefix provided is an existing directory and if yes > add "/tmp" automatically. > > This also works for processLexicalReorderingTableMin. > > Let me know, if you can use it now. > Best, > Marcin > > W dniu 22.01.2013 18:08, Barry Haddow pisze: >> Hi Marcin >> >> I'm also having a similar problem. I'm getting a a BUS error when >> trying to compact a fairly large (20G, gzipped) phrase table. My /tmp >> is on root, which has about 27G of free space. >> >> I've tried setting the TMPDIR, but processPhraseTableMon still writes >> to /tmp (see below). >> >> Any suggestions? >> >> cheers - Barry >> >> [bhaddow@skaol compact]$ export TMPDIR=$PWD; nohup nice >> /home/bhaddow/moses/dist/124c36a/bin/processPhraseTableMin -in >> 1009-13/phrase-table.sorted.gz -out 1009-13/phrase-table -nscores 5 >> -threads 4 &> 1009-13.log.1 & >> [1] 1479 >> [bhaddow@skaol compact]$ >> [bhaddow@skaol compact]$ ls -l /proc/1479/fd >> total 0 >> l-wx------ 1 bhaddow users 64 2013-01-22 17:05 0 -> /dev/null >> l-wx------ 1 bhaddow users 64 2013-01-22 17:05 1 -> >> /mnt/skaol3/bhaddow/experiments/accept/instance-selection/compact/1009-13.log.1 >> lr-x------ 1 bhaddow users 64 2013-01-22 17:05 10 -> >> /mnt/skaol3/bhaddow/experiments/accept/instance-selection/compact/1009-13/phrase-table.sorted.gz >> l-wx------ 1 bhaddow users 64 2013-01-22 17:05 2 -> >> /mnt/skaol3/bhaddow/experiments/accept/instance-selection/compact/1009-13.log.1 >> l-wx------ 1 bhaddow users 64 2013-01-22 17:05 3 -> >> /mnt/skaol3/bhaddow/experiments/accept/instance-selection/compact/1009-13/phrase-table.minphr >> lrwx------ 1 bhaddow users 64 2013-01-22 17:05 4 -> /tmp/tmpfr3cMiG >> (deleted) >> lrwx------ 1 bhaddow users 64 2013-01-22 17:05 5 -> /tmp/tmpfzCTl3q >> (deleted) >> lrwx------ 1 bhaddow users 64 2013-01-22 17:05 6 -> /tmp/tmpfz7WVNb >> (deleted) >> lrwx------ 1 bhaddow users 64 2013-01-22 17:05 7 -> /tmp/tmpfv2pwyW >> (deleted) >> lrwx------ 1 bhaddow users 64 2013-01-22 17:05 8 -> /tmp/tmpfhag7iH >> (deleted) >> lrwx------ 1 bhaddow users 64 2013-01-22 17:05 9 -> /tmp/tmpfhRpI3r >> (deleted) >> >> >> >> >> >> On 17/01/13 11:12, Marcin Junczys-Dowmunt wrote: >>> That might actually be it, although I would expect a segfault or a "bad >>> alloc" in that case. I do not have the equivalent of a -T option yet and >>> the intermediate representations of the phrase table before compression >>> are by default built in the system's temporary folder (although >>> invisible). So if your temporary folder is too small it will not work. A >>> quick work-around would be to symlink your /tmp folder to a partition >>> where you have more space. I'd say you need at least as much space in >>> your temporary folder as the size of the uncompressed text version of >>> the phrase table or reordering table. >>> >>> W dniu 17.01.2013 11:30, Jacob Dlougach pisze: >>>> Marcin Junczys-Dowmunt<junczys@...> writes: >>>>> And another question, do you have by any chance a non-standard >>>>> setup for >>>>> your temporary folder? Particularly small, located in memory etc.? >>>>> >>>> If by temporary directory you mean /tmp, then it is in fact rather >>>> small, and I >>>> had to specify -T parameter for "sort" to make it work. Talking of >>>> truncating >>>> phrase table, I'll do that as soon as I can. >>>> >>>> _______________________________________________ >>>> Moses-support mailing list >>>> [email protected] >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
