Hi again, I just pushed a fix for that. You can now specify for instance processPhraseTableMin ... -T $PWD/tmp
and it should create all temporary files in the current folder. This uses Kenneth's FMakeTemp. BTW: it might be useful if that function could check whether the prefix provided is an existing directory and if yes add "/tmp" automatically. This also works for processLexicalReorderingTableMin. Let me know, if you can use it now. Best, Marcin W dniu 22.01.2013 18:08, Barry Haddow pisze: > Hi Marcin > > I'm also having a similar problem. I'm getting a a BUS error when > trying to compact a fairly large (20G, gzipped) phrase table. My /tmp > is on root, which has about 27G of free space. > > I've tried setting the TMPDIR, but processPhraseTableMon still writes > to /tmp (see below). > > Any suggestions? > > cheers - Barry > > [bhaddow@skaol compact]$ export TMPDIR=$PWD; nohup nice > /home/bhaddow/moses/dist/124c36a/bin/processPhraseTableMin -in > 1009-13/phrase-table.sorted.gz -out 1009-13/phrase-table -nscores 5 > -threads 4 &> 1009-13.log.1 & > [1] 1479 > [bhaddow@skaol compact]$ > [bhaddow@skaol compact]$ ls -l /proc/1479/fd > total 0 > l-wx------ 1 bhaddow users 64 2013-01-22 17:05 0 -> /dev/null > l-wx------ 1 bhaddow users 64 2013-01-22 17:05 1 -> > /mnt/skaol3/bhaddow/experiments/accept/instance-selection/compact/1009-13.log.1 > lr-x------ 1 bhaddow users 64 2013-01-22 17:05 10 -> > /mnt/skaol3/bhaddow/experiments/accept/instance-selection/compact/1009-13/phrase-table.sorted.gz > l-wx------ 1 bhaddow users 64 2013-01-22 17:05 2 -> > /mnt/skaol3/bhaddow/experiments/accept/instance-selection/compact/1009-13.log.1 > l-wx------ 1 bhaddow users 64 2013-01-22 17:05 3 -> > /mnt/skaol3/bhaddow/experiments/accept/instance-selection/compact/1009-13/phrase-table.minphr > lrwx------ 1 bhaddow users 64 2013-01-22 17:05 4 -> /tmp/tmpfr3cMiG > (deleted) > lrwx------ 1 bhaddow users 64 2013-01-22 17:05 5 -> /tmp/tmpfzCTl3q > (deleted) > lrwx------ 1 bhaddow users 64 2013-01-22 17:05 6 -> /tmp/tmpfz7WVNb > (deleted) > lrwx------ 1 bhaddow users 64 2013-01-22 17:05 7 -> /tmp/tmpfv2pwyW > (deleted) > lrwx------ 1 bhaddow users 64 2013-01-22 17:05 8 -> /tmp/tmpfhag7iH > (deleted) > lrwx------ 1 bhaddow users 64 2013-01-22 17:05 9 -> /tmp/tmpfhRpI3r > (deleted) > > > > > > On 17/01/13 11:12, Marcin Junczys-Dowmunt wrote: >> That might actually be it, although I would expect a segfault or a "bad >> alloc" in that case. I do not have the equivalent of a -T option yet and >> the intermediate representations of the phrase table before compression >> are by default built in the system's temporary folder (although >> invisible). So if your temporary folder is too small it will not work. A >> quick work-around would be to symlink your /tmp folder to a partition >> where you have more space. I'd say you need at least as much space in >> your temporary folder as the size of the uncompressed text version of >> the phrase table or reordering table. >> >> W dniu 17.01.2013 11:30, Jacob Dlougach pisze: >>> Marcin Junczys-Dowmunt<junczys@...> writes: >>>> And another question, do you have by any chance a non-standard >>>> setup for >>>> your temporary folder? Particularly small, located in memory etc.? >>>> >>> If by temporary directory you mean /tmp, then it is in fact rather >>> small, and I >>> had to specify -T parameter for "sort" to make it work. Talking of >>> truncating >>> phrase table, I'll do that as soon as I can. >>> >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
