Seems th giza crashes when or after loading the corpus, you should check the corpus, whether there are empty lines, that is a common cause of the problem. --Q
On Tue, Dec 28, 2010 at 12:31 PM, amin farajian <[email protected]>wrote: > snt2cooc.out is installed, and it can be seen by moses. I reinstalled > giza++ with and without the flag and I tested moses in both situations, but > nothing changed. > Here are the messages that appeared on the screen: > > executing > /home/amin/mt/work/baseline/phraseBased/3lm/steps/1/TRAINING_run-giza-inverse.1 > via sh (1) > executing > /home/amin/mt/work/baseline/phraseBased/3lm/steps/1/TRAINING_run-giza.1 via > sh (2) > *** buffer overflow detected ***: /home/amin/mt/bin/GIZA++ terminated > ======= Backtrace: ========= > [0x817af29] > [0x817aedd] > [0x817acf8] > [0x815cd3e] > [0x8187eb5] > [0x817ad94] > [0x817aced] > [0x806ce9d] > [0x806d903] > [0x8073b70] > [0x8147b3f] > [0x8048201] > ======= Memory map: ======== > 00fa7000-00fa8000 r-xp 00000000 00:00 0 [vdso] > 08048000-081fb000 r-xp 00000000 08:04 804586 /home/amin/mt/bin/GIZA++ > 081fc000-081fe000 rw-p 001b3000 08:04 804586 /home/amin/mt/bin/GIZA++ > 081fe000-08206000 rw-p 00000000 00:00 0 > 08d39000-08dad000 rw-p 00000000 00:00 0 [heap] > bfd80000-bfda1000 rw-p 00000000 00:00 0 [stack] > *** buffer overflow detected ***: /home/amin/mt/bin/GIZA++ terminated > ======= Backtrace: ========= > [0x817af29] > [0x817aedd] > [0x817acf8] > [0x815cd3e] > [0x8187eb5] > [0x817ad94] > [0x817aced] > [0x806ce9d] > [0x806d903] > [0x8073b70] > [0x8147b3f] > [0x8048201] > ======= Memory map: ======== > 00cc2000-00cc3000 r-xp 00000000 00:00 0 [vdso] > 08048000-081fb000 r-xp 00000000 08:04 804586 /home/amin/mt/bin/GIZA++ > 081fc000-081fe000 rw-p 001b3000 08:04 804586 /home/amin/mt/bin/GIZA++ > 081fe000-08206000 rw-p 00000000 00:00 0 > 09df6000-09e6a000 rw-p 00000000 00:00 0 [heap] > bfaf0000-bfb11000 rw-p 00000000 00:00 0 [stack] > step TRAINING:run-giza-inverse crashed > step TRAINING:run-giza crashed > number of steps doable or running: 0 > > and here is the content of "TRAINING_run-giza.1.STDERR": > > Using SCRIPTS_ROOTDIR: /home/amin/mt/bin/scripts > Using single-thread GIZA > (2) running giza on 5 cooc parts @ Tue Dec 28 20:35:28 IRST 2010 > Executing: mkdir -p /home/amin/mt/work/baseline/ > phraseBased/3lm/training/prepared.1/part1 > Executing: mkdir -p > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/part2 > Executing: mkdir -p > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/part3 > Executing: mkdir -p > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/part4 > Executing: mkdir -p > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/part5 > (2.1a) running snt2cooc fr-en @ Tue Dec 28 20:35:28 IRST 2010 > > Executing: mkdir -p > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part1 > Executing: /home/amin/mt/bin/snt2cooc.out > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/en.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/part1/fr-en-int-train.snt > > > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part1/fr-en.cooc > END. > (2.1a) running snt2cooc fr-en @ Tue Dec 28 20:35:28 IRST 2010 > > Executing: mkdir -p > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part2 > Executing: /home/amin/mt/bin/snt2cooc.out > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/en.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/part2/fr-en-int-train.snt > > > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part2/fr-en.cooc > END. > (2.1a) running snt2cooc fr-en @ Tue Dec 28 20:35:28 IRST 2010 > > Executing: mkdir -p > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part3 > Executing: /home/amin/mt/bin/snt2cooc.out > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/en.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/part3/fr-en-int-train.snt > > > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part3/fr-en.cooc > END. > (2.1a) running snt2cooc fr-en @ Tue Dec 28 20:35:28 IRST 2010 > > Executing: mkdir -p > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part4 > Executing: /home/amin/mt/bin/snt2cooc.out > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/en.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/part4/fr-en-int-train.snt > > > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part4/fr-en.cooc > END. > (2.1a) running snt2cooc fr-en @ Tue Dec 28 20:35:28 IRST 2010 > > Executing: mkdir -p > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part5 > Executing: /home/amin/mt/bin/snt2cooc.out > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/en.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr.vcb > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/part5/fr-en-int-train.snt > > > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/part5/fr-en.cooc > END. > (2.1b) running giza fr-en @ Tue Dec 28 20:35:29 IRST 2010 > /home/amin/mt/bin/GIZA++ -CoocurrenceFile > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/fr-en.cooc -c > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr-en-int-train.snt > -m1 5 -m2 0 -m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 > -nodumps 1 -nsmooth 4 -o > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/fr-en > -onlyaldumps 1 -p0 0.999 -s > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/en.vcb -t > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr.vcb > Executing: /home/amin/mt/bin/GIZA++ -CoocurrenceFile > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/fr-en.cooc -c > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr-en-int-train.snt > -m1 5 -m2 0 -m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 > -nodumps 1 -nsmooth 4 -o > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/fr-en > -onlyaldumps 1 -p0 0.999 -s > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/en.vcb -t > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr.vcb > ERROR: Execution of: /home/amin/mt/bin/GIZA++ -CoocurrenceFile > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/fr-en.cooc -c > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr-en-int-train.snt > -m1 5 -m2 0 -m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4 > -nodumps 1 -nsmooth 4 -o > /home/amin/mt/work/baseline/phraseBased/3lm/training/giza.1/fr-en > -onlyaldumps 1 -p0 0.999 -s > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/en.vcb -t > /home/amin/mt/work/baseline/phraseBased/3lm/training/prepared.1/fr.vcb > died with signal 6, without coredump > > > On Tue, Dec 28, 2010 at 8:25 PM, Qin Gao <[email protected]> wrote: > >> If you comment out -DBINARY_SEARCH_FOR_TTABLE then giza will use hash >> table instead of binary search for lexicon translation model, that will use >> more memory. The problem seems to be snt2cooc.out is not installed. My >> suggestion is to keep the compiler flag and install snt2cooc.out, make sure >> that can be seen by moses. It is included in both giza++ and mgiza package. >> >> --Q >> >> >> On Tue, Dec 28, 2010 at 11:45 AM, amin farajian <[email protected]>wrote: >> >>> Hi All, >>> One month ago, when I wanted to use moses to build a baseline SMT, I >>> faced a problem during training the models. moses crashed on "run-giza" and >>> "run-giza inverse". After some searches I found that I have to uncomment a >>> line in giza++ which contains "-DBINARY_SEARCH_FOR_TTABLE", and make it >>> again. that time it worked. but, 2 days ago my linux crashed and it made me >>> to reinstall it. after installing the newest version of kubuntu, I installed >>> giza++, mkcls, srilm and moses. when I wanted to build the base-line MT, >>> moses crashed in "run-giza" and "run-giza inverse",again. I uncommented that >>> line in the make file of giza++ but nothing changed in running moses. What >>> can be the cause of this problem? >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >>> >> >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
