I appreciate your interest Mr. Hieu,
The command I wrote in a shell script file like that
“/home/SMT/moses_srilm/scripts/training/train-model.perl -root-dir $1 -corpus
$2 -f src -e trg -alignment grow-diag-final-and -reordering
msd-bidirectional-fe -external-bin-dir=/home/SMT/Bin/ --lm 0:3:$3:0”
Where $1 is the root directory and $2 is the corus path and $3 is the language
model file
Also you will find the log file contain the error in the attacements
From: Hieu Hoang [mailto:[email protected]]
Sent: Sunday, September 28, 2014 8:17 PM
To: [email protected]
Cc: 'moses-support'
Subject: Re: [Moses-support] Training error ("Terminate called after throwing
an instance of 'std::bad_alloc' ")
there are ways of making training run ok even with large amount of data.
However, we need to know exactly where the problem is. What the the EXACT
command you ran, including all arguments? Can you send me the entire command
line output that contains the error, not just the single line with the error
On 28/09/14 15:07, Ihab Ramadan wrote:
I was training the translation model the language model is done successfully,
the command was train-model.perl
It generates the corpus folder and start to make the other files in the
giza.src-trg files using GIZA program, after a while it generates this error
I agree with you that could be a run out of memory exception because the data I
use this time is much bigger than the previous one but what you mean by
“something odd with the data”
From: <mailto:[email protected]> [email protected] [
<mailto:[email protected]> mailto:[email protected]] On Behalf Of Hieu Hoang
Sent: Sunday, September 28, 2014 3:38 PM
To: <mailto:[email protected]> [email protected]
Cc: moses-support
Subject: Re: [Moses-support] Training error ("Terminate called after throwing
an instance of 'std::bad_alloc' ")
what is the exact command you ran? Were you training the translation model or
the language model? and can you provide more details on the output.
it seems to have run out of memory. That shouldn't happen unless there's
something odd with the data
On 28 September 2014 11:02, Ihab Ramadan <[email protected]> wrote:
Dears,
I got this error during the training process of some big parallel files ( 600
MB for each) “Terminate called after throwing an instance of 'std::bad_alloc'”
Note : I ran the clean script before training
Any suggestions?
Best Regards
Ihab Ramadan| Senior Developer| <http://www.saudisoft.com/> Saudisoft - Egypt
| Tel +2 02 330 320 37 Ext- 0 | Mob+201007570826 <tel:%2B201007570826> |
Fax+20233032036 <tel:%2B20233032036> | Follow us on
<http://www.linkedin.com/company/77017?trk=vsrp_companies_res_name&trkInfo=VSRPsearchId%3A1489659901402995947155%2CVSRPtargetId%3A77017%2CVSRPcmpt%3Aprimary>
linked |
<https://www.facebook.com/pages/Saudisoft-Co-Ltd/289968997768973?ref_type=bookmark>
ZA102637861 | <https://twitter.com/Saudisoft> ZA102637858
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
e 686000
line 687000
line 688000
line 689000
line 690000
line 691000
line 692000
line 693000
line 694000
line 695000
line 696000
line 697000
line 698000
line 699000
line 700000
line 701000
line 702000
line 703000
line 704000
line 705000
line 706000
line 707000
line 708000
line 709000
line 710000
line 711000
line 712000
line 713000
line 714000
line 715000
line 716000
line 717000
line 718000
line 719000
line 720000
line 721000
line 722000
line 723000
line 724000
line 725000
line 726000
line 727000
line 728000
line 729000
line 730000
line 731000
line 732000
line 733000
line 734000
line 735000
line 736000
line 737000
line 738000
line 739000
line 740000
line 741000
line 742000
line 743000
line 744000
line 745000
line 746000
line 747000
line 748000
line 749000
line 750000
line 751000
line 752000
line 753000
line 754000
line 755000
line 756000
line 757000
line 758000
line 759000
line 760000
line 761000
line 762000
line 763000
line 764000
line 765000
line 766000
line 767000
line 768000
line 769000
line 770000
line 771000
line 772000
line 773000
line 774000
line 775000
line 776000
line 777000
line 778000
line 779000
line 780000
line 781000
line 782000
line 783000
line 784000
line 785000
line 786000
line 787000
line 788000
line 789000
line 790000
line 791000
line 792000
line 793000
line 794000
line 795000
line 796000
line 797000
line 798000
line 799000
line 800000
line 801000
line 802000
line 803000
line 804000
line 805000
line 806000
line 807000
line 808000
line 809000
line 810000
line 811000
line 812000
line 813000
line 814000
line 815000
line 816000
line 817000
line 818000
line 819000
line 820000
line 821000
END.
(2.1b) running giza trg-src @ Mon Sep 29 15:46:50 EET 2014
/home/SMT/Bin/GIZA++ -CoocurrenceFile
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/giza.trg-src/trg-src.cooc
-c
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/corpus/trg-src-int-train.snt
-m1 5 -m2 0 -m3 3 -m4 3 -model1dumpfrequency 1 -model4smoothfactor 0.4
-nodumps 1 -nsmooth 4 -o
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/giza.trg-src/trg-src
-onlyaldumps 1 -p0 0.999 -s
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/corpus/src.vcb -t
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/corpus/trg.vcb
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/giza.trg-src/trg-src.A3.final.gz
seems finished, reusing.
(3) generate word alignment @ Mon Sep 29 15:46:50 EET 2014
Combining forward and inverted alignment from files:
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/giza.src-trg/src-trg.A3.final.{bz2,gz}
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/giza.trg-src/trg-src.A3.final.{bz2,gz}
Executing: mkdir -p /home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model
Executing: /home/SMT/mosesdecoder-master/scripts/training/giza2bal.pl -d "gzip
-cd
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/giza.trg-src/trg-src.A3.final.gz"
-i "gzip -cd
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/giza.src-trg/src-trg.A3.final.gz"
|/home/SMT/mosesdecoder-master/scripts/../bin/symal -alignment="grow"
-diagonal="yes" -final="yes" -both="yes" >
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/aligned.grow-diag-final-and
symal: computing grow alignment: diagonal (1) final (1)both-uncovered (1)
skip=<0> counts=<821805>
(4) generate lexical translation table 0-0 @ Mon Sep 29 15:47:38 EET 2014
(/home/SMT/domains/IT_English_Arabic/IT_English_Arabic.src,/home/SMT/domains/IT_English_Arabic/IT_English_Arabic.trg,/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex)
reusing: /home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.f2e
and /home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.e2f
(5) extract phrases @ Mon Sep 29 15:47:38 EET 2014
MAX 7 1 0
/home/SMT/mosesdecoder-master/scripts/generic/extract-parallel.perl 1 split
"sort " /home/SMT/mosesdecoder-master/scripts/../bin/extract
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic.trg
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic.src
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/aligned.grow-diag-final-and
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract 7
orientation --model wbe-msd --GZOutput
Executing: /home/SMT/mosesdecoder-master/scripts/generic/extract-parallel.perl
1 split "sort " /home/SMT/mosesdecoder-master/scripts/../bin/extract
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic.trg
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic.src
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/aligned.grow-diag-final-and
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract 7
orientation --model wbe-msd --GZOutput
Started Mon Sep 29 15:47:38 2014
total=821805 line-per-split=821806
Executing: ln -s /home/SMT/domains/IT_English_Arabic/IT_English_Arabic.trg
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/target.00000
Executing: ln -s /home/SMT/domains/IT_English_Arabic/IT_English_Arabic.src
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/source.00000
Executing: ln -s
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/aligned.grow-diag-final-and
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/align.00000
glueArg=
/home/SMT/mosesdecoder-master/scripts/../bin/extract
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/target.00000
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/source.00000
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/align.00000
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/extract.00000
7 orientation --model wbe-msd --GZOutput --SentenceOffset 0 2>> /dev/stderr
PhraseExtract v1.4, written by Philipp Koehn
phrase extraction from an aligned parallel corpus
..................................................................................merging
extract / extract.inv
gunzip -c
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/extract.00000.gz
| LC_ALL=C sort -T
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986 2>>
/dev/stderr | gzip -c >
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.sorted.gz
2>> /dev/stderr
gunzip -c
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/extract.00000.inv.gz
| LC_ALL=C sort -T
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986 2>>
/dev/stderr | gzip -c >
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.inv.sorted.gz
2>> /dev/stderr
gunzip -c
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986/extract.00000.o.gz
| LC_ALL=C sort -T
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986 2>>
/dev/stderr | gzip -c >
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.o.sorted.gz
2>> /dev/stderr
rm -rf /home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.7986
Finished Mon Sep 29 15:53:55 2014
(6) score phrases @ Mon Sep 29 15:53:55 EET 2014
(6.1) creating table half
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.f2e
@ Mon Sep 29 15:53:55 EET 2014
/home/SMT/mosesdecoder-master/scripts/generic/score-parallel.perl 1 "sort "
/home/SMT/mosesdecoder-master/scripts/../bin/score
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.sorted.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.f2e
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.f2e.gz
0
Executing: /home/SMT/mosesdecoder-master/scripts/generic/score-parallel.perl 1
"sort " /home/SMT/mosesdecoder-master/scripts/../bin/score
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.sorted.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.f2e
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.f2e.gz
0
Started Mon Sep 29 15:53:56 2014
ln -s
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.sorted.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8045/extract.0.gz
/home/SMT/mosesdecoder-master/scripts/../bin/score
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8045/extract.0.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.f2e
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8045/phrase-table.half.00000.gz
2>> /dev/stderr
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8045/run.0.shScore
v2.1 -- scoring methods for extracted rules
Loading lexical translation table from
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.f2e.............
ERROR: faulty line 26101: """"""""" ||| بÙاسطة
*** Segmentation fault
Register dump:
RAX: 000000000aa3d0b8 RBX: 0000000000000001 RCX: 00007f69c4ed1620
RDX: 000000000aa3d0b0 RSI: 00007fff09437470 RDI: 000000000aa3d0b0
RBP: 000000000aa3d0b0 R8 : 0000000000000001 R9 : 0000000000000000
R10: 00007fff09437120 R11: 00007f69c56cce40 R12: 000000000aa3d0c8
R13: 00007fff094374a0 R14: 000000000aa3cf20 R15: 0000000000000002
RSP: 00007fff09437300
RIP: 000000000041253d EFLAGS: 00010202
CS: 0033 FS: 0000 GS: 0000
Trap: 0000000e Error: 00000004 OldMask: 00000000 CR2: 00000022
FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000
RIP: 00000000 RDP: 00000000
ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000
ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000
ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000
ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000
mxcsr: 1fa0
XMM0: 0000000000000000000000003f800000 XMM1: 0000000000000000000000003f800000
XMM2: 0000000000000000000000003f800000 XMM3: 0000000000000000000000003f800000
XMM4: 0000000000000000000000003f800000 XMM5: 0000000000000000000000003f800000
XMM6: 0000000000000000000000003f800000 XMM7: 0000000000000000000000003f800000
XMM8: 0000000000000000000000003f800000 XMM9: 0000000000000000000000003f800000
XMM10: 0000000000000000000000003f800000 XMM11: 0000000000000000000000003f800000
XMM12: 0000000000000000000000003f800000 XMM13: 0000000000000000000000003f800000
XMM14: 0000000000000000000000003f800000 XMM15: 0000000000000000000000003f800000
Backtrace:
/home/SMT/mosesdecoder-master/scripts/../bin/score[0x41253d]
/home/SMT/mosesdecoder-master/scripts/../bin/score[0x411547]
/home/SMT/mosesdecoder-master/scripts/../bin/score[0x40ec96]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f69c4b49be5]
/home/SMT/mosesdecoder-master/scripts/../bin/score[0x40fef9]
Memory map:
00400000-00471000 r-xp 00000000 08:05 1051265
/home/SMT/mosesdecoder-master/bin/score
00670000-00671000 r--p 00070000 08:05 1051265
/home/SMT/mosesdecoder-master/bin/score
00671000-00672000 rw-p 00071000 08:05 1051265
/home/SMT/mosesdecoder-master/bin/score
00672000-00673000 rw-p 00000000 00:00 0
01032000-0aa3f000 rw-p 00000000 00:00 0 [heap]
7f69c4116000-7f69c4718000 rw-p 00000000 00:00 0
7f69c4919000-7f69c4927000 r-xp 00000000 08:04 9248
/usr/lib64/libbz2.so.1.0.6
7f69c4927000-7f69c4b26000 ---p 0000e000 08:04 9248
/usr/lib64/libbz2.so.1.0.6
7f69c4b26000-7f69c4b27000 r--p 0000d000 08:04 9248
/usr/lib64/libbz2.so.1.0.6
7f69c4b27000-7f69c4b28000 rw-p 0000e000 08:04 9248
/usr/lib64/libbz2.so.1.0.6
7f69c4b28000-7f69c4ccd000 r-xp 00000000 08:04 652843
/lib64/libc-2.18.so
7f69c4ccd000-7f69c4ecd000 ---p 001a5000 08:04 652843
/lib64/libc-2.18.so
7f69c4ecd000-7f69c4ed1000 r--p 001a5000 08:04 652843
/lib64/libc-2.18.so
7f69c4ed1000-7f69c4ed3000 rw-p 001a9000 08:04 652843
/lib64/libc-2.18.so
7f69c4ed3000-7f69c4ed7000 rw-p 00000000 00:00 0
7f69c4ed7000-7f69c4eef000 r-xp 00000000 08:04 652849
/lib64/libpthread-2.18.so
7f69c4eef000-7f69c50ef000 ---p 00018000 08:04 652849
/lib64/libpthread-2.18.so
7f69c50ef000-7f69c50f0000 r--p 00018000 08:04 652849
/lib64/libpthread-2.18.so
7f69c50f0000-7f69c50f1000 rw-p 00019000 08:04 652849
/lib64/libpthread-2.18.so
7f69c50f1000-7f69c50f5000 rw-p 00000000 00:00 0
7f69c50f5000-7f69c510b000 r-xp 00000000 08:04 652811
/lib64/libgcc_s.so.1
7f69c510b000-7f69c530a000 ---p 00016000 08:04 652811
/lib64/libgcc_s.so.1
7f69c530a000-7f69c530b000 r--p 00015000 08:04 652811
/lib64/libgcc_s.so.1
7f69c530b000-7f69c530c000 rw-p 00016000 08:04 652811
/lib64/libgcc_s.so.1
7f69c530c000-7f69c540e000 r-xp 00000000 08:04 652834
/lib64/libm-2.18.so
7f69c540e000-7f69c560d000 ---p 00102000 08:04 652834
/lib64/libm-2.18.so
7f69c560d000-7f69c560e000 r--p 00101000 08:04 652834
/lib64/libm-2.18.so
7f69c560e000-7f69c560f000 rw-p 00102000 08:04 652834
/lib64/libm-2.18.so
7f69c560f000-7f69c56f9000 r-xp 00000000 08:04 9450
/usr/lib64/libstdc++.so.6.0.18
7f69c56f9000-7f69c58f8000 ---p 000ea000 08:04 9450
/usr/lib64/libstdc++.so.6.0.18
7f69c58f8000-7f69c5900000 r--p 000e9000 08:04 9450
/usr/lib64/libstdc++.so.6.0.18
7f69c5900000-7f69c5902000 rw-p 000f1000 08:04 9450
/usr/lib64/libstdc++.so.6.0.18
7f69c5902000-7f69c5917000 rw-p 00000000 00:00 0
7f69c5917000-7f69c591e000 r-xp 00000000 08:04 652838
/lib64/librt-2.18.so
7f69c591e000-7f69c5b1d000 ---p 00007000 08:04 652838
/lib64/librt-2.18.so
7f69c5b1d000-7f69c5b1e000 r--p 00006000 08:04 652838
/lib64/librt-2.18.so
7f69c5b1e000-7f69c5b1f000 rw-p 00007000 08:04 652838
/lib64/librt-2.18.so
7f69c5b1f000-7f69c5b37000 r-xp 00000000 08:04 33033
/usr/lib64/libboost_iostreams.so.1.53.0
7f69c5b37000-7f69c5d36000 ---p 00018000 08:04 33033
/usr/lib64/libboost_iostreams.so.1.53.0
7f69c5d36000-7f69c5d38000 r--p 00017000 08:04 33033
/usr/lib64/libboost_iostreams.so.1.53.0
7f69c5d38000-7f69c5d39000 rw-p 00019000 08:04 33033
/usr/lib64/libboost_iostreams.so.1.53.0
7f69c5d39000-7f69c5d4e000 r-xp 00000000 08:04 652916
/lib64/libz.so.1.2.8
7f69c5d4e000-7f69c5f4d000 ---p 00015000 08:04 652916
/lib64/libz.so.1.2.8
7f69c5f4d000-7f69c5f4e000 r--p 00014000 08:04 652916
/lib64/libz.so.1.2.8
7f69c5f4e000-7f69c5f4f000 rw-p 00015000 08:04 652916
/lib64/libz.so.1.2.8
7f69c5f4f000-7f69c5f53000 r-xp 00000000 08:04 652822
/lib64/libSegFault.so
7f69c5f53000-7f69c6152000 ---p 00004000 08:04 652822
/lib64/libSegFault.so
7f69c6152000-7f69c6153000 r--p 00003000 08:04 652822
/lib64/libSegFault.so
7f69c6153000-7f69c6154000 rw-p 00004000 08:04 652822
/lib64/libSegFault.so
7f69c6154000-7f69c6157000 r-xp 00000000 08:04 14873
/usr/lib64/libboost_system.so.1.53.0
7f69c6157000-7f69c6356000 ---p 00003000 08:04 14873
/usr/lib64/libboost_system.so.1.53.0
7f69c6356000-7f69c6357000 r--p 00002000 08:04 14873
/usr/lib64/libboost_system.so.1.53.0
7f69c6357000-7f69c6358000 rw-p 00003000 08:04 14873
/usr/lib64/libboost_system.so.1.53.0
7f69c6358000-7f69c636d000 r-xp 00000000 08:04 15378
/usr/lib64/libboost_thread.so.1.53.0
7f69c636d000-7f69c656d000 ---p 00015000 08:04 15378
/usr/lib64/libboost_thread.so.1.53.0
7f69c656d000-7f69c656f000 r--p 00015000 08:04 15378
/usr/lib64/libboost_thread.so.1.53.0
7f69c656f000-7f69c6570000 rw-p 00017000 08:04 15378
/usr/lib64/libboost_thread.so.1.53.0
7f69c6570000-7f69c6573000 r-xp 00000000 08:04 652848
/lib64/libdl-2.18.so
7f69c6573000-7f69c6772000 ---p 00003000 08:04 652848
/lib64/libdl-2.18.so
7f69c6772000-7f69c6773000 r--p 00002000 08:04 652848
/lib64/libdl-2.18.so
7f69c6773000-7f69c6774000 rw-p 00003000 08:04 652848
/lib64/libdl-2.18.so
7f69c6774000-7f69c6794000 r-xp 00000000 08:04 652832
/lib64/ld-2.18.so
7f69c6967000-7f69c6970000 rw-p 00000000 00:00 0
7f69c6992000-7f69c6993000 rw-p 00000000 00:00 0
7f69c6993000-7f69c6994000 r--p 0001f000 08:04 652832
/lib64/ld-2.18.so
7f69c6994000-7f69c6995000 rw-p 00020000 08:04 652832
/lib64/ld-2.18.so
7f69c6995000-7f69c6996000 rw-p 00000000 00:00 0
7fff0941e000-7fff0946a000 rw-p 00000000 00:00 0 [stack]
7fff095fe000-7fff09600000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
[vsyscall]
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8045/run.0.sh:
line 1: 8050 Segmentation fault
/home/SMT/mosesdecoder-master/scripts/../bin/score
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8045/extract.0.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.f2e
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8045/phrase-table.half.00000.gz
2>> /dev/stderr
mv
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8045/phrase-table.half.00000.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.f2e.gzrm
-rf /home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8045
Finished Mon Sep 29 15:53:59 2014
(6.3) creating table half
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.e2f
@ Mon Sep 29 15:53:59 EET 2014
/home/SMT/mosesdecoder-master/scripts/generic/score-parallel.perl 1 "sort "
/home/SMT/mosesdecoder-master/scripts/../bin/score
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.inv.sorted.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.e2f
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.e2f.gz
--Inverse 1
Executing: /home/SMT/mosesdecoder-master/scripts/generic/score-parallel.perl 1
"sort " /home/SMT/mosesdecoder-master/scripts/../bin/score
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.inv.sorted.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.e2f
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.e2f.gz
--Inverse 1
Started Mon Sep 29 15:53:59 2014
ln -s
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.inv.sorted.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056/extract.0.gz
/home/SMT/mosesdecoder-master/scripts/../bin/score
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056/extract.0.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.e2f
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056/phrase-table.half.00000.gz
--Inverse 2>> /dev/stderr
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056/run.0.shScore
v2.1 -- scoring methods for extracted rules
using inverse mode
Loading lexical translation table from
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.e2f.............
ERROR: faulty line 41565: """"
*** Segmentation fault
Register dump:
RAX: 000000000c5edfd8 RBX: 0000000000000001 RCX: 00007fbb130ea620
RDX: 000000000c5edfd0 RSI: 00007fff3b426f20 RDI: 000000000c5edfd0
RBP: 000000000c5edfd0 R8 : 0000000000000001 R9 : 0000000000000000
R10: 00007fff3b426bd0 R11: 00007fbb138e5e40 R12: 000000000c5edfe8
R13: 00007fff3b426f50 R14: 000000000c5edda0 R15: 0000000000000002
RSP: 00007fff3b426db0
RIP: 000000000041253d EFLAGS: 00010202
CS: 0033 FS: 0000 GS: 0000
Trap: 0000000e Error: 00000004 OldMask: 00000000 CR2: 00000022
FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000
RIP: 00000000 RDP: 00000000
ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000
ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000
ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000
ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000
mxcsr: 1fa0
XMM0: 0000000000000000000000003f800000 XMM1: 0000000000000000000000003f800000
XMM2: 0000000000000000000000003f800000 XMM3: 0000000000000000000000003f800000
XMM4: 0000000000000000000000003f800000 XMM5: 0000000000000000000000003f800000
XMM6: 0000000000000000000000003f800000 XMM7: 0000000000000000000000003f800000
XMM8: 0000000000000000000000003f800000 XMM9: 0000000000000000000000003f800000
XMM10: 0000000000000000000000003f800000 XMM11: 0000000000000000000000003f800000
XMM12: 0000000000000000000000003f800000 XMM13: 0000000000000000000000003f800000
XMM14: 0000000000000000000000003f800000 XMM15: 0000000000000000000000003f800000
Backtrace:
/home/SMT/mosesdecoder-master/scripts/../bin/score[0x41253d]
/home/SMT/mosesdecoder-master/scripts/../bin/score[0x411547]
/home/SMT/mosesdecoder-master/scripts/../bin/score[0x40ec96]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fbb12d62be5]
/home/SMT/mosesdecoder-master/scripts/../bin/score[0x40fef9]
Memory map:
00400000-00471000 r-xp 00000000 08:05 1051265
/home/SMT/mosesdecoder-master/bin/score
00670000-00671000 r--p 00070000 08:05 1051265
/home/SMT/mosesdecoder-master/bin/score
00671000-00672000 rw-p 00071000 08:05 1051265
/home/SMT/mosesdecoder-master/bin/score
00672000-00673000 rw-p 00000000 00:00 0
023e3000-0c60d000 rw-p 00000000 00:00 0 [heap]
7fbb1232f000-7fbb12931000 rw-p 00000000 00:00 0
7fbb12b32000-7fbb12b40000 r-xp 00000000 08:04 9248
/usr/lib64/libbz2.so.1.0.6
7fbb12b40000-7fbb12d3f000 ---p 0000e000 08:04 9248
/usr/lib64/libbz2.so.1.0.6
7fbb12d3f000-7fbb12d40000 r--p 0000d000 08:04 9248
/usr/lib64/libbz2.so.1.0.6
7fbb12d40000-7fbb12d41000 rw-p 0000e000 08:04 9248
/usr/lib64/libbz2.so.1.0.6
7fbb12d41000-7fbb12ee6000 r-xp 00000000 08:04 652843
/lib64/libc-2.18.so
7fbb12ee6000-7fbb130e6000 ---p 001a5000 08:04 652843
/lib64/libc-2.18.so
7fbb130e6000-7fbb130ea000 r--p 001a5000 08:04 652843
/lib64/libc-2.18.so
7fbb130ea000-7fbb130ec000 rw-p 001a9000 08:04 652843
/lib64/libc-2.18.so
7fbb130ec000-7fbb130f0000 rw-p 00000000 00:00 0
7fbb130f0000-7fbb13108000 r-xp 00000000 08:04 652849
/lib64/libpthread-2.18.so
7fbb13108000-7fbb13308000 ---p 00018000 08:04 652849
/lib64/libpthread-2.18.so
7fbb13308000-7fbb13309000 r--p 00018000 08:04 652849
/lib64/libpthread-2.18.so
7fbb13309000-7fbb1330a000 rw-p 00019000 08:04 652849
/lib64/libpthread-2.18.so
7fbb1330a000-7fbb1330e000 rw-p 00000000 00:00 0
7fbb1330e000-7fbb13324000 r-xp 00000000 08:04 652811
/lib64/libgcc_s.so.1
7fbb13324000-7fbb13523000 ---p 00016000 08:04 652811
/lib64/libgcc_s.so.1
7fbb13523000-7fbb13524000 r--p 00015000 08:04 652811
/lib64/libgcc_s.so.1
7fbb13524000-7fbb13525000 rw-p 00016000 08:04 652811
/lib64/libgcc_s.so.1
7fbb13525000-7fbb13627000 r-xp 00000000 08:04 652834
/lib64/libm-2.18.so
7fbb13627000-7fbb13826000 ---p 00102000 08:04 652834
/lib64/libm-2.18.so
7fbb13826000-7fbb13827000 r--p 00101000 08:04 652834
/lib64/libm-2.18.so
7fbb13827000-7fbb13828000 rw-p 00102000 08:04 652834
/lib64/libm-2.18.so
7fbb13828000-7fbb13912000 r-xp 00000000 08:04 9450
/usr/lib64/libstdc++.so.6.0.18
7fbb13912000-7fbb13b11000 ---p 000ea000 08:04 9450
/usr/lib64/libstdc++.so.6.0.18
7fbb13b11000-7fbb13b19000 r--p 000e9000 08:04 9450
/usr/lib64/libstdc++.so.6.0.18
7fbb13b19000-7fbb13b1b000 rw-p 000f1000 08:04 9450
/usr/lib64/libstdc++.so.6.0.18
7fbb13b1b000-7fbb13b30000 rw-p 00000000 00:00 0
7fbb13b30000-7fbb13b37000 r-xp 00000000 08:04 652838
/lib64/librt-2.18.so
7fbb13b37000-7fbb13d36000 ---p 00007000 08:04 652838
/lib64/librt-2.18.so
7fbb13d36000-7fbb13d37000 r--p 00006000 08:04 652838
/lib64/librt-2.18.so
7fbb13d37000-7fbb13d38000 rw-p 00007000 08:04 652838
/lib64/librt-2.18.so
7fbb13d38000-7fbb13d50000 r-xp 00000000 08:04 33033
/usr/lib64/libboost_iostreams.so.1.53.0
7fbb13d50000-7fbb13f4f000 ---p 00018000 08:04 33033
/usr/lib64/libboost_iostreams.so.1.53.0
7fbb13f4f000-7fbb13f51000 r--p 00017000 08:04 33033
/usr/lib64/libboost_iostreams.so.1.53.0
7fbb13f51000-7fbb13f52000 rw-p 00019000 08:04 33033
/usr/lib64/libboost_iostreams.so.1.53.0
7fbb13f52000-7fbb13f67000 r-xp 00000000 08:04 652916
/lib64/libz.so.1.2.8
7fbb13f67000-7fbb14166000 ---p 00015000 08:04 652916
/lib64/libz.so.1.2.8
7fbb14166000-7fbb14167000 r--p 00014000 08:04 652916
/lib64/libz.so.1.2.8
7fbb14167000-7fbb14168000 rw-p 00015000 08:04 652916
/lib64/libz.so.1.2.8
7fbb14168000-7fbb1416c000 r-xp 00000000 08:04 652822
/lib64/libSegFault.so
7fbb1416c000-7fbb1436b000 ---p 00004000 08:04 652822
/lib64/libSegFault.so
7fbb1436b000-7fbb1436c000 r--p 00003000 08:04 652822
/lib64/libSegFault.so
7fbb1436c000-7fbb1436d000 rw-p 00004000 08:04 652822
/lib64/libSegFault.so
7fbb1436d000-7fbb14370000 r-xp 00000000 08:04 14873
/usr/lib64/libboost_system.so.1.53.0
7fbb14370000-7fbb1456f000 ---p 00003000 08:04 14873
/usr/lib64/libboost_system.so.1.53.0
7fbb1456f000-7fbb14570000 r--p 00002000 08:04 14873
/usr/lib64/libboost_system.so.1.53.0
7fbb14570000-7fbb14571000 rw-p 00003000 08:04 14873
/usr/lib64/libboost_system.so.1.53.0
7fbb14571000-7fbb14586000 r-xp 00000000 08:04 15378
/usr/lib64/libboost_thread.so.1.53.0
7fbb14586000-7fbb14786000 ---p 00015000 08:04 15378
/usr/lib64/libboost_thread.so.1.53.0
7fbb14786000-7fbb14788000 r--p 00015000 08:04 15378
/usr/lib64/libboost_thread.so.1.53.0
7fbb14788000-7fbb14789000 rw-p 00017000 08:04 15378
/usr/lib64/libboost_thread.so.1.53.0
7fbb14789000-7fbb1478c000 r-xp 00000000 08:04 652848
/lib64/libdl-2.18.so
7fbb1478c000-7fbb1498b000 ---p 00003000 08:04 652848
/lib64/libdl-2.18.so
7fbb1498b000-7fbb1498c000 r--p 00002000 08:04 652848
/lib64/libdl-2.18.so
7fbb1498c000-7fbb1498d000 rw-p 00003000 08:04 652848
/lib64/libdl-2.18.so
7fbb1498d000-7fbb149ad000 r-xp 00000000 08:04 652832
/lib64/ld-2.18.so
7fbb14b80000-7fbb14b89000 rw-p 00000000 00:00 0
7fbb14bab000-7fbb14bac000 rw-p 00000000 00:00 0
7fbb14bac000-7fbb14bad000 r--p 0001f000 08:04 652832
/lib64/ld-2.18.so
7fbb14bad000-7fbb14bae000 rw-p 00020000 08:04 652832
/lib64/ld-2.18.so
7fbb14bae000-7fbb14baf000 rw-p 00000000 00:00 0
7fff3b40e000-7fff3b45a000 rw-p 00000000 00:00 0 [stack]
7fff3b5fe000-7fff3b600000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
[vsyscall]
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056/run.0.sh:
line 1: 8061 Segmentation fault
/home/SMT/mosesdecoder-master/scripts/../bin/score
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056/extract.0.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/lex.e2f
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056/phrase-table.half.00000.gz
--Inverse 2>> /dev/stderr
gunzip -c
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056/phrase-table.half.*.gz
2>> /dev/stderr| LC_ALL=C sort -T
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056 | gzip -c
>
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.e2f.gz
2>> /dev/stderr
gzip:
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056/phrase-table.half.00000.gz:
unexpected end of file
rm -rf /home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/tmp.8056
Finished Mon Sep 29 15:54:04 2014
(6.6) consolidating the two halves @ Mon Sep 29 15:54:04 EET 2014
Executing: /home/SMT/mosesdecoder-master/scripts/../bin/consolidate
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.f2e.gz
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.e2f.gz
/dev/stdout | gzip -c >
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.gz
Consolidate v2.0 written by Philipp Koehn
consolidating direct and indirect rule tables
ERROR: target phrase does not match in line 12: 'â\""SQL Server and
Windows\"" authentication."' != '! "" # $ ` ( )'
Executing: rm -f
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/phrase-table.half.*
(7) learn reordering model @ Mon Sep 29 15:54:04 EET 2014
(7.1) [no factors] learn reordering model @ Mon Sep 29 15:54:04 EET 2014
(7.2) building tables @ Mon Sep 29 15:54:04 EET 2014
Executing:
/home/SMT/mosesdecoder-master/scripts/../bin/lexical-reordering-score
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/extract.o.sorted.gz
0.5
/home/SMT/domains/IT_English_Arabic/IT_English_Arabic/model/reordering-table.
--model "wbe msd wbe-msd-bidirectional-fe"
Lexical Reordering Scorer
scores lexical reordering models of several types (hierarchical, phrase-based
and word-based-extraction
(8) learn generation model @ Mon Sep 29 15:55:01 EET 2014
no generation model requested, skipping step
(9) create moses.ini @ Mon Sep 29 15:55:01 EET 2014_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
