In addition to a variety of perl and python, I believe the scripts tree contains some C++ code that needs compiling. There are also dependencies that are managed through the Makefile.
On Fri, 23 Sep 2011 12:02:12 +0100, Kenneth Heafield <[email protected]> wrote: > Stupid question: why do they need to be compiled at all? They're > scripts. > > I think "compiling" might mean hard-coding paths of dependencies into > the source. The correct way to do this is use e.g. bash's $0 to find > where the executable is then use relative paths and readlink -f to > fix > the top-level source directory. > > Kenneth > > On 09/23/11 11:44, Tom Hoar wrote: >> The date-time stamp also helps prevent accidental deletion of user >> changes when recompiling the scripts. >> >> Also, when DoMY installs Moses and compiles the scripts, we >> abstract >> the date-time folder with a symbolic link "scripts", then set the >> environment variable to the symbolic link. >> >> Our DoMY CE installation takes a different approach that might work >> here. Our user's scripts are in folders without date-time stamps. >> When >> the re-install or upgrade, we rename any existing folders with a >> date-time stamp and the users must migrate their changes. I think >> this >> approach with the moses scripts could work. >> >> Tom >> >> >> On Fri, 23 Sep 2011 09:46:06 +0200, Ondrej Bojar >> <[email protected]> wrote: >>> Hi, >>> >>> mert-moses-new.pl is outdated, isn't it? There should be only >>> mert-moses.pl these days. >>> >>> The 'scripts releasing' is of mine and dates back to JHU workshop >>> in >>> 2006. Back then, we were hacking the scripts and running >>> experiments >>> at >>> the same time, so we needed some track of what version of scripts >>> was >>> used for that particular failing experiment. >>> >>> As soon as anyone has some spare time I'd suggest: >>> >>> - delete the 'releasing' code in the Makefile >>> - make sure the main make compiles everything: all moseses, all >>> auxiliary binaries in scripts etc. >>> - I'd avoid implementing 'make install', because all the tools in >>> scripts know where their frieds are sitting in terms of relative >>> paths. >>> The install would need to preserve the complicated structure anyway >>> so >>> there's not much point in having the install as 'cp -r' does the >>> same >>> job. >>> >>> Cheers, O. >>> >>> On 09/23/2011 09:20 AM, Hieu Hoang wrote: >>>> agreed, we were having an offlist moan about the same thing. The >>>> separation >>>> between decoding& training, and release date-stamp thingy is >>>> historical and >>>> quite silly. >>>> >>>> The directories& release procedure has to be rationalised. >>>> >>>> Someone will do it eventually... >>>> >>>> On 23 September 2011 14:04, Joerg >>>> Tiedemann<[email protected]>wrote: >>>> >>>>> By the way, what is the use of a date-stamped directory anyway? >>>>> I find if rather disturbing that all the binaries and scripts are >>>>> distributed all over the place. >>>>> moses-cmd/src >>>>> moses-chart-cmd/src >>>>> misc >>>>> scripts >>>>> released scripts in a date-stamped directory of your choice >>>>> ... >>>>> >>>>> J�rg >>>>> >>>>> >>>>> On Thu, Sep 22, 2011 at 4:34 PM, Barry Haddow >>>>> <[email protected]> wrote: >>>>>> Hi Hieu et al >>>>>> >>>>>> I replied to the OP on this, but forgot to CC it to the list. >>>>>> >>>>>> There was a change in mert some time in the summer (from Prague) >>>>>> meaning >>>>> that >>>>>> if you run an old mert-moses.perl with a new mert binary, you >>>>>> get >>>>>> an >>>>> error >>>>>> with exit code 3. I suspect this is the problem here since the >>>>> scripts-rootdir >>>>>> suggests the scripts are from March. Checking mert.log would >>>>>> confirm the >>>>>> diagnosis. >>>>>> >>>>>> The solution is to rerun 'make release' in the scripts >>>>>> directory, >>>>>> and use >>>>> the >>>>>> new scripts-rootdir. >>>>>> >>>>>> As an aside, I should say that I'm not keen on our two-level >>>>>> make >>>>>> setup, >>>>> and >>>>>> would like to find something simpler. Probably a one level make, >>>>>> with a >>>>>> conventional 'make install', but with the default being to >>>>>> install >>>>>> in a >>>>> date >>>>>> stamped directory. >>>>>> >>>>>> cheers - Barry >>>>>> >>>>>> >>>>>> On Thursday 22 Sep 2011 14:59:26 Hieu Hoang wrote: >>>>>>> i'm afraid i don't know what the problem is. The n-best-list >>>>>>> and >>>>>>> ini >>>>>>> files look ok. I'm running mert and moses-chart that was >>>>>>> checked >>>>>>> out on >>>>>>> the 13th September and ran ok. >>>>>>> >>>>>>> i'm not familiar with the mert code so i can't tell you if the >>>>>>> changes >>>>>>> you made are good. However, you're welcome to post the changes >>>>>>> to >>>>>>> the >>>>>>> mailing list and someone might be know better than i do. >>>>>>> >>>>>>> On 22/09/2011 15:34, Prasanth K wrote: >>>>>>>> Hi Hieu, >>>>>>>> >>>>>>>> I am attaching the following files: >>>>>>>> >>>>>>>> multi-threads_tune-run1.100best.out - first 1000 lines from >>>>>>>> the >>>>>>>> multi-threaded decoder >>>>>>>> single-threads_tune-run1.100best.out - the same from the >>>>>>>> single-threaded decoder >>>>>>>> >>>>>>>> multi-threads_tune-run1.moses.ini - the ini file used at the >>>>>>>> beginning >>>>>>>> of the tuning when using threads >>>>>>>> single-threads_tune-run1.moses.ini - the ini file used at the >>>>>>>> beginning of the tuning when threads were not used >>>>>>>> single-threads_tune-run2.moses.ini - the ini file used at the >>>>>>>> beginning of the second iteration when threads were not used >>>>>>>> >>>>>>>> - Prasanth >>>>>>>> >>>>>>>> On Thu, Sep 22, 2011 at 10:19 AM, Hieu >>>>>>>> Hoang<[email protected] >>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>> >>>>>>>> can you send me your ini file, and a few lines of the >>>>>>>> n-best list. >>>>>>>> For accuracy, send them as attachements, not cut& paste >>>>>>>> into the >>>>>>>> email. >>>>>>>> >>>>>>>> I'll try& see what the problem is >>>>>>>> >>>>>>>> >>>>>>>> On 22 September 2011 14:57, Prasanth >>>>>>>> K<[email protected] >>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> I am facing the same error that Cyrine Nasri >>>>>>>> mentioned >>>>>>>> in this >>>>>>>> thread. >>>>>>>> >>>>>>>> I will try and give more information that what has >>>>>>>> already >>>>>>>> been mentioned. >>>>>>>> 1. I was using a single-threaded version of moses >>>>>>>> until >>>>>>>> earlier and was having no problem with the >>>>>>>> experiments >>>>>>>> using >>>>> EMS. >>>>>>>> 2. I recently shifted to a multi-threaded version, >>>>>>>> and >>>>>>>> tried >>>>>>>> the same experiment again with EMS. >>>>>>>> This time, the tuning process crashes after a single >>>>>>>> iteration >>>>>>>> with exactly the same error as mentioned below. >>>>>>>> 3. I have used 10 threads for decoding in the tuning >>>>>>>> process, >>>>>>>> and am using a server with 32Gb ram for running the >>>>>>>> experiments. (Might not be related, but just thought >>>>>>>> I >>>>>>>> should >>>>>>>> mention!) >>>>>>>> >>>>>>>> I am not sure if Cyrine's problem was with the >>>>>>>> multi-threaded >>>>>>>> version as well, but could some one point out as to >>>>>>>> what might >>>>>>>> be wrong in this picture ? >>>>>>>> >>>>>>>> - Prasanth >>>>>>>> >>>>>>>> On Wed, Jul 27, 2011 at 5:59 PM, Cyrine NASRI >>>>>>>> >>>>>>>> <[email protected]<mailto:[email protected]>> >>>>> wrote: >>>>>>>> Hello, >>>>>>>> I 'm trying to launch the mert using this >>>>>>>> command: >>>>>>>> >>>>>>>> ./mert-moses-new.pl<http://mert-moses-new.pl> >>>>>>>> /users/parole/cnasri/moses_work/corpus/source2TOK >>>>>>>> /users/parole/cnasri/moses_work/corpus/ref2TOK >>>>>>>> >>>>>>>> /users/parole/cnasri/moses_work/moses/moses-cmd/src/moses >>>>>>>> >>>>>>>> >>>>> >>>>> /users/parole/cnasri/moses_work/moses-scripts/scripts-20110727-1543/train >>>>>>>> ing/moses.ini --working-dir >>>>>>>> /users/parole/cnasri/moses_work/tuning/ >>>>>>>> --mertdir /users/parole/cnasri/moses_work/moses/mert >>>>>>>> >>>>>>>> But there is only one iteration, after it stops >>>>>>>> and >>>>> itshow: >>>>>>>> Peeking at the beginning of nbestlist to get >>>>>>>> order >>>>>>>> of >>>>>>>> scores: run1.best100.out >>>>>>>> The decoder returns the scores in this order: d >>>>>>>> lm >>>>>>>> w tm tm >>>>> tm >>>>>>>> Executing: gzip -f run1.best100.out >>>>>>>> Scoring the nbestlist. >>>>>>>> Executing: >>>>>>>> >>>>>>>> /users/parole/cnasri/moses_work/moses/mert/extractor >>>>>>>> --scconfig case:true --scfile run1.scores.dat >>>>>>>> --ffile >>>>>>>> run1.features.dat -r >>>>>>>> /users/parole/cnasri/moses_work/corpus/ref2TOK -n >>>>>>>> run1.best100.out.gz> extract.out 2> extract.err >>>>>>>> Executing: \cp -f init.opt run1.init.opt >>>>>>>> Executing: >>>>>>>> /users/parole/cnasri/moses_work/moses/mert/mert >>>>>>>> -d 6 --scconfig case:true -n 20 --ffile >>>>>>>> run1.features.dat >>>>>>>> --scfile run1.scores.dat --ifile run1.init.opt 2> >>>>>>>> mert.log >>>>>>>> Exit code: 3 >>>>>>>> Failed to run mert at ./mert-moses-new.pl >>>>>>>> <http://mert-moses-new.pl> line 752. >>>>>>>> and in the mert log : >>>>>>>> >>>>>>>> Seeding random numbers with system clock >>>>>>>> >>>>>>>> :Too few minimum weights. >>>>>>>> >>>>>>>> error could not initialize start point with >>>>>>>> >>>>>>>> I have not idea how i resolve this problem. >>>>>>>> Any idea please? >>>>>>>> >>>>>>>> Thank you >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Moses-support mailing list >>>>>>>> >>>>>>>> [email protected]<mailto:[email protected]> >>>>>>>> >>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> --- J.B.S. Haldane >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Moses-support mailing list >>>>>>>> [email protected]<mailto:[email protected]> >>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> --- J.B.S. Haldane >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected] >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> >>>>> >>>>> ********************************************************************************** >>>>> J�rg Tiedemann >>>>> [email protected] >>>>> Dep. of Linguistics and Philology >>>>> http://stp.lingfil.uu.se/~joerg/ >>>>> Uppsala University tel: +46 >>>>> (0)18 - 471 >>>>> 1412 >>>>> Box 635, SE-751 26 Uppsala/SWEDEN fax: +46 (0)18 - 471 1094 >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Moses-support mailing list >>>> [email protected] >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
