The date-time stamp also helps prevent accidental deletion of user changes when recompiling the scripts.
Also, when DoMY installs Moses and compiles the scripts, we abstract the date-time folder with a symbolic link "scripts", then set the environment variable to the symbolic link. Our DoMY CE installation takes a different approach that might work here. Our user's scripts are in folders without date-time stamps. When the re-install or upgrade, we rename any existing folders with a date-time stamp and the users must migrate their changes. I think this approach with the moses scripts could work. Tom On Fri, 23 Sep 2011 09:46:06 +0200, Ondrej Bojar <[email protected]> wrote: > Hi, > > mert-moses-new.pl is outdated, isn't it? There should be only > mert-moses.pl these days. > > The 'scripts releasing' is of mine and dates back to JHU workshop in > 2006. Back then, we were hacking the scripts and running experiments > at > the same time, so we needed some track of what version of scripts was > used for that particular failing experiment. > > As soon as anyone has some spare time I'd suggest: > > - delete the 'releasing' code in the Makefile > - make sure the main make compiles everything: all moseses, all > auxiliary binaries in scripts etc. > - I'd avoid implementing 'make install', because all the tools in > scripts know where their frieds are sitting in terms of relative > paths. > The install would need to preserve the complicated structure anyway > so > there's not much point in having the install as 'cp -r' does the same > job. > > Cheers, O. > > On 09/23/2011 09:20 AM, Hieu Hoang wrote: >> agreed, we were having an offlist moan about the same thing. The >> separation >> between decoding& training, and release date-stamp thingy is >> historical and >> quite silly. >> >> The directories& release procedure has to be rationalised. >> >> Someone will do it eventually... >> >> On 23 September 2011 14:04, Joerg >> Tiedemann<[email protected]>wrote: >> >>> By the way, what is the use of a date-stamped directory anyway? >>> I find if rather disturbing that all the binaries and scripts are >>> distributed all over the place. >>> moses-cmd/src >>> moses-chart-cmd/src >>> misc >>> scripts >>> released scripts in a date-stamped directory of your choice >>> ... >>> >>> J�rg >>> >>> >>> On Thu, Sep 22, 2011 at 4:34 PM, Barry Haddow >>> <[email protected]> wrote: >>>> Hi Hieu et al >>>> >>>> I replied to the OP on this, but forgot to CC it to the list. >>>> >>>> There was a change in mert some time in the summer (from Prague) >>>> meaning >>> that >>>> if you run an old mert-moses.perl with a new mert binary, you get >>>> an >>> error >>>> with exit code 3. I suspect this is the problem here since the >>> scripts-rootdir >>>> suggests the scripts are from March. Checking mert.log would >>>> confirm the >>>> diagnosis. >>>> >>>> The solution is to rerun 'make release' in the scripts directory, >>>> and use >>> the >>>> new scripts-rootdir. >>>> >>>> As an aside, I should say that I'm not keen on our two-level make >>>> setup, >>> and >>>> would like to find something simpler. Probably a one level make, >>>> with a >>>> conventional 'make install', but with the default being to install >>>> in a >>> date >>>> stamped directory. >>>> >>>> cheers - Barry >>>> >>>> >>>> On Thursday 22 Sep 2011 14:59:26 Hieu Hoang wrote: >>>>> i'm afraid i don't know what the problem is. The n-best-list and >>>>> ini >>>>> files look ok. I'm running mert and moses-chart that was checked >>>>> out on >>>>> the 13th September and ran ok. >>>>> >>>>> i'm not familiar with the mert code so i can't tell you if the >>>>> changes >>>>> you made are good. However, you're welcome to post the changes to >>>>> the >>>>> mailing list and someone might be know better than i do. >>>>> >>>>> On 22/09/2011 15:34, Prasanth K wrote: >>>>>> Hi Hieu, >>>>>> >>>>>> I am attaching the following files: >>>>>> >>>>>> multi-threads_tune-run1.100best.out - first 1000 lines from the >>>>>> multi-threaded decoder >>>>>> single-threads_tune-run1.100best.out - the same from the >>>>>> single-threaded decoder >>>>>> >>>>>> multi-threads_tune-run1.moses.ini - the ini file used at the >>>>>> beginning >>>>>> of the tuning when using threads >>>>>> single-threads_tune-run1.moses.ini - the ini file used at the >>>>>> beginning of the tuning when threads were not used >>>>>> single-threads_tune-run2.moses.ini - the ini file used at the >>>>>> beginning of the second iteration when threads were not used >>>>>> >>>>>> - Prasanth >>>>>> >>>>>> On Thu, Sep 22, 2011 at 10:19 AM, Hieu Hoang<[email protected] >>>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> can you send me your ini file, and a few lines of the >>>>>> n-best list. >>>>>> For accuracy, send them as attachements, not cut& paste >>>>>> into the >>>>>> email. >>>>>> >>>>>> I'll try& see what the problem is >>>>>> >>>>>> >>>>>> On 22 September 2011 14:57, Prasanth >>>>>> K<[email protected] >>>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> I am facing the same error that Cyrine Nasri mentioned >>>>>> in this >>>>>> thread. >>>>>> >>>>>> I will try and give more information that what has >>>>>> already >>>>>> been mentioned. >>>>>> 1. I was using a single-threaded version of moses >>>>>> until >>>>>> earlier and was having no problem with the experiments >>>>>> using >>> EMS. >>>>>> 2. I recently shifted to a multi-threaded version, and >>>>>> tried >>>>>> the same experiment again with EMS. >>>>>> This time, the tuning process crashes after a single >>>>>> iteration >>>>>> with exactly the same error as mentioned below. >>>>>> 3. I have used 10 threads for decoding in the tuning >>>>>> process, >>>>>> and am using a server with 32Gb ram for running the >>>>>> experiments. (Might not be related, but just thought I >>>>>> should >>>>>> mention!) >>>>>> >>>>>> I am not sure if Cyrine's problem was with the >>>>>> multi-threaded >>>>>> version as well, but could some one point out as to >>>>>> what might >>>>>> be wrong in this picture ? >>>>>> >>>>>> - Prasanth >>>>>> >>>>>> On Wed, Jul 27, 2011 at 5:59 PM, Cyrine NASRI >>>>>> <[email protected]<mailto:[email protected]>> >>> wrote: >>>>>> >>>>>> Hello, >>>>>> I 'm trying to launch the mert using this command: >>>>>> >>>>>> ./mert-moses-new.pl<http://mert-moses-new.pl> >>>>>> /users/parole/cnasri/moses_work/corpus/source2TOK >>>>>> /users/parole/cnasri/moses_work/corpus/ref2TOK >>>>>> >>>>>> /users/parole/cnasri/moses_work/moses/moses-cmd/src/moses >>>>>> >>>>>> >>> >>> /users/parole/cnasri/moses_work/moses-scripts/scripts-20110727-1543/train >>>>>> ing/moses.ini --working-dir >>>>>> /users/parole/cnasri/moses_work/tuning/ >>>>>> --mertdir /users/parole/cnasri/moses_work/moses/mert >>>>>> >>>>>> But there is only one iteration, after it stops and >>> itshow: >>>>>> >>>>>> Peeking at the beginning of nbestlist to get order >>>>>> of >>>>>> scores: run1.best100.out >>>>>> The decoder returns the scores in this order: d lm >>>>>> w tm tm >>> tm >>>>>> Executing: gzip -f run1.best100.out >>>>>> Scoring the nbestlist. >>>>>> Executing: >>>>>> >>>>>> /users/parole/cnasri/moses_work/moses/mert/extractor >>>>>> --scconfig case:true --scfile run1.scores.dat >>>>>> --ffile >>>>>> run1.features.dat -r >>>>>> /users/parole/cnasri/moses_work/corpus/ref2TOK -n >>>>>> run1.best100.out.gz> extract.out 2> extract.err >>>>>> Executing: \cp -f init.opt run1.init.opt >>>>>> Executing: >>>>>> /users/parole/cnasri/moses_work/moses/mert/mert >>>>>> -d 6 --scconfig case:true -n 20 --ffile >>>>>> run1.features.dat >>>>>> --scfile run1.scores.dat --ifile run1.init.opt 2> >>>>>> mert.log >>>>>> Exit code: 3 >>>>>> Failed to run mert at ./mert-moses-new.pl >>>>>> <http://mert-moses-new.pl> line 752. >>>>>> and in the mert log : >>>>>> >>>>>> Seeding random numbers with system clock >>>>>> >>>>>> :Too few minimum weights. >>>>>> >>>>>> error could not initialize start point with >>>>>> >>>>>> I have not idea how i resolve this problem. >>>>>> Any idea please? >>>>>> >>>>>> Thank you >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected]<mailto:[email protected]> >>>>>> >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --- J.B.S. Haldane >>>>>> >>>>>> _______________________________________________ >>>>>> Moses-support mailing list >>>>>> [email protected]<mailto:[email protected]> >>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --- J.B.S. Haldane >>>>> >>>> _______________________________________________ >>>> Moses-support mailing list >>>> [email protected] >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> >>> >>> >>> >>> -- >>> >>> >>> ********************************************************************************** >>> J�rg Tiedemann >>> [email protected] >>> Dep. of Linguistics and Philology >>> http://stp.lingfil.uu.se/~joerg/ >>> Uppsala University tel: +46 >>> (0)18 - 471 >>> 1412 >>> Box 635, SE-751 26 Uppsala/SWEDEN fax: +46 (0)18 - 471 1094 >>> >> >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
