You would probably get a lot of problems when throwing everything into one bin-directory. There are, for example, two 'score' binaries (one for phrase scoring and one for lexical reordering). But it would be nice if everything executable ends up in appropriate sub-directories of one common bin directory. Maybe two directories (scripts & bin). Even links (or copies) of the helper tools (GIZA++, mkcls, ...) could be added there. Installing everything to global directories is probably not necessary. It would always require admin-permissions and most Moses-users do not have them on university machines etc. And it wouldn't help much if the scripts and binaries are still not in the global exec-path but in several sub-dirs.
But in general, I would also prefer the solution with using the --prefix parameter for people who really like to keep several versions of the binaries in parallel. Jörg On Wed, Sep 28, 2011 at 11:55 AM, Barry Haddow <[email protected]> wrote: > Hi > > So it looks like a date-stamped release folder is not popular. It would be > possible to have a version stamp inserted into the binaries and scripts and > have this read by ems - would this be useful? > > And would a 'make install' be of any use? It would mean that anyone who wanted > a date-stamped release could make one with --prefix, and would help users to > separate files needed at runtime from those needed at build time. > > If we do have a 'make install', should the scripts be squished into the bin > directory, or have their directory structure preserved? If we go for the > latter, then there would be the option of running them from a source directory > too. > > cheers - Barry > > > > On Tuesday 27 September 2011 23:53:29 Raphael Payen wrote: >> I like your idea of a VERSION.txt file. Maybe containing also the >> configure line. >> >> By the way, there is already a make install which is configurable with >> "configure --prefix=..." ; I dont understand if you want more than >> this. >> >> Raphael >> >> 2011/9/27 Lane Schwartz <[email protected]>: >> > I would like to see a true make install option. I don't care what the >> > default location is, as long as it's configurable. >> > >> > At the least, I would prefer to see the binaries (and preferably >> > scripts, too) installed to a bin directory within the moses dir >> > instead of being spread throughout the source directories. >> > >> > I agree that it can be useful to know what version you're running on. >> > But I don't think time stamped directories are the way to do it. I >> > would propose that make install creates a VERSION.txt file that lists >> > the svn version number, or some other version number. >> > >> > Lane >> > >> > On Tue, Sep 27, 2011 at 6:07 AM, Hieu Hoang <[email protected]> wrote: >> >> I'd go with Ondrej and avoid any release procedure and run everything >> >> in-situ. We can debug and commit scripts easier. >> >> >> >> Or have a very simple release that moves all executables and scripts >> >> into a bin directory. If the user wants to move/rename, it's their >> >> choice >> >> >> >> On 27 September 2011 16:30, Barry Haddow <[email protected]> wrote: >> >>> Hi >> >>> >> >>> I think it's useful to have a date-stamped release directory, so that >> >>> you can >> >>> work on the code and have experiments running, but keep tracking of >> >>> which version was used in which experiment. Many people at Edinburgh >> >>> make a copy of >> >>> the moses binary with the svn version number as suffix on the name. >> >>> >> >>> How about having a 'make install, which by default installs to a >> >>> date-stamped >> >>> directory, with the standard bin, lib etc. pattern, with all the >> >>> scripts under >> >>> bin? How big a change would that be? >> >>> >> >>> cheers - Barry >> >>> >> >>> On Friday 23 September 2011 08:46:06 Ondrej Bojar wrote: >> >>> > Hi, >> >>> > >> >>> > mert-moses-new.pl is outdated, isn't it? There should be only >> >>> > mert-moses.pl these days. >> >>> > >> >>> > The 'scripts releasing' is of mine and dates back to JHU workshop in >> >>> > 2006. Back then, we were hacking the scripts and running experiments >> >>> > at the same time, so we needed some track of what version of scripts >> >>> > was used for that particular failing experiment. >> >>> > >> >>> > As soon as anyone has some spare time I'd suggest: >> >>> > >> >>> > - delete the 'releasing' code in the Makefile >> >>> > - make sure the main make compiles everything: all moseses, all >> >>> > auxiliary binaries in scripts etc. >> >>> > - I'd avoid implementing 'make install', because all the tools in >> >>> > scripts know where their frieds are sitting in terms of relative >> >>> > paths. The install would need to preserve the complicated structure >> >>> > anyway so there's not much point in having the install as 'cp -r' >> >>> > does the same job. >> >>> > >> >>> > Cheers, O. >> >>> > >> >>> > On 09/23/2011 09:20 AM, Hieu Hoang wrote: >> >>> > > agreed, we were having an offlist moan about the same thing. The >> >>> > > separation between decoding& training, and release date-stamp >> >>> > > thingy is >> >>> > > historical and quite silly. >> >>> > > >> >>> > > The directories& release procedure has to be rationalised. >> >>> > > >> >>> > > Someone will do it eventually... >> >>> > > >> >>> > > On 23 September 2011 14:04, Joerg >> >>> >> >>> Tiedemann<[email protected]>wrote: >> >>> > >> By the way, what is the use of a date-stamped directory anyway? >> >>> > >> I find if rather disturbing that all the binaries and scripts are >> >>> > >> distributed all over the place. >> >>> > >> moses-cmd/src >> >>> > >> moses-chart-cmd/src >> >>> > >> misc >> >>> > >> scripts >> >>> > >> released scripts in a date-stamped directory of your choice >> >>> > >> ... >> >>> > >> >> >>> > >> J�rg >> >>> > >> >> >>> > >> >> >>> > >> On Thu, Sep 22, 2011 at 4:34 PM, Barry Haddow >> >>> > >> >> >>> > >> <[email protected]> wrote: >> >>> > >>> Hi Hieu et al >> >>> > >>> >> >>> > >>> I replied to the OP on this, but forgot to CC it to the list. >> >>> > >>> >> >>> > >>> There was a change in mert some time in the summer (from Prague) >> >>> > >>> meaning >> >>> > >> >> >>> > >> that >> >>> > >> >> >>> > >>> if you run an old mert-moses.perl with a new mert binary, you get >> >>> > >>> an >> >>> > >> >> >>> > >> error >> >>> > >> >> >>> > >>> with exit code 3. I suspect this is the problem here since the >> >>> > >> >> >>> > >> scripts-rootdir >> >>> > >> >> >>> > >>> suggests the scripts are from March. Checking mert.log would >> >>> > >>> confirm the diagnosis. >> >>> > >>> >> >>> > >>> The solution is to rerun 'make release' in the scripts directory, >> >>> > >>> and >> >>> > >>> use >> >>> > >> >> >>> > >> the >> >>> > >> >> >>> > >>> new scripts-rootdir. >> >>> > >>> >> >>> > >>> As an aside, I should say that I'm not keen on our two-level make >> >>> > >>> setup, >> >>> > >> >> >>> > >> and >> >>> > >> >> >>> > >>> would like to find something simpler. Probably a one level make, >> >>> > >>> with a >> >>> > >>> conventional 'make install', but with the default being to >> >>> > >>> install in a >> >>> > >> >> >>> > >> date >> >>> > >> >> >>> > >>> stamped directory. >> >>> > >>> >> >>> > >>> cheers - Barry >> >>> > >>> >> >>> > >>> On Thursday 22 Sep 2011 14:59:26 Hieu Hoang wrote: >> >>> > >>>> i'm afraid i don't know what the problem is. The n-best-list and >> >>> > >>>> ini >> >>> > >>>> files look ok. I'm running mert and moses-chart that was checked >> >>> > >>>> out >> >>> > >>>> on the 13th September and ran ok. >> >>> > >>>> >> >>> > >>>> i'm not familiar with the mert code so i can't tell you if the >> >>> > >>>> changes >> >>> > >>>> you made are good. However, you're welcome to post the changes >> >>> > >>>> to the >> >>> > >>>> mailing list and someone might be know better than i do. >> >>> > >>>> >> >>> > >>>> On 22/09/2011 15:34, Prasanth K wrote: >> >>> > >>>>> Hi Hieu, >> >>> > >>>>> >> >>> > >>>>> I am attaching the following files: >> >>> > >>>>> >> >>> > >>>>> multi-threads_tune-run1.100best.out - first 1000 lines from the >> >>> > >>>>> multi-threaded decoder >> >>> > >>>>> single-threads_tune-run1.100best.out - the same from the >> >>> > >>>>> single-threaded decoder >> >>> > >>>>> >> >>> > >>>>> multi-threads_tune-run1.moses.ini - the ini file used at the >> >>> > >>>>> beginning of the tuning when using threads >> >>> > >>>>> single-threads_tune-run1.moses.ini - the ini file used at the >> >>> > >>>>> beginning of the tuning when threads were not used >> >>> > >>>>> single-threads_tune-run2.moses.ini - the ini file used at the >> >>> > >>>>> beginning of the second iteration when threads were not used >> >>> > >>>>> >> >>> > >>>>> - Prasanth >> >>> > >>>>> >> >>> > >>>>> On Thu, Sep 22, 2011 at 10:19 AM, Hieu >> >>> > >>>>> Hoang<[email protected] <mailto:[email protected]>> wrote: >> >>> > >>>>> >> >>> > >>>>> can you send me your ini file, and a few lines of the >> >>> > >>>>> n-best list. For accuracy, send them as attachements, not cut& >> >>> > >>>>> paste into >> >>> > >>>>> the email. >> >>> > >>>>> >> >>> > >>>>> I'll try& see what the problem is >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> On 22 September 2011 14:57, Prasanth >> >>> > >>>>> K<[email protected] >> >>> > >>>>> <mailto:[email protected]>> wrote: >> >>> > >>>>> >> >>> > >>>>> Hi all, >> >>> > >>>>> >> >>> > >>>>> I am facing the same error that Cyrine Nasri mentioned >> >>> > >>>>> in this thread. >> >>> > >>>>> >> >>> > >>>>> I will try and give more information that what has >> >>> > >>>>> already >> >>> > >>>>> been mentioned. >> >>> > >>>>> 1. I was using a single-threaded version of moses >> >>> > >>>>> until earlier and was having no problem with the experiments >> >>> > >>>>> using >> >>> > >> >> >>> > >> EMS. >> >>> > >> >> >>> > >>>>> 2. I recently shifted to a multi-threaded version, >> >>> > >>>>> and tried the same experiment again with EMS. >> >>> > >>>>> This time, the tuning process crashes after a single >> >>> > >>>>> iteration with exactly the same error as mentioned below. >> >>> > >>>>> 3. I have used 10 threads for decoding in the tuning >> >>> > >>>>> process, and am using a server with 32Gb ram for running the >> >>> > >>>>> experiments. (Might not be related, but just thought I should >> >>> > >>>>> mention!) >> >>> > >>>>> >> >>> > >>>>> I am not sure if Cyrine's problem was with the >> >>> > >>>>> multi-threaded version as well, but could some one point out as >> >>> > >>>>> to what might be wrong in this picture ? >> >>> > >>>>> >> >>> > >>>>> - Prasanth >> >>> > >>>>> >> >>> > >>>>> On Wed, Jul 27, 2011 at 5:59 PM, Cyrine NASRI >> >>> > >>>>> >> >>> > >>>>> <[email protected]<mailto:[email protected]>> >> >>> > >> >> >>> > >> wrote: >> >>> > >>>>> Hello, >> >>> > >>>>> I 'm trying to launch the mert using this command: >> >>> > >>>>> >> >>> > >>>>> ./mert-moses-new.pl<http://mert-moses-new.pl> >> >>> > >>>>> /users/parole/cnasri/moses_work/corpus/source2TOK >> >>> > >>>>> /users/parole/cnasri/moses_work/corpus/ref2TOK >> >>> > >>>>> >> >>> > >>>>> /users/parole/cnasri/moses_work/moses/moses-cmd/src/moses >> >>> > >> >> >>> > >> /users/parole/cnasri/moses_work/moses-scripts/scripts-20110727-154 >> >>> > >>3/trai n >> >>> > >> >> >>> > >>>>> ing/moses.ini --working-dir >> >>> > >>>>> /users/parole/cnasri/moses_work/tuning/ >> >>> > >>>>> --mertdir /users/parole/cnasri/moses_work/moses/mert >> >>> > >>>>> >> >>> > >>>>> But there is only one iteration, after it stops >> >>> > >>>>> and >> >>> > >> >> >>> > >> itshow: >> >>> > >>>>> Peeking at the beginning of nbestlist to get order >> >>> > >>>>> of scores: run1.best100.out >> >>> > >>>>> The decoder returns the scores in this order: d lm >> >>> > >>>>> w tm >> >>> > >>>>> tm >> >>> > >> >> >>> > >> tm >> >>> > >> >> >>> > >>>>> Executing: gzip -f run1.best100.out >> >>> > >>>>> Scoring the nbestlist. >> >>> > >>>>> Executing: >> >>> > >>>>> >> >>> > >>>>> /users/parole/cnasri/moses_work/moses/mert/extractor >> >>> > >>>>> --scconfig case:true --scfile run1.scores.dat --ffile >> >>> > >>>>> run1.features.dat -r >> >>> > >>>>> /users/parole/cnasri/moses_work/corpus/ref2TOK -n >> >>> > >>>>> run1.best100.out.gz> extract.out 2> extract.err >> >>> > >>>>> Executing: \cp -f init.opt run1.init.opt >> >>> > >>>>> Executing: >> >>> > >>>>> /users/parole/cnasri/moses_work/moses/mert/mert -d 6 >> >>> > >>>>> --scconfig case:true -n 20 --ffile run1.features.dat --scfile >> >>> > >>>>> run1.scores.dat --ifile run1.init.opt 2> mert.log Exit code: 3 >> >>> > >>>>> Failed to run mert at ./mert-moses-new.pl >> >>> > >>>>> <http://mert-moses-new.pl> line 752. >> >>> > >>>>> and in the mert log : >> >>> > >>>>> >> >>> > >>>>> Seeding random numbers with system clock >> >>> > >>>>> >> >>> > >>>>> :Too few minimum weights. >> >>> > >>>>> >> >>> > >>>>> error could not initialize start point with >> >>> > >>>>> >> >>> > >>>>> I have not idea how i resolve this problem. >> >>> > >>>>> Any idea please? >> >>> > >>>>> >> >>> > >>>>> Thank you >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> _______________________________________________ >> >>> > >>>>> Moses-support mailing list >> >>> > >>>>> >> >>> > >>>>> [email protected]<mailto:[email protected]> >> >>> > >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> --- J.B.S. Haldane >> >>> > >>>>> >> >>> > >>>>> _______________________________________________ >> >>> > >>>>> Moses-support mailing list >> >>> > >>>>> [email protected]<mailto:[email protected]> >> >>> > >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> >> >>> > >>>>> --- J.B.S. Haldane >> >>> > >>> >> >>> > >>> _______________________________________________ >> >>> > >>> Moses-support mailing list >> >>> > >>> [email protected] >> >>> > >>> http://mailman.mit.edu/mailman/listinfo/moses-support >> >>> > >> >> >>> > >> -- >> >>> > >> >> >>> > >> >> >>> > >> ****************************************************************** >> >>> > >>****** ********** J�rg Tiedemann >> >>> > >> [email protected] >> >>> > >> Dep. of Linguistics and Philology >> >>> > >> http://stp.lingfil.uu.se/~joerg/ >> >>> > >> Uppsala University tel: +46 >> >>> > >> (0)18 - >> >>> > >> 471 1412 >> >>> > >> Box 635, SE-751 26 Uppsala/SWEDEN fax: +46 (0)18 - 471 1094 >> >>> > > >> >>> > > _______________________________________________ >> >>> > > Moses-support mailing list >> >>> > > [email protected] >> >>> > > http://mailman.mit.edu/mailman/listinfo/moses-support >> >>> >> >>> -- >> >>> The University of Edinburgh is a charitable body, registered in >> >>> Scotland, with registration number SC005336. >> >>> >> >>> >> >>> _______________________________________________ >> >>> Moses-support mailing list >> >>> [email protected] >> >>> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> >> _______________________________________________ >> >> Moses-support mailing list >> >> [email protected] >> >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > >> > -- >> > When a place gets crowded enough to require ID's, social collapse is not >> > far away. It is time to go elsewhere. The best thing about space travel >> > is that it made it possible to go elsewhere. >> > -- R.A. Heinlein, "Time Enough For Love" >> > >> > _______________________________________________ >> > Moses-support mailing list >> > [email protected] >> > http://mailman.mit.edu/mailman/listinfo/moses-support >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > -- > Barry Haddow > University of Edinburgh > +44 (0) 131 651 3173 > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- ********************************************************************************** Jörg Tiedemann [email protected] Dep. of Linguistics and Philology http://stp.lingfil.uu.se/~joerg/ Uppsala University tel: +46 (0)18 - 471 1412 Box 635, SE-751 26 Uppsala/SWEDEN fax: +46 (0)18 - 471 1094 _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
