I'd like to respond to this thread. I, too, have limited resources (at work, at least) - a 3 GB RAM 32-bit Windows i5 machine with Linux running on VMWare with 2.5 GB RAM. Training and tuning take many hours; the machine is running BitParl over the weekend and may be done with NewsCommentary de-en (DE) on Monday when I get back to work. I'm afraid after all that I won't be able to subsequently run Collins on the English, train, tune, and decode all that and, even if it takes forever, expect it to run with the limited memory resources available. What I think I need to do is trim the corpora according to some criteria that isn't too complicated. Is it enough for learning purposes (we are a long way from any sort of real comparison of results, let alone production - this will receive proper hardware) to take the first n sentences, or every nth sentence? The result is simply to get a feel for the various modes of tree-based SMT, run hierarchical phrase, string-to-tree, tree-to-string and tree-to-tree without worrying which one is the best - the idea is just to get some experience with it.What would be a good number of sentences to take so that it runs relatively quickly, without killing RAM, but produces results that aren't useless? Thanks - and I'd like to thank everyone on the group for their eager helpfulness, and for discussing things that I as a newbie find very useful!
> Date: Sat, 9 Mar 2013 11:25:45 -0500 > From: [email protected] > To: [email protected] > Subject: Re: [Moses-support] Accelerate the tuning > > Hi, > > It won't fix everything, but there is a long-term TODO to rewrite > phrase table scoring to use binary files with vocabulary ids instead of > text files. > > Kenneth > > On 03/09/13 08:30, Per Tunedal wrote: > > > > Hi, > > the training seems to be an overwhelming task for my computer. If it > > ever succeeds, I will have to undertake the even more demanding task of > > tuning. Can anything be done to accelerate it? > > > > Specifically, I wonder if it's feasible to prune the translation table > > before doing the tuning. > > > > Yours, > > Per Tunedal > > > > PS I've abandoned the idea of building a Hierarchical phrase model, I'm > > now trying to make a phrase-based system. I suppose that would use less > > resources. > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
