Thank you for the info! It sounds very useful for me, and I hope it can be completed very soon.
Best wishes! Pidong On 3 February 2012 16:37, Marcin Junczys-Dowmunt <[email protected]> wrote: > Hi, > Since the topic of huge reordering tables appeared I just wanted to let > you now that I am working at the moment on a much smaller reordering > table and phrase table representation. The reordering-table is already > working and could be used within Moses without much problems. > For instance, I have a reduction from 114G to 5.2G for the reordering > table. Right now however, it needs to be kept fully in RAM. Some > memory-mapping capabilities are planned. It also needs quite a lot > memory during construction time, but I will do something with that, too. > > If someone wants to take a look or share some comments, it's in the WIPO > branch, LexicalReorderingDictionary.* (class > LexicalReoderingDictionaryHashed). > > Best, > Marcin > > W dniu 03.02.2012 05:50, Wang Pidong pisze: > > Thanks a lot! > > I am going to increase my disk space and try again. > > > > Best wishes! > > Pidong > > > > On 3 February 2012 12:11, Hieu Hoang <[email protected] > > <mailto:[email protected]>> wrote: > > > > yes, that's typical. > > > > as a comparison, my 3.9GB (text, gz) reordering table takes up 40GB > > when binarized. > > > > On Fri, Feb 3, 2012 at 10:49 AM, Wang Pidong <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi all, > > > > I am now trying to binarize a big lexical reordering table, > > *reordering-table.wbe-msd-bidirectional-fe.gz*, whose size is > > 2.4GB after gzip. > > After about 6-hour running of the *processLexicalTable* tool, > > the incomplete binarized table (about 68GB) used up all my disk > > space. > > > > Does anyone have the experience of binarizing big lexical > > reordering table? > > Is my case typical? do I need to increase my disk space then try > > again? > > > > Thank you in advance! > > > > Best wishes! > > Pidong > > > > > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] <mailto:[email protected]> > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > > > > > > -- > > Wang Pidong > > > > Department of Computer Science > > School of Computing > > National University of Singapore > > > > > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- Wang Pidong Department of Computer Science School of Computing National University of Singapore
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
