Hi Lane, Thanks for your interest in large files!
As Marcin said, HTTP using aria2c is the preferred method of download. I also produced zsync [1] files. That's something like rsync over http using a static metadata file which should be able to fix a broken download / recover from a partial download. Raw English text is still available here: http://statmt.org/ngrams/raw_en/ but I may shuffle the files around occasionally because they clog up our fileservers. If possible send me a notice before downloading so I can take care that they persist for that time. cheers, Christian [1] http://zsync.moria.org.uk/ On 25/01/2015 17:27, Marcin Junczys-Dowmunt wrote: > Hi, > I managed to download the 5.5 TB monster using just aria2c, worked > splendidly. BTW, what happend to the English language data text files? > > W dniu 25.01.2015 o 17:06, Lane Schwartz pisze: >> I'm interested in downloading some of the pre-trained models that are >> hosted at statmt.org, including the 5.5 TB English language model >> (http://statmt.org/ngrams/lm/). >> >> My University cluster strongly recommends that I make use of Globus >> Connect (https://www.globus.org/globus-connect-personal) to manage >> large file transfers. It appears that software takes care of a lot of >> the details (reconnections, etc) when transferring large files. >> >> I was wondering if it might be possible to set up that software (at >> least on a temporary basis) on whatever server is hosting the statmt >> files. Who would be the person for me to talk to about that? >> >> Thanks, >> Lane >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
