Re: [Wikitech-l] enwiki dump problems

2010-02-19 Thread Aryeh Gregor
On Fri, Feb 19, 2010 at 1:24 PM, Tei wrote: > On 19 February 2010 14:54, Jamie Morken wrote: >> I hope you guys are planning on adding some way to download the wikimedia >> commons images too at some point > > something that could be fun is git. git is not intended to handle large repositories,

Re: [Wikitech-l] enwiki dump problems

2010-02-19 Thread Tei
On 19 February 2010 14:54, Jamie Morken wrote: > I hope you guys are planning on adding some way to download the wikimedia > commons images too at some point something that could be fun is git. plus something like a "ticket system", where you ask for permissions to download a tree inside git, a

Re: [Wikitech-l] enwiki dump problems

2010-02-19 Thread Tomasz Finc
We were actually a week away from having a finished snapshot last month when we had an unscheduled change. Long and short of it is that Tim's re compression of ES has made huge progress in improving the speed of the work and we simply need to wait for that to finish to re-asses how much more we

[Wikitech-l] enwiki dump problems

2010-02-19 Thread Jamie Morken
Hi, There hasn't been a successful pages-meta-history.xml.bz2 or pages-meta-history.xml.7z dump from the http://download.wikimedia.org/enwiki/ site in the last 5 dumps.  How is the new dump system coming along for these large wiki files?  I personally am a bit concerned that these files haven't