[Wikitech-l] Collection extension
Hi, Is there a planned deployment date for the Collection (PediaPress) extension on Wikimedia wikis that currently don't have them? The blog post about it [1] mentioned March if everything went well. Best regards, Bence [1] http://blog.wikimedia.org/2009/02/20/wiki-to-print-feature-activated-in-six-more-wikipedia-languages/ ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] Dealing with Large Files when attempting a wikipedia database download.
Does anyone on the wikitech mailing list happen to know whether it would be possible for some of the larger wikipedia database downloads (which are, say, 16GB or so in size) to be split into parts so that they can be downloaded. For whatever reason, whenever I have attempted to download the ~14GB files (say, from http://static.wikipedia.org/downloads/2008-06/en/ ), I have found that only 2GB (presumably, the first 2GB) of what I have sought to download has actually been downloaded. Is there anyway around this? Could anyone possibly suggest what possible reasons there might be for this difficulty in downloading the material? Thanks. . ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Dealing with Large Files when attempting a wikipedia database download.
2009/4/10 Jameson Scanlon jameson.scan...@googlemail.com: Does anyone on the wikitech mailing list happen to know whether it would be possible for some of the larger wikipedia database downloads (which are, say, 16GB or so in size) to be split into parts so that they can be downloaded. For whatever reason, whenever I have attempted to download the ~14GB files (say, from http://static.wikipedia.org/downloads/2008-06/en/ ), I have found that only 2GB (presumably, the first 2GB) of what I have sought to download has actually been downloaded. Is there anyway around this? Could anyone possibly suggest what possible reasons there might be for this difficulty in downloading the material? Downloading to a filesystem that only does maximum 2GB files? - d. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Dealing with Large Files when attempting a wikipedia database download.
David Gerard schrieb: 2009/4/10 Jameson Scanlon jameson.scan...@googlemail.com: Does anyone on the wikitech mailing list happen to know whether it would be possible for some of the larger wikipedia database downloads (which are, say, 16GB or so in size) to be split into parts so that they can be downloaded. For whatever reason, whenever I have attempted to download the ~14GB files (say, from http://static.wikipedia.org/downloads/2008-06/en/ ), I have found that only 2GB (presumably, the first 2GB) of what I have sought to download has actually been downloaded. Is there anyway around this? Could anyone possibly suggest what possible reasons there might be for this difficulty in downloading the material? Downloading to a filesystem that only does maximum 2GB files? Also, several http clients don't like files over 2GB - this is because the large number of bytes in the Length field causes an integer overflow (2GB is the 31 bit limit). wget likes to die with a segmentation fault on those. I found that curl works. But of course, the file system also has to support very large files, as Gerard said. Finally: yes, it would be nive to have such dumps available in pieces of perhaps 1GB in size. -- daniel ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Dealing with Large Files when attempting a wikipedia database download.
http://en.wikipedia.org/wiki/Wikipedia_database has some information on how to deal with the large files henna On Fri, Apr 10, 2009 at 21:43, Daniel Kinzler dan...@brightbyte.de wrote: David Gerard schrieb: 2009/4/10 Jameson Scanlon jameson.scan...@googlemail.com: Does anyone on the wikitech mailing list happen to know whether it would be possible for some of the larger wikipedia database downloads (which are, say, 16GB or so in size) to be split into parts so that they can be downloaded. For whatever reason, whenever I have attempted to download the ~14GB files (say, from http://static.wikipedia.org/downloads/2008-06/en/ ), I have found that only 2GB (presumably, the first 2GB) of what I have sought to download has actually been downloaded. Is there anyway around this? Could anyone possibly suggest what possible reasons there might be for this difficulty in downloading the material? Downloading to a filesystem that only does maximum 2GB files? Also, several http clients don't like files over 2GB - this is because the large number of bytes in the Length field causes an integer overflow (2GB is the 31 bit limit). wget likes to die with a segmentation fault on those. I found that curl works. But of course, the file system also has to support very large files, as Gerard said. Finally: yes, it would be nive to have such dumps available in pieces of perhaps 1GB in size. -- daniel ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- Maybe you knew early on that your track went from point A to B, but unlike you I wasn't given a map at birth! Alyssa, Chasing Amy ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Dealing with Large Files when attempting a wikipedia database download.
I have downloaded the history dump file (~150 GB) using Firefox on XP and using wget on Ubuntu and it works fine. I have downloaded it using a download manager on Vista and it is fine also. A more probable reason is the file system limitations. bilal On Fri, Apr 10, 2009 at 3:49 PM, Finne Boonen hen...@gmail.com wrote: http://en.wikipedia.org/wiki/Wikipedia_database has some information on how to deal with the large files henna On Fri, Apr 10, 2009 at 21:43, Daniel Kinzler dan...@brightbyte.de wrote: David Gerard schrieb: 2009/4/10 Jameson Scanlon jameson.scan...@googlemail.com: Does anyone on the wikitech mailing list happen to know whether it would be possible for some of the larger wikipedia database downloads (which are, say, 16GB or so in size) to be split into parts so that they can be downloaded. For whatever reason, whenever I have attempted to download the ~14GB files (say, from http://static.wikipedia.org/downloads/2008-06/en/ ), I have found that only 2GB (presumably, the first 2GB) of what I have sought to download has actually been downloaded. Is there anyway around this? Could anyone possibly suggest what possible reasons there might be for this difficulty in downloading the material? Downloading to a filesystem that only does maximum 2GB files? Also, several http clients don't like files over 2GB - this is because the large number of bytes in the Length field causes an integer overflow (2GB is the 31 bit limit). wget likes to die with a segmentation fault on those. I found that curl works. But of course, the file system also has to support very large files, as Gerard said. Finally: yes, it would be nive to have such dumps available in pieces of perhaps 1GB in size. -- daniel ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] Dealing with Large Files when attempting a wikipedia database download.
I'm pretty sure it's impossible to encourage people to include relevant information in their OPs. You don't suppose you could have at least told us your operating system, whether you are running 32 or 64 bits? Are you on linux with no large file support? On Fri, Apr 10, 2009 at 12:21 PM, Jameson Scanlon jameson.scan...@googlemail.com wrote: Does anyone on the wikitech mailing list happen to know whether it would be possible for some of the larger wikipedia database downloads (which are, say, 16GB or so in size) to be split into parts so that they can be downloaded. For whatever reason, whenever I have attempted to download the ~14GB files (say, from http://static.wikipedia.org/downloads/2008-06/en/ ), I have found that only 2GB (presumably, the first 2GB) of what I have sought to download has actually been downloaded. Is there anyway around this? Could anyone possibly suggest what possible reasons there might be for this difficulty in downloading the material? Thanks. . ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
