Re: 2 giga file size limit ?
Hallo, Noel, I would suggest you to take http://download.fedora.redhat.com/pub/fedora/linux/core/development/SRPMS/wget-1.9.1-17.src.rpm then look carefully at redhat patches. I took wget-cvs as a basic, because I had an impression there were important bug fixes there. FC folks took wget 1.9.1, applied a battery of their patches, then tried to create their own LFS patch using my patch as a basis. This required an extra work which unfortunately (see https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=123524 ) did not go smoothly, but eventually it has succeeded. I do not know new bugs in FC wget-1.9.1-17.src.rpm Leonid
Re: 2 giga file size limit ?
Am Donnerstag, den 09.09.2004, 23:44 -0400 schrieb Leonid: Hello, There exists more than one LFS patch for wget. For Linux, Sun and HP-UX one can use http://software.lpetrov.net/wget-LFS/ This is a patch against the cvs version of wget. Do anybody know a LFS patch for 1.9.1? thx. -- Nol Kthe noel debian.org Debian GNU/Linux, www.debian.org signature.asc Description: Dies ist ein digital signierter Nachrichtenteil
Re: 2 giga file size limit ?
Jonathan, But i am curious. If patches exist, why have they not been merged into the 'vanilla' wget? Seems this is a very requested feature, and an often encountered bug... I am not familiar with wget developers, but I suppose they are currently busy with other projects. The beauty of free source software is that in such a situation users do not need to wait when the software developers will make fixes. I guess wget developers would like to make large file support in a portable way, which is much more difficult than to make it for three specific platforms. Anyway, redhat team is currently testing my LFS patch together with a bunch of their other patches (bug 123524), so we can expect that Fedora Core 3 will have wget with large file support. Leonid
Re: 2 giga file size limit ?
On Thu, 09 Sep 2004 23:44:09 -0400, Leonid [EMAIL PROTECTED] wrote: Jonathan, There exists more than one LFS patch for wget. For Linux, Sun and HP-UX one can use http://software.lpetrov.net/wget-LFS/ I shall have to try said patches. And then find a large file to download. But i am curious. If patches exist, why have they not been merged into the 'vanilla' wget? Seems this is a very requested feature, and an often encountered bug... -- Jonathan
Re: 2 giga file size limit ?
Yep sorry to being a pain , I've seen a bit later that the issue raised quite a lot of times in the past, My point is though that wget compiled on Tru64 OS does work with huge files. Jonathan Stewart wrote: Wget doesn't support 2GB files. It is a known issue that is brought up a lot. Please patch if you're able, so far no fix has been forthcoming. Cheers, Jonathan - Original Message - From: david coornaert [EMAIL PROTECTED] Date: Thu, 09 Sep 2004 12:41:31 +0200 Subject: 2 giga file size limit ? To: [EMAIL PROTECTED] Hi all, I'm trying to get around this kind of message on I*86 linux boxes with wget 1.9.1 --11:12:08-- ftp://ftp.ensembl.org/pub/current_human/data/mysql/homo_sapiens_snp_23_34e/RefSNP.txt.table.gz = `current_human/data/mysql/homo_sapiens_snp_23_34e/RefSNP.txt.table.gz' == CWD not required. == PASV ... done.== RETR RefSNP.txt.table.gz ... done. Length: -1,212,203,102 The file is actually more than 3giga, since my main goal is to mirror the whole thing @ensembl-org, It would be very fine if mirroring could be used with huge files too There is no trouble though on Tru64 machines. here is what the .listing file says for this file on the linux boxes : total 3753518 -rw-rw-r-- 1 00 97960 Jul 21 17:05 Assay.txt.table.gz -rw-rw-r-- 1 00 279 Jul 21 19:29 CHECKSUMS.gz -rw-rw-r-- 1 00 153157540 Jul 21 17:08 ContigHit.txt.table.gz -rw-rw-r-- 1 0032 Jul 21 17:08 DataSource.txt.table.gz -rw-rw-r-- 1 00 18359087 Jul 21 17:09 Freq.txt.table.gz -rw-rw-r-- 1 00 46848 Jul 21 17:09 GTInd.txt.table.gz -rw-rw-r-- 1 00 185265599 Jul 21 17:13 Hit.txt.table.gz -rw-rw-r-- 1 00 35914149 Jul 21 17:14 Locus.txt.table.gz -rw-rw-r-- 1 0020 Jul 21 17:14 Pop.txt.table.gz -rw-rw-r-- 1 003082764194 Jul 21 19:21 RefSNP.txt.table.gz -rw-rw-r-- 1 00 195 Jul 21 19:21 Resource.txt.table.gz -rw-rw-r-- 1 00 72306055 Jul 21 19:23 Strain.txt.table.gz -rw-rw-r-- 1 00 9480171 Jul 21 19:23 SubPop.txt.table.gz -rw-rw-r-- 1 00 286116716 Jul 21 19:27 SubSNP.txt.table.gz -rw-rw-r-- 1 00 49095 Jul 21 19:23 Submitter.txt.table.gz -rw-rw-r-- 1 00 1697 Jul 21 19:27 homo_sapiens_snp_23_34e.sql.gz You can see that the file is appropriately listed , though once the ftp session is started it reports a negative size.. any solution ?
Re: 2 giga file size limit ?
On Fri, 10 Sep 2004, david coornaert wrote: My point is though that wget compiled on Tru64 OS does work with huge files. That's most likely because Tru64 is a 64bit architecture where all longs are 64 bits already and thus most code need no particular fixes to support 4GB files. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: 2 giga file size limit ?
Wget doesn't support 2GB files. It is a known issue that is brought up a lot. Please patch if you're able, so far no fix has been forthcoming. Cheers, Jonathan - Original Message - From: david coornaert [EMAIL PROTECTED] Date: Thu, 09 Sep 2004 12:41:31 +0200 Subject: 2 giga file size limit ? To: [EMAIL PROTECTED] Hi all, I'm trying to get around this kind of message on I*86 linux boxes with wget 1.9.1 --11:12:08-- ftp://ftp.ensembl.org/pub/current_human/data/mysql/homo_sapiens_snp_23_34e/RefSNP.txt.table.gz = `current_human/data/mysql/homo_sapiens_snp_23_34e/RefSNP.txt.table.gz' == CWD not required. == PASV ... done.== RETR RefSNP.txt.table.gz ... done. Length: -1,212,203,102 The file is actually more than 3giga, since my main goal is to mirror the whole thing @ensembl-org, It would be very fine if mirroring could be used with huge files too There is no trouble though on Tru64 machines. here is what the .listing file says for this file on the linux boxes : total 3753518 -rw-rw-r-- 1 00 97960 Jul 21 17:05 Assay.txt.table.gz -rw-rw-r-- 1 00 279 Jul 21 19:29 CHECKSUMS.gz -rw-rw-r-- 1 00 153157540 Jul 21 17:08 ContigHit.txt.table.gz -rw-rw-r-- 1 0032 Jul 21 17:08 DataSource.txt.table.gz -rw-rw-r-- 1 00 18359087 Jul 21 17:09 Freq.txt.table.gz -rw-rw-r-- 1 00 46848 Jul 21 17:09 GTInd.txt.table.gz -rw-rw-r-- 1 00 185265599 Jul 21 17:13 Hit.txt.table.gz -rw-rw-r-- 1 00 35914149 Jul 21 17:14 Locus.txt.table.gz -rw-rw-r-- 1 0020 Jul 21 17:14 Pop.txt.table.gz -rw-rw-r-- 1 003082764194 Jul 21 19:21 RefSNP.txt.table.gz -rw-rw-r-- 1 00 195 Jul 21 19:21 Resource.txt.table.gz -rw-rw-r-- 1 00 72306055 Jul 21 19:23 Strain.txt.table.gz -rw-rw-r-- 1 00 9480171 Jul 21 19:23 SubPop.txt.table.gz -rw-rw-r-- 1 00 286116716 Jul 21 19:27 SubSNP.txt.table.gz -rw-rw-r-- 1 00 49095 Jul 21 19:23 Submitter.txt.table.gz -rw-rw-r-- 1 00 1697 Jul 21 19:27 homo_sapiens_snp_23_34e.sql.gz You can see that the file is appropriately listed , though once the ftp session is started it reports a negative size.. any solution ? -- Jonathan
Re: 2 giga file size limit ?
Jonathan, There exists more than one LFS patch for wget. For Linux, Sun and HP-UX one can use http://software.lpetrov.net/wget-LFS/ Leonid
