Re: [Bug-wget] New to this, large files constraints?
- Original Message - From: Veronica Watson vjsorz...@gmail.com To: bug-wget@gnu.org Cc: Sent: Thursday, September 15, 2011 6:04 PM Subject: [Bug-wget] New to this, large files constraints? Hello, I am trying to download a data file, but am unsuccessful. I have downloaded other files just fine, but I have a feeling that this one is either too large or too complex. The website itself may have to do something before the file is generated. However, when I paste the link into firefox, the file downloads easily. What should I do? Here is the file: http://socds.huduser.org/permits/output_monthly_csv.odb?outpref=csvgeoval=statedatatype=monthlyFvarlist=1%232%233yearlist=2000%232001%232002%232003%232004%232005%232006%232007%232008%232009%232010statelist=13%2337%2345msalist=+cbsalist=+bppllist=+cntylist=13033%2313073%2313189%2313245%2337007%2337025%2337071%2337119%2337179%2345001%2345003%2345005%2345007%2345009%2345011%2345013%2345015%2345017%2345019%2345021%2345023%2345025%2345027%2345029%2345031%2345033%2345035%2345037%2345039%2345041%2345043%2345045%2345047%2345049%2345051%2345053%2345055%2345057%2345059%2345061%2345063%2345067%2345069%2345065%2345071%2345073%2345075%2345077%2345079%2345081%2345083%2345085%2345087%2345089%2345091COUNTYSUM=YESCOUNTYALL=+COUNTYGRP=+STATESUM=+STATEALL=+METROSUM=+METROALL=+METRO=+CBSA=+PLACEGRP=+CSUMNAME=JSUMNAME=+geo=statechron=monthlyF Thanks, Veronica W. On Windows you may see older versions of wget give the error message Result too large but it means filename too long. In Linux File name too long. And wget 1.13 --trust-server-names doesn't work with this site's response.. should it? What you can do is something like this, with your URL quoted wget -O BuildingPermits.csv http://yourhudurl; wget -O BuildingPermits.csv http://socds.huduser.org/permits/output_monthly_csv.odb?outpref=csvgeoval=statedatatype=monthlyFvarlist=1%232%233yearlist=2000%232001%232002%232003%232004%232005%232006%232007%232008%232009%232010statelist=13%2337%2345msalist=+cbsalist=+bppllist=+cntylist=13033%2313073%2313189%2313245%2337007%2337025%2337071%2337119%2337179%2345001%2345003%2345005%2345007%2345009%2345011%2345013%2345015%2345017%2345019%2345021%2345023%2345025%2345027%2345029%2345031%2345033%2345035%2345037%2345039%2345041%2345043%2345045%2345047%2345049%2345051%2345053%2345055%2345057%2345059%2345061%2345063%2345067%2345069%2345065%2345071%2345073%2345075%2345077%2345079%2345081%2345083%2345085%2345087%2345089%2345091COUNTYSUM=YESCOUNTYALL=+COUNTYGRP=+STATESUM=+STATEALL=+METROSUM=+METROALL=+METRO=+CBSA=+PLACEGRP=+CSUMNAME=JSUMNAME=+geo=statechron=monthlyF; ---response begin--- HTTP/1.1 200 OK Content-Type: application/vnd.ms-excel Server: Microsoft-IIS/6.0 Content-Disposition: attachment; filename=BuildingPermits.csv; X-Powered-By: ASP.NET Date: Sat, 17 Sep 2011 05:58:06 GMT Connection: close ---response end--- 200 OK Length: unspecified [application/vnd.ms-excel] output_monthly_csv.odb?outpref=csvgeoval=statedatatype=monthlyFvarlist=1#2#3yearlist=2000#2001#2002#2003#2004#2005#2006#2007#2008#2009#2010statelist=13#37#45msalist=+cbsalist=+bppllist=+cntylist=13033#13073#13189#13245#37007#37025#37071#37119#37179#45001#45003#45005#45007#45009#45011#45013#45015#45017#45019#45021#45023#45025#45027#45029#45031#45033#45035#45037#45039#45041#45043#45045#45047#45049#45051#45053#45055#45057#45059#45061#45063#45067#45069#45065#45071#45073#45075#45077#45079#45081#45083#45085#45087#45089#45091COUNTYSUM=YESCOUNTYALL=+COUNTYGRP=+STATESUM=+STATEALL=+METROSUM=+METROALL=+METRO=+CBSA=+PLACEGRP=+CSUMNAME=JSUMNAME=+geo=statechron=monthlyF: File name too long Closed fd 3
Re: [Bug-wget] New to this, large files constraints?
This is really an interesting problem: http://socds.huduser.org/permits/output_monthly_csv.odb?outpref=csvgeoval=statedatatype=monthlyFvarlist=1%232%233yearlist=2000%232001%232002%232003%232004%232005%232006%232007%232008%232009%232010statelist=13%2337%2345msalist=+cbsalist=+bppllist=+cntylist=13033%2313073%2313189%2313245%2337007%2337025%2337071%2337119%2337179%2345001%2345003%2345005%2345007%2345009%2345011%2345013%2345015%2345017%2345019%2345021%2345023%2345025%2345027%2345029%2345031%2345033%2345035%2345037%2345039%2345041%2345043%2345045%2345047%2345049%2345051%2345053%2345055%2345057%2345059%2345061%2345063%2345067%2345069%2345065%2345071%2345073%2345075%2345077%2345079%2345081%2345083%2345085%2345087%2345089%2345091COUNTYSUM=YESCOUNTYALL=+COUNTYGRP=+STATESUM=+STATEALL=+METROSUM=+METROALL=+METRO=+CBSA=+PLACEGRP=+CSUMNAME=JSUMNAME=+geo=statechron=monthlyF On Windows you may see older versions of wget give the error message Result too large but it means filename too long. In Linux File name too long. And wget 1.13 --trust-server-names doesn't work with this site's response.. should it? Well, in theory it should work with --content-disposition=on, as the webapplication sends a Content-Disposition header with a filename: ---response begin--- HTTP/1.1 200 OK Content-Type: application/vnd.ms-excel Server: Microsoft-IIS/6.0 Content-Disposition: attachment; filename=BuildingPermits.csv; X-Powered-By: ASP.NET Date: Sat, 17 Sep 2011 05:58:06 GMT Connection: close ---response end--- ... but wget seems to bail out with the overlong filename *before* it reads the response headers. Best regards, Jochen Roderburg
Re: [Bug-wget] New to this, large files constraints?
Zitat von Jochen Roderburg roderb...@uni-koeln.de: This is really an interesting problem: http://socds.huduser.org/permits/output_monthly_csv.odb?outpref=csvgeoval=statedatatype=monthlyFvarlist=1%232%233yearlist=2000%232001%232002%232003%232004%232005%232006%232007%232008%232009%232010statelist=13%2337%2345msalist=+cbsalist=+bppllist=+cntylist=13033%2313073%2313189%2313245%2337007%2337025%2337071%2337119%2337179%2345001%2345003%2345005%2345007%2345009%2345011%2345013%2345015%2345017%2345019%2345021%2345023%2345025%2345027%2345029%2345031%2345033%2345035%2345037%2345039%2345041%2345043%2345045%2345047%2345049%2345051%2345053%2345055%2345057%2345059%2345061%2345063%2345067%2345069%2345065%2345071%2345073%2345075%2345077%2345079%2345081%2345083%2345085%2345087%2345089%2345091COUNTYSUM=YESCOUNTYALL=+COUNTYGRP=+STATESUM=+STATEALL=+METROSUM=+METROALL=+METRO=+CBSA=+PLACEGRP=+CSUMNAME=JSUMNAME=+geo=statechron=monthlyF On Windows you may see older versions of wget give the error message Result too large but it means filename too long. In Linux File name too long. And wget 1.13 --trust-server-names doesn't work with this site's response.. should it? Well, in theory it should work with --content-disposition=on, as the webapplication sends a Content-Disposition header with a filename: ---response begin--- HTTP/1.1 200 OK Content-Type: application/vnd.ms-excel Server: Microsoft-IIS/6.0 Content-Disposition: attachment; filename=BuildingPermits.csv; X-Powered-By: ASP.NET Date: Sat, 17 Sep 2011 05:58:06 GMT Connection: close ---response end--- ... but wget seems to bail out with the overlong filename *before* it reads the response headers. After further examination I must retract the before assumption. Debug outputs show the GET response headers with Content-Disposition and the error message comes after it, so it looks more as if for some unknown reason the Content-Disposition is simply ignored. Best regards, Jochen Roderburg
[Bug-wget] GNU wget 1.13.4 released
Hello, I am pleased to announce the new version of GNU wget. It fixes some bugs reported in the recent wget 1.13.3 release. It is available for download here: ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.gz ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.xz and the GPG detached signatures using the key C03363F4: ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.gz.sig ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.xz.sig To reduce load on the main server, you can use this redirector service which automatically redirects you to a mirror: http://ftpmirror.gnu.org/wget/wget-1.13.4.tar.gz http://ftpmirror.gnu.org/wget/wget-1.13.4.tar.xz * Noteworthy changes in Wget 1.13.4 ** Now --version and --help work again. ** Fix a build error on solaris 10 sparc. ** Now --timestamping and --continue work well together. ** Return a network failure when FTP downloads fail and --timestamping is specified. Please report any problem you may experience to the bug-wget@gnu.org mailing list. Have fun! Giuseppe
Re: [Bug-wget] GNU wget 1.13.4 released
I'm not a programmer, but have enjoyed _using_ wget for many years. The latest Windows binary that I find posted is 1.11.4 of several years ago. Would someone please post a current one? (One of the links to Windows binaries yields a 404 error.) Alternatively, would someone please post [a link to, or put it in the documentation] a recipe (manual script) for a non-programmer to compile a Windows binary for himself using a readily-available, inexpensive (free?) compiler? I have done some Visual Basic and VBA programming and am familiar with the VB IDE and VBA used for Microsoft Office macros. Many thanks, Fred Holmes At 03:49 PM 9/17/2011, Giuseppe Scrivano wrote: Hello, I am pleased to announce the new version of GNU wget. It fixes some bugs reported in the recent wget 1.13.3 release. It is available for download here: ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.gz ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.xz and the GPG detached signatures using the key C03363F4: ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.gz.sig ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.xz.sig To reduce load on the main server, you can use this redirector service which automatically redirects you to a mirror: http://ftpmirror.gnu.org/wget/wget-1.13.4.tar.gz http://ftpmirror.gnu.org/wget/wget-1.13.4.tar.xz * Noteworthy changes in Wget 1.13.4 ** Now --version and --help work again. ** Fix a build error on solaris 10 sparc. ** Now --timestamping and --continue work well together. ** Return a network failure when FTP downloads fail and --timestamping is specified. Please report any problem you may experience to the bug-wget@gnu.org mailing list. Have fun! Giuseppe
Re: [Bug-wget] GNU wget 1.13.4 released
From: Fred Holmes f...@his.com I'm not a programmer, but have enjoyed _using_ wget for many years. The latest Windows binary that I find posted is 1.11.4 of several years ago. Would someone please post a current one? (One of the links to Windows binaries yields a 404 error.) Alternatively, would someone please post [a link to, or put it in the documentation] a recipe (manual script) for a non-programmer to compile a Windows binary for himself using a readily-available, inexpensive (free?) compiler? I have done some Visual Basic and VBA programming and am familiar with the VB IDE and VBA used for Microsoft Office macros. Many thanks, Fred Holmes Ray Satiro posted this on 8/24. It isn't the latest but it works well. http://sourceforge.net/projects/getgnuwin32/files/getgnuwin32/test%20builds/wget-1.13.1.patched.zip/download I'm sure a Windows v1.13.4 will get posted in due time. -- Dave Multi-AV Scanning Tool - http://multi-av.thespykiller.co.uk http://www.pctipp.ch/downloads/dl/35905.asp
Re: [Bug-wget] GNU wget 1.13.4 released
From: Voytek voy...@sbt.net.au On Sun, September 18, 2011 10:20 am, David H. Lipman wrote: From: Fred Holmes f...@his.com Ray Satiro posted this on 8/24. It isn't the latest but it works well. http://sourceforge.net/projects/getgnuwin32/files/getgnuwin32/test%20buil ds/wget-1.13.1.patched.zip/download I'm sure a Windows v1.13.4 will get posted in due time. dumb question, what's the best or proper way to 'integrate' this into a windoze XP, so it's a path for execution and obeys any options in etc/wgetrc ? There is no best practice. I drope the EXE and required DLLs in %windir% (so it is in the PATH) and create a; C:\wgetrc.txt file and set the WGETRC environemntal variable to point to it. WGETRC=C:\wgetrc.txt -- Dave Multi-AV Scanning Tool - http://multi-av.thespykiller.co.uk http://www.pctipp.ch/downloads/dl/35905.asp
Re: [Bug-wget] GNU wget 1.13.4 released
I haven't done it yet, but, if it works like the earlier versions: Put the following files in some convenient folder that is in the PATH, e.g. C:\windows\system32\ wget.exe ssleay32.dll libeay32.dll msvcr71.dll msvcr80.dll You may not need the bottom two files. They may already be there. I've never used .wgetrc. I've always just put wget.exe in a batch file with the switches I want. I have a template batch file that contains all my default switches, and a whole bunch of comment (help) lines listing the switches I frequently use. But I presume that one writes a .wgetrc file and places it in the same place as the other files. Fred Holmes At 08:39 PM 9/17/2011, Voytek wrote: dumb question, what's the best or proper way to 'integrate' this into a windoze XP, so it's a path for execution and obeys any options in etc/wgetrc ?
