AW: AW: Problem mirroring a site using ftp over proxy
I just noticed these debug messages: ** DEBUG output created by Wget 1.10.2 on cygwin. --15:49:45-- ftp://ftpde.nai.com/CommonUpdater/ = `ftpde.nai.com/CommonUpdater/index.html' Resolving * 192.168.182.76 Caching * = 192.168.182.76 Connecting to *|192.168.182.76|:8080... connected. Created socket 4. Releasing 0x006a1390 (new refcount 1). ---request begin--- GET ftp://ftpde.nai.com/CommonUpdater/ HTTP/1.0 User-Agent: Wget/1.10.2 Accept: */* Host: ftpde.nai.com ---request end--- Proxy request sent, awaiting response... ---response begin--- HTTP/1.1 200 OK Server: Sun-Java-System-Web-Proxy-Server/4.0 Date: Fri, 08 Aug 2008 13:49:46 GMT Content-type: text/html Proxy-agent: Sun-Java-System-Web-Proxy-Server/4.0 Via: 1.1 proxy-internet Connection: close ---response end--- 200 OK Length: unspecified [text/html] 0K 7.11 MB/s Closed fd 4 Last-modified header missing -- time-stamps turned off. 15:49:47 (7.11 MB/s) - `ftpde.nai.com/CommonUpdater/index.html' saved [8786] FINISHED --15:49:47-- Downloaded: 8,786 bytes in 1 files ** The point is that wget sends rather a http request than a pure ftp command (GET ftp://ftpde.nai.com/CommonUpdater/ HTTP/1.0) which causes the proxy to send back a index.html. Do u agree? -Ursprüngliche Nachricht- Von: Micah Cowan [mailto:[EMAIL PROTECTED] Gesendet: Donnerstag, 7. August 2008 23:23 An: Juon, Stefan Cc: wget@sunsite.dk Betreff: Re: AW: Problem mirroring a site using ftp over proxy -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Well, considering that FTP proxied over HTTP is working fine for me, it's probably more a matter of the index.html file that's generated by the proxy (since one can't do a true LIST over a proxy). Perhaps you could supply the index.html files that are being generated (be sure to clean out any sensitive info first). It might also be informative to know what server program is doing the proxying. - -Micah Juon, Stefan wrote: ...problem exists also with version 1.11.4. So what might cause wget not to download the files as it has performed a LIST? Thanks, Stefan Juon, Stefan wrote: Hi there I'm trying to mirror a ftp site over a proxy (Sun Java Webproxy 4.0.4) using this wget-command: export ftp_proxy=http://proxy.company.com:8080 wget --follow-ftp --passive-ftp --proxy=on --mirror --output-file=./logfile.wget ftp://ftpde.nai.com/CommonUpdater What version of Wget are you running? If it's not the latest, please try the current 1.11.4 release. Please also try the --debug option, to see if Wget gives you more information. - -- Micah J. Cowan Programmer, musician, typesetting enthusiast, gamer. GNU Maintainer: wget, screen, teseq http://micah.cowan.name/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFIm2fF7M8hyUobTrERAv/BAJ9biwIIUFaIWZ9Ds7IZxiGAKriA7wCeJtn1 lYdaP8hzodianPg1Bp6b6gk= =+HQo -END PGP SIGNATURE-
Re: AW: AW: Problem mirroring a site using ftp over proxy
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Juon, Stefan wrote: The point is that wget sends rather a http request than a pure ftp command (GET ftp://ftpde.nai.com/CommonUpdater/ HTTP/1.0) which causes the proxy to send back a index.html. Do u agree? Well of course it does: it's using an HTTP proxy. How do you send FTP commands over HTTP? The problem isn't that the result is an HTML file; the problem is that the proxy sends an HTML file that Wget apparently can't parse. Perhaps the proxy's not really sending an HTML file at all, which would be unusual (but I'm not sure there are standards governing how FTP gets proxied across HTTP), in which case Wget would need to be modified to check whether the proxied results are a listing file. But until you show us what index.html file Wget is getting, I don't see how we can help. - -- Micah J. Cowan Programmer, musician, typesetting enthusiast, gamer. GNU Maintainer: wget, screen, teseq http://micah.cowan.name/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFInJpC7M8hyUobTrERAhGtAJ9/cY3nJk8xf1oWb+KCH8mQ54nXNACgg/is xD3eHrajIfnUDaRhnFI+X+s= =g1QP -END PGP SIGNATURE-
[PATCH] 1.11.4: Add missing $(datarootdir)
Hello, Here is a change that adds $(datarootdir) throughout that has been missed despite the prominent warning output by ./configure. :-( 2008-08-09 Maciej W. Rozycki [EMAIL PROTECTED] * Makefile.in (datarootdir): Add definition. * doc/Makefile.in (datarootdir): Likewise. * src/Makefile.in (datarootdir): Likewise. * tests/Makefile.in (datarootdir): Likewise. Please apply. Maciej wget-1.11.4-datarootdir.patch diff -up --recursive --new-file wget-1.11.4.macro/Makefile.in wget-1.11.4/Makefile.in --- wget-1.11.4.macro/Makefile.in 2008-05-29 03:29:54.0 + +++ wget-1.11.4/Makefile.in 2008-08-08 22:29:08.0 + @@ -47,6 +47,7 @@ VPATH = @srcdir@ # prefix = @prefix@ exec_prefix = @exec_prefix@ +datarootdir = @datarootdir@ bindir = @bindir@ infodir = @infodir@ sysconfdir = @sysconfdir@ diff -up --recursive --new-file wget-1.11.4.macro/doc/Makefile.in wget-1.11.4/doc/Makefile.in --- wget-1.11.4.macro/doc/Makefile.in 2008-04-27 04:48:19.0 + +++ wget-1.11.4/doc/Makefile.in 2008-08-08 22:31:03.0 + @@ -37,6 +37,7 @@ srcdir = @srcdir@ VPATH = @srcdir@ prefix = @prefix@ +datarootdir= @datarootdir@ infodir= @infodir@ mandir = @mandir@ manext = 1 diff -up --recursive --new-file wget-1.11.4.macro/src/Makefile.in wget-1.11.4/src/Makefile.in --- wget-1.11.4.macro/src/Makefile.in 2008-04-27 04:48:21.0 + +++ wget-1.11.4/src/Makefile.in 2008-08-08 22:30:29.0 + @@ -40,6 +40,7 @@ VPATH = @srcdir@ prefix = @prefix@ exec_prefix = @exec_prefix@ +datarootdir = @datarootdir@ bindir = @bindir@ sysconfdir = @sysconfdir@ datadir = @datadir@ diff -up --recursive --new-file wget-1.11.4.macro/tests/Makefile.in wget-1.11.4/tests/Makefile.in --- wget-1.11.4.macro/tests/Makefile.in 2008-04-27 04:48:24.0 + +++ wget-1.11.4/tests/Makefile.in 2008-08-08 22:31:11.0 + @@ -40,6 +40,7 @@ VPATH = @srcdir@ prefix = @prefix@ exec_prefix = @exec_prefix@ +datarootdir = @datarootdir@ bindir = @bindir@ sysconfdir = @sysconfdir@ datadir = @datadir@
Re: [PATCH] 1.11.4: Add missing $(datarootdir)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Maciej W. Rozycki wrote: Hello, Here is a change that adds $(datarootdir) throughout that has been missed despite the prominent warning output by ./configure. :-( 2008-08-09 Maciej W. Rozycki [EMAIL PROTECTED] * Makefile.in (datarootdir): Add definition. * doc/Makefile.in (datarootdir): Likewise. * src/Makefile.in (datarootdir): Likewise. * tests/Makefile.in (datarootdir): Likewise. Please apply. Maciej Hi Maciej, We're not anticipating any further 1.11.x releases for Wget. Active development for most of the last year has focused on 1.12, which is based on Automake (so we get datarootdir for free). But if any significant bugs are found in 1.11.4 that warrant a new 1.11.x release, we'll add this patch in. - -- Micah J. Cowan Programmer, musician, typesetting enthusiast, gamer. GNU Maintainer: wget, screen, teseq http://micah.cowan.name/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFInM4p7M8hyUobTrERAm6zAJ4sLyVEIkq/VVQ2XKylIKPDrNewSwCfUsIH rFK6XiRKYgVo/yZiU8Nf2iI= =gLU4 -END PGP SIGNATURE-
Re: [PATCH] 1.11.4: Add missing $(datarootdir)
Hi Micah, We're not anticipating any further 1.11.x releases for Wget. Active development for most of the last year has focused on 1.12, which is based on Automake (so we get datarootdir for free). Hmm, I must have missed that bit, but if I said I had been following the discussion at the wget mailing list, that would be an overstatement. ;) But if any significant bugs are found in 1.11.4 that warrant a new 1.11.x release, we'll add this patch in. No problem -- I just thought I would share a fix for what I discovered. Feel free to use it however you like. Maciej
