Re: Wget and Yahoo login?
On Mon, 8 Sep 2008, Donald Allen wrote: The page I get is what would be obtained if an un-logged-in user went to the specified url. Opening that same url in Firefox *does* correctly indicate that it is logged in as me and reflects my customizations. First, LiveHTTPHeaders is the Firefox plugin everyone who tries these stunts need. Then you read the capure and replay them as closely as possible using your tool. As you will find out, sites like this use all sorts of funny tricks to figure out you and to make it hard to automate what you're trying to do. They tend to use javascripts for redirects and for fiddling with cookies just to make sure you have a javascript and cookie enabled browser. So you need to work hard(er) when trying this with non-browsers. It's certainly still possible, even without using the browser to get the first cookie file. But it may take some effort. -- / daniel.haxx.se
Re: Connection management and pipelined Wget
On Thu, 7 Aug 2008, Micah Cowan wrote: "niwt" (which I like best so far: Nifty Integrated Web Tools). But the grand question is: how would that be pronounced? Like newt? :-) -- / daniel.haxx.se
Re: About Automated Unit Test for Wget
On Sat, 5 Apr 2008, Hrvoje Niksic wrote: This would mean we'd need to separate uses of read() and write() on normal files (which should continue to use the real calls, until we replace them with the file I/O abstractions), from uses of read(), write(), etc on sockets, which would be using our emulated versions. Unless you're willing to spend a lot of time in careful design of these abstractions, I think this is a mistake. Related: In the curl project we took a simpler route: we have our own dumb test servers in the test suite to run tests against and we have single files that describe each test case: what the server should respond, what the protocol dump should look like, what output to expect, what return code, etc. Then we have a script that reads the test case description, fires up the correct server(s), verifies all the ouputs (optionally using valgrind). This system allows us to write unit-tests if we'd like to, but mostly so far we've focused to test it system-wide. It is hard enough for us!
Re: About Automated Unit Test for Wget
On Sat, 5 Apr 2008, Micah Cowan wrote: Or did you mean to write wget version of socket interface? i.e. to write our version of socket, connect,write,read,close,bind, listen,accept,,,? sorry I'm confused. Yes! That's what I meant. (Except, we don't need listen, accept; and we only need bind to support --bind-address. We're a client, not a server. ;) ) Except, you do need listen, accept and bind in a server sense since even if wget is a client I believe it still supports the PORT command for ftp...
Re: wget2
On Thu, 29 Nov 2007, Alan Thomas wrote: Sorry for the misunderstanding. Honestly, Java would be a great language for what wget does. Perhaps, but not for where wget is used: on numerous platforms as a stand-alone downloadable tool, including on embedded and small-CPU devices. Environments where java doesn't excel. Lots of built-in support for web stuff. That's simply not true. It has some _basic_ support for some "web" protocols etc, but that's far from enough for a tool such as wget.
Re: Wget Name Suggestions
On Thu, 29 Nov 2007, Josh Williams wrote: I really like the name `fetch` because it does what it says it does. It's more UNIX-like than the other names :-) While I agree that a "unix-like" name is preferable, I just want to point out that 'fetch' is already used by a http/ftp transfer tool that's bundled with FreeBSD, so that would be a really bad choice...
Re: Thoughts on Wget 1.x, 2.0 (*LONG!*)
On Fri, 26 Oct 2007, Micah Cowan wrote: The obvious solution to that is to use c-ares, which does exactly that: handle DNS queries asynchronously. Actually, I didn't know this until just now, but c-ares was split off from ares to meet the needs of the curl developers. :) We needed an asynch name resolver for libcurl so c-ares started out that way, but perhaps mostly because the original author didn't care much for our improvements and bug fixes. ADNS is a known alternative, but we couldn't use that due to license restrictions. You (wget) don't have that same problem with it. I'm not able to compare them though, as I never used ADNS... Of course, if we're doing asynchronous net I/O stuff, rather than reinvent the wheel and try to maintain portability for new stuff, we're better off using a prepackaged deal, if one exists. Luckily, one does; a friend of mine (William Ahern) wrote a package called libevnet that handles all of that; When I made libcurl grok a vast number of simultaneous connections, I went straight with libevent for my test and example code. It's solid and fairly easy to use... Perhaps libevnet makes it even easier, I don't know. Plus, there is the following thought. While I've talked about not reinventing the wheel, using existing packages to save us the trouble of having to maintain portable async code, higher-level buffered-IO and network comm code, etc, I've been neglecting one more package choice. There is, after all, already a Free Software package that goes beyond handling asynchronous network operations, to specifically handle asynchronous _web_ operations; I'm speaking, of course, of libcurl. I guess I'm not the man to ask nor comment this a lot, but look what I found: http://www.mail-archive.com/wget@sunsite.dk/msg01129.html I've always thought and I still believe that wget's power and most appreciated abilities are in the features it adds on top of the transfer, like HTML parsing, ftp list parsing and the other things you mentioned. Of course, going one single unified transfer library is perhaps not the best thing from a software eco-system perspective, as competition tends to drive innovation and development, but the more users of a free software/open source project we get the better it will become.
Re: More portability stuff [Re: gettext configuration]
On Fri, 26 Oct 2007, Micah Cowan wrote: I very much doubt it does, since we check for it in the curl configure script, and I can see the output from it running on Tru64 clearly state: checking for sigsetjmp... yes Note that curl provides the additional check for a macro version in the configure script, rather than in the source; we should probably do it that way as well. I'm not sure how that helps for this, though: if the above test is failing, then either it's a function (no macro) and configure isn't picking it up; or else it's not defined in . Yes right, I had forgot about that! But in the Tru64 case the extra macro check isn't necessary. I don't remember exactly what system that needs it, but I believe it was some older linux or bsd.
Re: More portability stuff [Re: gettext configuration]
On Sat, 27 Oct 2007, Hrvoje Niksic wrote: Do you say that Tru64 lacks both sigsetjmp and siggetmask? Are you sure about that? That is the only system we are currently talking about. I find it hard to believe that Tru64 lacks both of those functions; for example, see http://h30097.www3.hp.com/docs/base_doc/DOCUMENTATION/V51_HTML/MAN/MAN3/0707.HTM It is quite possible that the Autoconf test for sigsetjmp yields a false negative. I very much doubt it does, since we check for it in the curl configure script, and I can see the output from it running on Tru64 clearly state: checking for sigsetjmp... yes (available here for another ten days or so: http://curl.haxx.se/auto/log.cgi?id=20071026080956-25212)
Re: wget 1.10.2 doesn't compile on NetBSD/i386 3.1
On Wed, 10 Oct 2007, Micah Cowan wrote: It appears from your description that Wget's check in http-ntlm.c: #if OPENSSL_VERSION_NUMBER < 0x00907001L is wrong. Your copy of openssl seems to be issuing a number lower than that, and yet has the newer, capitalized names. I don't think that check is wrong. We have the exact same check in libcurl (no suprise) and it has worked fine since I wrote it, and I've tested it myself on numerous different openssl versions. I would rather suspect that the problem is related to multiple openssl installations or similar.
scan.coverity.com
Hey Just to let you know, it seems that coverity.com scans/scanned wget as part of their scan project, and I belive a wget person could get a signin and get to see the details from that: http://scan.coverity.com/rung0.html We got curl added as well and coverity did find a range of rather non-trivial and non-obvious flaws in the code that we could fix. (I have no relation to these guys other than that I'm enjoying being part of a(nother) project that they scan for us.)
ohloh
Hi guys, ohloh.net keeps track of FLOSS authors and projects and do some interesting stats and numbers. Wget is listed too: http://www.ohloh.net/projects/7947?p=Wget (No I'm not involved with the site in any way but as a happy visitor and registered user.)
Re: FTP OS-dependence, and new FTP RFC
On Sat, 4 Aug 2007, Micah Cowan wrote: Just a word of caution here: while RFC1738 tells this is the way to do it, there are servers and times where this approach doesn't work. (lib)curl has an option to specify the CWD style (multiple cwd, single cwd or no cwd) due to this... Could you be more specific about how it fails to work on some systems? I understand (from comments in wget) that OS/400 doesn't support hierarchies: but then no legitimate OS/400 FTP URL would include more than one dir component in its path. Sorry, but the exact details have escaped so I can't provide them right now. I'm sure someone will remind us later on if you change wget to multi-CWD ;-) I remember one situation has been that the server doesn't allow you to CWD into the dir, but allows you to CWD into the dir's subdir! So that "CWD dir1" fails, but "CWD dir1/dir2" works Given that probably most other user-agents follow the RFC in this regard Why do think that? I remember back when we made curl do this (April 2003), a guy named David Balazic posted an RFC1738 compliance summary where he tested a range of FTP clients and as I recall it very few of them used the multi-CWD way. Of course time has since passed so the situation may have changed, but I believe many still use the single CWD or even no CWD approach. Unfortunately that summary URL is now a 404... I'm having a little problem seeing how an FTP server that can't handle this could survive, particularly since, from the beginning of time, it's been the only way to reliably specify the directory path one wants. Reliably for multiple servers, no. But reliable for a specific server that you're using, sure. Technically, we can violate the RFCs (specifically, RFC 1738, which defines the ftp URL scheme in terms of a string of CWDs from the starting directory), if the FTP server starts us out in a directory other than / The particular login directory doesn't affect the CWD approach at all, since the URL gives the path relative to the login directory... Do you mean that wget tacks the path onto the value it gets from PWD, then? No I don't, I don't know how wget creates the dir it sends with CWD. In the quote above you wrote that "we can violate" ... "*if* the FTP server starts us out in a directory other than /" (my emphasis), and I just mentioned that you can do the exact same work no matter what the login directory is.
Re: FTP OS-dependence, and new FTP RFC
On Fri, 3 Aug 2007, Micah Cowan wrote: I have a question: why do we attempt to generate absolute paths and such and CWD to those, instead of just doing the portable string-of-CWDs to get where we need to be? Just a word of caution here: while RFC1738 tells this is the way to do it, there are servers and times where this approach doesn't work. (lib)curl has an option to specify the CWD style (multiple cwd, single cwd or no cwd) due to this... Technically, we can violate the RFCs (specifically, RFC 1738, which defines the ftp URL scheme in terms of a string of CWDs from the starting directory), if the FTP server starts us out in a directory other than / The particular login directory doesn't affect the CWD approach at all, since the URL gives the path relative to the login directory...
Re: ignoring robots.txt
On Wed, 18 Jul 2007, Micah Cowan wrote: The manpage doesn't need to give as detailed explanations as the info manual (though, as it's auto-generated from the info manual, this could be hard to avoid); but it should fully describe essential features. I know GNU projects for some reason go with info, but I'm not in fan of that. Personally I always just use man pages and only revert to using info pages when forced. I simply don't like it when projects "hide" information in info pages.
Re: Subversion commit notifications
On Fri, 6 Jul 2007, Micah Cowan wrote: I forgot to mention that the wget-notify list is writable, as it's also intended to permit discussion of development as it is happening. So the svn notify list is effectively turning into a devel list?
Re: post-file does not work with post-data
On Fri, 29 Jun 2007, Micah Cowan wrote: For submitting actual files as form content, multipart/form-data is a much more natural mechanism. [...] Obviously, while this is something wget does not currently do, it is something wget ought to do. I'll look into how we might implement this in a future version. Allow me to mention that (lib)curl supports this and this could serve as inspiration for a future wget implementation.
Re: Basic auth by default
On Thu, 28 Jun 2007, Hrvoje Niksic wrote: It's easy to bring back the code itself, but it's not easy to integrate it with how Wget communicates with proxies, at least not without reworking a large chunk of HTTP code. That is why I started with support for simple client NTLM and postponed proxies for "later". Right, thanks for setting that straight.
RE: Basic auth by default
On Wed, 27 Jun 2007, Barnett, Rodney wrote: I agree. I discovered this when trying to use wget with an HTTP proxy that uses NTLM. (Is that on the list somewhere?) I'm pretty sure the original NTLM code I contributed to wget _had_ the ability to deal with proxies (as I wrote the support for both host and proxy at the same time). It should be fairly easy to bring back.
Re: New wget maintainer
On Tue, 26 Jun 2007, Micah Cowan wrote: The GNU Project has appointed me as the new maintainer for wget Welcome! Speaking of licensing changes, I don't see a specific exemption clause for linking wget with OpenSSL See the end of the README.
Re: Can Nagios/Lynx/wget/curl/links do NTLM authentication?
On Fri, 8 Jun 2007, Kelly Jones wrote: I want to use Nagios to monitor a site (running on Windows/IIS) that uses NTLM for authentication. Is there a plugin/script/library/etc that can help? Reason I'm cc'ing the lynx/wget/curl/links lists: if lynx/wget/curl/links can do NTLM, I can easily write a nagios wrapper. Both curl and wget can, curl is slightly better at it though. I can't speak for the others since I don't know. -- Commercial curl and libcurl Technical Support: http://haxx.se/curl.html
Re: Requests are always HTTP/1.0 ?!
On Tue, 1 May 2007, Greg Lindahl wrote: Host: kpic1 is a HTTP/1.1 feature. So this is non-sensical. Some pre-1.1 servers have required this header, I don't see how the 1.0 spec forbids it and by using it you can utilize name-based virtual hosting so I disagree with your conclusion.
RE: NTLM for proxy authentication
On Tue, 20 Feb 2007, Barnett, Rodney wrote: I'm referring to *proxy authentication*. I'm no expert on wget, but... According to the documentation: "For proxy authorization only the Basic authentication scheme is currently implemented." Oh, sorry my bad. I submitted the NTLM code and it was written to support proxies as well (and do so just fine in curl). It seems it was never put to use for proxies in wget.
Re: NTLM for proxy authentication
On Tue, 20 Feb 2007, Barnett, Rodney wrote: Is anyone working on NTLM proxy authentication for wget? If not, are there any major obstacles? NTLM support has been in wget for several years by now.
Re: Parallel downloads provide significant speedup
On Sat, 30 Sep 2006, Anthony L. Bryan wrote: Multithreaded downloads can increase speed quite a bit. I don't think anyone has argued about that downloading from several servers at once will be faster in many occasions (like when each server gives you less bandwidth for the transfer than what you have as download link bandwidth). I think what is debatable is wheather to download the same file off the same server using several connects. I believe the HTTP spec says that a "nice" HTTP client connect no more than twice to the same server. And twice is then only due to (the effects of) pipelining on the first connection and the second would be for getting "out of band" data.
Re: wget breaks round-robin dns?
On Sun, 23 Jul 2006, Hrvoje Niksic wrote: If it works without IPv6, then it means that getaddrinfo on your machine sorts the results, whereas gethostbyname doesn't. This effect has been seen (and reported) before on getaddrinfo vs gethostbyname and I've tried to get the glibc guys interested enough to answer why this is but have failed so far. I can easily repeat this (bad) behaviour using a stand-alone test program using a very recent glibc, so this is in no way wget's fault. If anyone would like to try the tests or dive further into this, my test program can be found here: http://curl.haxx.se/mail/lib-2005-11/0008.html -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([s\x]\)\([\xoi]\)xo un\2\1 is xg'`ol
Re: wget BUG: ftp file retrieval
On Fri, 25 Nov 2005, Steven M. Schweda wrote: Or, better yet, _DO_ forget to prepend the trouble-causing $CWD to those paths. I agree. What good would prepending do? It will most definately add problems such as those Steven describes. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: SSL certificate difficulties
On Tue, 1 Nov 2005, Hrvoje Niksic wrote: If nothing else works, grab the CA bundle from http://curl.haxx.se/docs/sslcerts.html I believe you meant to point out this page: http://curl.haxx.se/docs/caextract.html ... which provides the Mozilla/Firefox CA cert bundle in PEM format, suitable to use with wget, curl or others. (Updated frequently) -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Making wget reusing an ftp connection
On Tue, 1 Nov 2005, Oliver Schulze L. wrote: I sugested this aproach because it will be far more easy to add a new option for enabling the "reuse ftp connection" feature and groups all URLs from the same host, than implementing a full cache system. The logic for "grouping" the URL per host (and user+password) is just about as advanced as the logic for matching hosts in a connection cache. The difference is mainly that "grouping" effects in what order the downloading is done, while a cache keeps the order but risks getting the connection closed while the middle transfer is done (in the case host1 host2 host1 where the host1 transfer is very slow). Without knowing much about wget internals, one could imagine the same concept being used for HTTP persistent connections, downloading wildcard FTP files as well as multiple URLs specified on the command line. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: brief report on NTLM buffer overflow
On Fri, 14 Oct 2005, Noèl Köthe wrote:
The last paragraph says something like: "Notable is the fast time of
reaction of the Open Source developer: two days ago the problem was
reported, yesterday corrected packages were produced and details of the
vulnerability were published."
Just want to give you very possitive feedback and say thanks.:)
I mailed Hrvoje the patch roughly 50 minutes after the notification (and he
forwarded it to Mauro). Wget 1.10.2 was relased less than 14 hours after the
first mail brought our attention to this problem.
Both Hrvoje and Mauro acted swiftly and promptly. Nice going guys!
(The plan was originally to coordinate the security fix release with vendors
and between the curl and wget projects, but due to mistakes did the
notification accidentally become public immediately and we had to work really
fast to reduce the impact.)
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: wget 1.10.1 released
On Wed, 21 Sep 2005, Steven M. Schweda wrote: I'll admit that it'd be tedious for the program(s), but do you really think it'd be slow? If you access a (remote) site with very long round-trip delays the speed difference is at least noticable. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: wget 1.10.1 released
On Mon, 19 Sep 2005, Steven M. Schweda wrote: Interestingly, I've found yet another VMS FTP server variety which seems to require a change to the CWD code so that instead of doing "CWD abc/def/ghi" (as now), it would need to do "CWD abc", "CWD def", and "CWD ghi", as the RFC suggests. In project [beep] we took this route ages ago because of these two reasons: 1) the RFC says this is how to do it 2) some servers require it The challange that we have still to fix is to write code that would first attempt the full-path approach only to downgrade to the single-path-part approach on failure of the first. This would be for speed reasons of course, since CWDing into deep dir hierarchies get really slow with that lng series of CWD sending! ;-) -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: openssl server renogiation bug in wget
On Fri, 26 Aug 2005, Hrvoje Niksic wrote: + /* The OpenSSL library can handle renegotiations automatically, so + tell it to do so. */ + SSL_CTX_set_mode (ssl_ctx, SSL_MODE_AUTO_RETRY); + Just wanted to make sure that you are aware that this option is only available in OpenSSL 0.9.6 or later? I don't remember what oldest OpenSSL version you want to support... -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: wget Mailing List question
On Fri, 26 Aug 2005, Jonathan wrote:
Would it be possible (and is anyone else interested) to have the subject
line of messages posted to this list prefixed with '[wget]'?
Please don't. Subject real estate is precious and limited already is it is. I
find subject prefixes highly distdurbing.
There is already plenty of info in the headers of each single mail to allow
them to get filtered accurately without this being needed.
For example: "X-Mailing-List: wget@sunsite.dk"
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Issue tracker
On Tue, 23 Aug 2005, Hrvoje Niksic wrote: Would someone be willing to host an issue tracker for Wget? Doesn't http://savannah.gnu.org/ or similar provide such services that are sufficiently good? -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Controlling OpenSSL's use of read() and write()
On Wed, 25 May 2005, Hrvoje Niksic wrote: In SSL it is not enough to select() before SSL_read because SSL_read can and does call read multiple times, which breaks the intended timeout semantics. Is there a way to prevent this? I figure one way is to use non-blocking sockets, isn't it? -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: SSL option documentation
On Thu, 12 May 2005, Hrvoje Niksic wrote: (On my Debian installation the certificates come with the "ca-certificates" package and are apparently assembled from different sources, the most significant being Mozilla. On SuSE 9.2 the CA certificates come with the "openssl" package.) There are no license restrictions that prevent you from using/bundling/include the Mozilla one (if you want to). I have a little service up and running for those who wants the latest Mozilla ca cert bundle in PEM format: http://curl.haxx.se/docs/caextract.html The Debian wget packager will of coure be encouraged to make wget use the already installed cacert file. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: SSL code audit
On Tue, 10 May 2005, Hrvoje Niksic wrote: curl contains much more elaborate code in ssluse.c:verifyhost(). Although I'm not sure Wget requires an exact replica of curl's logic, *some* check seems necessary for, especially since we claim to verify the server's certificate by default. It does require a replica, exact or not. If you verify a server certificate, you must make sure the commonName field matches the host name you communicate with (inluding wildcards). Then add the subjectAltName fields to the mix and you get a fair amount of code to write. I believe RFC2818 details this. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: CONNECT and the Host header
On Thu, 28 Apr 2005, Hrvoje Niksic wrote: The original CONNECT proposal by Luotonen didn't mention `Host' at all. curl doesn't send it curl has sent CONNECT for many years without it, and nobody has ever reported a problem with it. That doesn't however mean it shouldn't be there to adhere to the protocol strictly of course. I have a pending patch that adds it, but I haven't yet decided if it is worth adding or not... -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: SSL option documentation
On Sun, 24 Apr 2005, Hrvoje Niksic wrote: Since you seem to be knowledgable about SSL implementation(s), what do you think about GNU TLS? Is its development active? How hard would it be to use it in Wget? I'm not Doug, but since I just recently made libcurl capable of using GnuTLS (as an option instead of OpenSSL) I think can fill in some info: GnuTLS is alive and it is working. It is even documented somewhat better than OpenSSL (which isn't saying a lot, I know). Converting an OpenSSL-using program into using GnuTLS instead isn't very hard. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: cygwin wget ssl
On Thu, 21 Apr 2005, Konrad Chan wrote: After browsing the openssl newsgroup per Hrvoje's suggestion, I came to a similar conclusion as well (cipher problem). However, I couldn't find instructions on how to change the cipher for wget, I tried all the different ciphers using curl but no luck, since "openssl ciphers" only had SSLv3. Let me repeat myself: "If you force it to do SSLv2 it works fine." wget --sslprotocol=1 -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: cygwin wget ssl
On Wed, 20 Apr 2005, Hrvoje Niksic wrote: It seems this site is sending something that the OpenSSL library cannot handle. For example, both Wget and curl on Linux display the same error: $ curl https://www.danier.com curl: (35) error:1408F455:SSL routines:SSL3_GET_RECORD:decryption failed or bad record mac Maybe this should be reported to the OpenSSL maintainers? If you force it to do SSLv2 it works fine. At times, old and bad SSL implementations make it hard for OpenSSL to autodetect version. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: NTLM authentication in CVS
On Thu, 7 Apr 2005, Hrvoje Niksic wrote: If someone has access to an NTLM-authorizing web server, please try it out with Wget and let us know how it goes. It should be noted that for NTLM, you can specify the domain name in the user name field as: "domain\user" or "domain/user" -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: NTLM test server
On Mon, 4 Apr 2005, Hrvoje Niksic wrote: Is there a test server where one can try out NTLM authentication? I'm working on adapting Daniel's code to Wget, and having a test server would be of great help. Just for your info: I had friends providing the test servers for both host and proxy authentication when I've worked on NTLM code. Once the basics worked, I could add test cases to the curl test suite. Nowadays I can test NTLM on my own with the curl test server. I'm afraid it is too specific for curl to be useful for wget. Also, since the day I provided that code, I've leared a few additional things: o I had a buffer security problem in the NTLM code, but related to base64 decode function and that is bound to be different when you adapt the code to wget conditions anwyay. o There was also another less alarming buffer problem with a memset() of 8 bytes instead of 5. You may have of course already have found and fixed this. o POSTing with NTLM auth is a pain, since NTLM is for connections and thus you cannot close the connection without breaking the auth so thus you are (more likely than with other multi-pass auth methods) forced to send the full request body multiple times. And probably a little more that I've forgotten to mention now. ;-) -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
RE: help!!!
On Mon, 21 Mar 2005, Richard Emanilov wrote: Thanks for your response, can you please for an example of how the string should look? Below is what I have tried with no success wget --http-user=login --http-passwd=passwd --http-post="login=login&password=passwd" https://siteIamTryingToDownLoad.html This looks just fine to me. There are however a wide range of possible reasons why this can still fail, and since you don't tell us more than "with no success" it is impossible for anyone to tell. My advice: learn how to use LiveHTTPHeaders when doing the operation with your browser, and then use that info when repeating the operation with wget or other tools. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: help!!!
On Mon, 21 Mar 2005, Richard Emanilov wrote: Is there a way I can concatenate both challenges into one string when trying to d/l the page? No, you should use both options. And be prepared that it'll respond with a set of headers that set one or more cookies and then redirect you to another page. is list receiving this email? Yes it does. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: string_t breaks compatibility?
On Sat, 19 Feb 2005, Hrvoje Niksic wrote: I noticed the addition of a `string_t.c' file that uses wchar_t and wide string literals without any protection. This will break compilation of Wget under older compilers. I believe this has already been discussed and Mauro said he's working on fixing that. "i will fix it ASAP" (Jan 12 2005) -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: post-file with PHP back end?
On Tue, 15 Feb 2005, Patrick H. Madden wrote:
Here's a subset of the HTML that works with the PHP back end. The actual
HTML page is at <http://vlsicad.cs.binghamton.edu/placebench>. This is for
uploading circuit layout information, and benchmarking.
This form expects the user-agent to provide the POST data MIME-encoded as
RFC1867 describes. AFAIK, wget does not provide that feature.
If you want to use wget for this, you need to produce a RFC1867-formatted file
first and then post that (you'd also need to provide a custom header that
specifies the boundary separator string).
Another option is to use a tool that already has support for what you're
asking.
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: post-file with PHP back end?
On Tue, 15 Feb 2005, Patrick H. Madden wrote:
Hi -- I'm trying to get post-file working with wget (1.9.1), talking to
Apache 2.0.48, using PHP 4.3.4. This is all running on an x86 Linux machine
with Mandrake 10.
If I do an http post of a file through an HTML form (fill in the blanks with
a browser, hit submit), the PHP script catches all the data and files, and
everything is cool. When I use wget, however, the file does not seem to
make it there; I'm sure I'm doing something incorrectly--the man page is a
bit vague on post-file.
The command I think should work with wget is
% wget -O foo.html --post-file data.txt --post-data "varname=worksfine"
http://localhost/test.php
Is the PHP script possibly assuming that the post data is sent using formpost
multipart encoding? Can you show us what the HTML tag looks like that
you use for a browser to do this?
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
large files
Hi I suggest: That you apply and commit the currently best patch for large files. That is probably the one Leonid works/worked on. Isn't that the one Redhat has applied? I suggest that off_t isn't used to store/keep large file sizes, but instead a privately typedefed type instead. Named wget_off_t for example. On linux and most unix-like systems, we can 'typedef off_t wget_off_t' and most things will be fine. A strtoll() replacement is needed for some platforms. I believe this works fine for VMS too. I could check later. On systems like Windows, large file support takes some more work, and then the wget_off_t is probably best made 'typedef signed __int64 wget_off_t' with MSVC (but all libc-calls using wget_off_t will need attention/replacements since the windows libc APIs aren't supporting large files). On Windows with mingw or watcom, the typedef is better made 'typedef long long wget_off_t'. On Windows CE, I believe it works best as 'typedef long wget_off_t'. On systems without large file support, off_t is most often still present and simply just a 32 bit type. Thus, a file-size-as-string-to-number function (basicly strtol() or strtoll() depending on the platform) is suitable, to convert a string to wget_off_t. Now, this is only a suggestion meant to kick-start some discussions and possibly implementations. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: wget does not handle sizes over 2GB
On Wed, 12 Jan 2005, Wincent Colaiuta wrote: Daniel really needs to do one of two things: Thanks for telling me what to do. Your listing wasn't 100% accurate though. Am I not allowed to discuss technical solutions for wget if that involves a term from a different Free Software project I am involved in? I guess not. As you can see, in the other four mentionings of your list, I did mention the other tool to HELP the users who were asking for features wget doesn't provide. When people ask for stuff on other mailing lists and I know wget can do them fine, I usually recommend wget to them. I guess that is stupid too. Perhaps I'll learn all this when I grow older. This is my last mail here on this topic. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: wget does not handle sizes over 2GB
On Tue, 11 Jan 2005, Mauro Tortonesi wrote: oh, come on. let's not fall to the "my software is better than yours" childish attitude. I'm sorry if it came out that way, it was not my intention. I just wanted to address the misinformation posted here. I have not said and do not think that X is better than Y, just different. And I have contributed to this project serveral times and I might very well continue to do so. I am not just an author of another tool. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: wget does not handle sizes over 2GB
On Tue, 11 Jan 2005, Leonid wrote: curl does not survive losing connection. Since the probability to lose connection when you download 2Gb+ files is very high even if you have a fast connection, This mailing list is for wget, not curl. We can talk about what curl does and does not on the curl mailing list. It _does_ support request retrying and it does support download resuming - it is however not a wget clone. Besides, curl already supports large file transfers portably. Wget does not. In general, TCP connections _are not_ likely to disconnect no matter how much data you transfer, it is only likely to happen if you have shaky network setup. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: wget does not handle sizes over 2GB
On Sun, 9 Jan 2005, Denis Doroshenko wrote: *size = strtol (respline + 4, NULL, 0); where size is defined as "long int *" in the function's declaration. BTW. why's the base given to strtol is "0", not "10"? isn't that too flexible for a defined protocol? Yes it is, SIZE returns a base-10 number. The limitation seems to be hardwired throughout the source code, so it is not that simple to me There is at least one (platform specific) patch floating around that introduces large file support to wget on Linux. Until the situation is changed, I can recommend using curl for this kind of transfers. It supports large files on all platforms that do. Having done lots of the adjustments in the curl code, I have to admit that the work (the transition to portable large file support) wasn't _that_ hard once we actually started working on it. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
RE: wGET - NTLM Support
On Mon, 3 Jan 2005, Mudliar, Anand wrote:
I would appreciate if you please let me know as soon as the code is posted
on the site.
It would be great help if you can point me to some site/link where the old
version is posted.
Let me just point out loud and clear that my files were not complete patches
that introduced NTLM support to wget. They were files ripped out from the curl
source tree that I re-licensed and handed to the wget project to allow someone
with insights to adjust it to be usable. I'm not the person to say how much
time or effort this requires, or even if anyone wants to do it.
The old version is not available anymore so posting the old URL is not gonna
help anyone.
If you want to get a grasp of what the code looks like in its original shape,
check the lib/http_ntlm.[ch] files in curl's source repository.
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
RE: wGET - NTLM Support
On Fri, 31 Dec 2004, Herold Heiko wrote: Daniel, could you resend that code to the current co-maintainer Mauro Tortonesi [EMAIL PROTECTED] ? Maybe sooner or later he finds some time for this. The files I prepared are no longer around (I put them up on a site in november 2003 and I fixed the FSF assignment stuff before 2003 ended). I'll try to get time off to fix a new version of the files in the beginning of next year. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Problems communicating with digest authorization
On Tue, 14 Dec 2004, Steve Glymph wrote: (I know very little about wget, I'm just bringing my observations.) I was expecting to see the Basic authorization used initially (as seen below) but that a 401 error would be returned instead of a 403. Possibly that happens because you provide authentication that isn't accepted. I find it rather amusing that wget tries to "auto-detect" Digest while providing Basic, since that means that first it'll send the password in plain text only to let the server ask for a method in which the password doesn't have to get sent in plain text... So, basicly there's no way to avoid showing the password for evesdroppers even if Digest would work! I have tried to run curl to the same node and it does perform the negotiation correctly. Maybe it is about time I bring up my old suggestion of moving wget over to use libcurl for transfers... :-) -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Bugreport: wget and ssl via proxy
On Thu, 2 Dec 2004, Jim Wright wrote: export http_proxy=http://user:[EMAIL PROTECTED]:3128 export https_proxy=http://user:[EMAIL PROTECTED]:3128 Should both environment variables be identical? How does the proxy know that you are using ssl in the second case? The proxy shouldn't care about it. wget should use CONNECT when doing HTTPS over a proxy and it does GET when doing HTTP. IIRC, this problem is fixed in the CVS version. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: wget support for ftp/SSL
On Wed, 1 Dec 2004 [EMAIL PROTECTED] wrote: Is there a way to use SSL authentication with ftp in wget? AFAIK, wget doesn't support it. But curl does: curl.haxx.se -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Windows and > 2 GB downloads
On Thu, 18 Nov 2004, Leonid wrote: This was tested for Unixes. I am not sure that it will run for M$-Windows, but if not, the changes are supposed to be small. Then, please post a windows-patch. I'm not a Windows person, but since I made the large file support for curl I can tell you that 'off_t' is 32 bit on Windows systems and thus this does not enable large file support for those. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Help needed ...
On Mon, 1 Nov 2004, Phani Kumar Anumolu wrote: Can some body send me a sample code with which I can download a single file from the URL given. I am writing in C and tried to understand Wget..which looked very complicated. Consider using libcurl: http://curl.haxx.se/libcurl/ ... or a competitor: http://curl.haxx.se/libcurl/competitors.html -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: [PATCH] support for Content-Encoding in wget
On Mon, 25 Oct 2004, Karsten Scheibler wrote: The attached patch adds the option --accept-encoding-gzip to wget. If you enable this option the line: `Accent-Encoding: gzip' is added to the Header wget sends to the server. If the server responds with `Content-Encoding: gzip' the received data will be decompressed with gzip -d -c -. Since wget is often built and used on systems without fork(), I believe you won't see it applied in this shape (assuming patches are applied at all, that is). libz is the way to go, and then you could easily make it support both deflate and gzip. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Suggestion, --range
On Thu, 30 Sep 2004, Robert Thomson wrote: It would be really advantageous if wget had a --range command line argument, that would download a range of bytes of a file, if the server supports it. Until this is implemented, you may find it useful to know that curl supports this option => http://curl.haxx.se/ -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: 2 giga file size limit ?
On Fri, 10 Sep 2004, david coornaert wrote: My point is though that wget compiled on Tru64 OS does work with huge files. That's most likely because Tru64 is a 64bit architecture where all longs are 64 bits already and thus most code need no particular fixes to support >4GB files. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: File size limit
On Fri, 10 Sep 2004, r3al1tych3ck wrote:
Try using ncftpget. It will resume your download that wget aborted as well.
Not necessarily. Many FTP servers that handle >4GB file sizes fine don't
properly deal with REST >4GB (I don't know why, but it has shown up several
times).
Allow me to mention that curl groks large files too.
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: wput mailing list
On Thu, 26 Aug 2004, Justin Gombos wrote:
This list appears either dead, or very low traffic.
20-30 mails per month or so the last couple of month. That certainly isn't
dead, but it might be called "low traffic" depending on your opinion about
mail volumes.
I wasn't able to find any mailing lists for wput, so I just established one
at <http://groups.yahoo.com/group/wput/>, if anyone is interested.
wput is not wget so I agree discussing it on a different list is a good idea.
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: -T parameter on HP-UX port
On Sat, 5 Jun 2004, Hrvoje Niksic wrote: > Believe it or not, I wasn't aware of that. Furthermore, I'm pretty certain > that older versions of Autoconf handled HPUX just fine. The level of > overengineering that went into Autoconf in the last several years is > appalling. :-( I agree with this, and I too am convinced that this used to work fine in the past and just suddenly stopped working at some point. > Would it make sense to simply always use this check? In my mind, this is what the function-exists test should do, but I thought that I'd leave the existing test do what it thinks is right first since there might be cases and systems around that works in ways I haven't considered. For example, this extra test might fail if the function name is defined as a macro. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: -T parameter on HP-UX port
On Thu, 3 Jun 2004, Hrvoje Niksic wrote:
> It seems configure's way of checking for select simply fails on HPUX.
> :-(
The default configure test that checks for the presence of a function is
certainly not optimal for all platforms and environments.
In curl's configure script, we've resorted to a secondary check if the default
check fails, and that one detects select fine even in cases like this.
The concept is basicly like this:
AC_CHECK_FUNCS( select, ,
dnl if not found, $ac_func is the name we check for
func="$ac_func"
AC_MSG_CHECKING([deeper for $func])
AC_TRY_LINK( [],
[ $func ();],
AC_MSG_RESULT(yes!)
eval "ac_cv_func_$func=yes"
def=`echo "HAVE_$func" | tr 'a-z' 'A-Z'`
AC_DEFINE_UNQUOTED($def, 1, [If you have $func]),
AC_MSG_RESULT(but still no)
)
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
RE: Is NTLM authentication with wget possible?
On Tue, 25 May 2004, Herold Heiko wrote:
> Wget does not support NTLM auth (yet).
For the ones who feel like helping out, I have already donated (and properly
assigned copyright to FSF with papers and everything) fully working NTLM code
to the wget project (mailed privately to Hrvoje) that I'm sure everyone would
be happy if it was incorporated in the wget code base "properly".
My code - proved working in curl - is based on the wonderfully detailed notes
done by Eric Glass => http://davenport.sourceforge.net/ntlm.html
I'm not motivated nor involved enough in wget to do the necessary changes
myself.
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Wget race condition vulnerability (fwd)
I found this on the bugtraq mailing list and since I haven't seen it discussed here, I thought it could be informative. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol -- Forwarded message -- Date: 16 May 2004 23:28:47 - From: Hugo Vázquez Caramés <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Subject: Wget race condition vulnerability Tested software: Wget 1.9, Wget 1.9.1 Wget checks for the presence of a file with the same name of the one invoqued at the command line, if the file exists, then it saves the downloaded file with a different name. The problem is that Wget does not lock the file, and directly writes to it. So there's a window time where Wget is exposed to a symlink attack (only on world writable directories) This is the attack sequence: 1) Wget process starts 2) File checking (but not locking!) <--- attacker creates symlink 3) Wget writes on the wrong place As a P.o.C. here you have a very simple script that exploits this flaw with an attack I have called: "file hijacking". 1)Open a shell and execute wget_race.sh with user A. 2)Open another shell and with root user launch wget from /tmp: wget http://www.kernel.org/pub/linux/kernel/v2.4/patch-2.4.26.bz2 3) Check the content of /tmp/patch-2.4.26.bz2 Smile :-) --- wget_race.sh #!/bin/bash rm -f salida.txt pid.txt *.wget /tmp/patch-2.4.26.bz2 echo "1">salida.txt a=`cat salida.txt` echo "Waiting for Wget execution..." while [ "$a" == 1 ] do ps auxw|grep wget|grep patch-2.4.26.bz2>>salida.txt a=`cat salida.txt` done echo "Process catched!" pgrep -u root wget>pid.txt ln -s /dev/null /tmp/patch-2.4.26.bz2 echo "/dev/null link created!" echo "Waiting for downloading to finish..." b=`pgrep -u root wget` touch $b.wget c=1 while [ "$c" == 1 ] do if [ -e .wget ] then c=0 echo "Downloading finished! Let's delete the original file, and put our trojaned file :-)" rm -f /tmp/patch-2.4.26.bz2 echo "Surprise!">/tmp/patch-2.4.26.bz2 echo "Does it worked?" ls -la /tmp/patch-2.4.26.bz2 else b=`pgrep -u root wget` touch $b.wget fi done - This flaw open a wide range of attack vectors. Any program wich runs wget from a world writable directory is vulnerable. Hugo Vazquez Caramés [EMAIL PROTECTED]
Re: Large Files Support for Wget
On Tue, 11 May 2004, Hrvoje Niksic wrote: > That in itself is not a problem because, under Windows, off_t will be > typedeffed to a 64-bit type if LFS is available, and to `int' otherwise. Sorry for my confusion, but when I think about it I believe Windows _do_ have an off_t, but that is merely 32bit. The plain libc in Windows seem to be limited to 32 bits, even though the actual underlying code support 64bit sizes. In curl we work around this by providing our own curl_off_t variable. > The point of my question was: should low-level code even care whether LFS is > in use (by using off_t for various variables), or should it use intmax_t to > get the largest representation available on the system? The latter is in > principle sort of like using `long', except you're not tied to the actual > size of `long'. In curl we went with curl_off_t for various variables and that is then 32bit or 64bit depending on platform. I can't see many benefits in using 64bit variables on systems that don't deal with 64bit filesizes. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Large Files Support for Wget
On Mon, 10 May 2004, [iso-8859-2] Dra?en Ka?ar wrote: > > * Change most (all?) occurrences of `long' in the code to `off_t'. Or > > should we go the next logical step and just use uintmax_t right > > away? > > Just use off_t. ... but Windows has no off_t... ;-) -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: http over auth/proxy
On Tue, 30 Mar 2004, Hrvoje Niksic wrote:
> * http.c (gethttp): Send the Proxy-Authorization header over
> non-SSL connections too.
I couldn't really tell from this diff, but I thought I'd remind you:
If you are using SSL over a proxy, you should not send the Proxy-Authorization
in the GET request, only in the CONNECT request.
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: compiling wget 1.9 with ssl
On Fri, 5 Mar 2004, Hrvoje Niksic wrote: > Maybe configure should try to detect running under Red Hat 9 and kludge in > the kerberos path for that broken system. I've done exactly this hack once already for curl. It doesn't have to be _that_ kludgy, since you can invoke "pkg-config" to query what link and compiler options to use when building with OpenSSL, and then it'll include the path to the kerberos' headers. Feel free to use snippets from this: dnl Detect the pkg-config tool, as it may have extra info about the dnl openssl installation we can use. I *believe* this is what we are dnl expected to do on really recent Redhat Linux hosts. AC_PATH_PROG( PKGCONFIG, pkg-config, no, $PATH:/usr/bin:/usr/local/bin) if test "$PKGCONFIG" != "no" ; then AC_MSG_CHECKING([for OpenSSL options using pkg-config]) $PKGCONFIG --exists openssl SSL_EXISTS=$? if test "$SSL_EXISTS" -eq "0"; then SSL_LIBS=`$PKGCONFIG --libs-only-l openssl 2>/dev/null` SSL_LDFLAGS=`$PKGCONFIG --libs-only-L openssl 2>/dev/null` SSL_CPPFLAGS=`$PKGCONFIG --cflags-only-I openssl 2>/dev/null` LIBS="$LIBS $SSL_LIBS" CPPFLAGS="$CPPFLAGS $SSL_CPPFLAGS" LDFLAGS="$LDFLAGS $SSL_LDFLAGS" AC_MSG_RESULT([yes]) else AC_MSG_RESULT([no]) fi fi -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
RE: Syntax question ...
On Thu, 22 Jan 2004, Simons, Rick wrote: > curl https://server/file -uuser:pass > Virtual user user logged in. > > No file created locally. Chalk it up as a http server flaw? Uh, curl doesn't create any file when used like that. It outputs the downloaded data to stdout unless you use an option to direct it elsewhere. You can use "curl -v -i [URL]" to get to see the full request curl sends and all the headers it receives back. Then those could be compared to what wget sends/gets. In my eyes, this looks like the correct output from curl. Wasn't it? -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: bit overflow in wget 1.9
On Wed, 21 Jan 2004, Hrvoje Niksic wrote: > Thanks for the report. Wget still doesn't handle large files (it doesn't > use off_t yet). I plan to fix this for the next release. Speaking of this subject, I'm currently doing just about the same in libcurl (adding large file support) and I thought I'd share a clue I've learned: off_t is not a good type to use for this (large sizes) on Windows, since it is 32 bits even though Windows _does_ support large files. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: passive-ftp as default
On Thu, 11 Dec 2003, Hrvoje Niksic wrote: > IIRC passive FTP is not documented by RFC 959 It was. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: SSL over proxy passthrough
On Fri, 28 Nov 2003, Hrvoje Niksic wrote: > The bottom line is: should I even acknowledge `https' proxies? I don't think you'll ever run into any. > Do the browsers work with them? I don't know. I've never seen anyone use a proxy like that. > Does curl handle `https_proxy' or `http_proxy' being a https URL? Nope. curl only speaks non-SSL HTTP with the proxy. (To be precise, it ignores the protocol part of the given proxy and connects to it non-SSL.) -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: SSL over proxy passthrough
On Fri, 28 Nov 2003, Hrvoje Niksic wrote: > > I find this wording a bit funny. What is a "non-SSL" proxy? CONNECT is the > > defined way to speak SSL when using a http proxy... > > What if someone explicitly uses http_proxy=https://...? Or even > https_proxy=https://...? Ah, right! *That* is indeed a very-much-SSL proxy. I didn't consider that case. I don't think it is ever used in practise. > > The 'A2' encoded chunk needs the 'CONNECT' string then. > > I'm not sure I understand this. Sorry, I cut down a bit too much on my wording. I meant that since the Digest authentication hash uses the HTTP method string, you need to make sure that 'CONNECT' is the method used when you use Digest for this case. But as you said, Digest is rarely used for proxy authentication. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
RE: SSL over proxy passthrough
On Fri, 28 Nov 2003, Hrvoje Niksic's patch: > This patch implements a first attempt of using the CONNECT method to > establish passthrough of SSL communication over non-SSL proxies. This will > require testing. I find this wording a bit funny. What is a "non-SSL" proxy? CONNECT is the defined way to speak SSL when using a http proxy... > + /* This does not appear right. Can't the proxy request, > + say, `Digest' authentication? */ Right, the proxy can of course require Digest (or other kinds of) authentication. The 'A2' encoded chunk needs the 'CONNECT' string then. Also, I couldn't really tell from this patch, but make sure that you don't accidentally pass on the proxy authentication in the following request to the actual remote server as well. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
MSG_PEEK (was Re: Testing on BEOS?)
On Wed, 26 Nov 2003, Hrvoje Niksic wrote:
> Speaking of testing, please be sure to test the latest CVS on Windows as
> well, where MSG_PEEK is said to be flaky. HTTPS is another thing that might
> work strangely because SSL_peek is undocumented (!).
Out of curiosity, why are you introducing this peeking? I mean, what's the
gain?
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: feature request: --second-guess-the-dns
On Mon, 17 Nov 2003, Hrvoje Niksic wrote: > Come to think of it, I've had need for this before; the switch makes at > least as much sense as `--bind-address', which I've never needed myself. > > Maybe `--connect-address' would be a good name for the option? It would > nicely parallel `--bind-address'. > > Are there any takers to implement it? In curl land we offer this functionality in a more implicit way, by allowing the user to override any tool-generated header from the command line. It might not be as easily accessible as this proposed option, but it offers even more power. In the case where you want to connect to 1.2.3.4, asking for the host abc.com you would use 'curl -H "Host: abc.com" http://1.2.3.4'. This of course also lets you fool around with port numbers like 'curl -H "Host: abc.com:8080" http://1.2.3.4:2003' I don't claim this is a better way, I'm only providing food for thought here. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: keep alive connections
On Tue, 11 Nov 2003, Hrvoje Niksic wrote: > I think the difference is that Wget closes the connection when it decides > not to read the request body. For example, it closes on redirections > because it (intentionally) ignores the body. Another approach could be to read and just ignore the body of redirect pages. You'd gain a close/connect but lose the transfer time. > With the HEAD method you never know when you'll stumble upon a CGI that > doesn't understand it and that will send the body anyway. But maybe it > would actually be a better idea to read (and discard) the body than to close > the connection and reopen it. That approach is just as hard, only depending on different things to work correctly. Since we're talking about silly servers, they could just as well return a body to the HEAD request, and the response is said to be persistant and the Content-Length: is set. The size of the Content-Length in a HEAD request is the size of the body that would be returned if GET is request so you'd have no idea how much data to read Been there. Seen it happen. There's just no good way to deal with HEAD requests that sends back a body. I mean besides yelling at the author of the server side. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Does HTTP allow this?
On Mon, 10 Nov 2003, Hrvoje Niksic wrote: > I'm already treating SSL and non-SSL connections as incompatible. But I'm > curious as to why you say name-based virtual hosting isn't possible over > SSL? To quote the Apache docs: "Name-based virtual hosting cannot be used with SSL secure servers because of the nature of the SSL protocol." Since you connect to the site in a secure manner, you can't select which host to get data from after a successful connection, as the connection will not be successul unless you have all the proper credentials already. > That's very generous, thanks! I'll prepare a C file and header and post them in a separate mail. They will need a little attention, but not much. Mainly to setup pointers to user name, password, etc. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Does HTTP allow this?
On Sun, 9 Nov 2003, Hrvoje Niksic wrote: > One thing that might break (but that Wget doesn't yet support anyway) is > NTLM, which seems to authorize the *connections* individual connections. Yes it does. It certainly makes things more complicated, as you would have to exclude such a connection from the checks (at least I think you want that, I don't think you'll be forced to do so). And you also need to exclude HTTPS-connections from this logic (since name-based virtual hosting over SSL isn't really possible). curl doesn't do such advanced IP-checking to detect existing connections to re-use, it only uses host-name based checking for connection re-use for persistant connections. > Does curl handle NTLM? Yes it does since a while back. I am willing to donate NTLM code to the wget project, if you want it. I'm not very familiar with the wget internals so they wouldn't be a fully working patch, but a set of (proved working) functions to be integrated by someone with more wget insights. (It depends on crypto- functions provided by OpenSSL.) Otherwise, I can recommend Eric Glass' superb web page for all bits and and details on the NTLM protocol: http://davenport.sourceforge.net/ntlm.html -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Does HTTP allow this?
On Sat, 8 Nov 2003, Hrvoje Niksic wrote:
> So if I have the connection to the endpoint, I should be able to reuse it.
> But on the other hand, a server might decide to connect a file descriptor to
> a handler for a specific virtual host, which would be unable to serve
> anything else. FWIW, it works fine with Apache.
I would say that your described approach would work nicely, and it would not
contradict anything in the HTTP standards. Each single request is stand-alone
and may indeed have its own Host: header, even when the connection is kept
alive.
At least this is how I interpret these things.
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: keep alive connections
On Fri, 7 Nov 2003, Hrvoje Niksic wrote: > Persistent connections were available prior to HTTP/1.1, although they were > not universally implemented. Wget uses the `Keep-Alive' request header to > request persistent connections, and understands both the HTTP/1.0 > `Keep-Alive' and the HTTP/1.1 `Connection: keep-alive' response header. HTTP 1.1 servers don't (normally) use "Connection: keep-alive". Since 1.1 assumes persistant connections by default they only send "Connection: close" if they aren't. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: The libtool situation
On Wed, 5 Nov 2003, Hrvoje Niksic wrote: > Is it me, or is there something really strange going on with libtool? I > tried updating Wget's libtool, but libtool 1.5 is *still* unavailable from > the GNU FTP site and its mirrors. It points to a "BACK-RSN" type of file > that explains how they've been compromised, and so on. It must be something that has happened recently, as the 1.5 version used to be around on all the GNU mirrors. In case it helps, I happened to find my own downloaded package lying around and I can offer it for download from here: http://www.contactor.se/~dast/libtool-1.5.tar.gz -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Cookie options
On Wed, 5 Nov 2003, Hrvoje Niksic wrote: > Good point, although I somewhat disagree about the "plain wrong" part. From > what I gather, the `cookies.txt' file to which Wget and curl dump their > cookies is meant to hold permanent cookies. Putting session cookies inside > might confuse other browsers. Not necessarily, since they are stored without any expire time (using zero) and ordinary "non-session" cookies always have a number different than zero in that field. But sure, they might still confuse a browser that reads the file. I've never considered that a problem and I've never got any problems with that behavior reported. I believe very few people actually try that stunt. > That notwithstanding, another problem is that, since session cookies have no > expiry time, saving them by default would make them effectively immortal, > which sounds quite wrong to me. I may be missing curl-specific details, > though. Yes, it makes them immortal and lets the user decide when the "session" ends (by using the correct option). There is no one but the user who can tell when one session ends and another starts. In most cases, people will make a fresh cookie file each time they start a new session and then just keep session cookies around until they're done for this time. (Based on what I've done myself and seen others do with automated curl operations using cookies.) > Hmm. Wget will not ignore session cookies at load-time (any more), > regardless of options. `--keep-session-cookies' causes it not to discard > them at save time. Right, but then again people who use wget are more likely (I'm guessing here) to use one single wget command line and consider that to be the full session. In curl's case, when using cookies you will very often enter multiple command lines to sort out a single session's business. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Cookie options
On Tue, 4 Nov 2003, Hrvoje Niksic wrote: > Interestingly enough, curl has a `--junk-session-cookies', which indicates > that it keeps them by default (?). Daniel, are you still listening? :-) I am, and that is correct! I never thought that single-shot curl invokes could be seen as single "sessions" and therefore I think removing (or not saving) session cookies is plain wrong and will only make things harder and more confusing to users. But then, I believe the average usage of wget and curl differ a bit. Many users that use curl to script actions and that use cookies, operate with multiple curl command lines. The option named --junk-session-cookies is a rather new baby in curl land, as previously curl never ignored session cookies at load-time. Also, curl doesn't have any option to send out-of-date cookies such as has been proposed here as an option for wget, and I've never got a request from a single soul to add it to curl. I'd say it is an indication that such an option would not be widely used. (Given that curl has been used for scripting POSTs, logins and cookie stuff for years already.) -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: POST followed by GET
On Tue, 14 Oct 2003, Tony Lewis wrote: > It would be the same logically equivalent to the following three commands: > > wget --user-agent='my robot' --post-data 'data=foo' POST URL1 > wget --user-agent='my robot' --post-data 'data=bar' POST URL2 > wget --user-agent='my robot' --referer=URL3 GET URL4 Just as a comparison, this approach is basicly what we've went with in curl (curl has supported this kind of operations for years, including support for multipart formposts which I guess is next up for adding to wget! ;-P). There are just too many options or specifics that you can set, so having them all possible to change between several URLs specified on the command line makes the command line parser complicated and the command lines even more complex. The main thing this described approach requires (that I can think of) is that wget would need to store session cookies as well in the cookie file (I believe I read that it doesn't atm). -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Using chunked transfer for HTTP requests?
On Tue, 7 Oct 2003, Hrvoje Niksic wrote: > My first impulse was to bemoan Wget's antiquated HTTP code which doesn't > understand "chunked" transfer. But, coming to think of it, even if Wget > used HTTP/1.1, I don't see how a client can send chunked requests and > interoperate with HTTP/1.0 servers. > > The thing is, to be certain that you can use chunked transfer, you > have to know you're dealing with an HTTP/1.1 server. But you can't > know that until you receive a response. And you don't get a response > until you've finished sending the request. A chicken-and-egg problem! The only way to deal with this automaticly, that I can think of, is to use a "Expect: 100-continue" request-header and based on the 100-response you can decide if the server is 1.1 or not. Other than that, I think a command line option is the only choice. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Portable representation of large integers
On Mon, 22 Sep 2003, Hrvoje Niksic wrote:
> The bottom line is, I really don't know how to solve this portably. Does
> anyone know how widely ported software deals with large files?
In curl, we provide our own *printf() code that works as expected on all
platforms.
(Not that we have proper 2GB support yet anyway, but that's another story.
For example, we have to face the problems with exposing an API using such a
variable type...)
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: IPv6 detection in configure
On Tue, 9 Sep 2003, Hrvoje Niksic wrote:
> Thanks to Daniel Stenberg who has either been reading my mind or has had the
> exact same needs, here is a patch that brings configure (auto-)detection for
> IPv6.
Of course I read your mind, what else could it be? :-P
I'm glad it helped.
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
IPv6 detection in configure
Hi Hrvoje,
These are two snippets that can be used to detect IPv6 support and a "working"
getaddrinfo() info. Adjust as you see fit!
--- snippet for acinclude.a4 or configure.ac ---
dnl
dnl check for working getaddrinfo()
dnl
AC_DEFUN(WGET_CHECK_WORKING_GETADDRINFO,[
AC_CACHE_CHECK(for working getaddrinfo, ac_cv_working_getaddrinfo,[
AC_TRY_RUN( [
#include
#include
#include
void main(void) {
struct addrinfo hints, *ai;
int error;
memset(&hints, 0, sizeof(hints));
hints.ai_family = AF_UNSPEC;
hints.ai_socktype = SOCK_STREAM;
error = getaddrinfo("127.0.0.1", "8080", &hints, &ai);
if (error) {
exit(1);
}
else {
exit(0);
}
}
],[
ac_cv_working_getaddrinfo="yes"
],[
ac_cv_working_getaddrinfo="no"
],[
ac_cv_working_getaddrinfo="yes"
])])
if test "$ac_cv_working_getaddrinfo" = "yes"; then
AC_DEFINE(HAVE_GETADDRINFO, 1, [Define if getaddrinfo exists and works])
AC_DEFINE(ENABLE_IPV6, 1, [Define if you want to enable IPv6 support])
IPV6_ENABLED=1
AC_SUBST(IPV6_ENABLED)
fi
])
--- snippet for configure.ac ---
dnl **
dnl Checks for IPv6
dnl **
AC_MSG_CHECKING([whether to enable ipv6])
AC_ARG_ENABLE(ipv6,
AC_HELP_STRING([--enable-ipv6],[Enable ipv6 support])
AC_HELP_STRING([--disable-ipv6],[Disable ipv6 support]),
[ case "$enableval" in
no)
AC_MSG_RESULT(no)
ipv6=no
;;
*) AC_MSG_RESULT(yes)
ipv6=yes
;;
esac ],
AC_TRY_RUN([ /* is AF_INET6 available? */
#include
#include
main()
{
if (socket(AF_INET6, SOCK_STREAM, 0) < 0)
exit(1);
else
exit(0);
}
],
AC_MSG_RESULT(yes)
ipv6=yes,
AC_MSG_RESULT(no)
ipv6=no,
AC_MSG_RESULT(no)
ipv6=no
))
if test "$ipv6" = "yes"; then
WGET_CHECK_WORKING_GETADDRINFO
fi
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: NTLM authentication in wget
On Mon, 8 Sep 2003, Wayne Eisenberg wrote: > I think it must have something to do with NTLM authentication in IIS 5.0, > but here's the debug output: [snip] > Date: Tue, 09 Sep 2003 09:14:34 GMT > WWW-Authenticate: Negotiate > WWW-Authenticate: NTLM > Content-Length: 4431 > Content-Type: text/html This is NTLM required, indeed. > Does anyone have any ideas on what's wrong and how to fix it (if possible)? The problem is that wget doesn't support NTLM. The fix is to implement it. A work-around would be to get a recent curl release, as it supports NTLM. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: autoconf 2.5x and automake support for wget 1.9 beta
On Tue, 2 Sep 2003, Mauro Tortonesi wrote: > > 2. Care to elaborate on why you introduced automake in wget? > > it makes the sources __MUCH__ easier to maintain, believe me. you only need > 6 months (i am __definitely__ not joking here) to understand how the > automake/autoconf/libtool/gettext stuff works and then No need for any lecture here. I'm quite well acquainted with how the autotools family works. > i have the same feeling of yours. and i can't understand why the wget > developers didn't use automake. Perhaps because a) automake doesn't add much value to wget while b) adding complexity to the output makefiles. > in fact, if you take a look at the Makefile.in's generated by automake in my > package you'll see that they aren't much different from the original > Makefile.in's contained in wget-1.9beta, so why bother maintaining files > that could be automatically generated from automake? In my eyes, I think you've simply moved the work to maintain src/Makefile.am instead of maintaining src/Makefile.in. The amount of work seems to be pretty much the same. > 1) the Makefile.in's generated by automake have more functions (like the > very useful make maintainer-clean) If you say so! ;-) I've never used that target, and I've maintained packages built with automake for many years! > 2) automake provides a seamless integration with libtool Why is libtool-integration necessary in wget? > 3) with automake is much easier to build the texi documentation and also > to generate html, ps and pdf docs Why? > 4) automake supports automatic de-ANSI-fication (which personally, i > hate, but many other developers like) Doesn't wget already have this support? > 5) with automake it is easier to generate and check tarballs Agreed. But this isn't something that is very hard to do without automake... > is this enough? ;-) You don't have to convince *me* since I'm just one of the interested guys in the audience, asking one of the players on stage! ;-) I'll go back to lurk mode again. Thanks for your lengthy reply anyway! -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: autoconf 2.5x and automake support for wget 1.9 beta
On Mon, 1 Sep 2003, Mauro Tortonesi wrote:
> i hope that you will like the new autoconf-2.5x/automake-1.6-ed package. i
> also hope that the wget maintainer will consider accepting this patch.
1. There is no current wget maintainer. Your CVS tree has a good chance
of becoming the official one if you want to.
2. Care to elaborate on why you introduced automake in wget? I have a feeling
this is not what earlier wget hackers would've wanted.
--
-=- Daniel Stenberg -=- http://daniel.haxx.se -=-
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
RE: https authentification with wget
On Tue, 5 Aug 2003, Bartel, Lawrence wrote: > > I would like to know if the authentification i.e. : wget > > https://user:[EMAIL PROTECTED] is send clearly to the https server, or if wget deal > > with it as a proper https clien, ie: it send the user/pwd in a ecrypt way. > According to http://www.gnu.org/manual/wget/html_mono/wget.html#SEC49 they > are not encrypted. That is just plain wrong. When using SSL, the client connects and does an SSL handshake before any actual HTTP authentication has been sent to the server. Thus, if the SSL handshake is successful, the HTTP authentication credentials will be sent to the server encrypted over the SSL connection. -- Daniel Stenberg - http://daniel.haxx.se - +46-705-44 31 77 ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Reminder: wget has no maintainer
On Tue, 12 Aug 2003, Aaron S. Hawley wrote: > > 1) is proficient in C That is next to common knowledge. > > 2) knows Internet protocols THe HTTP 1.0 and FTP RFC959 that wget supports today are pretty basic and no-frills. Besides, there are many proof-readers in the audience that help identifying flaws if any should occur. > > 3) is willing to learn the intricacies of wget Judging from the shear amount of patches that fly around, many people have walked down that road already. > > 4) has the time to go through months' worth of email and patches Nah, ignore them and look forward. If people really care they'll resubmit them or others will. Setting up a proper bug tracker for wget could be a separate task for someone to assist the new maintainer with... > > 5) expects to have time to continue to maintain wget Yes. A maintainer that can't maintain wouldn't be any improvement from what the situation is now. We know Hrvoje is there, somewhere, only lacking the opportunity to maintain wget. > 2a) All of them. :) > (Is familiar with application-level network protocols, web standards > and their usage, correct or otherwise . For an example see the HTML > comments thread. Knows (X)HTML and familiar with other web media > formats.) Yeah, but there's no reason all that knowledge would have to be possesed by one single do-it-all magician. > 6) Will move Wget sources to the savannah(.gnu.org) Whatever, as long as there are more than one person with CVS write access... > it seems like there's a lot of hackers, bug-reporters, users and supporters > who could collectively accomplish a lot of the work, we really just need an > all-knowing philosopher-queen or king to not necessarily do a whole lot of > work, but just have an active CVS account. so as technically threatening as > the Wget maintainer application looks, we could currently most benefit from > a trusted soul. Indeed. Or make that "trusted souls", as I believe it would be better to have more than one. -- -=- Daniel Stenberg -=- http://daniel.haxx.se -=- ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
