Re: [Bug-wget] new alpha tarball wget-1.12-2460.tar.bz2
Hi, It is still an issue that wget/openssl combo is broken in windows. Wget source has changed to file descriptors for sending and receiving data in windows. OpenSSL in windows expects a socket that it can recv/send. From OpenSSL's e_os.h for WINDOWS: #define readsocket(s,b,n)recv((s),(b),(n),0) #define writesocket(s,b,n)send((s),(b),(n),0) Anything in OpenSSL that tries to write to a socket will fail because it's passed a fd and not a socket. For example sock_write() in openssl's crypto/bio/bss_sock.c:153 calling send() and passing a fd will cause an error of WSAENOTSOCK. Here are three ideas: 1. Revert to sockets don't use gnulib select() etc 2. Although OpenSSL's BIOs are abstract enough that it is possible to change the pointer to their associated read/write functions in wget (specifically those used by bss_conn/dgram/sock, respectively). It's hacky to do this though, and it could cause problems if OpenSSL later changes their BIO code. 3. In wget's openssl.c to get the handle/socket from the fd using (SOCKET) _get_osfhandle ((fd)) and pass that, as is now done elsewhere in wget code. Although OpenSSL can then send/recv, I witnessed data dropouts while testing this. Has anyone else tested mingw wget build with mingw openssl since the change in wget to file descriptors on windows? Another thing is the configure test for openssl is still using ssl and crypto libs configure:22076: gcc -o conftest.exe -O2 -Wall conftest.c -lssl -lcrypto 5 but on windows you want -lssl -lcrypto -lws2_32 -lgdi32 As I mentioned at some other point in time what you'd expect is shared libs when building. Unfortunately a similar test for that will fail if the actual dll is not in the path. Would it be better for just an AC_CHECK_LIB on eay32 and ssl32? Another thing re ipv6 support: host.c: In function 'getaddrinfo_with_timeout_callback': host.c:383:3: warning: implicit declaration of function 'getaddrinfo' host.c: In function 'lookup_host': host.c:787:5: warning: implicit declaration of function 'freeaddrinfo' In windows ws2tcpip.h should be included in addition to winsock2.h. Some headers for ws2tcpip.h have the winsock2.h include some don't. The order is # include winsock2.h # include ws2tcpip.h When ipv6 is enabled _WIN32_WINNT should be defined = 0x0501 (WinXP) before includes. This means wget with ipv6 will not work on win2000. There's a solution for this but it requires rewriting code that is copyrighted microsoft for a getaddrinfo wrapper, unless someone has already done this. Is windows 2000 support still wanted? I have one request from last year but other than that I don't hear about it anymore. Thanks, Jay Satiro on behalf of the GetGnuWin32 project - Original Message From: Giuseppe Scrivano gscriv...@gnu.org To: bug-wget@gnu.org Sent: Sun, March 27, 2011 10:37:57 AM Subject: Re: [Bug-wget] new alpha tarball wget-1.12-2460.tar.bz2 Hello, as a followup to the last alpha release, I have uploaded a new tarball with some minor changes: ftp://alpha.gnu.org/gnu/wget/wget-1.12-2466.tar.bz2 and the detached GPG signature using the key C03363F4: ftp://alpha.gnu.org/gnu/wget/wget-1.12-2466.tar.bz2.sig If no problems are found, I am going to finally release wget 1.13 in the next few days. Cheers, Giuseppe
[Bug-wget] mirroring one sourceforge package?
The bug (?) -- running wget -m -np -nv \ http://sourceforge.net/projects/biblatex-biber/files/biblatex-biber/current/ ends up downloading many things above that directory, despite the -np. Doesn't that seem wrong? This is with wget 1.12 compiled from the original source. The request: does anyone know a way to mirror one package (ideally a subdirectory thereof) from sourceforge? Specifically, I want http://sourceforge.net/projects/biblatex-biber/files/biblatex-biber/current/ (I don't actually care about using wget, could be anything. :) I did a bunch of searching to no avail. My friends at CTAN have tried from time to time over the years but never found any really reliable method -- always ends up coming down to scraping the HTML :(. One possible answer, Joining the sourceforge mirror network (http://sourceforge.net/apps/trac/sourceforge/wiki/Instructions for joining the SourceForge.net mirror network) is not what I want to do. I just want one package, not to help sf. I suspect there is no good way, given sf's policies and setup, but thought I would ask here. Thanks, Karl
Re: [Bug-wget] mirroring one sourceforge package?
(03/30/2011 02:37 PM), Karl Berry wrote: The bug (?) -- running wget -m -np -nv \ http://sourceforge.net/projects/biblatex-biber/files/biblatex-biber/current/ ends up downloading many things above that directory, despite the -np. Doesn't that seem wrong? This is with wget 1.12 compiled from the original source. Definitely a bug; reproduced with Ubuntu Lucid's wget 1.12. Running with --debug, I see a lot of: Deciding whether to enqueue http://sourceforge.net/blog/;. Going to blog would escape projects/biblatex-biber/files/biblatex-biber/current with no_parent on. Decided NOT to load it. And then: Deciding whether to enqueue https://sourceforge.net/blog/;. Allowing path blog/ because of rule `'. Decided to load it. That link was apparently found in https://sourceforge.net/account/login.php So it looks like wget is correctly blocking the http URL, but incorrectly permitting the https URL. Adding -R login.php seems a decent workaround; I let it run awhile (not forever), and it seemed okay, though it did get a single link (so far) outside the expected hierarchy (once again, an https link; this time to a wiki page; the page fortunately appears not to have incurred other renegade links AFAICT). -- HTH, Micah J. Cowan http://micah.cowan.name/
Re: [Bug-wget] mirroring one sourceforge package?
Thanks Tony. I wonder if it's possible that that file is a redirection from a correct URL. Because wget would expect to download all URLs from a redirection, and would use the redirected name (but AIUI the current dev sources wouldn't use that name without --trust-server-name or something). In any event, it seems pretty clear that something busted between 1.11.4 and 1.12. -mjc (03/30/2011 03:06 PM), Tony Lewis wrote: It works as I would expect in 1.11.4, with the exception of downloading this file: sourceforge.net/projects/biblatex-biber/files/index.html Tony
Re: [Bug-wget] mirroring one sourceforge package?
Micah Cowan mi...@cowan.name writes:
So it looks like wget is correctly blocking the http URL, but
incorrectly permitting the https URL.
We check if the two schemes are similar but at the same time we require
the port to be identical.
I have relaxed this condition, now the two ports must be identical only
in the case the same protocol is used.
I have pushed this patch:
=== modified file 'src/recur.c'
--- src/recur.c 2011-01-01 12:19:37 +
+++ src/recur.c 2011-03-30 23:36:05 +
@@ -563,7 +563,8 @@
if (opt.no_parent
schemes_are_similar_p (u-scheme, start_url_parsed-scheme)
0 == strcasecmp (u-host, start_url_parsed-host)
- u-port == start_url_parsed-port
+ (u-scheme != start_url_parsed-scheme
+ || u-port == start_url_parsed-port)
!(opt.page_requisites upos-link_inline_p))
{
if (!subdir_p (start_url_parsed-dir, u-dir))
Applying it and launching wget using the same arguments used by Karl, I
get:
$ find sourceforge.net/ -maxdepth 3
sourceforge.net/
sourceforge.net/projects
sourceforge.net/projects/biblatex-biber
sourceforge.net/projects/biblatex-biber/files
sourceforge.net/robots.txt
Just in time before the release :-)
Cheers,
Giuseppe
