Re: wget 1.7 configure errors
On Tue, 5 Jun 2001, Jan Prikryl wrote: > > Yes, the SSL detection is broken on some systems. We are working on a > fix. > Since wget is now a crypto application, I suggest extreme caution here. On my system, only static SSL libraries exist. It bothers me greatly that an attacker could corrupt a shared crypto library and, thus, corrupt all crypto apps. Now your current testing uses rpath which is irrelevant with static libs, but building with rpath is not benign because that same attacker (or another one) could plant strange libraries in the rpath location and wget could be compromised. I would suggest either testing whether shared SSL libraries exist or providing configure options so the user can choose. > > Could you provide an example? On my machine, GCC is in /usr/local/bin > by default (I'm using the GCC 3.0 development version which has some > bugs, so I prevent to keep tow versions of the compiler). In this > setup, `CC=/usr/bin/gcc ./configure ... ; make' will build wget with > the "old" GCC 2.95.2 just fine. > This is my error. I have a config.site script and a cc definition crept in by mistake. I'm sorry that I bothered you with this. In the midst of all this, I built wget with debugging and gcc turned up these warnings: rbuf.c:68: warning: implicit declaration of function `ssl_iread' retr.c:122: warning: implicit declaration of function `ssl_iread' gen_sslfunc.c:155: warning: implicit declaration of function `select_fd' Some header files are not included and some are incomplete. Wget works because the defaults happen to match the actual functions. It is not a big deal though. Thanks. -- Jupiter 2001-06-06 19:07:22.660 UTC (JD 2452067.296790) X = 0.631953667, Y = 4.648285018, Z = 1.977027736 (au) X' = -0.007581252, Y' = 0.001124320, Z' = 0.000666516 (au/d)
Re: How do I get SSL support to work in 1.7?
On Wed, Jun 06, 2001 at 02:09:12PM -0400, Edward J. Sabol wrote: > H. I've tried connecting to various sites using https without success. > I've tried this on both IRIX 6.5.2 and Digital Unix 4.0d. > > When I installed OpenSSL 0.9.6a using the default configure options, it > didn't make any shared libraries, but I have libssl.a and libcrypto.a > installed, and wget's configure process does find them. (Do I need to install > the shared libraries?) > > For example, I can connect to https://www.apache-ssl.org/ in Netscape just > fine, but here's what happens when I try with wget 1.7: > > [... debug output removed ...] Not surprising. Neither IRIX 6.5 nor Tru64 UNIX 4.0D have /dev/random. So, you need either EGD/PRNGD to provide a substitute for your missing /dev/random. And, the *client* software has to be configured to support this. So, if wget doesn't call RAND_egd() from OpenSSL, there is *nothing* you can do. And, from a quick perusal of wget 1.7, it doesn't. So, 1.7 is useless for https:// on any system without /dev/random. Note that we have added such support to other programs and will hopefully get time soon to do it to wget. cURL already has support for EGD/PRNGD so we're just going to steal their solution. -- albert chin ([EMAIL PROTECTED])
Re: WGET is changing my URL!
Quoting Kohler Roberto ([EMAIL PROTECTED]): > I am using wget 1.5.3 to get the URL > "cache_object://localhost:/info" (cache statistics from Squid proxy) > and the program is changing the URL to > "ftp://cache_object:21/%2Flocalhost/info"; (guess because the > protocol cache_object is not known). Exactly. Wget knows nothing about "cache_object:" URL identifier and it assumes you wanted a FTP one. > I could not find an option so the program would accept the URL "as > is". There is no such an option, sorry. You may only use http://, https://, or ftp:// URLs with wget. -- jan +-- Jan Prikryl| vr|vis center for virtual reality and visualisation <[EMAIL PROTECTED]> | http://www.vrvis.at +--
WGET is changing my URL!
Hello. I am using wget 1.5.3 to get the URL "cache_object://localhost:/info" (cache statistics from Squid proxy) and the program is changing the URL to "ftp://cache_object:21/%2Flocalhost/info"; (guess because the protocol cache_object is not known). I could not find an option so the program would accept the URL "as is". /Roberto
How do I get SSL support to work in 1.7?
H. I've tried connecting to various sites using https without success.
I've tried this on both IRIX 6.5.2 and Digital Unix 4.0d.
When I installed OpenSSL 0.9.6a using the default configure options, it
didn't make any shared libraries, but I have libssl.a and libcrypto.a
installed, and wget's configure process does find them. (Do I need to install
the shared libraries?)
For example, I can connect to https://www.apache-ssl.org/ in Netscape just
fine, but here's what happens when I try with wget 1.7:
% ./src/wget --debug https://www.apache-ssl.org/
DEBUG output created by Wget 1.7 on osf4.0d.
parseurl ("https://www.apache-ssl.org/";) -> host www.apache-ssl.org -> opath -> dir
-> file -> ndir
newpath: /
--14:01:54-- https://www.apache-ssl.org/
=> `index.html'
Connecting to www.apache-ssl.org:443... Caching www.apache-ssl.org <-> 193.123.86.250
Created fd 4.
connected!
Unable to establish SSL connection.
Closing fd 4
Unable to establish SSL connection.
Re: win binary
Harald Gerber wrote: > T.Bharath <[EMAIL PROTECTED]> wrote ><[EMAIL PROTECTED]> on 05.06.01 > > > > Herold Heiko wrote: > > > > > A windows binary for 1.7 is present at > > > http://space.tin.it/computer/hherold . > > > Heiko > > > > The server seems to reset the connection when the source or binary > > is downloaded > > can you look into it > > worked fine from my place. Lots of resets here too, but I just let WGET get it! However, thats not much use if you haven't got it yet and are trying to download it! :-) Rick. -- Richard Travett,Email: [EMAIL PROTECTED] May all your communications be free from electro-magnetic disturbances, and your days be free from temporal distortions in the space time continuum This email does not represent a formal communication from Simoco.
Re: wget 1.7, linux, -rpath
On Wed, Jun 06, 2001 at 06:36:26PM +0200, Jan Prikryl wrote: > Quoting [EMAIL PROTECTED] ([EMAIL PROTECTED]): > > > The ssl support is much appreciated in wget 1.7. But there is a problem > > with the configure support that makes it think ssl can't be used, at > > least with gcc 2.95.2 on my redhat 6.2 system: > > Thanks for the report. Unfortunately the SSL test does not work on > linux at all. Replacing "-rpath " with "-Wl,rpath " will solve part of > the problems. You may want to try if the attached patch works for > you. Note that this is an unofficial patch and while it may help > solving the SSL check problem, it may break other things. Why don't you steal the --with-ssl option for cURL? It works. -- albert chin ([EMAIL PROTECTED])
Re: Wget 1.7-pre1 available for testing
On Wed, Jun 06, 2001 at 12:47:14PM +0200, Jan Prikryl wrote: > > Jan Prikryl <[EMAIL PROTECTED]> writes: > > > > > It seems that -lsocket is not found as it requires -lnsl for > > > linking. -lnsl is not detected as it does not contain > > > `gethostbyname()' function. > > > > That's weird. What does libnsl contain if not gethostbyname()? > > It seems to contain `gethostname()' ... see the config.log submitted > in one of the previous emails. But it's a very long distance shot: if, > after adding -lsocket -lnsl everything works correctly and if with > -lsocket only the linker complains about missing 'yp_*()' functions > and also missing `gethostname()' and `getdomainname()', I thinks it's > likely that these functions are defined in -lnsl. Of course, if -lnsl > has built in dependency on some other library, the situation might be > completely different. > > > Jan, you must be confusing something here. gethostname() only gets > > the local host name, and is just a wrapper for the appropriate > > uname() or sysinfo() call. It has nothing to do with name server > > lookups, which is what libnsl is supposed to do. > > Probably, but are you sure that this is true on _all_ systems? FYI, gethostname is in libc on Solaris. A better test is gethostent which is in libnsl. Also, on HP-UX 11.00, the yp_* functions are in libnsl while just about everywhere else they're in libc. -- albert chin ([EMAIL PROTECTED])
Re: wget 1.7, linux, -rpath
Quoting [EMAIL PROTECTED] ([EMAIL PROTECTED]):
> The ssl support is much appreciated in wget 1.7. But there is a problem
> with the configure support that makes it think ssl can't be used, at
> least with gcc 2.95.2 on my redhat 6.2 system:
Thanks for the report. Unfortunately the SSL test does not work on
linux at all. Replacing "-rpath " with "-Wl,rpath " will solve part of
the problems. You may want to try if the attached patch works for
you. Note that this is an unofficial patch and while it may help
solving the SSL check problem, it may break other things.
-- jan
+--
Jan Prikryl| vr|vis center for virtual reality and visualisation
<[EMAIL PROTECTED]> | http://www.vrvis.at
+--
Index: configure.in
===
RCS file: /pack/anoncvs/wget/configure.in,v
retrieving revision 1.17
diff -u -r1.17 configure.in
--- configure.in2001/05/28 22:02:47 1.17
+++ configure.in2001/06/06 05:26:20
@@ -174,8 +174,12 @@
AC_CHECK_FUNCS(strdup strstr strcasecmp strncasecmp)
AC_CHECK_FUNCS(gettimeofday mktime strptime)
AC_CHECK_FUNCS(strerror snprintf vsnprintf select signal symlink access isatty)
-AC_CHECK_FUNCS(uname gethostname)
+AC_CHECK_FUNCS(uname)
+AC_CHECK_FUNCS(gethostname, [], [
+ AC_CHECK_LIB(nsl, gethostname)
+])
+
AC_CHECK_FUNCS(gethostbyname, [], [
AC_CHECK_LIB(nsl, gethostbyname)
])
@@ -205,14 +209,18 @@
AC_MSG_CHECKING("for runtime libraries flag")
case "$host_os" in
sol2 ) dash_r="-R" ;;
- decosf* | linux* | irix*) dash_r="-rpath " ;;
+ decosf* | irix*) dash_r="-rpath " ;;
+ linux*) dash_r="-Wl,rpath " ;;
*)
dash_r=""
for try_dash_r in "-R" "-R " "-rpath "; do
OLD_LDFLAGS=$LDFLAGS
LDFLAGS="${try_dash_r}/no/such/file-or-directory $LDFLAGS"
+ dnl gcc seems to only produce a warning about nonexistent option
+ dnl `-R/no/such/file-or-directory' so the test comes thru
+ dnl (tested with "gcc version 3.0 20010308 (prerelease)")
AC_TRY_LINK(, , dash_r="$try_dash_r")
- LDFLAGS=$ODL_LDFLAGS
+ LDFLAGS=$OLD_LDFLAGS
test -n "$dash_r" && break
done ;;
esac
@@ -235,9 +243,6 @@
ssl_all_roots=$with_ssl
fi
- OLD_LIBS=$LIBS
- OLD_LDFLAGS=$LDFLAGS
-
dnl Unfortunately, as of this writing (OpenSSL 0.9.6), the libcrypto
dnl shared library doesn't record its dependency on libdl, so we
dnl need to check for it ourselves so we won't fail to link due to a
@@ -245,6 +250,9 @@
dnl shl_load().
AC_CHECK_LIB(dl,dlopen)
AC_CHECK_LIB(dl,shl_load)
+
+ OLD_LIBS=$LIBS
+ OLD_LDFLAGS=$LDFLAGS
ssl_linked=no
wget 1.7, linux, -rpath
The ssl support is much appreciated in wget 1.7. But there is a problem with the configure support that makes it think ssl can't be used, at least with gcc 2.95.2 on my redhat 6.2 system: configure:3660: checking for RSA_new in -lcrypto configure:3679: gcc -o conftest -g -O2 -L/usr/local/openssl/lib -rpath /usr/local/openssl/lib conftest.c -lcrypto 1>&5 gcc: unrecognized option `-rpath' /usr/local/openssl/lib: file not recognized: Is a directory This comes from configure: decosf* | linux* | irix*) dash_r="-rpath " ;; -rpath isn't recognized directly by gcc as far as I can tell. -Wl,-rpath works, though. I suspect the same would be needed on irix and decosf but don't know for sure. -R is recognized directly by gcc on Solaris, so that part is ok. Thanks, karl
problem with wget 1.7
please try: wget --mirror http://www.ire.pw.edu.pl/zejim/rois/ and wget loops ;( 1.6 version works fine. -- Arkadiusz Miśkiewicz, AM2-6BONE, 1024/3DB19BBD http://www.t17.ds.pwr.wroc.pl/~misiek/ipv6/ IPv6 ready PLD/Linux at http://www.pld.org.pl/
Re: Wget 1.7-pre1 available for testing
On 2001-06-06 12:47 +0200, Jan Prikryl wrote: > > Jan Prikryl <[EMAIL PROTECTED]> writes: > > > > > It seems that -lsocket is not found as it requires -lnsl for > > > linking. -lnsl is not detected as it does not contain > > > `gethostbyname()' function. > > > > That's weird. What does libnsl contain if not gethostbyname()? > > It seems to contain `gethostname()' ... see the config.log submitted > in one of the previous emails. But it's a very long distance shot: if, > after adding -lsocket -lnsl everything works correctly and if with > -lsocket only the linker complains about missing 'yp_*()' functions > and also missing `gethostname()' and `getdomainname()', I thinks it's > likely that these functions are defined in -lnsl. Of course, if -lnsl > has built in dependency on some other library, the situation might be > completely different. I've put the output of nm for libsocket and libnsl at http://www.teaser.fr/~amajorel/mpras/libnsl.so.nm.gz http://www.teaser.fr/~amajorel/mpras/libsocket.so.nm.gz -- André Majorel Work: <[EMAIL PROTECTED]> Home: <[EMAIL PROTECTED]> http://www.teaser.fr/~amajorel/
Re: Wget 1.7-pre1 available for testing
> Jan Prikryl <[EMAIL PROTECTED]> writes: > > > It seems that -lsocket is not found as it requires -lnsl for > > linking. -lnsl is not detected as it does not contain > > `gethostbyname()' function. > > That's weird. What does libnsl contain if not gethostbyname()? It seems to contain `gethostname()' ... see the config.log submitted in one of the previous emails. But it's a very long distance shot: if, after adding -lsocket -lnsl everything works correctly and if with -lsocket only the linker complains about missing 'yp_*()' functions and also missing `gethostname()' and `getdomainname()', I thinks it's likely that these functions are defined in -lnsl. Of course, if -lnsl has built in dependency on some other library, the situation might be completely different. > Jan, you must be confusing something here. gethostname() only gets > the local host name, and is just a wrapper for the appropriate > uname() or sysinfo() call. It has nothing to do with name server > lookups, which is what libnsl is supposed to do. Probably, but are you sure that this is true on _all_ systems? > Perhaps we really should try to write a libtool-based macro named > WGET_CHECK_EXTERNAL_LIB. Perhaps it would be more portable then. -- jan ---+ Jan Prikryl icq | vr|vis center for virtual reality and <[EMAIL PROTECTED]> 83242638 | visualisation http://www.vrvis.at ---+
Re: wget memory leak or expected operation
jolan <[EMAIL PROTECTED]> writes: > I'm currently using wget to mirror a rather extensive website. So > far I've mirrored 10 gigs and counting. wget has been running for roughly > 24 hours. My problem is the huge amount of memory wget has alloted to > it. Since I have it converting links, I was wondering if maybe all the > webpages I have been mirroring are being stored in memory for later > conversion to relative links. No, this is not the case. Yes, Wget can allocate a lot of memory when mirroring huge sites, but I have not been able to detect a memory leaks, and I tried it with many memory debugging tools. So if Wget has allocated 44M so far, it will probably not allocate much more. I still haven't investigated where all the memory goes and if it is a bug, but I plan to do so.
Re: Wget 1.7-pre1 available for testing
Jan Prikryl <[EMAIL PROTECTED]> writes: > It seems that -lsocket is not found as it requires -lnsl for > linking. -lnsl is not detected as it does not contain > `gethostbyname()' function. That's weird. What does libnsl contain if not gethostbyname()? > -AC_CHECK_FUNCS(uname gethostname) >+AC_CHECK_FUNCS(uname) > >+AC_CHECK_FUNCS(gethostname, [], [ >+ AC_CHECK_LIB(nsl, gethostname) >+]) Jan, you must be confusing something here. gethostname() only gets the local host name, and is just a wrapper for the appropriate uname() or sysinfo() call. It has nothing to do with name server lookups, which is what libnsl is supposed to do. >+ linux*) dash_r="-Wl,rpath " ;; This part is correct, except we probably want addition of -Wl, on otherh systems where we use Gcc, too. Perhaps we really should try to write a libtool-based macro named WGET_CHECK_EXTERNAL_LIB.
