Re: Wget 1.7-pre1 available for testing

2001-06-06 Thread Hrvoje Niksic

Jan Prikryl [EMAIL PROTECTED] writes:

 It seems that -lsocket is not found as it requires -lnsl for
 linking. -lnsl is not detected as it does not contain
 `gethostbyname()' function.

That's weird.  What does libnsl contain if not gethostbyname()?

 -AC_CHECK_FUNCS(uname gethostname)
+AC_CHECK_FUNCS(uname)
  
+AC_CHECK_FUNCS(gethostname, [], [
+  AC_CHECK_LIB(nsl, gethostname)
+])

Jan, you must be confusing something here.  gethostname() only gets
the local host name, and is just a wrapper for the appropriate uname()
or sysinfo() call.  It has nothing to do with name server lookups,
which is what libnsl is supposed to do.

+  linux*) dash_r=-Wl,rpath  ;;

This part is correct, except we probably want addition of -Wl, on
otherh systems where we use Gcc, too.

Perhaps we really should try to write a libtool-based macro named
WGET_CHECK_EXTERNAL_LIB.



Re: wget memory leak or expected operation

2001-06-06 Thread Hrvoje Niksic

jolan [EMAIL PROTECTED] writes:

 I'm currently using wget to mirror a rather extensive website. So
 far I've mirrored 10 gigs and counting. wget has been running for roughly
 24 hours. My problem is the huge amount of memory wget has alloted to
 it. Since I have it converting links, I was wondering if maybe all the
 webpages I have been mirroring are being stored in memory for later
 conversion to relative links.

No, this is not the case.

Yes, Wget can allocate a lot of memory when mirroring huge sites, but
I have not been able to detect a memory leaks, and I tried it with
many memory debugging tools.

So if Wget has allocated 44M so far, it will probably not allocate
much more.

I still haven't investigated where all the memory goes and if it is a
bug, but I plan to do so.



Re: Wget 1.7-pre1 available for testing

2001-06-06 Thread Jan Prikryl

 Jan Prikryl [EMAIL PROTECTED] writes:
 
  It seems that -lsocket is not found as it requires -lnsl for
  linking. -lnsl is not detected as it does not contain
  `gethostbyname()' function.
 
 That's weird.  What does libnsl contain if not gethostbyname()?

It seems to contain `gethostname()' ... see the config.log submitted
in one of the previous emails. But it's a very long distance shot: if,
after adding -lsocket -lnsl everything works correctly and if with
-lsocket only the linker complains about missing 'yp_*()' functions
and also missing `gethostname()' and `getdomainname()', I thinks it's
likely that these functions are defined in -lnsl. Of course, if -lnsl
has built in dependency on some other library, the situation might be
completely different.

 Jan, you must be confusing something here.  gethostname() only gets
 the local host name, and is just a wrapper for the appropriate
 uname() or sysinfo() call.  It has nothing to do with name server
 lookups, which is what libnsl is supposed to do.

Probably, but are you sure that this is true on _all_ systems?

 Perhaps we really should try to write a libtool-based macro named
 WGET_CHECK_EXTERNAL_LIB.

Perhaps it would be more portable then.

-- jan

---+
  Jan Prikryl  icq | vr|vis center for virtual reality and
  [EMAIL PROTECTED]  83242638 | visualisation http://www.vrvis.at
---+



Re: Wget 1.7-pre1 available for testing

2001-06-06 Thread Andre Majorel

On 2001-06-06 12:47 +0200, Jan Prikryl wrote:
  Jan Prikryl [EMAIL PROTECTED] writes:
  
   It seems that -lsocket is not found as it requires -lnsl for
   linking. -lnsl is not detected as it does not contain
   `gethostbyname()' function.
  
  That's weird.  What does libnsl contain if not gethostbyname()?
 
 It seems to contain `gethostname()' ... see the config.log submitted
 in one of the previous emails. But it's a very long distance shot: if,
 after adding -lsocket -lnsl everything works correctly and if with
 -lsocket only the linker complains about missing 'yp_*()' functions
 and also missing `gethostname()' and `getdomainname()', I thinks it's
 likely that these functions are defined in -lnsl. Of course, if -lnsl
 has built in dependency on some other library, the situation might be
 completely different.

I've put the output of nm for libsocket and libnsl at

  http://www.teaser.fr/~amajorel/mpras/libnsl.so.nm.gz
  http://www.teaser.fr/~amajorel/mpras/libsocket.so.nm.gz

-- 
André Majorel
Work: [EMAIL PROTECTED]
Home: [EMAIL PROTECTED] http://www.teaser.fr/~amajorel/



wget 1.7, linux, -rpath

2001-06-06 Thread karl

The ssl support is much appreciated in wget 1.7.  But there is a problem
with the configure support that makes it think ssl can't be used, at
least with gcc 2.95.2 on my redhat 6.2 system:

configure:3660: checking for RSA_new in -lcrypto
configure:3679: gcc -o conftest -g -O2  -L/usr/local/openssl/lib -rpath 
/usr/local/openssl/lib  conftest.c -lcrypto   15
gcc: unrecognized option `-rpath'
/usr/local/openssl/lib: file not recognized: Is a directory

This comes from configure:

  decosf* | linux* | irix*) dash_r=-rpath  ;;

-rpath isn't recognized directly by gcc as far as I can tell.
-Wl,-rpath works, though.  I suspect the same would be needed on irix
and decosf but don't know for sure.  -R is recognized directly by gcc on
Solaris, so that part is ok.

Thanks,
karl



Re: wget 1.7, linux, -rpath

2001-06-06 Thread Jan Prikryl

Quoting [EMAIL PROTECTED] ([EMAIL PROTECTED]):

 The ssl support is much appreciated in wget 1.7.  But there is a problem
 with the configure support that makes it think ssl can't be used, at
 least with gcc 2.95.2 on my redhat 6.2 system:

Thanks for the report. Unfortunately the SSL test does not work on
linux at all. Replacing -rpath  with -Wl,rpath  will solve part of
the problems. You may want to try if the attached patch works for
you. Note that this is an unofficial patch and while it may help
solving the SSL check problem, it may break other things.

-- jan

+--
 Jan Prikryl| vr|vis center for virtual reality and visualisation
 [EMAIL PROTECTED] | http://www.vrvis.at
+--


Index: configure.in
===
RCS file: /pack/anoncvs/wget/configure.in,v
retrieving revision 1.17
diff -u -r1.17 configure.in
--- configure.in2001/05/28 22:02:47 1.17
+++ configure.in2001/06/06 05:26:20
@@ -174,8 +174,12 @@
 AC_CHECK_FUNCS(strdup strstr strcasecmp strncasecmp)
 AC_CHECK_FUNCS(gettimeofday mktime strptime)
 AC_CHECK_FUNCS(strerror snprintf vsnprintf select signal symlink access isatty)
-AC_CHECK_FUNCS(uname gethostname)
+AC_CHECK_FUNCS(uname)
 
+AC_CHECK_FUNCS(gethostname, [], [
+  AC_CHECK_LIB(nsl, gethostname)
+])
+
 AC_CHECK_FUNCS(gethostbyname, [], [
   AC_CHECK_LIB(nsl, gethostbyname)
 ])
@@ -205,14 +209,18 @@
 AC_MSG_CHECKING(for runtime libraries flag)
 case $host_os in
   sol2 ) dash_r=-R ;;
-  decosf* | linux* | irix*) dash_r=-rpath  ;;
+  decosf* | irix*) dash_r=-rpath  ;;
+  linux*) dash_r=-Wl,rpath  ;;
   *)
 dash_r=
 for try_dash_r in -R -R  -rpath ; do
   OLD_LDFLAGS=$LDFLAGS
   LDFLAGS=${try_dash_r}/no/such/file-or-directory $LDFLAGS
+  dnl gcc seems to only produce a warning about nonexistent option
+  dnl `-R/no/such/file-or-directory' so the test comes thru
+  dnl (tested with gcc version 3.0 20010308 (prerelease))
   AC_TRY_LINK(, , dash_r=$try_dash_r)
-  LDFLAGS=$ODL_LDFLAGS
+  LDFLAGS=$OLD_LDFLAGS
   test -n $dash_r  break
 done ;;
 esac
@@ -235,9 +243,6 @@
 ssl_all_roots=$with_ssl
   fi
 
-  OLD_LIBS=$LIBS
-  OLD_LDFLAGS=$LDFLAGS
-
   dnl Unfortunately, as of this writing (OpenSSL 0.9.6), the libcrypto
   dnl shared library doesn't record its dependency on libdl, so we
   dnl need to check for it ourselves so we won't fail to link due to a
@@ -245,6 +250,9 @@
   dnl shl_load().
   AC_CHECK_LIB(dl,dlopen)
   AC_CHECK_LIB(dl,shl_load)
+
+  OLD_LIBS=$LIBS
+  OLD_LDFLAGS=$LDFLAGS
 
   ssl_linked=no
 



Re: wget 1.7, linux, -rpath

2001-06-06 Thread wget

On Wed, Jun 06, 2001 at 06:36:26PM +0200, Jan Prikryl wrote:
 Quoting [EMAIL PROTECTED] ([EMAIL PROTECTED]):
 
  The ssl support is much appreciated in wget 1.7.  But there is a problem
  with the configure support that makes it think ssl can't be used, at
  least with gcc 2.95.2 on my redhat 6.2 system:
 
 Thanks for the report. Unfortunately the SSL test does not work on
 linux at all. Replacing -rpath  with -Wl,rpath  will solve part of
 the problems. You may want to try if the attached patch works for
 you. Note that this is an unofficial patch and while it may help
 solving the SSL check problem, it may break other things.

Why don't you steal the --with-ssl option for cURL? It works.

-- 
albert chin ([EMAIL PROTECTED])



Re: win binary

2001-06-06 Thread Richard Travett

Harald Gerber wrote:

 T.Bharath [EMAIL PROTECTED] wrote 
[EMAIL PROTECTED] on 05.06.01
 
  Herold Heiko wrote:
 
   A windows binary for 1.7 is present at
   http://space.tin.it/computer/hherold .
   Heiko
 
  The server seems to reset the connection when the source or binary
  is downloaded
  can you look into it
 
 worked fine from my place.

Lots of resets here too, but I just let WGET get it! However, thats not
much use if you haven't got it yet and are trying to download it! :-)

Rick.
-- 
Richard Travett,Email: [EMAIL PROTECTED] 
  May all your communications be free from electro-magnetic disturbances,
and your days be free from temporal distortions in the space time continuum
 This email does not represent a formal communication from Simoco.



WGET is changing my URL!

2001-06-06 Thread Kohler Roberto

Hello.

I am using wget 1.5.3 to get the URL cache_object://localhost:/info (cache 
statistics from Squid proxy) and the program is changing the URL to 
ftp://cache_object:21/%2Flocalhost/info; (guess because the protocol cache_object is 
not known).

I could not find an option so the program would accept the URL as is.

/Roberto



Re: WGET is changing my URL!

2001-06-06 Thread Jan Prikryl

Quoting Kohler Roberto ([EMAIL PROTECTED]):

 I am using wget 1.5.3 to get the URL
 cache_object://localhost:/info (cache statistics from Squid proxy)
 and the program is changing the URL to
 ftp://cache_object:21/%2Flocalhost/info; (guess because the
 protocol cache_object is not known).

Exactly. Wget knows nothing about cache_object: URL identifier and
it assumes you wanted a FTP one.

 I could not find an option so the program would accept the URL as
 is.

There is no such an option, sorry. You may only use http://,
https://, or ftp:// URLs with wget.

-- jan

+--
 Jan Prikryl| vr|vis center for virtual reality and visualisation
 [EMAIL PROTECTED] | http://www.vrvis.at
+--



Re: How do I get SSL support to work in 1.7?

2001-06-06 Thread wget

On Wed, Jun 06, 2001 at 02:09:12PM -0400, Edward J. Sabol wrote:
 H. I've tried connecting to various sites using https without success.
 I've tried this on both IRIX 6.5.2 and Digital Unix 4.0d.
 
 When I installed OpenSSL 0.9.6a using the default configure options, it
 didn't make any shared libraries, but I have libssl.a and libcrypto.a
 installed, and wget's configure process does find them. (Do I need to install
 the shared libraries?)
 
 For example, I can connect to https://www.apache-ssl.org/ in Netscape just
 fine, but here's what happens when I try with wget 1.7:

 [... debug output removed ...]

Not surprising. Neither IRIX 6.5 nor Tru64 UNIX 4.0D have /dev/random.
So, you need either EGD/PRNGD to provide a substitute for your missing
/dev/random. And, the *client* software has to be configured to
support this. So, if wget doesn't call RAND_egd() from OpenSSL, there
is *nothing* you can do. And, from a quick perusal of wget 1.7, it
doesn't. So, 1.7 is useless for https:// on any system without
/dev/random.

Note that we have added such support to other programs and will
hopefully get time soon to do it to wget. cURL already has support for
EGD/PRNGD so we're just going to steal their solution.

-- 
albert chin ([EMAIL PROTECTED])



Re: wget 1.7 configure errors

2001-06-06 Thread TenThumbs

On Tue, 5 Jun 2001, Jan Prikryl wrote:


 Yes, the SSL detection is broken on some systems. We are working on a
 fix.


Since wget is now a crypto application, I suggest extreme caution here.

On my system, only static SSL libraries exist. It bothers me greatly
that an attacker could corrupt a shared crypto library and, thus,
corrupt all crypto apps. Now your current testing uses rpath which is
irrelevant with static libs, but building with rpath is not benign
because that same attacker (or another one) could plant strange
libraries in the rpath location and wget could be compromised. I would
suggest either testing whether shared SSL libraries exist or providing
configure options so the user can choose.



 Could you provide an example? On my machine, GCC is in /usr/local/bin
 by default (I'm using the GCC 3.0 development version which has some
 bugs, so I prevent to keep tow versions of the compiler). In this
 setup, `CC=/usr/bin/gcc ./configure ... ; make' will build wget with
 the old GCC 2.95.2 just fine.


This is my error. I have a config.site script and a cc definition
crept in by mistake. I'm sorry that I bothered you with this.

In the midst of all this, I built wget with debugging and gcc turned up
these warnings:

rbuf.c:68: warning: implicit declaration of function `ssl_iread'
retr.c:122: warning: implicit declaration of function `ssl_iread'
gen_sslfunc.c:155: warning: implicit declaration of function `select_fd'

Some header files are not included and some are incomplete. Wget works
because the defaults happen to match the actual functions. It is not a
big deal though.

Thanks.

-- 
Jupiter
2001-06-06 19:07:22.660 UTC (JD 2452067.296790)
X  =   0.631953667, Y  =   4.648285018, Z  =   1.977027736 (au)
X' =  -0.007581252, Y' =   0.001124320, Z' =   0.000666516 (au/d)