perl auto-downloader thingy and wget

2003-10-12 Thread Jenny Eastman
Hi,
Im writing a perl auto-downloader that is supposed to maintain the directory
structure. My problem is that with a link like http://www.cisco.com/abcd extracted
from a file i have no know no way of knowing whether what i am downloading
is a file or directory(because when you try to d/l a dir the server sends the default 
page).
As a result i dont know whther to store m the file the server sent with abcd or as 
xyz.html.

Is it true that when you request a dir www.cisco.com/abcd the server sends a redirect 
to
www.cisco.com/abcd/ .How are you handling this problem.
Please advice.
Bye


Re: perl auto-downloader thingy and wget

2003-10-12 Thread Hrvoje Niksic
Jenny Eastman [EMAIL PROTECTED] writes:

 Im writing a perl auto-downloader that is supposed to maintain the
 directory structure. My problem is that with a link like
 http://www.cisco.com/abcd extracted from a file i have no know no
 way of knowing whether what i am downloading is a file or
 directory(because when you try to d/l a dir the server sends the
 default page).  As a result i dont know whther to store m the file
 the server sent with abcd or as xyz.html.

 Is it true that when you request a dir www.cisco.com/abcd the server
 sends a redirect to www.cisco.com/abcd/ .How are you handling this
 problem.

Wget handles this by changing the file name to abcd/index.html if/when
the server sends the redirect.  Therefore the file abcd will be
saved as abcd, while a directory abcd will be saved as
abcd/index.html.



Greetings, and a question

2003-10-12 Thread Robert Poole
Greetings!  I'm new to this mailing list, though not to wget.  I've 
successfully compiled wget on a variety of platforms in the past, but I 
seem to be having a problem, and I'm hoping someone can help me.

I am attempting to build wget 1.8.2 on one of the new G5 PowerMac 
machines (dual processor) from Apple.  I also compiled OpenSSL 0.9.7c 
and installed it; the tests all ran without a hitch, so I'm assuming 
that OpenSSL is compiled and running correctly.

When building wget, the link phase bombs with the following error 
message:
gcc -O2 -Wall -Wno-implicit -o wget cmpt.o connect.o cookies.o 
fnmatch.o ftp.o ftp-basic.o ftp-ls.o ftp-opie.o getopt.o hash.o 
headers.o host.o html-parse.o html-url.o http.o init.o log.o main.o 
gen-md5.o netrc.o progress.o rbuf.o recur.o res.o retr.o safe-ctype.o 
snprintf.o gen_sslfunc.o url.o utils.o version.o  -L/usr/local/lib 
-lssl -lcrypto
ld: Undefined symbols:
_OPENSSL_add_all_algorithms_noconf
make[1]: *** [wget] Error 1
make: *** [src] Error 2

I've successfully built wget with OpenSSL support previously on a G4 
PowerMac, so I'm wondering if there's an issue with the G5's compiler 
(gcc 3.3 with G5 optimizations) or if there's something else wrong.  
Perhaps I forgot to configure something correctly with OpenSSL or wget?

Any suggestions?

Best Regards,
Rob Poole
[EMAIL PROTECTED]


Compile and link problems with wget 1.9 beta5

2003-10-12 Thread Robert Poole
After ploughing through the archives of this mailing list, looking for 
additional clues why wget 1.8.2 wasn't linking correctly, I found that 
wget 1.9 beta 5 was released recently.  I downloaded the source code 
for wget 1.9 beta 5 and am getting the same link problems I was getting 
with 1.8.2:

/bin/sh ../libtool --mode=link gcc -O2 -Wall -Wno-implicit  -o wget  
cmpt.o connect.o convert.o cookies.o ftp.o ftp-basic.o ftp-ls.o 
ftp-opie.o getopt.o hash.o headers.o host.o html-parse.o html-url.o 
http.o init.o log.o main.o gen-md5.o netrc.o progress.o rbuf.o recur.o 
res.o retr.o safe-ctype.o snprintf.o gen_sslfunc.o url.o utils.o 
version.o -lssl -lcrypto
mkdir .libs
gcc -O2 -Wall -Wno-implicit -o wget cmpt.o connect.o convert.o 
cookies.o ftp.o ftp-basic.o ftp-ls.o ftp-opie.o getopt.o hash.o 
headers.o host.o html-parse.o html-url.o http.o init.o log.o main.o 
gen-md5.o netrc.o progress.o rbuf.o recur.o res.o retr.o safe-ctype.o 
snprintf.o gen_sslfunc.o url.o utils.o version.o  -lssl -lcrypto
ld: Undefined symbols:
_OPENSSL_add_all_algorithms_noconf
make[1]: *** [wget] Error 1
make: *** [src] Error 2

I've tried to determine if my OpenSSL installation was built wrong, but 
as far as I can determine, it's OK.  That doesn't mean that there's 
nothing wrong with OpenSSL on this platform, but so far, this link 
error has been the only problem I've encountered.

The platform is a dual-processor G5 running Mac OS X 10.2.8 with the 
latest developer tools installed (gcc 3.3 with G5 optimizer settings 
available, although I haven't used any of the command line switches to 
turn those optimizations on).

Help?

Best Regards,
Rob Poole
[EMAIL PROTECTED]


RE: Compile and link problems with wget 1.9 beta5

2003-10-12 Thread Post, Mark K
Do you see the missing symbol when you do an nm -D command against either
libssl.so or libcrypto.so?  (It shows up on my Linux system in
libcrypto.so.)


Mark Post

-Original Message-
From: Robert Poole [mailto:[EMAIL PROTECTED]
Sent: Sunday, October 12, 2003 2:23 PM
To: [EMAIL PROTECTED]
Subject: Compile and link problems with wget 1.9 beta5


After ploughing through the archives of this mailing list, looking for 
additional clues why wget 1.8.2 wasn't linking correctly, I found that 
wget 1.9 beta 5 was released recently.  I downloaded the source code 
for wget 1.9 beta 5 and am getting the same link problems I was getting 
with 1.8.2:

/bin/sh ../libtool --mode=link gcc -O2 -Wall -Wno-implicit  -o wget  
cmpt.o connect.o convert.o cookies.o ftp.o ftp-basic.o ftp-ls.o 
ftp-opie.o getopt.o hash.o headers.o host.o html-parse.o html-url.o 
http.o init.o log.o main.o gen-md5.o netrc.o progress.o rbuf.o recur.o 
res.o retr.o safe-ctype.o snprintf.o gen_sslfunc.o url.o utils.o 
version.o -lssl -lcrypto
mkdir .libs
gcc -O2 -Wall -Wno-implicit -o wget cmpt.o connect.o convert.o 
cookies.o ftp.o ftp-basic.o ftp-ls.o ftp-opie.o getopt.o hash.o 
headers.o host.o html-parse.o html-url.o http.o init.o log.o main.o 
gen-md5.o netrc.o progress.o rbuf.o recur.o res.o retr.o safe-ctype.o 
snprintf.o gen_sslfunc.o url.o utils.o version.o  -lssl -lcrypto
ld: Undefined symbols:
_OPENSSL_add_all_algorithms_noconf
make[1]: *** [wget] Error 1
make: *** [src] Error 2

I've tried to determine if my OpenSSL installation was built wrong, but 
as far as I can determine, it's OK.  That doesn't mean that there's 
nothing wrong with OpenSSL on this platform, but so far, this link 
error has been the only problem I've encountered.

The platform is a dual-processor G5 running Mac OS X 10.2.8 with the 
latest developer tools installed (gcc 3.3 with G5 optimizer settings 
available, although I haven't used any of the command line switches to 
turn those optimizations on).

Help?

Best Regards,
Rob Poole
[EMAIL PROTECTED]


Re: Compile and link problems with wget 1.9 beta5

2003-10-12 Thread Hrvoje Niksic
Robert Poole [EMAIL PROTECTED] writes:

 gcc -O2 -Wall -Wno-implicit -o wget cmpt.o connect.o convert.o
 cookies.o ftp.o ftp-basic.o ftp-ls.o ftp-opie.o getopt.o hash.o
 headers.o host.o html-parse.o html-url.o http.o init.o log.o main.o
 gen-md5.o netrc.o progress.o rbuf.o recur.o res.o retr.o safe-ctype.o
 snprintf.o gen_sslfunc.o url.o utils.o version.o  -lssl -lcrypto
 ld: Undefined symbols:
 _OPENSSL_add_all_algorithms_noconf

As far as I can tell, the call to SSLeay_add_all_algorithms in
gen_sslfunc.c expands to OPENSSL_add_all_algorithms_noconf (without
the leading underscore).  I really don't know why it fails to link for
you; if I had to guess, I'd try checking two things.

It could be a mismatch between OpenSSL headers and the installed
libraries.  Please check that the headers Wget is picking up and the
libraries it's linking with are from the same OpenSSL version.

Or, it could be a problem with libtool, which Wget uses to be able to
(easily) link with external libraries.  Try running the link line
manually, outside make and without libtool.  See if linking works
then.

For better or worse, noone has reported this problem so far.  It's
either specific to your architecture or to your setup.  Do other SSL
programs, such as curl, compile successfully?



Re: Compile and link problems with wget 1.9 beta5

2003-10-12 Thread Robert Poole
I did a little digging.  Even though I compiled openssl, and it 
installed things under /usr/local, there are other libssl and libcrypto 
versions on the system; so I'm kind of scratching my head, trying to 
figure out how to force wget to build against the stuff that I actually 
compiled, instead of something else.  I can only guess that somehow the 
linker is finding the wrong libraries and is blowing up when it tries 
to link against them.

I examined libssl.a and libcrypto.a, and the _ 
OPENSSL_add_all_algorithms_noconf symbol is defined in libcrypto.a that 
was installed when I built openssl.  However, there's a libssl.so and a 
libcrypto.so elsewhere on the system, and a libssl.dylib and 
libcrypto.dylib elsewhere as well...

It's been quite some time since I've had to dig through a linker 
problem like this, so I'm kind of lost.  I'd be willing to let someone 
I could trust ssh into my system and dig around and see what they see.

As far as I can tell, the only headers for openssl are the ones that I 
installed with openssl 0.9.7c.

I'm trying to search on the net, but nobody seems to be compiling the 
code from scratch; almost everyone is using fink.

Best Regards,
Rob Poole
[EMAIL PROTECTED]
On Sunday, October 12, 2003, at 01:51 PM, Hrvoje Niksic wrote:

Robert Poole [EMAIL PROTECTED] writes:

gcc -O2 -Wall -Wno-implicit -o wget cmpt.o connect.o convert.o
cookies.o ftp.o ftp-basic.o ftp-ls.o ftp-opie.o getopt.o hash.o
headers.o host.o html-parse.o html-url.o http.o init.o log.o main.o
gen-md5.o netrc.o progress.o rbuf.o recur.o res.o retr.o safe-ctype.o
snprintf.o gen_sslfunc.o url.o utils.o version.o  -lssl -lcrypto
ld: Undefined symbols:
_OPENSSL_add_all_algorithms_noconf
As far as I can tell, the call to SSLeay_add_all_algorithms in
gen_sslfunc.c expands to OPENSSL_add_all_algorithms_noconf (without
the leading underscore).  I really don't know why it fails to link for
you; if I had to guess, I'd try checking two things.
It could be a mismatch between OpenSSL headers and the installed
libraries.  Please check that the headers Wget is picking up and the
libraries it's linking with are from the same OpenSSL version.
Or, it could be a problem with libtool, which Wget uses to be able to
(easily) link with external libraries.  Try running the link line
manually, outside make and without libtool.  See if linking works
then.
For better or worse, noone has reported this problem so far.  It's
either specific to your architecture or to your setup.  Do other SSL
programs, such as curl, compile successfully?




Re: some wget patches against beta3

2003-10-12 Thread Arkadiusz Miskiewicz
On Tuesday 07 of October 2003 12:08, Hrvoje Niksic wrote:
 Thanks!

btw. looking into test4 I see that autoconf conception is used in weird way. 
Normally aclocal.m4 is autogenerated by aclocal command (and it includes 
sources of macros from /usr/share/autoconf/autoconf/*.m4 and from local 
acinclude.m4).

You are seem to put all macros (even local macros!) inside of aclocal.m4. This 
is bad because it doesn't allow to regenerate autoconf/aclocal stuff in case 
when it's needed. aclocal command will overwrite aclocal.m4 and all local 
macros will be lost.

We at PLD are right now patching wget to move all local macros into 
acinclude.m4 but it would be better if that was done properly at wget 
maintainers level :)

ps. I see that beta5 is there so time to look into this one... argh, same 
things as in test4
-- 
Arkadiusz MikiewiczCS at FoE, Wroclaw University of Technology
arekm.pld-linux.org AM2-6BONE, 1024/3DB19BBD, arekm(at)ircnet, PLD/Linux



Re: some wget patches against beta3

2003-10-12 Thread Hrvoje Niksic
Arkadiusz Miskiewicz [EMAIL PROTECTED] writes:

 On Tuesday 07 of October 2003 12:08, Hrvoje Niksic wrote:
 Thanks!

 btw. looking into test4 I see that autoconf conception is used in
 weird way. Normally aclocal.m4 is autogenerated by aclocal command

Normally only if you're also using Automake.  Wget doesn't use
Automake and therefore has no use for `aclocal'.

 You are seem to put all macros (even local macros!) inside of
 aclocal.m4.

Yes, local macros are in aclocal.m4 -- that's how it got its name.
Autoconf manual says:

To create a `configure' script with Autoconf, you need to write an
Autoconf input file `configure.ac' (or `configure.in') and run
`autoconf' on it.  If you write your own feature tests to
supplement those that come with Autoconf, you might also write
files called `aclocal.m4' and `acsite.m4'.

Then, later:

Those files [`aclocal.m4' and `acsite.m4'] can contain your site's
or the package's own Autoconf macro definitions.

`acinclude.m4' is not even mentioned, so I must assume it's another
Automake thing.

 This is bad because it doesn't allow to regenerate autoconf/aclocal

The macros in aclocal.m4 are not autogenerated, therefore there is no
need for regeneration.  Am I missing something?

 We at PLD are right now patching wget to move all local macros into
 acinclude.m4 but it would be better if that was done properly at
 wget maintainers level :)

Could you explain the motivation behind the move from aclocal.m4 to
acinclude.m4?



Re: How do you pronounce Hrvoje?

2003-10-12 Thread Tony Lewis
Hrvoje and I have had an off-list dialogue about this subject. We've settled
on HUR-voy-eh as the closest phonetic rendition of his name for English
speakers. It helps to remember that the r is rolled.

Tony



Re: Compile and link problems with wget 1.9 beta5

2003-10-12 Thread Robert Poole
I tried what you suggested, and of course did a make clean to make 
sure everything got recompiled from scratch, but the link still bombs 
with the same error message.  I even copied the libssl.a and 
libcrypto.a files to /usr/local/ssl/lib (which was empty up to this 
point, very strange), and then ran the configure script as follows:

./configure --with-ssl=/usr/local/ssl

Still nothing.  Hrm.

On a hunch, rebuilt OpenSSL without specifying any directories, 
reconfiguring to use all the default values.  This time, 
/usr/local/ssl/lib has everything you'd expect to be in it.

But, wget still won't link correctly.

The really frustrating thing is, I had successfully built wget 1.8.2 
against openssl 0.9.7 on an iBook and a G4 Cube with a virtually 
identical setup.  The only difference is that the G5 is using an 
updated version of the dev tools with support for G5-specific 
optimizations (which I'm not using at the moment).  I'm tempted to 
regress to the older version of the development tools and see if that 
helps; if so, then the linker got broken in the new version of the 
development tools.

Unfortunately, although I attempted to regress to the previous version 
of the dev tools (which would roll me back to gcc 3.1), the linking 
still seems to be broken; so for now, I've returned to using the latest 
dev tools (gcc 3.3, etc.) and am going to submit a bug report to Apple 
developer relations.

I haven't given up hope of getting this thing built -- but considering 
that I was able to build and link these packages successfully on a 
lowly iBook and a G4 Cube, both of which were running the older dev 
tools suite, I suspect the problem is the newer compiler and linker.

Best Regards,
Rob Poole
On Sunday, October 12, 2003, at 03:55 PM, Hrvoje Niksic wrote:

Robert Poole [EMAIL PROTECTED] writes:

I did a little digging.  Even though I compiled openssl, and it
installed things under /usr/local, there are other libssl and
libcrypto versions on the system; so I'm kind of scratching my head,
trying to figure out how to force wget to build against the stuff
that I actually compiled, instead of something else.
If I remember correctly, OpenSSL installs under /usr/local/ssl by
default.  Wget, on the other hand, tries to be a decent citizen and
first attempt to look for libraries in the standard locations, where
standard means whatever is in the compiler's include/library search
path.  To quote configure.in:
dnl OpenSSL's root was not specified, so we have to guess.  First
dnl try the system default location, then /usr/local/ssl (where
dnl OpenSSL installs by default), then /usr/local (traditional
dnl choice for installation root), then /opt.
ssl_all_roots=system-default /usr/local/ssl /usr/local /opt
You can specify OpenSSL root with `--with-ssl=ROOT'.  For example,
`./configure --with-ssl=/usr/local/ssl' will prefer the libraries in
/usr/local/ssl/lib and includes in /usr/local/ssl/include over any
other.
I examined libssl.a and libcrypto.a, and the _
OPENSSL_add_all_algorithms_noconf symbol is defined in libcrypto.a
that was installed when I built openssl.  However, there's a
libssl.so and a libcrypto.so elsewhere on the system, and a
libssl.dylib and libcrypto.dylib elsewhere as well...
[...]
As far as I can tell, the only headers for openssl are the ones that
I installed with openssl 0.9.7c.
That almost confirms my suspicion.  I'm guessing that Wget links with
an older libssl.so (because they're in the compiler's library search
path), but finds the includes from the newer version (because those
are the only SSL includes on the system, and they're in the compiler's
include path).
Please try using `./configure --with-ssl=/usr/local' or `./configure
--with-ssl=/usr/local/ssl' and see if the problem goes away.