Georg Bauhaus [EMAIL PROTECTED] writes:
Hrvoje Niksic [EMAIL PROTECTED] writes:
What do you think about this patch:
+ if (USLEEP_usec 0) \
+usleep (USLEEP_usec);\
+} while (0)
Could you change this to have proper number conversions
I guess I don't see the usefulness of the protocol independence, as
the term is being applied here. Wget is not protocol-independent, it
uses TCP and depends on it in various places of the code.
The only advantage is arguably clearer code in functions like
`lookup_host', but even that does not
Kazu Yamamoto ($B;3K\OBI'(B) [EMAIL PROTECTED] writes:
(B
(B Hello all,
(B
(B Thank you for your comments, Hrvoje.
(B
(B From: Hrvoje Niksic [EMAIL PROTECTED]
(B Subject: Re: IPv6 support of wget v 1.9.1
(B
(B I guess I don't see the usefulness of the protocol independence, as
(B
Voelker Bernhard [EMAIL PROTECTED] writes:
I recognized that the password passed to wget with the option
--http-passwd= can be seen as clear text with tools like ps and
top.
Yes. I don't think there is a portable and reliable way to prevent
this. You can work around it, though. Put your
Is it my weird setup, or has spam reemerged on this list? If others
are seeing the spams that seem to originate from the list, I'll try to
contact the admins.
Kazu Yamamoto ($B;3K\OBI'(B) [EMAIL PROTECTED] writes:
(B
(B My current motivation is style improvement. Since people
(B misunderstood the new style, which you can see in Thomas's summary,
(B we would like to show that it was misunderstanding.
(B
(BYou are right. It was indeed wrong to
Greg Underwood [EMAIL PROTECTED] writes:
Yeah, I got some spam from the list.
It seems that the spam are in fact viruses, most likely originating
from infected subscribers. A sunsite staff member told me they have
just updated the virus scanner, which should make the problem go away.
Christian Biere [EMAIL PROTECTED] writes:
[...]
Hey, these are great catches! Thanks for taking the time to
investigate the code.
I've discovered 2 buffer overflows which affects at least people using
2 certain locales but nobody who's using wget built with --disable-nls.
I've bumped the
YOSHIFUJI Hideaki / [EMAIL PROTECTED](B [EMAIL PROTECTED] writes:
(B
(B It is likely there're no users infected since the "From:" field is very
(B likely forged by viruses and spammers.
(B
(BI think the mailing list software looks at the "From " line, not the
(B"From:" header field, but I
Greg Underwood [EMAIL PROTECTED] writes:
I took a peek at my cookies while logging into the site in a regular
browser. It definitely adds a session cookie when I log in,
I think your problem should be solvable with `--keep-session-cookies'.
The server will have no way of knowing that the two
[ This discussion is about a patch that determines the screen width on
Windows console. ]
Herold Heiko [EMAIL PROTECTED] writes:
Note: for a complete look-and-feel similar to the unix version we
still need a detection when the size changes (on unix this is done
with received_sigwinch in
This is a bug -- regardless of whether it supports large files or not,
Wget shouldn't crash when dealing with them.
I plan to look into large file support for the next version of Wget.
It sounds to me like you could do the equivalent with a simple shell
script. For example:
while read url
do
wget --limit-rate=2k $url
# Your commands go here.
done URL-LIST-FILE
problems.
2004-01-29 Hrvoje Niksic [EMAIL PROTECTED]
* utils.c (determine_screen_width): Return 0 if not running on
Windows or on a TIOCGWINSZ-capable system.
Index: src/utils.c
===
RCS file: /pack/anoncvs/wget/src
Greg Underwood [EMAIL PROTECTED] writes:
On Tuesday 27 January 2004 05:23 pm, Hrvoje Niksic wrote:
Greg Underwood [EMAIL PROTECTED] writes:
I took a peek at my cookies while logging into the site in a regular
browser. It definitely adds a session cookie when I log in,
I think your
Roger Binns [EMAIL PROTECTED] writes:
Has anyone compiled wget for Windows CE (I have PocketPC 2002 Phone
Edition)?
Not that I'm aware of. I don't know what kind of software is required
for such compilation.
Note that it is not hard to compile Wget, so if you have a development
environment
Gerald Oskoboiny [EMAIL PROTECTED] writes:
Hi, the man page for wget 1.8.2 says:
-q
--quiet
Turn off Wget's output.
-nv
--non-verbose
Non-verbose output---turn off verbose without being completely
quiet (use -q for that), which means that error messages
Could someone check how other Windows console programs check
interactive size changes?
For now it's probably OK to check the progress bar size at the start.
After all, the size was hardcoded to 80 before David's patch.
Noname NoLast [EMAIL PROTECTED] writes:
reasonable so that it can save them) otherwise it will
give me error messages such as:
Cannot write to
'http://www.osnews.com/comment.php?news_id=5602offset=105rows=120'
I don't understand this error message. Wget should never try to write
to a
Noname NoLast [EMAIL PROTECTED] writes:
Is there any way to download the structure (what files, directories,
size of files, etc.) on an FTP site without downloading the actual
files?
I think `-R *' should work.
Please try generating the debug output, using the `-d' option. That
way we'll see what kind of incorrect response Wget is getting, and
in response to which FTP command.
francois eric [EMAIL PROTECTED] writes:
after some test:
bug is when: ftp, with username and password, with bind address specifyed
bug is not when: http, ftp without username and password
looks like memory leaks. so i made some modification before bind:
src/connect.c:
--
...
/*
Jens Rösner [EMAIL PROTECTED] writes:
PS: One note to the manual editor(s?): The -e switch could be
(briefly?) mentioned also at the wgetrc commands paragraph. I
think it would make sense to mention it there again without
clustering the manual too much. Currently it is only mentioned in
I don't see any obvious reason why timestamping would work in one
case, but not in the other. One possible explanation might be that
the second server does not provide correct time-stamping data. Debug
output (with the `-d' switch) might shed some light on this.
Dan LeGate [EMAIL PROTECTED] writes:
What I'm wanting to do is Mirror a site, but keep backups of any
local files that get replaced because newer versions are being
downloaded.
You might want to try the undocumented option `--backups', which does
what you want, i.e. forces the use of numbered
Nick Hogg [EMAIL PROTECTED] writes:
Hi...
Stuck on a problem with wget.
Am using --ignore-length -o wget.log -R
jpg,jpeg,gif,mpeg,mpg,avi,au,ps,pdf,mp3,tmp,bmp,png,tiff,mov,wmv,qt,wav,ogg,rm,ram,doc,ppt,xls,zip,tar,gz,bz2,rar,arj,swf
--random-wait --recursive --no-parent
Nick Hogg [EMAIL PROTECTED] writes:
It tries 4 times, getting a read error each time. Have run it
several times and every time it fails, don't think that making the
tries infinite will make it run!!
How about removing the ten-second timeout you're also specifying? If
you or the server is on
Nick Hogg [EMAIL PROTECTED] writes:
Increased timeout to 900. No difference unfortunately. Still getting
the same errors.
Did it wait for 900 seconds before timeouting? Generally, does the
error occur at the same byte position each time?
Nick Hogg [EMAIL PROTECTED] writes:
At 11:07 06/02/2004, Hrvoje Niksic wrote:
Did it wait for 900 seconds before timeouting? Generally, does the
error occur at the same byte position each time?
Seems to...It seems to happen on the same byte. But its happening on a
few domains I am trying
to remote
addresses, or it would have been noticed ages ago.
Thanks for the pointer. This patch should fix the problem in the CVS
version:
2004-02-06 Hrvoje Niksic [EMAIL PROTECTED]
* connect.c (sockaddr_set_data): Zero out
sockaddr_in/sockaddr_in6. Apparently BSD-derived stacks
Dan LeGate [EMAIL PROTECTED] writes:
Thanks Hrvoje,
Yes, --backups does seem to be what I want.
However, when I run the following command-line:
wget -m --backups -A gif,jpg,js,inc,css -o wget.log
ftp://userid:[EMAIL PROTECTED]/dir1/dir2
It returns:
wget: backups: Invalid specification
I've now installed this fix, thanks.
Jens Rösner [EMAIL PROTECTED] writes:
Hi Hrvoje!
PS: One note to the manual editor(s?): The -e switch could be
(briefly?) mentioned also at the wgetrc commands paragraph. I
think it would make sense to mention it there again without
clustering the manual too much. Currently it is only
Jens Rösner [EMAIL PROTECTED] writes:
distribution. See
http://cvs.sunsite.dk/viewcvs.cgi/*checkout*/wget/PATCHES?rev=1.5
Thanks, I tried to understand that. Let's see if I understood it.
You're close. You forgot the `-u' option to diff (very important),
and you snipped the beginning of
Thanks tracing this one. It would never have occurred to me that the
file name c:\/foo could cause such a problem.
I see two different bugs here:
1. The routine that merges the .netrc file name with the directory
name should be made aware of Windows, so that it doesn't append
another
Juhana Sadeharju [EMAIL PROTECTED] writes:
Hello.
What goes wrong in the following? (I will read replies from the list
archives.)
% wget http://www.maqamworld.com/
--16:59:21-- http://www.maqamworld.com:80/
= `index.html'
Connecting to www.maqamworld.com:80...
Thanks for the cleanup! I've now installed this patch in CVS.
I've now applied this patch (with minor editing), thanks.
chatiman [EMAIL PROTECTED] writes:
I'm trying to download a robots.txt protexted directory and I'm having the
following problem:
- wget downloads the files but delete them after they are downloaded with
the following :message (translated from french):
Destroyed file because it must be
patrick robinson [EMAIL PROTECTED] writes:
That message has nothing to do with robots.txt, it means that you
have rejected the file using the `-R' or equivalent option.
Here you go again with this IMHO stupid implemented option.
Why thank you.
I'm using it too but on some suffixes it acts
The SOCKS support was added to Wget at a very early date and was
unmaintained for a long time, up to the point where it wouldn't build
at all. Since I didn't have the SOCKS library installed and noone
even reported the failures, I decided to remove the `--with-socks'
option from configure until
Craig Sowadski [EMAIL PROTECTED] writes:
Ok, I have attached a new patch that moves the local time into
http_stat. I am also sending this to [EMAIL PROTECTED] for others to
try out. It seems to work great for me.
I was about to apply this patch, but noted a small problem:
+ if
Dan LeGate [EMAIL PROTECTED] writes:
When using the following syntax:
wget -m -o wget.log --backups 1
ftp://user:[EMAIL PROTECTED]/dir1/dir2
is there a way to specify the local directory it copies to and have
it NOT create a www.company.com parent directory?
Sure, just use `-nH'.
Thanks for the modification, I've now applied the patch to my
workspace and given it some testing. There's one thing I don't quite
understand. Before the patch, Wget's timestamping was based on
analyzing the Last-Modified header, working like this:
1. Send a HEAD request and get the response.
Tapan Mehta [EMAIL PROTECTED] writes:
1) If there is a connection loss then the next time
does wget automatically start from the point it had stopped or is
there a switch I need to apply to enable this option
No special option is necessary, Wget continues where it left off in
case of
David Fritz [EMAIL PROTECTED] writes:
Attached is a patch that removes the ws_help() function from
mswindows.[ch] and the call to it from print_help() in main.c. Also
attached is an alternate patch that will fix ws_help(), which I
neglected to update when I changed ws_mypath().
I find this
Does the same thing happen with Wget 1.9.1? I remember some bugs in
that area that have been fixed since 1.8.2.
Craig Sowadski [EMAIL PROTECTED] writes:
1. I can implement a patch like my original that only uses the
if-modified-since when the last-modified field is excluded from the
head-only request.
Isn't that what your most recent patch implements?
2. Send the if-modified-since request, then get
Michael Bingel [EMAIL PROTECTED] writes:
I thought great, problem solved, but Cygwin wget version 1.9 does
not accept -O, although the NEWS file does not state removal of
this feature.
-O is still there. How exactly are you invoking Wget and what error
message is it printing?
Your official
David Fritz [EMAIL PROTECTED] writes:
But, I'd guess you probably had a non-option argument before -O.
For a while now, the version of getopt_long() included with Cygwin
has had argument permutation disabled by default.
What on Earth were they thinking?! I've never considered the
possibility
D Richard Felker III [EMAIL PROTECTED] writes:
The following code in url.c makes it impossible to request urls that
contain multiple slashes in a row in their query string:
[...]
That code is removed in CVS, so multiple slashes now work correctly.
Think of something like
D Richard Felker III [EMAIL PROTECTED] writes:
Think of something like http://foo/bar/redirect.cgi?http://...
wget translates this into: [...]
Which version of Wget are you using? I think even Wget 1.8.2 didn't
collapse multiple slashes in query strings, only in paths.
I was using
Gisle Vanem [EMAIL PROTECTED] writes:
We could also fix this by calling ws_changetitle() unconditionally.
Should the title bar be affected by verbosity?
IMHO yes, Quiet is quiet.
I agree.
Applied now, thanks.
Gisle Vanem [EMAIL PROTECTED] writes:
ws_percenttitle() should not be called in quiet mode since
ws_changetitle() AFAICS is only called in verbose mode. That caused
an assert in mswindows.c. An easy patch:
[...]
I've applied this patch, thanks.
The whole matter of conversion of / to /index.html on the file
system is a hack. But I really don't know how to better represent
empty trailing file name on the file system.
D Richard Felker III [EMAIL PROTECTED] writes:
The request log shows that the slashes are apparently respected.
I retried a test case and found the same thing -- the slashes were
respected.
OK.
Then I remembered that I was using -i. Wget seems to work fine with
the url on the command
Jamie Zawinski [EMAIL PROTECTED] writes:
- that pulled in krb5-devel (good so far)
- then I had to do ./configure --with-ssl=/usr/kerberos
because it needs /usr/kerberos/include/krb5.h
I had the same problem on Red Hat 9, and solved it the same way. I
think that was actually a bug
--troms-zua.no ?
The Host header is (I think) not URL-escaped, so we can simply send
the 8-bit characters as we received them.
Here's a patch; please let me know if it works for you.
2004-03-19 Hrvoje Niksic [EMAIL PROTECTED]
* url.c (url_parse): Decode %HH sequences in host name
Gisle Vanem [EMAIL PROTECTED] writes:
Hrvoje Niksic [EMAIL PROTECTED] said:
If it where not for the Host: header, the name could remain
un-escaped. I don't know what the standard say about this case.
Should the header contain Host:www.xn--troms-zua.no ?
The Host header is (I think
david-zhan [EMAIL PROTECTED] writes:
WGET is popular FTP software for UNIX. But, after the files were
downloaded for the first time, WGET always use the date and time,
matching those on the remote server, for the downloaded files. If
WGET is executed in temporary directory in which the files
Rick Goyette [EMAIL PROTECTED] writes:
The local and remote files have different sizes, which I thought
(after reading the man page) should flag wget to grab it. But it
does not.
It should. Do you use HTTP or FTP to get the file? Can you post a
debug log (possibly edited for confidential
Scott Chapman [EMAIL PROTECTED] writes:
--22:25:13-- http://%253/
= `%253/index.html'
Resolving %253... failed: Host not found.
--22:25:13-- http://%254/
= `%254/index.html'
Resolving %254... failed: Host not found.
It converted what it snagged. I'd like to know
Michael Dolan [EMAIL PROTECTED] writes:
Greetings,
How does one copy a remote subdir to a local subdir without getting the
entire path?
For example:
local dir - /home/user/database
remote dir - /pub/this/database
I want the remote /database to to be copying (update) my local
For now I'd start with applying David's patch, so that people can test
its functionality. It is easy to fix the behavior of `wget -q -b'
later.
David, can I apply your patch now?
=?utf-8?B?0JLQsNGB0LjQu9C10LLRgdC60LjQuSDQodC10YDQs9C10Lk=?= [EMAIL PROTECTED]
writes:
url: list.php?cat_id=Àêñåññóàðû è êàáåëè
Wget 1.8.2 on freebsd4.7. escape url befor save to disk.
Wget 1.9.1 on freebsd4.7. not escape url befor save to disk.
How can I use 1.9.1 with escaping?
Thanks for the patch, I've now applied it to CVS.
You might want to add a comment in front of fake_fork() explaining
what it does, and why. The comment doesn't have to be long, only
several sentences so that someone reading the code later understands
what the heck a fake fork is and why we're
Juhana Sadeharju [EMAIL PROTECTED] writes:
Command: wgetdir http://liarliar.sourceforge.net;.
Problem: Files are named as
content.php?content.2
content.php?content.3
content.php?content.4
which are interpreted, e.g., by Nautilus as manual pages and are
displayed as plain texts. Could
David Fritz [EMAIL PROTECTED] writes:
Ok, I'll submit a patch latter tonight. Do you think it would be a
good idea to include README.fork in windows/ (the directory with the
Windows Makefiles, etc. in it)?
I don't think that's necessary. Simply explain how the fork emulation
works in
Thanks for the report; I've now asked the GNU people to redirect
bug-wget to the correct address.
It's a bug caused by an oversight, thanks for the report. Please let
me know if this patch corrects the problem:
2004-03-30 Hrvoje Niksic [EMAIL PROTECTED]
* http.c (gethttp): Send the Proxy-Authorization header over
non-SSL connections too.
Index: src/http.c
Daniel Stenberg [EMAIL PROTECTED] writes:
On Tue, 30 Mar 2004, Hrvoje Niksic wrote:
* http.c (gethttp): Send the Proxy-Authorization header over
non-SSL connections too.
I couldn't really tell from this diff, but I thought I'd remind you:
If you are using SSL over a proxy, you
I've now applied this patch.
none none [EMAIL PROTECTED] writes:
(URL changed for privacy)
$ wget http://1.2.3.4/?.file
--00:00:00-- http://1.2.3.4/%E9.file
= `?.file'
Connecting to 1.2.3.4:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
00:00:00 ERROR 403: Forbidden.
$ wget -V
Noname NoLast [EMAIL PROTECTED] writes:
Is there any way to download files from an FTP site depending on a
certain string in their names. For example, download all files that
contain # or sharp from ftp://72.0.0.1/Tutorials/
This might work:
wget ftp://72.0.0.1/Tutorials/*%23*;
Noname NoLast [EMAIL PROTECTED] writes:
I'm trying to download a file from an FTP site that contains a # in
it's name.
# is a URL metacharacter. Use %23 instead:
wget ftp://194.85.35.67/C%23 Tutorial.rar
BTW --user-agent and --referer don't do anything for FTP downloads.
I'm afraid Wget doesn't understand JavaScript. As your example
demonstrates, it is impossible to extract URLs from JavaScript by
merely parsing it -- you need to actually execute it.
It seems the site has an anti-leech protection designed to throw off
Wget. Adding something like `-U Mozilla' seems to make it work.
Ilya N. Golubev [EMAIL PROTECTED] writes:
A future version of
Wget will probably parse comments in a non-compliant fashion, by
considering everything between !-- and -- to be a comment
Installed 1.9.1 (unfortunately, there are no good binary rpms still;
this is why ran it uninstalled so
Bettinger, Imelda [EMAIL PROTECTED] writes:
We moved from web based authentication to form based last year and I
can't figure out how to get wget to get past the authenication. Most
of our content is behind the authentication.
By form based authentication I assume you mean that you enter your
JFL [EMAIL PROTECTED] writes:
I would like to have Wget download several files simultaneously.
Any solutions for that?
You can always start several instances of Wget.
Lawrance, Mark [EMAIL PROTECTED] writes:
I am unable to get wget to work via a proxy for HTTPS sites.
It does work via proxy for HTTP
It does work with HTTPS NOT through proxy
Any ideas? Should this work?
It works in the CVS version of Wget. Please try it and see if it
works
Daniel Stenberg [EMAIL PROTECTED] writes:
On Mon, 10 May 2004, [iso-8859-2] Dra?en Ka?ar wrote:
* Change most (all?) occurrences of `long' in the code to `off_t'. Or
should we go the next logical step and just use uintmax_t right
away?
Just use off_t.
... but Windows has no
Axel Pettinger [EMAIL PROTECTED] writes:
Is there a reason for (or a solution to avoid it) the following
message: wget: strdup: Not enough memory. [1]
Does Wget exit after the error, or does it keep running?
Arno Schuring [EMAIL PROTECTED] writes:
The manual (man wget) doesn't say anything about redirecting the logs to
stdout; however, but since -O - is explicitly mentioned I figured I could
use the same for -o.
Sorry about that. Since -o prints to stdout (ok, stderr) by default,
I didn't
Axel Pettinger [EMAIL PROTECTED] writes:
Hrvoje Niksic wrote:
Axel Pettinger [EMAIL PROTECTED] writes:
Is there a reason for (or a solution to avoid it) the following
message: wget: strdup: Not enough memory. [1]
Does Wget exit after the error, or does it keep running?
Wget
(NULL), which results in calling strdup (NULL), which returns
null and makes xstrdup think that strdup signalled not enough
memory.
This patch should fix the problem. Please let me know if it works for
you:
2004-05-08 Hrvoje Niksic [EMAIL PROTECTED]
* ftp-basic.c (ftp_pwd): Handle PWD
[ Moving discussion to [EMAIL PROTECTED] ]
Gisle Vanem [EMAIL PROTECTED] writes:
Hrvoje Niksic [EMAIL PROTECTED] said:
Supporting large files in a really portable manner is unfortunately
not trivial.
It's not that hard. Look at how we did this in libcurl; basically
some define trickery
Unfortunately, Wget does not yet support downloading of large files.
There have been several patches that add such support, and it is
probable that one of them will make it into the next version.
Axel Pettinger [EMAIL PROTECTED] writes:
Hrvoje Niksic wrote:
I think I understand where the bug is. The server doesn't seem to
send PWD in the format the code expects (in fact, it doesn't seem to
be sending it at all).
So one could indeed say that it is a strange ftp server
Gisle Vanem [EMAIL PROTECTED] writes:
printf (_(The file is FILE_OFF_FMT octets long.\n), size);
I assume Wget needs a msg-entry for each string.
The file is %Ld octets long.\n
The file is %lld octets long.\n
The problem is, which of those message entries should end up in
Gerriet M. Denkmann [EMAIL PROTECTED] writes:
So: either the -P option should work as it does - than the man page
should mention this. Or it is a -P bug.
It's a bug, fixed in CVS.
Axel Pettinger [EMAIL PROTECTED] writes:
I added the five lines to ftp-basic.c and recompiled Wget. Now I can say
that your patch is indeed working for me![1] Thank you very much.
BTW, I was a little bit confused because of the last line in your patch.
Instead of FREE_MAYBE (*pwd); my
David Fritz [EMAIL PROTECTED] writes:
IIUC, GNU coreutils uses uintmax_t to store large numbers relating to
the file system and prints them with something like this:
char buf[INT_BUFSIZE_BOUND (uintmax_t)];
printf (_(The file is %s octets long.\n), umaxtostr (size, buf));
That's
Dan Jacobson [EMAIL PROTECTED] writes:
On the man page the interaction between -O vs. -nc is not mentioned!
Nor perhaps -O vs. -N.
Indeed, why don't you cause an error when you find both -O and -nc
used,
Good point. I'll take a look at doing that.
Thanks for forwarding this. The idea was for Wget to print the file
name it will write to, and yet to refrain from creating the file until
the data arrives.
One way to solve this is to use O_EXCL when opening the file, and
refusing to write to files that cannot be so opened. Essentially,
Wget
Ferry van Steen [EMAIL PROTECTED] writes:
I'm having trouble recursively sucking in a ftp site. The problem is in
the extremely weird username which is in this form:
admin.company.amsterdam#company.amsterdam.ispname.tld
Yes, that's only the username :)
Try replacing the # with %23.
Fred Holmes [EMAIL PROTECTED] writes:
If I have a URL that has %20 in place of spaces, and I use the URL
directly as the argument of WGET, it seems that the file is always
not found. I've discovered that if I replace each %20 with a
space, and put quotation marks around the entire URL, it
Fred Holmes [EMAIL PROTECTED] writes:
But I want WGET to convert %20 to space (I think).
Why would you want that? A URL with a literal space is illegal, at
least for HTTP -- Wget would have to convert the space to %20 to be
able to send the URL to the HTTP server anyway.
OTOH, if you're
Jacob Friis Larsen [EMAIL PROTECTED] writes:
How can I make Wget only download files less than 5mb?
Currently you can't, sorry.
801 - 900 of 1457 matches
Mail list logo
