minor memory leak risk
This subject says it all. The leak is minor, the fix could be made something
like this:
diff -u -r1.21 utils.c
--- utils.c 2001/05/27 19:35:12 1.21
+++ utils.c 2001/11/20 10:10:17
@@ -903,7 +903,12 @@
while (fgets (line + length, bufsize - length, fp))
{
length += strlen (line + length);
- assert (length 0);
+ if (0 == length)
+{
+ /* bad input file */
+ xfree(line);
+ return NULL;
+}
if (line[length - 1] == '\n')
break;
/* fgets() guarantees to read the whole line, or to use up the
--
Daniel Stenberg - http://daniel.haxx.se - +46-705-44 31 77
ech`echo xiun|tr nu oc|sed 'sx\([sx]\)\([xoi]\)xo un\2\1 is xg'`ol
Re: Wget 1.7.1 released
Tomas Hjelmberg [EMAIL PROTECTED] writes: Does anyone know when the meta name=robots tag will work? 1.8? Doesn't it work already? It's supposed to work as of 1.7.
Re: minor memory leak risk
Daniel Stenberg [EMAIL PROTECTED] writes: This subject says it all. The leak is minor, the fix could be made something like this: What memory leak are you referring to in the subject? Your patch replaces an assert() with a return NULL. The only way that assert() could be tripped is by having a line begin with \0 (which I didn't think of when I was writing the code; I was guarding against an impossible condition to protect the logic of the code.)
Re: SSL bad random seed
I remember discussions about this, but I am hardly the person to fix it, as I have zero knowledge about (and need for) SSL. Christian, could you take a look at this? I can dig up old messages about this if you don't have them. No problem. I will try to implement it (more or less) like Daniel Stenberg suggested in his mail [EMAIL PROTECTED]
Re: minor memory leak risk
[EMAIL PROTECTED] writes: It's not just a memory leak. Length = 0 is declared as a can't happen. If length is zero, wget will suddenly end due to the assert. If a bad input file can lead to length being zero, then using assert is bad on principle. One should never assert external input. For the record, I agree with all you said here. The assert was not meant to guard against external input. As I said in the previous message, I now see a way for LENGTH to be zero, so I will remove the assert. Before I do that, I'd like to understand what problem Daniel was trying to fix, just in case I missed something.
Re: Patch: --range switch implemented
hi! Here is my IMO (in case someone is really interested in:)) all ranges 0-based, support few syntax-es: --range=0..1024-- closed-closed --range=0-1024 -- closed-open --range=1024+2048 -- take 3..4 K's :) i.e. get 2k starting on pos 1024 (well last one could be like --range=2048@1024 just for fun) implementation of all cases is trivial and I cannot see why not having them all? P! Vladi. Hrvoje Niksic wrote: Herold Heiko [EMAIL PROTECTED] writes: However, of the top of my head I can't remember many occasions where 0-n means closed-open There are. (And note that it's n-m in the general case, not just 0-n.) Off the top of my head, the Java string subscripts, Lisp array-related functions, Python slices, various Emacs functions, etc. The Python example is easy to demonstrate: range(0, 10) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] Also: a = [0, 1, 2, 3, 4, 5] a[2:4] [2, 3] This makes perfect sense to me, but not everyone would agree. The nicest thing about it is that it allows this: a[0:3] + a[3:] [0, 1, 2, 3, 4, 5] I.e. you can construct the original interval by appending the touching subintervals. This is nice for downloads because it allows you to download 0-2k, 2k-5k, etc., without the one overlapping byte. Common Lisp: [1] (setq a '(0 1 2 3 4 5)) (0 1 2 3 4 5) [2] (subseq a 2 4) (2 3) Perl avoids the potential confusion by having its SUBSTR take offset (0-based) and length, which is clear to everyone. while there are at least Pascal and Perl where 0..n The Pascal reference is to 1..n, not 0..n. Which is one point you seem to have missed: IMHO [start, end] makes more sense with intervals that start with 1, and [start, end) makes more sense with intervals with start with 0. 0..10 #11 bytes including first one, like Perl, Pascal 1-10 #10 bytes including first one 1-10 is what I meant by the Pascal way because most Pascal programs use 1-based arrays. Again, assuming we want to download 16 bytes, the three options are, in my order of preference: 1 .. 16 # end-closed 1-based, Pascal-like 0 .. 15 # end-closed 0-based, Perl-like 0 .. 16 # end-open 0-based, Python-like Maybe we should support all 3, but document only one in --help? That way most users will not notice the complexity. Also, the first option could well be ignored since 1-based arrays are for wimps. :-) -- Vladi Belperchinov-Shabanski [EMAIL PROTECTED] [EMAIL PROTECTED] Personal home page at http://www.biscom.net/~cade DataMax Ltd. http://www.datamax.bg Too many hopes and dreams won't see the light... smime.p7s Description: S/MIME Cryptographic Signature
RE: Patch: --range switch implemented
From: Hrvoje Niksic [mailto:[EMAIL PROTECTED]] Vladi Belperchinov-Shabanski [EMAIL PROTECTED] writes: Here is my IMO (in case someone is really interested in:)) ... So what would be a nice alternative syntax for closed-open? 0:1024? Hyphen is easier to type, though. Damn, sometimes it's so hard to win. :-) Don't forget you need a symbol for the start-size syntax,too ... + would be perfect, --range 4096+1k or --range 4095+1k (shudder) Maybe what we really need is not a different syntax for every kind of range definition but a default syntax (whichever symbol you use) and number modifiers... for example, for the first kb, and suppose we want to accomodate endusers: default 1-1024 or 1:1024 or 1..1024 or ]0-1023] (same with : or ..) or [1-1024] (same with : or ..) or [0-1024[ (same with : or ..) or 0+1024 or 0+1k or [1+1024 or [1+1k or ]0+1024 or ]0+1k you get the point, sorry but I'm in a hurry, possibly I got some braces wrong. Heiko -- -- PREVINET S.p.A.[EMAIL PROTECTED] -- Via Ferretto, 1ph x39-041-5907073 -- I-31021 Mogliano V.to (TV) fax x39-041-5907087 -- ITALY
newbee not getting all nested (recursive) links
Dear all, Q #1 - URGENT! Am attempting to get 'all' recursive (2 levels deep) links URL. Am getting some but not all. Have tried: wget -o log.txt -r -l 2 -nc -T 10 -t inf URL... Where URL is: http://jobsearch.monster.com/jobsearch.asp?cy=USbrd=1q=%22risk+assessment%22 Have checked log file and saw no timeout related 'Errors' - ie: host seemed to respond to all. Why am I not getting all 'http://' links? NOTE: Have tried similar with WebMiner and it seems to work - ie: it successfully fetches all links successfully. - - - Q #2 Is there a PC (precompiled) version of 'wget'? If so, where? - - - Q #3 If/when I put 'wget' in script file if spawns some temp tasks/processes that terminate eventually. 'wget' then executes in background mode. If possible, I would need 'wget' to execute in foreground mode! If possible, how? If not possible, I have work-around. Thank you all Attila Horvath (US-VA)
RE: Patch: --range switch implemented
From: Hrvoje Niksic [mailto:[EMAIL PROTECTED]] Herold Heiko [EMAIL PROTECTED] writes: Don't forget you need a symbol for the start-size syntax,too ... + would be perfect, Yes. That's +, as implemented in the original patch. Noone is disupting that one. --range 4096+1k or --range 4095+1k (shudder) Did you mean 4097 here? Yes in fact, for the 1-based syntax. Heiko -- -- PREVINET S.p.A.[EMAIL PROTECTED] -- Via Ferretto, 1ph x39-041-5907073 -- I-31021 Mogliano V.to (TV) fax x39-041-5907087 -- ITALY
Re: minor memory leak risk
Daniel Stenberg [EMAIL PROTECTED] writes: You're right of course. Hm. No, it can probably only happen if the first byte in an existing file is a zero-byte, so that strlen() return 0... Yup. Or the first byte of any line. I.e. Wget will die if it encounters the \n\0 sequence of characters, which is a bug. This should be fixed by replacing the assert with continue. That line may be impossible to process given the standard handling of C strings, but it doesn't mean that other lines in the file cannot be used.
newbee followup...
In case the question pops up, I'm using version 1.5.3 of 'wget'. Thx Attila
wget suggestion...
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 One small suggestion for a possible later release... a mask for all files.. wget -m http://localhost/*.txt for example. Other than that.. all's good =) Regards.. Total K http://www.oc32.cjb.net ~ OC32 Home http://www.digiserv.cjb.net ~ Home of [Total K] http://www.digitaldisorder.cjb.net/php/download.php?sec=pgpsub_sec=f=totalk.asc ~ PGP Key - --- If mini skirts get any higher, said the Fairy to the Gnome, We'll have two more cheeks to powder, and a few more hairs to comb. -BEGIN PGP SIGNATURE- Version: PGP 6.5.3 iQA/AwUBO/ptAuVkNn/VM/QPEQKhfQCgqsmh85/7XZlWdFNYHS2tyt8g0hUAnR0H k6ekAq6xnmZQMU23vzHKTccA =QtzJ -END PGP SIGNATURE-
Re: ftp bug in 1.8-dev
Herold Heiko [EMAIL PROTECTED] writes:
With 1.8, build on windows, wget -d ftp:// ... I get consistently
-- PWD
257 / is current directory.
done.
== TYPE ...
-- TYPE
500 'TYPE ': Invalid number of parameters
Thanks for the report. This patch should fix the problem.
2001-11-20 Hrvoje Niksic [EMAIL PROTECTED]
* url.c (parseurl): Don't depend on the now-obsolete TYPE.
Index: src/url.c
===
RCS file: /pack/anoncvs/wget/src/url.c,v
retrieving revision 1.51
diff -u -r1.51 url.c
--- src/url.c 2001/11/19 16:15:42 1.51
+++ src/url.c 2001/11/20 15:59:59
@@ -423,7 +423,7 @@
the scheme was explicitly named,
i.e. it wasn't deduced from the URL
format. */
- uerr_t type;
+ uerr_t type = URLUNKNOWN;
DEBUGP ((parseurl (\%s\) - , url));
recognizable = url_has_scheme (url);
@@ -442,7 +442,17 @@
else if (i == ARRAY_SIZE (supported_schemes))
type = URLUNKNOWN;
else
-u-scheme = type = supported_schemes[i].scheme;
+{
+ u-scheme = supported_schemes[i].scheme;
+ if (u-scheme == SCHEME_HTTP)
+ type = URLHTTP;
+#ifdef HAVE_SSL
+ if (u-scheme == SCHEME_HTTPS)
+ type = URLHTTPS;
+#endif
+ if (u-scheme == SCHEME_FTP)
+ type = URLFTP;
+}
if (type == URLUNKNOWN)
l = 0;
@@ -505,12 +515,12 @@
/* Some delimiter troubles... */
if (url[i] == '/' url[i - 1] != ':')
++i;
- if (type == URLHTTP)
+ if (u-scheme == SCHEME_HTTP)
while (url[i] url[i] == '/')
++i;
u-path = (char *)xmalloc (strlen (url + i) + 8);
strcpy (u-path, url + i);
- if (type == URLFTP)
+ if (u-scheme == SCHEME_FTP)
{
u-ftp_type = process_ftp_type (u-path);
/* We don't handle type `d' correctly yet. */
@@ -534,7 +544,7 @@
/* Simplify the directory. */
path_simplify (u-dir);
/* Remove the leading `/' in HTTP. */
- if (type == URLHTTP *u-dir == '/')
+ if (u-scheme == SCHEME_HTTP *u-dir == '/')
strcpy (u-dir, u-dir + 1);
DEBUGP ((ndir %s\n, u-dir));
/* Strip trailing `/'. */
Re: 134% ready
On 17 Nov 2001, at 14:24, Hrvoje Niksic wrote:
Ferenc VERES [EMAIL PROTECTED] writes:
My download stopped, then next day I continued with -c, from 34%.
At the end I saw: 134% downloaded, in end of the lines ;-) (it was
FTP:// transfer, a 680MB iso image)
GNU Wget
1.5.3
Try it with a later version. I believe that this bug has been fixed
in most of its variations.
That stuff only works for recursive transfers, not when resuming
transfer of a single file.
For a recursive resume, wget knows what size the file is supposed
to be based on a previous directory listing, and can therefore
compensate for the two different ways in which a server can report
the size of a REST transfer. However, when resuming transfer of a
single file, wget currently has no idea what the size of the file
is supposed to be and assumes that the size reported by the server
for a REST transfer is the total size of the file. If the server
actually reports the remaining size, the transfer log will show
incorrect percentages.
Dave Turner wrote a patch back in the middle of August which I
modified slightly. The patch attempts to use the SIZE command which
is commonly supported by servers, but not mentioned in RFC959. I
suppose an alternative would be to do a directory listing for the
file. See these messages:
http://www.mail-archive.com/wget@sunsite.dk/msg01463.html
http://www.mail-archive.com/wget@sunsite.dk/msg01468.html
http://www.mail-archive.com/wget@sunsite.dk/msg01469.html
http://www.mail-archive.com/wget@sunsite.dk/msg01481.html
http://www.mail-archive.com/wget@sunsite.dk/msg01480.html
The patch in msg01481 (my modified version) still applies okay
against the current CVS, but I have recreated it against the
current revisions in the repository below.
Here is the ChangeLog entry from August (you may wish to change the
date):
2001-08-21 Dave Turner [EMAIL PROTECTED]
* ftp-basic.c (ftp_size): New function to send non-standard SIZE
command to server to request file size.
* ftp.h (ftp_size): Export it.
* ftp.c (getftp): Use new ftp_size function if restoring
transfer of a file with unknown size.
And here is the patch against the current repository:
Index: src/ftp-basic.c
===
RCS file: /pack/anoncvs/wget/src/ftp-basic.c,v
retrieving revision 1.10
diff -u -r1.10 ftp-basic.c
--- src/ftp-basic.c 2001/05/27 19:34:57 1.10
+++ src/ftp-basic.c 2001/11/20 17:01:32
@@ -21,6 +21,8 @@
#include stdio.h
#include stdlib.h
+#include errno.h
+
#ifdef HAVE_STRING_H
# include string.h
#else
@@ -626,6 +628,63 @@
FREE_MAYBE (*pwd);
*pwd = xstrdup (request);
+
+ xfree (respline);
+ /* All OK. */
+ return FTPOK;
+}
+/* Sends the SIZE command to the server, and returns the value in 'size'.
+ * If an error occurs, size is set to zero. */
+uerr_t
+ftp_size (struct rbuf *rbuf, const char *file, long int *size)
+{
+ char *request, *respline;
+ int nwritten;
+ uerr_t err;
+
+ /* Send PWD request. */
+ request = ftp_request (SIZE, file);
+ nwritten = iwrite (RBUF_FD (rbuf), request, strlen (request));
+ if (nwritten 0)
+{
+ xfree (request);
+ *size = 0;
+ return WRITEFAILED;
+}
+ xfree (request);
+ /* Get appropriate response. */
+ err = ftp_response (rbuf, respline);
+ if (err != FTPOK)
+{
+ xfree (respline);
+ *size = 0;
+ return err;
+}
+ if (*respline == '5')
+{
+ /*
+ * Probably means SIZE isn't supported on this server.
+ * Error is nonfatal since SIZE isn't in RFC 959
+ */
+ xfree (respline);
+ *size = 0;
+ return FTPOK;
+}
+
+ errno = 0;
+ *size = strtol (respline + 4, NULL, 0);
+ if (errno)
+{
+ /*
+ * Couldn't parse the response for some reason. On the (few)
+ * tests I've done, the response is 213 SIZE with nothing else -
+ * maybe something a bit more resilient is necessary. It's not a
+ * fatal error, however.
+ */
+ xfree (respline);
+ *size = 0;
+ return FTPOK;
+}
xfree (respline);
/* All OK. */
Index: src/ftp.c
===
RCS file: /pack/anoncvs/wget/src/ftp.c,v
retrieving revision 1.44
diff -u -r1.44 ftp.c
--- src/ftp.c 2001/11/16 19:38:03 1.44
+++ src/ftp.c 2001/11/20 17:01:36
@@ -462,6 +462,38 @@
else /* do not CWD */
logputs (LOG_VERBOSE, _(== CWD not required.\n));
+ if ((cmd DO_RETR) restval *len == 0)
+{
+ if (opt.verbose)
+ {
+ if (!opt.server_response)
+ logprintf (LOG_VERBOSE, == SIZE %s ... , u-file);
+ }
+
+ err = ftp_size(con-rbuf, u-file, len);
+ /* FTPRERR */
+ switch (err)
+ {
+ case FTPRERR:
+ case FTPSRVERR :
+ logputs (LOG_VERBOSE, \n);
+ logputs (LOG_NOTQUIET, _(\
+Error in server response, closing control
Re: 134% ready
Ian Abbott [EMAIL PROTECTED] writes: Try it with a later version. I believe that this bug has been fixed in most of its variations. That stuff only works for recursive transfers, not when resuming transfer of a single file. You're right. The patch in msg01481 (my modified version) still applies okay against the current CVS, but I have recreated it against the current revisions in the repository below. Thanks for the patch; I've now applied it to the source.
Get your GREEN CARD for FREE
Title: Untitled Document Don't miss your opportunity The Diversity Immigrant Visa Lottery (Green Card Lottery) has been established in the 1996 Immigration Act in order to give immigration opportunity to natives from countries other than the main source of immigration to the U.S.A. 55,000 permanent resident visas (Green Cards) are awarded each year. A Green Card gives you the legal right to live and work permanently in the United States. Green card holders also receive health, education, and other rights. A Green Card holder may later apply for United States citizenship. When you receive a Green Card through the lottery, your husband or wife and all unmarried children under 21 years of age will also receive Green Cards. There is no fee to enter the lottery. Don't miss your chance to enter the Green Card Lottery. Please Visit Our Website CLICK HERE
