Re: wget 1.11.1 make test fails
On Thursday, April 3, 2008 at 22:37:41 +0200, Hrvoje Niksic wrote: Or it could be that you're picking up a different fnmatch.h that sets up a different value for FNM_PATHNAME. Do you have more than one fnmatch.h installed on your system? I have only /usr/include/fnmatch.h installed, identical to the file in the libc-5.4.33 tarball, and defining the same values as wget's src/sysdep.h (even comments are identical). Just my fnmatch.h defines two more flags, FNM_LEADING_DIR=8 and FNM_CASEFOLD=16, and defines an FNM_FILE_NAME alias (commented as Preferred GNU name) to FNM_PATHNAME=1 (the libc code uses only this alias). Anyway I had noticed your comment about incompatible headers, and double-checked your little test program also with explicit value 1: same results. BTW everybody should be able to reproduce the make test failure, on any system, just by #undefining SYSTEM_FNMATCH in src/sysdep.h Alain.
Re: wget 1.11.1 make test fails
On Thursday, April 3, 2008 at 9:14:52 -0700, Micah Cowan wrote: Are you certain you rebuilt cmpt.o? This seems pretty unlikely, to me. Certain: make test after touching src/sysdep.h rebuilds both cmpt.o, the normal in src/ and the one in tests/. And both those cmpt.o become 784 bytes bigger without SYSTEM_FNMATCH. Alain.
Re: wget 1.11.1 make test fails
On Wednesday, April 2, 2008 at 23:09:52 +0200, Hrvoje Niksic wrote: Micah Cowan [EMAIL PROTECTED] writes: It's hard for me to imagine an fnmatch that ignores FNM_PATHNAME The libc 5.4.33 fnmatch() supports FNM_PATHNAME, and there is code apparently intending to return FNM_NOMATCH on a slash. But this code seems to be rather broken. | printf(%d\n, fnmatch(foo*, foo/bar, FNM_PATHNAME)); It should print a non-zero value. Zero on the old system, FNM_NOMATCH on a recent one. Alain.
Re: wget 1.11.1 make test fails
On Thursday, April 3, 2008 at 11:08:27 +0200, Hrvoje Niksic wrote: Well, it would point to a problem with both the fnmatch replacement and the older system fnmatch. Our fnmatch (coming from an old release of Bash The fnmatch()es in libc 5.4.33 and in Wget are twins. They differ on some minor details like FNM_CASEFOLD support, and cosmetic things like parenthesis around return(code). The part dealing with * in pattern is functionaly identical. Maybe you could put a breakpoint in fnmatch and see what goes wrong? The for loop intended to eat several characters from the string also advances the pattern pointer. This one reaches the end of the pattern, and points to a NUL. It is not a '*' anymore, so the loop exits prematurely. Just below, a test for NUL returns 0. The body of the loop, returning FNM_NOMATCH on a slash, is not executed at all. That isn't moderately broken, is it? Alain.
Re: wget 1.11.1 make test fails
Hello Micah,
On Monday, March 31, 2008 at 11:39:43 -0700, Micah Cowan wrote:
could you try to isolate which part of test_dir_matches_p is failing?
The only failing src/utils.c test_array[] line is:
| { { *COMPLETE, NULL, NULL }, foo/!COMPLETE, false },
I don't understand enough of dir_matches_p() and fnmatch() to guess what
is supposed to happen. But with false replaced by true, this test and
following succeed.
| ALL TESTS PASSED
| Tests run: 7
Of course this test then fails on newer systems.
Alain.
wget 1.11.1 make test fails
Hello,
With an old gcc 2.7.2.1 compiler, wget 1.11.1 make test fails:
| gcc -I. -I. -I./../src -DHAVE_CONFIG_H
-DSYSTEM_WGETRC=\/usr/local/etc/wgetrc\
-DLOCALEDIR=\/usr/local/share/locale\ -O2 -Wall -DTESTING -c ../src/test.c
| ../src/test.c: In function `all_tests':
| ../src/test.c:51: parse error before `const'
| ../src/test.c:51: `message' undeclared (first use this function)
| ../src/test.c:51: (Each undeclared identifier is reported only once
| ../src/test.c:51: for each function it appears in.)
| ../src/test.c:52: parse error before `const'
| ../src/test.c:53: parse error before `const'
| ../src/test.c:54: parse error before `const'
| ../src/test.c:55: parse error before `const'
| ../src/test.c:56: parse error before `const'
| ../src/test.c:57: parse error before `const'
| make[1]: *** [test.o] Error 1
| make[1]: Leaving directory `/tmp/wget-1.11.1/tests'
| make: *** [test] Error 2
The attached make-test.patch seems to fix this. However later the 3rd
test fails:
| ./unit-tests
| RUNNING TEST test_parse_content_disposition...
| PASSED
|
| RUNNING TEST test_subdir_p...
| PASSED
|
| RUNNING TEST test_dir_matches_p...
| test_dir_matches_p: wrong result
| Tests run: 3
| make[1]: *** [run-unit-tests] Error 1
| make[1]: Leaving directory `/tmp/wget-1.11.1/tests'
| make: *** [test] Error 2
Alain.
diff -prud wget-1.11.1.orig/src/test.h wget-1.11.1/src/test.h
--- wget-1.11.1.orig/src/test.h Mon Mar 24 22:53:58 2008
+++ wget-1.11.1/src/test.h Mon Mar 31 15:19:31 2008
@@ -34,8 +34,9 @@ as that of the covered work. */
#define mu_assert(message, test) do { if (!(test)) return message; } while (0)
#define mu_run_test(test) \
do { \
+ const char *message; \
puts(RUNNING TEST #test ...); \
- const char *message = test(); \
+ message = test(); \
tests_run++; \
if (message) return message; \
puts(PASSED\n); \
Re: Wget 1.11 build fails on old Linux
On Monday, February 25, 2008 at 16:32:21 +0100, Alain Guibert wrote: On an old Debian Bo system (kernel 2.0.40, gcc 2.7.2.1, libc 5.4.33), building Wget 1.11 fails: While wget 1.11.1 builds and works OK. Thank you very much, gentlemen! Alain.
Wget 1.11 build fails on old Linux
Hello gentlemen, On an old Debian Bo system (kernel 2.0.40, gcc 2.7.2.1, libc 5.4.33), building Wget 1.11 fails: | gcc -I. -I. -DHAVE_CONFIG_H -DSYSTEM_WGETRC=\/usr/local/etc/wgetrc\ -DLOCALEDIR=\/usr/local/share/locale\ -O2 -Wall -c connect.c | connect.c: In function `select_fd': | connect.c:620: storage size of `tmout' isn't known | connect.c:623: warning: implicit declaration of function `FD_ZERO' | connect.c:624: warning: implicit declaration of function `FD_SET' | connect.c:634: warning: implicit declaration of function `select' | connect.c:620: warning: unused variable `tmout' | connect.c: In function `test_socket_open': | connect.c:655: storage size of `to' isn't known | connect.c:655: warning: unused variable `to' | make[1]: *** [connect.o] Error 1 | make[1]: Leaving directory `/tmp/wget-1.11/src' | make: *** [src] Error 2 On such old platform, the needed definitions are in linux/time.h, included by sys/time.h, included by time.h. At least one of them used to be included by src/sysdep.h in Wget 1.10.2. But in Wget 1.11, src/sysdep.h #includes time.h only #ifndef HAVE_TIMEGM. This platform does HAVE_TIMEGM, but its select() still needs time.h. Once src/sysdep.h reverted to #include time.h or sys/time.h as before, Wget 1.11 compiles and works fine. Alain.
Re: wrong timestamps
On Tuesday, July 5, 2005 at 6:59:04 PM +0200, Hrvoje Niksic wrote: Am I completely off the mark, or can mktime_from_utc (or in fact timegm) be replaced by something as simple as the following? Or is it too good to be true? It returns a number of seconds since epoch neglecting leap seconds. This number is later given to the system for file timestamps, display, and such. I believe this can unfortunately be wrong by about 22s on platforms without timegm() but counting leap seconds. Untested though. It seems many platforms directly or optionally account for leap seconds. Example Linux Woody by default neglects but optionally accounts leap seconds when timezone info comes from the right directory. | $ TZ=UTC date --date 9 Sep 2001 1:46:40 UTC +%s | 10 | $ TZ=right/UTC date --date 9 Sep 2001 1:46:40 UTC +%s | 100022 The first is the common Posix simplification, the later is The Truth. By the way: After 7 years of stability, the IERS announced officialy that the next added leap second will be December 31 2005 23:59:60 UTC. Alain.
Re: wrong timestamps
On Saturday, May 14, 2005 at 10:09:32 PM +0200, Hrvoje Niksic wrote: Alain Guibert [EMAIL PROTECTED] writes: Maybe cmpt.c mktime is failing because of incompatible timezone and daylight infos on the platform? If you change __tzset() to tzset() and remove the surrounding #ifdef _LIBC, does it then work? Same false result: Files from 3:30 and 4:30 GMT get minus one hour on libc5, while on Glibc2 the 3:30 file gets minus one hour and the 4:30 gets download date. Cygwin behaves as Glibc2. Alain.
Re: wrong timestamps
On Thursday, May 19, 2005 at 3:24:39 PM +0200, Hrvoje Niksic wrote: * Is GMT0 a portable value for TZ? TZ=GMT0 works on Linux libc5/Glibc, Cygwin, and MinGW. setenv is not available on Windows, or at least Borland's headers don't include a prototype. (But they do include putenv.) (un)setenv are not available in MinGW, but exist in Cygwin. Putenv is available on both. Additional problem with putenv(): To remove TZ Linux and Cygwin need putenv(TZ), while MinGW wants putenv(TZ=) with equal. I really don't understand how setting TZ to an empty string makes it point to UTC. Not portable: TZ= works more or less as GMT0 on Linux and Cygwin, but not on MinGW where it behaves as with unset TZ, taking system timezone from control panel. MinGW just uses Windows libc equivalent dll, so behaviour may be the same with other compilers. Alain.
Re: wrong timestamps
Hello Roger, On Thursday, May 19, 2005 at 8:09:51 AM -0700, Roger L. Beeman wrote: I have been trying to follow this thread, but have not yet devoted enough time to fully understand the issues, so I would very much appreciate the summary. The [EMAIL PROTECTED] address is no longer valid, but I am subscribed to the list. Edward Sabol was an integral contributer to the current version and I would ask that he also lend a hand in revisiting this issue. Let's try the summary. I see 3 points: (1) Libc 5.4.33 own mktime() produces wrong by some minutes results for all summer dates when tm_isdst is forced to false 0. Wget's mktime_from_utc() forces tm_isdst=0 at a stage, and produces wrong by some minutes result only for one hour, beginning at DST transition plus one hour. (2) Replacing wget's mktime_from_utc() by a TZ=GMT0 mktime() scheme. Solves problem one, is faster, may seem cleaner (no discontinuities to support), but introduces portability issues. Still at discussion. (3) When wget's cmpt.c:mktime() is forced to override platform's mktime(), then mktime_from_utc() produces wrong results for two hours, beginning at to-DST transition plus two hours. Wrong by an hour less, or totally wrong. Even on platforms not affected by problem one. I have been assuming that the behavior being observed was from a mechanism similar to the nonexistent localtime hour issue currently addressed in the code, so if you have explored that possibility and ruled it out, I would benefit from your analysis. The nonexistant localtime hour is here the hour between 2:00 CET and 3:00 CEST. Problem one begins later at 4:00 CEST (until 5:00), while problem three begins at 5:00 CEST (until 7:00). Alain.
Re: wrong timestamps
On Monday, May 9, 2005 at 5:53:26 PM +0200, Hrvoje Niksic wrote:
Alain Guibert [EMAIL PROTECTED] writes:
What about another method: putenv(TZ=GMT0), tzset(), mktime(), reset
TZ to previous state, tzset() again. Not portable?
That sounds at least as non-portable as the current method, and it
might be slow in addition. Have you tried that method? Does it help
in your case?
It seems to work correctly on Linux and Cygwin, and is roughly 3
times faster. I tried this:
| static time_t
| mktime_from_utc (struct tm *t)
| {
| time_t time;
| char *tz;
| char save_tz[100];
|
| tz=getenv(TZ);
| if (tz)
| strncpy(save_tz, tz, sizeof(save_tz));
| setenv(TZ, GMT0, 1);
| tzset();
| time=mktime(t);
| if (tz)
| setenv(TZ, save_tz, 1);
| else
| unsetenv(TZ);
| tzset();
| return (time);
| }
But I'm sure it will fail on some platforms, so not good.
I find it strange that the cmpt.c mktime produces garbage when that
version is taken from glibc.
Note that with TZ=GMT0 method and #undef HAVE_MKTIME, wget gives correct
timestamps. Maybe cmpt.c mktime is failing because of incompatible
timezone and daylight infos on the platform? It calls __tzset() but does
not provide it.
| init.c:214: structure has no member named `random_file'
| init.c:214: initializer element for `commands[76].place' is not constant
This has been fixed in CVS; the fix is to wrap the offending init.c
line in #ifdef HAVE_SSL.
That's it, thank you!
Alain.
Re: wrong timestamps
On Saturday, May 7, 2005 at 6:56:53 PM +0200, Hrvoje Niksic wrote: I was lead to believe that it is quite reasonable to set tm_isdst to zero before calling mktime. In fact, the logic in mktime_from_utc, along with setting tm_isdst to zero, was explicitly written to support discontinuities at DST transition points -- see Roger Beeman's changelog entry from 2001-04-25. And comments in code, which are very detailed, thanks. Is behaviour of mktime() with limit false input values defined in standards? What about another method: putenv(TZ=GMT0), tzset(), mktime(), reset TZ to previous state, tzset() again. Not portable? Fortunately there is an easy solution for this: simply #undef HAVE_MKTIME in config.h. This will enable the use of Wget's mktime replacement that comes from glibc 2 and does not have the bug. This gives a minus one hour error during 2 hours (3:00:00-4:59:59): | $ TZ=GMT0 vdir --full-time /var/www/testfile[34] | -rw-r--r-- 1 root root0 Sun Mar 26 03:30:00 2000 /var/www/testfile3 | -rw-r--r-- 1 root root0 Sun Mar 26 04:30:00 2000 /var/www/testfile4 | $ TZ=GMT0 vdir --full-time /tmp/testfile[34] | -rw-r--r-- 1 ag users 0 Sun Mar 26 02:30:00 2000 /tmp/testfile3 | -rw-r--r-- 1 ag users 0 Sun Mar 26 03:30:00 2000 /tmp/testfile4 Tried same #undefs on Woody: testfile3 gets same wrong 2:30 GMT timestamp, while testfile4 gets the current date (9 May 2005). BTW unrelated I tried building alpha-3 on another machine with more or less same Debian Bo setup, and got this error: | gcc -I. -I. -I/opt/include -DHAVE_CONFIG_H -DSYSTEM_WGETRC=\/usr/local/etc/wgetrc\ -DLOCALEDIR=\/usr/local/share/locale\ -O2 -Wall -Wno-implicit -c init.c | init.c:214: structure has no member named `random_file' | init.c:214: initializer element for `commands[76].place' is not constant | make[1]: *** [init.o] Error 1 | make[1]: Leaving directory `/tmp/wget-1.10-alpha3/src' | make: *** [src] Error 2 Alain.
wrong timestamps
Hello,
On Debian Bo (kernel 2.0.40, gcc 2.7.2.1, libc 5.4.33), wget 1.9.1 and
1.10-alpha3 give a wrong timestamp to downloaded files, when the
timestamp is in the hour after the hour following a DST transition.
Example here timezone CET +0100, transition to CEST +0200 on last Sunday
of March at 1:00 GMT, if file is between 2:00:00 and 2:59:59 GMT (would
be the same on last Sunday of October):
| # touch --date 26 Mar 2000 2:30:00 GMT /var/www/testfile2
| $ vdir --full-time /var/www/testfile2
| -rw-r--r-- 1 root root0 Sun Mar 26 04:30:00 2000
/var/www/testfile2
| $ wget-1.10-alpha3 --debug http://localhost/testfile2 21 | grep -i
^last-modified:
| Last-modified: Sun, 26 Mar 2000 02:30:00 GMT
| $ vdir --full-time /tmp/testfile2
| -rw-r--r-- 1 ag users 0 Sun Mar 26 04:20:39 2000
/tmp/testfile2
| $ TZ=GMT0 vdir --full-time /tmp/testfile2
| -rw-r--r-- 1 ag users 0 Sun Mar 26 02:20:39 2000
/tmp/testfile2
The file correctly served as 2:30 gets downloaded wrongly as 2:20:39 GMT.
No such problem with Woody and Glibc. This problem seems to be half Wget
half libc 5.4.33. In src/http.c:mktime_from_utc() and http_atotm(), Wget
manipulates tm structs forcing tm_isdst to 0. Which is false 7 monthes a
year, and hurts libc mktime() function giving garbage result. Due to the
way mktime_from_utc() does twice the calculation, the error is
compensated all year long, but not during these post transition hours.
Setting tm_isdst to -1 (unknown DST) corrects this 9 minutes inaccuracy,
but then the complex logic of mktime_from_utc() fails and gives full
hour errors.
The libc5 problem is that, given a tm with forced wrong tm_isdst,
mktime() garbles the tm content and returns a consistent but wrong
time_t. Example given a tm containing {0, 0, 0, 1, 7, 105, 0, 0, 0}
(1 Aug 2005 0:00), mktime() gives {39, 50, 0, 1, 7, 105, 1, 212, 1}
(1 Aug 2005 0:50:39 CEST) and returns 1122850239.
Alain.
Re: 1.10 alpha 3 build fails
On Saturday, April 30, 2005 at 8:31:14 PM +0200, Hrvoje Niksic wrote: Alain Guibert [EMAIL PROTECTED] writes: On Friday, April 29, 2005 at 2:15:55 PM +0200, Hrvoje Niksic wrote: The problem was that the check for sockaddr_in6 set ipv6 to yes if successful, overriding other tests that set it to no. This patch should fix the problem. I can't confirm, stuck in auto-requirement I don't know how to solve whithout breaking other things. Autoconf 2.59 installation was difficult for me, with required Perl upgrade from 5.003.07 to 5.8.6 failing some tests and filling my $HOME, but with your helpfull explanations I succeeded. Thank you very much! I can now confirm: Alpha3+configure.in patch builds cleanly on Debian Bo even without --disable-ipv6: | checking for getaddrinfo... no | configure: Disabling IPv6 support: your system does not support getaddrinfo(3) Thank you again, Hrvoje! Alain.
Re: 1.10 alpha 3 build fails
Hello Hrvoje, On Friday, April 29, 2005 at 1:30:34 PM +0200, Hrvoje Niksic wrote: | ptimer.c:143: `CLOCK_MONOTONIC' undeclared (first use in this function) Does this this patch fix the problem? Yes: Alpha3+ptimer.c patch builds OK, and the binary works. Much thanks Hrvoje! | checking for getaddrinfo... no | checking for INET6 protocol support... yes Does Wget build if you specify --disable-ipv6 on the command line? Yes: Alpha3 builds OK with --disable-ipv6, and the binary seems to work. On Friday, April 29, 2005 at 2:15:55 PM +0200, Hrvoje Niksic wrote: The problem was that the check for sockaddr_in6 set ipv6 to yes if successful, overriding other tests that set it to no. This patch should fix the problem. I can't confirm, stuck in auto-requirement I don't know how to solve whithout breaking other things. | cd . autoconf | configure.in:33: error: Autoconf version 2.57 or higher is required for this script | configure.in:33: the top level | make: *** [configure] Error 1 Alain.
1.10 alpha 3 build fails
Hello, On Debian Woody (kernel 2.4.18, gcc 2.95.4, Glibc 2.2.5), building Wget 1.10 alpha 3 fails: | checking for clock_gettime... no | checking for clock_gettime in -lrt... yes [...] | gcc -I. -I.-DHAVE_CONFIG_H -DSYSTEM_WGETRC=\/usr/local/etc/wgetrc\ -DLOCALEDIR=\/usr/local/share/locale\ -O2 -Wall -Wno-implicit -c ptimer.c | ptimer.c: In function `posix_init': | ptimer.c:143: `CLOCK_MONOTONIC' undeclared (first use in this function) | ptimer.c:143: (Each undeclared identifier is reported only once | ptimer.c:143: for each function it appears in.) | ptimer.c:143: initializer element is not constant | ptimer.c:143: (near initialization for `clocks[0].id') | make[1]: *** [ptimer.o] Error 1 | make[1]: Leaving directory `/tmp/wget-1.10-alpha3/src' | make: *** [src] Error 2 On older Debian Bo (kernel 2.0.40, gcc 2.7.2.1, libc 5.4.33) also fails: | checking for getaddrinfo... no | configure: Disabling IPv6 support: your system does not support getaddrinfo(3) | checking for INET6 protocol support... yes | checking for struct sockaddr_in6... yes | checking for struct sockaddr_storage... no | checking for struct sockaddr_in6.sin6_scope_id... no [...] | gcc -I. -I. -I/opt/include -DHAVE_CONFIG_H -DSYSTEM_WGETRC=\/usr/local/etc/wgetrc\ -DLOCALEDIR=\/usr/local/share/locale\ -O2 -Wall -Wno-implicit -c connect.c | In file included from connect.c:61: | host.h:63: field `addr' has incomplete type | connect.c: In function `connect_to_ip': | connect.c:262: storage size of `ss' isn't known | connect.c:319: storage size of `bind_ss' isn't known | connect.c:319: warning: unused variable `bind_ss' | connect.c:262: warning: unused variable `ss' | connect.c: In function `bind_local': | connect.c:424: storage size of `ss' isn't known | connect.c:424: warning: unused variable `ss' | connect.c: In function `accept_connection': | connect.c:496: storage size of `ss' isn't known | connect.c:496: warning: unused variable `ss' | connect.c: In function `socket_ip_address': | connect.c:523: storage size of `storage' isn't known | connect.c:523: warning: unused variable `storage' | make[1]: *** [connect.o] Error 1 | make[1]: Leaving directory `/tmp/wget-1.10-alpha3/src' | make: *** [src] Error 2 Alain.
