Re: [Bug-wget] FTP tests fail on MS-Windows
Tim Rühsen wrote: P.S. Now I'm beginning to wonder whether anyone runs the test suite on Windows, or cares about the results... Looks like nobody does :-( I've tried, but failed since my Perl (Strawberry Perl 5.18.4) doesn't support the fork() call. I don't think ActiveState Perl does either. -- --gv
Re: [Bug-wget] FTP tests fail on MS-Windows
Date: Mon, 22 Dec 2014 12:18:27 + From: Gisle Vanem gva...@yahoo.no CC: bug-wget@gnu.org Tim Rühsen wrote: P.S. Now I'm beginning to wonder whether anyone runs the test suite on Windows, or cares about the results... Looks like nobody does :-( I've tried, but failed since my Perl (Strawberry Perl 5.18.4) doesn't support the fork() call. I don't think ActiveState Perl does either. The only way this is going to work is to use MSYS/MSYS2 (and its Perl). The test suite needs too many Posix features, and OTOH needs to be able to invoke the native wget correctly. Nothing else except MSYS fits that bill, AFAIK.
Re: [Bug-wget] FTP tests fail on MS-Windows
Am Sonntag, 21. Dezember 2014, 05:43:09 schrieb Eli Zaretskii:
From: Tim Rühsen tim.rueh...@gmx.de
Date: Sat, 20 Dec 2014 22:16:16 +0100
# FTP Server has to start with english locale due to use of
strftime
month names in LIST command
setlocale(LC_ALL, 'C');
$self-_launch_server(
Thanks. But how can this explain the 'index.html' thingy appearing in
the FTP listing, instead of the expected afile.txt etc.?
The .listing file can't be parsed correctly when the month names are
incorrect.
I see. But then this is not the problem in my case. Here's the
listing I see in one of the FTP test logs:
226 Listing complete. Data connection has been closed.
-r--r--r-- 1 0 0 12 Dec 12:43 franτais.txt
2014-12-19 12:43:03 (362 KB/s) - '.listing' saved [48]
As you see, even though the file name includes non-ASCII characters,
the month name is in English (which is what I'd expect, given the
locale I have here).
But I know, there are a few Windows users / developers reading this.
Maybe they can help or bring some light !?
I certainly hope so.
Could the problem be that the listing has a CR-LF end-of-line format?
Could that interfere with its parsing?
Thanks.
Back to Test-ftp-bad-list.px... if you add the following line to
FTPServer.pm, you can see the difference between the GNU/Linux and the Windows
.listing. The listing has two lines, so a difference of 4 bytes (reported in
your original post) seems not to be a CRLF problem.
Also, as you can see below, even on GNU/Linux CRLF is used and not LF alone.
diff --git a/tests/FTPServer.pm b/tests/FTPServer.pm
index 3d7d8a5..261e819 100644
--- a/tests/FTPServer.pm
+++ b/tests/FTPServer.pm
@@ -134,6 +134,7 @@ sub _LIST_command
{
for my $item (@$listing)
{
+print STDERR $item\r\n;
print $sock $item\r\n;
}
}
Tim
signature.asc
Description: This is a digitally signed message part.
Re: [Bug-wget] FTP tests fail on MS-Windows
From: Tim Rühsen tim.rueh...@gmx.de Date: Sun, 21 Dec 2014 13:13:50 +0100 The .listing file can't be parsed correctly when the month names are incorrect. I see. But then this is not the problem in my case. Here's the listing I see in one of the FTP test logs: 226 Listing complete. Data connection has been closed. -r--r--r-- 1 0 0 12 Dec 12:43 franτais.txt 2014-12-19 12:43:03 (362 KB/s) - '.listing' saved [48] As you see, even though the file name includes non-ASCII characters, the month name is in English (which is what I'd expect, given the locale I have here). But I know, there are a few Windows users / developers reading this. Maybe they can help or bring some light !? I certainly hope so. Could the problem be that the listing has a CR-LF end-of-line format? Could that interfere with its parsing? Thanks. Back to Test-ftp-bad-list.px... if you add the following line to FTPServer.pm, you can see the difference between the GNU/Linux and the Windows .listing. The listing has two lines, so a difference of 4 bytes (reported in your original post) seems not to be a CRLF problem. Right you are. I looked at it from the other end: enabled debugging output from wget, and then looked carefully at the output. And I saw this: .listing[ = ] 88 --.-KB/s in 0s Closed fd 4 226 Listing complete. Data connection has been closed. 2014-12-21 18:16:05 (693 KB/s) - '.listing' saved [88] PLAINFILE; perms 444; size: 0; month: Dec; day: 18; time: 00:00:00 (no yr); Skipping. ^^ PLAINFILE; perms 444; size: 0; month: Dec; day: 18; time: 00:00:00 (no yr); Skipping. Removed '.listing'. Which then prompted me to take a closer look at the listing wget receives as result of LIST: -r--r--r-- 1 0 0 12 Dec 18:17 afile.txt ^^^ Look, ma: no day of the month! And that leads me to this line in FTPServer.pm: my $date = strftime(%b %e %H:%M, localtime); Which is the root cause of the problem: it uses %e, which is a C99 feature, and is not supported on MS-Windows. I replaced it with %d, and now all the FTP tests succeed! So I suggest to use %d there on all platforms: that listing is not for human consumption, so leading zeros will not annoy anyone. Thanks. P.S. Now I'm beginning to wonder whether anyone runs the test suite on Windows, or cares about the results...
Re: [Bug-wget] FTP tests fail on MS-Windows
Am Sonntag, 21. Dezember 2014, 18:30:28 schrieb Eli Zaretskii: From: Tim Rühsen tim.rueh...@gmx.de Date: Sun, 21 Dec 2014 13:13:50 +0100 The .listing file can't be parsed correctly when the month names are incorrect. I see. But then this is not the problem in my case. Here's the listing I see in one of the FTP test logs: 226 Listing complete. Data connection has been closed. -r--r--r-- 1 0 0 12 Dec 12:43 franτais.txt 2014-12-19 12:43:03 (362 KB/s) - '.listing' saved [48] As you see, even though the file name includes non-ASCII characters, the month name is in English (which is what I'd expect, given the locale I have here). But I know, there are a few Windows users / developers reading this. Maybe they can help or bring some light !? I certainly hope so. Could the problem be that the listing has a CR-LF end-of-line format? Could that interfere with its parsing? Thanks. Back to Test-ftp-bad-list.px... if you add the following line to FTPServer.pm, you can see the difference between the GNU/Linux and the Windows .listing. The listing has two lines, so a difference of 4 bytes (reported in your original post) seems not to be a CRLF problem. Right you are. I looked at it from the other end: enabled debugging output from wget, and then looked carefully at the output. And I saw this: .listing[ = ] 88 --.-KB/s in 0s Closed fd 4 226 Listing complete. Data connection has been closed. 2014-12-21 18:16:05 (693 KB/s) - '.listing' saved [88] PLAINFILE; perms 444; size: 0; month: Dec; day: 18; time: 00:00:00 (no yr); Skipping. ^^ PLAINFILE; perms 444; size: 0; month: Dec; day: 18; time: 00:00:00 (no yr); Skipping. Removed '.listing'. Which then prompted me to take a closer look at the listing wget receives as result of LIST: -r--r--r-- 1 0 0 12 Dec 18:17 afile.txt ^^^ Look, ma: no day of the month! Outch ! And that leads me to this line in FTPServer.pm: my $date = strftime(%b %e %H:%M, localtime); Which is the root cause of the problem: it uses %e, which is a C99 feature, and is not supported on MS-Windows. I replaced it with %d, and now all the FTP tests succeed! Thanks for testing, I'll set up a patch for using %d. %e is not C99 but Single Unix Specification. And Windows / MS doesn't care for it. FYI, see http://msdn.microsoft.com/en-us/library/fe06s4ak.aspx I normally never use strftime in my applications, now I remember one of the reasons ;-). P.S. Now I'm beginning to wonder whether anyone runs the test suite on Windows, or cares about the results... Looks like nobody does :-( Tim signature.asc Description: This is a digitally signed message part.
Re: [Bug-wget] FTP tests fail on MS-Windows
Am Freitag, 19. Dezember 2014, 18:18:31 schrieb Eli Zaretskii: I've built wget 1.16.1 on GNU/Linux as well, and compared the test results to try and figure out why some tests fail on Windows. When I run Test-ftp-bad-list.px on GNU/Linux, I see this output: Running test Test-ftp-bad-list Calling /srv/data/home/e/eliz/wget-1.16.1/tests/../src/wget -nH -Nc -r ftp://localhost:40221/ --2014-12-19 11:13:27-- ftp://localhost:40221/ = ‘.listing’ Resolving localhost (localhost)... ::1, 127.0.0.1 Connecting to localhost (localhost)|::1|:40221... failed: Connection refused. Connecting to localhost (localhost)|127.0.0.1|:40221... connected. Logging in as anonymous ... Logged in! == SYST ... done.== PWD ... done. == TYPE I ... done. == CWD not needed. == PASV ... done.== LIST ... _LIST_command - dir is: / done. .listing[ = ] 92 --.-KB/s in 0s 2014-12-19 11:13:27 (3.65 MB/s) - ‘.listing’ saved [92] Removed ‘.listing’. The sizes do not match (local 12) -- retrieving. --2014-12-19 11:13:27-- ftp://localhost:40221/afile.txt = ‘afile.txt’ == CWD not required. == SIZE afile.txt ... 12 File has already been retrieved. 2014-12-19 11:13:27 (0.00 B/s) - ‘afile.txt’ saved [12] But on Windows the same test yields this: Running test Test-ftp-bad-list Calling /d/gnu/wget-1.16.1/tests/../src/wget -nH -Nc -r ftp://localhost:3244/ --2014-12-19 18:14:54-- ftp://localhost:3244/ = '.listing' Resolving localhost (localhost)... 127.0.0.1 Connecting to localhost (localhost)|127.0.0.1|:3244... connected. Logging in as anonymous ... Logged in! == SYST ... done.== PWD ... done. == TYPE I ... done. == CWD not needed. == PASV ... done.== LIST ... _LIST_command - dir is: / done. .listing[ = ] 88 --.-KB/s in 0s 2014-12-19 18:14:54 (818 KB/s) - '.listing' saved [88] Removed '.listing'. --2014-12-19 18:14:54-- ftp://localhost:3244/ = 'index.html' == CWD not required. == SIZE ... done. == PASV ... done.== RETR ... No such file ''. Note the 'index.html' thing -- that's where the difference between the two systems begins. Looks like the server behaves differently, doesn't it? Does anyone have a clue what is going on here? Any bells ring for anyone? I fixed a locale thing a while ago. It was FTPServer.pm using strftime() generating non-english month names. Wget expects english month names (of course). What fixed that for me was a change in WgetTests.pm: ... # FTP Server has to start with english locale due to use of strftime month names in LIST command setlocale(LC_ALL, 'C'); $self-_launch_server( ... Maybe this is not enough in your environment... That also might be the reason for other failures. Do you have a recent version of the auto* tools installed ? Maybe you can tell me your locale settings and/or try using the same locale settings on Gnu/Linux. 'make check' explicitly sets LC_ALL=C. So i could produce these errors only with TESTS_ENVIRONMENT=LC_ALL=de_DE.utf8 make check Since these things are 'fixed for me', I can't reproduce them any more. And thus can't be of much further help. Tim signature.asc Description: This is a digitally signed message part.
Re: [Bug-wget] FTP tests fail on MS-Windows
From: Tim Rühsen tim.rueh...@gmx.de Date: Sat, 20 Dec 2014 18:18:20 +0100 But on Windows the same test yields this: Running test Test-ftp-bad-list Calling /d/gnu/wget-1.16.1/tests/../src/wget -nH -Nc -r ftp://localhost:3244/ --2014-12-19 18:14:54-- ftp://localhost:3244/ = '.listing' Resolving localhost (localhost)... 127.0.0.1 Connecting to localhost (localhost)|127.0.0.1|:3244... connected. Logging in as anonymous ... Logged in! == SYST ... done.== PWD ... done. == TYPE I ... done. == CWD not needed. == PASV ... done.== LIST ... _LIST_command - dir is: / done. .listing[ = ] 88 --.-KB/s in 0s 2014-12-19 18:14:54 (818 KB/s) - '.listing' saved [88] Removed '.listing'. --2014-12-19 18:14:54-- ftp://localhost:3244/ = 'index.html' == CWD not required. == SIZE ... done. == PASV ... done.== RETR ... No such file ''. Note the 'index.html' thing -- that's where the difference between the two systems begins. Looks like the server behaves differently, doesn't it? Does anyone have a clue what is going on here? Any bells ring for anyone? I fixed a locale thing a while ago. It was FTPServer.pm using strftime() generating non-english month names. Wget expects english month names (of course). What fixed that for me was a change in WgetTests.pm: ... # FTP Server has to start with english locale due to use of strftime month names in LIST command setlocale(LC_ALL, 'C'); $self-_launch_server( Thanks. But how can this explain the 'index.html' thingy appearing in the FTP listing, instead of the expected afile.txt etc.? Do you have a recent version of the auto* tools installed ? For some value of recent, yes. How are autotools related to this issue? Maybe you can tell me your locale settings and/or try using the same locale settings on Gnu/Linux. It's the Windows equivalent of en_US.cp1255. 'make check' explicitly sets LC_ALL=C. So i could produce these errors only with TESTS_ENVIRONMENT=LC_ALL=de_DE.utf8 make check I see the same problems no matter whether I run make check or the individual tests from the Bash prompt.
Re: [Bug-wget] FTP tests fail on MS-Windows
Am Samstag, 20. Dezember 2014, 22:06:01 schrieb Eli Zaretskii: From: Tim Rühsen tim.rueh...@gmx.de Date: Sat, 20 Dec 2014 18:18:20 +0100 But on Windows the same test yields this: Running test Test-ftp-bad-list Calling /d/gnu/wget-1.16.1/tests/../src/wget -nH -Nc -r ftp://localhost:3244/ --2014-12-19 18:14:54-- ftp://localhost:3244/ = '.listing' Resolving localhost (localhost)... 127.0.0.1 Connecting to localhost (localhost)|127.0.0.1|:3244... connected. Logging in as anonymous ... Logged in! == SYST ... done.== PWD ... done. == TYPE I ... done. == CWD not needed. == PASV ... done.== LIST ... _LIST_command - dir is: / done. .listing[ = ] 88 --.-KB/s in 0s 2014-12-19 18:14:54 (818 KB/s) - '.listing' saved [88] Removed '.listing'. --2014-12-19 18:14:54-- ftp://localhost:3244/ = 'index.html' == CWD not required. == SIZE ... done. == PASV ... done.== RETR ... No such file ''. Note the 'index.html' thing -- that's where the difference between the two systems begins. Looks like the server behaves differently, doesn't it? Does anyone have a clue what is going on here? Any bells ring for anyone? I fixed a locale thing a while ago. It was FTPServer.pm using strftime() generating non-english month names. Wget expects english month names (of course). What fixed that for me was a change in WgetTests.pm: ... # FTP Server has to start with english locale due to use of strftime month names in LIST command setlocale(LC_ALL, 'C'); $self-_launch_server( Thanks. But how can this explain the 'index.html' thingy appearing in the FTP listing, instead of the expected afile.txt etc.? The .listing file can't be parsed correctly when the month names are incorrect. Do you have a recent version of the auto* tools installed ? For some value of recent, yes. How are autotools related to this issue? 'make check' explicitly sets LC_ALL=C. For some reason it sems to fail in your environment. You could add a print line to the server perl code to see what the settings are it runs with. Maybe you can tell me your locale settings and/or try using the same locale settings on Gnu/Linux. It's the Windows equivalent of en_US.cp1255. 'make check' explicitly sets LC_ALL=C. So i could produce these errors only with TESTS_ENVIRONMENT=LC_ALL=de_DE.utf8 make check I see the same problems no matter whether I run make check or the individual tests from the Bash prompt. Either auto* tools are broken (or let's say the test suite code) or your environment behavesw unexpected. But I simply can't fix it for you, I just don't (and won't) have your system / environment and can't reproduce these issues. I just can guess from what you tell me. But I know, there are a few Windows users / developers reading this. Maybe they can help or bring some light !? Tim signature.asc Description: This is a digitally signed message part.
Re: [Bug-wget] FTP tests fail on MS-Windows
From: Tim Rühsen tim.rueh...@gmx.de Date: Sat, 20 Dec 2014 22:16:16 +0100 # FTP Server has to start with english locale due to use of strftime month names in LIST command setlocale(LC_ALL, 'C'); $self-_launch_server( Thanks. But how can this explain the 'index.html' thingy appearing in the FTP listing, instead of the expected afile.txt etc.? The .listing file can't be parsed correctly when the month names are incorrect. I see. But then this is not the problem in my case. Here's the listing I see in one of the FTP test logs: 226 Listing complete. Data connection has been closed. -r--r--r-- 1 0 0 12 Dec 12:43 franτais.txt 2014-12-19 12:43:03 (362 KB/s) - '.listing' saved [48] As you see, even though the file name includes non-ASCII characters, the month name is in English (which is what I'd expect, given the locale I have here). But I know, there are a few Windows users / developers reading this. Maybe they can help or bring some light !? I certainly hope so. Could the problem be that the listing has a CR-LF end-of-line format? Could that interfere with its parsing? Thanks.
[Bug-wget] FTP tests fail on MS-Windows
I've built wget 1.16.1 on GNU/Linux as well, and compared the test results to try and figure out why some tests fail on Windows. When I run Test-ftp-bad-list.px on GNU/Linux, I see this output: Running test Test-ftp-bad-list Calling /srv/data/home/e/eliz/wget-1.16.1/tests/../src/wget -nH -Nc -r ftp://localhost:40221/ --2014-12-19 11:13:27-- ftp://localhost:40221/ = ‘.listing’ Resolving localhost (localhost)... ::1, 127.0.0.1 Connecting to localhost (localhost)|::1|:40221... failed: Connection refused. Connecting to localhost (localhost)|127.0.0.1|:40221... connected. Logging in as anonymous ... Logged in! == SYST ... done.== PWD ... done. == TYPE I ... done. == CWD not needed. == PASV ... done.== LIST ... _LIST_command - dir is: / done. .listing[ = ] 92 --.-KB/s in 0s 2014-12-19 11:13:27 (3.65 MB/s) - ‘.listing’ saved [92] Removed ‘.listing’. The sizes do not match (local 12) -- retrieving. --2014-12-19 11:13:27-- ftp://localhost:40221/afile.txt = ‘afile.txt’ == CWD not required. == SIZE afile.txt ... 12 File has already been retrieved. 2014-12-19 11:13:27 (0.00 B/s) - ‘afile.txt’ saved [12] But on Windows the same test yields this: Running test Test-ftp-bad-list Calling /d/gnu/wget-1.16.1/tests/../src/wget -nH -Nc -r ftp://localhost:3244/ --2014-12-19 18:14:54-- ftp://localhost:3244/ = '.listing' Resolving localhost (localhost)... 127.0.0.1 Connecting to localhost (localhost)|127.0.0.1|:3244... connected. Logging in as anonymous ... Logged in! == SYST ... done.== PWD ... done. == TYPE I ... done. == CWD not needed. == PASV ... done.== LIST ... _LIST_command - dir is: / done. .listing[ = ] 88 --.-KB/s in 0s 2014-12-19 18:14:54 (818 KB/s) - '.listing' saved [88] Removed '.listing'. --2014-12-19 18:14:54-- ftp://localhost:3244/ = 'index.html' == CWD not required. == SIZE ... done. == PASV ... done.== RETR ... No such file ''. Note the 'index.html' thing -- that's where the difference between the two systems begins. Looks like the server behaves differently, doesn't it? Does anyone have a clue what is going on here? Any bells ring for anyone? TIA P.S. Please CC me on any replies, as I'm not subscribed.
