Am 20.01.2014 11:21, schrieb Daniel Stenberg: > Yes, I wouldn't be surprised. A directory listing is done in text mode > with FTP so I'd say CRLF line endings would be expected on Windows and > LF elsewhere. > > This is probably part of the known bug #21, "FTP ASCII transfers do > not follow RFC959".
I think that curl actually does nothing to the FTP LIST output. ftpserver.pl actually does not send a correct FTP LISTing since it is using [CR] for test case 100 instead of [CR][LF] in accordance with RFC959. At least that is as for as I understand and interpret section 2.2 of RFC959. Please correct me if I am wrong here. I also figured that it would probably make more sense to change the test suite to convert the expected output to Windows line-endings for text-aware test cases instead of forcing the actual output to contain only [LF]. And since runtests.pl already supported text-mode data sections, I quickly added support for it in datacheck-section. Attached you will find 4 patches that should improve quite a lot of the FTP LISTing tests on Windows. Before pushing to the git repository, I would appreciate if someone could run them on his well-tested Unix build environment to make sure that I did not break the tests on Unix systems. Now we only need to figure out how to improve the remaining FTP-data and non-FTP test cases where the curl tool automatically outputs [CR][LF] through the Console buffer, but [LF] actually is expected since no conversion is desired. Best regards, Marc
>From 389d9a796d672903f6f980d442f137a9a3f0dab1 Mon Sep 17 00:00:00 2001 From: Marc Hoersken <[email protected]> Date: Sun, 26 Jan 2014 11:23:11 +0100 Subject: [PATCH 1/4] runtests.pl: reverse line-ending conversion on Windows It makes more sense to convert the expected output to [CR][LF] on Windows than to force the actual, probably correct, output to [LF]. This way it is actually possible to see if curl outputs the correct line-ending excepted by a text-aware test case. --- tests/runtests.pl | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/tests/runtests.pl b/tests/runtests.pl index b459c33..e63aa03 100755 --- a/tests/runtests.pl +++ b/tests/runtests.pl @@ -3616,7 +3616,8 @@ sub singletest { my $filemode=$hash{'mode'}; if($filemode && ($filemode eq "text") && $has_textaware) { # text mode when running on windows: fix line endings - map s/\r\n/\n/g, @actual; + map s/\r\n/\n/g, @validstdout; + map s/\n/\r\n/g, @validstdout; } if($hash{'nonewline'}) { @@ -3644,7 +3645,8 @@ sub singletest { my $filemode=$hash{'mode'}; if($filemode && ($filemode eq "text") && $has_textaware) { # text mode when running on windows: fix line endings - map s/\r\n/\n/g, @out; + map s/\r\n/\n/g, @reply; + map s/\n/\r\n/g, @reply; } $res = compare($testnum, $testname, "data", \@out, \@reply); @@ -3788,9 +3790,9 @@ sub singletest { my $filemode=$hash{'mode'}; if($filemode && ($filemode eq "text") && $has_textaware) { - # text mode when running on windows means adding an extra - # strip expression - push @stripfile, "s/\r\n/\n/"; + # text mode when running on windows: fix line endings + map s/\r\n/\n/g, @outfile; + map s/\n/\r\n/g, @outfile; } my $strip; -- 1.8.1.msysgit.1
>From 72c64836babb5815c19c712c63a587d472f7f29f Mon Sep 17 00:00:00 2001 From: Marc Hoersken <[email protected]> Date: Sun, 26 Jan 2014 11:30:41 +0100 Subject: [PATCH 2/4] ftpserver.pl: directory LISTings use [CR][LF] for ASCII transfer According to section 2.2 of RFC959 the End-of-Line is defined as: The end-of-line sequence defines the separation of printing lines. The sequence is Carriage Return, followed by Line Feed. Verified by sniffing traffic between a Windows FTP client (FileZilla) and Unix-hosted FTP server (ProFTPD). --- tests/ftpserver.pl | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/ftpserver.pl b/tests/ftpserver.pl index f2b12eb..0116e20 100755 --- a/tests/ftpserver.pl +++ b/tests/ftpserver.pl @@ -2160,6 +2160,9 @@ my @ftpdir=("total 20\r\n", my @data = getpart("reply", "data"); for(@data) { my $send = $_; + # convert all \n to \r\n for ASCII transfer + $send =~ s/\r\n/\n/g; + $send =~ s/\n/\r\n/g; logmsg "send $send as data\n"; senddata $send; } -- 1.8.1.msysgit.1
>From 518a44b360c52159204d8e0fb4942c71e5be8860 Mon Sep 17 00:00:00 2001 From: Marc Hoersken <[email protected]> Date: Sun, 26 Jan 2014 11:31:52 +0100 Subject: [PATCH 3/4] runtests.pl: added support for text-mode within datacheck section --- tests/runtests.pl | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/tests/runtests.pl b/tests/runtests.pl index e63aa03..6509694 100755 --- a/tests/runtests.pl +++ b/tests/runtests.pl @@ -3147,15 +3147,20 @@ sub singletest { my @reply = getpart("reply", "data"); my @replycheck = getpart("reply", "datacheck"); + my %replyattr = getpartattr("reply", "data"); + my %replycheckattr = getpartattr("reply", "datacheck"); + if (@replycheck) { # we use this file instead to check the final output against - my %hash = getpartattr("reply", "datacheck"); - if($hash{'nonewline'}) { + if($replycheckattr{'nonewline'}) { # Yes, we must cut off the final newline from the final line # of the datacheck chomp($replycheck[$#replycheck]); } + if($replycheckattr{'mode'}) { + $replyattr{'mode'} = $replycheckattr{'mode'}; + } @reply=@replycheck; } @@ -3636,13 +3641,11 @@ sub singletest { $ok .= "-"; # stdout not checked } - my %replyattr = getpartattr("reply", "data"); if(!$replyattr{'nocheck'} && (@reply || $replyattr{'sendzero'})) { # verify the received data my @out = loadarray($CURLOUT); - my %hash = getpartattr("reply", "data"); # get the mode attribute - my $filemode=$hash{'mode'}; + my $filemode=$replyattr{'mode'}; if($filemode && ($filemode eq "text") && $has_textaware) { # text mode when running on windows: fix line endings map s/\r\n/\n/g, @reply; -- 1.8.1.msysgit.1
>From 7324f720dac1833a13fead596f5992a05421903e Mon Sep 17 00:00:00 2001 From: Marc Hoersken <[email protected]> Date: Sun, 26 Jan 2014 11:32:44 +0100 Subject: [PATCH 4/4] FTP tests: enable text-mode for data and datacheck sections --- tests/data/test100 | 2 +- tests/data/test101 | 2 +- tests/data/test130 | 2 +- tests/data/test131 | 2 +- tests/data/test132 | 2 +- tests/data/test133 | 2 +- tests/data/test134 | 2 +- tests/data/test215 | 2 +- tests/data/test250 | 2 +- tests/data/test251 | 2 +- tests/data/test280 | 2 +- tests/data/test294 | 2 +- tests/data/test350 | 2 +- tests/data/test351 | 2 +- tests/data/test352 | 2 +- tests/data/test353 | 2 +- tests/data/test521 | 2 +- tests/data/test539 | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/tests/data/test100 b/tests/data/test100 index 6e0f734..72f9c85 100644 --- a/tests/data/test100 +++ b/tests/data/test100 @@ -11,7 +11,7 @@ LIST <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<data> +<data mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test101 b/tests/data/test101 index 815292b..09faade 100644 --- a/tests/data/test101 +++ b/tests/data/test101 @@ -10,7 +10,7 @@ LIST <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test130 b/tests/data/test130 index dcc46fc..827b1a0 100644 --- a/tests/data/test130 +++ b/tests/data/test130 @@ -12,7 +12,7 @@ netrc <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test131 b/tests/data/test131 index 86501c6..6e99d8a 100644 --- a/tests/data/test131 +++ b/tests/data/test131 @@ -13,7 +13,7 @@ netrc # # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test132 b/tests/data/test132 index ed96953..51cb89c 100644 --- a/tests/data/test132 +++ b/tests/data/test132 @@ -12,7 +12,7 @@ netrc <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test133 b/tests/data/test133 index 085092d..0b4d2ff 100644 --- a/tests/data/test133 +++ b/tests/data/test133 @@ -12,7 +12,7 @@ netrc <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test134 b/tests/data/test134 index 8a3ba62..e314b66 100644 --- a/tests/data/test134 +++ b/tests/data/test134 @@ -12,7 +12,7 @@ netrc <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test215 b/tests/data/test215 index 8770efb..987fd99 100644 --- a/tests/data/test215 +++ b/tests/data/test215 @@ -9,7 +9,7 @@ FTP <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test250 b/tests/data/test250 index aeed134..3d0a8f3 100644 --- a/tests/data/test250 +++ b/tests/data/test250 @@ -10,7 +10,7 @@ FTP <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test251 b/tests/data/test251 index d50e1b3..cc8cbfa 100644 --- a/tests/data/test251 +++ b/tests/data/test251 @@ -9,7 +9,7 @@ FTP <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test280 b/tests/data/test280 index c169993..de7534e 100644 --- a/tests/data/test280 +++ b/tests/data/test280 @@ -12,7 +12,7 @@ LIST <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test294 b/tests/data/test294 index 4d7f2b8..a9752a2 100644 --- a/tests/data/test294 +++ b/tests/data/test294 @@ -13,7 +13,7 @@ ACCT <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test350 b/tests/data/test350 index 9c22d30..6cb53fa 100644 --- a/tests/data/test350 +++ b/tests/data/test350 @@ -11,7 +11,7 @@ LIST <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test351 b/tests/data/test351 index 5d44a64..219677f 100644 --- a/tests/data/test351 +++ b/tests/data/test351 @@ -11,7 +11,7 @@ LIST <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test352 b/tests/data/test352 index d868894..5f498a2 100644 --- a/tests/data/test352 +++ b/tests/data/test352 @@ -11,7 +11,7 @@ LIST <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test353 b/tests/data/test353 index 417b8c3..1adee37 100644 --- a/tests/data/test353 +++ b/tests/data/test353 @@ -11,7 +11,7 @@ LIST <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test521 b/tests/data/test521 index 9bc5501..90f51b3 100644 --- a/tests/data/test521 +++ b/tests/data/test521 @@ -11,7 +11,7 @@ CURLOPT_PORT <reply> # When doing LIST, we get the default list output hard-coded in the test # FTP server -<datacheck> +<datacheck mode="text"> total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . drwxr-xr-x 8 98 98 512 Oct 22 13:06 .. diff --git a/tests/data/test539 b/tests/data/test539 index 15da794..593fcac 100644 --- a/tests/data/test539 +++ b/tests/data/test539 @@ -12,7 +12,7 @@ FTP file contents </data> -<datacheck> +<datacheck mode="text"> file contents total 20 drwxr-xr-x 8 98 98 512 Oct 22 13:06 . -- 1.8.1.msysgit.1
------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
