Fabian Keil <[email protected]> wrote: > Fabian Keil <[email protected]> wrote: > > > Daniel Stenberg <[email protected]> wrote: > > > > > On Wed, 21 May 2014, Fabian Keil wrote: > > > > > > > I'm still doing that and would like to get the required runtests.pl > > > > changes > > > > upstream after making them more generic. > > > > > > Thanks for keeping it up Fabian. I'm in favor of making sure that the > > > curl > > > test suite can be re-used for more purposes as I believe it can help > > > improving > > > more code and it will help the test suite and thus it also improves curl > > > itself. > > > > Great. I'll look into making the patches more generic in the near feature. > > Attached are generic versions of the patches.
Additional patches are attached. For my use case I only "need" patches 1 and 2 (and some of the previous patches). Patch 3 is convenient but the implementation could be considered ugly and it could be replaced with a script that copies an existing test and updates the test number on the fly. Instead of patch 2, httpserver.pl's and sws's "--srcdir" arguments could be renamed "--testdir". This would be a more intrusive change, but it would make the purpose of the argument more obvious (unless I miss something, srcdir is only used to derive the testdir) and wouldn't require the testdir to be named "data". Fabian
From 7f785fdcef1eaebcd7ba86a0378742835c9e574a Mon Sep 17 00:00:00 2001 From: Fabian Keil <[email protected]> Date: Sun, 15 Jun 2014 17:19:58 +0200 Subject: [PATCH 1/6] Allow to overwrite $TESTDIR through the environment This makes it more convenient to execute custom tests that aren't relevant for curl itself and thus not upstreamed. --- tests/runtests.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/runtests.pl b/tests/runtests.pl index a8ffd3c..74f07a2 100755 --- a/tests/runtests.pl +++ b/tests/runtests.pl @@ -153,7 +153,7 @@ my $VCURL=$CURL; # what curl binary to use to verify the servers with # just built hangs or crashes and thus prevent verification my $DBGCURL=$CURL; #"../src/.libs/curl"; # alternative for debugging my $LOGDIR="log"; -my $TESTDIR="$srcdir/data"; +my $TESTDIR = $ENV{'TESTDIR'} || "$srcdir/data"; my $LIBDIR="./libtest"; my $UNITDIR="./unit"; # TODO: change this to use server_inputfilename() -- 1.9.0
From 87b574a186813d5f621c395da9208df784e20cd0 Mon Sep 17 00:00:00 2001 From: Fabian Keil <[email protected]> Date: Mon, 16 Jun 2014 14:59:11 +0200 Subject: [PATCH 2/6] runtests.pl: Let runhttpserver() use $TESTDIR instead of $srcdir --- tests/runtests.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/runtests.pl b/tests/runtests.pl index 74f07a2..d6401a4 100755 --- a/tests/runtests.pl +++ b/tests/runtests.pl @@ -1240,7 +1240,7 @@ sub runhttpserver { $flags .= $verbose_flag if($debugprotocol); $flags .= "--pidfile \"$pidfile\" --logfile \"$logfile\" "; $flags .= "--id $idnum " if($idnum > 1); - $flags .= "--ipv$ipvnum --port $port --srcdir \"$srcdir\""; + $flags .= "--ipv$ipvnum --port $port --srcdir \"$TESTDIR/..\""; my $cmd = "$exe $flags"; my ($httppid, $pid2) = startnew($cmd, $pidfile, 15, 0); -- 1.9.0
From c54a1092fdb66be0e73d54a2444b642fdfebd289 Mon Sep 17 00:00:00 2001 From: Fabian Keil <[email protected]> Date: Mon, 16 Jun 2014 16:03:00 +0200 Subject: [PATCH 3/6] runtests.pl: Add a %TESTNUMBER variable to make copying tests more convenient --- tests/runtests.pl | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/tests/runtests.pl b/tests/runtests.pl index d6401a4..29544a0 100755 --- a/tests/runtests.pl +++ b/tests/runtests.pl @@ -2625,7 +2625,7 @@ sub checksystem { # a command, in either case passed by reference # sub subVariables { - my ($thing) = @_; + my ($thing, $testnum) = @_; # ports @@ -2679,6 +2679,7 @@ sub subVariables { $$thing =~ s/%PWD/$pwd/g; $$thing =~ s/%SRCDIR/$srcdir/g; $$thing =~ s/%USER/$USER/g; + $$thing =~ s/%TESTNUMBER/$testnum/g; # The purpose of FTPTIME2 and FTPTIME3 is to provide times that can be # used for time-out tests and that whould work on most hosts as these @@ -2693,10 +2694,11 @@ sub subVariables { } sub fixarray { + my $testnum = shift; my @in = @_; for(@in) { - subVariables \$_; + subVariables \$_, $testnum; } return @in; } @@ -3115,7 +3117,7 @@ sub singletest { if(@setenv) { foreach my $s (@setenv) { chomp $s; - subVariables \$s; + subVariables \$s, $testnum; if($s =~ /([^=]*)=(.*)/) { my ($var, $content) = ($1, $2); # remember current setting, to restore it once test runs @@ -3155,7 +3157,7 @@ sub singletest { if(@precheck) { $cmd = $precheck[0]; chomp $cmd; - subVariables \$cmd; + subVariables \$cmd, $testnum; if($cmd) { my @p = split(/ /, $cmd); if($p[0] !~ /\//) { @@ -3223,17 +3225,17 @@ sub singletest { } # this is the valid protocol blurb curl should generate - my @protocol= fixarray ( getpart("verify", "protocol") ); + my @protocol= fixarray ( $testnum, getpart("verify", "protocol") ); # this is the valid protocol blurb curl should generate to a proxy - my @proxyprot = fixarray ( getpart("verify", "proxy") ); + my @proxyprot = fixarray ( $testnum, getpart("verify", "proxy") ); # redirected stdout/stderr to these files $STDOUT="$LOGDIR/stdout$testnum"; $STDERR="$LOGDIR/stderr$testnum"; # if this section exists, we verify that the stdout contained this: - my @validstdout = fixarray ( getpart("verify", "stdout") ); + my @validstdout = fixarray ( $testnum, getpart("verify", "stdout") ); # if this section exists, we verify upload my @upload = getpart("verify", "upload"); @@ -3280,7 +3282,7 @@ sub singletest { # make some nice replace operations $cmd =~ s/\n//g; # no newlines please # substitute variables in the command line - subVariables \$cmd; + subVariables \$cmd, $testnum; } else { # there was no command given, use something silly @@ -3301,7 +3303,7 @@ sub singletest { return -1; } my $fileContent = join('', @inputfile); - subVariables \$fileContent; + subVariables \$fileContent, $testnum; # logmsg "DEBUG: writing file " . $filename . "\n"; open(OUTFILE, ">$filename"); binmode OUTFILE; # for crapage systems, use binary @@ -3633,7 +3635,7 @@ sub singletest { if(@postcheck) { $cmd = $postcheck[0]; chomp $cmd; - subVariables \$cmd; + subVariables \$cmd, $testnum; if($cmd) { logmsg "postcheck $cmd\n" if($verbose); my $rc = runclient("$cmd"); @@ -3680,7 +3682,7 @@ sub singletest { my @actual = loadarray($STDOUT); # variable-replace in the stdout we have from the test case file - @validstdout = fixarray(@validstdout); + @validstdout = fixarray($testnum, @validstdout); # get all attributes my %hash = getpartattr("verify", "stdout"); @@ -3881,7 +3883,7 @@ sub singletest { @generated = @newgen; } - @outfile = fixarray(@outfile); + @outfile = fixarray($testnum, @outfile); $res = compare($testnum, $testname, "output ($filename)", \@generated, \@outfile); -- 1.9.0
From 8eece31f6198f1ffc854c3ddc9c5a3ab5fa6acce Mon Sep 17 00:00:00 2001 From: Fabian Keil <[email protected]> Date: Sun, 15 Jun 2014 17:21:14 +0200 Subject: [PATCH 4/6] getpart.pm: Fix a comment typo --- tests/getpart.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/getpart.pm b/tests/getpart.pm index 4548833..0729cf4 100644 --- a/tests/getpart.pm +++ b/tests/getpart.pm @@ -239,7 +239,7 @@ sub writearray { } # -# Load a specified file an return it as an array +# Load a specified file and return it as an array # sub loadarray { my ($filename)=@_; -- 1.9.0
From cf74104545324109c40d8205f901b55574331908 Mon Sep 17 00:00:00 2001 From: Fabian Keil <[email protected]> Date: Thu, 19 Jun 2014 16:40:06 +0200 Subject: [PATCH 5/6] runtests.pl: Don't expect $TESTDIR/DISABLED to exist If a non-standard $TESTDIR is used the file may not be necessary. Previously a "missing" file resulted in the warning: readline() on closed filehandle D at ./runtests.pl line 4940. --- tests/runtests.pl | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/tests/runtests.pl b/tests/runtests.pl index 29544a0..686c623 100755 --- a/tests/runtests.pl +++ b/tests/runtests.pl @@ -4960,20 +4960,21 @@ if(!$listonly) { } ####################################################################### -# Fetch all disabled tests +# Fetch all disabled tests, if there are any # -open(D, "<$TESTDIR/DISABLED"); -while(<D>) { - if(/^ *\#/) { - # allow comments - next; - } - if($_ =~ /(\d+)/) { - $disabled{$1}=$1; # disable this test number +if(open(D, "<$TESTDIR/DISABLED")) { + while(<D>) { + if(/^ *\#/) { + # allow comments + next; + } + if($_ =~ /(\d+)/) { + $disabled{$1}=$1; # disable this test number + } } + close(D); } -close(D); ####################################################################### # If 'all' tests are requested, find out all test numbers -- 1.9.0
From e22fc18654727eeb5c0e97074a91ffd82456fb04 Mon Sep 17 00:00:00 2001 From: Fabian Keil <[email protected]> Date: Thu, 19 Jun 2014 17:31:35 +0200 Subject: [PATCH 6/6] runtests.pl: Remove filteroff() which hasn't been used since 2001 --- tests/runtests.pl | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/tests/runtests.pl b/tests/runtests.pl index 686c623..729ce60 100755 --- a/tests/runtests.pl +++ b/tests/runtests.pl @@ -2230,37 +2230,10 @@ sub cleardir { } ####################################################################### -# filter out the specified pattern from the given input file and store the -# results in the given output file -# -sub filteroff { - my $infile=$_[0]; - my $filter=$_[1]; - my $ofile=$_[2]; - - open(IN, "<$infile") - || return 1; - - open(OUT, ">$ofile") - || return 1; - - # logmsg "FILTER: off $filter from $infile to $ofile\n"; - - while(<IN>) { - $_ =~ s/$filter//; - print OUT $_; - } - close(IN); - close(OUT); - return 0; -} - -####################################################################### # compare test results with the expected output, we might filter off # some pattern that is allowed to differ, output test results # sub compare { - # filter off patterns _before_ this comparison! my ($testnum, $testname, $subject, $firstref, $secondref)=@_; my $result = compareparts($firstref, $secondref); -- 1.9.0
signature.asc
Description: PGP signature
------------------------------------------------------------------- List admin: http://cool.haxx.se/list/listinfo/curl-library Etiquette: http://curl.haxx.se/mail/etiquette.html
