Re: [HACKERS] vcregress support for single TAP tests

[Andrew Dunstan]

On 05/02/2017 12:19 AM, Vaishnavi Prabakaran wrote:
>
>
>
> On Mon, May 1, 2017 at 11:01 PM, Andrew Dunstan
>  > wrote:
>
>
>
> In the absence of further comments I'm going to apply this and
> back-patch it so we can get a significant improvement in how the
> buildfarm reports results from TAP tests, as well as increased
> coverage,
> on Windows/MSVC machines.
>
>
> I see vcregress is not processing the second argument passed to it.
> For example, "vcregress.bat check serial" runs the regress test in
> parallel mode.
> Though this issue is present even before this patch is applied, am
> writing here because this patch touches the argument passing to
> sub-routine logic. 
> Attached is the one fix I find it working for me.
>
>


Oh, you're right, my bad. Thanks. Will apply.

cheers

andrew

-- 
Andrew Dunstanhttps://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] vcregress support for single TAP tests

[Vaishnavi Prabakaran]

On Mon, May 1, 2017 at 11:01 PM, Andrew Dunstan <
andrew.duns...@2ndquadrant.com> wrote:

>
>
> In the absence of further comments I'm going to apply this and
> back-patch it so we can get a significant improvement in how the
> buildfarm reports results from TAP tests, as well as increased coverage,
> on Windows/MSVC machines.
>
>
I see vcregress is not processing the second argument passed to it. For
example, "vcregress.bat check serial" runs the regress test in parallel
mode.
Though this issue is present even before this patch is applied, am writing
here because this patch touches the argument passing to sub-routine logic.
Attached is the one fix I find it working for me.

Thanks & Regards,
Vaishnavi,
Fujitsu Australia.


0001-Argument-passing-to-sub-routine-corrected-in-vcregre.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] vcregress support for single TAP tests

[Andrew Dunstan]

On 04/28/2017 08:54 AM, Andrew Dunstan wrote:
>
> On 04/26/2017 10:32 PM, Peter Eisentraut wrote:
>> On 4/23/17 17:09, Andrew Dunstan wrote:
>>> Here's a patch that will allow calling vcregress.pl to run a single TAP
>>> test set. It would work like this:
>>>
>>>
>>> vcregress.pl src/test/recover true
>>>
>>>
>>> The second argument if true (in the perl sense, of course) would trigger
>>> a temp install before running the tests. It defaults to off, in an
>>> attempt to minimize the unnecessary running of installs.
>> Seems kind of weird to have that functionality only for the tap tests,
>> though.
>>
>
>
> Yeah, you're right, I think we'd be much better mimicking what the
> makefiles do and honouring the NO_TEMP_INSTALL environment variable.
> Revised patch that works that way attached.
>
> So to run an arbitrary TAP test set, you would do (for example)
>
> vcregress.pl taptest src/test/recover
>
>


In the absence of further comments I'm going to apply this and
back-patch it so we can get a significant improvement in how the
buildfarm reports results from TAP tests, as well as increased coverage,
on Windows/MSVC machines.

cheers

andrew

-- 
Andrew Dunstanhttps://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] vcregress support for single TAP tests

[Andrew Dunstan]

On 04/26/2017 10:32 PM, Peter Eisentraut wrote:
> On 4/23/17 17:09, Andrew Dunstan wrote:
>> Here's a patch that will allow calling vcregress.pl to run a single TAP
>> test set. It would work like this:
>>
>>
>> vcregress.pl src/test/recover true
>>
>>
>> The second argument if true (in the perl sense, of course) would trigger
>> a temp install before running the tests. It defaults to off, in an
>> attempt to minimize the unnecessary running of installs.
> Seems kind of weird to have that functionality only for the tap tests,
> though.
>



Yeah, you're right, I think we'd be much better mimicking what the
makefiles do and honouring the NO_TEMP_INSTALL environment variable.
Revised patch that works that way attached.

So to run an arbitrary TAP test set, you would do (for example)

vcregress.pl taptest src/test/recover


cheers

andrew

-- 
Andrew Dunstanhttps://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 8933920..d387274 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -34,7 +34,7 @@ if (-e "src/tools/msvc/buildenv.pl")
 
 my $what = shift || "";
 if ($what =~
-/^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck|recoverycheck)$/i
+/^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck|recoverycheck|taptest)$/i
   )
 {
 	$what = uc $what;
@@ -54,13 +54,6 @@ copy("$Config/dummy_seclabel/dummy_seclabel.dll", "src/test/regress");
 
 $ENV{PATH} = "$topdir/$Config/libpq;$ENV{PATH}";
 
-my $schedule = shift;
-unless ($schedule)
-{
-	$schedule = "serial";
-	$schedule = "parallel" if ($what eq 'CHECK' || $what =~ /PARALLEL/);
-}
-
 if ($ENV{PERL5LIB})
 {
 	$ENV{PERL5LIB} = "$topdir/src/tools/msvc;$ENV{PERL5LIB}";
@@ -90,13 +83,14 @@ my %command = (
 	ISOLATIONCHECK => \&isolationcheck,
 	BINCHECK   => \&bincheck,
 	RECOVERYCHECK  => \&recoverycheck,
-	UPGRADECHECK   => \&upgradecheck,);
+	UPGRADECHECK   => \&upgradecheck,
+	TAPTEST=> \&taptest,);
 
 my $proc = $command{$what};
 
 exit 3 unless $proc;
 
-&$proc();
+&$proc(@_);
 
 exit 0;
 
@@ -104,6 +98,7 @@ exit 0;
 
 sub installcheck
 {
+	my $schedule = shift || 'serial';
 	my @args = (
 		"../../../$Config/pg_regress/pg_regress",
 		"--dlpath=.",
@@ -119,6 +114,7 @@ sub installcheck
 
 sub check
 {
+	my $schedule = shift || 'parallel';
 	InstallTemp();
 	chdir "${topdir}/src/test/regress";
 	my @args = (
@@ -145,7 +141,7 @@ sub ecpgcheck
 	exit $status if $status;
 	InstallTemp();
 	chdir "$topdir/src/interfaces/ecpg/test";
-	$schedule = "ecpg";
+	my $schedule = "ecpg";
 	my @args = (
 		"../../../../$Config/pg_regress_ecpg/pg_regress_ecpg",
 		"--bindir=",
@@ -219,6 +215,17 @@ sub bincheck
 	exit $mstat if $mstat;
 }
 
+sub taptest
+{
+	my $dir = shift;
+
+	die "no tests found!" unless -d "$topdir/$dir/t";
+
+	InstallTemp();
+	my $status = tap_check("$topdir/$dir");
+	exit $status if $status;
+}
+
 sub plcheck
 {
 	chdir "../../pl";
@@ -516,7 +523,6 @@ sub fetchRegressOpts
 	$m =~ s{\\\r?\n}{}g;
 	if ($m =~ /^\s*REGRESS_OPTS\s*\+?=(.*)/m)
 	{
-
 		# Substitute known Makefile variables, then ignore options that retain
 		# an unhandled variable reference.  Ignore anything that isn't an
 		# option starting with "--".
@@ -588,17 +594,21 @@ sub GetTests
 
 sub InstallTemp
 {
-	print "Setting up temp install\n\n";
-	Install("$tmp_installdir", "all", $config);
+	unless ($ENV{NO_TEMP_INSTALL})
+	{
+		print "Setting up temp install\n\n";
+		Install("$tmp_installdir", "all", $config);
+	}
 	$ENV{PATH} = "$tmp_installdir/bin;$ENV{PATH}";
 }
 
 sub usage
 {
 	print STDERR
-	  "Usage: vcregress.pl  [  ]\n\n",
+	  "Usage: vcregress.pl  [ arg [ , arg] ... ]\n\n",
 	  "Options for :\n",
 	  "  bincheck   run tests of utilities in src/bin/\n",
+	  "  taptest   run an arbitrary TAP test\n",
 	  "  check  deploy instance and run regression tests on it\n",
 	  "  contribcheck   run tests of modules in contrib/\n",
 	  "  ecpgcheck  run regression tests of ECPG\n",
@@ -608,8 +618,10 @@ sub usage
 	  "  plcheckrun tests of PL languages\n",
 	  "  recoverycheck  run recovery test suite\n",
 	  "  upgradecheck   run tests of pg_upgrade\n",
-	  "\nOptions for :\n",
+	  "\nOptions for arg: (used by check and installcheck)\n",
 	  "  serial serial mode\n",
-	  "  parallel   parallel mode\n";
+	  "  parallel   parallel mode\n",
+	  "\nOptions for args for taptest\n",
+	  "  test-dir   (required) directory where tests reside\n",
 	exit(1);
 }

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] vcregress support for single TAP tests

[Peter Eisentraut]

On 4/23/17 17:09, Andrew Dunstan wrote:
> 
> Here's a patch that will allow calling vcregress.pl to run a single TAP
> test set. It would work like this:
> 
> 
> vcregress.pl src/test/recover true
> 
> 
> The second argument if true (in the perl sense, of course) would trigger
> a temp install before running the tests. It defaults to off, in an
> attempt to minimize the unnecessary running of installs.

Seems kind of weird to have that functionality only for the tap tests,
though.

-- 
Peter Eisentraut  http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] vcregress support for single TAP tests

[Andrew Dunstan]

Here's a patch that will allow calling vcregress.pl to run a single TAP
test set. It would work like this:


vcregress.pl src/test/recover true


The second argument if true (in the perl sense, of course) would trigger
a temp install before running the tests. It defaults to off, in an
attempt to minimize the unnecessary running of installs.

Once we have this I will switch the buildfarm client to use it. The
previous bincheck procedure is kept to avoid breakage of scripts,
buildfarm clients etc.


cheers


andrew


-- 
Andrew Dunstanhttps://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 8933920..56c7a2d 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -34,7 +34,7 @@ if (-e "src/tools/msvc/buildenv.pl")
 
 my $what = shift || "";
 if ($what =~
-/^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck|recoverycheck)$/i
+/^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck|recoverycheck|taptest)$/i
   )
 {
 	$what = uc $what;
@@ -54,13 +54,6 @@ copy("$Config/dummy_seclabel/dummy_seclabel.dll", "src/test/regress");
 
 $ENV{PATH} = "$topdir/$Config/libpq;$ENV{PATH}";
 
-my $schedule = shift;
-unless ($schedule)
-{
-	$schedule = "serial";
-	$schedule = "parallel" if ($what eq 'CHECK' || $what =~ /PARALLEL/);
-}
-
 if ($ENV{PERL5LIB})
 {
 	$ENV{PERL5LIB} = "$topdir/src/tools/msvc;$ENV{PERL5LIB}";
@@ -90,13 +83,14 @@ my %command = (
 	ISOLATIONCHECK => \&isolationcheck,
 	BINCHECK   => \&bincheck,
 	RECOVERYCHECK  => \&recoverycheck,
-	UPGRADECHECK   => \&upgradecheck,);
+	UPGRADECHECK   => \&upgradecheck,
+	TAP_TEST   => \&taptest,);
 
 my $proc = $command{$what};
 
 exit 3 unless $proc;
 
-&$proc();
+&$proc(@_);
 
 exit 0;
 
@@ -104,6 +98,7 @@ exit 0;
 
 sub installcheck
 {
+	my $schedule = shift || 'serial';
 	my @args = (
 		"../../../$Config/pg_regress/pg_regress",
 		"--dlpath=.",
@@ -119,6 +114,7 @@ sub installcheck
 
 sub check
 {
+	my $schedule = shift || 'parallel';
 	InstallTemp();
 	chdir "${topdir}/src/test/regress";
 	my @args = (
@@ -145,7 +141,7 @@ sub ecpgcheck
 	exit $status if $status;
 	InstallTemp();
 	chdir "$topdir/src/interfaces/ecpg/test";
-	$schedule = "ecpg";
+	my $schedule = "ecpg";
 	my @args = (
 		"../../../../$Config/pg_regress_ecpg/pg_regress_ecpg",
 		"--bindir=",
@@ -219,6 +215,17 @@ sub bincheck
 	exit $mstat if $mstat;
 }
 
+sub taptest
+{
+	my ($dir, $needs_install) = @_;
+
+	die "no tests found!" unless -d "$topdir/$dir/t";
+
+	InstallTemp() if $needs_install;
+	my $status = tap_check("$topdir/$dir");
+	exit $status if $status;
+}
+
 sub plcheck
 {
 	chdir "../../pl";
@@ -516,7 +523,6 @@ sub fetchRegressOpts
 	$m =~ s{\\\r?\n}{}g;
 	if ($m =~ /^\s*REGRESS_OPTS\s*\+?=(.*)/m)
 	{
-
 		# Substitute known Makefile variables, then ignore options that retain
 		# an unhandled variable reference.  Ignore anything that isn't an
 		# option starting with "--".
@@ -596,9 +602,10 @@ sub InstallTemp
 sub usage
 {
 	print STDERR
-	  "Usage: vcregress.pl  [  ]\n\n",
+	  "Usage: vcregress.pl  [ arg [ , arg] ... ]\n\n",
 	  "Options for :\n",
 	  "  bincheck   run tests of utilities in src/bin/\n",
+	  "  taptest   run an arbitrary TAP test\n",
 	  "  check  deploy instance and run regression tests on it\n",
 	  "  contribcheck   run tests of modules in contrib/\n",
 	  "  ecpgcheck  run regression tests of ECPG\n",
@@ -608,8 +615,11 @@ sub usage
 	  "  plcheckrun tests of PL languages\n",
 	  "  recoverycheck  run recovery test suite\n",
 	  "  upgradecheck   run tests of pg_upgrade\n",
-	  "\nOptions for :\n",
+	  "\nOptions for arg: (used by check and installcheck)\n",
 	  "  serial serial mode\n",
-	  "  parallel   parallel mode\n";
+	  "  parallel   parallel mode\n",
+	  "\nOptions for args for taptest\n",
+	  "  test-dir   (required) directory where tests reside\n",
+	  "  install-needed (optional) if non-zero a temp install will be done\n";
 	exit(1);
 }

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers