Re: [HACKERS] pg_upgrade automatic testing

2011-12-05 Thread Peter Eisentraut
On sön, 2011-11-27 at 18:17 -0500, Tom Lane wrote:
 Peter Eisentraut pete...@gmx.net writes:
  I've committed it now, and some buildfarm members are failing with lack
  of shared memory, semaphores, or disk space.  Don't know what to do with
  that or why so many are failing like that.  We could create a way to
  omit the test if it becomes a problem.
 
 I believe the issue is that those BF members have kernel settings that
 only support running one postmaster at a time.  The way you've got this
 set up, it launches a new private postmaster during a make installcheck;
 which is not only problematic from a resource consumption standpoint,
 but seems to me to violate the spirit of make installcheck, because
 what it's testing is not the installed postmaster but a local instance.
 
 Can you confine the test to only occur in make check mode, not make
 installcheck, please?

FWIW, the original definition of installcheck is that it tests the
already installed programs, which is what this does (did).  But I agree
that the difference is minimal in this case.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-12-05 Thread Peter Eisentraut
On sön, 2011-11-27 at 19:12 -0500, Andrew Dunstan wrote:
 Contrib tests are only run by the buildfarm in installcheck mode, so 
 that will probably turn the tests off for the buildfarm, so +1 on that 

Does the new buildfarm modular thing support that some members could run
the checks through explicit configuration?

 I think these tests are probably somewhat ill-conceived. Note also 
 that shell scripts are not portable, and so these tests would not be 
 able to run on MSVC buildfarm members, even if they had been enabled in 
 the MSVC regression driver, which they have not. If we need a regression 
 driver script it needs to be written in Perl.

Anyone is free to rewrite the thing in a different language.

 Also note that the test as written is likely to add significantly to 
 buildfarm run times, as it will run the entire base regression suite 
 again, possibly several times.

Are there any restrictions on how long a buildfarm run is supposed to
take?

 Finally, I think that this is probably not what we really need.

What do you base your thinking on?

This test suite has already found a number of bugs in the pg_upgrade
procedure that no one else was able to find.  By that measure, it's
exactly what we need.

 I have 
 already started work (as I mentioned some weeks ago) on having the 
 buildfarm stash away a successful build and data directory, to be used 
 later in cross-version upgrade testing, which seems to me much more what 
 we need from something like the buildfarm. Maybe we could discuss how to 
 run something like that.

That is one part of the puzzle.  But once you have stashed away the old
source and data directory, you still need a test runner, which is
exactly what this provides you.

But note, cross-version pg_upgrade checks will not give you the full
value, even assuming that you can make them work at all in an unattended
way, because by default you won't be able to get a clean dumps match
result, at least without a lot of additional work to mangle the dump
output.  Most (or all) of the bugs found so far with this test suite
were for upgrades *from* whatever was the current version.  If we don't
have a current-to-current upgrade test suite, then we would only find
those years from now.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-11-28 Thread Andrew Dunstan



On 11/27/2011 06:17 PM, Tom Lane wrote:

Peter Eisentrautpete...@gmx.net  writes:

I've committed it now, and some buildfarm members are failing with lack
of shared memory, semaphores, or disk space.  Don't know what to do with
that or why so many are failing like that.  We could create a way to
omit the test if it becomes a problem.

I believe the issue is that those BF members have kernel settings that
only support running one postmaster at a time.  The way you've got this
set up, it launches a new private postmaster during a make installcheck;
which is not only problematic from a resource consumption standpoint,
but seems to me to violate the spirit of make installcheck, because
what it's testing is not the installed postmaster but a local instance.

Can you confine the test to only occur in make check mode, not make
installcheck, please?




Another thing that's annoying about this is that it doesn't give you any 
idea of how it's failing if there's a database difference. All we get is:


   Files 
/home/pgrunner/bf/root/HEAD/pgsql.3188/contrib/pg_upgrade/tmp_check/dump1.sql 
and 
/home/pgrunner/bf/root/HEAD/pgsql.3188/contrib/pg_upgrade/tmp_check/dump2.sql 
differ


See 
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=frogmouthdt=2011-11-28%2019%3A30%3A03 
for an example. For buildfarm purposes this is pretty low grade info, ISTM.


cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-11-27 Thread Peter Eisentraut
On lör, 2011-11-05 at 18:45 +0200, Peter Eisentraut wrote:
 On mån, 2011-09-19 at 07:06 +0300, Peter Eisentraut wrote:
  I found a simpler way to get this working.  Just hack up the catalogs
  for the new path directly.  So I can now run this test suite against
  older versions as well, like this:
  
  contrib/pg_upgrade$ make installcheck oldsrc=somewhere oldbindir=elsewhere
 
 Any comments on how to proceed with this?  I think it has been useful in
 detecting pg_upgrade breakage a few times already, so I'd like to commit
 it and start using it.

I've committed it now, and some buildfarm members are failing with lack
of shared memory, semaphores, or disk space.  Don't know what to do with
that or why so many are failing like that.  We could create a way to
omit the test if it becomes a problem.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-11-27 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 I've committed it now, and some buildfarm members are failing with lack
 of shared memory, semaphores, or disk space.  Don't know what to do with
 that or why so many are failing like that.  We could create a way to
 omit the test if it becomes a problem.

I believe the issue is that those BF members have kernel settings that
only support running one postmaster at a time.  The way you've got this
set up, it launches a new private postmaster during a make installcheck;
which is not only problematic from a resource consumption standpoint,
but seems to me to violate the spirit of make installcheck, because
what it's testing is not the installed postmaster but a local instance.

Can you confine the test to only occur in make check mode, not make
installcheck, please?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-11-27 Thread Andrew Dunstan



On 11/27/2011 06:17 PM, Tom Lane wrote:

Peter Eisentrautpete...@gmx.net  writes:

I've committed it now, and some buildfarm members are failing with lack
of shared memory, semaphores, or disk space.  Don't know what to do with
that or why so many are failing like that.  We could create a way to
omit the test if it becomes a problem.

I believe the issue is that those BF members have kernel settings that
only support running one postmaster at a time.  The way you've got this
set up, it launches a new private postmaster during a make installcheck;
which is not only problematic from a resource consumption standpoint,
but seems to me to violate the spirit of make installcheck, because
what it's testing is not the installed postmaster but a local instance.

Can you confine the test to only occur in make check mode, not make
installcheck, please?





Contrib tests are only run by the buildfarm in installcheck mode, so 
that will probably turn the tests off for the buildfarm, so +1 on that 
:-) I think these tests are probably somewhat ill-conceived. Note also 
that shell scripts are not portable, and so these tests would not be 
able to run on MSVC buildfarm members, even if they had been enabled in 
the MSVC regression driver, which they have not. If we need a regression 
driver script it needs to be written in Perl.


Also note that the test as written is likely to add significantly to 
buildfarm run times, as it will run the entire base regression suite 
again, possibly several times.


Finally, I think that this is probably not what we really need. I have 
already started work (as I mentioned some weeks ago) on having the 
buildfarm stash away a successful build and data directory, to be used 
later in cross-version upgrade testing, which seems to me much more what 
we need from something like the buildfarm. Maybe we could discuss how to 
run something like that.


And yes, some buildfarm members run on fairly scarce machine resources, 
of memory, CPU time and disk space, and we need not to increase what our 
tests use without due notice and care.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-11-10 Thread Bruce Momjian
Peter Eisentraut wrote:
 On m?n, 2011-09-19 at 07:06 +0300, Peter Eisentraut wrote:
  I found a simpler way to get this working.  Just hack up the catalogs
  for the new path directly.  So I can now run this test suite against
  older versions as well, like this:
  
  contrib/pg_upgrade$ make installcheck oldsrc=somewhere oldbindir=elsewhere
 
 Any comments on how to proceed with this?  I think it has been useful in
 detecting pg_upgrade breakage a few times already, so I'd like to commit
 it and start using it.

I don't have a problem with adding it.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-11-05 Thread Peter Eisentraut
On mån, 2011-09-19 at 07:06 +0300, Peter Eisentraut wrote:
 I found a simpler way to get this working.  Just hack up the catalogs
 for the new path directly.  So I can now run this test suite against
 older versions as well, like this:
 
 contrib/pg_upgrade$ make installcheck oldsrc=somewhere oldbindir=elsewhere

Any comments on how to proceed with this?  I think it has been useful in
detecting pg_upgrade breakage a few times already, so I'd like to commit
it and start using it.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-28 Thread Bruce Momjian
Bruce Momjian wrote:
 Tom Lane wrote:
  Bruce Momjian br...@momjian.us writes:
   I propose I just remove the 8.4
   test and always allow toast table names not to match --- the oids are
   still checked and are preserved.
  
  +1.  You'll still make the check for non-toast tables, of course?
 
 Yes, only toast tables will skip the check.  Proposed patch attached.

I was wrong. I can check for the version number because the toast file
name is made to match when pg_upgrade completes on the 9.0+ cluster. 
Updated patch attached that adds comments and checks for 9.0 instead of
8.4.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +
diff --git a/contrib/pg_upgrade/info.c b/contrib/pg_upgrade/info.c
new file mode 100644
index e41ab2b..b55bd6d
*** a/contrib/pg_upgrade/info.c
--- b/contrib/pg_upgrade/info.c
*** gen_db_file_maps(DbInfo *old_db, DbInfo 
*** 57,68 
     old_db-db_name, old_rel-reloid, new_rel-reloid);
  
  		/*
! 		 * In pre-8.4, TOAST table names change during CLUSTER;  in = 8.4
! 		 * TOAST relation names always use heap table oids, hence we cannot
! 		 * check relation names when upgrading from pre-8.4.
  		 */
  		if (strcmp(old_rel-nspname, new_rel-nspname) != 0 ||
! 			((GET_MAJOR_VERSION(old_cluster.major_version) = 804 ||
  			  strcmp(old_rel-nspname, pg_toast) != 0) 
  			 strcmp(old_rel-relname, new_rel-relname) != 0))
  			pg_log(PG_FATAL, Mismatch of relation names in database \%s\: 
--- 57,71 
     old_db-db_name, old_rel-reloid, new_rel-reloid);
  
  		/*
! 		 * TOAST table names initially match the heap pg_class oid.
! 		 * In pre-8.4, TOAST table names change during CLUSTER; in pre-9.0,
! 		 * TOAST table names change during ALTER TABLE ALTER COLUMN SET TYPE.
! 		 * In = 9.0, TOAST relation names always use heap table oids, hence
! 		 * we cannot check relation names when upgrading from pre-9.0.
! 		 * Clusters upgraded to 9.0 will get matching TOAST names.
  		 */
  		if (strcmp(old_rel-nspname, new_rel-nspname) != 0 ||
! 			((GET_MAJOR_VERSION(old_cluster.major_version) = 900 ||
  			  strcmp(old_rel-nspname, pg_toast) != 0) 
  			 strcmp(old_rel-relname, new_rel-relname) != 0))
  			pg_log(PG_FATAL, Mismatch of relation names in database \%s\: 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-28 Thread Bruce Momjian
Bruce Momjian wrote:
 Bruce Momjian wrote:
  Tom Lane wrote:
   Bruce Momjian br...@momjian.us writes:
I propose I just remove the 8.4
test and always allow toast table names not to match --- the oids are
still checked and are preserved.
   
   +1.  You'll still make the check for non-toast tables, of course?
  
  Yes, only toast tables will skip the check.  Proposed patch attached.
 
 I was wrong. I can check for the version number because the toast file
 name is made to match when pg_upgrade completes on the 9.0+ cluster. 
 Updated patch attached that adds comments and checks for 9.0 instead of
 8.4.

Applied to head and 9.1.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-27 Thread Bruce Momjian
Bruce Momjian wrote:
 Peter Eisentraut wrote:
  8.4 - master upgrade fails like this:
  
  Restoring user relation files
  Mismatch of relation names in database regression: old name 
  pg_toast.pg_toast_27437, new name pg_toast.pg_toast_27309
  Failure, exiting
  
  This has been 100% reproducible for me.
 
 I can now reproduce this failure and will research the cause, probably
 not before next week though.  :-( What is interesting is that loading
 the regression tests from an SQL dump does not show the failure, but
 running the regression tests and then upgrading does.

OK, I found time to research this and I think I have a fix.  The problem
is caused by an ALTER TABLE in 8.4 not preserving a toast table name
that matches the heap oid.  Below you can see that 8.4 does not preserve
this, while 9.0 does:

8.4
---
test= CREATE TABLE test5(aa TEXT, bb TEXT);
CREATE TABLE
test= INSERT INTO test5 VALUES ('123', '323');
INSERT 0 1
test= ALTER TABLE test5 ALTER COLUMN aa TYPE INTEGER USING 
bit_length(aa);
ALTER TABLE
test= SELECT oid, reltoastrelid FROM pg_class WHERE relname = 'test5';
  oid  | reltoastrelid
---+---
  ---   16406 | 16415
(1 row)

test= SELECT relname FROM pg_class WHERE oid = 16415;
relname

 pg_toast_16412  ---
(1 row)

9.0
---
test= CREATE TABLE test5(aa TEXT, bb TEXT);
CREATE TABLE
test= INSERT INTO test5 VALUES ('123', '323');
INSERT 0 1
test= ALTER TABLE test5 ALTER COLUMN aa TYPE INTEGER USING
bit_length(aa);
ALTER TABLE
test= SELECT oid, reltoastrelid FROM pg_class WHERE relname = 'test5';
  oid  | reltoastrelid
---+---
  ---   16409 | 16418
(1 row)

test= SELECT relname FROM pg_class WHERE oid = 16418;
relname

 pg_toast_16409  ---
(1 row)

We must have fixed this in 9.0 and I missed it.  Anyway, the pg_upgrade
code already assumes pre-8.4 doesn't have stable toast names:

/*
 * In pre-8.4, TOAST table names change during CLUSTER;  in = 8.4
 * TOAST relation names always use heap table oids, hence we cannot
 * check relation names when upgrading from pre-8.4.
 */
if (strcmp(old_rel-nspname, new_rel-nspname) != 0 ||
((GET_MAJOR_VERSION(old_cluster.major_version) = 804 ||
  strcmp(old_rel-nspname, pg_toast) != 0) 
 strcmp(old_rel-relname, new_rel-relname) != 0))
pg_log(PG_FATAL, Mismatch of relation names in database \%s\: 
   old name \%s.%s\, new name \%s.%s\\n,
   old_db-db_name, old_rel-nspname, old_rel-relname,
   new_rel-nspname, new_rel-relname);

Looking at this code now, I realize it is wrong even without the 8.4
ALTER issue.  If someone uses pg_upgrade to go from 8.3 to 8.4, they
would then still have the toast table name mismatch when going to 9.0,
so the test in itself is wrong anyway.  I propose I just remove the 8.4
test and always allow toast table names not to match --- the oids are
still checked and are preserved.

The current code is just too conservative and throws an error during
upgrade (but not during check mode) when it shouldn't.  This code only
exists in 9.1 and HEAD.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-27 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 I propose I just remove the 8.4
 test and always allow toast table names not to match --- the oids are
 still checked and are preserved.

+1.  You'll still make the check for non-toast tables, of course?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-27 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian br...@momjian.us writes:
  I propose I just remove the 8.4
  test and always allow toast table names not to match --- the oids are
  still checked and are preserved.
 
 +1.  You'll still make the check for non-toast tables, of course?

Yes, only toast tables will skip the check.  Proposed patch attached.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +
diff --git a/contrib/pg_upgrade/info.c b/contrib/pg_upgrade/info.c
new file mode 100644
index e41ab2b..7b1ab36
*** a/contrib/pg_upgrade/info.c
--- b/contrib/pg_upgrade/info.c
*** gen_db_file_maps(DbInfo *old_db, DbInfo 
*** 57,69 
     old_db-db_name, old_rel-reloid, new_rel-reloid);
  
  		/*
! 		 * In pre-8.4, TOAST table names change during CLUSTER;  in = 8.4
! 		 * TOAST relation names always use heap table oids, hence we cannot
! 		 * check relation names when upgrading from pre-8.4.
  		 */
  		if (strcmp(old_rel-nspname, new_rel-nspname) != 0 ||
! 			((GET_MAJOR_VERSION(old_cluster.major_version) = 804 ||
! 			  strcmp(old_rel-nspname, pg_toast) != 0) 
  			 strcmp(old_rel-relname, new_rel-relname) != 0))
  			pg_log(PG_FATAL, Mismatch of relation names in database \%s\: 
     old name \%s.%s\, new name \%s.%s\\n,
--- 57,73 
     old_db-db_name, old_rel-reloid, new_rel-reloid);
  
  		/*
! 		 * TOAST table names initially match the heap pg_class oid.
! 		 * However, in pre-8.4, TOAST table names change during CLUSTER, and
! 		 * in pre-9.0, TOAST table names change during ALTER TABLE.  Because
! 		 * an 8.3 or 8.4 system might be upgraded to 9.0 and then 9.1 (and
! 		 * still have a mismatch between toast table name and heap oid),
! 		 * we can't use the old cluster version to know if all toast
! 		 * table names match.  Hence we don't check a match of toast table
! 		 * names.
  		 */
  		if (strcmp(old_rel-nspname, new_rel-nspname) != 0 ||
! 			(strcmp(old_rel-nspname, pg_toast) != 0 
  			 strcmp(old_rel-relname, new_rel-relname) != 0))
  			pg_log(PG_FATAL, Mismatch of relation names in database \%s\: 
     old name \%s.%s\, new name \%s.%s\\n,

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-26 Thread Bruce Momjian
Peter Eisentraut wrote:
 8.4 - master upgrade fails like this:
 
 Restoring user relation files
 Mismatch of relation names in database regression: old name 
 pg_toast.pg_toast_27437, new name pg_toast.pg_toast_27309
 Failure, exiting
 
 This has been 100% reproducible for me.

I can now reproduce this failure and will research the cause, probably
not before next week though.  :-( What is interesting is that loading
the regression tests from an SQL dump does not show the failure, but
running the regression tests and then upgrading does.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-18 Thread Peter Eisentraut
On mån, 2011-09-05 at 23:42 +0300, Peter Eisentraut wrote:
 On lör, 2011-09-03 at 19:58 -0400, Tom Lane wrote:
  Anyway, after giving up on that I went back to plan A, namely install
  regress.so and friends into $libdir.  That turns out to be really quite
  straightforward, though I had to hack pg_regress.c a bit to get its idea
  of $libdir to match up exactly with the way the backend sees it.
  (The only reason this matters is that there's one error report in the
  regression tests where the full expansion of $libdir is reported.
  Maybe we should just drop that one test case instead of maintaining
  the infrastructure for replacing @libdir@ in pg_regress.c.)
  
  Attached is a draft patch for HEAD.  It passes make check and make
  installcheck on Unix, but I've not touched the MSVC scripts.
  Comments? 
 
 I'll try to integrate this with my pg_upgrade test runner to see if it
 gets the job done.

I found a simpler way to get this working.  Just hack up the catalogs
for the new path directly.  So I can now run this test suite against
older versions as well, like this:

contrib/pg_upgrade$ make installcheck oldsrc=somewhere oldbindir=elsewhere

The status is:

master - master works.

9.1 - master works.

9.0 - master kind of works.  The upgrade succeeds, but the dump has
differences because the languages are now dumped as extension commands.
It's easy to inspect manually, but won't work for any kind of automated
test runs.

8.4 - master upgrade fails like this:

Restoring user relation files
Mismatch of relation names in database regression: old name 
pg_toast.pg_toast_27437, new name pg_toast.pg_toast_27309
Failure, exiting

This has been 100% reproducible for me.

8.3 - master upgrade doesn't work at all, because the regression test
database contains columns of type name and pg_upgrade won't upgrade
those from this version.

diff --git a/contrib/pg_upgrade/.gitignore b/contrib/pg_upgrade/.gitignore
index 03ec737..79c8cdb 100644
--- a/contrib/pg_upgrade/.gitignore
+++ b/contrib/pg_upgrade/.gitignore
@@ -1 +1,5 @@
 /pg_upgrade
+# Generated by test suite
+delete_old_cluster.sh
+/log/
+/tmp_check/
diff --git a/contrib/pg_upgrade/Makefile b/contrib/pg_upgrade/Makefile
index 8f3fd7c..7f97b3d 100644
--- a/contrib/pg_upgrade/Makefile
+++ b/contrib/pg_upgrade/Makefile
@@ -21,3 +21,11 @@ top_builddir = ../..
 include $(top_builddir)/src/Makefile.global
 include $(top_srcdir)/contrib/contrib-global.mk
 endif
+
+check: test.sh
+	MAKE=$(MAKE) bindir=$(bindir) libdir=$(libdir) $(SHELL) $ --install
+
+installcheck: test.sh
+	MAKE=$(MAKE) bindir=$(bindir) libdir=$(libdir) $(SHELL) $
+
+EXTRA_CLEAN = delete_old_cluster.sh log/ tmp_check/
diff --git a/contrib/pg_upgrade/test.sh b/contrib/pg_upgrade/test.sh
new file mode 100644
index 000..9aebee9
--- /dev/null
+++ b/contrib/pg_upgrade/test.sh
@@ -0,0 +1,123 @@
+#!/bin/sh
+
+# contrib/pg_upgrade/test.sh
+#
+# Test driver for pg_upgrade.  Initializes a new database cluster,
+# runs the regression tests (to put in some data), runs pg_dumpall,
+# runs pg_upgrade, runs pg_dumpall again, compares the dumps.
+#
+# Portions Copyright (c) 1996-2011, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+
+set -e
+
+: ${MAKE=make}
+: ${PGPORT=50432}
+export PGPORT
+
+temp_root=$PWD/tmp_check
+
+if [ $1 = '--install' ]; then
+	temp_install=$temp_root/install
+	bindir=$temp_install/$bindir
+	libdir=$temp_install/$libdir
+
+	$MAKE -s -C ../.. install DESTDIR=$temp_install
+	$MAKE -s -C ../pg_upgrade_support install DESTDIR=$temp_install
+	$MAKE -s -C . install DESTDIR=$temp_install
+
+	# platform-specific magic to find the shared libraries; see pg_regress.c
+	LD_LIBRARY_PATH=$libdir:$LD_LIBRARY_PATH
+	export LD_LIBRARY_PATH
+	DYLD_LIBRARY_PATH=$libdir:$DYLD_LIBRARY_PATH
+	export DYLD_LIBRARY_PATH
+	LIBPATH=$libdir:$LIBPATH
+	export LIBPATH
+	PATH=$libdir:$PATH
+
+	# We need to make it use psql from our temporary installation,
+	# because otherwise the installcheck run below would try to
+	# use psql from the proper installation directory, which might
+	# be outdated or missing.
+	EXTRA_REGRESS_OPTS=--psqldir=$bindir
+	export EXTRA_REGRESS_OPTS
+fi
+
+: ${oldbindir=$bindir}
+
+: ${oldsrc=../..}
+oldsrc=`cd $oldsrc  pwd`
+newsrc=`cd ../..  pwd`
+
+PATH=$bindir:$PATH
+export PATH
+
+PGDATA=$temp_root/data
+export PGDATA
+rm -rf $PGDATA $PGDATA.old
+
+logdir=$PWD/log
+rm -rf $logdir
+mkdir $logdir
+
+set -x
+
+$oldbindir/initdb
+$oldbindir/pg_ctl start -l $logdir/postmaster1.log -w
+if $MAKE -C $oldsrc installcheck; then
+	pg_dumpall $temp_root/dump1.sql || pg_dumpall1_status=$?
+	if [ $newsrc != $oldsrc ]; then
+		oldpgversion=`psql -A -t -d regression -c SHOW server_version_num`
+		fix_sql=
+		case $oldpgversion in
+			804??)
+fix_sql=UPDATE pg_proc SET probin = replace(probin::text, '$oldsrc', '$newsrc')::bytea WHERE probin LIKE '$oldsrc%'; DROP FUNCTION public.myfunc(integer);
+;;
+			900??)
+fix_sql=SET 

Re: [HACKERS] pg_upgrade automatic testing

2011-09-05 Thread Peter Eisentraut
On lör, 2011-09-03 at 19:58 -0400, Tom Lane wrote:
 Anyway, after giving up on that I went back to plan A, namely install
 regress.so and friends into $libdir.  That turns out to be really quite
 straightforward, though I had to hack pg_regress.c a bit to get its idea
 of $libdir to match up exactly with the way the backend sees it.
 (The only reason this matters is that there's one error report in the
 regression tests where the full expansion of $libdir is reported.
 Maybe we should just drop that one test case instead of maintaining
 the infrastructure for replacing @libdir@ in pg_regress.c.)
 
 Attached is a draft patch for HEAD.  It passes make check and make
 installcheck on Unix, but I've not touched the MSVC scripts.
 Comments? 

I'll try to integrate this with my pg_upgrade test runner to see if it
gets the job done.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-05 Thread Andrew Dunstan



On 09/03/2011 07:58 PM, Tom Lane wrote:


Anyway, after giving up on that I went back to plan A, namely install
regress.so and friends into $libdir.  That turns out to be really quite
straightforward, though I had to hack pg_regress.c a bit to get its idea
of $libdir to match up exactly with the way the backend sees it.
(The only reason this matters is that there's one error report in the
regression tests where the full expansion of $libdir is reported.
Maybe we should just drop that one test case instead of maintaining
the infrastructure for replacing @libdir@ in pg_regress.c.)

Attached is a draft patch for HEAD.  It passes make check and make
installcheck on Unix, but I've not touched the MSVC scripts.
Comments?



This looks like it should work.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-03 Thread Peter Eisentraut
On fre, 2011-09-02 at 19:49 -0400, Tom Lane wrote:
 The only one that's problematic is pg_regress.so; contrib modules are
 already installed in $libdir.  I still think that installing
 pg_regress.so in $libdir may be the most reasonable solution, assuming
 that the delta involved isn't too great.  Yeah, we would have to
 back-patch the changes into every release branch we want to test
 upgrading from, but how risky is that really?  The *only* thing it
 affects is the regression tests.

Or maybe make use of dynamic_library_path.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-03 Thread Bruce Momjian
Peter Eisentraut wrote:
 But if you think about it, it doesn't really test pg_upgrade, it tests
 pg_dump.  So the test could just as well be moved to src/bin/pg_dump/
 and be labeled pg_dump smoke test or whatever.  (Minor detail: The bug
 fix above involved the --binary-upgrade flag, so it is somewhat
 pg_upgrade related.)
 
 A real pg_upgrade test suite should naturally upgrade across binary
 incompatible versions.  The real question is how you develop a useful
 test input.  Most pg_upgrade issues are not bugs of omission or
 regression but unexpected corner cases discovered with databases of
 nontrivial usage patterns.  (The recent one related to upgrade from 8.3
 is an exception.)  Because the basic premise of pg_upgrade is, dump and
 restore the schema, move over the files, that's it, and the rest of the
 code is workarounds for obscure details that are difficult to anticipate
 let alone test for.

You might want to read my blog entry on why pg_upgrade relies so much on
external tools:

http://momjian.us/main/blogs/pgblog/2011.html#June_15_2011_2

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-03 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On fre, 2011-09-02 at 19:49 -0400, Tom Lane wrote:
 The only one that's problematic is pg_regress.so; contrib modules are
 already installed in $libdir.  I still think that installing
 pg_regress.so in $libdir may be the most reasonable solution, assuming
 that the delta involved isn't too great.  Yeah, we would have to
 back-patch the changes into every release branch we want to test
 upgrading from, but how risky is that really?  The *only* thing it
 affects is the regression tests.

 Or maybe make use of dynamic_library_path.

That seemed like a promising idea at first, but on reflection I thought
probably it's not such a great idea.  The problem is, where do you
inject the setting?  In a temp installation we could put it in
postgresql.conf, but for make installcheck the only way that seems
like it would work at all is to apply it as a database configuration
(ALTER DATABASE SET), and that seems problematic.  In particular, it
would not work for testing pg_dump, since pg_dump doesn't copy those
settings.  (I know we've talked about making it do so, but we'd
certainly not wish to back-patch such a change.)

(BTW, this also strikes me as a counterexample for the recently
proposed change to make pg_dumpall dump such settings at the end.
If you've got datatypes or indexes that depend on a shared library
that's found via ALTER DATABASE SET dynamic_library_path, it will
absolutely not work to restore the database contents before that
setting is applied.)

Anyway, after giving up on that I went back to plan A, namely install
regress.so and friends into $libdir.  That turns out to be really quite
straightforward, though I had to hack pg_regress.c a bit to get its idea
of $libdir to match up exactly with the way the backend sees it.
(The only reason this matters is that there's one error report in the
regression tests where the full expansion of $libdir is reported.
Maybe we should just drop that one test case instead of maintaining
the infrastructure for replacing @libdir@ in pg_regress.c.)

Attached is a draft patch for HEAD.  It passes make check and make
installcheck on Unix, but I've not touched the MSVC scripts.
Comments?

regards, tom lane

diff --git a/contrib/dummy_seclabel/Makefile b/contrib/dummy_seclabel/Makefile
index 105400f..3a5f968 100644
*** a/contrib/dummy_seclabel/Makefile
--- b/contrib/dummy_seclabel/Makefile
*** top_builddir = ../..
*** 12,14 
--- 12,24 
  include $(top_builddir)/src/Makefile.global
  include $(top_srcdir)/contrib/contrib-global.mk
  endif
+ 
+ # Build/install only the files needed for regression test support
+ REGRESSION_MODULES = dummy_seclabel
+ 
+ .PHONY: regression-modules install-regression-modules
+ 
+ regression-modules: $(addsuffix $(DLSUFFIX), $(REGRESSION_MODULES))
+ 
+ install-regression-modules: $(addsuffix $(DLSUFFIX), $(REGRESSION_MODULES))
+ 	$(INSTALL_SHLIB) $(addsuffix $(DLSUFFIX), $(REGRESSION_MODULES)) '$(DESTDIR)$(pkglibdir)/'
diff --git a/contrib/spi/Makefile b/contrib/spi/Makefile
index 0c11bfc..83578fb 100644
*** a/contrib/spi/Makefile
--- b/contrib/spi/Makefile
*** top_builddir = ../..
*** 28,30 
--- 28,40 
  include $(top_builddir)/src/Makefile.global
  include $(top_srcdir)/contrib/contrib-global.mk
  endif
+ 
+ # Build/install only the files needed for regression test support
+ REGRESSION_MODULES = autoinc refint
+ 
+ .PHONY: regression-modules install-regression-modules
+ 
+ regression-modules: $(addsuffix $(DLSUFFIX), $(REGRESSION_MODULES))
+ 
+ install-regression-modules: $(addsuffix $(DLSUFFIX), $(REGRESSION_MODULES))
+ 	$(INSTALL_SHLIB) $(addsuffix $(DLSUFFIX), $(REGRESSION_MODULES)) '$(DESTDIR)$(pkglibdir)/'
diff --git a/src/test/regress/GNUmakefile b/src/test/regress/GNUmakefile
index 90aea6c..96839db 100644
*** a/src/test/regress/GNUmakefile
--- b/src/test/regress/GNUmakefile
*** EXTRADEFS = '-DHOST_TUPLE=$(host_tuple)
*** 41,47 
  
  # Build regression test driver
  
! all: pg_regress$(X)
  
  pg_regress$(X): pg_regress.o pg_regress_main.o | submake-libpgport
  	$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@
--- 41,47 
  
  # Build regression test driver
  
! all: pg_regress$(X) regression-modules
  
  pg_regress$(X): pg_regress.o pg_regress_main.o | submake-libpgport
  	$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@
*** pg_regress.o: pg_regress.c $(top_builddi
*** 53,65 
  $(top_builddir)/src/port/pg_config_paths.h: $(top_builddir)/src/Makefile.global
  	$(MAKE) -C $(top_builddir)/src/port pg_config_paths.h
  
! install: all installdirs
  	$(INSTALL_PROGRAM) pg_regress$(X) '$(DESTDIR)$(pgxsdir)/$(subdir)/pg_regress$(X)'
  
  installdirs:
  	$(MKDIR_P) '$(DESTDIR)$(pgxsdir)/$(subdir)'
  
! uninstall:
  	rm -f '$(DESTDIR)$(pgxsdir)/$(subdir)/pg_regress$(X)'
  
  
--- 53,65 
  $(top_builddir)/src/port/pg_config_paths.h: $(top_builddir)/src/Makefile.global
  	$(MAKE) -C 

Re: [HACKERS] pg_upgrade automatic testing

2011-09-02 Thread Peter Eisentraut
On tor, 2011-09-01 at 18:55 -0400, Bruce Momjian wrote:
 Tom Lane wrote:
  Peter Eisentraut pete...@gmx.net writes:
   +# contrib/pg_upgrade/test.sh
   +#
   +# Test driver for pg_upgrade.  Initializes a new database cluster,
   +# runs the regression tests (to put in some data), runs pg_dumpall,
   +# runs pg_upgrade, runs pg_dumpall again, compares the dumps.
  
  Hm .. my experience is that that doesn't work at all, because the
  regression tests set up assorted C functions whose implementations are
  in pg_regress.so, and it creates them with absolute path references
  to pg_regress.so.  When you try to load that into another installation
  that's a different version of PG, it quite properly fails.  So I think
  that as given, this script is only useful for testing pg_upgrade of
  $currentversion to $currentversion.  Which is surely better than no test
 
 Reminder --- you can't use pg_upgrade to go from the same catalog
 version to the same catalog version because the catalog version is
 embedded in the tablespace directory name.

Well, it does work, but only because the regression tests don't keep a
tablespace around at the end.  Would pg_upgrade complain otherwise?


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-02 Thread Andrew Dunstan



On 09/02/2011 10:36 AM, Peter Eisentraut wrote:

On tor, 2011-09-01 at 18:55 -0400, Bruce Momjian wrote:

Tom Lane wrote:

Peter Eisentrautpete...@gmx.net  writes:

+# contrib/pg_upgrade/test.sh
+#
+# Test driver for pg_upgrade.  Initializes a new database cluster,
+# runs the regression tests (to put in some data), runs pg_dumpall,
+# runs pg_upgrade, runs pg_dumpall again, compares the dumps.

Hm .. my experience is that that doesn't work at all, because the
regression tests set up assorted C functions whose implementations are
in pg_regress.so, and it creates them with absolute path references
to pg_regress.so.  When you try to load that into another installation
that's a different version of PG, it quite properly fails.  So I think
that as given, this script is only useful for testing pg_upgrade of
$currentversion to $currentversion.  Which is surely better than no test

Reminder --- you can't use pg_upgrade to go from the same catalog
version to the same catalog version because the catalog version is
embedded in the tablespace directory name.

Well, it does work, but only because the regression tests don't keep a
tablespace around at the end.  Would pg_upgrade complain otherwise?



In any case, it would be good to get rid of the limitation if possible. 
Then we could look at creating an automated test that we could use in 
the buildfarm.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-02 Thread Bruce Momjian
Peter Eisentraut wrote:
 On tor, 2011-09-01 at 18:55 -0400, Bruce Momjian wrote:
  Tom Lane wrote:
   Peter Eisentraut pete...@gmx.net writes:
+# contrib/pg_upgrade/test.sh
+#
+# Test driver for pg_upgrade.  Initializes a new database cluster,
+# runs the regression tests (to put in some data), runs pg_dumpall,
+# runs pg_upgrade, runs pg_dumpall again, compares the dumps.
   
   Hm .. my experience is that that doesn't work at all, because the
   regression tests set up assorted C functions whose implementations are
   in pg_regress.so, and it creates them with absolute path references
   to pg_regress.so.  When you try to load that into another installation
   that's a different version of PG, it quite properly fails.  So I think
   that as given, this script is only useful for testing pg_upgrade of
   $currentversion to $currentversion.  Which is surely better than no test
  
  Reminder --- you can't use pg_upgrade to go from the same catalog
  version to the same catalog version because the catalog version is
  embedded in the tablespace directory name.
 
 Well, it does work, but only because the regression tests don't keep a
 tablespace around at the end.  Would pg_upgrade complain otherwise?

The restriction is only for old clusters that contain tablespaces, and
you get this error message during the check phase:

Cannot migrate to/from the same system catalog version when
using tablespaces.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-02 Thread Bruce Momjian
Andrew Dunstan wrote:
 
 
 On 09/02/2011 10:36 AM, Peter Eisentraut wrote:
  On tor, 2011-09-01 at 18:55 -0400, Bruce Momjian wrote:
  Tom Lane wrote:
  Peter Eisentrautpete...@gmx.net  writes:
  +# contrib/pg_upgrade/test.sh
  +#
  +# Test driver for pg_upgrade.  Initializes a new database cluster,
  +# runs the regression tests (to put in some data), runs pg_dumpall,
  +# runs pg_upgrade, runs pg_dumpall again, compares the dumps.
  Hm .. my experience is that that doesn't work at all, because the
  regression tests set up assorted C functions whose implementations are
  in pg_regress.so, and it creates them with absolute path references
  to pg_regress.so.  When you try to load that into another installation
  that's a different version of PG, it quite properly fails.  So I think
  that as given, this script is only useful for testing pg_upgrade of
  $currentversion to $currentversion.  Which is surely better than no test
  Reminder --- you can't use pg_upgrade to go from the same catalog
  version to the same catalog version because the catalog version is
  embedded in the tablespace directory name.
  Well, it does work, but only because the regression tests don't keep a
  tablespace around at the end.  Would pg_upgrade complain otherwise?
 
 
 In any case, it would be good to get rid of the limitation if possible. 
 Then we could look at creating an automated test that we could use in 
 the buildfarm.

Well, the idea of using the catalog version was that it is something we
expect would change during any change in the system catalogs or internal
data format that would require the use of pg_upgrade.  I am unclear what
other fixed value we could use for this.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-02 Thread Andrew Dunstan



On 09/02/2011 01:55 PM, Bruce Momjian wrote:

Andrew Dunstan wrote:


On 09/02/2011 10:36 AM, Peter Eisentraut wrote:

On tor, 2011-09-01 at 18:55 -0400, Bruce Momjian wrote:

Tom Lane wrote:

Peter Eisentrautpete...@gmx.net   writes:

+# contrib/pg_upgrade/test.sh
+#
+# Test driver for pg_upgrade.  Initializes a new database cluster,
+# runs the regression tests (to put in some data), runs pg_dumpall,
+# runs pg_upgrade, runs pg_dumpall again, compares the dumps.

Hm .. my experience is that that doesn't work at all, because the
regression tests set up assorted C functions whose implementations are
in pg_regress.so, and it creates them with absolute path references
to pg_regress.so.  When you try to load that into another installation
that's a different version of PG, it quite properly fails.  So I think
that as given, this script is only useful for testing pg_upgrade of
$currentversion to $currentversion.  Which is surely better than no test

Reminder --- you can't use pg_upgrade to go from the same catalog
version to the same catalog version because the catalog version is
embedded in the tablespace directory name.

Well, it does work, but only because the regression tests don't keep a
tablespace around at the end.  Would pg_upgrade complain otherwise?


In any case, it would be good to get rid of the limitation if possible.
Then we could look at creating an automated test that we could use in
the buildfarm.

Well, the idea of using the catalog version was that it is something we
expect would change during any change in the system catalogs or internal
data format that would require the use of pg_upgrade.  I am unclear what
other fixed value we could use for this.


Why not use a prefix like 'd_' and 's_' so they can't be identical?

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-02 Thread Tom Lane
Bruce Momjian br...@momjian.us writes:
 Andrew Dunstan wrote:
 In any case, it would be good to get rid of the limitation if possible. 
 Then we could look at creating an automated test that we could use in 
 the buildfarm.

 Well, the idea of using the catalog version was that it is something we
 expect would change during any change in the system catalogs or internal
 data format that would require the use of pg_upgrade.  I am unclear what
 other fixed value we could use for this.

IMO there's next to no value in testing that scenario anyway, since
nobody would ever use it in the field.  What *would* be of value is
testing upgrades from previous release versions.  Probably that will
take some work in the buildfarm infrastructure as well as figuring out a
non-problematic test case to use, but that's the direction we need to
head in.

The other reasonable use-case for pg_upgrade is migrating a development
or beta-test installation across a catversion bump, but again the
tablespace directory name is not a restriction.  Perhaps we could have
a test that involves checking out the
commit-just-before-the-last-catversion-change and seeing if we can
migrate from that.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-02 Thread Andrew Dunstan



On 09/02/2011 03:04 PM, Tom Lane wrote:

Bruce Momjianbr...@momjian.us  writes:

Andrew Dunstan wrote:

In any case, it would be good to get rid of the limitation if possible.
Then we could look at creating an automated test that we could use in
the buildfarm.

Well, the idea of using the catalog version was that it is something we
expect would change during any change in the system catalogs or internal
data format that would require the use of pg_upgrade.  I am unclear what
other fixed value we could use for this.

IMO there's next to no value in testing that scenario anyway, since
nobody would ever use it in the field.  What *would* be of value is
testing upgrades from previous release versions.  Probably that will
take some work in the buildfarm infrastructure as well as figuring out a
non-problematic test case to use, but that's the direction we need to
head in.



I'm working on this right now.

Basically the idea is to stash away build and data dirs (after we've run 
regression, PL and contrib testing) for stable branches (via a command 
line option) and then test upgrading them. A trial run on the first part 
is currently running. Once I have that sorted out I'll work on the 
testing bit ;-)


cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-02 Thread Bruce Momjian
Andrew Dunstan wrote:
 
 
 On 09/02/2011 01:55 PM, Bruce Momjian wrote:
  Andrew Dunstan wrote:
 
  On 09/02/2011 10:36 AM, Peter Eisentraut wrote:
  On tor, 2011-09-01 at 18:55 -0400, Bruce Momjian wrote:
  Tom Lane wrote:
  Peter Eisentrautpete...@gmx.net   writes:
  +# contrib/pg_upgrade/test.sh
  +#
  +# Test driver for pg_upgrade.  Initializes a new database cluster,
  +# runs the regression tests (to put in some data), runs pg_dumpall,
  +# runs pg_upgrade, runs pg_dumpall again, compares the dumps.
  Hm .. my experience is that that doesn't work at all, because the
  regression tests set up assorted C functions whose implementations are
  in pg_regress.so, and it creates them with absolute path references
  to pg_regress.so.  When you try to load that into another installation
  that's a different version of PG, it quite properly fails.  So I think
  that as given, this script is only useful for testing pg_upgrade of
  $currentversion to $currentversion.  Which is surely better than no test
  Reminder --- you can't use pg_upgrade to go from the same catalog
  version to the same catalog version because the catalog version is
  embedded in the tablespace directory name.
  Well, it does work, but only because the regression tests don't keep a
  tablespace around at the end.  Would pg_upgrade complain otherwise?
 
  In any case, it would be good to get rid of the limitation if possible.
  Then we could look at creating an automated test that we could use in
  the buildfarm.
  Well, the idea of using the catalog version was that it is something we
  expect would change during any change in the system catalogs or internal
  data format that would require the use of pg_upgrade.  I am unclear what
  other fixed value we could use for this.
 
 Why not use a prefix like 'd_' and 's_' so they can't be identical?

What would 'd' and 's' mean?  Destination and Source?  That doesn't help
us because a destination today might be a source tomorrow and we don't
rename these tablespace directory names.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade automatic testing

2011-09-02 Thread