Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-03-02 Thread Peter Eisentraut
On tis, 2012-02-28 at 11:00 -0800, Daniel Farina wrote:
 I'd really like to support libraries (C or otherwise) of multiple
 versions at the same time, when the underlying library permits.

What's preventing you from doing that now?  You need to name all the
symbols differently, of course.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-03-02 Thread Daniel Farina
On Fri, Mar 2, 2012 at 10:37 AM, Peter Eisentraut pete...@gmx.net wrote:
 On tis, 2012-02-28 at 11:00 -0800, Daniel Farina wrote:
 I'd really like to support libraries (C or otherwise) of multiple
 versions at the same time, when the underlying library permits.

 What's preventing you from doing that now?  You need to name all the
 symbols differently, of course.

That's the problem: not really practical in a wider ecosystem of C
libraries, especially if the library produces multiple versions.  (Or,
not practical unless someone writes some credible
symbol-version-mangling-magic)

But is it unsurmountable? -- dlsym returns a function pointer, and one
would build up the operator table for the version of the extension at
hand, so one might have ltree version 1.01 and ltree version 2.3
fields in the same database.

-- 
fdr

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-03-02 Thread Robert Haas
On Fri, Mar 2, 2012 at 1:53 PM, Daniel Farina dan...@heroku.com wrote:
 On Fri, Mar 2, 2012 at 10:37 AM, Peter Eisentraut pete...@gmx.net wrote:
 On tis, 2012-02-28 at 11:00 -0800, Daniel Farina wrote:
 I'd really like to support libraries (C or otherwise) of multiple
 versions at the same time, when the underlying library permits.

 What's preventing you from doing that now?  You need to name all the
 symbols differently, of course.

 That's the problem: not really practical in a wider ecosystem of C
 libraries, especially if the library produces multiple versions.  (Or,
 not practical unless someone writes some credible
 symbol-version-mangling-magic)

 But is it unsurmountable? -- dlsym returns a function pointer, and one
 would build up the operator table for the version of the extension at
 hand, so one might have ltree version 1.01 and ltree version 2.3
 fields in the same database.

That might be possible, but it seems largely unrelated to the topic of
this thread, unless I am missing something.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-03-02 Thread Daniel Farina
On Fri, Mar 2, 2012 at 1:50 PM, Robert Haas robertmh...@gmail.com wrote:
 But is it unsurmountable? -- dlsym returns a function pointer, and one
 would build up the operator table for the version of the extension at
 hand, so one might have ltree version 1.01 and ltree version 2.3
 fields in the same database.

 That might be possible, but it seems largely unrelated to the topic of
 this thread, unless I am missing something.

It might be, and might not be, because if we start getting into where
to put shared objects for functionality linked to the database (rather
just letting the dynamic linker do whatever) it could crash head-on
into data field versioning and replica  issues. If someone thinks
about it for a few minutes and says nope, orthogonal, I'm without
objection, but I did want to raise this spectre to other minds because
it seems to be on the edge of that maelstrom.

I also look at the inability for pg_upgrade to help when certain
contribs are used between versions and can only say phew, I'm glad
that doesn't affect most people (a sad flip side: people aren't using
contribs and database extension as much as I like, but things are
moving towards more of that and not less)  because then the expected
duration of the upgrade process becomes a very ugly compatibility
matrix that nobody wants to see...somewhat dissatisfying for a service
provider.   Yet, formats can and will change requiring a lot of
gradual I/O, and a version byte is not always practical, so the best
option I can think of is to support multiple versions of formats,
operators and types at the same time, and then gradually rewrite
things.

I'm not intending to open discussion about any of that right now
(unless someone else is interested and wants to start a new thread),
but I wanted to provide insight as to to why I'm especially
preoccupied about this.

-- 
fdr

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-28 Thread Daniel Farina
On Sun, Feb 26, 2012 at 7:36 AM, Peter Eisentraut pete...@gmx.net wrote:
 On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote:
 Well, I'm trying to invoke the extension's make check target at
 extension build time. I do have a temporary installation I own
 somehwere in my $HOME, but that is still trying to find extensions in
 /usr/share/postgresql/9.1/extension/*.control, because I am using the
 system's postgresql version. The build process is not running as root,
 so I cannot do an install of the extension to its final location.
 Still it would be nice to run regression tests. All that seems to be
 missing is the ability to put

 extension_control_path = /home/buildd/tmp/extension

 into the postgresql.conf of the temporary PG installation, or some
 other way like CREATE EXTENSION foobar WITH CONTROL
 '/home/buildd/...'.

 Yeah, of course, the extension path is not related to the data
 directory.  So we do need some kind of path setting, just like
 dynamic_library_path.

A thought that may or may not be related:

I'd really like to support libraries (C or otherwise) of multiple
versions at the same time, when the underlying library permits.  The
critical use case is contribs that iterate on their storage format,
which right now is really painful.  While perhaps contribs should use
a version byte or something and be backwards-compatible forever, to
some extent that is clearly impossible and sometimes undesirable for
contribs that emit small on-disk representations, and would rather
use, say, the type system to disambiguate what operators fit with what
data.

I've been struggling to think of a more graceful way to load the
correct version of extra dependencies (including non-PG intrinsic
things like libgeos or libproj) when new database replicas are
created, as well as helping people with upgrade issues.  Is this
tunable possibly related to a solution for this?

-- 
fdr

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-27 Thread Sandro Santilli
On Fri, Feb 24, 2012 at 08:21:05PM +0200, Peter Eisentraut wrote:
 On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote:
  We don't initdb with PostGIS regression testing framework
  but I've considered doing it for this specific case and it stroke me
  that even then we couldn't control SHAREDIR.
 
 I would always create a new instance using initdb for test runs.  It's
 easy and avoids all these problems.

Doesn't avoid the SHAREDIR problem, that's what I'm saying.
SHAREDIR is a compile-time setting in PostgreSQL.

--strk;

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-27 Thread Sandro Santilli
On Sun, Feb 26, 2012 at 09:50:04PM -0500, Robert Haas wrote:
 On Sun, Feb 26, 2012 at 10:36 AM, Peter Eisentraut pete...@gmx.net wrote:
  On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote:
  Well, I'm trying to invoke the extension's make check target at
  extension build time. I do have a temporary installation I own
  somehwere in my $HOME, but that is still trying to find extensions in
  /usr/share/postgresql/9.1/extension/*.control, because I am using the
  system's postgresql version. The build process is not running as root,
  so I cannot do an install of the extension to its final location.
  Still it would be nice to run regression tests. All that seems to be
  missing is the ability to put
 
  extension_control_path = /home/buildd/tmp/extension
 
  into the postgresql.conf of the temporary PG installation, or some
  other way like CREATE EXTENSION foobar WITH CONTROL
  '/home/buildd/...'.
 
  Yeah, of course, the extension path is not related to the data
  directory.  So we do need some kind of path setting, just like
  dynamic_library_path.
 
 That logic seems sound to me, so +1.

+1 here as well. I may attempt to produce a patch if this gets consensus.

--strk; 

  ,--o-. 
  |   __/  |Delivering high quality PostGIS 2.0 !
  |  / 2.0 |http://strk.keybit.net - http://vizzuality.com
  `-o--'


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-26 Thread Peter Eisentraut
On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote:
 Well, I'm trying to invoke the extension's make check target at
 extension build time. I do have a temporary installation I own
 somehwere in my $HOME, but that is still trying to find extensions in
 /usr/share/postgresql/9.1/extension/*.control, because I am using the
 system's postgresql version. The build process is not running as root,
 so I cannot do an install of the extension to its final location.
 Still it would be nice to run regression tests. All that seems to be
 missing is the ability to put
 
 extension_control_path = /home/buildd/tmp/extension
 
 into the postgresql.conf of the temporary PG installation, or some
 other way like CREATE EXTENSION foobar WITH CONTROL
 '/home/buildd/...'. 

Yeah, of course, the extension path is not related to the data
directory.  So we do need some kind of path setting, just like
dynamic_library_path.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-26 Thread Robert Haas
On Sun, Feb 26, 2012 at 10:36 AM, Peter Eisentraut pete...@gmx.net wrote:
 On lör, 2012-02-25 at 14:21 +0100, Christoph Berg wrote:
 Well, I'm trying to invoke the extension's make check target at
 extension build time. I do have a temporary installation I own
 somehwere in my $HOME, but that is still trying to find extensions in
 /usr/share/postgresql/9.1/extension/*.control, because I am using the
 system's postgresql version. The build process is not running as root,
 so I cannot do an install of the extension to its final location.
 Still it would be nice to run regression tests. All that seems to be
 missing is the ability to put

 extension_control_path = /home/buildd/tmp/extension

 into the postgresql.conf of the temporary PG installation, or some
 other way like CREATE EXTENSION foobar WITH CONTROL
 '/home/buildd/...'.

 Yeah, of course, the extension path is not related to the data
 directory.  So we do need some kind of path setting, just like
 dynamic_library_path.

That logic seems sound to me, so +1.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-25 Thread Christoph Berg
Re: Peter Eisentraut 2012-02-24 1330107599.32452.15.ca...@vanquo.pezone.net
 On fre, 2012-02-24 at 11:53 -0500, Tom Lane wrote:
   We have the same problem with testing extensions at build-time in
  the
   Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
   only writable by root, while the build is running as buildd user, so
   there is no way to do create extension whatimbuildingrightnow to
  be
   able to run regression tests, even if this is a cluster I have just
   created with initdb.
  
  This seems like nonsense.  If the build process has installed the
  software, you surely have got permissions to write in that directory.
 
 The build process just installs the software in a fake root where it
 will be wrapped up by the packaging software.
 
  If you haven't installed the software, you need to do testing in a
  temporary installation per make check, and we are able to test
  extensions that way too.
  
 That looks like the right answer.

Well, I'm trying to invoke the extension's make check target at
extension build time. I do have a temporary installation I own
somehwere in my $HOME, but that is still trying to find extensions in
/usr/share/postgresql/9.1/extension/*.control, because I am using the
system's postgresql version. The build process is not running as root,
so I cannot do an install of the extension to its final location.
Still it would be nice to run regression tests. All that seems to be
missing is the ability to put

extension_control_path = /home/buildd/tmp/extension

into the postgresql.conf of the temporary PG installation, or some
other way like CREATE EXTENSION foobar WITH CONTROL '/home/buildd/...'.

Christoph
-- 
c...@df7cb.de | http://www.df7cb.de/


signature.asc
Description: Digital signature


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Christoph Berg
Re: Sandro Santilli 2012-02-22 20120222101656.GB6125@gnash
 I'm not really looking for inline extensions.
 I do want to install the extension objects somewhere, just NOT
 in the PostgreSQL builtin SHAREDIR but in an arbitrary staging
 directory to use for QA the extension before distribution.

We have the same problem with testing extensions at build-time in the
Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
only writable by root, while the build is running as buildd user, so
there is no way to do create extension whatimbuildingrightnow to be
able to run regression tests, even if this is a cluster I have just
created with initdb.

Christoph
-- 
c...@df7cb.de | http://www.df7cb.de/


signature.asc
Description: Digital signature


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Dimitri Fontaine
Daniel Farina dan...@heroku.com writes:
 I'm pretty sure Sandro is hacking PostGIS, so inline extensions are of
 no help here.

Sorry about the sloppy mention, one of the things under discussion (but
out of scope for 9.2) in the inline extension thread is about where to
store the shared objects and extension files so that we have a real
per-database objects here, rather than per major version deployments
that happen to be able to impact several clusters and databases in
different and subtle ways.

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Sandro Santilli
On Fri, Feb 24, 2012 at 09:59:12AM +0100, Christoph Berg wrote:
 Re: Sandro Santilli 2012-02-22 20120222101656.GB6125@gnash
  I'm not really looking for inline extensions.
  I do want to install the extension objects somewhere, just NOT
  in the PostgreSQL builtin SHAREDIR but in an arbitrary staging
  directory to use for QA the extension before distribution.
 
 We have the same problem with testing extensions at build-time in the
 Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
 only writable by root, while the build is running as buildd user, so
 there is no way to do create extension whatimbuildingrightnow to be
 able to run regression tests, even if this is a cluster I have just
 created with initdb.

Exactly. We don't initdb with PostGIS regression testing framework
but I've considered doing it for this specific case and it stroke me
that even then we couldn't control SHAREDIR.

--strk;

  ,--o-. 
  |   __/  |Delivering high quality PostGIS 2.0 !
  |  / 2.0 |http://strk.keybit.net - http://vizzuality.com
  `-o--'


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Sandro Santilli
On Thu, Feb 23, 2012 at 06:53:05PM -0800, Daniel Farina wrote:
 On Tue, Feb 21, 2012 at 1:34 PM, Dimitri Fontaine
 dimi...@2ndquadrant.fr wrote:
  Sandro Santilli s...@keybit.net writes:
  Please see the inline extension thread where answers to your problem
  have been discussed.
 
 I'm pretty sure Sandro is hacking PostGIS, so inline extensions are of
 no help here.
 
 Can you tell us why alternative approaches that don't require adding a
 knob (such as copying/symlinking of compiled artifacts) is such a huge
 pain for you?

It's not specifically a pain for me, but it breaks the
tests can be run in a staged environment idiom.

See Christoph Berg mail, that's exactly the same issue.

--strk; 

  ,--o-. 
  |   __/  |Delivering high quality PostGIS 2.0 !
  |  / 2.0 |http://strk.keybit.net - http://vizzuality.com
  `-o--'


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Tom Lane
Christoph Berg c...@df7cb.de writes:
 Re: Sandro Santilli 2012-02-22 20120222101656.GB6125@gnash
 I'm not really looking for inline extensions.
 I do want to install the extension objects somewhere, just NOT
 in the PostgreSQL builtin SHAREDIR but in an arbitrary staging
 directory to use for QA the extension before distribution.

 We have the same problem with testing extensions at build-time in the
 Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
 only writable by root, while the build is running as buildd user, so
 there is no way to do create extension whatimbuildingrightnow to be
 able to run regression tests, even if this is a cluster I have just
 created with initdb.

This seems like nonsense.  If the build process has installed the
software, you surely have got permissions to write in that directory.
If you haven't installed the software, you need to do testing in a
temporary installation per make check, and we are able to test
extensions that way too.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Peter Eisentraut
On fre, 2012-02-24 at 11:53 -0500, Tom Lane wrote:
  We have the same problem with testing extensions at build-time in
 the
  Debian packages. The server's SHAREDIR /usr/share/postgresql/... is
  only writable by root, while the build is running as buildd user, so
  there is no way to do create extension whatimbuildingrightnow to
 be
  able to run regression tests, even if this is a cluster I have just
  created with initdb.
 
 This seems like nonsense.  If the build process has installed the
 software, you surely have got permissions to write in that directory.

The build process just installs the software in a fake root where it
will be wrapped up by the packaging software.

 If you haven't installed the software, you need to do testing in a
 temporary installation per make check, and we are able to test
 extensions that way too.
 
That looks like the right answer.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Peter Eisentraut
On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote:
 We don't initdb with PostGIS regression testing framework
 but I've considered doing it for this specific case and it stroke me
 that even then we couldn't control SHAREDIR.

I would always create a new instance using initdb for test runs.  It's
easy and avoids all these problems.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 10:21 AM, Peter Eisentraut pete...@gmx.net wrote:
 On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote:
 We don't initdb with PostGIS regression testing framework
 but I've considered doing it for this specific case and it stroke me
 that even then we couldn't control SHAREDIR.

 I would always create a new instance using initdb for test runs.  It's
 easy and avoids all these problems.

Having been in this position once before in a different but similar
situation, there's one big caveat: initdb is *really* slow, so it is
really painful for people who write Postgres-linked code that is
compiled separately, whereby the initdb binary is stable.  The
frustration is not unlike when I have to run ./configure, except I had
to do it all the time, every time.

A ccache-like solution has worked well for at least one person I know,
and wasn't hard to implement(?) for personal use.

But by hook or crook, a fresh database is the right thing for sure.

-- 
fdr

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Andrew Dunstan



On 02/24/2012 06:31 PM, Daniel Farina wrote:

On Fri, Feb 24, 2012 at 10:21 AM, Peter Eisentrautpete...@gmx.net  wrote:

On fre, 2012-02-24 at 17:26 +0100, Sandro Santilli wrote:

We don't initdb with PostGIS regression testing framework
but I've considered doing it for this specific case and it stroke me
that even then we couldn't control SHAREDIR.

I would always create a new instance using initdb for test runs.  It's
easy and avoids all these problems.

Having been in this position once before in a different but similar
situation, there's one big caveat: initdb is *really* slow, so it is
really painful for people who write Postgres-linked code that is
compiled separately, whereby the initdb binary is stable.  The
frustration is not unlike when I have to run ./configure, except I had
to do it all the time, every time.



Really? Here's what I just got on a severely under-resourced SL6 VM:

   [andrew@xena ~]$ time /usr/pgsql-9.1/bin/initdb -A ident foo  /dev/null

   real0m1.441s
   user0m1.131s
   sys0m0.309s

1.5s doesn't seem terribly slow.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Daniel Farina
On Fri, Feb 24, 2012 at 6:31 PM, Andrew Dunstan and...@dunslane.net wrote:
 Really? Here's what I just got on a severely under-resourced SL6 VM:

 1.5s doesn't seem terribly slow.

You are right. Come to think of it, I do seem to recall that initdb
got some speed improvements; these were in 8.3 days, I think.

Here are my numbers for comparison, in some various forms:

env time /usr/lib/postgresql/9.1/bin/initdb  /tmp/foo
0.94user 0.28system 0:01.87elapsed 64%CPU (0avgtext+0avgdata 79920maxresident)k
14288inputs+85144outputs (36major+66622minor)pagefaults 0swaps

env time eatmydata /usr/lib/postgresql/9.1/bin/initdb  /tmp/foo
0.90user 0.27system 0:01.26elapsed 92%CPU (0avgtext+0avgdata 79984maxresident)k
856inputs+85144outputs (0major+67202minor)pagefaults 0swaps

env time /usr/lib/postgresql/9.1/bin/initdb  /dev/shm/blah
0.90user 0.19system 0:01.15elapsed 95%CPU (0avgtext+0avgdata 79984maxresident)k
0inputs+0outputs (0major+67192minor)pagefaults 0swaps

Retracted.

-- 
fdr

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-24 Thread Maciek Sakrejda
On Fri, Feb 24, 2012 at 3:31 PM, Daniel Farina dan...@heroku.com wrote:
 Having been in this position once before in a different but similar
 situation, there's one big caveat: initdb is *really* slow, so it is
 really painful for people who write Postgres-linked code that is
 compiled separately, whereby the initdb binary is stable.  The
 frustration is not unlike when I have to run ./configure, except I had
 to do it all the time, every time.

 A ccache-like solution has worked well for at least one person I know,
 and wasn't hard to implement(?) for personal use.

Being the person in question: I don't have the full context so I'm not
sure how useful this will be, but here's hoping it's not just line
noise. I'm probably not at liberty to share the original code, but it
was a fairly trivial shell script that did more or less the following:

1. Capture all arguments (to pass to initdb itself) except the path,
and figure out the path to the cluster (via looking for
-D/--pgdata/$PGDATA)
2. Create a hash of the concatenation of the arguments (again, sans
cluster path)
3. Create a template path based on a local cache directory and the
argument hash
4. a) If the directory with this path did not exist, run initdb with
-D set to this and the rest of the arguments as captured; then copy to
*actual* path
b) If it does exist, that's because you've run initdb with these
arguments before: just copy that to the cluster path you captured in
step (1) instead of starting fresh

This was not terribly robust (e.g., I remember that Dan had pointed
out several shortcomings), but none of these were practical issues for
me, and I never had a problem. There  is some more work around
capturing the cluster path in the initdb output and mangling it back
from the template version into the original (so the output of the
cached initdb looks like output of a real initdb), but that's not a
big deal. The approach could probably be made solid enough for
general-purpose use.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-23 Thread Daniel Farina
On Tue, Feb 21, 2012 at 1:34 PM, Dimitri Fontaine
dimi...@2ndquadrant.fr wrote:
 Sandro Santilli s...@keybit.net writes:
 Please see the inline extension thread where answers to your problem
 have been discussed.

I'm pretty sure Sandro is hacking PostGIS, so inline extensions are of
no help here.

Can you tell us why alternative approaches that don't require adding a
knob (such as copying/symlinking of compiled artifacts) is such a huge
pain for you?

--
fdr

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-22 Thread Sandro Santilli
On Tue, Feb 21, 2012 at 10:34:42PM +0100, Dimitri Fontaine wrote:
 Sandro Santilli s...@keybit.net writes:
  On Tue, Feb 21, 2012 at 10:21:17AM -0500, Tom Lane wrote:
  Sandro Santilli s...@keybit.net writes:
   I'm trying to understand what options I have to test CREATE EXTENSION
   w/out installing the extension files in their final destination.
  
  There aren't any.  Generally speaking, if you want to be testing an
  extension, you should be doing it in a test installation anyway;
  so I don't see what's the objection to sticking the files into the
  installation's real SHAREDIR.
 
  We do a staged installation of the extension library and scripts
  in an ad-hoc dir which does not match the built-in SHAREDIR of PostgreSQL.
 
  It works fine for testing the extension directly sourcing the scripts.
  It's only CREATE EXTENSION that doesn't play nicely with that setup.
 
 Please see the inline extension thread where answers to your problem
 have been discussed.
 
   https://commitfest.postgresql.org/action/patch_view?id=746

I'm not really looking for inline extensions.
I do want to install the extension objects somewhere, just NOT
in the PostgreSQL builtin SHAREDIR but in an arbitrary staging
directory to use for QA the extension before distribution.

--strk; 

  ,--o-. 
  |   __/  |Delivering high quality PostGIS 2.0 !
  |  / 2.0 |http://strk.keybit.net
  `-o--'


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-21 Thread Tom Lane
Sandro Santilli s...@keybit.net writes:
 I'm trying to understand what options I have to test CREATE EXTENSION
 w/out installing the extension files in their final destination.

There aren't any.  Generally speaking, if you want to be testing an
extension, you should be doing it in a test installation anyway;
so I don't see what's the objection to sticking the files into the
installation's real SHAREDIR.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-21 Thread Sandro Santilli
On Tue, Feb 21, 2012 at 10:21:17AM -0500, Tom Lane wrote:
 Sandro Santilli s...@keybit.net writes:
  I'm trying to understand what options I have to test CREATE EXTENSION
  w/out installing the extension files in their final destination.
 
 There aren't any.  Generally speaking, if you want to be testing an
 extension, you should be doing it in a test installation anyway;
 so I don't see what's the objection to sticking the files into the
 installation's real SHAREDIR.

We do a staged installation of the extension library and scripts
in an ad-hoc dir which does not match the built-in SHAREDIR of PostgreSQL.

It works fine for testing the extension directly sourcing the scripts.
It's only CREATE EXTENSION that doesn't play nicely with that setup.

--strk; 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Runtime SHAREDIR for testing CREATE EXTENSION

2012-02-21 Thread Dimitri Fontaine
Sandro Santilli s...@keybit.net writes:
 On Tue, Feb 21, 2012 at 10:21:17AM -0500, Tom Lane wrote:
 Sandro Santilli s...@keybit.net writes:
  I'm trying to understand what options I have to test CREATE EXTENSION
  w/out installing the extension files in their final destination.
 
 There aren't any.  Generally speaking, if you want to be testing an
 extension, you should be doing it in a test installation anyway;
 so I don't see what's the objection to sticking the files into the
 installation's real SHAREDIR.

 We do a staged installation of the extension library and scripts
 in an ad-hoc dir which does not match the built-in SHAREDIR of PostgreSQL.

 It works fine for testing the extension directly sourcing the scripts.
 It's only CREATE EXTENSION that doesn't play nicely with that setup.

Please see the inline extension thread where answers to your problem
have been discussed.

  https://commitfest.postgresql.org/action/patch_view?id=746

Regards,
-- 
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers