Re: [HACKERS] Patch: Perl xsubpp

2011-11-27 Thread Mr. Aaron W. Swenson
On Sat, Nov 26, 2011 at 03:28:57PM -0500, Andrew Dunstan wrote:
 On 10/12/2011 08:55 PM, Alex Hunsaker wrote:
  On Wed, Oct 12, 2011 at 17:53, David E.
  Wheelerda...@kineticode.com wrote:
  On Sep 15, 2011, at 3:04 PM, Alex Hunsaker wrote:
 
  Close, seems I was wrong about the typemap ExtUtils::ParseXS does not
  install a new one so we still need to point to the one in privlib.
  Also xsubpp is not executable so the test should be -r or something.
 
  Also don't think we should change the configure switch tests to
  test XSUBPPDIR.
 
  Find those plus some minor typos fixed in the attached.
  xsubpp_v3.patch
  --
  Doesn't look like this has been applied yet. I think it ought to
  be backported, too, frankly. DId I miss it?
  Nah, probably should add it to the next commit fest so it does not get
  forgotten.
 
 
 committed.
 
 cheers
 
 andrew

Has this been backpatched as well?

-- 
Mr. Aaron W. Swenson
Gentoo Linux Developer
Email: titanof...@gentoo.org
GnuPG FP : 2C00 7719 4F85 FB07 A49C  0E31 5713 AA03 D1BB FDA0
GnuPG ID : D1BBFDA0


pgpOvprJOI60g.pgp
Description: PGP signature


Re: [HACKERS] Patch: Perl xsubpp

2011-11-27 Thread Mr. Aaron W. Swenson
On Sun, Nov 27, 2011 at 12:12:41PM -0800, David E. Wheeler wrote:
 On Nov 27, 2011, at 6:11 AM, Andrew Dunstan wrote:
 
  Has this been backpatched as well?
  
  It has been to 9.1.
 
 There may be a simple workaround, but it's non-obvious. I think it should be 
 back-patched all the way.
 
 Best,
 
 David

That's my vote, too. It's preventing users of all versions from compiling
against ExtUtils-ParseXS-3.20.0.

-- 
Mr. Aaron W. Swenson
Gentoo Linux
Developer, Proxy Committer
Email: titanof...@gentoo.org
GnuPG FP : 2C00 7719 4F85 FB07 A49C  0E31 5713 AA03 D1BB FDA0
GnuPG ID : D1BBFDA0


pgpyHoCvdur9C.pgp
Description: PGP signature


Re: [HACKERS] Add socket dir to pg_config..?

2011-10-29 Thread Mr. Aaron W. Swenson
On Fri, Oct 28, 2011 at 06:33:39PM +0200, Dimitri Fontaine wrote:
 Andrew Dunstan and...@dunslane.net writes:
  Er, which distros other than debian/ubuntu?
 
 Well, any and all derivatives I guess, to begin with.
 
   http://distrowatch.com/dwres.php?resource=independence#debian
   Based on Debian GNU/Linux: 129 Distributions
 
 More seriously, I'm not sure how to understand why some people will both
 frown upon distribution allowing themselves to patch the version of
 PostgreSQL they are packaging, and vote against making their life
 easier.
 
 If /tmp is the only decent place where to put the socket file on Unix
 when security and other concerns are considered, then sure, making
 distro life difficult is a good thing to do. But then let's take it to
 the FHS that debian and ubuntu are implementing, AFAIUI.
 
 I'm puzzled, maybe I'm not understanding a key point here though.
 
 Regards,
 -- 
 Dimitri Fontaine
 http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

In Gentoo, we change the socket directory to /var/run/postgresql via
pg_config_manual.h. However, I'm not too terribly interested in pg_config
outputting the directory location.

We inform users at the end of every install where the default location
is. Further, all of the packages we maintain build against the sources so
the packages automatically know where the socket directory is located.

-- 
Mr. Aaron W. Swenson
Gentoo Linux Developer
Email: titanof...@gentoo.org
GnuPG FP : 2C00 7719 4F85 FB07 A49C  0E31 5713 AA03 D1BB FDA0
GnuPG ID : D1BBFDA0


pgph6Z3VCbZWh.pgp
Description: PGP signature


Re: [HACKERS] Bug with pg_ctl -w/wait and config-only directories

2011-10-05 Thread Mr. Aaron W. Swenson
On Wed, Oct 05, 2011 at 10:44:38AM -0400, Bruce Momjian wrote:
 Peter Eisentraut wrote:
  On m?n, 2011-10-03 at 15:09 -0400, Bruce Momjian wrote:
   Why were people not using pg_ctl?
  
  Actually, a slight correction/addition here: The Debian init script does
  use pg_ctl to start the service.  Seems to work fine.
 
 Yes.  The script authors discovered a working behavior, which matches my
 research:
 
 pg_ctl startspecify config directory
 pg_ctl -w start impossible
 ...


Maybe I'm misunderstanding what you've written, but for 'pg_ctl -w' it is
possible. The following command does work (I've replaced the variables
with their default values to make it easier to read):

  su -l postgres \
-c env PGPORT=\5432\ /usr/lib/postgresql-9.1/bin/pg_ctl start -w \
-t 60 -s -D /var/lib/postgresql/9.1/data/ \
 -o '-D /etc/postgresql-9.1/ \
--data-directory=/var/lib/postgresql/9.1/data/ \
--silent-mode=true'

-- 
Mr. Aaron W. Swenson
Gentoo Linux Developer
Email: titanof...@gentoo.org
GnuPG FP : 2C00 7719 4F85 FB07 A49C  0E31 5713 AA03 D1BB FDA0
GnuPG ID : D1BBFDA0


pgpW11PKX3kCL.pgp
Description: PGP signature


Re: [HACKERS] Bug with pg_ctl -w/wait and config-only directories

2011-10-05 Thread Mr. Aaron W. Swenson
On Wed, Oct 05, 2011 at 07:20:16PM -0400, Bruce Momjian wrote:
 Mr. Aaron W. Swenson wrote:
 -- Start of PGP signed section.
  On Wed, Oct 05, 2011 at 10:44:38AM -0400, Bruce Momjian wrote:
   Peter Eisentraut wrote:
On m?n, 2011-10-03 at 15:09 -0400, Bruce Momjian wrote:
 Why were people not using pg_ctl?

Actually, a slight correction/addition here: The Debian init script does
use pg_ctl to start the service.  Seems to work fine.
   
   Yes.  The script authors discovered a working behavior, which matches my
   research:
   
   pg_ctl startspecify config directory
   pg_ctl -w start impossible
   ...
  
  
  Maybe I'm misunderstanding what you've written, but for 'pg_ctl -w' it is
  possible. The following command does work (I've replaced the variables
  with their default values to make it easier to read):
  
su -l postgres \
  -c env PGPORT=\5432\ /usr/lib/postgresql-9.1/bin/pg_ctl start -w \
  -t 60 -s -D /var/lib/postgresql/9.1/data/ \
   -o '-D /etc/postgresql-9.1/ \
  --data-directory=/var/lib/postgresql/9.1/data/ \
  --silent-mode=true'
 
 Wow.  So you are telling pg_ctl to use the real data directory, but are
 passing flags to the postmaster to start with the config directory and
 also setting data-directory to the real data directory.  I am impressed.
 
 I can see how that would work.  Did you look at the C code to figure
 this out or did you just test until it worked, or did it just seem
 logical to you?

No, I didn't look into the C code. (Though, I have poked around in there a
bit.)

Being a Gentoo user has taught me a few things like one should always
'read the friendly manual.' It just seemed logical after reading the
pg_ctl documentation that passing options to the postmaster would do the
trick, which spurred my testing several variations until it worked. But, I
only knew that I could give 'postgres'/'postmaster' the configuration
directory and it would do the right thing.
 
 With the patch I am going to commit, you will not need to use one of the
 -D flags because pg_ctl will find the data directory location;  you will
 just specify the config-only directory with one -D, and the
 --data-directory.

So, you're saying that my passing the config directory to pg_ctl through
'-D' will eliminate the need -- as far as config and data directories are
concerned -- to pass options directly to the postmaster?

-- 
Mr. Aaron W. Swenson
Gentoo Linux Developer
Email: titanof...@gentoo.org
GnuPG FP : 2C00 7719 4F85 FB07 A49C  0E31 5713 AA03 D1BB FDA0
GnuPG ID : D1BBFDA0


pgpZN3TMQo6L7.pgp
Description: PGP signature


Re: [HACKERS] Bug with pg_ctl -w/wait and config-only directories

2011-10-05 Thread Mr. Aaron W. Swenson
On Wed, Oct 05, 2011 at 07:59:07PM -0400, Bruce Momjian wrote:
 Mr. Aaron W. Swenson wrote:
   With the patch I am going to commit, you will not need to use one of the
   -D flags because pg_ctl will find the data directory location;  you will
   just specify the config-only directory with one -D, and the
   --data-directory.
  
  So, you're saying that my passing the config directory to pg_ctl through
  '-D' will eliminate the need -- as far as config and data directories are
  concerned -- to pass options directly to the postmaster?
 
 Yes, in PG 9.2.

\o/

-- 
Mr. Aaron W. Swenson
Gentoo Linux Developer
Email: titanof...@gentoo.org
GnuPG FP : 2C00 7719 4F85 FB07 A49C  0E31 5713 AA03 D1BB FDA0
GnuPG ID : D1BBFDA0


pgpedIo5z4j4c.pgp
Description: PGP signature


Re: [HACKERS] Bug with pg_ctl -w/wait and config-only directories

2011-10-04 Thread Mr. Aaron W. Swenson
On Tue, Oct 04, 2011 at 09:42:42AM -0400, Bruce Momjian wrote:
 Peter Eisentraut wrote:
  On m?n, 2011-10-03 at 18:44 -0400, Bruce Momjian wrote:
   Agreed.  You could argue that pg_ctl 9.1 is much better than anything
   anyone would be able to craft in a script.
  
  And what facts support that argument?
 
 Because pg_ctl 9.1 will read postmaster.pid and find the port number,
 socket location, and listen host for wait mode --- I doubt someone would
 do that work in a script.

I don't know why we chose pg_ctl, but I chose to stick with pg_ctl over
rolling my own precisely because of that. pg_ctl does all sorts of fancy
things that would have taken me a much longer time than figuring out a way
to allow a configuration-only directory and a data-only directory.

-- 
Mr. Aaron W. Swenson
Gentoo Linux Developer
Email: titanof...@gentoo.org
GnuPG FP : 2C00 7719 4F85 FB07 A49C  0E31 5713 AA03 D1BB FDA0
GnuPG ID : D1BBFDA0


pgpVNrf0vvvgS.pgp
Description: PGP signature


Re: [HACKERS] Bug with pg_ctl -w/wait and config-only directories

2011-10-04 Thread Mr. Aaron W. Swenson
On Mon, Oct 03, 2011 at 04:49:21PM -0300, Alvaro Herrera wrote:
 
 Excerpts from Bruce Momjian's message of lun oct 03 16:09:08 -0300 2011:
 
  Yes, auto-creation of symlinks would be useful, but at that point pg_ctl
  and pg_upgrade would have to use the real data directory, so I again
  wonder what the config-only directory is getting us.
 
 Not mixing config stuff (in /etc per FHS) with server data (/var/lib,
 again per FHS).  It's Debian policy anyway.  I don't judge whether this
 is sane or not.  See
 http://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard
 
  Why were people not using pg_ctl?  Because of the limitations which were
  fixed in PG 9.1?  As Dave already said, windows already has to use pg_ctl.
 
 As I said, Debian has their own version pg_ctlcluster because of their
 work to allow multiple major versions to work simultaneously in the same
 server.  I dunno what about Gentoo.

I implemented separated configuration and data directories to adhere to
FHS. Actually, I had a bug on bugs.gentoo.org -- again, actually, there
have been a few bugs over the years -- requesting that I make PostgreSQL
adhere to the FHS.

I made it work using pg_ctl. The more curious among you can take a look at
the initscripts and related config files from my devspace. [1]

'/etc/init.d/postgresql-9.0 restart' will actually call stop() and
start(). Otherwise, I've mostly paralleled initscript functionality with
the functionality of pg_ctl. Multiple initscripts are installed
side-by-side to control multiple major versions.

1. http://dev.gentoo.org/~titanofold/

-- 
Mr. Aaron W. Swenson
Gentoo Linux Developer
Email: titanof...@gentoo.org
GnuPG FP : 2C00 7719 4F85 FB07 A49C  0E31 5713 AA03 D1BB FDA0
GnuPG ID : D1BBFDA0


pgpv5M1wDmlL1.pgp
Description: PGP signature


Re: [HACKERS] Bug with pg_ctl -w/wait and config-only directories

2011-10-01 Thread Mr. Aaron W. Swenson
On Sat, Oct 01, 2011 at 02:08:33PM -0400, Bruce Momjian wrote:
 In researching pg_ctl -w/wait mode for pg_upgrade, I found that pg_ctl
 -w's handling of configuration-only directories is often incorrect.  For
 example, 'pg_ctl -w stop' checks for the postmaster.pid file to
 determine when the server is shut down, but there is no postmaster.pid
 file in the config directory, so it fails, i.e. does nothing.  What is
 interesting is that specifying the real data directory does work.  
 
 Similarly, pg_ctl references these data directory files:
 
 snprintf(postopts_file, MAXPGPATH, %s/postmaster.opts, pg_data);
 snprintf(backup_file, MAXPGPATH, %s/backup_label, pg_data);
 snprintf(recovery_file, MAXPGPATH, %s/recovery.conf, pg_data);
 snprintf(promote_file, MAXPGPATH, %s/promote, pg_data);
 
 I assume things that use these files also don't work for config-only
 directories.  
 
 You might think that you can always just specify the real data
 directory, but that doesn't work if the server has to be started because
 you need to point to postgresql.conf.  pg_ctl -w restart is a classic
 case of something that needs both the config directory and the real data
 directory.  Basically, this stuff all seems broken and needs to be fixed
 or documented.
 
 What is even worse is that pre-9.1, pg_ctl start would read ports from
 the pg_ctl -o command line, but in 9.1 we changed this to force reading
 the postmaster.pid file to find the port number and socket directory
 location --- meaning, new in PG 9.1, 'pg_ctl -w start' doesn't work for
 config-only directories either.  And, we can't easily connect to the
 server to get the 'data_directory' because we need to read
 postmaster.pid to get the connection settings.  :-(
 
 I think this points to the need for a command-line tool to output the
 data directory location;  I am not sure what to do about the new 9.1
 breakage.
 
 pg_upgrade can work around these issues by starting using the config
 directory and stopping using the real data directory, but it cannot work
 around the 9.1 pg_ctl -w start problem for config-only directories.
 
 -- 
   Bruce Momjian  br...@momjian.ushttp://momjian.us
   EnterpriseDB http://enterprisedb.com
 
   + It's impossible for everything to be true. +

I went through several iterations trying to find a command that can work
the way we'd like it to. (Essentially is works the way you're describing
it should.) So, in Gentoo, for the initscript, we have this really ugly
command to start the server:

su -l postgres \
-c env PGPORT=\${PGPORT}\ ${PG_EXTRA_ENV} \
/usr/lib/postgresql-9.0/bin/pg_ctl \
start ${WAIT_FOR_START} -t ${START_TIMEOUT} -s -D ${DATA_DIR} \
-o '-D ${PGDATA} --data-directory=${DATA_DIR} \
--silent-mode=true ${PGOPTS}'

And to stop the server:

su -l postgres \
-c env PGPORT=\${PGPORT}\ ${PG_EXTRA_ENV} \
/usr/lib/postgresql-9.0/bin/pg_ctl \
stop ${WAIT_FOR_STOP} -t ${NICE_TIMEOUT} -s -D ${DATA_DIR} \
-m smart

The default values for these are:

PGPORT='5432'
PG_EXTRA_ENV=''
WAIT_FOR_START='-w'
START_TIMEOUT='60'
WAIT_FOR_STOP='-w'
NICE_TIMEOUT='60'
DATA_DIR='/var/lib/postgresql/9.0/data'
PGDATA='/etc/postgresql-9.0'
PGOPTS=''

We don't use 'pg_ctl restart', instead we stop and then start the
server. So, I don't have an answer for that. I'd imagine passing '-D
${DATA_DIR}' would do the trick there as well.

Of course, simplifying this a bit would be welcome. 

-- 
Mr. Aaron W. Swenson
Gentoo Linux Developer
Email: titanof...@gentoo.org
GnuPG FP : 2C00 7719 4F85 FB07 A49C  0E31 5713 AA03 D1BB FDA0
GnuPG ID : D1BBFDA0


pgpLbW9GnlOgM.pgp
Description: PGP signature


Re: [HACKERS] pg_upgrade - add config directory setting

2011-09-29 Thread Mr. Aaron W. Swenson
On Thu, Sep 29, 2011 at 10:44:29AM -0300, Alvaro Herrera wrote:
 
 Excerpts from Bruce Momjian's message of jue sep 29 09:56:09 -0300 2011:
  
  Thinking some more, I don't need to know the data directory while the
  server is down --- I already am starting it. pg_upgrade starts both old
  and new servers during its check phase, and it could look up the
  data_directory GUC variable and use that value when accessing files. 
  That would work for old and new servers.  However, I assume that is
  something we would not backpatch to 9.1.  
 
 Why not?  We've supported separate data/config dirs for a long time now,
 so it seems to me that pg_upgrade not coping with them is a bug.  If
 pg_upgrade starts postmaster, it seems simple to grab the data_directory
 setting, is it not?

I was going to say. I'd view this as bringing the behavior of pg_upgrade
to a consistent state with postgres. I vote for it being backpatched to
9.0 even. For whatever my vote is worth.

-- 
Mr. Aaron W. Swenson
Pseudonym: TitanOfOld
Gentoo Developer


pgpRbF13InZSg.pgp
Description: PGP signature


Re: [HACKERS] pg_upgrade - add config directory setting

2011-09-27 Thread Mr. Aaron W. Swenson
On Tue, Sep 27, 2011 at 04:13:41PM -0700, Steve Crawford wrote:
 It would perhaps be useful to add optional --old-confdir and 
 --new-confdir parameters to pg_upgrade. If these parameters are absent 
 then pg_upgrade would work as it does now and assume that the config 
 files are in the datadir.
 
 The reason for this suggestion is that packages for Ubuntu (and I 
 suppose Debian and possibly others) place the config files in a 
 different directory than the data files.
 
 The Ubuntu packaging, for example, puts all the configuration files in 
 /etc/postgresql/VERSION/main/.
 
 If I set the data-directories to /var/lib/postgresql/VERSION/main then 
 pg_upgrade complains about missing config files.
 
 If I set the data directories to /etc/postgresql/VERSION/main/ then 
 pg_upgrade complains that the base subdirectory is missing.
 
 Temporarily symlinking postgresql.conf and pg_hba.conf from the config 
 directory to the data directory allowed the upgrade to run successfully 
 but is a bit more kludgey and non-obvious.
 
 Cheers,
 Steve

I was just about to submit this suggestion. We do the same on Gentoo, as a
default anyway. (Users can pick their own locations for the configuration files
and data directories.) It would simplify the upgrade process by eliminating two
to four steps. (Symlink/copy configuration files in /etc/postgresql-${SLOT}
to /var/lib/postgresql-${SLOT}, same to $version++, pg_upgrade, remove 
symlinks.)

-- 
Mr. Aaron W. Swenson
Pseudonym: TitanOfOld
Gentoo Developer


pgpA97Tt6jV2d.pgp
Description: PGP signature