[HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Simon Riggs

I notice that there are more than a few projects on pgfoundry that are
marked as BSD licence but then the project files don't contain any
mention of the licence details. In some cases, projects are also clearly
marked Copyright of people or organizations.

For example, pg_batch is clearly marked BSD licence, yet the docs and
many of the files are marked Copyright (c) 2010, NIPPON TELEGRAPH AND
TELEPHONE CORPORATION.

pg_lesslog does contain a BSD-looking licence in the COPYRIGHT file, but
is also marked with copyrights.

My understanding is that we had a policy of copyright novation to the
PGDG. Is that not followed up for pgfoundry projects? I think we should
move to a policy of explicit licencing.

In the absence of a licence file, when a project is marked BSD licence
on pgfoundry I think it is safe to presume that the licence for those
files is the same as PostgreSQL's licence.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Fujii Masao
On Tue, May 18, 2010 at 2:26 PM, Simon Riggs si...@2ndquadrant.com wrote:
 On Tue, 2010-05-18 at 12:02 +0900, Fujii Masao wrote:
 On Mon, May 17, 2010 at 9:01 PM, Simon Riggs si...@2ndquadrant.com wrote:
  (1)
  Smart or fast shutdown requested in PM_STARTUP state always removes
  the backup_label file if it exists. But it might be still required
  for subsequent recovery. I changed your patch so that additionally
  the postmaster skips deleting the backup_label in that case.
 
  Don't like the name NeedBackupLabel seems too specific. That really
  corresponds to we were in recovery. We should have a couple of
  super-states that correspond to am in recovery/am not in recovery so we
  can drive it from that.

 ISTM that we can use XLogCtl-SharedRecoveryInProgress for that.
 Is this OK?

 That can change state at any time. Would that work?

Yes. XLogCtl-SharedRecoveryInProgress is set to TRUE only when
XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's
set to FALSE only at the end of recovery.

Here is the updated patch (fix_smart_shutdown_in_recovery_v3_fujii.patch).
I used XLogCtl-SharedRecoveryInProgress instead of NeedBackupLabel,
which made the patch very simple. And I prevented the postmaster
from invoking walreceiver after shutdown or recovery.

 (2)
 pg_ctl -ms stop emits the following warning whenever there is the
 backup_label file in $PGDATA.

   WARNING: online backup mode is active
   Shutdown will not complete until pg_stop_backup() is called.

 This warning doesn't fit in with the shutdown during recovery case.
 Since smart shutdown might be requested by other than pg_ctl, the
 warning should be emitted in server side rather than client, I think.
 How about moving the warning to the server side?

Though I'm not sure if this should be fixed for 9.0, I attached the
patch (move_bkp_cancel_warning_v1.patch).

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center


fix_smart_shutdown_in_recovery_v3_fujii.patch
Description: Binary data


move_bkp_cancel_warning_v1.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Simon Riggs
On Tue, 2010-05-18 at 15:09 +0900, Fujii Masao wrote:
 On Tue, May 18, 2010 at 2:26 PM, Simon Riggs si...@2ndquadrant.com wrote:
  On Tue, 2010-05-18 at 12:02 +0900, Fujii Masao wrote:
  On Mon, May 17, 2010 at 9:01 PM, Simon Riggs si...@2ndquadrant.com wrote:
   (1)
   Smart or fast shutdown requested in PM_STARTUP state always removes
   the backup_label file if it exists. But it might be still required
   for subsequent recovery. I changed your patch so that additionally
   the postmaster skips deleting the backup_label in that case.
  
   Don't like the name NeedBackupLabel seems too specific. That really
   corresponds to we were in recovery. We should have a couple of
   super-states that correspond to am in recovery/am not in recovery so we
   can drive it from that.
 
  ISTM that we can use XLogCtl-SharedRecoveryInProgress for that.
  Is this OK?
 
  That can change state at any time. Would that work?
 
 Yes. XLogCtl-SharedRecoveryInProgress is set to TRUE only when
 XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's
 set to FALSE only at the end of recovery.

You should be using RecoveryInProgress()

 Here is the updated patch (fix_smart_shutdown_in_recovery_v3_fujii.patch).
 I used XLogCtl-SharedRecoveryInProgress instead of NeedBackupLabel,
 which made the patch very simple. And I prevented the postmaster
 from invoking walreceiver after shutdown or recovery.
 
  (2)
  pg_ctl -ms stop emits the following warning whenever there is the
  backup_label file in $PGDATA.
 
WARNING: online backup mode is active
Shutdown will not complete until pg_stop_backup() is called.
 
  This warning doesn't fit in with the shutdown during recovery case.
  Since smart shutdown might be requested by other than pg_ctl, the
  warning should be emitted in server side rather than client, I think.
  How about moving the warning to the server side?
 
 Though I'm not sure if this should be fixed for 9.0, I attached the
 patch (move_bkp_cancel_warning_v1.patch).
 
 Regards,
 
-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Fujii Masao
On Tue, May 18, 2010 at 3:24 PM, Simon Riggs si...@2ndquadrant.com wrote:
 On Tue, 2010-05-18 at 15:09 +0900, Fujii Masao wrote:
 On Tue, May 18, 2010 at 2:26 PM, Simon Riggs si...@2ndquadrant.com wrote:
  On Tue, 2010-05-18 at 12:02 +0900, Fujii Masao wrote:
  On Mon, May 17, 2010 at 9:01 PM, Simon Riggs si...@2ndquadrant.com 
  wrote:
   (1)
   Smart or fast shutdown requested in PM_STARTUP state always removes
   the backup_label file if it exists. But it might be still required
   for subsequent recovery. I changed your patch so that additionally
   the postmaster skips deleting the backup_label in that case.
  
   Don't like the name NeedBackupLabel seems too specific. That really
   corresponds to we were in recovery. We should have a couple of
   super-states that correspond to am in recovery/am not in recovery so we
   can drive it from that.
 
  ISTM that we can use XLogCtl-SharedRecoveryInProgress for that.
  Is this OK?
 
  That can change state at any time. Would that work?

 Yes. XLogCtl-SharedRecoveryInProgress is set to TRUE only when
 XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's
 set to FALSE only at the end of recovery.

 You should be using RecoveryInProgress()

Isn't access to a bool variable atomic?

And I think that postmaster should not take any locks since its
deadlock would cause a fatal situation. No?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Dave Page
On Tue, May 18, 2010 at 6:57 AM, Simon Riggs si...@2ndquadrant.com wrote:

 I notice that there are more than a few projects on pgfoundry that are
 marked as BSD licence but then the project files don't contain any
 mention of the licence details. In some cases, projects are also clearly
 marked Copyright of people or organizations.

I agree that projects should make their licence clear. Gurjeet and I
were just talking about this in relation to Slony, which has only a
copy of the PostgreSQL licence tucked away in an SGML file in the guts
of the tarball, with no text at all to say it's the licence used for
Slony, and not just for PG.

 For example, pg_batch is clearly marked BSD licence, yet the docs and
 many of the files are marked Copyright (c) 2010, NIPPON TELEGRAPH AND
 TELEPHONE CORPORATION.

Don't mix up copyright and licence. They are not the same thing at all.

 pg_lesslog does contain a BSD-looking licence in the COPYRIGHT file, but
 is also marked with copyrights.

 My understanding is that we had a policy of copyright novation to the
 PGDG. Is that not followed up for pgfoundry projects? I think we should
 move to a policy of explicit licencing.

No - pgFoundry projects are licenced and copyright-attributed as their
authors see fit (as long as it's an open source licence of course).

 In the absence of a licence file, when a project is marked BSD licence
 on pgfoundry I think it is safe to presume that the licence for those
 files is the same as PostgreSQL's licence.

The PostgreSQL Licence is not the same as any of the BSD variants, so
that is not a safe presumption to make.

-- 
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Fujii Masao
On Tue, May 18, 2010 at 3:45 PM, Fujii Masao masao.fu...@gmail.com wrote:
 On Tue, May 18, 2010 at 3:24 PM, Simon Riggs si...@2ndquadrant.com wrote:
 On Tue, 2010-05-18 at 15:09 +0900, Fujii Masao wrote:
 On Tue, May 18, 2010 at 2:26 PM, Simon Riggs si...@2ndquadrant.com wrote:
  On Tue, 2010-05-18 at 12:02 +0900, Fujii Masao wrote:
  On Mon, May 17, 2010 at 9:01 PM, Simon Riggs si...@2ndquadrant.com 
  wrote:
   (1)
   Smart or fast shutdown requested in PM_STARTUP state always removes
   the backup_label file if it exists. But it might be still required
   for subsequent recovery. I changed your patch so that additionally
   the postmaster skips deleting the backup_label in that case.
  
   Don't like the name NeedBackupLabel seems too specific. That really
   corresponds to we were in recovery. We should have a couple of
   super-states that correspond to am in recovery/am not in recovery so we
   can drive it from that.
 
  ISTM that we can use XLogCtl-SharedRecoveryInProgress for that.
  Is this OK?
 
  That can change state at any time. Would that work?

 Yes. XLogCtl-SharedRecoveryInProgress is set to TRUE only when
 XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's
 set to FALSE only at the end of recovery.

 You should be using RecoveryInProgress()

 Isn't access to a bool variable atomic?

If it's not atomic, I'll add the following comment into CancelBackup():

Don't bother with lock to access XLogCtl-SharedRecoveryInProgress,
because there should be no other processes running when this code
is reached.

Thought?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Simon Riggs
On Tue, 2010-05-18 at 07:53 +0100, Dave Page wrote:

  For example, pg_batch is clearly marked BSD licence, yet the docs and
  many of the files are marked Copyright (c) 2010, NIPPON TELEGRAPH AND
  TELEPHONE CORPORATION.
 
 Don't mix up copyright and licence. They are not the same thing at all.

I didn't mix those things up, I just used them in the same sentence.
They are two aspects of ownership and appear to offer conflicting
messages, which is a concern to some users.

  pg_lesslog does contain a BSD-looking licence in the COPYRIGHT file, but
  is also marked with copyrights.
 
  My understanding is that we had a policy of copyright novation to the
  PGDG. Is that not followed up for pgfoundry projects? I think we should
  move to a policy of explicit licencing.
 
 No - pgFoundry projects are licenced and copyright-attributed as their
 authors see fit (as long as it's an open source licence of course).

Yes, are they open source licences?

  In the absence of a licence file, when a project is marked BSD licence
  on pgfoundry I think it is safe to presume that the licence for those
  files is the same as PostgreSQL's licence.
 
 The PostgreSQL Licence is not the same as any of the BSD variants, so
 that is not a safe presumption to make.

If, as you say, the licence is unclear then whether-or-not it is an open
source licence must also be unclear.

The copyright holders can change the licence in future as they see fit,
as we've witnessed on other formerly open source projects.

Since the licence is unclear now and the future is subject to change, I
think its safe to say that those projects are fairly unsafe for open
source users.

I'm sure the various other Telco companies out there don't want to hear
that they are using software that NTT might decide in the future to
contest as to whether it was open source or not. Nothing against NTT,
though the principle is clear and effects everything on pgfoundry.

That puts a fairly large hole in recommending that people visit
pgFoundry. That either needs to fixed or users will no longer be able to
trust PgFoundry.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Simon Riggs
On Tue, 2010-05-18 at 16:05 +0900, Fujii Masao wrote:
   ISTM that we can use XLogCtl-SharedRecoveryInProgress for that.
   Is this OK?
  
   That can change state at any time. Would that work?
 
  Yes. XLogCtl-SharedRecoveryInProgress is set to TRUE only when
  XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's
  set to FALSE only at the end of recovery.
 
  You should be using RecoveryInProgress()
 
  Isn't access to a bool variable atomic?
 
 If it's not atomic, I'll add the following comment into CancelBackup():
 
 Don't bother with lock to access XLogCtl-SharedRecoveryInProgress,
 because there should be no other processes running when this code
 is reached.

Call it via a function. There is no need for postmaster to know the
innards of xlog.c, which could change in future. Modularity.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Dave Page
On Tue, May 18, 2010 at 9:06 AM, Simon Riggs si...@2ndquadrant.com wrote:
 On Tue, 2010-05-18 at 07:53 +0100, Dave Page wrote:

  For example, pg_batch is clearly marked BSD licence, yet the docs and
  many of the files are marked Copyright (c) 2010, NIPPON TELEGRAPH AND
  TELEPHONE CORPORATION.

 Don't mix up copyright and licence. They are not the same thing at all.

 I didn't mix those things up, I just used them in the same sentence.
 They are two aspects of ownership and appear to offer conflicting
 messages, which is a concern to some users.

No, copyright is about ownership. The licence is a right granted by
the copyright holders to other to govern their *use* of the code.

 No - pgFoundry projects are licenced and copyright-attributed as their
 authors see fit (as long as it's an open source licence of course).

 Yes, are they open source licences?

All the options on pgFoundry are, yes.

 The PostgreSQL Licence is not the same as any of the BSD variants, so
 that is not a safe presumption to make.

 If, as you say, the licence is unclear then whether-or-not it is an open
 source licence must also be unclear.

Not at all. If it's listed on www.opensource.org, then a licence is
open source. Why do you think I busted a gut to get the PostgreSQL
licence approved when we realised it wasn't BSD?

 The copyright holders can change the licence in future as they see fit,
 as we've witnessed on other formerly open source projects.

 Since the licence is unclear now and the future is subject to change, I
 think its safe to say that those projects are fairly unsafe for open
 source users.

That is the case for *anything*. We could change the PostgreSQL
licence if we wanted, but it would take a huge amount of effort and
approval of every contributor ever whose work could be considered an
artistic contribution.

With PostgreSQL we rely on the sheer number of contributors to ensure
the licence will never actually change. We cannot have such a
guarantee for most smaller projects of course - simply attributing
copyright to a non-existent legal entity such as PGDG (or as I
understand it, even an actual entity) doesn't actually change who
legally owns the copyright.

To get the protection I think you seek, I believe we'd need to create
a legal entity to own the copyright and then have every contributor to
anything on pgFoundry sign a copyright assignment agreement that
grants the legal entity copyright on the current and all future
versions of that work, as hosted on there. And even then, there's no
guarantee that the legal entity couldn't be bought or change it's
charter, unless there's some way to irrevocably build things into its
statutes.

Of course, as you know I'm not a lawyer but have spent a fair bit
of^W^W^Wfar too much time talking to them about this sort of stuff, so
I at least *think* I know what I'm talking about :-)

-- 
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Fujii Masao
On Tue, May 18, 2010 at 5:10 PM, Simon Riggs si...@2ndquadrant.com wrote:
 On Tue, 2010-05-18 at 16:05 +0900, Fujii Masao wrote:
   ISTM that we can use XLogCtl-SharedRecoveryInProgress for that.
   Is this OK?
  
   That can change state at any time. Would that work?
 
  Yes. XLogCtl-SharedRecoveryInProgress is set to TRUE only when
  XLogCtl structure is initialized (i.e., XLOGShmemInit), and it's
  set to FALSE only at the end of recovery.
 
  You should be using RecoveryInProgress()
 
  Isn't access to a bool variable atomic?

 If it's not atomic, I'll add the following comment into CancelBackup():

     Don't bother with lock to access XLogCtl-SharedRecoveryInProgress,
     because there should be no other processes running when this code
     is reached.

 Call it via a function. There is no need for postmaster to know the
 innards of xlog.c, which could change in future. Modularity.

In the patch, it's accessed in CancelBackup() which is in xlog.c.
CancelBackup() should call further wrapping function?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Simon Riggs
On Tue, 2010-05-18 at 09:33 +0100, Dave Page wrote:

  No - pgFoundry projects are licenced and copyright-attributed as their
  authors see fit (as long as it's an open source licence of course).
 
  Yes, are they open source licences?
 
 All the options on pgFoundry are, yes.
 
  The PostgreSQL Licence is not the same as any of the BSD variants, so
  that is not a safe presumption to make.
 
  If, as you say, the licence is unclear then whether-or-not it is an open
  source licence must also be unclear.
 
 Not at all. If it's listed on www.opensource.org, then a licence is
 open source. Why do you think I busted a gut to get the PostgreSQL
 licence approved when we realised it wasn't BSD?

Dave, this is important and so this thread must have a clear resolution,
so we must stick to a single point and be clear about our logic and our
statements.

You're saying these two things, I think, or if you or anybody else
disagrees, please so clearly.

* When project realised that the PostgreSQL licence wasn't actually a
BSD licence, that PostgreSQL was clarified to be the TPL, yet pgfoundry
was not covered by that clarification for some reason.

* In the absence of any licence text in any of the files of a project on
a certain date, then if the project is advertised on PgFoundry on that
date as having a BSD licence then the software will be covered by 
http://www.opensource.org/licenses/bsd-license.php 

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Partitioning/inherited tables vs FKs

2010-05-18 Thread Greg Stark
On Thu, May 6, 2010 at 10:38 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 * the index grows as the size of the total data set, it's not limited
 by partition size

 * can't cheaply drop one partition any more, you have to vacuum the
 (big) index first

So I wholeheartedly agree with the general sentiment that if you need
global indexes then partitioning just isn't really the right tool for
you.

But it occurs to me that we could defer the vacuum safely. I'm
assuming a index heap-tid pointer for a global index would include a
relid or some other identifier to specify which partition the tuple is
in. If you drop that partition those can all just be left as dangling
pointers as long as we don't reuse that id. So all we would need is
some way to leave a catalog entry reserving that id. The data files
can be truncated and deleted normally and whenever vacuum does run
against the index it can clean up the catalog entries for the deleted
partitions.

But I would rather work on having unique and foreign key constraints
that work on keys which include the partition key than work on global
indexes.
-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Greg Stark
On Tue, May 18, 2010 at 4:06 AM, Simon Riggs si...@2ndquadrant.com wrote:
 If, as you say, the licence is unclear then whether-or-not it is an open
 source licence must also be unclear.

I would suggest you, or anyone else who notices, open bugs on any
packages you want to use for which you find no LICENSE file matching
the license asserted in pgfoundry.

Are there so many that we need a more organized mass effort? Do we
need automated checks for this?

 The copyright holders can change the licence in future as they see fit,
 as we've witnessed on other formerly open source projects.

This is always true. The protection open source licenses have for this
is that they're irrevocable. So while NTT could stop releasing future
work under an open source license, the code which was already released
would still be available under the license it was released under and
anyone who wants to could pay anyone willing to support it without
asking NTT for permission.

The question that arises then is whether pgfoundry archives the source
it has in a way that the project maintainer can't delete. If an author
decides to stop releasing a package and deletes the source from
pgfoundry can we get the last version they released from pgfoundry and
put it back up as an orphaned project or with a new set of
maintainers? As long as we have the infrastructure to do that
conveniently I think we're protected against this danger.


-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Dave Page
On Tue, May 18, 2010 at 9:59 AM, Simon Riggs si...@2ndquadrant.com wrote:
 Dave, this is important and so this thread must have a clear resolution,
 so we must stick to a single point and be clear about our logic and our
 statements.

OK. I thought you were talking about copyright and licences though.

 You're saying these two things, I think, or if you or anybody else
 disagrees, please so clearly.

 * When project realised that the PostgreSQL licence wasn't actually a
 BSD licence, that PostgreSQL was clarified to be the TPL, yet pgfoundry
 was not covered by that clarification for some reason.

No. The licences never changed on anything - all we did was get it
approved by the OSI, and clarify our *naming* of the licence in
PostgreSQL (and pgAdmin). It's entirely up to the maintainers of each
project on pgFoundry to decide whether the licence text or the licence
name is what they intend, and to carify accordingly for their
projects.

 * In the absence of any licence text in any of the files of a project on
 a certain date, then if the project is advertised on PgFoundry on that
 date as having a BSD licence then the software will be covered by
 http://www.opensource.org/licenses/bsd-license.php

Yes, I believe that is a fair and safe assumption.



-- 
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Simon Riggs
On Tue, 2010-05-18 at 06:32 -0400, Greg Stark wrote:
 On Tue, May 18, 2010 at 4:06 AM, Simon Riggs si...@2ndquadrant.com wrote:
  If, as you say, the licence is unclear then whether-or-not it is an open
  source licence must also be unclear.
 
 I would suggest you, or anyone else who notices, open bugs on any
 packages you want to use for which you find no LICENSE file matching
 the license asserted in pgfoundry.

I'm not personally going to do this, though I will point out to people
the dangers of imprecisely licenced software when they ask.

 Are there so many that we need a more organized mass effort? Do we
 need automated checks for this?

I would say so. 

  The copyright holders can change the licence in future as they see fit,
  as we've witnessed on other formerly open source projects.
 
 This is always true. The protection open source licenses have for this
 is that they're irrevocable. So while NTT could stop releasing future
 work under an open source license, the code which was already released
 would still be available under the license it was released under and
 anyone who wants to could pay anyone willing to support it without
 asking NTT for permission.
 
 The question that arises then is whether pgfoundry archives the source
 it has in a way that the project maintainer can't delete. If an author
 decides to stop releasing a package and deletes the source from
 pgfoundry can we get the last version they released from pgfoundry and
 put it back up as an orphaned project or with a new set of
 maintainers? As long as we have the infrastructure to do that
 conveniently I think we're protected against this danger.

Well, whoever runs pgfoundry.org gets to make that decision. They may
choose how they respond if someone says I request X, in the name of
PostgreSQL and open source,  

There may or may not keep archived copies. If they just keep a latest
backup, then once the developer quietly deletes stuff then its gone
forever. Who could monitor that to make sure it never takes place??

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Peter Geoghegan
 That is the case for *anything*. We could change the PostgreSQL
 licence if we wanted, but it would take a huge amount of effort and
 approval of every contributor ever whose work could be considered an
 artistic contribution.

I doubt it. Do you think that every single contributor is contactable?
Haven't some died? My guess is that it would be completely impossible.

 With PostgreSQL we rely on the sheer number of contributors to ensure
 the licence will never actually change. We cannot have such a
 guarantee for most smaller projects of course - simply attributing
 copyright to a non-existent legal entity such as PGDG (or as I
 understand it, even an actual entity) doesn't actually change who
 legally owns the copyright.

 To get the protection I think you seek, I believe we'd need to create
 a legal entity to own the copyright and then have every contributor to
 anything on pgFoundry sign a copyright assignment agreement that
 grants the legal entity copyright on the current and all future
 versions of that work, as hosted on there. And even then, there's no
 guarantee that the legal entity couldn't be bought or change it's
 charter, unless there's some way to irrevocably build things into its
 statutes.

IANAL, but I know that there was a similar situation when Trolltech
still existed and controlled the Qt framework. It was dual licensed
GPL2/proprietary (it is now dual LGPL/proprietary). Contributors were
required to sign reams of paperwork, which had to be sent out by fax
(I'm not sure why), to assign the copyright to Trolltech. Thankfully,
that situation has changed under Nokia - contributors retain the
copyright, and there is minimal red tape. Contributors are now asked
to grant Qt Software a non-exclusive right to re-use code as a part of
Qt, the first time they submit code for inclusion.

Regards,
Peter Geoghegan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synchronous replication patch built on SR

2010-05-18 Thread Fujii Masao
Thanks for your reply!

On Fri, May 14, 2010 at 10:33 PM, Boszormenyi Zoltan z...@cybertec.at wrote:
 In your design, the transaction commit on the master waits for its XID
 to be read from the XLOG_XACT_COMMIT record and replied by the standby.
 Right? This design seems not to be extensible to #2 and #3 since
 walreceiver cannot read XID from the XLOG_XACT_COMMIT record.

 Yes, this was my problem, too. I would have had to
 implement a custom interpreter into walreceiver to
 process the WAL records and extract the XIDs.

Isn't reading the same WAL twice (by walreceiver and startup process)
inefficient? In synchronous replication, the overhead of walreceiver
directly affects the performance of the master. We should not assign
such a hard work to walreceiver, I think.

 But at least the supporting details, i.e. not opening another
 connection, instead being able to do duplex COPY operations in
 a server-acknowledged way is acceptable, no? :-)

Though I might not understand your point (sorry), it's OK for the standby
to send the reply to the master by using CopyData message. Currently
PQputCopyData() cannot be executed in COPY OUT, but we can relax
that.

  How about
 using LSN instead of XID? That is, the transaction commit waits until
 the standby has reached its LSN. LSN is more easy-used for walreceiver
 and startup process, I think.


 Indeed, using the LSN seems to be more appropriate for
 the walreceiver, but how would you extract the information
 that a certain LSN means a COMMITted transaction? Or
 we could release a locked transaction in case the master receives
 an LSN greater than or equal to the transaction's own LSN?

Yep, we can ensure that the transaction has been replicated by
comparing its own LSN with the smallest LSN in the latest LSNs
of each connected synchronous standby.

 Sending back all the LSNs in case of long transactions would
 increase the network traffic compared to sending back only the
 XIDs, but the amount is not clear for me. What I am more
 worried about is the contention on the ProcArrayLock.
 XIDs are rarer then LSNs, no?

No. For example, when WAL data sent by walsender at a time
has two XLOG_XACT_COMMIT records, in XID approach, walreceiver
would need to send two replies. OTOH, in LSN approach, only
one reply which indicates the last received location would
need to be sent.

 What if the synchronous standby starts up from the very old backup?
 The transaction on the master needs to wait until a large amount of
 outstanding WAL has been applied? I think that synchronous replication
 should start with *asynchronous* replication, and should switch to the
 sync level after the gap between servers has become enough small.
 What's your opinion?


 It's certainly one option, which I think partly addressed
 with the strict_sync_replication knob below.
 If strict_sync_replication = off, then the master doesn't make
 its transactions wait for the synchronous reports, and the client(s)
 can work through their WALs. IIRC, the walreceiver connects
 to the master only very late in the recovery process, no?

No, the master might have a large number of WAL files which
the standby doesn't have.

 I have added 3 new options, two GUCs in postgresql.conf and one
 setting in recovery.conf. These options are:

 1. min_sync_replication_clients = N

 where N is the number of reports for a given transaction before it's
 released as committed synchronously. 0 means completely asynchronous,
 the value is maximized by the value of max_wal_senders. Anything
 in between 0 and max_wal_senders means different levels of partially
 synchronous replication.

 2. strict_sync_replication = boolean

 where the expected number of synchronous reports from standby
 servers is further limited to the actual number of connected synchronous
 standby servers if the value of this GUC is false. This means that if
 no standby servers are connected yet then the replication is asynchronous
 and transactions are allowed to finish without waiting for synchronous
 reports. If the value of this GUC is true, then transactions wait until
 enough synchronous standbys connect and report back.


 Why are these options necessary?

 Can these options cover more than three synchronization levels?


 I think I explained it in my mail.

 If  min_sync_replication_clients == 0, then the replication is async.
 If  min_sync_replication_clients == max_wal_senders then the
 replication is fully synchronous.
 If 0  min_sync_replication_clients  max_wal_senders then
 the replication is partially synchronous, i.e. the master can wait
 only for say, 50% of the clients to report back before it's considered
 synchronous and the relevant transactions get released from the wait.

Seems s/min_sync_replication_clients/max_sync_replication_clients

min_sync_replication_clients is required to prevent outside attacker
from connecting to the master as synchronous standby, and degrading
the performance on the master? Other usecase?

Regards,

-- 
Fujii 

Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Andrew Dunstan



Simon Riggs wrote:

That puts a fairly large hole in recommending that people visit
pgFoundry. That either needs to fixed or users will no longer be able to
trust PgFoundry.

  


pgFoundry is a resource we provide the community. The projects there are 
the responsibility of their individual owners. We are not going to start 
being the license police. I at least have neither the time to do that 
nor any interest in doing it. If people want to use what is on pgFoundry 
then it is up to them to make sure it has whatever licence meets their 
requirements.


What we should do is add the PostgreSQL license to the list of available 
licenses and make sure it is the default for new projects.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Robert Haas
On Mon, May 17, 2010 at 10:40 PM, Fujii Masao masao.fu...@gmail.com wrote:
 On Mon, May 17, 2010 at 10:20 PM, Robert Haas robertmh...@gmail.com wrote:
 OK, I think I understand now.  But, the SIGTERM sent by the postmaster
 doesn't kill the recovery process unconditionally.  It will invoke
 StartupProcShutdownHandler(), which will set set shutdown_requested =
 true.  That gets checked by RestoreArchivedFile() and
 HandleStartupProcInterrupts(), and I think that neither of those can
 get invoked until after the control file has been updated.  Do you see
 a way it can happen?

 Yeah, the way is:
 StartupXLOG() -- ReadCheckpointRecord() -- ReadRecord() --
 XLogPageRead() -- XLogFileReadAnyTLI() -- XLogFileRead() --
 RestoreArchivedFile()

 ReadCheckpointRecord() is called before pg_control is updated.

OK.  In that case, I'm wondering if we should reverse course and
rejigger the logic so that the shutdown gets processed when we
transition to PM_RECOVERY.  Seems like that might be simpler.

 ISTM that walreceiver might be invoked even after shutdown is requested.
 We should prevent the postmaster from starting up walreceiver if
 Shutdown  NoShutdown?

Well, when we did the previous shutdown patch, we decided it was not
right to kill walreceiver until all backends had exited, so it seems
inconsistent to make the opposite decision here.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synchronous replication patch built on SR

2010-05-18 Thread Fujii Masao
On Sat, May 15, 2010 at 4:59 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
 BTW, What I'd like to see as a very first patch first is to change the
 current poll loops in walreceiver and walsender to, well, not poll.
 That's a requirement for synchronous replication, is very useful on its
 own, and requires a some design and implementation effort to get right.
 It would be nice to get that out of the way before/during we discuss the
 more user-visible behavior.

Yeah, we should wake up the walesender from sleep to send WAL data
as soon as it's flushed. But why do we need to change the loop of
walreceiver? Or you mean changing the poll loop in the startup process?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread jesper
Hi

I tried running pg_upgrade from the current snapshot of postgresql and
upgrading from 8.4.4 to the snapshot version. Everything seem to look fine
in the process and all that came out was only ok's but when I tried a
simple query on the databse it keeps throwing these message out of the back
side.

DETAIL:  You might have already suffered transaction-wraparound data loss.
WARNING:  some databases have not been vacuumed in over 2 billion
transactions


The database was around 600GB and it took a couple of minutes to run
pg_upgrade after I had all the binaries in the correct place.

It is not really an easy task to throw around 600GB of data, so I cannot
gaurantee that the above is reproducible, but I'll see if I can get time
and try to reproduce it.

Jesper


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Stefan Kaltenbrunner
On 05/18/2010 07:32 AM, Andrew Dunstan wrote:
 
 
 Simon Riggs wrote:
 That puts a fairly large hole in recommending that people visit
 pgFoundry. That either needs to fixed or users will no longer be able to
 trust PgFoundry.

   
 
 pgFoundry is a resource we provide the community. The projects there are
 the responsibility of their individual owners. We are not going to start
 being the license police. I at least have neither the time to do that
 nor any interest in doing it. If people want to use what is on pgFoundry
 then it is up to them to make sure it has whatever licence meets their
 requirements.

I agree there - pgfoundry is just the resource provider, we are not a
licence police (and given that none of the pgf admins is an actual
lawyer there is no sense in even trying).
People wanting to get some sort of indemnification or whatever need to
look into commercial providers (or use distribution provided packages
for stuff because those are usually very well checked for licence stuff
in all major linux distributions).


 
 What we should do is add the PostgreSQL license to the list of available
 licenses and make sure it is the default for new projects.

I can look into that...


Stefan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Simon Riggs
On Tue, 2010-05-18 at 07:32 -0400, Andrew Dunstan wrote:
 
 Simon Riggs wrote:
  That puts a fairly large hole in recommending that people visit
  pgFoundry. That either needs to fixed or users will no longer be able to
  trust PgFoundry.
 

 pgFoundry is a resource we provide the community. The projects there are 
 the responsibility of their individual owners. We are not going to start 
 being the license police. I at least have neither the time to do that 
 nor any interest in doing it. If people want to use what is on pgFoundry 
 then it is up to them to make sure it has whatever licence meets their 
 requirements.

Agreed, though that significantly lessens the value of that resource for
everybody. If somebody would like to try to improve that by attempting
to improve or police the licencing, it would be appreciated.

 What we should do is add the PostgreSQL license to the list of available 
 licenses and make sure it is the default for new projects.

Good idea.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Bug with ordering aggregates?

2010-05-18 Thread Stephen Frost
Greetings,

  This doesn't seem right to me:

postgres=# select
postgres-# string_agg(column1::text order by column1 asc,',')
postgres-# from (values (3),(4),(1),(2)) a;
 string_agg 

 1234
(1 row)

  I'm thinking we should toss a syntax error here and force the 'order
  by' to be at the end of any arguments to the aggregate.
  Alternatively, we should actually make this work like this one does:

postgres=# select
postgres-# string_agg(column1::text,',' order by column1 asc)
postgres-# from (values (3),(4),(1),(2)) a;
 string_agg 

 1,2,3,4
(1 row)

Thanks,

Stephen


signature.asc
Description: Digital signature


Re: [HACKERS] Bug with ordering aggregates?

2010-05-18 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes:
   This doesn't seem right to me:

 postgres=# select
 postgres-# string_agg(column1::text order by column1 asc,',')
 postgres-# from (values (3),(4),(1),(2)) a;
  string_agg 
 
  1234
 (1 row)

Looks fine to me: you have two ordering columns (the second rather
useless,  but that's no matter).

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Bug with ordering aggregates?

2010-05-18 Thread Thom Brown
On 18 May 2010 16:37, Stephen Frost sfr...@snowman.net wrote:
 Greetings,

  This doesn't seem right to me:

 postgres=# select
 postgres-# string_agg(column1::text order by column1 asc,',')
 postgres-# from (values (3),(4),(1),(2)) a;
  string_agg
 
  1234
 (1 row)

  I'm thinking we should toss a syntax error here and force the 'order
  by' to be at the end of any arguments to the aggregate.
  Alternatively, we should actually make this work like this one does:

 postgres=# select
 postgres-# string_agg(column1::text,',' order by column1 asc)
 postgres-# from (values (3),(4),(1),(2)) a;
  string_agg
 
  1,2,3,4
 (1 row)


I find that 2nd example confusing.  It suggests the delimiter is being
ordered as the order by clause appears in its parameter.  But I can
see why the first one is returning the wrong result.  The order by
clause conflicts with the delimiter parameter as obviously the order
by clause prevents you specifying a 2nd parameter in the aggregate
function.  The delimiter would either need to be the first parameter,
or the order by clause would require a way to terminate it's order by
list.

Thom

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Bug with ordering aggregates?

2010-05-18 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote:
 Stephen Frost sfr...@snowman.net writes:
This doesn't seem right to me:
 
  postgres=# select
  postgres-# string_agg(column1::text order by column1 asc,',')
  postgres-# from (values (3),(4),(1),(2)) a;
   string_agg 
  
   1234
  (1 row)
 
 Looks fine to me: you have two ordering columns (the second rather
 useless,  but that's no matter).

Ah, yeah, guess I'll just complain that having the order by look like
it's an argument to an aggregate makes things confusing.  Not much to be
done about it though.

Thanks,

Stephen


signature.asc
Description: Digital signature


[HACKERS] Documentation Bug/Misnomer?

2010-05-18 Thread Stephen Frost
Greetings,

Under:

http://developer.postgresql.org/pgdocs/postgres/runtime-config-file-locations.html

We have:

ident_file (string)

Specifies the configuration file for ident authentication
(customarily called pg_ident.conf). This parameter can only be set
at server start.

That's not really accurate anymore, is it?  It's referring to the
username maps now, which are used for Ident, GSSAPI, SSL, etc..

Thanks,

Stephen


signature.asc
Description: Digital signature


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread Bruce Momjian
jes...@krogh.cc wrote:
 Hi
 
 I tried running pg_upgrade from the current snapshot of postgresql and
 upgrading from 8.4.4 to the snapshot version. Everything seem to look fine
 in the process and all that came out was only ok's but when I tried a
 simple query on the databse it keeps throwing these message out of the back
 side.
 
 DETAIL:  You might have already suffered transaction-wraparound data loss.
 WARNING:  some databases have not been vacuumed in over 2 billion
 transactions
 
 
 The database was around 600GB and it took a couple of minutes to run
 pg_upgrade after I had all the binaries in the correct place.
 
 It is not really an easy task to throw around 600GB of data, so I cannot
 gaurantee that the above is reproducible, but I'll see if I can get time
 and try to reproduce it.

This certainly should never have happened, so I am guessing it is a bug.
pg_upgrade tries hard to make sure all your datfrozenxid and
relfrozenxid are properly migrated from the old server, and the
transaction id is set properly.  Unfortunately this is the first time I
have heard of such a problem, so I am unclear on its cause.

The warning is issued from vacuum.c::vac_truncate_clog().  Can you run
this query and show us the output:

SELECT datname, datfrozenxid FROM pg_database;

It would be good to see these numbers on both the old and new servers.
I would also like to see:

SELECT txid_current();

on the old and new servers, but if you can only provide these values on
one of the two servers, it is still useful.  Thanks.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread jesper
 jes...@krogh.cc wrote:
 Hi

 I tried running pg_upgrade from the current snapshot of postgresql and
 upgrading from 8.4.4 to the snapshot version. Everything seem to look
 fine
 in the process and all that came out was only ok's but when I tried a
 simple query on the databse it keeps throwing these message out of the
 back
 side.

 DETAIL:  You might have already suffered transaction-wraparound data
 loss.
 WARNING:  some databases have not been vacuumed in over 2 billion
 transactions


 The database was around 600GB and it took a couple of minutes to run
 pg_upgrade after I had all the binaries in the correct place.

 It is not really an easy task to throw around 600GB of data, so I cannot
 gaurantee that the above is reproducible, but I'll see if I can get time
 and try to reproduce it.

 This certainly should never have happened, so I am guessing it is a bug.
 pg_upgrade tries hard to make sure all your datfrozenxid and
 relfrozenxid are properly migrated from the old server, and the
 transaction id is set properly.  Unfortunately this is the first time I
 have heard of such a problem, so I am unclear on its cause.

 The warning is issued from vacuum.c::vac_truncate_clog().  Can you run
 this query and show us the output:

   SELECT datname, datfrozenxid FROM pg_database;

 It would be good to see these numbers on both the old and new servers.
 I would also like to see:

   SELECT txid_current();

 on the old and new servers, but if you can only provide these values on
 one of the two servers, it is still useful.  Thanks.

Hi Bruce, thanks for your prompt response.

First the new one..

j...@pal:~$ psql -p 5433
psql (9.0beta1)
Type help for help.

data=# SELECT datname, datfrozenxid FROM pg_database;
  datname  | datfrozenxid
---+--
 template0 |  654
 postgres  |   2374592801
 data  |   2023782337
 jk|   2023822188
 template1 |   2374592801
 workqueue |   2023822188
(6 rows)

data=# SELECT txid_current();
 txid_current
--
   2375384556
(1 row)

data=# \q

Then the old one.

j...@pal:~$ psql data
psql (9.0beta1, server 8.4.1)
WARNING: psql version 9.0, server version 8.4.
 Some psql features might not work.
Type help for help.

data# SELECT datname, datfrozenxid FROM pg_database;
  datname  | datfrozenxid
---+--
 template0 |   2073823552
 postgres  |   2023820521
 data  |   2023782337
 jk|   2023822188
 template1 |   2073823552
 workqueue |   2023822188
(6 rows)

data=# SELECT txid_current();
 txid_current
--
   2390524243
(1 row)


The old database has been copied over using rsync and
pg_start_backup()/pg_stop_backup() procecures and started up
using a recovery.conf file.

Jesper



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Unexpected data beyond EOF during heavy writes

2010-05-18 Thread Tony Sullivan
Hello everyone,

We are seeing the following error message occasionally in the postgres logs:

2010-05-13 23:49:03 PDT ERROR: unexpected data beyond EOF in block 4106698 of 
relation custom_discoveryprofile
2010-05-13 23:49:03 PDT HINT: This has been seen to occur with buggy kernels; 
consider updating your system.

I have read several threads on this, including:
http://archives.postgresql.org/pgsql-general/2007-03/msg01535.php
and
http://archives.postgresql.org/pgsql-general/2009-07/msg01011.php

I am wondering if anyone has ever placed a bug report to any of the linux 
vendors or open source sites?
I could not find a bug report that matched this at http://bugzilla.redhat.com/ 
and am trying to track down what is needed to resolve the error.

We are running a an IBM Blade Center server 64 bit Red Hat 5.3 kernel with 16 
cores doing heavy I/O to the database.
uname -a
Linux 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008 x86_64 x86_64 x86_64 
GNU/Linux x86_64 x86_64 x86_64 GNU/Linux

Thanks for any pointers,

Tony Sullivan


Re: [HACKERS] Clarifications of licences on pgfoundry

2010-05-18 Thread Stefan Kaltenbrunner
On 05/18/2010 09:22 AM, Simon Riggs wrote:
 On Tue, 2010-05-18 at 07:32 -0400, Andrew Dunstan wrote:

 Simon Riggs wrote:
 That puts a fairly large hole in recommending that people visit
 pgFoundry. That either needs to fixed or users will no longer be able to
 trust PgFoundry.

 
 pgFoundry is a resource we provide the community. The projects there are 
 the responsibility of their individual owners. We are not going to start 
 being the license police. I at least have neither the time to do that 
 nor any interest in doing it. If people want to use what is on pgFoundry 
 then it is up to them to make sure it has whatever licence meets their 
 requirements.
 
 Agreed, though that significantly lessens the value of that resource for
 everybody. If somebody would like to try to improve that by attempting
 to improve or police the licencing, it would be appreciated.

even if somebody steps up and tries to to that - we have hundreds of
projects on pgf and I think it is impossible to do anything that would
actually provide some sort of guarantee that the licence stuff is
properly done fore every project which is the only thing that would
prevent you to do your own research or evaluation. However it makes
sense to the the projects you where you ran into an issue about so it
can be fixed (technically this is simply a bug that needs to be reported).

 
 What we should do is add the PostgreSQL license to the list of available 
 licenses and make sure it is the default for new projects.
 
 Good idea.

done


Stefan

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Documentation Bug/Misnomer?

2010-05-18 Thread Magnus Hagander
On Tue, May 18, 2010 at 12:49 PM, Stephen Frost sfr...@snowman.net wrote:
 Greetings,

 Under:

 http://developer.postgresql.org/pgdocs/postgres/runtime-config-file-locations.html

 We have:

 ident_file (string)

    Specifies the configuration file for ident authentication
        (customarily called pg_ident.conf). This parameter can only be set
        at server start.

 That's not really accurate anymore, is it?  It's referring to the
 username maps now, which are used for Ident, GSSAPI, SSL, etc..

That definitely looks wrong. How about this?

diff -c -r1.276 config.sgml
*** config.sgml 3 May 2010 10:31:29 -   1.276
--- config.sgml 18 May 2010 18:44:30 -
***
*** 242,249 
/indexterm
listitem
 para
!  Specifies the configuration file for
!  applicationident/ authentication
   (customarily called filenamepg_ident.conf/).
   This parameter can only be set at server start.
 /para
--- 242,248 
/indexterm
listitem
 para
!  Specifies the configuration file for username mapping
   (customarily called filenamepg_ident.conf/).
   This parameter can only be set at server start.
 /para


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Documentation Bug/Misnomer?

2010-05-18 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote:
 That definitely looks wrong. How about this?

Looks good to me, but I'd add a link to the Username Maps page.

Thanks!

Stephen


signature.asc
Description: Digital signature


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread Bruce Momjian
jes...@krogh.cc wrote:
  on the old and new servers, but if you can only provide these values on
  one of the two servers, it is still useful.  Thanks.
 
 Hi Bruce, thanks for your prompt response.
 
 First the new one..

Great.

 j...@pal:~$ psql -p 5433
 psql (9.0beta1)
 Type help for help.
 
 data=# SELECT datname, datfrozenxid FROM pg_database;
   datname  | datfrozenxid
 ---+--
  template0 |  654
  postgres  |   2374592801
  data  |   2023782337
  jk|   2023822188
  template1 |   2374592801
  workqueue |   2023822188
 (6 rows)
 
 data=# SELECT txid_current();
  txid_current
 --
2375384556
 (1 row)
 
 data=# \q
 
 Then the old one.
 
 j...@pal:~$ psql data
 psql (9.0beta1, server 8.4.1)
 WARNING: psql version 9.0, server version 8.4.
  Some psql features might not work.
 Type help for help.
 
 data# SELECT datname, datfrozenxid FROM pg_database;
   datname  | datfrozenxid
 ---+--
  template0 |   2073823552

This line above looks very odd because I didn't think the template0
datfrozenxid could be advanced.  Can I see the output of this query:

SELECT datname, datfrozenxid, datallowconn FROM pg_database;

I am wondering if you set datallowconn for template0 to 'true'.

  postgres  |   2023820521
  data  |   2023782337
  jk|   2023822188
  template1 |   2073823552
  workqueue |   2023822188
 (6 rows)
 
 data=# SELECT txid_current();
  txid_current
 --
2390524243
 (1 row)
 
 
 The old database has been copied over using rsync and
 pg_start_backup()/pg_stop_backup() procecures and started up
 using a recovery.conf file.

My other idea is that somehow recovery touches datallowconn for
template0.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Documentation Bug/Misnomer?

2010-05-18 Thread Magnus Hagander
On Tue, May 18, 2010 at 2:46 PM, Stephen Frost sfr...@snowman.net wrote:
 * Magnus Hagander (mag...@hagander.net) wrote:
 That definitely looks wrong. How about this?

 Looks good to me, but I'd add a link to the Username Maps page.

Done and applied.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread Jesper Krogh

On 2010-05-18 20:52, Bruce Momjian wrote:

This line above looks very odd because I didn't think the template0
datfrozenxid could be advanced.  Can I see the output of this query:

SELECT datname, datfrozenxid, datallowconn FROM pg_database;

   


Only from the old database:
data=# SELECT datname, datfrozenxid, datallowconn FROM pg_database;
  datname  | datfrozenxid | datallowconn
---+--+--
 template0 |   2073823552 | f
 postgres  |   2023820521 | t
 data  |   2023782337 | t
 jk|   2023822188 | t
 template1 |   2073823552 | t
 workqueue |   2023822188 | t
(6 rows)


I am wondering if you set datallowconn for template0 to 'true'.


From this database, I cannot give any more results, I ran some other
queries and then restarted postgres, subsequently it seemed
totally broken. I'm in the process of running the test over again, but
it'll take a while before data is in.  I'll report back.

--
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread Bruce Momjian
Jesper Krogh wrote:
 On 2010-05-18 20:52, Bruce Momjian wrote:
  This line above looks very odd because I didn't think the template0
  datfrozenxid could be advanced.  Can I see the output of this query:
 
  SELECT datname, datfrozenxid, datallowconn FROM pg_database;
 
 
 
 Only from the old database:
 data=# SELECT datname, datfrozenxid, datallowconn FROM pg_database;
datname  | datfrozenxid | datallowconn
 ---+--+--
   template0 |   2073823552 | f
   postgres  |   2023820521 | t
   data  |   2023782337 | t
   jk|   2023822188 | t
   template1 |   2073823552 | t
   workqueue |   2023822188 | t
 (6 rows)

OK, datallowconn = false is right for template0, but I am still confused
how it got set to that high value.

  I am wondering if you set datallowconn for template0 to 'true'.
 
  From this database, I cannot give any more results, I ran some other
 queries and then restarted postgres, subsequently it seemed
 totally broken. I'm in the process of running the test over again, but
 it'll take a while before data is in.  I'll report back.

OK, thanks.  This does seem odd.  Frankly, having template0's
datfrozenxid be wrong would not cause any kind of instability because
template0 is used only by pg_dump, so I am wondering if something else
is seriously wrong.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Synchronous replication patch built on SR

2010-05-18 Thread Heikki Linnakangas

On 18/05/10 07:41, Fujii Masao wrote:

On Sat, May 15, 2010 at 4:59 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com  wrote:

BTW, What I'd like to see as a very first patch first is to change the
current poll loops in walreceiver and walsender to, well, not poll.
That's a requirement for synchronous replication, is very useful on its
own, and requires a some design and implementation effort to get right.
It would be nice to get that out of the way before/during we discuss the
more user-visible behavior.


Yeah, we should wake up the walesender from sleep to send WAL data
as soon as it's flushed. But why do we need to change the loop of
walreceiver? Or you mean changing the poll loop in the startup process?


Yeah, changing the poll loop in the startup process is what I meant.

--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread Bruce Momjian
jes...@krogh.cc wrote:
 First the new one..
 
 j...@pal:~$ psql -p 5433
 psql (9.0beta1)
 Type help for help.
 
 data=# SELECT datname, datfrozenxid FROM pg_database;
   datname  | datfrozenxid
 ---+--
  template0 |  654
  postgres  |   2374592801
  data  |   2023782337
  jk|   2023822188
  template1 |   2374592801
  workqueue |   2023822188
 (6 rows)
 
 data=# SELECT txid_current();
  txid_current
 --
2375384556
 (1 row)

I just ran a test and all the datfrozenxids are less than the current
xid, so the only database that could be generating a wraparound warning
is 'template0'.  But, again, I though that template0 was not touched for
wraparound protection --- I am starting to think I am wrong.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread Alvaro Herrera
Excerpts from jesper's message of mar may 18 13:22:12 -0400 2010:

 j...@pal:~$ psql data
 psql (9.0beta1, server 8.4.1)
 WARNING: psql version 9.0, server version 8.4.
  Some psql features might not work.
 Type help for help.
 
 data# SELECT datname, datfrozenxid FROM pg_database;
   datname  | datfrozenxid
 ---+--
  template0 |   2073823552
  postgres  |   2023820521
  data  |   2023782337
  jk|   2023822188
  template1 |   2073823552
  workqueue |   2023822188
 (6 rows)

Does the old server have pg_database.datallowconn = true for template0?
-- 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread Bruce Momjian
Alvaro Herrera wrote:
 Excerpts from jesper's message of mar may 18 13:22:12 -0400 2010:
 
  j...@pal:~$ psql data
  psql (9.0beta1, server 8.4.1)
  WARNING: psql version 9.0, server version 8.4.
   Some psql features might not work.
  Type help for help.
  
  data# SELECT datname, datfrozenxid FROM pg_database;
datname  | datfrozenxid
  ---+--
   template0 |   2073823552
   postgres  |   2023820521
   data  |   2023782337
   jk|   2023822188
   template1 |   2073823552
   workqueue |   2023822188
  (6 rows)
 
 Does the old server have pg_database.datallowconn = true for template0?

The user reported back that it did not:

Only from the old database:
data=# SELECT datname, datfrozenxid, datallowconn FROM pg_database;
   datname  | datfrozenxid | datallowconn
---+--+--
  template0 |   2073823552 | f
  postgres  |   2023820521 | t
  data  |   2023782337 | t
  jk|   2023822188 | t
  template1 |   2073823552 | t
  workqueue |   2023822188 | t
(6 rows)

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Unexpected data beyond EOF during heavy writes

2010-05-18 Thread Alvaro Herrera
Excerpts from Tony Sullivan's message of mar may 18 13:19:13 -0400 2010:
 Hello everyone,
 
 We are seeing the following error message occasionally in the postgres logs:
 
 2010-05-13 23:49:03 PDT ERROR: unexpected data beyond EOF in block 4106698 of 
 relation custom_discoveryprofile
 2010-05-13 23:49:03 PDT HINT: This has been seen to occur with buggy kernels; 
 consider updating your system.

What's your storage?

-- 

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] fillfactor gets set to zero for toast tables

2010-05-18 Thread Alvaro Herrera
Excerpts from Tom Lane's message of vie may 14 15:03:57 -0400 2010:

 Maybe a better solution is to have some kind of notion of a default-only
 entry, which is sufficient to insert the default into the struct but
 isn't accepted as a user-settable item.

This patch (for 8.4, but applies fuzzily to 9.0) implements this idea.
Note that there's no explicit check that every heap option has a
corresponding toast option; that's left to the developer's judgement to
add.  I added the new member to relopt_gen struct so that existing
entries did not require changes in initializers.

(I'll leave the error as ERRCODE_CANT_CHANGE_RUNTIME_PARAM unless
someone thinks ERRCODE_INVALID_PARAMETER_VALUE is better)

I checked and this is the only heap setting that did not have a toast
setting.

I'll leave this open to comments for a bit more time than usual, to give
room for pgcon beers and such ...
-- 


relopt-toast-ff.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] BYTEA / DBD::Pg change in 9.0 beta

2010-05-18 Thread Jesper Krogh

Hi.

I'm trying to do a test move of one of our applications onto 9.0beta1.
We use storable and serializes data into a bytea column in the database.
This script uses that:

#!/usr/bin/perl
use strict;
use warnings;
use Storable;
use DBI;
use DBD::Pg;
use Data::Dumper;

my $dbh = DBI-connect(dbi:Pg:dbname=testdb,,,{AutoCommit = 1});
my $sql = END
create table testtable (id serial, testbytea bytea);
END
;

eval {
$dbh-do($sql);
};
$dbh-do(delete from testtable);
my $href = { this = 1, that = 2};
print Before:  . Dumper($href) . \n;
my $sth = $dbh-prepare(insert into testtable (testbytea) values (?));
my $frozen = Storable::nfreeze($href);
$sth-bind_param(1, $frozen, { pg_type=DBD::Pg::PG_BYTEA });
$sth-execute;
$sth = $dbh-prepare(select testbytea from testtable);
$sth-execute();
my $row = $sth-fetchrow_hashref();
my $href2 = Storable::thaw($row-{testbytea});

print Dumper($href2);


Running it against 8.4 gives:

$ perl bin/test-bytea
NOTICE:  CREATE TABLE will create implicit sequence testtable_id_seq1 
for serial column testtable.id
DBD::Pg::db do failed: ERROR:  relation testtable already exists at 
bin/efam/test-bytea line 16.

Before: $VAR1 = {
  'that' = '2',
  'this' = '1'
};

$VAR1 = {
  'that' = '2',
  'this' = '1'
};

Whereas 9.0beta1 gives:
$ perl bin/test-bytea
NOTICE:  CREATE TABLE will create implicit sequence testtable_id_seq1 
for serial column testtable.id
DBD::Pg::db do failed: ERROR:  relation testtable already exists at 
bin/efam/test-bytea line 16.

Before: $VAR1 = {
  'that' = '2',
  'this' = '1'
};

Storable binary image v60.48 more recent than I am (v2.7) at 
../../lib/Storable.pm (autosplit into ../../lib/auto/Storable/thaw.al) 
line 366, at bin/test-bytea line 28


Inspecting the data seems that it is the insert that does something to 
the data:


8.4
 id |   testbytea
+
  9 | 
\005\007\003\000\000\000\002\012\0012\000\000\000\004that\012\0011\000\000\000\004this

(1 row)

9.0beta1
 id |  testbytea
+--
  3 | \x05070300020a01320004746861740a0131000474686973
(1 row)


Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Keepalive for max_standby_delay

2010-05-18 Thread Heikki Linnakangas

On 17/05/10 04:40, Simon Riggs wrote:

On Sun, 2010-05-16 at 16:53 +0100, Simon Riggs wrote:


Attached patch rearranges the walsender loops slightly to fix the above.
XLogSend() now only sends up to MAX_SEND_SIZE bytes (== XLOG_SEG_SIZE /
2) in one round and returns to the main loop after that even if there's
unsent WAL, and the main loop no longer sleeps if there's unsent WAL.
That way the main loop gets to respond to signals quickly, and we also
get to update the shared memory status and PS display more often when
there's a lot of catching up to do.

Comments


8MB at a time still seems like a large batch to me.

libpq is going to send it in smaller chunks anyway, so I don't see the
importance of trying to keep the batch too large. It just introduces
delay into the sending process. We should be sending chunks that matches
libpq better.


More to the point the logic will fail if XLOG_BLCKSZ  PQ_BUFFER_SIZE
because it will send partial pages.


I don't see a failure. We rely on not splitting WAL records across 
messages, but we're talking about libpq-level CopyData messages, not TCP 
messages.



Having MAX_SEND_SIZE  PQ_BUFFER_SIZE is pointless, as libpq currently
stands.


Well, it does affect the size of the read() in walsender, and I'm sure 
there's some overhead in setting the ps display and the other small 
stuff we do once per message. But you're probably right that we could 
easily make MAX_SEND_SIZE much smaller with no noticeable affect on 
performance, while making walsender more responsive to signals. I'll 
decrease it to, say, 512 kB.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread Jesper Krogh

On 2010-05-18 21:56, Bruce Momjian wrote:

Jesper Krogh wrote:
   

On 2010-05-18 20:52, Bruce Momjian wrote:
 

This line above looks very odd because I didn't think the template0
datfrozenxid could be advanced.  Can I see the output of this query:

SELECT datname, datfrozenxid, datallowconn FROM pg_database;


   

Only from the old database:
data=# SELECT datname, datfrozenxid, datallowconn FROM pg_database;
datname  | datfrozenxid | datallowconn
---+--+--
   template0 |   2073823552 | f
   postgres  |   2023820521 | t
   data  |   2023782337 | t
   jk|   2023822188 | t
   template1 |   2073823552 | t
   workqueue |   2023822188 | t
(6 rows)
 

OK, datallowconn = false is right for template0, but I am still confused
how it got set to that high value.
   


This is the production system. I have absolutely no indications that
anything should be wrong in there. It has run rock-solid since it got
migrated (dump/restore) to 8.4 for about 7 months now. So I am a bit
scared about you telling that it seems wrong. (but that cannot be
attributed to pg_upgrade)


OK, thanks.  This does seem odd.  Frankly, having template0's
datfrozenxid be wrong would not cause any kind of instability because
template0 is used only by pg_dump, so I am wondering if something else
is seriously wrong.
   

I also think that something was seriously wrong with the pg_upgrade'd
version. I'll try to reproduce and be a bit more carefull in tracking 
the steps

this time.

--
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Keepalive for max_standby_delay

2010-05-18 Thread Heikki Linnakangas

On 17/05/10 12:36, Jim Nasby wrote:

On May 15, 2010, at 12:05 PM, Heikki Linnakangas wrote:

What exactly is the user trying to monitor? If it's how far behind is
the standby, the difference between pg_current_xlog_insert_location()
in the master and pg_last_xlog_replay_location() in the standby seems
more robust and well-defined to me. It's a measure of XLOG location (ie.
bytes) instead of time, but time is a complicated concept.


I can tell you that end users *will* want a time-based indication of how far behind we 
are. DBAs will understand we're this many transactions behind, but managers 
and end users won't. Unless it's unreasonable to provide that info, we should do so.


No doubt about that, the problem is that it's hard to provide a reliable 
time-based indication.


--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] BYTEA / DBD::Pg change in 9.0 beta

2010-05-18 Thread Alex Hunsaker
On Tue, May 18, 2010 at 14:54, Jesper Krogh jes...@krogh.cc wrote:
 Hi.

 I'm trying to do a test move of one of our applications onto 9.0beta1.
 We use storable and serializes data into a bytea column in the database.
 [ snip insert/select using bytea ]

 8.4
  id |                                       testbytea
 +
  9 |
 \005\007\003\000\000\000\002\012\0012\000\000\000\004that\012\0011\000\000\000\004this
 (1 row)

 9.0beta1
  id |                          testbytea
 +--
  3 | \x05070300020a01320004746861740a0131000474686973
 (1 row)


Try adding $db-do(set bytea_output 'escape';); as 9.0 defaults to
hex encoding.  Id bet DBD::Pg does not account for that yet.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Bug with ordering aggregates?

2010-05-18 Thread Caleb Welton
This is an area that the SQL standard didn't think through very clearly
(IMHO).  They actually have two ways of specifying functions like this, one
is the ordered aggregate section that this syntax is modeled on, which is
indeed very confusing for multi-parameter aggregates.  The other is the
hypothetical set function syntax which is actually much clearer for this
sort of operation, though I haven't dug deep enough into the standard to be
sure this wouldn't include any gotchas:

  SELECT agg(parameter1, parameter2) WITHIN GROUP (ORDER BY column1 asc)

(See section 10.9 on aggregate function syntax)

Supporting the hypthothetical set functions could give a preferable syntax.

Regards,
  Caleb

On 5/18/10 9:42 AM, Stephen Frost sfr...@snowman.net wrote:

 * Tom Lane (t...@sss.pgh.pa.us) wrote:
 Stephen Frost sfr...@snowman.net writes:
   This doesn't seem right to me:
 
 postgres=# select
 postgres-# string_agg(column1::text order by column1 asc,',')
 postgres-# from (values (3),(4),(1),(2)) a;
  string_agg 
 
  1234
 (1 row)
 
 Looks fine to me: you have two ordering columns (the second rather
 useless,  but that's no matter).
 
 Ah, yeah, guess I'll just complain that having the order by look like
 it's an argument to an aggregate makes things confusing.  Not much to be
 done about it though.
 
 Thanks,
 
 Stephen


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_upgrade - link mode and transaction-wraparound data loss

2010-05-18 Thread Bruce Momjian
Jesper Krogh wrote:
 On 2010-05-18 21:56, Bruce Momjian wrote:
  Jesper Krogh wrote:
 
  On 2010-05-18 20:52, Bruce Momjian wrote:
   
  This line above looks very odd because I didn't think the template0
  datfrozenxid could be advanced.  Can I see the output of this query:
 
SELECT datname, datfrozenxid, datallowconn FROM pg_database;
 
 
 
  Only from the old database:
  data=# SELECT datname, datfrozenxid, datallowconn FROM pg_database;
  datname  | datfrozenxid | datallowconn
  ---+--+--
 template0 |   2073823552 | f
 postgres  |   2023820521 | t
 data  |   2023782337 | t
 jk|   2023822188 | t
 template1 |   2073823552 | t
 workqueue |   2023822188 | t
  (6 rows)
   
  OK, datallowconn = false is right for template0, but I am still confused
  how it got set to that high value.
 
 
 This is the production system. I have absolutely no indications that
 anything should be wrong in there. It has run rock-solid since it got
 migrated (dump/restore) to 8.4 for about 7 months now. So I am a bit
 scared about you telling that it seems wrong. (but that cannot be
 attributed to pg_upgrade)

I am on chat with Alvaro now and it seems we do somehow connect to
template0 for transaction id wraparound.  I think Alvaro will post
shortly on this.

  OK, thanks.  This does seem odd.  Frankly, having template0's
  datfrozenxid be wrong would not cause any kind of instability because
  template0 is used only by pg_dump, so I am wondering if something else
  is seriously wrong.
 
 I also think that something was seriously wrong with the pg_upgrade'd
 version. I'll try to reproduce and be a bit more carefull in tracking 
 the steps
 this time.

Thanks, but I think the entire problem might be this template0 xid issue
that Alvaro and I are researching.  I can now see how invalid template0
xids could cause the instability you saw in the new database.  Odd no
one has seen this bug before.

-- 
  Bruce Momjian  br...@momjian.ushttp://momjian.us
  EnterpriseDB http://enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Keepalive for max_standby_delay

2010-05-18 Thread Simon Riggs
On Tue, 2010-05-18 at 17:06 -0400, Heikki Linnakangas wrote:
 On 17/05/10 04:40, Simon Riggs wrote:
  On Sun, 2010-05-16 at 16:53 +0100, Simon Riggs wrote:
 
  Attached patch rearranges the walsender loops slightly to fix the above.
  XLogSend() now only sends up to MAX_SEND_SIZE bytes (== XLOG_SEG_SIZE /
  2) in one round and returns to the main loop after that even if there's
  unsent WAL, and the main loop no longer sleeps if there's unsent WAL.
  That way the main loop gets to respond to signals quickly, and we also
  get to update the shared memory status and PS display more often when
  there's a lot of catching up to do.
 
  Comments
 
  8MB at a time still seems like a large batch to me.
 
  libpq is going to send it in smaller chunks anyway, so I don't see the
  importance of trying to keep the batch too large. It just introduces
  delay into the sending process. We should be sending chunks that matches
  libpq better.
 
  More to the point the logic will fail if XLOG_BLCKSZ  PQ_BUFFER_SIZE
  because it will send partial pages.
 
 I don't see a failure. We rely on not splitting WAL records across 
 messages, but we're talking about libpq-level CopyData messages, not TCP 
 messages.

OK

  Having MAX_SEND_SIZE  PQ_BUFFER_SIZE is pointless, as libpq currently
  stands.
 
 Well, it does affect the size of the read() in walsender, and I'm sure 
 there's some overhead in setting the ps display and the other small 
 stuff we do once per message. But you're probably right that we could 
 easily make MAX_SEND_SIZE much smaller with no noticeable affect on 
 performance, while making walsender more responsive to signals. I'll 
 decrease it to, say, 512 kB.

I'm pretty certain we don't need to set the ps display once per message.
ps doesn't need an update 5 times per second on average.

There's no reason that the buffer size we use for XLogRead() should be
the same as the send buffer, if you're worried about that. My point is
that pq_putmessage contains internal flushes so at the libpq level you
gain nothing by big batches. The read() will be buffered anyway with
readahead so not sure what the issue is. We'll have to do this for sync
rep anyway, so what's the big deal? Just do it now, once. Do we really
want 9.1 code to differ here?

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] BYTEA / DBD::Pg change in 9.0 beta

2010-05-18 Thread Jesper Krogh

On 2010-05-18 23:12, Alex Hunsaker wrote:

set bytea_output 'escape';


That was it. Knowing what the problem was I had no problem finding it in 
the release notes.


May I ask whats the reason is for breaking the compatibillity?

--
Jesper

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Keepalive for max_standby_delay

2010-05-18 Thread Simon Riggs
On Tue, 2010-05-18 at 17:08 -0400, Heikki Linnakangas wrote:
 On 17/05/10 12:36, Jim Nasby wrote:
  On May 15, 2010, at 12:05 PM, Heikki Linnakangas wrote:
  What exactly is the user trying to monitor? If it's how far behind is
  the standby, the difference between pg_current_xlog_insert_location()
  in the master and pg_last_xlog_replay_location() in the standby seems
  more robust and well-defined to me. It's a measure of XLOG location (ie.
  bytes) instead of time, but time is a complicated concept.
 
  I can tell you that end users *will* want a time-based indication of how 
  far behind we are. DBAs will understand we're this many transactions 
  behind, but managers and end users won't. Unless it's unreasonable to 
  provide that info, we should do so.
 
 No doubt about that, the problem is that it's hard to provide a reliable 
 time-based indication.

I think I have one now.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Keepalive for max_standby_delay

2010-05-18 Thread Heikki Linnakangas

On 18/05/10 17:17, Simon Riggs wrote:

There's no reason that the buffer size we use for XLogRead() should be
the same as the send buffer, if you're worried about that. My point is
that pq_putmessage contains internal flushes so at the libpq level you
gain nothing by big batches. The read() will be buffered anyway with
readahead so not sure what the issue is. We'll have to do this for sync
rep anyway, so what's the big deal? Just do it now, once. Do we really
want 9.1 code to differ here?


Do what? What exactly is it that you want instead, then?

--
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] BYTEA / DBD::Pg change in 9.0 beta

2010-05-18 Thread Alex Hunsaker
On Tue, May 18, 2010 at 15:20, Jesper Krogh jes...@krogh.cc wrote:
 On 2010-05-18 23:12, Alex Hunsaker wrote:

 set bytea_output 'escape';

 That was it. Knowing what the problem was I had no problem finding it in the
 release notes.

 May I ask whats the reason is for breaking the compatibillity?

There were a couple IIRC, the big ones being speed and size.  Id look
at the archives for more.

I imagine at some point DBD::Pg will handle this transparently.  I
also imagine Greg would happily accept patches :-)

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] BYTEA / DBD::Pg change in 9.0 beta

2010-05-18 Thread Bernd Helmle



--On 18. Mai 2010 23:20:26 +0200 Jesper Krogh jes...@krogh.cc wrote:


That was it. Knowing what the problem was I had no problem finding it in
the release notes.

May I ask whats the reason is for breaking the compatibillity?


Efficency, if i am allowed to call it this way. The new hex 
representation should be more efficient to retrieve and to handle than the 
old one. I think bytea_output was set to hex for testing purposes on the 
first hand, but not sure wether there was a consensus to leave it there 
finally later.


--
Thanks

Bernd

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] BYTEA / DBD::Pg change in 9.0 beta

2010-05-18 Thread Kenneth Marshall
On Tue, May 18, 2010 at 03:26:17PM -0600, Alex Hunsaker wrote:
 On Tue, May 18, 2010 at 15:20, Jesper Krogh jes...@krogh.cc wrote:
  On 2010-05-18 23:12, Alex Hunsaker wrote:
 
  set bytea_output 'escape';
 
  That was it. Knowing what the problem was I had no problem finding it in the
  release notes.
 
  May I ask whats the reason is for breaking the compatibillity?
 
 There were a couple IIRC, the big ones being speed and size.  Id look
 at the archives for more.
 
 I imagine at some point DBD::Pg will handle this transparently.  I
 also imagine Greg would happily accept patches :-)
 

Yes, the new format is much faster, more space efficient, and uses
less CPU to do the encoding. The older format caused the COPY for
bytea to be CPU limited in many more situations.

Regards,
Ken

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Keepalive for max_standby_delay

2010-05-18 Thread Simon Riggs
On Tue, 2010-05-18 at 17:25 -0400, Heikki Linnakangas wrote:
 On 18/05/10 17:17, Simon Riggs wrote:
  There's no reason that the buffer size we use for XLogRead() should be
  the same as the send buffer, if you're worried about that. My point is
  that pq_putmessage contains internal flushes so at the libpq level you
  gain nothing by big batches. The read() will be buffered anyway with
  readahead so not sure what the issue is. We'll have to do this for sync
  rep anyway, so what's the big deal? Just do it now, once. Do we really
  want 9.1 code to differ here?
 
 Do what? What exactly is it that you want instead, then?

Read and write smaller messages, so the latency is minimised. Libpq will
send in 8192 byte packets, so writing anything larger gains nothing when
the WAL data is also chunked at exactly the same size.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] ecmascript 5 DATESTYLE

2010-05-18 Thread Ben Hockey

hi,

i mistakenly had posted this to pgsql-bugs already and got a response  
(see below - edited).  i'm posting here since afaik it is the way i  
should be requesting new features.  my suggestion is to add a  
DATESTYLE format to match the format specified for date time strings  
in ecmascript 5.


the following is from the ecmascript 5 specification at http://www.ecmascript.org/docs/tc39-2009-043.pdf 
 page 168:



15.9.1.15 Date Time String Format
ECMAScript defines a string interchange format for date-times based  
upon a simplification of the ISO 8601

Extended Format.  The format is as follows: -MM-DDTHH:mm:ss.sssZ


ecmascript 5 is the most recent specification for JavaScript and i  
would think that having a DATESTYLE format to simplify  
interoperability with JavaScript applications would be highly desirable.


thanks,

ben...

On May 16, 2010, at 2:22 AM, Pavel Stehule wrote:

I have nothing against some new datestyles - xml, ecma5 and I am able
to add to pg when hackers will agree





Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle

2010-05-18 Thread Florian Pflug
On May 17, 2010, at 3:30 , Robert Haas wrote:
 On Sun, May 16, 2010 at 9:07 PM, Florian Pflug f...@phlo.org wrote:
 On May 14, 2010, at 22:54 , Robert Haas wrote:
 On Thu, May 13, 2010 at 5:39 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Florian Pflug f...@phlo.org writes:
 All in all, I believe that SHARE and UPDATE row-level locks should be
 changed to cause concurrent UPDATEs to fail with a serialization
 error.
 
 I don't see an argument for doing that for FOR SHARE locks, and it
 already happens for FOR UPDATE (at least if the row actually gets
 updated).  AFAICS this proposal mainly breaks things, in pursuit of
 an unnecessary and probably-impossible-anyway goal of making FK locking
 work with only user-level snapshots.
 
 After giving this considerable thought and testing the behavior at
 some length, I think the OP has it right.  One thing I sometimes need
 to do is denormalize a copy of a field, e.g.
 
 snipped example
 
 I've whipped up a quick and still rather dirty patch that implements the 
 behavior I proposed, at least for the case of conflicts between FOR UPDATE 
 locks and updates. With the patch, any attempt to UPDATE or FOR UPDATE lock 
 a row that has concurrently been FOR UPDATE locked will cause a 
 serialization error. (The same for an actually updated row of course, but 
 that happened before too).
 
 While this part of the patch was fairly straight forward, make FOR SHARE 
 conflict too seems to be much harder. The assumption that a lock becomes 
 irrelevant after the transaction(s) that held it completely is built deeply 
 into the multi xact machinery that powers SHARE locks. That machinery 
 therefore assumes that once all members of a multi xact have completed the 
 multi xact is dead also. But my proposal depends on a SERIALIZABLE 
 transaction being able to find if any of the lockers of a row are invisible 
 under it's snapshot - for which it'd need any multi xact containing 
 invisible xids to outlive its snapshot.
 
 Thanks for putting this together. I suggest adding it to the open
 CommitFest - even if we decide to go forward with this, I don't
 imagine anyone is going to be excited about changing it during beta.
 
 https://commitfest.postgresql.org/action/commitfest_view/open


Will do. Thanks for the link.

Here is an updated version that works for SHARE locks too.

(This message mainly serves as a way to link the updated patch to the commit 
fest entry. Is this how I'm supposed to do that, or am I doing something wrong?)

best regards,
Florian Pflug


serializable_lock_consistency.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Fujii Masao
On Tue, May 18, 2010 at 8:35 PM, Robert Haas robertmh...@gmail.com wrote:
 On Mon, May 17, 2010 at 10:40 PM, Fujii Masao masao.fu...@gmail.com wrote:
 On Mon, May 17, 2010 at 10:20 PM, Robert Haas robertmh...@gmail.com wrote:
 OK, I think I understand now.  But, the SIGTERM sent by the postmaster
 doesn't kill the recovery process unconditionally.  It will invoke
 StartupProcShutdownHandler(), which will set set shutdown_requested =
 true.  That gets checked by RestoreArchivedFile() and
 HandleStartupProcInterrupts(), and I think that neither of those can
 get invoked until after the control file has been updated.  Do you see
 a way it can happen?

 Yeah, the way is:
 StartupXLOG() -- ReadCheckpointRecord() -- ReadRecord() --
 XLogPageRead() -- XLogFileReadAnyTLI() -- XLogFileRead() --
 RestoreArchivedFile()

 ReadCheckpointRecord() is called before pg_control is updated.

 OK.  In that case, I'm wondering if we should reverse course and
 rejigger the logic so that the shutdown gets processed when we
 transition to PM_RECOVERY.  Seems like that might be simpler.

You mean keeping shutdown waiting until the postmaster has reached
PM_RECOVERY, i.e., the startup process has sent PMSIGNAL_RECOVERY_STARTED?

The startup process must call ReadCheckpointRecord() before sending
that signal. ReadCheckpointRecord() might get stuck for some reasons,
e.g., neither master nor standby might have the recovery starting
checkpoint WAL record. So that signal might not be sent forever,
in this case, shutdown would get stuck.

 ISTM that walreceiver might be invoked even after shutdown is requested.
 We should prevent the postmaster from starting up walreceiver if
 Shutdown  NoShutdown?

 Well, when we did the previous shutdown patch, we decided it was not
 right to kill walreceiver until all backends had exited, so it seems
 inconsistent to make the opposite decision here.

Oh, right. How about allowing the postmaster only in PM_STARTUP,
PM_RECOVERY, PM_HOT_STANDBY or PM_WAIT_READONLY state to invoke
walreceiver? We can keep walreceiver alive until all read only
backends have gone, and prevent unexpected startup of walreceiver.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Robert Haas
On Tue, May 18, 2010 at 10:40 PM, Fujii Masao masao.fu...@gmail.com wrote:
 On Tue, May 18, 2010 at 8:35 PM, Robert Haas robertmh...@gmail.com wrote:
 On Mon, May 17, 2010 at 10:40 PM, Fujii Masao masao.fu...@gmail.com wrote:
 On Mon, May 17, 2010 at 10:20 PM, Robert Haas robertmh...@gmail.com wrote:
 OK, I think I understand now.  But, the SIGTERM sent by the postmaster
 doesn't kill the recovery process unconditionally.  It will invoke
 StartupProcShutdownHandler(), which will set set shutdown_requested =
 true.  That gets checked by RestoreArchivedFile() and
 HandleStartupProcInterrupts(), and I think that neither of those can
 get invoked until after the control file has been updated.  Do you see
 a way it can happen?

 Yeah, the way is:
 StartupXLOG() -- ReadCheckpointRecord() -- ReadRecord() --
 XLogPageRead() -- XLogFileReadAnyTLI() -- XLogFileRead() --
 RestoreArchivedFile()

 ReadCheckpointRecord() is called before pg_control is updated.

 OK.  In that case, I'm wondering if we should reverse course and
 rejigger the logic so that the shutdown gets processed when we
 transition to PM_RECOVERY.  Seems like that might be simpler.

 You mean keeping shutdown waiting until the postmaster has reached
 PM_RECOVERY, i.e., the startup process has sent PMSIGNAL_RECOVERY_STARTED?

 The startup process must call ReadCheckpointRecord() before sending
 that signal. ReadCheckpointRecord() might get stuck for some reasons,
 e.g., neither master nor standby might have the recovery starting
 checkpoint WAL record. So that signal might not be sent forever,
 in this case, shutdown would get stuck.

Ah, OK.

In terms of removing the backup label file, can we simply have an
additional boolean in the postmaster that indicates whether we've ever
reached PM_RUN, and only consider removing the backup file if so?

 ISTM that walreceiver might be invoked even after shutdown is requested.
 We should prevent the postmaster from starting up walreceiver if
 Shutdown  NoShutdown?

 Well, when we did the previous shutdown patch, we decided it was not
 right to kill walreceiver until all backends had exited, so it seems
 inconsistent to make the opposite decision here.

 Oh, right. How about allowing the postmaster only in PM_STARTUP,
 PM_RECOVERY, PM_HOT_STANDBY or PM_WAIT_READONLY state to invoke
 walreceiver? We can keep walreceiver alive until all read only
 backends have gone, and prevent unexpected startup of walreceiver.

Yes, that seems like something we should be checking, if we aren't already.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Row-level Locks SERIALIZABLE transactions, postgres vs. Oracle

2010-05-18 Thread Robert Haas
On Tue, May 18, 2010 at 8:15 PM, Florian Pflug f...@phlo.org wrote:
 Will do. Thanks for the link.

 Here is an updated version that works for SHARE locks too.

 (This message mainly serves as a way to link the updated patch to the commit 
 fest entry. Is this how I'm supposed to do that, or am I doing something 
 wrong?)

Yeah - just go to the existing CF entry and say New Comment then
select type Patch.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ecmascript 5 DATESTYLE

2010-05-18 Thread Robert Haas
On Tue, May 18, 2010 at 6:26 PM, Ben Hockey neonstalw...@gmail.com wrote:
 hi,
 i mistakenly had posted this to pgsql-bugs already and got a response (see
 below - edited).  i'm posting here since afaik it is the way i should be
 requesting new features.  my suggestion is to add a DATESTYLE format to
 match the format specified for date time strings in ecmascript 5.
 the following is from the ecmascript 5 specification
 at http://www.ecmascript.org/docs/tc39-2009-043.pdf page 168:

 15.9.1.15 Date Time String Format

 ECMAScript defines a string interchange format for date-times based upon a
 simplification of the ISO 8601

 Extended Format.  The format is as follows: -MM-DDTHH:mm:ss.sssZ

 ecmascript 5 is the most recent specification for JavaScript and i would
 think that having a DATESTYLE format to simplify interoperability with
 JavaScript applications would be highly desirable.

I don't object, if someone wants to write a patch.  I guess the
question is whether to keep adding named formats, or try to create a
general mechanism to allow the user to specify an arbitrary format, as
we do with to_char().

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ecmascript 5 DATESTYLE

2010-05-18 Thread Pavel Stehule
2010/5/19 Robert Haas robertmh...@gmail.com:
 On Tue, May 18, 2010 at 6:26 PM, Ben Hockey neonstalw...@gmail.com wrote:
 hi,
 i mistakenly had posted this to pgsql-bugs already and got a response (see
 below - edited).  i'm posting here since afaik it is the way i should be
 requesting new features.  my suggestion is to add a DATESTYLE format to
 match the format specified for date time strings in ecmascript 5.
 the following is from the ecmascript 5 specification
 at http://www.ecmascript.org/docs/tc39-2009-043.pdf page 168:

 15.9.1.15 Date Time String Format

 ECMAScript defines a string interchange format for date-times based upon a
 simplification of the ISO 8601

 Extended Format.  The format is as follows: -MM-DDTHH:mm:ss.sssZ

 ecmascript 5 is the most recent specification for JavaScript and i would
 think that having a DATESTYLE format to simplify interoperability with
 JavaScript applications would be highly desirable.

 I don't object, if someone wants to write a patch.  I guess the
 question is whether to keep adding named formats, or try to create a
 general mechanism to allow the user to specify an arbitrary format, as
 we do with to_char().


I can write patch. I am against to general solution - It can be new
way for SQL injection.

Regards

Pavel

 --
 Robert Haas
 EnterpriseDB: http://www.enterprisedb.com
 The Enterprise Postgres Company

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Stefan's bug (was: max_standby_delay considered harmful)

2010-05-18 Thread Fujii Masao
On Wed, May 19, 2010 at 12:59 PM, Robert Haas robertmh...@gmail.com wrote:
 In terms of removing the backup label file, can we simply have an
 additional boolean in the postmaster that indicates whether we've ever
 reached PM_RUN, and only consider removing the backup file if so?

Yes, but I prefer XLogCtl-SharedRecoveryInProgress, which is the almost
same indicator as the boolean you suggested. Thought?

 ISTM that walreceiver might be invoked even after shutdown is requested.
 We should prevent the postmaster from starting up walreceiver if
 Shutdown  NoShutdown?

 Well, when we did the previous shutdown patch, we decided it was not
 right to kill walreceiver until all backends had exited, so it seems
 inconsistent to make the opposite decision here.

 Oh, right. How about allowing the postmaster only in PM_STARTUP,
 PM_RECOVERY, PM_HOT_STANDBY or PM_WAIT_READONLY state to invoke
 walreceiver? We can keep walreceiver alive until all read only
 backends have gone, and prevent unexpected startup of walreceiver.

 Yes, that seems like something we should be checking, if we aren't already.

I'll do that.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers