Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Fujii Masao
Hi,

On Thu, Nov 12, 2009 at 4:32 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
 Fujii Masao wrote:
 The problem is that fsync needs to be issued too frequently, which would
 be harmless in asynchronous replication, but not in synchronous one.
 A transaction would have to wait for the primary's and standby's fsync
 before returning a success to a client.

 So I'm inclined to change the startup process and bgwriter, instead of
 walreceiver, so as to fsync the WAL for the WAL rule.

 Let's keep it simple for now. Just make the walreceiver do the fsync. We
 can optimize later. For now, we're only going to have async mode anyway.

Okey, I'll do that; the walreceiver issues the fsync for each arrival of
the WAL records, and the startup process replays only the records already
fsynced.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Simon Riggs
On Thu, 2009-11-12 at 17:03 +0900, Fujii Masao wrote:

 On Thu, Nov 12, 2009 at 4:32 PM, Heikki Linnakangas
 heikki.linnakan...@enterprisedb.com wrote:
  Fujii Masao wrote:
  The problem is that fsync needs to be issued too frequently, which would
  be harmless in asynchronous replication, but not in synchronous one.
  A transaction would have to wait for the primary's and standby's fsync
  before returning a success to a client.
 
  So I'm inclined to change the startup process and bgwriter, instead of
  walreceiver, so as to fsync the WAL for the WAL rule.
 
  Let's keep it simple for now. Just make the walreceiver do the fsync. We
  can optimize later. For now, we're only going to have async mode anyway.
 
 Okey, I'll do that; the walreceiver issues the fsync for each arrival of
 the WAL records, and the startup process replays only the records already
 fsynced.

I agree with you, though it has taken some time to understand what you
said and at first my reaction was to disagree. I think the responses you
got on this are because you dived straight in with a question before
explaining other things around this.

We already have a number of options for how to handle incoming WAL. We
can choose to fsync or not when WAL arrives. Choosing *not* to fsync
would be the typical choice because it provides reasonable performance;
fsyncing after each transaction commit would be worse. In any case, if
WAL receiver does the fsyncs then we will get worse performance. If we
reduce the number of fsyncs it does we just get spiky behaviour around
the fsyncs.

If recovery starts reading WAL records that have not been fsynced then
we may need to flush a shared buffer to disk that depends upon a
non-fsynced(yet) WAL record. Fsyncing WAL after *every* WAL record is
going to make performance suck even worse and is completely out of the
question. So implementing the fsync-WAL-before-buffer-flush rule during
recovery makes much more sense. It's also only small change during
XlogFlush().

Another way of doing this would be to only allow recovery to progress as
far as has been fsynced. That seems a more plausible approach, but would
lead to delays if we had a small number of long write transactions. The
benefit of streaming is that it potentially allows us to keep as near to
real-time recovery as possible.

So overall, yes, we need to do as you suggested: implement WAL rule in
recovery. WALreceiver smoothly does write(), Startup replays and we
leave the WAL file fsyncs to be performed by the bgwriter. 

But I also agree with Heikki. Let's plan to do this later in this
release.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] New VACUUM FULL

2009-11-12 Thread Simon Riggs
On Tue, 2009-10-27 at 13:55 +0900, Itagaki Takahiro wrote:
 Here is a patch to support rewrite version of VACUUM FULL.
 It was called VACUUM REWRITE in the past disucussin,
 but I choose the following syntax for now:
 
 VACUUM ( FULL [ INPLACE | REPLACE ] )  [ table_name ]
 
 The reason is to contrast the new REPLACE behavior with the old INPLACE
 behavior. I cannot find any good terms of opposite meaning of REWRITE.
 
 The new version works like as CLUSTER USING ctid or rewriting in
 ALTER TABLE. It must be faster than them if we have many dead tuples
 and are not interested in physical order of tuples.
 
 We still need traditional VACUUM FULL behavior for system catalog because
 we cannot change relfilenode for them. Also, VACUUM FULL REPLACE is not
 always better than traditional VACUUM FULL; the new version requires
 additional disk space and might be slower if we have a few dead tuples.
 
 VACUUM FULL and VACUUM (FULL) are synonyms for VACUUM (FULL REPLACE),
 but if the target table is s system catalog, instead FULL INPLACE is
 used automatically.

 If this approach is reasonable, I'll go for other supplemental parts.
 (documentations, vacuumdb, etc.)

Rough approach looks fine to me.

The internal logic is fairly hard to read. I'd suggest you have option
flags VACUUM_FULL and VACUUM_REPLACE, rather than VACUUM_INPLACE and
VACUUM_REPLACE. So VACUUM_REPLACE can only be set iff VACUUM_FULL. That
will reduce much of the logic.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Simon Riggs
On Sun, 2009-11-08 at 20:52 -0500, Robert Haas wrote:

 I would personally prefer not to be involved in the management of the
 next CommitFest.  Having done all of the July CommitFest and a good
 chunk of the September CommitFest, I am feeling a bit burned out.

You did a grand job and everybody appreciates it. 

 I feel like Simon Riggs and
 Fujii Masao really pulled out all the stops to get these ready in time
 for the September CommitFest, and while I'm not in a hurry to break
 the world, I think the sooner these can hit the tree, the better of
 we'll be in terms of releasing 8.5.

Sprinting is hard and we all need to rest afterwards.

How about we just slow the pace down a little? Nobody wants you to quit,
we just need to set a sustainable pace.

Looking at the submissions so far, it seems you've done such a grand job
at clearing the backlog that there are few patches in the next fest.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] CommitFest 2009-09, two weeks on

2009-11-12 Thread Boszormenyi Zoltan
Alvaro Herrera írta:
 Boszormenyi Zoltan escribió:

   
 I have split up (and cleaned up a little) the dynamic
 cursorname patch into smaller logical, easier-to-review
 pieces. Descriptions below.

 1) 1a-unified-optfromin-fetch-ctxdiff.patch

 ecpg supports optional FROM/IN in FETCH and
 MOVE statements (mainly because it's required by
 Informix-compatibility). Unify core and ecpg grammar
 as per Tom Lane's suggestion.
 

 I have applied this patch after some tinkering.  I mainly added support
 for fetch_args: FORWARD opt_from_in name and BACKWARD opt_from_in
 name in ecpg.addons which apparently you forgot.
   

Thanks. Your fix is correct if this patch is considered
standalone. This means I have to re-post the later
patches to fix the reject this fix causes in them.

Best regards,
Zoltán Böszörményi

-- 
Bible has answers for everything. Proof:
But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil. (Matthew 5:37) - basics of digital technology.
May your kingdom come - superficial description of plate tectonics

--
Zoltán Böszörményi
Cybertec Schönig  Schönig GmbH
http://www.postgresql.at/


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] CommitFest 2009-09, two weeks on

2009-11-12 Thread Boszormenyi Zoltan
Alvaro Herrera írta:
 Boszormenyi Zoltan escribió:

   
 2) 1b-cursor_name-ctxdiff.patch

 name - cursor_name transition in core grammar
 and ecpg grammar. Currently it does nothing, it's a
 preparation phase. Depends on patch 1.
 

 Applied this part too.

 I cannot apply the other ones -- they belong to the ECPG maintainer.  I
 hope I cleared his road a bit.
   

Thanks and best regards,
Zoltán Böszörményi

-- 
Bible has answers for everything. Proof:
But let your communication be, Yea, yea; Nay, nay: for whatsoever is more
than these cometh of evil. (Matthew 5:37) - basics of digital technology.
May your kingdom come - superficial description of plate tectonics

--
Zoltán Böszörményi
Cybertec Schönig  Schönig GmbH
http://www.postgresql.at/


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Joachim Wieland
On Thu, Nov 12, 2009 at 2:12 AM, Andrew Gierth
and...@tao11.riddles.org.uk wrote:
 Does it cope with the case where a trigger is doing NOTIFY, and you do
 a whole-table update, therefore dumping potentially millions of
 notifications in at once?

 (for example a rare maintenance operation on a table which has a
 listen/notify arrangement triggered by single inserts or updates)

 The existing implementation copes with that just fine.

As long as you use the existing way to send out notifications by just
sending NOTIFY foo, then yes, it will cope with millions of them
just fine because it will collapse them into one notification as does
the current implementation (contrary to the current implementation a
notification will be received from every transaction that has sent one
- the current implementation even collapses notifications from
different transactions).

However if you decide to use the new syntax and add a distinct payload
to every NOTIFY, then you also send out millions of notifications...

With the proposed patch the queue has space for up to about 81,787,680
notifications. The limits are mainly imposed by slru.c which defines
page size, pages per segment and the number of segments available. We
could easily bump that up to a multiple of the current limits if we
have decided that we need to...


Joachim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch committers

2009-11-12 Thread Dimitri Fontaine
Bruce Momjian br...@momjian.us writes:

 Peter Eisentraut wrote:
 discernible benefits.  But you can't expect a lot of people or employers
 to reserve time on top of that for handholding other people's patches
 and for other community stuff that has no easy to measure benefits.

 Totally agree.  It is that zero-return work that is hard to justify for
 people and companies.  It is clearly something that requires
 self-sacrifice, and personally I think a culture of self-sacrifice is
 what has given us such great success and such a nuturing community
 environment.

I don't know what you're talking about here, or refering to.

The easy way to explain Open Source contributing to PHB and finance
people, in my mind, is the following: Open Source is not free as in free
beer. At all. Rather than paying licences costs, what you have to pay
for is your employees salary when they take the time to participate into
this community things. And if you're using Open Source Software, you
*need* employees that take part of the community processes.

That also mean you have real experts now, and ones able to drive the
project in a direction that will be to your profit as an employer. For
example, this hairy bug in your production environment will get fixed
the day you report about it, rather that when the licencing company
think it matches their roadmap, if ever.

Oh and some other niceties about it. Let's try the story telling
style... When I worked on preprepare, I though we had an urgent need
here to run it. It happened that we had yet more urgent to do, so I
barely had time to have it working and commit it on some pgfoundry
CVS. Less than a week after, I'm told on IRC that the product is running
fine in a production environment somewhere, taking real load. Now I
don't have to make tests before deploying. That has been done for
me. That's Open Source.

Regards,
-- 
dim

Is it 2009 and we're still having this discussion about how
participating into Open Source communities is a good way to spend your
money as an employer? Wow.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Parsing config files in a directory

2009-11-12 Thread Dimitri Fontaine
Hi,

Josh Berkus j...@agliodbs.com writes:
 Let's NOT start that discussion again.

Don't worry, no aim here to.

 Overall, I'm seeing this patch proposal suffer from an extreme excess of
 bike-shedding.

Not only that. See above.

 My proposal is this:

 (1) that we support the idea of a patch which allows people to add
 directory includes to postgresql.conf, in the same manner that we now
 support file includes, with files in the included directory to be
 processed alphanumerically.

+1

 (2) that we put out a TODO for making the configuration variables which
 take lists able to take an accumulator as well as an assignment, syntax
 TBA.

+1

 These two above seem like nice, small incremental changes to 8.5 which
 require no sweeping redesigns of how people handle conf files, but do
 allow people who want to develop new management strategies to do so.

+1

What made us have this long a thread is the premise that having the
include directory facility will make any good to solving the problem of
editing the configuration from a program. So everyone tried either to
explain how it helps or solves it, or to explan how it does not help at
all. My position is the later, as I hope to have made clear before.

But you're very right when saying that this facility is worth it
independantly of how much it helps solving the programatic API to
configuration.

Regards,
-- 
dim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Joachim Wieland
On Thu, Nov 12, 2009 at 4:25 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 One possible solution would be to write to the queue before committing
 and adding the TransactionID.  Then other backends can check if our
 TransactionID has successfully committed or not. Not sure if this is
 worth the overhead however...

 That sounds reasonable, and it's certainly no more overhead than the
 tuple visibility checks that happen in the current implementation.

I am not too concerned about the runtime of the visibility checks,
instead I suppose that most of the overhead will come from waiting for
another transaction to either commit or abort...

If transaction t1 scans the queue and at some point finds
notifications from t2, then it will ask for the status of t2. If it
finds out that t2 is still running, then it has no other option than
to stop working on the queue and wait (it will be signalled again
later once t2 has finished).

This also means that we cannot at the same time write notifications to
the queue and read from it and if a transaction places a few million
notifications into the queue, readers need to wait until it has
finished and only after that they can continue and read the
notifications...

And it means that if the queue is full, we might run into a
deadlock... A transaction adding notifications will wait for the
readers to proceed and the readers wait for the transaction to commit
or abort...

One option could be to write new notifications to a subdirectory, and
create a bunch of new segment files there. Once this is done, the
segment files could be moved over and renamed, so that they continue
the slru queue... If we run out of disk space while filling that
temporary subdirectory, then we can just delete the subdirectory and
nobody has been blocked. We could still run into errors moving and
renaming the segment files (e.g. permission problems) so that we still
might need to abort the transaction...


 2. The payload parameter is optional. A notifying client can either call
 NOTIFY foo; or NOTIFY foo 'payload';. The length of the payload is
 currently limited to 128 characters... Not sure if we should allow longer
 payload strings...

 Might be a good idea to make the max the same as the max length for
 prepared transaction GUIDs?  Not sure anyone would be shipping those
 around, but it's a pre-existing limit of about the same size.

Yes, sounds reasonable to have the same limit for user-defined identifiers...


Joachim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 5:04 AM, Simon Riggs si...@2ndquadrant.com wrote:
 On Sun, 2009-11-08 at 20:52 -0500, Robert Haas wrote:

 I would personally prefer not to be involved in the management of the
 next CommitFest.  Having done all of the July CommitFest and a good
 chunk of the September CommitFest, I am feeling a bit burned out.

 You did a grand job and everybody appreciates it.

Thanks.

 I feel like Simon Riggs and
 Fujii Masao really pulled out all the stops to get these ready in time
 for the September CommitFest, and while I'm not in a hurry to break
 the world, I think the sooner these can hit the tree, the better of
 we'll be in terms of releasing 8.5.

 Sprinting is hard and we all need to rest afterwards.

 How about we just slow the pace down a little? Nobody wants you to quit,
 we just need to set a sustainable pace.

I'm not sure exactly what point you're aiming at here, so I'll respond
with a few thoughts that may or may not pertain.

I think it would be really, really good if we could make this release
come out on the schedule previously discussed.  8.4 slipped quite a
bit for reasons that were, IMHO, quite preventable: and, worse, it's
not clear that the slippage really bought us anything, because we
still ended up with a bunch of embarassing bugs.  Having said that,
I'm not capable of single-handedly effecting an on-time release, and I
don't particularly want to.  In a community where people can disappear
or change roles in the snap of a finger, it's bad to be relying on any
one person to do too much.  We need larger, more robust pools of
committers, reviewers, commitfest managers, etc.

Perhaps for next release we should consider spacing the CommitFests
out a little more.  I think one CommitFest every 2 months is a little
too tight a schedule.  As Peter and others have mentioned previously,
it doesn't leave a lot of time to work on your own patches (behold the
lack of any of my patches in this CommitFest).  I think a CommitFest
every 3 months would be too long, but maybe something in the middle.
The trick is to navigate around major holidays while ending around the
right time.  Possibly the amount of time between CommitFests doesn't
even need to be constant throughout the release cycle - maybe shorter
at the beginning and longer towards the end.

 Looking at the submissions so far, it seems you've done such a grand job
 at clearing the backlog that there are few patches in the next fest.

Thanks for your kind words.  It does seem that most of the major
patches we've seen so far landed last CommitFest, with the exception
of HS and SR.  That's not all me, of course - among other people, Tom
did a tremendous amount of work - but I'm glad I was able to help move
it along.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Andres Freund
Hi,

On Thursday 12 November 2009 12:46:46 Robert Haas wrote:
 Perhaps for next release we should consider spacing the CommitFests
 out a little more. 
That may lead to quite a bit frustration on the contributor side though. It 
can be very frustrating to have no input for a even longer timeframe...

Andres

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 6:49 AM, Andres Freund and...@anarazel.de wrote:
 On Thursday 12 November 2009 12:46:46 Robert Haas wrote:
 Perhaps for next release we should consider spacing the CommitFests
 out a little more.
 That may lead to quite a bit frustration on the contributor side though. It
 can be very frustrating to have no input for a even longer timeframe...

Yep, that's the flip side.  But you'll notice I was suggesting
extending it by a pretty small amount.  Anyway, it was just a thought.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Fujii Masao
On Thu, Nov 12, 2009 at 6:27 PM, Simon Riggs si...@2ndquadrant.com wrote:
 I agree with you, though it has taken some time to understand what you
 said and at first my reaction was to disagree. I think the responses you
 got on this are because you dived straight in with a question before
 explaining other things around this.

Thanks for clarifying this topic ;)

 If recovery starts reading WAL records that have not been fsynced then
 we may need to flush a shared buffer to disk that depends upon a
 non-fsynced(yet) WAL record. Fsyncing WAL after *every* WAL record is
 going to make performance suck even worse and is completely out of the
 question. So implementing the fsync-WAL-before-buffer-flush rule during
 recovery makes much more sense. It's also only small change during
 XlogFlush().

Agreed. This approach has lesser impact on the performance.

But, as I said on my first post on this thread, even such low-frequent
fsync-WAL-before-buffer-flush might cause a response time spike on the
primary because the walreceiver must sleep during that fsync. I think
that leaving the WAL-logging business to another process like walwriter
is a good idea for reducing further the impact on the walreceiver; In
typical case,

* The walreceiver receives WAL records, returns the ACK to the primary,
  saves them in the wal_buffers, and lets the startup process know
  the arrival.

* The walwriter writes and fsyncs the WAL records in the wal_buffers.

* The startup process applies the WAL records in the wal_buffers
  when it receives the notice of the arrival.

* The startup process and bgwriter fsyncs the WAL before the buffer
  flush.

Of course, since this approach is too complicated, it's out of the scope
of the development for v8.5.

 But I also agree with Heikki. Let's plan to do this later in this
 release.

Okey. I implement nothing around this topic until the core part of
asynchronous replication will have been committed.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Simon Riggs
On Thu, 2009-11-12 at 21:45 +0900, Fujii Masao wrote:

 But, as I said on my first post on this thread, even such low-frequent
 fsync-WAL-before-buffer-flush might cause a response time spike on the
 primary because the walreceiver must sleep during that fsync. I think
 that leaving the WAL-logging business to another process like walwriter
 is a good idea for reducing further the impact on the walreceiver; In
 typical case,

Agree completely.

 Of course, since this approach is too complicated, it's out of the scope
 of the development for v8.5.

It's out of scope for phase 1, certainly.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] New VACUUM FULL

2009-11-12 Thread Alvaro Herrera
Itagaki Takahiro wrote:
 Here is a patch to support rewrite version of VACUUM FULL.
 It was called VACUUM REWRITE in the past disucussin,
 but I choose the following syntax for now:
 
 VACUUM ( FULL [ INPLACE | REPLACE ] )  [ table_name ]
 
 The reason is to contrast the new REPLACE behavior with the old INPLACE
 behavior. I cannot find any good terms of opposite meaning of REWRITE.

I thought the idea is to rip out the current implementation altogether.
If that's the case, then it doesn't make sense to use a different
syntax.  Just rip the innards of VACUUM FULL and replace them with your
new implementation.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] New VACUUM FULL

2009-11-12 Thread Alvaro Herrera
Itagaki Takahiro wrote:

 We still need traditional VACUUM FULL behavior for system catalog because
 we cannot change relfilenode for them. Also, VACUUM FULL REPLACE is not
 always better than traditional VACUUM FULL; the new version requires
 additional disk space and might be slower if we have a few dead tuples.

Tom was saying that we could fix the problem that relfilenode could not
be changed by having a flat file filenode map.  It would only be needed
for nailed system catalogs (the rest of the tables grab their
relfilenode from pg_class as usual) so it wouldn't have the problems
that the previous flatfiles had (which was that they could grow too
much).  I don't recall if this got implemented (I don't think it did).

As for it being slower with few dead tuples, I don't think this is a
problem -- just use lazy vacuum in that case.

I also remember we agreed on something about the need for extra disk
space, but I can't remember what it was.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] New VACUUM FULL

2009-11-12 Thread Alvaro Herrera

BTW I think the vacstmt.options change, which seems a reasonable idea to
me, could be extracted from the patch and applied separately.  That'd
reduce the size of the patch somewhat.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Andrew Chernow



2. The payload parameter is optional. A notifying client can either call
NOTIFY foo; or NOTIFY foo 'payload';. The length of the payload is
currently limited to 128 characters... Not sure if we should allow longer
payload strings...

Might be a good idea to make the max the same as the max length for
prepared transaction GUIDs?  Not sure anyone would be shipping those
around, but it's a pre-existing limit of about the same size.


Yes, sounds reasonable to have the same limit for user-defined identifiers...



[..begging..] Can this be increased significantly?  I don't get it, is there any 
technical reason to make the limit soo small?  This drastically reduces the 
usefulness of the payload.  I've wanted this feature for quite sometime and it 
is quite disappointing that I could not even use it because it is unjustifiably 
limited.


One use case I need is making the payload an absolute path, which saves us a 
round trip (commonly internet latency) and a query in a section of the system 
that's extremely performance sensitive.  That sure ain't going to fit in 128 
bytes.


I'm sure I'm not the only one who finds this limit too small.  I can almost 
guarentee complaints would come in if released that way.


--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] not logging caught exceptions

2009-11-12 Thread Peter Eisentraut
On ons, 2009-11-11 at 19:45 -0500, Tom Lane wrote:
  try
  insert
  catch unique_constraint_violation
  update
  end try
 
  this will end up cluttering the logs with all the constraint violation
  messages.
 
 Really?  It's not supposed to.

There might be a different bug here.  This doesn't look right:

CREATE LANGUAGE plpgsql;

CREATE TABLE keytest (a int PRIMARY KEY, b text);

CREATE OR REPLACE FUNCTION insert_or_update(new_a int, new_b text)
RETURNS text
LANGUAGE plpgsql
AS $$
BEGIN
INSERT INTO keytest VALUES (new_a, new_b);
RETURN 'inserted';
EXCEPTION
  WHEN integrity_constraint_violation THEN
UPDATE keytest SET a = new_a, b = new_b;
RETURN 'updated';
END;
$$;

SELECT insert_or_update(1, 'one');
SELECT insert_or_update(1, 'one');
SELECT insert_or_update(2, 'two');
SELECT insert_or_update(2, 'two');

Results in:

 insert_or_update
--
 inserted
(1 row)

 insert_or_update
--
 updated
(1 row)

 insert_or_update
--
 inserted
(1 row)

ERROR:  duplicate key value violates unique constraint keytest_pkey
CONTEXT:  SQL statement UPDATE keytest SET a =  $1 , b =  $2 
PL/pgSQL function insert_or_update line 6 at SQL statement



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Merlin Moncure
On Thu, Nov 12, 2009 at 8:25 AM, Andrew Chernow a...@esilo.com wrote:

 2. The payload parameter is optional. A notifying client can either call
 NOTIFY foo; or NOTIFY foo 'payload';. The length of the payload is
 currently limited to 128 characters... Not sure if we should allow
 longer
 payload strings...

 Might be a good idea to make the max the same as the max length for
 prepared transaction GUIDs?  Not sure anyone would be shipping those
 around, but it's a pre-existing limit of about the same size.

 Yes, sounds reasonable to have the same limit for user-defined
 identifiers...


 [..begging..] Can this be increased significantly?  I don't get it, is there
 any technical reason to make the limit soo small?  This drastically reduces
 the usefulness of the payload.  I've wanted this feature for quite sometime
 and it is quite disappointing that I could not even use it because it is
 unjustifiably limited.

+1

What advantage is there in limiting it to a tiny size?  This is a
'payload' after all...an arbitrary data block. Looking at the patch I
noticed the payload structure (AsyncQueueEntry) is fixed length and
designed to lay into QUEUE_PAGESIZE (set to) BLCKSZ sized pages.

Couple of questions:

*) is BLCKSZ a hard requirement, that is, coming from the slru
implementation, or can QUEUE_PAGESIZE be bumped independently of block
size.

*) why not make the AsyncQueueEntry divide evenly into BLCKSZ, that
is, make the whole structure a size that is a multiple of two?  (this
would make the payload length 'weird')

*) is there any downside you see to making the AsyncQueueEntry
structure exactly BLCKSZ bytes in size?  Are we worried about the
downsides of spinning the notifications out to disk?

*) Is a variable length AsyncQueueEntry possible? (presumably bounded
by the max page size).  Or does complicate the implementation too
much?

merlin





merlin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] not logging caught exceptions

2009-11-12 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 There might be a different bug here.  This doesn't look right:

The UPDATE lacks a WHERE clause :-(

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Tom Lane
Fujii Masao masao.fu...@gmail.com writes:
 The problem is that fsync needs to be issued too frequently, which would
 be harmless in asynchronous replication, but not in synchronous one.
 A transaction would have to wait for the primary's and standby's fsync
 before returning a success to a client.

Surely that is exactly what is *required* if the user has asked for
synchronous replication.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] NULL-handling in aggregate(DISTINCT ...)

2009-11-12 Thread Tom Lane
Andrew Gierth and...@tao11.riddles.org.uk writes:
 A followup question: currently the code uses the datum interface for
 tuplesort. Obviously with multiple columns the slot interface is used
 instead; but is there any performance advantage for staying with the
 datum interface for the single-column case?

No idea ... measure it and see.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Andrew Chernow




What advantage is there in limiting it to a tiny size?  This is a
'payload' after all...an arbitrary data block. Looking at the patch I
noticed the payload structure (AsyncQueueEntry) is fixed length and
designed to lay into QUEUE_PAGESIZE (set to) BLCKSZ sized pages.



H.  Looks like the limitation comes from slru.  The true payload 
limit is (8K - struct members) the way this is implemented.


--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] plperl and inline functions -- first draft

2009-11-12 Thread Joshua Tolley
On Fri, Nov 06, 2009 at 09:53:20PM -0500, Andrew Dunstan wrote:
 Joshua Tolley wrote:
  I looked through the
 regression tests and didn't find any that used plperl -- should we add one 
 for
 this (or for this and all kinds of other stuff)? Is there some way to make
 running the regression test conditional on having built --with-perl in the
 first place?

 Look in src/pl/plperl/{sql,expected}

 cheers

 andrew

FWIW, I've added this to the upcoming commitfest page.

https://commitfest.postgresql.org/action/patch_view?id=206

--
Joshua Tolley / eggyknap
End Point Corporation
http://www.endpoint.com


signature.asc
Description: Digital signature


Re: [HACKERS] PL/Python array support

2009-11-12 Thread Peter Eisentraut
On ons, 2009-11-04 at 16:02 +0200, Peter Eisentraut wrote:
 Here is a patch to support arrays in PL/Python as parameters and
 return values.

Slightly updated version with fixed reference counting.
diff --git a/src/pl/plpython/expected/plpython_types.out b/src/pl/plpython/expected/plpython_types.out
index 2dd498c..cbc93a2 100644
--- a/src/pl/plpython/expected/plpython_types.out
+++ b/src/pl/plpython/expected/plpython_types.out
@@ -477,3 +477,106 @@ CONTEXT:  PL/Python function test_type_conversion_bytea10
 ERROR:  value for domain bytea10 violates check constraint bytea10_check
 CONTEXT:  while creating return value
 PL/Python function test_type_conversion_bytea10
+--
+-- Arrays
+--
+CREATE FUNCTION test_type_conversion_array_int4(x int4[]) RETURNS int4[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_int4(ARRAY[0, 100]);
+INFO:  ([0, 100], type 'list')
+CONTEXT:  PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4 
+-
+ {0,100}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[0,-100,55]);
+INFO:  ([0, -100, 55], type 'list')
+CONTEXT:  PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4 
+-
+ {0,-100,55}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[NULL,1]);
+INFO:  ([None, 1], type 'list')
+CONTEXT:  PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4 
+-
+ {NULL,1}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[]::integer[]);
+INFO:  ([], type 'list')
+CONTEXT:  PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4 
+-
+ {}
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(NULL);
+INFO:  (None, type 'NoneType')
+CONTEXT:  PL/Python function test_type_conversion_array_int4
+ test_type_conversion_array_int4 
+-
+ 
+(1 row)
+
+SELECT * FROM test_type_conversion_array_int4(ARRAY[[1,2,3],[4,5,6]]);
+ERROR:  cannot convert multidimensional array to Python list
+DETAIL:  PL/Python only supports one-dimensional arrays.
+CONTEXT:  PL/Python function test_type_conversion_array_int4
+CREATE FUNCTION test_type_conversion_array_bytea(x bytea[]) RETURNS bytea[] AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_bytea(ARRAY[E'\\xdeadbeef'::bytea, NULL]);
+INFO:  (['\xde\xad\xbe\xef', None], type 'list')
+CONTEXT:  PL/Python function test_type_conversion_array_bytea
+ test_type_conversion_array_bytea 
+--
+ {\\xdeadbeef,NULL}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_mixed1() RETURNS text[] AS $$
+return [123, 'abc']
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_mixed1();
+ test_type_conversion_array_mixed1 
+---
+ {123,abc}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_mixed2() RETURNS int[] AS $$
+return [123, 'abc']
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_mixed2();
+ERROR:  invalid input syntax for integer: abc
+CONTEXT:  while creating return value
+PL/Python function test_type_conversion_array_mixed2
+CREATE FUNCTION test_type_conversion_array_record() RETURNS type_record[] AS $$
+return [None]
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_record();
+ERROR:  PL/Python functions cannot return type type_record[]
+DETAIL:  PL/Python does not support conversion to arrays of row types.
+CREATE FUNCTION test_type_conversion_array_string() RETURNS text[] AS $$
+return 'abc'
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_string();
+ test_type_conversion_array_string 
+---
+ {a,b,c}
+(1 row)
+
+CREATE FUNCTION test_type_conversion_array_tuple() RETURNS text[] AS $$
+return ('abc', 'def')
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_array_tuple();
+ test_type_conversion_array_tuple 
+--
+ {abc,def}
+(1 row)
+
diff --git a/src/pl/plpython/plpython.c b/src/pl/plpython/plpython.c
index 6fd4aca..8f097b3 100644
--- a/src/pl/plpython/plpython.c
+++ b/src/pl/plpython/plpython.c
@@ -89,6 +89,9 @@ typedef struct PLyDatumToOb
 	Oid			typoid;			/* The OID of the type */
 	Oid			typioparam;
 	bool		typbyval;
+	int16		typlen;
+	char		typalign;
+	struct PLyDatumToOb *elm;
 } PLyDatumToOb;
 
 typedef struct PLyTupleToOb
@@ -120,6 +123,9 @@ typedef struct PLyObToDatum
 	Oid			typoid;			/* The OID of the type */
 	Oid			typioparam;
 	bool		typbyval;
+	int16		typlen;
+	char		typalign;
+	struct PLyObToDatum *elm;
 } PLyObToDatum;
 
 typedef struct PLyObToTuple
@@ -284,6 +290,7 @@ static PyObject *PLyInt_FromInt32(PLyDatumToOb *arg, Datum d);
 static PyObject *PLyLong_FromInt64(PLyDatumToOb *arg, Datum d);
 static PyObject 

Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Greg Stark
On Thu, Nov 12, 2009 at 3:09 PM, Andrew Chernow a...@esilo.com wrote:

 What advantage is there in limiting it to a tiny size?  This is a
 'payload' after all...an arbitrary data block. Looking at the patch I
 noticed the payload structure (AsyncQueueEntry) is fixed length and
 designed to lay into QUEUE_PAGESIZE (set to) BLCKSZ sized pages.


 H.  Looks like the limitation comes from slru.  The true payload limit
 is (8K - struct members) the way this is implemented.

So I'm beginning to think that adding a payload to NOTIFY is a bad
idea altogether.

The original use case NOTIFY was designed to handle was to implement
condition variables. You have clients wait on some data structure and
whenever someone changes that data structure they notify all waiting
clients to reread the data structure and check if their condition is
true. The condition might be that the data structure doesn't match
their locally cached copy and they have to rebuild their cache, it
might be that some work queue is non-empty, etc.

So the way to build a queue would be to use a table to hold your work
queue and then use NOTIFY whenever you insert into the queue and any
idle workers would know to go dequeue something from the queue.

It sounds like what people are trying to do is use NOTIFY as a queue
directly. The problem with this is that it was specifically not
designed to do this. As evidenced by the fact that it can't store
arbitrary data structures (nothing else in postgres is limited to only
handling text data, this would be an annoying arbitrary limitation),
can't handle even moderately large data efficiently, etc.

I'm beginning to think the right solution is to reject the idea of
adding a payload to the NOTIFY mechanism and instead provide a queue
contrib module which provides the interface people want.


-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] array_to_string bug?

2009-11-12 Thread David Fetter
Folks,

Here's expected behavior:

davidfet...@postgres=# SELECT array(values(1),(null));
 ?column? 
──
 {1,NULL}
(1 row)

The next one is just plain unexpected.  Either it's a bug, or it needs
more documentation in the function description in the docs, \df+
output, etc.

davidfet...@postgres=# SELECT array_to_string(array(values(1),(null)),'');
 array_to_string 
─
 1
(1 row)

I would have expected it to come out as NULL.

What say?

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Joachim Wieland
On Thu, Nov 12, 2009 at 3:30 PM, Merlin Moncure mmonc...@gmail.com wrote:
 Couple of questions:

 *) is BLCKSZ a hard requirement, that is, coming from the slru
 implementation, or can QUEUE_PAGESIZE be bumped independently of block
 size.

It's the size of slru's pages.

 *) why not make the AsyncQueueEntry divide evenly into BLCKSZ, that
 is, make the whole structure a size that is a multiple of two?  (this
 would make the payload length 'weird')

it's possible and yes, it would make the payload length weird. Also if
we wanted to add further structure members later on (commit time, xid,
notify counter, whatever) we'd have to announce that payload in a new
release cannot have the same length as in a previous one, so I tried
to keep both independent right from the beginning.

 *) is there any downside you see to making the AsyncQueueEntry
 structure exactly BLCKSZ bytes in size?  Are we worried about the
 downsides of spinning the notifications out to disk?

Yes, and also we'd need a lot more page slots to work efficiently and
I fear quite some queue managing overhead (locking). Currently we are
using 4 page slots and can have ~160 notifications mapped into memory.

 *) Is a variable length AsyncQueueEntry possible? (presumably bounded
 by the max page size).  Or does complicate the implementation too
 much?

That's an option to look at, I think it's technically possible, we'd
just need to throw in some more logic but I didnt want to invest too
much effort before we have a clear way to go.

However I share Greg's concerns that people are trying to use NOTIFY
as a message queue which it is not designed to be.


Joachim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] plpgsql GUC variable: custom or built-in?

2009-11-12 Thread Tom Lane
So I think we're agreed on adding a variable_conflict GUC for plpgsql.
The straight-and-narrow way to do it would be to make this a custom
GUC that's defined only when plpgsql is dynamically loaded into the
backend.  However that implies a lot of notational overhead, notably
having to put plpgsql into custom_variable_classes if you want to set
the variable in postgresql.conf.  Maybe that's okay, particularly if
you are of the opinion that most people will leave it at default.
But it could also be argued that plpgsql is so widely used that we
should bend the rules for it, and make this a built-in GUC that just
happens to only be consulted by plpgsql.

I'm leaning to the custom GUC approach because it seems a little
cleaner from a code point of view, but I wanted to see if anyone
wishes to argue for the other way.

One reason to argue for the other way is that maybe it wouldn't only
be consulted by plpgsql.  In particular I can easily imagine SQL
functions having the same issue as soon as someone gets around to
letting them use names for their parameters.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Bernd Helmle



--On 12. November 2009 15:48:44 + Greg Stark gsst...@mit.edu wrote:


I'm beginning to think the right solution is to reject the idea of
adding a payload to the NOTIFY mechanism and instead provide a queue
contrib module which provides the interface people want.


Isn't PgQ already the solution you have in mind?

--
Thanks

Bernd

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] array_to_string bug?

2009-11-12 Thread Tom Lane
David Fetter da...@fetter.org writes:
 The next one is just plain unexpected.

array_to_string ignores null elements.  What do you think it should do
with them?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] plpgsql GUC variable: custom or built-in?

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 11:13 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 One reason to argue for the other way is that maybe it wouldn't only
 be consulted by plpgsql.  In particular I can easily imagine SQL
 functions having the same issue as soon as someone gets around to
 letting them use names for their parameters.

I don't have a strong feeling on the core issue but I don't agree with
this point.  AIUI, we are implementing multiple behaviors here for
reasons of backward and competing-product compatibility.  Presumably,
if we're starting from scratch, we'll pick a sensible behavior -
probably error in the case of SQL - and stick with it.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 11:30 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Joachim Wieland j...@mcknight.de writes:
 However I share Greg's concerns that people are trying to use NOTIFY
 as a message queue which it is not designed to be.

 Yes.  Particularly those complaining that they want to have very large
 payload strings --- that's pretty much a dead giveaway that it's not
 being used as a condition signal.

 Now you might say that yeah, that's the point, we're trying to enable
 using NOTIFY in a different style.  The problem is that if you are
 trying to use NOTIFY as a queue, you will soon realize that it has
 the wrong semantics for that --- in particular, losing notifies across
 a system crash or client crash is OK for a condition notification,
 not so OK for a message queue.  The difference is that the former style
 assumes that the authoritative data is in a table somewhere, so you can
 still find out what you need to know after reconnecting.  If you are
 doing messaging you are likely to think that you don't need any backing
 store for the system state.

 So while a payload string for NOTIFY has been on the to-do list since
 forever, I have to think that Greg's got a good point questioning
 whether it is actually a good idea.

I think there could be cases where the person writing the code can
know, extrinsic to the system, that lost notifications are OK, and
still want to deliver a payload.  But I think the idea of enabling a
huge payload is not wise, as it sounds like it will sacrifice
performance for a feature that is by definition not essential to
anyone who is using this now.  A small payload seems like a reasonable
compromise.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Merlin Moncure
On Thu, Nov 12, 2009 at 11:30 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Joachim Wieland j...@mcknight.de writes:
 However I share Greg's concerns that people are trying to use NOTIFY
 as a message queue which it is not designed to be.

 Yes.  Particularly those complaining that they want to have very large
 payload strings --- that's pretty much a dead giveaway that it's not
 being used as a condition signal.

 Now you might say that yeah, that's the point, we're trying to enable
 using NOTIFY in a different style.  The problem is that if you are
 trying to use NOTIFY as a queue, you will soon realize that it has
 the wrong semantics for that --- in particular, losing notifies across
 a system crash or client crash is OK for a condition notification,
 not so OK for a message queue.  The difference is that the former style
 assumes that the authoritative data is in a table somewhere, so you can
 still find out what you need to know after reconnecting.  If you are
 doing messaging you are likely to think that you don't need any backing
 store for the system state.

 So while a payload string for NOTIFY has been on the to-do list since
 forever, I have to think that Greg's got a good point questioning
 whether it is actually a good idea.

You guys are assuming it's being used in a queue, which is only a
fraction of cases where this feature is useful.  In fact, having a
payload can remove the need for a queue completely where is currently
required for no other reason to deliver payload messages.

I'm sorry, the 128 character limit is simply lame (other than for
unsolvable implementation/performance complexity which I doubt is the
case here), and if that constraint is put in by the implementation,
than the implementation is busted and should be reworked until it's
right.  A feature that is being used for things not intended is a sign
of a strong feature, not a weak one, and the idea that a payload
should be length limited in order to prevent use in ways that are
'wrong' is a very weak argument IMO.  People have been asking for this
feature since the beginning of time, and nobody said: 'please limit it
to 128 bytes'. A limit of 4k - 64k is much more appropriate if you
even want a hard limit at all...

merlin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Merlin Moncure
On Thu, Nov 12, 2009 at 11:39 AM, Robert Haas robertmh...@gmail.com wrote:
 On Thu, Nov 12, 2009 at 11:30 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 Joachim Wieland j...@mcknight.de writes:
 However I share Greg's concerns that people are trying to use NOTIFY
 as a message queue which it is not designed to be.

 Yes.  Particularly those complaining that they want to have very large
 payload strings --- that's pretty much a dead giveaway that it's not
 being used as a condition signal.

 Now you might say that yeah, that's the point, we're trying to enable
 using NOTIFY in a different style.  The problem is that if you are
 trying to use NOTIFY as a queue, you will soon realize that it has
 the wrong semantics for that --- in particular, losing notifies across
 a system crash or client crash is OK for a condition notification,
 not so OK for a message queue.  The difference is that the former style
 assumes that the authoritative data is in a table somewhere, so you can
 still find out what you need to know after reconnecting.  If you are
 doing messaging you are likely to think that you don't need any backing
 store for the system state.

 So while a payload string for NOTIFY has been on the to-do list since
 forever, I have to think that Greg's got a good point questioning
 whether it is actually a good idea.

 I think there could be cases where the person writing the code can
 know, extrinsic to the system, that lost notifications are OK, and
 still want to deliver a payload.  But I think the idea of enabling a
 huge payload is not wise, as it sounds like it will sacrifice
 performance for a feature that is by definition not essential to

'premature optimization is the root of all evil' :-)

merlin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] plpgsql GUC variable: custom or built-in?

2009-11-12 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Thu, Nov 12, 2009 at 11:13 AM, Tom Lane t...@sss.pgh.pa.us wrote:
 One reason to argue for the other way is that maybe it wouldn't only
 be consulted by plpgsql.  In particular I can easily imagine SQL
 functions having the same issue as soon as someone gets around to
 letting them use names for their parameters.

 I don't have a strong feeling on the core issue but I don't agree with
 this point.  AIUI, we are implementing multiple behaviors here for
 reasons of backward and competing-product compatibility.  Presumably,
 if we're starting from scratch, we'll pick a sensible behavior -
 probably error in the case of SQL - and stick with it.

Fair enough.  I'll start writing the custom GUC then.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Simon Riggs
On Thu, 2009-11-12 at 06:46 -0500, Robert Haas wrote:

 Having said that,
 I'm not capable of single-handedly effecting an on-time release

You're bloody good and the task needs to fit our capability anyway. 

So, yes, you are.

 We need larger, more robust pools of
 committers, reviewers, commitfest managers, etc.

We're living in a desert. We just need to remember it. Plan hard, focus
on the important and be real. Move at a smooth pace to save resources.
Don't give up when the going gets tough, just rest up and then continue.

Not a new idea, but I think we should require all patch submitters to do
one review per submission. There needs to be a balance between time
spent on review and time spent on dev. The only real way this happens in
any community is by peer review. 

All patch submitters need to know that they must also take their turn as
patch reviewers. If it is a hard rule, then patch *sponsors* would be
forced to accept that they must *also* pay for review time. It is the
sponsors that need to be forced to accept that reality, though we can
only get at them through controlling developer behaviour. So, I
propose that we simply ignore patches from developers until they have
done sufficient review to be allowed to develop again.

-- 
 Simon Riggs   www.2ndQuadrant.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 11:31 AM, Simon Riggs si...@2ndquadrant.com wrote:
 Not a new idea, but I think we should require all patch submitters to do
 one review per submission. There needs to be a balance between time
 spent on review and time spent on dev. The only real way this happens in
 any community is by peer review.

 All patch submitters need to know that they must also take their turn as
 patch reviewers. If it is a hard rule, then patch *sponsors* would be
 forced to accept that they must *also* pay for review time. It is the
 sponsors that need to be forced to accept that reality, though we can
 only get at them through controlling developer behaviour. So, I
 propose that we simply ignore patches from developers until they have
 done sufficient review to be allowed to develop again.

I agree.  I would quibble only with the details.  I think we should
require patch authors to act as a reviewer for any CommitFest for
which they have submitted patches.  We don't need every patch author
to review as many patches as they submit, because some people will
review extras, and some non-patch-authors will review.  If they review
one patch each, that's probably more than enough.  It's also easier
for bookkeeping purposes.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Andrew Chernow



Now you might say that yeah, that's the point, we're trying to enable
using NOTIFY in a different style.  The problem is that if you are
trying to use NOTIFY as a queue, you will soon realize that it has
the wrong semantics for that --- in particular, losing notifies across
a system crash or client crash is OK for a condition notification,
not so OK for a message queue.  The difference is that the former style
assumes that the authoritative data is in a table somewhere, so you can
still find out what you need to know after reconnecting.  If you are
doing messaging you are likely to think that you don't need any backing
store for the system state.



I simply don't agree that the semantics have to change.  You call it a 
queue, I call it sesison data.  There is no reason why the documentation 
can't state that notifies may not be delivered due to crashes, so make 
sure to use persistent storage for any payload worth keeping post-session.


--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Dave Page
On Thu, Nov 12, 2009 at 4:30 PM, Tom Lane t...@sss.pgh.pa.us wrote:

 So while a payload string for NOTIFY has been on the to-do list since
 forever, I have to think that Greg's got a good point questioning
 whether it is actually a good idea.

Here's an example of why I'd like a payload (and not a queue in an
add-on module). Say you have multiple users running pgAdmin. One of
them creates a new table. Currently, unless the other user refreshes
his view of the database, he won't see it. I'd like to be able to
notify the other pgAdmin sessions that a new table has been created,
so they can automatically display it, without having to poll pg_class
or be manually refreshed. The payload could contain details of type of
object that has been created, and it's OID/identifier to minimise the
work required of the other sessions to find and display the new
object.

And as I'm sure you're already thinking it, yes, I know it doesn't
help if the new table is created using psql, but there are lots of
shops where pgAdmin is the default tool, and it could help them and
just exhibit the current behaviour if someone does break out psql.


-- 
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Tom Lane
Joachim Wieland j...@mcknight.de writes:
 However I share Greg's concerns that people are trying to use NOTIFY
 as a message queue which it is not designed to be.

Yes.  Particularly those complaining that they want to have very large
payload strings --- that's pretty much a dead giveaway that it's not
being used as a condition signal.

Now you might say that yeah, that's the point, we're trying to enable
using NOTIFY in a different style.  The problem is that if you are
trying to use NOTIFY as a queue, you will soon realize that it has
the wrong semantics for that --- in particular, losing notifies across
a system crash or client crash is OK for a condition notification,
not so OK for a message queue.  The difference is that the former style
assumes that the authoritative data is in a table somewhere, so you can
still find out what you need to know after reconnecting.  If you are
doing messaging you are likely to think that you don't need any backing
store for the system state.

So while a payload string for NOTIFY has been on the to-do list since
forever, I have to think that Greg's got a good point questioning
whether it is actually a good idea.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Selena Deckelmann
On Wed, Nov 11, 2009 at 2:03 PM, Josh Berkus j...@agliodbs.com wrote:

 I'd be happy to get together at some pre-appointed hour this weekend
 (Saturday / Sunday) to talk it over by phone / IRC. PDXPUG was already
 planning to get together to review some patches this Sunday from 3-5pm
 PST, so that is a convenient time for me.

 Aren't you running OpenSQL this weekend?

Yes :) But we have lots of volunteers. It's an unconference, and I delegated.

-selena

-- 
http://chesnok.com/daily - me
http://endpoint.com - work

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Andrew Chernow



However I share Greg's concerns that people are trying to use NOTIFY
as a message queue which it is not designed to be.


When you have an established libpq connection waiting for notifies it is 
not unreasonable to expect/desire a payload.  ISTM, the problem is that 
the initial design was half-baked.  NOTIFY is event-driven, ie. no 
polling!


--
Andrew Chernow
eSilo, LLC
every bit counts
http://www.esilo.com/

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Selena Deckelmann
On Thu, Nov 12, 2009 at 8:31 AM, Simon Riggs si...@2ndquadrant.com wrote:

 Not a new idea, but I think we should require all patch submitters to do
 one review per submission. There needs to be a balance between time
 spent on review and time spent on dev. The only real way this happens in
 any community is by peer review.

I like this idea. Perhaps also if a person is reviewing a patch for
the first time, we could make some effort to get a more experienced
person paired up with them.

Pairing up was really useful for the PDXPUG folks working on review.

-- 
http://chesnok.com/daily - me
http://endpoint.com - work

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] write ahead logging in standby (streaming replication)

2009-11-12 Thread Greg Smith

Tom Lane wrote:

Fujii Masao masao.fu...@gmail.com writes:
  

The problem is that fsync needs to be issued too frequently, which would
be harmless in asynchronous replication, but not in synchronous one.
A transaction would have to wait for the primary's and standby's fsync
before returning a success to a client.



Surely that is exactly what is *required* if the user has asked for
synchronous replication.
  
This a distressingly common thing people get wrong about replication.  
You can either have synchronous replication, which as you say has to be 
slow:  you must wait for an fsync ACK from the secondary and a return 
trip before you can say something is committed on the primary.  Or you 
can get better performance by not waiting for all of those things, but 
the minute you do that it's *not* synchronous replication anymore.  You 
can't get high-performance and true synchronous behavior; you have to 
pick one.  The best you can do if you need both is work on accelerating 
fsync everywhere using the standard battery-backed write cache technique.


--
Greg Smith2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com  www.2ndQuadrant.com



[HACKERS] CommitFest 2009-11 Call for Reviewers

2009-11-12 Thread Greg Smith
In a few days the 3rd 8.5 development CommitFest, 2009-11, is going to 
kick off, with the end goal being an alpha3 prerelease.  If you have a 
patch in progress, you'll need to submit it before the deadline of 
2009-11-15 00:00:00 GMT for it to be considered during this round:  
http://wiki.postgresql.org/wiki/Submitting_a_Patch


The actual process of the CommitFest itself is fairly well documented at 
this point:


http://wiki.postgresql.org/wiki/Reviewing_a_Patch
http://wiki.postgresql.org/wiki/RRReviewers
http://wiki.postgresql.org/wiki/Running_a_CommitFest

For lack of a more qualified volunteer, I'll be handling the initial 
round of patch assignments and reviewer organization.  I suspect we'll 
reorganize on the fly as things proceed based on who has time; I'd 
certainly welcome patch-chasing help in addition to reviewing.  Since 
the backlog for this CommitFest is so far lighter than we've seen 
recently, the small patches that don't already have an active reviewer 
shouldn't be too difficult to get through.


Please send me an email (without copying the list) if you are available 
to help with review.  Include any information that might be helpful in 
assigning you an appropriate patch.  If there's a specific one you want 
to claim, by all means let me know that.  All reviewers will need to be 
subscribed to the RRR mailing list, so when you write me please also 
follow the subscription link at 
http://archives.postgresql.org/pgsql-rrreviewers/ to add yourself to 
that list, too, if you're not already there.


The set of patches I have the least feel for are the five ECPG 
submissions, some of which were reviewed already.  I would particularly 
appreciate any early information reviewers might provide about their 
capability/willingness to work on that set.  Those are not so easy to 
just split among multiple people due to how they relate to one another.


--
Greg Smith2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com  www.2ndQuadrant.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Euler Taveira de Oliveira
Simon Riggs escreveu:
 So, I
 propose that we simply ignore patches from developers until they have
 done sufficient review to be allowed to develop again.
 
Is it really impolite for a first-contributor, no?


-- 
  Euler Taveira de Oliveira
  http://www.timbira.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] array_to_string bug?

2009-11-12 Thread David Fetter
On Thu, Nov 12, 2009 at 11:20:26AM -0500, Tom Lane wrote:
 David Fetter da...@fetter.org writes:
  The next one is just plain unexpected.
 
 array_to_string ignores null elements.  What do you think it should
 do with them?

It should concatenate them, i.e. return a NULL if the array includes
any NULLs.  That, or it should explain that it doesn't do that because
there's nothing in the docs that would indicate this behavior.

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [RRR] CommitFest 2009-11 Call for Reviewers

2009-11-12 Thread Josh Berkus
On 11/12/09 9:45 AM, Greg Smith wrote:
 For lack of a more qualified volunteer, I'll be handling the initial
 round of patch assignments and reviewer organization.

Hmmm?  Who's more qualified than you, exactly?

--Josh Berkus

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch committers

2009-11-12 Thread Josh Berkus

 That's basically just it:  Assume bashing is part of the process.  Don't
 think of it as bashing.  Take the constructive criticism from it, ignore
 the rest.  Assume only one out of three feature ideas will make it.
 Apply the prerequisite amount of gamesmanship to the system and tune
 your bikeshedding detectors.  Don't take anything personally.  Live and
 learn.

I think what Emmanuel is complaining about here ... with some
justification ... is not the revision requirements of our patch process
but the extremely confusing and frustrating nature of it for new
contributors.  For example, how exactly is a new contributor supposed to
know the difference between bikeshedding and constructive criticism,
when (a) they don't know the people involved, and (b) even our more
dedicated committers engage in bikeshedding periodically?

This isn't just Emmanuel; I've heard this complaint from numerous
first-time contributors, and have seen several talented people walk away
from our project because of it.  Even people who have stuck with us,
such as Josh Tolley, have remarked on the hazing ritual inherent in
getting a first-time contribution in.

Probably, the only possible solution is for each new contributor to have
a mentor who sticks with them throughout the process of getting their
first contribution accepted, explaining the process, pre-reviewing their
submissions, and explaining to them which criticism they should be
paying attention to and which they shouldn't.  I *thought* that Bruce
was doing that for AsterData, but apparently not.

This would require a pool of experienced contributors volunteering to be
mentors, which I'm not sure we have.

It would also help if people on this list *in general*, were to be a bit
more consistent about phrasing criticism as constructive criticism.
I've seen far too much how stupid are you? on this list for the good
health of our developer community.

--Josh Berkus

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] array_to_string bug?

2009-11-12 Thread Steve Crawford

Tom Lane wrote:

David Fetter da...@fetter.org writes:
  

The next one is just plain unexpected.



array_to_string ignores null elements.  What do you think it should do
with them?

regards, tom lane

  
This seems somewhat related to the long-running discussion from back in 
February-April regarding string_to_array with empty input which faded 
away somewhere around here: 
http://archives.postgresql.org/pgsql-hackers/2009-04/msg00363.php. At 
the time the decision was to defer any decision to after 8.4.


Perhaps there is a solution which can address both cases - ideally one 
which would, to the extent practical, allow string_to_array to be the 
inverse of array_to_string. This could be particularly useful when 
dealing with clients that don't know how to directly deal with 
PostgreSQL arrays but which can generally easily deal with strings.


Although it might cause a fair amount of backward-compatibility trouble, 
the string representation could either use NULL to represent a null 
element as is allowed in other contexts or require that empty-string 
elements be represented as  to differentiate ,, (empty-string 
element) from ,, (null element).


Cheers,
Steve



Re: [HACKERS] next CommitFest

2009-11-12 Thread Josh Berkus

 I like this idea. Perhaps also if a person is reviewing a patch for
 the first time, we could make some effort to get a more experienced
 person paired up with them.

When I was CFM last year, I assigned patches for review first if the
patch author was doing a review themselves.  Patches whose authors were
not doing reviews often got reviewed last, which means that sometimes
they waited a commitfest.

Of course, Simon submitted so many patches that despite him doing
reviews, it was hard to get them all assigned.  Simon: slow down a
little!  ;-)

--Josh Berkus


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 1:25 PM, Albert Cervera i Areny
alb...@nan-tic.com wrote:
 A Dijous, 12 de novembre de 2009, Euler Taveira de Oliveira va escriure:
 Simon Riggs escreveu:
  So, I
  propose that we simply ignore patches from developers until they have
  done sufficient review to be allowed to develop again.

 Is it really impolite for a first-contributor, no?


 I don't think so, as long as it's properly explained.

Personally, I would not propose to impose this rule of first-time
contributors, or even second-time contributors.  But by about patch #3
I think everyone should be pitching in.

Just MHO, of course.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Albert Cervera i Areny
A Dijous, 12 de novembre de 2009, Euler Taveira de Oliveira va escriure:
 Simon Riggs escreveu:
  So, I
  propose that we simply ignore patches from developers until they have
  done sufficient review to be allowed to develop again.
 
 Is it really impolite for a first-contributor, no?
 

I don't think so, as long as it's properly explained.

-- 
Albert Cervera i Areny
http://www.NaN-tic.com
Mòbil: +34 669 40 40 18

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] array_to_string bug?

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 1:28 PM, Steve Crawford
scrawf...@pinpointresearch.com wrote:
 Although it might cause a fair amount of backward-compatibility trouble, the
 string representation could either use NULL to represent a null element as
 is allowed in other contexts or require that empty-string elements be
 represented as  to differentiate ,, (empty-string element) from ,, (null
 element).

That would cause a substantial amount of grief to people who might not
want that behavior, though.  I use these functions for creating
human-readable output, not for serialization.  Simple, predictable
behavior is very important.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Brendan Jurd
2009/11/13 Euler Taveira de Oliveira eu...@timbira.com:
 Simon Riggs escreveu:
 So, I
 propose that we simply ignore patches from developers until they have
 done sufficient review to be allowed to develop again.

 Is it really impolite for a first-contributor, no?


I support Simon's proposal, but I think we would certainly need to
make an exception for first-time contributors.

Cheers,
BJ

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Listen / Notify rewrite

2009-11-12 Thread Merlin Moncure
On Thu, Nov 12, 2009 at 11:40 AM, Merlin Moncure mmonc...@gmail.com wrote:
 I'm sorry, the 128 character limit is simply lame (other than for
 unsolvable implementation/performance complexity which I doubt is the
 case here), and if that constraint is put in by the implementation,
 than the implementation is busted and should be reworked until it's
 right.

After some reflection, I realized this was an overly strong statement
and impolite to the OP.  It's easy to yarp from the gallery with the
other peanuts :-).  It's not the implementation I have an issue with,
just the _idea_ that we should be restricted to small payloads for
religious reasons...until that came upI was already scheming on how to
both extend the patch to be more flexible in terms of payload size,
and to backpatch and test it on 8.4  (no point if the community has no
interest however).  In any event, sorry for the strong words.

merlin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Jeff Davis
On Thu, 2009-11-12 at 15:52 -0200, Euler Taveira de Oliveira wrote:
 Simon Riggs escreveu:
  So, I
  propose that we simply ignore patches from developers until they have
  done sufficient review to be allowed to develop again.
  
 Is it really impolite for a first-contributor, no?

I believe the community has always given extra help to first-time
contributors.

If anyone sees a new contributor being ignored, they should make an
effort to help.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Alvaro Herrera
Robert Haas escribió:
 On Thu, Nov 12, 2009 at 1:25 PM, Albert Cervera i Areny
 alb...@nan-tic.com wrote:
  A Dijous, 12 de novembre de 2009, Euler Taveira de Oliveira va escriure:
  Simon Riggs escreveu:
   So, I
   propose that we simply ignore patches from developers until they have
   done sufficient review to be allowed to develop again.
 
  Is it really impolite for a first-contributor, no?
 
  I don't think so, as long as it's properly explained.
 
 Personally, I would not propose to impose this rule of first-time
 contributors, or even second-time contributors.  But by about patch #3
 I think everyone should be pitching in.

+1.  It's OK to not review anything if you're just doing a one-off.  If
you submit a bunch of patches you start becoming part of the community.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] plpgsql GUC variable: custom or built-in?

2009-11-12 Thread Tom Lane
I wrote:
 Fair enough.  I'll start writing the custom GUC then.

While testing that, I noticed a pre-existing GUC limitation that maybe
we should do something about now: it does not work very nicely for
custom SUSET GUC variables.  Normally, if a variable is SUSET, an
ordinary user can't do ALTER USER SET or ALTER DATABASE SET to modify
it.  However, a superuser can do that on his behalf, and then the
variable will default in the desired way for that user or database.
But this logic doesn't work for custom variables: the system doesn't
record whether the option was set by a superuser or not, so it's
afraid to allow the value to be applied when the defining module
gets loaded.  (Instead you get a WARNING and the value reverts to
default.)

I think we discussed this once before and came up with the idea that
it wouldn't be a problem if ALTER USER/DATABASE SET disallowed setting
of variables not currently known to the system.  Right now, if
plpgsql is in custom_variable_classes, you can do
ALTER USER foouser SET plpgsql.variable_conflict = variable_first;
and the system will take it even if plpgsql isn't loaded.  If we
required plpgsql to be loaded then we could be sure that the appropriate
permissions check had been made when the ALTER was done, and then in
subsequent sessions it would be safe to apply the variable value while
loading plpgsql.

One possible objection is that a loadable module couldn't safely upgrade
a USERSET variable to SUSET (except across a major version boundary),
since the permissions check would already have been made implicitly for
any ALTER settings.  This doesn't seem like a big problem compared to
the current situation of not being able to use SUSET effectively at all,
though.

Another issue is that pg_dumpall output would fail to reload with such a
restriction, since the dump script would most likely be executed in a
session that hadn't loaded the relevant loadable module.  We could get
around that by still allowing superusers to issue ALTER SET for unknown
variables, but that seems a tad weird.  OTOH the current rule (must be
in custom_variable_classes) is pretty hazardous for dump reloading, too.

Comments?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] array_to_string bug?

2009-11-12 Thread David Fetter
On Thu, Nov 12, 2009 at 01:33:41PM -0500, Robert Haas wrote:
 On Thu, Nov 12, 2009 at 1:28 PM, Steve Crawford
 scrawf...@pinpointresearch.com wrote:
  Although it might cause a fair amount of backward-compatibility
  trouble, the string representation could either use NULL to
  represent a null element as is allowed in other contexts or
  require that empty-string elements be represented as  to
  differentiate ,, (empty-string element) from ,, (null element).
 
 That would cause a substantial amount of grief to people who might
 not want that behavior, though.  I use these functions for creating
 human-readable output, not for serialization.  Simple, predictable
 behavior is very important.

My question boils down to, why is this string concatenation different
from all other string concatenations?

For now, the answer can be, it behaves differently with respect to
NULLs, and we just document this.  We can later decide whether this
behavior should change.

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] array_to_string bug?

2009-11-12 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Thu, Nov 12, 2009 at 1:28 PM, Steve Crawford
 scrawf...@pinpointresearch.com wrote:
 Although it might cause a fair amount of backward-compatibility trouble, the
 string representation could either use NULL to represent a null element as
 is allowed in other contexts or require that empty-string elements be
 represented as  to differentiate ,, (empty-string element) from ,, (null
 element).

 That would cause a substantial amount of grief to people who might not
 want that behavior, though.  I use these functions for creating
 human-readable output, not for serialization.  Simple, predictable
 behavior is very important.

I agree --- we don't want to start introducing quoting rules into
array_to_string.  I think the viable alternatives are the current
behavior, or treating a NULL element as if it were an empty string.
David's idea that the entire output should go to NULL might be sane
from a strict semantics point of view, but it also seems to make the
function just about entirely useless in practice.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Jeff Davis
On Thu, 2009-11-12 at 11:36 -0500, Robert Haas wrote:
 I agree.  I would quibble only with the details.  I think we should
 require patch authors to act as a reviewer for any CommitFest for
 which they have submitted patches.  We don't need every patch author
 to review as many patches as they submit, because some people will
 review extras, and some non-patch-authors will review.  If they review
 one patch each, that's probably more than enough.  It's also easier
 for bookkeeping purposes.

Not all contributors are fluent English readers and writers.

I certainly do not discourage those people from reviewing, but I can
understand how it might be more frustrating and less productive for
them. An important part of review is to read the relevant mailing list
threads and try to tie up loose ends and find a consensus.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch committers

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 1:27 PM, Josh Berkus j...@agliodbs.com wrote:

 That's basically just it:  Assume bashing is part of the process.  Don't
 think of it as bashing.  Take the constructive criticism from it, ignore
 the rest.  Assume only one out of three feature ideas will make it.
 Apply the prerequisite amount of gamesmanship to the system and tune
 your bikeshedding detectors.  Don't take anything personally.  Live and
 learn.

 I think what Emmanuel is complaining about here ... with some
 justification ... is not the revision requirements of our patch process
 but the extremely confusing and frustrating nature of it for new
 contributors.  For example, how exactly is a new contributor supposed to
 know the difference between bikeshedding and constructive criticism,
 when (a) they don't know the people involved, and (b) even our more
 dedicated committers engage in bikeshedding periodically?

 This isn't just Emmanuel; I've heard this complaint from numerous
 first-time contributors, and have seen several talented people walk away
 from our project because of it.  Even people who have stuck with us,
 such as Josh Tolley, have remarked on the hazing ritual inherent in
 getting a first-time contribution in.

I used to feel this way, too.  I'm not sure whether it's really worse
at first, or whether it just seems worse a first until you get used to
it.  There is no getting around the fact that this is a community of
very smart people.  I work at a company where I'm the only developer.
Guess what - my development ideas are all brilliant - or at least
there's no contrary evidence!  Then I come here, where there are
plenty of other developers just as smart as I am, or possibly smarter,
and here some of my ideas crash and burn.  On the other hand,
sometimes someone objects to my idea and the *objections* crash and
burn.  You can't expect to win them all when you're dealing with
people who are as smart as you and more knowledgeable about the code.

I try to be gentle with newcomers but it's not always easy to figure
out what that means.  When someone submits a bad patch and I review
it, they sometimes get two or three pages of feedback detailing
inadequacies of various types.  I understand that can be hard on the
ego, but I don't think it does anyone any favors to give them a
PARTIAL list of things they need to fix - in fact I think breaking up
the pain into multiple rounds is quite a bit worse, because then you
create the illusion of progress without the actuality.  I don't give
people feedback because I want to hurt them or cause them pain, but I
know from experience that until those things are fixed, it *will*
*not* *get* *committed*.  If I say nothing now, someone else will say
it later.

I am afraid that the difference between bikeshedding and legitimate
criticism is something that contributors have to learn for themselves.
 I can tell you which things *I* think are bikeshedding, but someone
else may have a longer, shorter, or same-size non-overlapping list.
No one can speak with authority on this topic because no one is the
boss.

All that having been said, I feel your pain.  I don't want to scare
off new contributors, but I don't necessarily know how to improve the
situation.

 Probably, the only possible solution is for each new contributor to have
 a mentor who sticks with them throughout the process of getting their
 first contribution accepted, explaining the process, pre-reviewing their
 submissions, and explaining to them which criticism they should be
 paying attention to and which they shouldn't.  I *thought* that Bruce
 was doing that for AsterData, but apparently not.

Well, this is a good idea, although I'm not sure how well it will play
out in practice.  I'm always game if anyone wants to email me off-list
to discuss issues of this type, or...

 This would require a pool of experienced contributors volunteering to be
 mentors, which I'm not sure we have.

...I'm willing to do this.

 It would also help if people on this list *in general*, were to be a bit
 more consistent about phrasing criticism as constructive criticism.
 I've seen far too much how stupid are you? on this list for the good
 health of our developer community.

I don't see very much of this at all, so maybe this is another area
where it boils down to perspective.  I find this list to be quite
civilized most of the time; I wouldn't hang around if I didn't.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER TABLE...ALTER COLUMN vs inheritance

2009-11-12 Thread Bernd Helmle



--On 4. November 2009 09:57:27 -0500 Tom Lane t...@sss.pgh.pa.us wrote:


 I think the
consensus was that the way to fix this (along with some other problems)
is to start representing NOT NULL constraints in pg_constraint, turning
attnotnull into just a bit of denormalization for performance.


I've just started looking into this and wonder how this should look like. 
My first idea is to just introduce a special contype in pg_constraint 
representing a NOT NULL constraint on a column, which holds all required 
information to do the mentioned maintenance stuff on them and to keep most 
of the current infrastructure. Utility commands need to track all changes 
in pg_constraint and keep pg_attribute.attnotnull up to date.


Another possibility is to change the representation of NOT NULL to be a 
CHECK constraint (e.g. CHECK(col IS NOT NULL)) internally and leave all the 
responsibility up to the current existing check constraint infrastructure 
(which already does the right thing for inheritance, e.g. it's not possible 
to drop such a constraint if it was inherited).
ALTER TABLE ... SET NOT NULL and DROP NOT NULL will be just syntactic sugar 
then.
I don't know the original design decisions for the current representation, 
but it seems it wasn't essential?


Of course, there's still the requirement to special case those check 
constraints in various places, since pg_dump and psql have to do the right 
thing.


--
Thanks

Bernd

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [RRR] CommitFest 2009-11 Call for Reviewers

2009-11-12 Thread Greg Smith

Josh Berkus wrote:

On 11/12/09 9:45 AM, Greg Smith wrote:
  

For lack of a more qualified volunteer, I'll be handling the initial
round of patch assignments and reviewer organization.



Hmmm?  Who's more qualified than you, exactly?
  

I was alluding to the fact that Robert isn't available to handle this one.

--
Greg Smith2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com  www.2ndQuadrant.com



Re: [HACKERS] Patch committers

2009-11-12 Thread Selena Deckelmann
I'm not addressing the complaints Emmanuel has about patch submission,
but was inspired by what he said. I hesitated to post this to
-hackers, but my hope is that my comments are taken in the spirit of
reflection.

I've been thinking about the Postgres community and things that
differentiate it from other FOSS projects for a few years. We lack
both a benevolent dictator, or a single-corporate backer... We have an
incredible reputation for technical expertise -- most knowledgeable
FOSS developers fear and are in awe of the quality of code our
community produces. I hear criticism for the way in which our
development community operates, and how we communicate with
outsiders. But few people are able to offer concrete suggestions on
/how/ we might change, as opposed to the /particular things/ that they
think should change.

So on this point:

On Wed, Nov 11, 2009 at 7:30 PM, Emmanuel Cecchet m...@frogthinker.org wrote:

 The commitfest started to
 address some of the most obvious issues but there is probably much more that
 can be done by looking at what is happening in the other open source
 communities.

Speaking from a cultural, and organizational standpoint:

* Distributed revision control as standard for the project

Using a distributed revision control system, and working from
developer branches which are pulled  pushed directly between
developers is consistently reported to have a positive impact on
solving problems and sharing information. Summer of Code mentors
consistently mention DVCS as having a huge positive impact on
productivity for their students - both because it's easy and cheap for
students to commit to their local repos, and because the merge/pull
tools are intuitive for the mentors and students.

This is how people work and share code out there.  I realize that
Postgres has an established cultural norm of posting patches to the
mailing list, and I'm not suggesting that people stop doing that.

I think more developers should be sharing branches as well.
git.postgresql.org should probably be reorganized to clearly divide
repos between postgresql-core and other projects (contrib, add-ons,
whatever).

We should also publish more information about how our developers are
sharing code with each other through repos.

* Decision making

Is there a project you'd point to as a model for decision making?
(serious question)

This is a volunteer organization, and as such, we're subject to who
has the expertise + time/energy to devote to the patches and projects.
Going back to what Bruce has said about skill drain -- if more
individuals were able to get their time sponsored to devote to the
technical challenges posed by the patches, then it might be possible
to have more coordinated/streamlined responses and decision making.

My experience in FOSS projects is that Postgres does fairly well on
execution compared to other projects (getting a new release out the
door every year, and now the alphas every couple months), but has some
issues with a couple large, important patches. Part of the problem
there appears to be people being stretched too thin, and inconsistent
communication practices.

More involvement from people really focusing on improving
communication between developers would be helpful to make those issues
easier to work around.

* Volunteers, funding and burnout

It's challenging to keep up with the technical discussions on -hackers
even if you are a very knowledgeable developer. If you're only doing
that on your free time, and say, you have a family? Most of the
developers that I know in other projects in leadership/commit manager
roles don't have young children, and very few are women.

So, another thing that we could use are more people like Bruce that
can devote a great deal of *work* time to thinking about and acting on
the issues around community management and recruitment. Robert Haas'
time spent on commitfest, for example, if sponsored by a foundation
would be very helpful. Maybe he could have bought a new dishwasher. :)

Also, we could use an individual dedicated to raising funds and
consistently executing on distributing the money out for non-code
infrastructure projects. Like:

* release management,
* marketing,
* dedicated public relations (driving things like new software release
articles, organizing interviews with press, and getting new case
studies written), and
* keeping track of our relationships with other organizations (Summer
of Code, conferences, commercial forks).

A lot of this work falls on Josh Berkus to coordinate. There are many
more tasks, and some are delegated to others  coordinated on
-advocacy. Again, this is work that could be funded through a
foundation, or as part of positions in particular companies.

IMO, it's about providing incentives that encourage people to use
their time wisely, and to help prevent burnout on the difficult tasks
around people management. And to say definitively that the people
tasks require organization and management, and we as a 

[HACKERS] Check constraint on domain over an array not executed for array literals

2009-11-12 Thread Florian G. Pflug

Hi

While trying to create a domain over an array type to enforce a certain
shape or certain contents of an array (like the array being only
one-dimensional or not containing NULLs), I've stumbled over what I
believe to be a bug in postgresql 8.4

It seems that check constraints on domains are *not* executed for
literals of the domain-over-array-type - in other words, for expressions
like:
array[...]::my-domain-over-array-type.

They are, however, executed if I first force the array to be of the base
type, and then cast it to the array type.

Here is an example that reproduces the problem:

create domain myintarray as int[] check (
  -- Check that the array is neither null, nor empty,
  -- nor multi-dimensional
  (value is not null) and
  (array_length(value,1) is not null) and
  (array_length(value,1)  0) and
  (array_length(value,2) is null)
);

select null::myintarray; -- Fails (Right)

select array[]::myintarray; -- Succeeds (Wrong)
select array[]::int[]::myintarray; -- Fails (Right)

select array[1]::myintarray; -- Succeeds (Right)
select array[1]::int[]::myintarray; -- Succeeds (Right)

select array[array[1]]::myintarray; -- Succeeds (Wrong)
select array[array[1]]::int[][]::myintarray; -- Fails (Right)


I guess the reason is that the ::arraytype part of
array[...]::arraytype isn't really a cast at all, but instead part of
the array literal syntax. Hence, array[]::myintarray probably creates an
empty myintarray instance, and then adds the elements between the square
brackets (none) - with none of this steps triggering a run of the check
constraint.

I still have the feeling that this a bug, though. First, because it
leaves you with no way at guarantee that values of a given domain always
fulfill certain constraints. And second because array[...]::arraytype
at least *looks* like a cast, and hence should behave like one too.

best regards,
Florian Pflug



smime.p7s
Description: S/MIME Cryptographic Signature


Re: [HACKERS] CommitFest 2009-11 Call for Reviewers

2009-11-12 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes:
 The set of patches I have the least feel for are the five ECPG 
 submissions, some of which were reviewed already.  I would particularly 
 appreciate any early information reviewers might provide about their 
 capability/willingness to work on that set.  Those are not so easy to 
 just split among multiple people due to how they relate to one another.

AFAIK the ecpg patches are all waiting on Michael Meskes to have time
to review/commit them.  ecpg is pretty much his turf and no other
committers are likely to touch these patches.  If anyone really wants
to review them, of course, that's fine ... but I don't think you should
assign them to someone just because the CF process says to.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] pg_dump enhancement proposal

2009-11-12 Thread Mark Hammonds

Hello Everyone,

I recently spent some time comparing the various techniques used to  
backup/export MySQL, SQLite, and PostgreSQL databases. So far, pg_dump  
is the best tool I've used. I was well pleased with its efficient  
syntax and functionality. However, I have two feature requests that I  
think can make it even better:


1. View Based Exports

In addition to being able to dump specific tables, it seems useful to  
be able to quickly specify and dump entire views.


2.  Custom Query Exports

In my use of mysqldump, I found one feature very useful: the ability  
to execute a custom SELECT. . .WHERE statement and then dump only the  
results. This feature currently provides MySQL users with the ability  
to quickly and easily export very granular data subsets, and I see no  
reason why PostgreSQL users wouldn't benefit from the same capability.  
While it is true that this functionality can already be achieved in  
PostgreSQL using Copy, it seems to me that it would logically fit well  
as an extension to pg_dump, especially since many beginning and even  
some intermediate PostgreSQL users aren't aware of the alternatives.


If the community finds value in the above features, I would be glad to  
begin developing them for possible inclusion in the next PostgreSQL  
release.


Best Regards,
Mark Hammonds
Web Engineer
OmniTI Computer Consulting
technology.markhammonds.ws

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 1:45 PM, Jeff Davis pg...@j-davis.com wrote:
 On Thu, 2009-11-12 at 11:36 -0500, Robert Haas wrote:
 I agree.  I would quibble only with the details.  I think we should
 require patch authors to act as a reviewer for any CommitFest for
 which they have submitted patches.  We don't need every patch author
 to review as many patches as they submit, because some people will
 review extras, and some non-patch-authors will review.  If they review
 one patch each, that's probably more than enough.  It's also easier
 for bookkeeping purposes.

 Not all contributors are fluent English readers and writers.

 I certainly do not discourage those people from reviewing, but I can
 understand how it might be more frustrating and less productive for
 them. An important part of review is to read the relevant mailing list
 threads and try to tie up loose ends and find a consensus.

Unfortunately, those people's patches also typically require more work
from other community members.  Discussions are longer and more
confused, and someone has to rewrite the documentation and comments
before commit.  If anything, people whose patches require more help
need to find ways to contribute MORE to the community, not less.  I
understand that's not easy, but it's necessary.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] array_to_string bug?

2009-11-12 Thread David E. Wheeler
On Nov 12, 2009, at 10:46 AM, David Fetter wrote:

 My question boils down to, why is this string concatenation different
 from all other string concatenations?

Does it have something to do with afikoman?

David

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] CommitFest 2009-09, two weeks on

2009-11-12 Thread Alvaro Herrera
Boszormenyi Zoltan escribió:
 Alvaro Herrera írta:

  I have applied this patch after some tinkering.  I mainly added support
  for fetch_args: FORWARD opt_from_in name and BACKWARD opt_from_in
  name in ecpg.addons which apparently you forgot.
 
 Thanks. Your fix is correct if this patch is considered
 standalone. This means I have to re-post the later
 patches to fix the reject this fix causes in them.

Yeah.  BTW I don't think the rest of the pieces in this series make
sense to apply separately, because they don't do anything useful by
themselves (one of them introduces an unused function, what good is in
that?).  I think they should be submitted as a single patch.

If you want to submit patches in a series like this one, they need to be
considered standalone, I think.  The Linux kernel devs work differently
than us here.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] CommitFest 2009-11 Call for Reviewers

2009-11-12 Thread Alvaro Herrera
Tom Lane wrote:
 Greg Smith g...@2ndquadrant.com writes:
  The set of patches I have the least feel for are the five ECPG 
  submissions, some of which were reviewed already.  I would particularly 
  appreciate any early information reviewers might provide about their 
  capability/willingness to work on that set.  Those are not so easy to 
  just split among multiple people due to how they relate to one another.
 
 AFAIK the ecpg patches are all waiting on Michael Meskes to have time
 to review/commit them.  ecpg is pretty much his turf and no other
 committers are likely to touch these patches.  If anyone really wants
 to review them, of course, that's fine ... but I don't think you should
 assign them to someone just because the CF process says to.

FWIW I committed the parts of one of these patches that touched the core
grammar mostly, because I think those might have been holding Michael
back a bit.  Hopefully that'll make it easier for him to review the
rest.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] array_to_string bug?

2009-11-12 Thread David Fetter
On Thu, Nov 12, 2009 at 11:46:54AM -0800, David Wheeler wrote:
 On Nov 12, 2009, at 10:46 AM, David Fetter wrote:
 
  My question boils down to, why is this string concatenation
  different from all other string concatenations?
 
 Does it have something to do with afikoman?

I was wondering who'd comment on that first ;)

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Python 3.1 support

2009-11-12 Thread Peter Eisentraut
Here's the patch to support Python =3.1 with PL/Python.  The
compatibility code is mostly in line with the usual 2-3 C porting
practice and is documented inline.

I needed to create an arguably weird hack to manage the regression
tests.  Instead of creating a new expected file for pretty much every
test file and also for some input files (where Python syntax had
changed), a sed script creates a complete Python 3 compatible set of
input and output files.  Doesn't look pretty but works quite well.  If
anyone has a better idea, please let me know.
diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index 373bc79..840b874 100644
--- a/src/pl/plpython/Makefile
+++ b/src/pl/plpython/Makefile
@@ -88,8 +88,32 @@ installdirs: installdirs-lib
 
 uninstall: uninstall-lib
 
+ifneq (,$(findstring 3.,$(python_version)))
+# Adjust regression tests for Python 3 compatibility
+prep3:
+	mkdir -p 3 3/sql 3/expected
+	for file in $(srcdir)/sql/* $(srcdir)/expected/*; do \
+	  sed -r -e 's/except ([[:alpha:].]+), ?(\w+):/except \1 as \2:/g' \
+	 -e s/type 'exceptions\.(\w+)'/class '\1'/g \
+	 -e s/type 'long'/class 'int'/g \
+	 -e s/\b([0-9]+)L\b/\1/g \
+	 -e 's/\bu//g' \
+	 -e s/\bu'/'/g \
+	 -e s/def next\b/def __next__/g \
+	$$file `echo $$file | sed 's,$(srcdir),3,'`; \
+	done
+
+clean3:
+	rm -rf 3/
+
+installcheck: submake prep3
+	cd 3  $(top_builddir)/../src/test/regress/pg_regress --inputdir=. --psqldir=$(PSQLDIR) $(REGRESS_OPTS) $(REGRESS)
+
+clean: clean3
+else
 installcheck: submake
 	$(top_builddir)/src/test/regress/pg_regress --inputdir=$(srcdir) --psqldir=$(PSQLDIR) $(REGRESS_OPTS) $(REGRESS)
+endif
 
 .PHONY: submake
 submake:
diff --git a/src/pl/plpython/expected/README b/src/pl/plpython/expected/README
index 47f31e8..a187937 100644
--- a/src/pl/plpython/expected/README
+++ b/src/pl/plpython/expected/README
@@ -8,3 +8,5 @@ plpython_unicode_0.out		any version, when server encoding != SQL_ASCII and clien
 plpython_unicode_2.out		Python 2.2
 plpython_unicode_3.out		Python 2.3, 2.4
 plpython_unicode_5.out		Python 2.5, 2.6
+
+plpython_types_3.out		Python 3.1
diff --git a/src/pl/plpython/expected/plpython_types_3.out b/src/pl/plpython/expected/plpython_types_3.out
new file mode 100644
index 000..3fcb0f4
--- /dev/null
+++ b/src/pl/plpython/expected/plpython_types_3.out
@@ -0,0 +1,479 @@
+--
+-- Test data type behavior
+--
+--
+-- Base/common types
+--
+CREATE FUNCTION test_type_conversion_bool(x bool) RETURNS bool AS $$
+plpy.info(x, type(x))
+return x
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_bool(true);
+INFO:  (True, class 'bool')
+CONTEXT:  PL/Python function test_type_conversion_bool
+ test_type_conversion_bool 
+---
+ t
+(1 row)
+
+SELECT * FROM test_type_conversion_bool(false);
+INFO:  (False, class 'bool')
+CONTEXT:  PL/Python function test_type_conversion_bool
+ test_type_conversion_bool 
+---
+ f
+(1 row)
+
+SELECT * FROM test_type_conversion_bool(null);
+INFO:  (None, class 'NoneType')
+CONTEXT:  PL/Python function test_type_conversion_bool
+ test_type_conversion_bool 
+---
+ 
+(1 row)
+
+-- test various other ways to express Booleans in Python
+CREATE FUNCTION test_type_conversion_bool_other(n int) RETURNS bool AS $$
+# numbers
+if n == 0:
+   ret = 0
+elif n == 1:
+   ret = 5
+# strings
+elif n == 2:
+   ret = ''
+elif n == 3:
+   ret = 'fa' # true in Python, false in PostgreSQL
+# containers
+elif n == 4:
+   ret = []
+elif n == 5:
+   ret = [0]
+plpy.info(ret, not not ret)
+return ret
+$$ LANGUAGE plpythonu;
+SELECT * FROM test_type_conversion_bool_other(0);
+INFO:  (0, False)
+CONTEXT:  PL/Python function test_type_conversion_bool_other
+ test_type_conversion_bool_other 
+-
+ f
+(1 row)
+
+SELECT * FROM test_type_conversion_bool_other(1);
+INFO:  (5, True)
+CONTEXT:  PL/Python function test_type_conversion_bool_other
+ test_type_conversion_bool_other 
+-
+ t
+(1 row)
+
+SELECT * FROM test_type_conversion_bool_other(2);
+INFO:  ('', False)
+CONTEXT:  PL/Python function test_type_conversion_bool_other
+ test_type_conversion_bool_other 
+-
+ f
+(1 row)
+
+SELECT * FROM test_type_conversion_bool_other(3);
+INFO:  ('fa', True)
+CONTEXT:  PL/Python function test_type_conversion_bool_other
+ test_type_conversion_bool_other 
+-
+ t
+(1 row)
+
+SELECT * FROM test_type_conversion_bool_other(4);
+INFO:  ([], False)
+CONTEXT:  PL/Python function test_type_conversion_bool_other
+ test_type_conversion_bool_other 
+-
+ f
+(1 row)
+
+SELECT * FROM test_type_conversion_bool_other(5);
+INFO:  ([0], True)
+CONTEXT:  PL/Python function test_type_conversion_bool_other
+ test_type_conversion_bool_other 
+-
+ t
+(1 row)
+
+CREATE FUNCTION test_type_conversion_char(x char) RETURNS 

Re: [HACKERS] Patch committers

2009-11-12 Thread Alvaro Herrera
Robert Haas escribió:

 I used to feel this way, too.  I'm not sure whether it's really worse
 at first, or whether it just seems worse a first until you get used to
 it.  There is no getting around the fact that this is a community of
 very smart people.  I work at a company where I'm the only developer.
 Guess what - my development ideas are all brilliant - or at least
 there's no contrary evidence!  Then I come here, where there are
 plenty of other developers just as smart as I am, or possibly smarter,
 and here some of my ideas crash and burn.  On the other hand,
 sometimes someone objects to my idea and the *objections* crash and
 burn.  You can't expect to win them all when you're dealing with
 people who are as smart as you and more knowledgeable about the code.

Maybe this paragraph (and some others from this email and Josh's) should
be part of the how to submit a patch wiki page.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread Andreas 'ads' Scherbaum
On Tue, 03 Nov 2009 10:32:17 -0800 Josh Berkus wrote:

 The same goes for other OSS projects.  There's quite a few random OSS
 apps which were created on PG 7.4 and have never offered their users an
 upgrade path (Gnuworld comes to mind).  They need an EOL announcement to
 get them motivated to upgrade.

I know several customers who decided to move from 7.3 only after the
EOL was announced. If 7.3 would not has see an EOL, they would never
ever have moved to a newer version.



 We'd want to do a full publicity around this, including a how do I
 upgrade page and an what does EOL mean for an OSS project page.  If
 this goes well, we could EOL 8.0 after 8.5 comes out, and thus decrease
 our maintenance burden.

+1

-- 
Andreas 'ads' Scherbaum
German PostgreSQL User Group
European PostgreSQL User Group - Board of Directors
Volunteer Regional Contact, Germany - PostgreSQL Project

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] CommitFest 2009-09, two weeks on

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 2:49 PM, Alvaro Herrera
alvhe...@commandprompt.com wrote:
 Boszormenyi Zoltan escribió:
 Alvaro Herrera írta:

  I have applied this patch after some tinkering.  I mainly added support
  for fetch_args: FORWARD opt_from_in name and BACKWARD opt_from_in
  name in ecpg.addons which apparently you forgot.

 Thanks. Your fix is correct if this patch is considered
 standalone. This means I have to re-post the later
 patches to fix the reject this fix causes in them.

 Yeah.  BTW I don't think the rest of the pieces in this series make
 sense to apply separately, because they don't do anything useful by
 themselves (one of them introduces an unused function, what good is in
 that?).  I think they should be submitted as a single patch.

 If you want to submit patches in a series like this one, they need to be
 considered standalone, I think.  The Linux kernel devs work differently
 than us here.

Zoltan broke them up because Michael asked him to do so.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Jeff Davis
On Thu, 2009-11-12 at 14:43 -0500, Robert Haas wrote:
  Not all contributors are fluent English readers and writers.
 
  I certainly do not discourage those people from reviewing, but I can
  understand how it might be more frustrating and less productive for
  them. An important part of review is to read the relevant mailing list
  threads and try to tie up loose ends and find a consensus.
 
 Unfortunately, those people's patches also typically require more work
 from other community members.  Discussions are longer and more
 confused, and someone has to rewrite the documentation and comments
 before commit.  If anything, people whose patches require more help
 need to find ways to contribute MORE to the community, not less.  I
 understand that's not easy, but it's necessary.

Keep in mind that many of these people may also be regional contacts,
translators, local conference (or user group) organizers, and provide
support to users in languages other than English. They may even organize
development efforts in languages other than English.

I think the best policies encourage people to help in the ways that they
are most effective, and/or enjoy the most. I know that's a general
statement, and it doesn't translate (excuse the pun) easily into
policy. 

I don't have a good answer for that. But sometimes the policy
discussions here seem a little abstract, and I have a hard time seeing
how they apply in real situations. It seems like you have a few
freeloaders in mind, but I have a hard time imagining it's more than a
couple people. Maybe they just need a nice reminder to be a more helpful
community member if they want timely feedback on their work?

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread Andrew Dunstan



Andreas 'ads' Scherbaum wrote:

On Tue, 03 Nov 2009 10:32:17 -0800 Josh Berkus wrote:

  

The same goes for other OSS projects.  There's quite a few random OSS
apps which were created on PG 7.4 and have never offered their users an
upgrade path (Gnuworld comes to mind).  They need an EOL announcement to
get them motivated to upgrade.



I know several customers who decided to move from 7.3 only after the
EOL was announced. If 7.3 would not has see an EOL, they would never
ever have moved to a newer version.
  



Nobody that I have seen is arguing against EOLing 7.4.



  

We'd want to do a full publicity around this, including a how do I
upgrade page and an what does EOL mean for an OSS project page.  If
this goes well, we could EOL 8.0 after 8.5 comes out, and thus decrease
our maintenance burden.



+1
  


The only burden of significance I have seen actually mentioned, as 
opposed to supposed, is from Dave Page.


What I and others have been arguing is necessary to do EOL right is a 
serious amount of notice, by way of press releases etc. We can't expect 
users to keep polling our web site to see if there's an EOL. That means 
we need to prepare for an EOL months or a year in advance, ISTM.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread Josh Berkus

 I think that's the key argument here. We have several customers, which
 need a very careful and time consuming evaluation before they can go
 into production with a new platform, which is quite time consuming and
 needs significant preparation for them. Announcing an EOL early in time
 would give them the required time before the version used disappears.

So, should we announce it for June?

--Josh Berkus


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] cvs head doesn't pass make check on one of the machines here

2009-11-12 Thread Grzegorz Jaskiewicz
consistently fails when compiled on ubuntu 9.10 here (on mini 10v). 



regression.diffs.gz
Description: GNU Zip compressed data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER TABLE...ALTER COLUMN vs inheritance

2009-11-12 Thread Tom Lane
Bernd Helmle maili...@oopsware.de writes:
 My first idea is to just introduce a special contype in pg_constraint 
 representing a NOT NULL constraint on a column, which holds all required 
 information to do the mentioned maintenance stuff on them and to keep most 
 of the current infrastructure. Utility commands need to track all changes 
 in pg_constraint and keep pg_attribute.attnotnull up to date.

 Another possibility is to change the representation of NOT NULL to be a 
 CHECK constraint (e.g. CHECK(col IS NOT NULL)) internally and leave all the 
 responsibility up to the current existing check constraint infrastructure 
 (which already does the right thing for inheritance, e.g. it's not possible 
 to drop such a constraint if it was inherited).

I'd go for the first of those, for sure.  Testing attnotnull is
significantly cheaper than enforcing a generic constraint expression,
and NOT NULL is a sufficiently common case to be worth worrying about
optimizing it.  Furthermore, removing attnotnull would break an unknown
but probably not-negligible amount of client-side code that cares
whether columns are known not null (I think both JDBC and ODBC track
that).  And it would significantly complicate code in the backend that
wants to determine whether a column is known not null --- there are
several proposals for planner optimizations that would depend on knowing
that, for example.

You will find yourself copying-and-pasting a fair amount of the
check-constraint code if you implement this as a completely separate
code path, and so it might be worthwhile to be creative about exactly
what the pg_constraint representations look like --- maybe you can
design it to share some of that code.  But I recommend strongly that
attnotnull stay there.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Python 3.1 support

2009-11-12 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 Here's the patch to support Python =3.1 with PL/Python.  The
 compatibility code is mostly in line with the usual 2-3 C porting
 practice and is documented inline.

There was considerable debate earlier about whether we wanted to treat
Python 3 as a separate PL so it could be available in parallel with
plpython 2, because of the user-level coding incompatibilities.  It
looks like this patch simply ignores that problem.  What is going to
happen to plpython functions that depend on 2.x behavior?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] next CommitFest

2009-11-12 Thread Robert Haas
On Thu, Nov 12, 2009 at 3:12 PM, Jeff Davis pg...@j-davis.com wrote:
 On Thu, 2009-11-12 at 14:43 -0500, Robert Haas wrote:
  Not all contributors are fluent English readers and writers.
 
  I certainly do not discourage those people from reviewing, but I can
  understand how it might be more frustrating and less productive for
  them. An important part of review is to read the relevant mailing list
  threads and try to tie up loose ends and find a consensus.

 Unfortunately, those people's patches also typically require more work
 from other community members.  Discussions are longer and more
 confused, and someone has to rewrite the documentation and comments
 before commit.  If anything, people whose patches require more help
 need to find ways to contribute MORE to the community, not less.  I
 understand that's not easy, but it's necessary.

 Keep in mind that many of these people may also be regional contacts,
 translators, local conference (or user group) organizers, and provide
 support to users in languages other than English. They may even organize
 development efforts in languages other than English.

They may, but it's beyond my power to keep track of everything that a
person may or may not do.  If we're going to have any chance of
enforcing a policy, it has to be simple and clear.

 I don't have a good answer for that. But sometimes the policy
 discussions here seem a little abstract, and I have a hard time seeing
 how they apply in real situations. It seems like you have a few
 freeloaders in mind, but I have a hard time imagining it's more than a
 couple people. Maybe they just need a nice reminder to be a more helpful
 community member if they want timely feedback on their work?

Well, you can go back and look at the list of people who had patches
reviewed last CF (see commitfest.postgresql.org), and then go back and
look at the reviewers (see pgsql-rrreviewers archives).  It's all
public data.  It's not 1 or 2 people - I count at least 8.  I could
cross-reference for you and list the names, but that seems like
embarassing those people without accomplishing anything useful.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] cvs head doesn't pass make check on one of the machines here

2009-11-12 Thread Andrew Dunstan



Grzegorz Jaskiewicz wrote:
consistently fails when compiled on ubuntu 9.10 here (on mini 10v). 
  



+ ERROR:  incompatible library 
/home/kgrittn/postgresql-8.4.0/src/test/regress/refint.so: version 
mismatch

+ DETAIL:  Server is version 8.4, library is version 8.3.


You might take this as a clue as to what you're doing wrong ...

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] cvs head doesn't pass make check on one of the machines here

2009-11-12 Thread Tom Lane
Grzegorz Jaskiewicz g...@pointblue.com.pl writes:
 consistently fails when compiled on ubuntu 9.10 here (on mini 10v). 

Locale issue maybe?  Looks like it has some odd ideas about text
sort order.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] CommitFest 2009-11 Call for Reviewers

2009-11-12 Thread Greg Smith

Tom Lane wrote:

AFAIK the ecpg patches are all waiting on Michael Meskes to have time
to review/commit them.  ecpg is pretty much his turf and no other
committers are likely to touch these patches.
Great to know, and since some of the regular reviewers already made a 
pass through them there's probably not too much general feedback left 
anyway.  I just marked all of those as having Michael as the reviewer.  
If it gets to where those are the main remaining hold-up I guess we'll 
revisit who else might help out then.  Would rather get the patches it's 
more obvious how to handle out of the way first.


Not considering those, HS/SR, or other patches with an already assigned 
reviewer, we're at 16 patches in the queue, and I've got 9 reviewer 
volunteers just so far today.  Barring a flood of last-minute entries, 
if I can get each reviewer to handle one patch and a moderate percentage 
of them to handle two, that should be all it takes for this round.  Will 
move the rest of the discussion here to just rrreviewers.


--
Greg Smith2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com  www.2ndQuadrant.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] pg_dump enhancement proposal

2009-11-12 Thread Tom Lane
Mark Hammonds mhammo...@omniti.com writes:
 1. View Based Exports

 In addition to being able to dump specific tables, it seems useful to  
 be able to quickly specify and dump entire views.

Isn't this pretty much the same thing as your #2?  A view in PG isn't
materialized, it's nothing but a custom SELECT.

 2.  Custom Query Exports

 In my use of mysqldump, I found one feature very useful: the ability  
 to execute a custom SELECT. . .WHERE statement and then dump only the  
 results. This feature currently provides MySQL users with the ability  
 to quickly and easily export very granular data subsets, and I see no  
 reason why PostgreSQL users wouldn't benefit from the same capability.  
 While it is true that this functionality can already be achieved in  
 PostgreSQL using Copy, it seems to me that it would logically fit well  
 as an extension to pg_dump, especially since many beginning and even  
 some intermediate PostgreSQL users aren't aware of the alternatives.

As you say, we already have this using COPY, and I don't agree that
it would be a good idea to plaster it into pg_dump as well.  pg_dump
is intended for dumping and restoring data, not for ETL-type tasks.
Furthermore, pg_dump is a overly complex beast already --- much more
so than one could wish, for a tool that is absolutely fundamental to
database reliability.  Putting requirements on it that are well outside
its charter seems like a short route to maintenance disaster.

There has been some occasional chatter about developing one or more
tools focused on ETL rather than dump/restore, and my thought is that
this idea would fit better there.  An ETL tool would not have the
kind of requirements pg_dump has for coping with multiple server
versions and knowing everything there is to know about database
contents, so it seems like it could address new areas of functionality
without a complexity explosion.

You might want to check the archives for previous discussions ---
I think the last go-round started with someone wanting to add an
arbitrary WHERE filter to pg_dump dumps.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread Andreas 'ads' Scherbaum
On Thu, 12 Nov 2009 15:23:06 -0500 Andrew Dunstan wrote:

 Andreas 'ads' Scherbaum wrote:
  On Tue, 03 Nov 2009 10:32:17 -0800 Josh Berkus wrote:
 

  The same goes for other OSS projects.  There's quite a few random OSS
  apps which were created on PG 7.4 and have never offered their users an
  upgrade path (Gnuworld comes to mind).  They need an EOL announcement to
  get them motivated to upgrade.
  
 
  I know several customers who decided to move from 7.3 only after the
  EOL was announced. If 7.3 would not has see an EOL, they would never
  ever have moved to a newer version.

 
 
 Nobody that I have seen is arguing against EOLing 7.4.

True. But as Josh pointed out: some people/projects/companies need
more motivation to actually consider an upgrade at all.



 What I and others have been arguing is necessary to do EOL right is a 
 serious amount of notice, by way of press releases etc. We can't expect 
 users to keep polling our web site to see if there's an EOL. That means 
 we need to prepare for an EOL months or a year in advance, ISTM.

Months. The software will not stop working once we announced the EOL.
And yes, i'm +1 for having a rule for EOL, like 5 versions are
supported.



Bye

-- 
Andreas 'ads' Scherbaum
German PostgreSQL User Group
European PostgreSQL User Group - Board of Directors
Volunteer Regional Contact, Germany - PostgreSQL Project

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] plpgsql GUC variable: custom or built-in?

2009-11-12 Thread Alvaro Herrera
Tom Lane escribió:
 I wrote:
  Fair enough.  I'll start writing the custom GUC then.
 
 While testing that, I noticed [...]

With all this fallout, I think it would be cleaner to step back and make
it a regular GUC variable, not custom.  Pretending that plpgsql is
somehow not an integral part of the system is not completely true
anyway.  Yes, we're playing favorites in the PL camp here, but so what?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] EOL for 7.4?

2009-11-12 Thread Joshua D. Drake
On Thu, 2009-11-12 at 23:09 +0100, Andreas 'ads' Scherbaum wrote:
 On Thu, 12 Nov 2009 15:23:06 -0500 Andrew Dunstan wrote:
 
  Andreas 'ads' Scherbaum wrote:
   On Tue, 03 Nov 2009 10:32:17 -0800 Josh Berkus wrote:
  
 
   The same goes for other OSS projects.  There's quite a few random OSS
   apps which were created on PG 7.4 and have never offered their users an
   upgrade path (Gnuworld comes to mind).  They need an EOL announcement to
   get them motivated to upgrade.
   
  
   I know several customers who decided to move from 7.3 only after the
   EOL was announced. If 7.3 would not has see an EOL, they would never
   ever have moved to a newer version.
 
  
  
  Nobody that I have seen is arguing against EOLing 7.4.
 
 True. But as Josh pointed out: some people/projects/companies need
 more motivation to actually consider an upgrade at all.

We have discussed in the past EOLing 7.4 I thought at the end of this
year. IMO 7.4 and 8.0 both need to be EOL. Can we just set a date and
call it good? March 31st sounds good.

Let's write up a quick announcement, add the letters EOL to the download
pages and call it good.

Joshua D. Drake


-- 
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering
If the world pushes look it in the eye and GRR. Then push back harder. - 
Salamander


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] plpgsql GUC variable: custom or built-in?

2009-11-12 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes:
 With all this fallout, I think it would be cleaner to step back and make
 it a regular GUC variable, not custom.  Pretending that plpgsql is
 somehow not an integral part of the system is not completely true
 anyway.  Yes, we're playing favorites in the PL camp here, but so what?

True, but on the other hand, if plpgsql needs this to work nicely, then
$YOURPL probably needs it too.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER TABLE...ALTER COLUMN vs inheritance

2009-11-12 Thread Alex Hunsaker
On Thu, Nov 12, 2009 at 13:55, Tom Lane t...@sss.pgh.pa.us wrote:

 I'd go for the first of those, for sure.  Testing attnotnull is
 significantly cheaper than enforcing a generic constraint expression,
 and NOT NULL is a sufficiently common case to be worth worrying about
 optimizing it.

When I looked at doing this, I thought about just using check
constraints just for the book keeping and leaving attnotnull as it is.
 If would be easier, but it seemed quite ugly.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Patch committers

2009-11-12 Thread Kevin Grittner
On the subject topic, I have to say that I don't see where Robert
hasn't met the qualifications mentioned so far on this thread as
required to promote someone to the committer level; are there some
requirements which exist but haven't been mentioned?
 
Regarding the specific issues below, which seem to be more about the
related topic of drawing in new contributors...
 
Robert Haas robertmh...@gmail.com wrote:
 Josh Berkus j...@agliodbs.com wrote:

 That's basically just it:  Assume bashing is part of the process. 
 
 I've heard this complaint from numerous first-time contributors,
 and have seen several talented people walk away from our project
 because of it.  Even people who have stuck with us, such as Josh
 Tolley, have remarked on the hazing ritual inherent in getting a
 first-time contribution in.
 
 I used to feel this way, too.  I'm not sure whether it's really
 worse at first, or whether it just seems worse a first until you get
 used to it.
 
I would say that I don't often see anything I would consider as
bashing, but that I have seen newcomers who appeared to want to help
contribute to the project handled with less finesse than one could
perhaps expect.  It seems to me that those needing assistance
(especially if they are in dire straights or are trying to convert to
PostgreSQL) are treated very gently.  Someone with a question, if it
isn't something too obviously documented in the FAQ or the online
manual, is given a polite and informative response.  The responses get
pretty blunt if the person offers suggestions which show that the
person has (or assumes) technical competence, when those suggestions
aren't acceptable to the community.  At that point it takes a pretty
thick skin to find some other way to contribute, rather than just
walking away.  Again, I wouldn't call it bashing, but a level of
bluntness which could be disconcerting if the person hasn't followed
these lists long enough to understand how it is intended.
 
 When someone submits a bad patch and I review it, they sometimes get
 two or three pages of feedback detailing inadequacies of various
 types.
 
When done in a professional manner, which is typical of this list, I
don't think there's any reasonable alternative.  I don't see that as a
problem.
 
Well, there is one thing which must be frustrating.  If I was seated
near others who read this list, I would start a pool every time a
newcomer submitted a large patch -- how long until someone tells them
to break it into smaller patches, and how long until someone else
tells them it makes no sense separated like that and should be put
back together?
 
 the only possible solution is for each new contributor to have a
 mentor who sticks with them throughout the process of getting their
 first contribution accepted
 
I don't know how common it is, but I felt that Bruce did that for me
when I worked on the standard_conforming_strings TODO item for 8.2. 
(Perhaps part of the reason was that it was urgently needed to address
problems we found in production after we converted a critical database
to PostgreSQL; but whatever the reason, I was grateful for the help.) 
Having gone through that experience, I feel confident that such
assistance for first-time contributors can make a difference both in
the success of their efforts (which will then, presumably, benefit the
community in general) and in their willingness to contribute in other
ways.
 
 It would also help if people on this list *in general*, were to be
 a bit more consistent about phrasing criticism as constructive
 criticism.  I've seen far too much how stupid are you? on this
 list for the good health of our developer community.
 
 I don't see very much of this at all, so maybe this is another area
 where it boils down to perspective.
 
I don't see much of that either, although describing another's
statement as nonsense or something similar isn't unheard of.  It's
not usually an unfair characterization, but some could take such
bluntness as hostility.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] ALTER TABLE...ALTER COLUMN vs inheritance

2009-11-12 Thread Tom Lane
Alex Hunsaker bada...@gmail.com writes:
 On Thu, Nov 12, 2009 at 13:55, Tom Lane t...@sss.pgh.pa.us wrote:
 I'd go for the first of those, for sure.  Testing attnotnull is
 significantly cheaper than enforcing a generic constraint expression,
 and NOT NULL is a sufficiently common case to be worth worrying about
 optimizing it.

 When I looked at doing this, I thought about just using check
 constraints just for the book keeping and leaving attnotnull as it is.

Yeah, you could definitely attack it like that.  The code that fixes up
attnotnull would have to look for check constraints that look like foo
NOT NULL rather than something more instantly recognizable, but
presumably ALTER TABLE is not a performance-critical path.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] actualised funcs typmod patch

2009-11-12 Thread Alvaro Herrera
Pavel Stehule escribió:
 Hello
 
 actualised version: enhance check inside sql function

What is this against?  It has a few suspicious chunks, such as

***
*** 358,364 
  
funcexpr-funcid = funcid;
funcexpr-funcresulttype = rettype;
!   funcexpr-funcresulttypmod = -1;  // TODO STEHULE
funcexpr-funcretset = retset;
funcexpr-funcformat = COERCE_EXPLICIT_CALL;
funcexpr-args = fargs;
--- 365,371 
  
funcexpr-funcid = funcid;
funcexpr-funcresulttype = rettype;
!   funcexpr-funcresulttypmod = rettypmod;
funcexpr-funcretset = retset;
funcexpr-funcformat = COERCE_EXPLICIT_CALL;
funcexpr-args = fargs;



This ended up in ./src/backend/parser/parse_func.c.rej

There are other rejects and I'm not sure if I should fix them (at least
the first couple look trivial, though it's against a bunch of code
that last changed in Aug. last year) or ask for a better version ...

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


  1   2   >