Re: [HACKERS] pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Tom Lane]

Alvaro Herrera  writes:
> The reason this is like this is that the indent binary modifies the
> prototype exactly like the function definition, and then the awk program
> that's used in the pipeline "pulls up" the second line:

> #  Move prototype names to the same line as return type.  Useful for ctags. 
> #  Indent should do this, but it does not.  It formats prototypes just
> #  like real functions.

> In this day and age there's probably no reason to do this.

Um, sorry, no reason to do which?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] TODO: fix priority of ordering of read and write light-weight locks

[Tom Lane]

Jeff Janes  writes:
> The wiki todo has the topic "Fix priority ordering of read and write
> light-weight locks" and
> references http://archives.postgresql.org/pgsql-hackers/2004-11/msg00893.php
> (lwlocks and starvation)

> Having read the referenced discussion, I'm trying to figure out what
> remains to be done.

AFAIR that patch hasn't been applied because nobody's demonstrated
an overall win from changing it.  It's not only a matter of showing
an improvement for some one workload, but providing some confidence
that no other case gets a lot worse.  If you go back to circa 2000
or 2001 to see the previous iteration of tweaking the lock algorithms,
you'll see that we found that it's important to avoid unnecessary
trips through the kernel scheduler.  If we force a incoming shared
requestor to block, there's a very real chance that the overhead from
descheduling and later rescheduling him will mean a net performance
degradation regardless of any other benefits.  As against this there's
the risk of long-term starvation of an exclusive requestor --- but
there is little if any evidence that that's a serious problem in
practice.

Just to make things more interesting, the context has changed a lot
since 2000-2001 --- far more people have multi-CPU machines now.
So it's possible the tradeoffs have changed.

> Since lwlock covers a rather heterogeneous bunch of lock purposes, it
> seems unlikely to me any one strategy is going to be applicable to all
> of those purposes if extreme optimization is what we are after .  How
> much are we willing to sacrifice modularity and abstraction in order
> to get a little extra performance out of things protected by lwlock?

My answer is "not a lot, unless it's a *lot* of extra performance".
We've talked sometimes about having more than one type of LWLock
to address different scheduling needs, but nobody's come up with
evidence that that'd really be helpful.

Also, did you see this thread:
http://archives.postgresql.org/pgsql-performance/2009-03/msg00104.php
That proposal was bounced because it seemed likely to hurt in a
much wider set of cases than it helped (see extra-scheduling-overhead
argument).  But I'd still be interested to see an unbiased analysis
of what was going on in the case where it seemed to help.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Alvaro Herrera]

Tom Lane escribió:

> Ah.  That's a bit idiosyncratic to pgindent.  What it does for a
> function definition makes sense, I think: it lines up all the
> parameters to start in the same column:
> 
> static int
> myfunction(int foo,
>int bar)
> 
> What is not obvious is that the same amount of hanging indent is
> used for the function's *prototype*, even though the first line
> is not the same:
> 
> static int myfunction(int foo,
>int bar);
> 
> ie, the indent is length of function name plus 1.  I find this a bit
> stupid myself 

The reason this is like this is that the indent binary modifies the
prototype exactly like the function definition, and then the awk program
that's used in the pipeline "pulls up" the second line:

#  Move prototype names to the same line as return type.  Useful for ctags. 
#  Indent should do this, but it does not.  It formats prototypes just
#  like real functions.

In this day and age there's probably no reason to do this.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Mark Mielke]

On 08/11/2009 11:19 PM, Robert Haas wrote:

On Tue, Aug 11, 2009 at 9:44 PM, Greg Stark  wrote:
   

No! This is *not* what "hot standby" means, at least not in the Oracle world.
 


I'm perplexed by this.  For example: http://en.wikipedia.org/wiki/Hot_standby

Admittedly, wikipedia is not an authoritative source, but I've always
understood cold/warm/hot just as Peter described them upthread.  Cold
means it's on the shelf.  Warm means it's plugged in, but you have to
have to do something to get it going.  Hot means it just takes over
when needed.

But of course I guess Oracle can call their features what they want to...
   


To further confuse things, the "temperature" might apply to only a 
particular aspect of the solution. For example, "hot swappable disk 
drives" are drives that probably do sit on a shelf until they are 
needed, and the "hot" aspect only implies that the server does not need 
to be shut down to install the drives, and initialize them for service. :-)


For databases, people seem to be associating "hot" with the ability to 
issue read only queries. As somebody else said - under a definition of 
hot that includes read-only clones, pg_dump/pg_restore could be 
considered a "hot standby" strategy.


I don't agree with that definition. For the clone to be able to perform 
read-only queries does not imply "hot" nor does it imply "standby". It 
implies "slave". The original poster correctly raised this concern.


For myself, I associate "hot" to mean "ready to replace the master", 
which implies that the data is up-to-date or nearly up-to-date, and 
implies that the server is within a bit toggle of accepting mastership 
of the data and serving ALL queries that the master would serve. Key to 
this is "nearly up-to-date" (requires some sort of streaming) AND "ALL 
queries" (not just read queries!).


If the server happens to be able to do read queries while it is waiting 
in standby more - that's convenient and could be useful to somebody, but 
that's not the value of a "hot standby" - that's the value of a 
"read-only slave". The feature being provided is not "hot standby".


In the case of Oracle, I believe their "hot standby" provides the nearly 
up-to-date, and the capability to switch over, which satisfies my 
requirements. It might *also* allow read-only slave for the standby, but 
that's just convenience - it's not really part of the definition or 
expectation of "hot standby".




"log based replication", "read-only slaves", and "hot standby" are all
100% accurate descriptions of what the hot standby patch enables. I do
like "read only slaves" because it's the most precise and meaningful.
 


Me too.
   


Read only slave works for me.

Cheers,
mark

--
Mark Mielke

Re: [HACKERS] pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Tom Lane]

Robert Haas  writes:
> On Tue, Aug 11, 2009 at 10:08 PM, Tom Lane wrote:
>> If that's not it, you'd need to mention details.

> Well, one thing I've noticed is that when a function prototype wraps
> around to the next line, you often change the number of spaces in the
> hanging indent.

Ah.  That's a bit idiosyncratic to pgindent.  What it does for a
function definition makes sense, I think: it lines up all the
parameters to start in the same column:

static int
myfunction(int foo,
   int bar)

What is not obvious is that the same amount of hanging indent is
used for the function's *prototype*, even though the first line
is not the same:

static int myfunction(int foo,
   int bar);

ie, the indent is length of function name plus 1.  I find this a bit
stupid myself and would prefer that the prototypes looked like this:

static int myfunction(int foo,
  int bar);

which is what my editor will do with it if left to its own devices.
So depending on whether I had occasion to touch the prototype, you might
see it get reindented to the second style.  (I know pgindent will change
it again, but I'm not sufficiently anal to override Emacs here.)  Also,
I tend to make sure that there are not so many parameters declared on
the same line that it would look bad if pgindent ever starts doing
things my way, ie, if I see

static someveryverylongtypename *somefunctionname(int foo,
 int bar, int baz, int quux, int somethingelse, int more);

I'll probably split the second line, even though there's room for it
according to pgindent's current rule.

So, yeah, I'm being a bit anal about this and yet I'm not always
consistent.  But that's what pgindent is good at fixing ...

> I am also a bit confused about the rule for indenting the variables in
> variable declarations.  I get that the *, if any, is suppose to hang
> to the left of the name, so that the first character of each variable
> name lines up - and in particular, line up on a tab stop.  But it's
> not obvious to me how to choose which tab stop to line everything up
> on.

I'm not too sure either.  It's the fourth tab stop for declarations at
the outer level of a function, but I've never entirely figured out what
pgindent's rule is for nested declarations.  It seems to be willing to
move the target column over to some extent, but eventually it gives up
and doesn't add any more whitespace.  I don't usually try very hard to
align declarations exactly where pgindent will put them, although if
it's visually ragged as-submitted sometimes I'll fix that.

One thing I frequently *do* do is remove submitted whitespace changes
if I know that pgindent would undo them anyway, just to reduce the size
of the final diff.  A case that happened at least once in this
commitfest is that somebody had taken a block of variables aligned to
the fourth tab stop:

int foo;
int bar;
int baz;

and realigned them to the fifth stop, because he was adding another
variable with a longer type name:

int foo;
int bar;
int baz;
longtypename   *ptr;

Well, that looks nice, but pgindent is going to do this anyway:

int foo;
int bar;
int baz;
longtypename *ptr;

so I tend to revert such dead-end whitespace changes just as part of
making sure there wasn't an unintentional change of the existing lines.
(This is the sort of thing that having submitters pgindent beforehand
might avoid ...)

Bored yet?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Robert Haas]

On Tue, Aug 11, 2009 at 9:44 PM, Greg Stark wrote:
> On Tue, Aug 11, 2009 at 10:13 PM, Robert Haas wrote:
>> On Tue, Aug 11, 2009 at 4:07 PM, Josh Berkus wrote:
>>> So really, the "streaming replication" patch should be called "hot
>>> standby",
>>
>> No.  AIUI, hot standby means that when your primary falls over, the
>> secondary automatically promotes itself and takes over.
>
> No! This is *not* what "hot standby" means, at least not in the Oracle world.

I'm perplexed by this.  For example: http://en.wikipedia.org/wiki/Hot_standby

Admittedly, wikipedia is not an authoritative source, but I've always
understood cold/warm/hot just as Peter described them upthread.  Cold
means it's on the shelf.  Warm means it's plugged in, but you have to
have to do something to get it going.  Hot means it just takes over
when needed.

But of course I guess Oracle can call their features what they want to...

> "log based replication", "read-only slaves", and "hot standby" are all
> 100% accurate descriptions of what the hot standby patch enables. I do
> like "read only slaves" because it's the most precise and meaningful.

Me too.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] TODO: fix priority of ordering of read and write light-weight locks

[Jeff Janes]

The wiki todo has the topic "Fix priority ordering of read and write
light-weight locks" and
references http://archives.postgresql.org/pgsql-hackers/2004-11/msg00893.php
(lwlocks and starvation)

Having read the referenced discussion, I'm trying to figure out what
remains to be done.  Tom proposed a patch back in 2004, which still
seems rather applicable today, which would correct the "new Shared
request trumps queued Exclusive waiter" problem except for not fixing
it during a race condition.  It seems that this patch was not pursued
because no one thought it was evident that it would actually make
things better (is Exclusive Waiter starvation actually a documentable
problem?), while the ways it could make things worse are obvious
(adding a context switches, often to little end).

So what is waiting to be done?

Do we want a fix that doesn't suffer the race condition (a waiter has
been removed from the queue
and signaled, but has not yet been dispatched to the CPU)?  It is hard
to imagine one of those that is not worse than the disease.

Are we looking for empirical evidence that the proposed patch is
actually an improvement for at least one plausible workload?  In which
case, does anyone have suggestions about what such a work-load might
look like?

Since lwlock covers a rather heterogeneous bunch of lock purposes, it
seems unlikely to me any one strategy is going to be applicable to all
of those purposes if extreme optimization is what we are after .  How
much are we willing to sacrifice modularity and abstraction in order
to get a little extra performance out of things protected by lwlock?

Cheers,

Jeff

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Robert Haas]

On Tue, Aug 11, 2009 at 10:08 PM, Tom Lane wrote:
> Robert Haas  writes:
>> What is a bit frustrating to me is that a number of Tom's changes to
>> the first two patches were trivial whitespace changes that required me
>> to rebase for no obvious reason.   Either those changes were made
>> accidentally as Tom was fooling around with what I had done, or they
>> were made because Tom had some reason to believe that they would play
>> more nicely with pgindent, though what those reasons may have been is
>> entirely opaque to me.
>
> I tend to do some manual adjustment to pgindent rules in places where
> I feel it makes the code more readable.  (Bear in mind that I've been
> looking at the Postgres code base for long enough that large variations
> from pgindent style are automatically less readable for me...)  If I'd
> realized you had followon patches touching the same spots I would've
> possibly refrained from that.  Although the nearby suggestions that we
> should run pgindent more often would render it moot ;-).
>
> If that's not it, you'd need to mention details.

Well, one thing I've noticed is that when a function prototype wraps
around to the next line, you often change the number of spaces in the
hanging indent.  I would do them the way that you want them if I could
figure out what the rule was, but I can't.  When I asked before, you
said that you let emacs do what it wants, but with apologies for using
the Other Editor, that doesn't actually help me very much.

I am also a bit confused about the rule for indenting the variables in
variable declarations.  I get that the *, if any, is suppose to hang
to the left of the name, so that the first character of each variable
name lines up - and in particular, line up on a tab stop.  But it's
not obvious to me how to choose which tab stop to line everything up
on.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Tom Lane]

Robert Haas  writes:
> What is a bit frustrating to me is that a number of Tom's changes to
> the first two patches were trivial whitespace changes that required me
> to rebase for no obvious reason.   Either those changes were made
> accidentally as Tom was fooling around with what I had done, or they
> were made because Tom had some reason to believe that they would play
> more nicely with pgindent, though what those reasons may have been is
> entirely opaque to me.

I tend to do some manual adjustment to pgindent rules in places where
I feel it makes the code more readable.  (Bear in mind that I've been
looking at the Postgres code base for long enough that large variations
from pgindent style are automatically less readable for me...)  If I'd
realized you had followon patches touching the same spots I would've
possibly refrained from that.  Although the nearby suggestions that we
should run pgindent more often would render it moot ;-).

If that's not it, you'd need to mention details.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Greg Stark]

As I see it we potentially have the following modes to deal with:

Archive file mode asynchronous  archive
Archive file mode asynchronous  standby slave
Streaming mode  asynchronous  standby slave
Streaming mode  synchronousstandby slave
Archive file mode asynchronous  read-only slave
Streaming mode  asynchronous  read-only slave
Streaming mode  synchronousread-only slave

By "standby slave" I mean a server which is constantly in recovery
mode but isn't open for connections. This is what we (and Oracle) have
been calling a warm standby. By "read-only slave" I mean a server
which is open for connections and can handle read-only queries, which
as I mentioned Oracle calls a hot standby. Note that *all* of these
are "log-based replication".

I'm not actually certain we can handle streaming synchronous mode to a
standby slave. Does the slave need to have connections enabled to
handle feeding wal sync status to the master?

I don't see any particular reason to come up with names for each of
these combination of modes. I'm pretty happy just saying there are
three different configuration options and certain options depend on
other options so only certain combinations are legal.

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: getting rid of the pg_database flat file

[Tom Lane]

Alvaro Herrera  writes:
> Tom Lane wrote:
>> To actually get rid of the pg_database flat file, we'd need to take the
>> further step of teaching the AV launcher to read pg_database for itself,
>> or else refactor things so that the AV workers can do that for it.
>> (Alvaro, any comments about the best way to proceed there?)

> Hmm.  I don't see any easy way out of that at the moment ... the
> launcher would have to become a pseudo-backend, at least to the point
> where it is able to read pg_database.  I don't see how could workers
> help the launcher with that, unless we made them write a flatfile
> representation of pg_database, which would put us back where we started ...

> I'll have a deeper look around.

What was sort of in the back of my mind was to have every n'th AV worker
examine pg_database and report back to the launcher (probably through
shared memory) with an indication of the next few databases that should
be vacuumed and when.  Not sure how inefficient that might be though.
Is there a real downside to promoting the launcher to be a
pseudo-backend?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Tom Lane]

Robert Haas  writes:
> Given that the anum.h stuff is gone, "vastly" might be an
> overstatement.  I'm pretty surprised to find out that people don't
> like the idea of having dependencies be correct from anywhere in the
> tree.  Even if I'm the only developer who does partial builds, the
> cost seems to me to be next to nil, so I'm not quite sure what anyone
> gets out of rejecting this patch.

It's not that having the dependencies be 100% up to date wouldn't be
nice; it's that there's a limit to how much we're willing to uglify
the Makefiles to have that.  The makefiles need maintenance too,
you know, and putting things far away from where they should be is
not any better in the makefiles than it is in C code.

As far as I can tell, if you've used --enable-depend then things will
get updated properly before you can ever attempt to run the code
(ie, install a rebuilt postmaster).  The only situation where you'd
actually get an improvement from redoing the dependencies like this
is where lack of an update to a derived file results in a compiler
error/warning.  But there aren't many such cases.  The only one I can
even think of offhand is lack of an fmgroids.h symbol for a newly-added
function ... but we don't use F_XXX symbols enough to make that a
convincing example.  We've intentionally arranged things so that
more-fragile cases like gram.h are not referenced outside their own
directories.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Bruce Momjian]

Tom Lane wrote:
> Bruce Momjian  writes:
> > I compared 8.3 CVS against 8.4 CVS and see the removal of a column in
> > pg_amop.h for exacly the lines you listed:
> 
> Ah.  The removal of amopreqcheck was so long ago I'd forgotten about it ;-)
> Yeah, the column additions/removals since 8.3 would give pgindent an
> excuse to fiddle with the whitespace here.

And all it needs is an excuse.  ;-)

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Bruce Momjian]

Robert Haas wrote:
> What is a bit frustrating to me is that a number of Tom's changes to
> the first two patches were trivial whitespace changes that required me
> to rebase for no obvious reason.   Either those changes were made
> accidentally as Tom was fooling around with what I had done, or they
> were made because Tom had some reason to believe that they would play
> more nicely with pgindent, though what those reasons may have been is
> entirely opaque to me.
> 
> I think that it is good for us as a community to talk about the
> reasons why Tom changes patches.  If some of them are bad reasons,
> maybe talking about it will persuade him to stop.  If they are good
> reasons, perhaps the rest of us can learn from them.  But I think it
> behooves us to talk about specific problems rather than engage in
> open-ended griping.  I haven't been a member of this community for a
> super-long time, but already I can see that there is a correlation
> between who wrote the patch and how heavily it gets edited on commit.

Sometimes I clean up code style in patches because I don't know there
are other patches depending on it;  if I knew there were, I would be
less likely to change patched code, but it is hard to remember which
patches have others pending and which don't.  Perhaps being explicit
might help remind patch appliers.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hot standby and synchronous replication status

[Fujii Masao]

Hi,

On Wed, Aug 12, 2009 at 6:50 AM, Robert Haas wrote:
> I don't really know how you COULD pick a safe default location.
> Presumably any location that's in the default postgresql.conf file
> would be under $PGDATA, which kind of defeats the purpose of the whole
> thing.  In other words, you're always going to have to move it anyway,
> so why bother with a default that is bound to be wrong?

Or, how about introducing the new option which specifies the archive location
for initdb, like -X option. This location is used to set up the default
archive_command by initdb. Though I'm not sure if this meets the need.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Alvaro Herrera]

Robert Haas escribió:

> Given that the anum.h stuff is gone, "vastly" might be an
> overstatement.  I'm pretty surprised to find out that people don't
> like the idea of having dependencies be correct from anywhere in the
> tree.  Even if I'm the only developer who does partial builds, the
> cost seems to me to be next to nil, so I'm not quite sure what anyone
> gets out of rejecting this patch.

I actually kinda like this patch.  I tend to do partial builds
frequently.

> That having been said, it's not
> really worth it to me to spend a lot of time arguing about it.  So
> far, the only coherent argument why this is bad is that it moves some
> logic into a shared Makefile rather than a directory-specific
> Makefile, which might be confusing to someone trying to maintain the
> Makefiles.  I don't really buy that because they're already complex
> enough that you have to read them all to understand what they are
> doing, and nothing in this quite small patch is going to change that
> picture very much, but I guess that's just me.

The action-at-a-distance rules in the shared makefile is a pain, but I
think I'd live with it -- just make sure it is properly documented in
both places.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Greg Stark]

On Tue, Aug 11, 2009 at 10:13 PM, Robert Haas wrote:
> On Tue, Aug 11, 2009 at 4:07 PM, Josh Berkus wrote:
>> So really, the "streaming replication" patch should be called "hot
>> standby",
>
> No.  AIUI, hot standby means that when your primary falls over, the
> secondary automatically promotes itself and takes over.

No! This is *not* what "hot standby" means, at least not in the Oracle world.

The "hot" in "hot standby" is not a reference to the hot backups or
replication behaviour. It means to the fact that the server is able to
take queries. Ie, It's a living breathing server, not one in some kind
of half-alive state like a warm standby server.

Essentially "hot" vs "warm" versus uh, "plain old standby server"
refer to how alive the server is. A traditional standby server was a
machine which was available to take over after a restore. A warm
standby was one that was constantly replaying logs and could be given
life quickly. A hot standby was actually alive.

I think the term "warm standby" and "hot standby" were actually by
analogy to the older term "hot spare" which referred to a machine
which was on and hooked up and ready to go as opposed to one which was
in the cupboard and could be deployed after manual intervention.

>> and the "hot standby" patch should be called "read only slaves"?
>
> Yes.
>
>> And *why* can't we call it log-based replication?
>
> Well, we can call it anything we want.  For example, up until now
> we've been calling it "hot standby", even though that's clearly wrong.
> :-)

"log based replication", "read-only slaves", and "hot standby" are all
100% accurate descriptions of what the hot standby patch enables. I do
like "read only slaves" because it's the most precise and meaningful.

By comparison, it's not clear whether "log based replication" is what
we've had since 8.0 or  whether it's multi-master read-write
replication. And "hot standby" is buzzwordy and depends on Oracle
experience to acquire any meaning.

When we have sync replication you'll also have to distinguish whether
the slave/standby/replica are sync slaves or async slaves and if
they're async what asynchronous mode they're in, streaming or
file-based. Using "hot" to describe those aspects would be horribly
confusing to Oracle users.

-- 
greg
http://mit.edu/~gsstark/resume.pdf

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Andrew Dunstan]

Robert Haas wrote:

On Tue, Aug 11, 2009 at 8:10 PM, Greg Stark wrote:
  

Of course
in all likelihood tom would have rewritten their first patch
anyways...



Maybe I'm taking life too seriously at the moment, but I find this
comment kind of snide and unhelpful.  
  


Yes, you're taking life too seriously ;-) I took it that there was an 
implied smiley at the end of this remark. It certainly didn't strike me 
that it was snide.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Tom Lane]

Bruce Momjian  writes:
> I compared 8.3 CVS against 8.4 CVS and see the removal of a column in
> pg_amop.h for exacly the lines you listed:

Ah.  The removal of amopreqcheck was so long ago I'd forgotten about it ;-)
Yeah, the column additions/removals since 8.3 would give pgindent an
excuse to fiddle with the whitespace here.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: getting rid of the pg_database flat file

[Alvaro Herrera]

Tom Lane wrote:

> To actually get rid of the pg_database flat file, we'd need to take the
> further step of teaching the AV launcher to read pg_database for itself,
> or else refactor things so that the AV workers can do that for it.
> (Alvaro, any comments about the best way to proceed there?)

Hmm.  I don't see any easy way out of that at the moment ... the
launcher would have to become a pseudo-backend, at least to the point
where it is able to read pg_database.  I don't see how could workers
help the launcher with that, unless we made them write a flatfile
representation of pg_database, which would put us back where we started ...

I'll have a deeper look around.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Robert Haas]

On Tue, Aug 11, 2009 at 9:19 PM, Tom Lane wrote:
> Robert Haas  writes:
>> On Tue, Aug 11, 2009 at 9:00 PM, Tom Lane wrote:
>>> Surely the answer to that is "you should be configuring with 
>>> --enable-depend".
>
>> Uhm, the point is that this is broken even with ---enable-depend.
>
> Oh, okay, but that's only for *generated* header files.  I'm not excited
> about that.  We've pretty much managed to minimize compile-time
> dependencies on generated files.  Now of course your other patch wants
> to vastly expand the use of generated files, and I can see that we might
> have a problem of this sort if we accepted that patch --- but it seems
> Peter's not excited about that one either.

Given that the anum.h stuff is gone, "vastly" might be an
overstatement.  I'm pretty surprised to find out that people don't
like the idea of having dependencies be correct from anywhere in the
tree.  Even if I'm the only developer who does partial builds, the
cost seems to me to be next to nil, so I'm not quite sure what anyone
gets out of rejecting this patch.  That having been said, it's not
really worth it to me to spend a lot of time arguing about it.  So
far, the only coherent argument why this is bad is that it moves some
logic into a shared Makefile rather than a directory-specific
Makefile, which might be confusing to someone trying to maintain the
Makefiles.  I don't really buy that because they're already complex
enough that you have to read them all to understand what they are
doing, and nothing in this quite small patch is going to change that
picture very much, but I guess that's just me.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Robert Haas]

On Tue, Aug 11, 2009 at 8:10 PM, Greg Stark wrote:
> Of course
> in all likelihood tom would have rewritten their first patch
> anyways...

Maybe I'm taking life too seriously at the moment, but I find this
comment kind of snide and unhelpful.  I just went through the
experience of getting a series of four patches committed.  I believe
Tom changed the first two only a little.  Tom asked me to make
modifications that to the third one that were a lot of work but with
which I fundamentally agreed, and made some modifications of his own
to the fourth one which were improvements - minor, but improvements.

What is a bit frustrating to me is that a number of Tom's changes to
the first two patches were trivial whitespace changes that required me
to rebase for no obvious reason.   Either those changes were made
accidentally as Tom was fooling around with what I had done, or they
were made because Tom had some reason to believe that they would play
more nicely with pgindent, though what those reasons may have been is
entirely opaque to me.

I think that it is good for us as a community to talk about the
reasons why Tom changes patches.  If some of them are bad reasons,
maybe talking about it will persuade him to stop.  If they are good
reasons, perhaps the rest of us can learn from them.  But I think it
behooves us to talk about specific problems rather than engage in
open-ended griping.  I haven't been a member of this community for a
super-long time, but already I can see that there is a correlation
between who wrote the patch and how heavily it gets edited on commit.

Of course, it's not 100%.  Some of the thing that Tom wants are, at
least AFAICS, things that Tom wants merely because he wants them, and
not because they are better.  That's not wonderful, but any of us
would probably do the same thing, to some degree or other, if we were
in his shoes.  Magnus Hagander recently submitted a patch against my
pgcommitfest application, and just look what I did to him:

http://git.postgresql.org/gitweb?p=pgcommitfest.git;a=summary

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Alpha 1 release notes

[Tom Lane]

Robert Haas  writes:
> Hmm, I thought we had some examples in there, but now that I look we
> only have an example for COSTS OFF, not for FORMAT.  That does seem
> like an oversight.

I thought about adding one when I committed the patch, but concluded
that there was no point until the output format stabilized.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Tom Lane]

Robert Haas  writes:
> On Tue, Aug 11, 2009 at 9:00 PM, Tom Lane wrote:
>> Surely the answer to that is "you should be configuring with 
>> --enable-depend".

> Uhm, the point is that this is broken even with ---enable-depend.

Oh, okay, but that's only for *generated* header files.  I'm not excited
about that.  We've pretty much managed to minimize compile-time
dependencies on generated files.  Now of course your other patch wants
to vastly expand the use of generated files, and I can see that we might
have a problem of this sort if we accepted that patch --- but it seems
Peter's not excited about that one either.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: getting rid of the pg_database flat file

[Tom Lane]

Andrew Dunstan  writes:
> Tom Lane wrote:
>> Really?  And how would pgbouncer be depending on that?

>  auth_file 

> The name of the file to load user names and passwords from. The file format 
> is the same as the PostgreSQL pg_auth/pg_pwd file, so this setting can be 
> pointed directly to one of those backend files.

Considering that we feel free to whack that file format around, and that
it's stored in a directory that non-postgres users shouldn't even be
able to read, I think this argument is just silly.  The only alternative
fix that was proposed in this thread
http://archives.postgresql.org/pgsql-performance/2009-08/msg00094.php
involved changing the flat files into a format that would definitely not
be easily readable by other programs, anyway.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Tom Lane]

Greg Stark  writes:
> What would happen if we ran pgindent immediately after every commit?
> So nobody would ever see a checkout that wasn't pgindent-clean?

> The only losers I see would be people working on multi-part patches.

... which we're trying to encourage ...

Actually, the case that started this thread is relevant: a forced
reindent would be *more* likely to have broken other patches than
not doing one.  I don't really like removing human judgment from
the process.

It definitely would help if large sections of new code were properly
indented before they got committed (assuming there are not followup
patches already prepared).  In the case of modifications to existing
code, the question is not nearly so black-and-white.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Alpha 1 release notes

[Robert Haas]

On Tue, Aug 11, 2009 at 8:32 PM, Alvaro
Herrera wrote:
> Mike wrote:
>
>> The thing that caused me the most trouble was that the "," wasn't very
>> noticeable sitting near the end of this line:
>>
>> EXPLAIN [ ( { ANALYZE boolean | VERBOSE boolean | COSTS boolean |
>> FORMAT { TEXT | XML | JSON } } [, ...] ) ] statement
>>
>> It may just be me, but I read that as the comma being optional, not
>> mandatory in circumstances where you want to specify more then one
>> option.
>
> It is you -- what's optional is having more than one option, but as soon
> as you have a second one, the comma is mandatory.
>
> I agree some examples would be useful.

Hmm, I thought we had some examples in there, but now that I look we
only have an example for COSTS OFF, not for FORMAT.  That does seem
like an oversight.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Robert Haas]

On Tue, Aug 11, 2009 at 5:56 PM, Alvaro
Herrera wrote:
> Robert Haas escribió:
>
>> *shrug*  You don't have to accept the patch, but I'm unclear as to how
>> make from a subdirectory will ever work reliably without something
>> like this.  The problem occurs when .c files have dependencies on
>> files that are generated by a Makefile in some other subdirectory.  So
>> it's not random stuff: it's the stuff where that particular thing
>> happens.
>
> But what use would have to build that particular .o file, and not the
> whole postgres binary?

I don't think that I can spell it out any more clearly than I did in
the paragraph immediately preceding the one you have quoted here.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Robert Haas]

On Tue, Aug 11, 2009 at 9:00 PM, Tom Lane wrote:
> Robert Haas  writes:
>> On Tue, Aug 11, 2009 at 4:38 PM, Peter Eisentraut wrote:
>>> I'm not convinced by this use case.
>
>> Well, my original motivation for developing this patch was that I
>> often go into a subdirectory and start hacking on a .c file and then
>> type make to rebuild just that subdirectory (so it's easier to see
>> warnings and errors).  But sometimes I would edit a relevant header
>> file and then the rebuild wouldn't happen.  That bugged me.  Your
>> mileage may vary.
>
> Surely the answer to that is "you should be configuring with --enable-depend".

Uhm, the point is that this is broken even with ---enable-depend.
Please refer to the OP.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Tom Lane]

Robert Haas  writes:
> On Tue, Aug 11, 2009 at 4:38 PM, Peter Eisentraut wrote:
>> I'm not convinced by this use case.

> Well, my original motivation for developing this patch was that I
> often go into a subdirectory and start hacking on a .c file and then
> type make to rebuild just that subdirectory (so it's easier to see
> warnings and errors).  But sometimes I would edit a relevant header
> file and then the rebuild wouldn't happen.  That bugged me.  Your
> mileage may vary.

Surely the answer to that is "you should be configuring with --enable-depend".

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: getting rid of the pg_database flat file

[Andrew Dunstan]

Tom Lane wrote:

Andrew Dunstan  writes:
  

Tom Lane wrote:


I'd also like to look into getting rid of the pg_auth flat file.
  


  

That would be sad for many users of pgbouncer.



Really?  And how would pgbouncer be depending on that?
  


"

 auth_file 

The name of the file to load user names and passwords from. The file format 
is the same as the PostgreSQL pg_auth/pg_pwd file, so this setting can be 
pointed directly to one of those backend files.


"

see 


cheers

andrew




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: getting rid of the pg_database flat file

[Andres Freund]

On Wednesday 12 August 2009 02:30:53 Tom Lane wrote:
> Andrew Dunstan  writes:
> > Tom Lane wrote:
> >> I'd also like to look into getting rid of the pg_auth flat file.
> > That would be sad for many users of pgbouncer.
> Really?  And how would pgbouncer be depending on that?
One can currently use a copy of that file for its authentication (which it does 
seperately).
I actually don't see a big problem there  - if its contents change often its 
not safe to use that file from outside anyway, and in the other case you easily 
can update it with a select...

Andres

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Collation

[Alvaro Herrera]

David Fetter wrote:
> Folks,
> 
> While trying unsuccessfully to help someone in IRC, they pointed me to
> this:
> 
> http://www.flexiguided.de/publications.pgcollkey.en.html
> 
> Is anybody working with the kind people of FlexiGuided GmbH to see
> about integrating this feature more tightly with PostgreSQL?
> 
> If not, how would we make such an approach?

This has been discussed at length several times.  See the following
messages and their followups:

http://archives.postgresql.org/message-id/de5165440807010758h147d5fw1fc969bc66dcb...@mail.gmail.com
http://archives.postgresql.org/message-id/de5165440808020639j4e7226bbu32e840e225e15...@mail.gmail.com
http://archives.postgresql.org/message-id/de5165440807101424l14fb535byf43fc665351c4...@mail.gmail.com
http://archives.postgresql.org/message-id/de5165440808311432t5618efb5j2f7b3c0f62da8...@mail.gmail.com
http://archives.postgresql.org/message-id/48bd2897.1020...@gmx.net
http://archives.postgresql.org/message-id/48c7855a.40...@enterprisedb.com
http://archives.postgresql.org/message-id/87abd21k3b@oxford.xeocode.com


-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: getting rid of the pg_database flat file

[Tom Lane]

Andrew Dunstan  writes:
> Tom Lane wrote:
>> I'd also like to look into getting rid of the pg_auth flat file.

> That would be sad for many users of pgbouncer.

Really?  And how would pgbouncer be depending on that?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Alpha 1 release notes

[Josh Berkus]

On 8/11/09 3:27 PM, Peter Eisentraut wrote:
> OK, since there was no clear consensus or volunteer for preparing release 
> notes for alpha 1, I have started something.  Let me know what you think.

Actually, the "consensus" was that Bruce was not going to share, so
Robert and I didn't want to bother trying to compose formal release notes.

Can we put your text file somewhere we can collaborate on it?  I started
a page on the wiki, but would gladly use your version as the seed instead:

http://wiki.postgresql.org/wiki/85AlphaFeatures


-- 
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Alpha 1 release notes

[Alvaro Herrera]

Mike wrote:

> The thing that caused me the most trouble was that the "," wasn't very
> noticeable sitting near the end of this line:
> 
> EXPLAIN [ ( { ANALYZE boolean | VERBOSE boolean | COSTS boolean |
> FORMAT { TEXT | XML | JSON } } [, ...] ) ] statement
> 
> It may just be me, but I read that as the comma being optional, not
> mandatory in circumstances where you want to specify more then one
> option. 

It is you -- what's optional is having more than one option, but as soon
as you have a second one, the comma is mandatory.

I agree some examples would be useful.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Alpha 1 release notes

[Mike]

If I didn't read this email I would still be trying to figure out how
to use the explain XML patch. Thanks Albert.

I found the syntax for the explain xml format to be quite difficult to
understand at first, it would be nice to give an example or two, ie:

EXPLAIN (ANALYZE, FORMAT XML) SELECT * FROM foo WHERE i = 4;

The thing that caused me the most trouble was that the "," wasn't very
noticeable sitting near the end of this line:

EXPLAIN [ ( { ANALYZE boolean | VERBOSE boolean | COSTS boolean |
FORMAT { TEXT | XML | JSON } } [, ...] ) ] statement

It may just be me, but I read that as the comma being optional, not
mandatory in circumstances where you want to specify more then one
option. 

Nonetheless, an additional example with multiple options would sure
help.

Thanks.


On Wed, 12 Aug 2009 01:08:54 +0200
Albert Cervera i Areny  wrote:

> A Dimecres, 12 d'agost de 2009, Peter Eisentraut va escriure:
> > OK, since there was no clear consensus or volunteer for preparing
> > release notes for alpha 1, I have started something.  Let me know
> > what you think.
> >
> > (reStructuredText, if you want to play around)
> 
> Maybe I'd be interesting to add development docs URL [1] so testers
> can easily find the syntax of new features, or given that they'll be
> downloading the alpha tarball they should use that documentation?
> 
> [1] http://developer.postgresql.org/pgdocs/postgres/index.html
> 


-- 
Mike

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Greg Stark]

On Tue, Aug 11, 2009 at 4:56 PM, Tom Lane wrote:
> A more aggressive approach would be to run pgindent immediately after
> the close of *each* commitfest, but that would tend to break patches
> that had gotten punted to the next fest.


What would happen if we ran pgindent immediately after every commit?
So nobody would ever see a checkout that wasn't pgindent-clean?

The only losers I see would be people working on multi-part patches.
If just one patch was committed they would have to resolve the
conflicts in their subsequent patches before resubmitting. Of course
in all likelihood tom would have rewritten their first patch
anyways...

-- 
greg
http://mit.edu/~gsstark/resume.pdf

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Bruce Momjian]

Robert Haas wrote:
> On Tue, Aug 11, 2009 at 4:07 PM, Josh Berkus wrote:
> > So really, the "streaming replication" patch should be called "hot
> > standby",
> 
> No.  AIUI, hot standby means that when your primary falls over, the
> secondary automatically promotes itself and takes over.  It requires
> things like heartbeat monitoring and STONITH and is unrelated to
> anything we currently have under consideration.
> 
> > and the "hot standby" patch should be called "read only slaves"?
> 
> Yes.
> 
> > And *why* can't we call it log-based replication?
> 
> Well, we can call it anything we want.  For example, up until now
> we've been calling it "hot standby", even though that's clearly wrong.
> :-)

How about "streaming archive logging" for synchronous replication, and
"continuous archive slave" for read-only queries on a warm standby.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: getting rid of the pg_database flat file

[Andrew Dunstan]

Tom Lane wrote:

I'd also like to look into getting rid of the pg_auth flat file.
  


That would be sad for many users of pgbouncer.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] HEAD docs

[Peter Eisentraut]

On Tuesday 11 August 2009 15:37:42 Andrew Dunstan wrote:
> have the latest docs at
>  stopped
> being built? It sure looks like it somehow.

Fixed

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Bruce Momjian]

Alvaro Herrera wrote:
> Andrew Dunstan escribi?:
> > 
> > 
> > Tom Lane wrote:
> > >Andrew Dunstan  writes:
> > >>Robert Haas wrote:
> > >>>Where it really bit me as when it reindented the DATA() statements
> > >>>that were touched by ALTER TABLE ... SET STATISTICS DISTINCT.  It's
> > >>>not so hard to compare code, but comparing DATA() lines is the pits.
> > >
> > >>Oh? Maybe that's a problem we need to address more directly. I
> > >>just looked at what it did to the DATA lines - it seems to have
> > >>changed 501 of them, and all the changes seem to be to do with
> > >>tabbing.
> > >
> > >That's interesting --- the whitespace in those macros has always been
> > >wildly inconsistent, so I assumed pgindent wasn't touching them at all.
> > >I wonder what it thinks it's doing...
> > 
> > Here's the extract attached.  I replace tabs with a literal '\t' so
> > I could see what it was doing. I can't make much head or tail of it
> > either.
> 
> pgindent uses entab/detab, which counts spaces and replaces them with
> tabs.  It is wildly undocumented.  See src/tools/entab

I am not sure what documentation you want for it that isn't already
there.  There is an entab.man, and it is mentioned in the developer's
FAQ, and it understands 'entab -h' for help.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Alpha 1 release notes

[Albert Cervera i Areny]

A Dimecres, 12 d'agost de 2009, Peter Eisentraut va escriure:
> OK, since there was no clear consensus or volunteer for preparing release
> notes for alpha 1, I have started something.  Let me know what you think.
>
> (reStructuredText, if you want to play around)

Maybe I'd be interesting to add development docs URL [1] so testers can easily 
find the syntax of new features, or given that they'll be downloading the 
alpha tarball they should use that documentation?

[1] http://developer.postgresql.org/pgdocs/postgres/index.html

-- 
Albert Cervera i Areny
http://www.NaN-tic.com
Mòbil: +34 669 40 40 18

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] WIP: getting rid of the pg_database flat file

[Tom Lane]

In the discussion of bug #4919 I wrote:
> In some sense this is a bootstrap problem: what does it take to get to
> the point of being able to read pg_database and its indexes?  That is
> necessarily not dependent on the particular database we want to join.
> Maybe we could solve it by having the relcache write a "global" cache
> file containing only entries for the global tables, and load that before
> we have identified the database we want to join (after which, we'll load
> another cache file for the local entries).  It would doubtless take some
> rearrangement of the backend startup sequence, but it doesn't seem
> obviously impossible.

Attached is a proof-of-concept patch which shows that this idea makes it
possible to start backends without the pg_database flat file, and that the
required search of pg_database can be done with an index as long as we
have the shared relcache cache file available (which should always be true
except for the first backend start after postmaster bootup or crash
recovery).  There are a few loose ends yet to fix, but on the whole it
was easier than I expected.  The main costs of doing it this way are:

* pg_database has to become a nailed-in-cache relation, as does its
index on datname.  (Its index on OID will have to be nailed too, unless
we can get rid of the kluge that lets autovacuum give InitPostgres a
database OID instead of database name.  I have not looked at autovacuum
yet.)  This doesn't really cost anything except a few more bytes in the
relcache ... and in reality I suspect pg_database is always in that
cache anyway.

* We have to have a Schema_pg_database macro in pg_attribute.h.  This
means a little more hand maintenance (unless we accept Robert Haas'
patch to autogenerate all that stuff); but it's still not a big problem.

I think this is clearly worth cleaning up and committing, since even
without any further progress it eliminates number-of-databases as a
significant factor in backend startup time.  Does anyone have any
objection to the above side-effects?

To actually get rid of the pg_database flat file, we'd need to take the
further step of teaching the AV launcher to read pg_database for itself,
or else refactor things so that the AV workers can do that for it.
(Alvaro, any comments about the best way to proceed there?)

I'd also like to look into getting rid of the pg_auth flat file.
As previously noted, that means postponing client auth to later in the
startup sequence.  If we were willing to eliminate role membership as an
available piece of information for auth method selection, we could still
do much of the auth work before initializing the backend proper; in
particular we could issue a password challenge and wait for a response,
which would be good in terms of reducing our exposure to lightweight DDOS
attacks.  I'm not sure if anyone will think that's a good tradeoff though,
since any attacker who can connect to the postmaster port can probably
DDOS the postmaster just fine anyway.

Comments?

regards, tom lane



binLpaO4qRx57.bin
Description: look-ma-no-flat-file-1.patch.gz

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Bruce Momjian]

Tom Lane wrote:
> Alvaro Herrera  writes:
> > Andrew Dunstan escribi?:
> >> Here's the extract attached.  I replace tabs with a literal '\t' so
> >> I could see what it was doing. I can't make much head or tail of it
> >> either.
> 
> > pgindent uses entab/detab, which counts spaces and replaces them with
> > tabs.  It is wildly undocumented.  See src/tools/entab
> 
> What surprises me is that it seems to be changing lines that have been
> there for quite some time.  Bruce, have you been changing pgindent
> underneath us?

Not unless I am CVS committing without showing anyone.  ;-)

These are the only commits I see recently, which are harmless:

revision 1.101
date: 2009/06/11 22:21:44;  author: momjian;  state: Exp;  lines: +7 -2
Document struct/union problem with pgindent.

revision 1.100
date: 2008/11/03 15:56:47;  author: momjian;  state: Exp;  lines: +2 -2
Small shell syntax improvement.

revision 1.99
date: 2008/04/16 21:03:08;  author: momjian;  state: Exp;  lines: +2 -2
Ignore blank lines in typedef file.

revision 1.98
date: 2008/01/16 20:13:44;  author: momjian;  state: Exp;  lines: +2 -2
Improve usage message for pgindent.

revision 1.97
date: 2007/12/21 14:20:36;  author: momjian;  state: Exp;  lines: +11
-2008
Modify pgindent to use an external typedefs file rather than included
list.

and entab is similarly unchanged:

revision 1.18
date: 2007/02/08 11:10:27;  author: petere;  state: Exp;  lines: +2 -2
Normalize fgets() calls to use sizeof() for calculating the buffer size
where possible, and fix some sites that apparently thought that fgets()
will overwrite the buffer by one byte.

Also add some strlcpy() to eliminate some weird memory handling.

I am stumped, but can keep researching.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Quick pointer required re indexing geometry

[Paul Matthews]

Dimitri Fontaine wrote:

  Paul Matthews  writes:
  
  
Witting a box@>point function easy. Having a spot of trouble trying to
figure out where and how to graft this into the GiST stuff. Could
someone please point me in the general direction?

  
  
You want index support for it, I suppose?
  

Yes

  Without index support (but needed anyway), you implement your code in a
C module then make it visible from SQL.  ...cut... 
COMMENT ON OPERATOR @>(box, point) IS 'box contains point?';
  

All done but the comment part :-) 

  
Now for adding support for index lookups, you have to see documentation
about OPERATOR CLASS and OPERATOR FAMILY.
Hope this helps, regards,
  

Thanks overlooked CLASS and FAMILY in my hurry. Hopefully that is where
the problem is.

What I am hoping to do is provide a "real" patch to make box@>point
available for all. But we can get to that later. A lot of code in
pgsql, but not much in the way of comments.

Re: [HACKERS] "Hot standby"?

[Gianni Ciolli]

On Wed, Aug 12, 2009 at 12:11:28AM +0300, Peter Eisentraut wrote:
> On Tuesday 11 August 2009 18:16:04 Gianni Ciolli wrote:
> > As for "warm/hot", it depends on what you exactly mean with "get
> > ready":
> >
> > (A) If you mean "it is possible to connect to the second node", then
> > Simon's patch is "hot".
> 
> Yeah, but by that definiton doing a pg_dump/pg_restore every hour is also 
> "hot". ;-)

OK, but only if (a) the data is so small that the restore takes less
than one hour, and if (b) the workload is far from 100% ;-)

Since there seem to be multiple views about terminology, it may be
useful to recall the proposed wording from
http://wiki.postgresql.org/wiki/Hot_Standby :

* the first node is called either "Primary" or "Master" 

* the second node is called "Standby" 

* the Standby is referred to as a "Clone" instead of a "Slave", to
  mean that it is an exact copy, which, instead of being built by
  repeating the actions of the master, is constructed just by
  implementing their effects.

Best regards,
Dr. Gianni Ciolli - 2ndQuadrant Italia
PostgreSQL Training, Services and Support
gianni.cio...@2ndquadrant.it | www.2ndquadrant.it


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Collation

[David Fetter]

Folks,

While trying unsuccessfully to help someone in IRC, they pointed me to
this:

http://www.flexiguided.de/publications.pgcollkey.en.html

Is anybody working with the kind people of FlexiGuided GmbH to see
about integrating this feature more tightly with PostgreSQL?

If not, how would we make such an approach?

Cheers,
David.
-- 
David Fetter  http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Alpha 1 release notes

[Peter Eisentraut]

OK, since there was no clear consensus or volunteer for preparing release 
notes for alpha 1, I have started something.  Let me know what you think.

(reStructuredText, if you want to play around)
.. -*- mode: rst -*-
=
Release 8.5alpha1
=
.. last commit: Simplify and speed up man page installation

Overview


PostgreSQL alpha releases are snapshots of development code.  They are
intended to preview and test upcoming features and to provide the
possibility for early feedback.  They should not be used in production
installations or active development projects.  While the PostgreSQL
code is continuously subject to a number of automated and manual
tests, alpha releases might have serious bugs.  Also features may be
changed incompatibly or removed at any time during the development
cycle.

The development cycle of a PostgreSQL major release alternates between
periods of development and periods of integration work, called commit
fests, normally one month each.  Alpha releases are planned to be
produced at the end of every commit fest, thus every two months.
Since the first commit fest starts within a month from the beginning
of development altogether, early alpha releases are not indicative of
the likely feature set of the final release.

The release notes below highlight user visible changes and new
features.  There are normally numerous bug fixes and performance
improvements in every new snapshot of PostgreSQL, and it would be too
bulky to attempt to list them all.  Note that many bug fixes are also
backported to stable releases of PostgreSQL, and you should be using
those if you are looking for bug-fix-only upgrades for your current
installations.

Migration
-

To upgrade from any release to an alpha release or from an alpha
release to any other release will most likely require a dump/restore
upgrade procedure.  It may happen that this is not necessary in
particular cases, but that is not verified beforehand.  (The server
will warn you in any case when a dump/restore is necessary if you
attempt to use it with an old data directory.)  Note, however, that
the dump/restore upgrade procedure is expected to work for alpha
releases, and problems in this area should be reported.

Changes
---

Server
~~

- Added log_line_prefix placeholder %e to contain the current SQL
  state

- Allow parentheses around the query expression that follows a WITH
  clause.

- Fixed ancient bug in handling of to_char modifier 'TH', when used
  with HH.

- Use floor() not rint() when reducing precision of fractional seconds
  in timestamp_trunc, timestamptz_trunc, and interval_trunc() for the
  float-datetime case.

- More sensible values for character_octet_length column in
  information schema.

- Information schema updated to SQL:2008

- Backend header files are now safe to use with C++.

- Make GEQO's planning deterministic by having it start from a
  predictable random number seed each time.

- Rewrite GEQO's gimme_tree function so that it always finds a legal
  join sequence.  Previously, it could have failed to produce a plan
  in some cases.

- Simplify the forms foo <> true and foo <> false to foo = true and
  foo = false.

- DROP IF EXISTS for columns and constraints

- Tweak TOAST code so that columns marked with MAIN storage strategy
  are not forced out-of-line unless that is necessary to make the row
  fit on a page.  Previously, they were forced out-of-line if needed
  to get the row down to the default target size (1/4th page).

- EXPLAIN can now take generic options

- EXPLAIN allows output of plans in XML or JSON format.

- Support  (scientific notation) in to_char().

- Allow * as parameter for FORCE QUOTE for COPY CSV

- Support deferrable uniqueness constraints.

- Added ALTER TABLE ... ALTER COLUMN ... SET STATISTICS DISTINCT

- has_sequence_privilege()

- New hex-string input and output for type BYTEA.

- Fast shutdown stop should forcibly disconnect any active backends,
  even if a smart shutdown is already in progress. Backpatched to 8.3.

pg_dump
~~~

- Fixed up --binary-upgrade option so that it behaves properly with
  inherited columns and check constraints.

- Properly restore pg_largeobject.relfozenxid in binary upgrade mode.

- Make pg_dump/pg_restore --clean options drop large objects too.

- Modify parallel pg_restore logic to avoid potential O(N^2) slowdown
  in extreme cases.

psql


- Have \\d show child tables that inherit from the specified parent

- Show definition of index columns in \\d on index

ecpg


- Added STRING datatype for Informix compatibility mode.

Procedural Languages


- Improve plpgsql's ability to cope with rowtypes containing dropped
  columns,

Contrib
~~~

- Multi-threaded version of pgbench

- Adds the ability to retrieve async notifications using dblink, via
  the addition of the function dblink_get_notify().

- Add matchorig, matchsynonyms, and keepsynonyms options to
  contrib/di

Re: [HACKERS] [COMMITTERS] pgsql: Ship documentation without intermediate tarballs Documentation

[Peter Eisentraut]

On Monday 10 August 2009 18:59:51 Tom Lane wrote:
> After doing "make" then "make distclean" in doc/src/sgml, I see the
> following undesirable files left behind:
>
> -rw-rw-r-- 1 tgl tgl 58 Aug 10 11:51 version.sgml
> -rw-rw-r-- 1 tgl tgl  38548 Aug 10 11:51 features-unsupported.sgml
> -rw-rw-r-- 1 tgl tgl  42014 Aug 10 11:51 features-supported.sgml
> -rw-rw-r-- 1 tgl tgl 345398 Aug 10 11:52 HTML.index
> -rw-rw-r-- 1 tgl tgl 298859 Aug 10 11:52 bookindex.sgml
> -rw-rw-r-- 1 tgl tgl  0 Aug 10 11:53 html-stamp
>
> I would argue that both "make clean" and "make distclean" should remove
> these.

OK, I fixed that, but html-stamp has to stay to keep the dependencies 
satisfied.  (Well, there are alternatives.  We could make the whole thing 
depend on html/index.html or whatever.)

> Also, we seem to need .cvsignore entries for the html/ and manN/
> subdirectories.  IMO the policy for .cvsignore is that anything
> intentionally left behind by make distclean is to be cvsignore'd.

Fixed.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] TODO item: Allow more complex user/database default GUC settings

[Alvaro Herrera]

Hi,

There's a longstanding TODO item, in subject.  Previous discussion was
here:

http://archives.postgresql.org/pgsql-hackers/2006-09/msg02341.php

In looking what it would take to implement it, I find that it is
trivial.  The only part that looks complex is the UI for it.  Is anyone
interested in giving this patch a shot?

Implementation-side, it requires a new catalog (pg_settings), with the
following columns:

setdatabase oid
setrole oid
setconfig   text[]

datconfig and rolconfig are removed.

ALTER DATABASE / SET sets setdatabase to the OID of the database in
command, and setrole to 0 (InvalidOid); ALTER ROLE / SET sets setrole
and leaves setdatabase 0.

A new command allows one to set a config that applies to both database
and role.

At startup, the settings are applied in the following order:
database=value  role=0
database=0  role=value
database=value  role=value

This way, current behavior is maintained (ALTER ROLE trumps ALTER
DATABASE).

The only real work in this is figuring out what the grammar for the new
command looks like.  Maybe we could have some like

ALTER ROLE foo ALTER DATABASE bar SET config

There are of course many possible variations but this looks the most
reasonable one.  Any better ideas?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Josh Berkus]

>> Incidentally, we billed pg_dump as hot backup at some point.

It *is* "hot backup" as in taken while the database is running and fully
accessible.

> mysql calls "mysqlhotcopy" a script that locks and flushes all tables,
> then makes a copy of the database directory (all queries being locked
> out while this is in progress, of course).

Which is actually 'cold backup'; it's only "hot" compared to earlier
MySQLs where you had to shut the db down.  So it's sort of a "lukewarm
backup".


-- 
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgsql: Ship documentation without intermediate tarballs Documentation

[Alvaro Herrera]

Peter Eisentraut wrote:
> On Monday 10 August 2009 23:03:12 Alvaro Herrera wrote:
> > Hmm, I notice that this rule to install manpages is pretty slow:

> Yeah, that was really freakishly complicated.  Fixed now.

It is much better now, thanks.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Andrew Dunstan]

Pierre Frédéric Caillaud wrote:



Incidentally, we billed pg_dump as hot backup at some point.


mysql calls "mysqlhotcopy" a script that locks and flushes all tables, 
then makes a copy of the database directory (all queries being locked 
out while this is in progress, of course).





Doesn't sound very "hot" to me. Tepid at best.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] machine-readable explain output v4

[Andrew Dunstan]

Robert Haas wrote:

On Tue, Aug 11, 2009 at 3:59 PM, Andrew Dunstan wrote:
  

Good. I had a look at this for a little while yesterday. I built it, did an
install, loaded auto_explain and then ran the regression tests. I didn't
like the output much. It looks like the XML has been dumbed down to fit a
data model that works for JSON as well, particularly in the lack of use of
attributes. An XML processor won't care that much, but humans will certainly
find it more tiresome to read. In effect we are swapping horizontal
expansion for vertical expansion. It would be nicer to be able to fit a plan
into a screen.



Isn't that what text format is for?

Not that the XML format is perfect; I haven't heard one person say
that they like anything about it except that it is already
implemented.


  


Well, I don't think that the fact that we are producing machine-readable 
output means we can just ignore the human side of it. It is more than 
likely that such output will be read by both machines and humans. 
Obviously, we need to make sure that it meets machine processing needs 
first, but once we have done that we should not ignore the human 
requirements.


BTW, I think it's great that we have machine-readable formats. This 
opens the door to a lot of tools. Well done.


No doubt there will be disagreements about the formats. Our community 
certainly has a tendency to argue over matters of appearance, sometimes 
to the point of silliness. My understanding was that the idea of getting 
this in now was to let people have a look, and play to some extent.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hot standby and synchronous replication status

[Robert Haas]

On Tue, Aug 11, 2009 at 5:40 PM, Dimitri Fontaine wrote:
> Le 11 août 09 à 23:30, Robert Haas a écrit :
>
>> On Tue, Aug 11, 2009 at 5:20 PM, Dimitri Fontaine
>> wrote:
>>>
>>> We should somehow provide a default archive and restore command
>>> integrated
>>> into the main product, so that it's as easy as turning it 'on' in the
>>> configuration for users to have something trustworthy: PostgreSQL will
>>> keep
>>> past logs into a pg_xlog/archives subdir or some other default place, and
>>> will know about the setup at startup time when/if needed.
>>
>> I might be missing something, but isn't this completely silly?  If you
>> archive your logs to the same partition where you keep your database
>> cluster, it seems to me that you might as well delete them.  Even
>> better, turn off XLogArchiving altogether and save yourself the
>> overhead of not using WAL-bypass.
>
> Nice, the pushback is about the default location, thanks for supporting the
> idea :)
>
> Seriously, debian package will install pg_xlog in $PGDATA which is often not
> what I want. So first thing after install, I stop the cluster, move the
> pg_xlog, setup a ln -s and restart. I figured having to do the same for
> setting up archiving would make my day, when compared to current
> documentation setup. Any better idea for a safe enough default location is
> welcome, of course.

*scratches head*

I don't really know how you COULD pick a safe default location.
Presumably any location that's in the default postgresql.conf file
would be under $PGDATA, which kind of defeats the purpose of the whole
thing.  In other words, you're always going to have to move it anyway,
so why bother with a default that is bound to be wrong?

Maybe I'm all wet?

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Alvaro Herrera]

Robert Haas escribió:

> *shrug*  You don't have to accept the patch, but I'm unclear as to how
> make from a subdirectory will ever work reliably without something
> like this.  The problem occurs when .c files have dependencies on
> files that are generated by a Makefile in some other subdirectory.  So
> it's not random stuff: it's the stuff where that particular thing
> happens.

But what use would have to build that particular .o file, and not the
whole postgres binary?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hot standby and synchronous replication status

[Joshua D. Drake]

On Tue, 2009-08-11 at 17:30 -0400, Robert Haas wrote:
> On Tue, Aug 11, 2009 at 5:20 PM, Dimitri Fontaine 
> wrote:
> > We should somehow provide a default archive and restore command integrated
> > into the main product, so that it's as easy as turning it 'on' in the
> > configuration for users to have something trustworthy: PostgreSQL will keep
> > past logs into a pg_xlog/archives subdir or some other default place, and
> > will know about the setup at startup time when/if needed.
> 
> I might be missing something, but isn't this completely silly?  If you
> archive your logs to the same partition where you keep your database
> cluster, it seems to me that you might as well delete them.  Even
> better, turn off XLogArchiving altogether and save yourself the
> overhead of not using WAL-bypass.

Depends on all kinds of factors. For example, PITRTools will keep a copy
local until it knows that the remote has received it.

Joshua D. Drake

> 
> ...Robert
> 
-- 
PostgreSQL - XMPP: jdr...@jabber.postgresql.org
   Consulting, Development, Support, Training
   503-667-4564 - http://www.commandprompt.com/
   The PostgreSQL Company, serving since 1997


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] machine-readable explain output v4

[Kevin Grittner]

Robert Haas  wrote: 
> Andrew Dunstan wrote:
 
>> find it more tiresome to read. In effect we are swapping horizontal
>> expansion for vertical expansion. It would be nicer to be able to
>> fit a plan into a screen.
> 
> Isn't that what text format is for?
 
In my experience XML which represents anything of any real complexity
is tiresome to read, period.  I would worry more about it accurately
representing the data and being easy to massage with software.  That's
not an opinion on any particular choice here, just on what should
matter in making a choice.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgsql: Ship documentation without intermediate tarballs Documentation

[Peter Eisentraut]

On Monday 10 August 2009 23:03:12 Alvaro Herrera wrote:
> Hmm, I notice that this rule to install manpages is pretty slow:
>
> for file in /pgsql/source/00head/doc/src/sgml/man1/*.1
> /pgsql/source/00head/doc/src/sgml/man3/*.3
> /pgsql/source/00head/doc/src/sgml/man7/*.7; do /bin/sh
> /pgsql/source/00head/config/install-sh -c -m 644 $file
> /pgsql/install/00head/share/man/`echo $file | sed
> 's,^/pgsql/source/00head/doc/src/sgml/,,'` || exit; done
>
> Can we use "basename" here instead of the `echo | sed` hack?
>
> Hmm, oh, I see it's stripping everything except the last directory level.
> I guess I'd go for doing a simple cp inside each man directory.

Yeah, that was really freakishly complicated.  Fixed now.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hot standby and synchronous replication status

[Dimitri Fontaine]

Le 11 août 09 à 23:30, Robert Haas a écrit :

On Tue, Aug 11, 2009 at 5:20 PM, Dimitri Fontaine> wrote:
We should somehow provide a default archive and restore command  
integrated

into the main product, so that it's as easy as turning it 'on' in the
configuration for users to have something trustworthy: PostgreSQL  
will keep
past logs into a pg_xlog/archives subdir or some other default  
place, and

will know about the setup at startup time when/if needed.


I might be missing something, but isn't this completely silly?  If you
archive your logs to the same partition where you keep your database
cluster, it seems to me that you might as well delete them.  Even
better, turn off XLogArchiving altogether and save yourself the
overhead of not using WAL-bypass.


Nice, the pushback is about the default location, thanks for  
supporting the idea :)


Seriously, debian package will install pg_xlog in $PGDATA which is  
often not what I want. So first thing after install, I stop the  
cluster, move the pg_xlog, setup a ln -s and restart. I figured having  
to do the same for setting up archiving would make my day, when  
compared to current documentation setup. Any better idea for a safe  
enough default location is welcome, of course.


Oh, and I hope you didn't read that the archive mode be 'on' by  
default in my proposal, because that's not what I meant.


Regards,
--
dim
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] machine-readable explain output v4

[Robert Haas]

On Tue, Aug 11, 2009 at 3:59 PM, Andrew Dunstan wrote:
> Good. I had a look at this for a little while yesterday. I built it, did an
> install, loaded auto_explain and then ran the regression tests. I didn't
> like the output much. It looks like the XML has been dumbed down to fit a
> data model that works for JSON as well, particularly in the lack of use of
> attributes. An XML processor won't care that much, but humans will certainly
> find it more tiresome to read. In effect we are swapping horizontal
> expansion for vertical expansion. It would be nicer to be able to fit a plan
> into a screen.

Isn't that what text format is for?

Not that the XML format is perfect; I haven't heard one person say
that they like anything about it except that it is already
implemented.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Any tutorial or FAQ on building an extension?

[Robert Haas]

On Tue, Aug 11, 2009 at 4:00 PM, Kevin
Grittner wrote:
> Matt Culbreth  wrote:
>
>> My new component intercepts this, and decides if it wants to do
>> something
>
>> If it does, it passes the request over to my new server (via
>> sockets), does its work, and pass back the results
>
> That's still too vague to allow people to give very specific advice.
> For example, I don't have a clue yet whether the query rewrite rules
> might satisfy your needs:
>
> http://www.postgresql.org/docs/8.4/interactive/sql-createrule.html

Yeah.  I suspect you are going to better off using some combination of
views, triggers, set-returning functions, and SQL-level permissions to
do whatever it is that you are trying to do here rather than inventing
a whole middleware layer.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hot standby and synchronous replication status

[Robert Haas]

On Tue, Aug 11, 2009 at 5:20 PM, Dimitri Fontaine wrote:
> We should somehow provide a default archive and restore command integrated
> into the main product, so that it's as easy as turning it 'on' in the
> configuration for users to have something trustworthy: PostgreSQL will keep
> past logs into a pg_xlog/archives subdir or some other default place, and
> will know about the setup at startup time when/if needed.

I might be missing something, but isn't this completely silly?  If you
archive your logs to the same partition where you keep your database
cluster, it seems to me that you might as well delete them.  Even
better, turn off XLogArchiving altogether and save yourself the
overhead of not using WAL-bypass.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Robert Haas]

On Tue, Aug 11, 2009 at 4:38 PM, Peter Eisentraut wrote:
> On Sunday 28 June 2009 21:21:35 Robert Haas wrote:
>> I think that our dependencies for generated header files (gram.h,
>> fmgroids.h, probes.h) are not as good as they could be.  What we do
>> right now is make src/backend/Makefile rebuild these before recursing
>> through its subdirectories.  This works OK for a top-level make, but
>> if you run make further down in the tree (like under
>> src/backend/commands) it won't necessarily rebuild everything that it
>> should.
>>
>> The attached patch moves some of this logic from src/backend/Makefile
>> to src/Makefile.global.in.  That way, if you --enable-depend and then
>> do something like "touch src/include/catalog/pg_proc.h" and then "cd
>> src/backend/commands; make vacuum.o", it rebuilds fmgroids.h and then
>> recompiles vacuum.c.  Under HEAD, it just tells you that vacuum.o is
>> up to date.
>
> I'm not convinced by this use case.  Why do you want to rebuild vacuum.o
> anyway?  If you change something, then you probably want to rebuild postgres,
> and that works fine without this change.

Well, my original motivation for developing this patch was that I
often go into a subdirectory and start hacking on a .c file and then
type make to rebuild just that subdirectory (so it's easier to see
warnings and errors).  But sometimes I would edit a relevant header
file and then the rebuild wouldn't happen.  That bugged me.  Your
mileage may vary.

> I'm concerned how this change moves random make rules into a global makefile.
> This is exacerbated by your next patch, which among other things moves the
> entire catalog list to build the various output files into Makefile.global,
> far away from its home.

*shrug*  You don't have to accept the patch, but I'm unclear as to how
make from a subdirectory will ever work reliably without something
like this.  The problem occurs when .c files have dependencies on
files that are generated by a Makefile in some other subdirectory.  So
it's not random stuff: it's the stuff where that particular thing
happens.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Pierre Frédéric Caillau d]

Incidentally, we billed pg_dump as hot backup at some point.


mysql calls "mysqlhotcopy" a script that locks and flushes all tables,  
then makes a copy of the database directory (all queries being locked out  
while this is in progress, of course).


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Hot standby and synchronous replication status

[Dimitri Fontaine]

Hi,

Le 11 août 09 à 07:50, Heikki Linnakangas a écrit :

>2009/8/11 Robert Haas 
> We should probably have a separate discussion about what the least
> committable unit would be for this patch.  I wonder if it might be
> sufficient to provide a facility for streaming WAL, plus a  
standalone

> tool for receving it and storing it to a file.

That's an interesting idea. That would essentially be another method  
to set up a WAL archive. I'm not sure it's worthwhile on its own,  
but once we have the wal-sender infrastructure in place it should be  
easy to write such a tool.


Well it might be over engineering time *again* but here's what it  
makes me think about:


We should somehow provide a default archive and restore command  
integrated into the main product, so that it's as easy as turning it  
'on' in the configuration for users to have something trustworthy:  
PostgreSQL will keep past logs into a pg_xlog/archives subdir or some  
other default place, and will know about the setup at startup time  
when/if needed.


Now, the archive and restore commands would make a independent  
subsystem, the only one (for modularisation sake) allowed to work with  
the archives. So we extend it to support sending and receiving  
archives to/from a remote PostgreSQL server, using libpq and the  
already proposed protocol in the current patch form.


It could be that for integration purpose we'd need something like the  
autovacuum launcher, an archive manager daemon child of postmaster  
accepting signals in order to spawn a specific tasks. Sender part  
could be launched more than once at any time, e.g.


Of course the included automatic and easy to setup daemon wouldn't  
care about setting up a remote archiving policy, but on the other hand  
a remote PostgreSQL instance could easily be set up as a wal receiver  
from the master's archive. The archive retention policy and how it  
applies to known list of receivers is to discuss :)


As far as the (a)Sync Rep patch is concerned, this could solve the  
setup part of it, this step where starting from a filesystem level  
backup you realize you need archived WALs before being able to apply  
currently received entries (LSN granularity).


Regards,
--
dim


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] autogenerating headers & bki stuff

[Robert Haas]

On Tue, Aug 11, 2009 at 4:52 PM, Peter Eisentraut wrote:
> On Tuesday 30 June 2009 06:59:51 Robert Haas wrote:
>> The attached patch merges all of the logic currently in genbki.sh and
>> Gen_fmgrtab.{sh,pl} into a single script called gen_catalog.pl
>
> I can't really convince myself to like this change.  I think there is some
> merit that these scripts are separate.  I'm not sure what the combined script
> buys us except that it is a lot bigger and does everything at once instead of
> in two or three steps.
>
> That together with the big makefile moving around makes me think that this
> would cause more confusion and not much advantage.
>
> Btw., is this stamp-h business really necessary?  I guess you copied that from
> pg_config.h, but the reason there is that everything depends on pg_config.h,
> and rerunning configure would normally cause everything to be rebuilt.  The
> files we are looking at here should only change when something is edited.

It definitely has less appeal without the anum.h stuff.

I think there is some benefit in having a single script because it
means that if we want to add additional syntax in the future (like
BKI_EXECUTE or DATA_DEFAULTS or some kind of more human-readable
notation for functions or opclasses, all of which were discussed
upthread) there is only one place to change.  But is that sufficient
reason to commit it at this point, given that we don't have a
fully-fleshed out design for any of those things?  Not sure.  Building
schemapg.h automatically seems definitely nice to me.

The stamp-h stuff is quite important for minimizing unnecessary
rebuilds.  Without that, any change to any include/catalog/*.h file
will trigger a massive rebuild.  With it, it only triggers a rebuild
if one of the outputs actually changes, and only for those portions
for which the output actually changed.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] autogenerating headers & bki stuff

[Alvaro Herrera]

Peter Eisentraut escribió:
> On Tuesday 30 June 2009 06:59:51 Robert Haas wrote:
> > The attached patch merges all of the logic currently in genbki.sh and
> > Gen_fmgrtab.{sh,pl} into a single script called gen_catalog.pl
> 
> I can't really convince myself to like this change.  I think there is some 
> merit that these scripts are separate.  I'm not sure what the combined script 
> buys us except that it is a lot bigger and does everything at once instead of 
> in two or three steps.

Maybe we can move forward in little steps.  For example it would be
excellent to have the schemapg.h file autogenerated instead of having to
edit two copies of those definitions.

I'm not sure that it buys us a lot to have all those things in a single
script.  Why not have a script to generate schemapg.h, another one to
generate the bki stuff, another one to generate the fmgrtab header?
They don't seem to share a lot of code anyway (and even if they do,
surely we can add a .pm module containing common code).

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Robert Haas]

On Tue, Aug 11, 2009 at 4:07 PM, Josh Berkus wrote:
> So really, the "streaming replication" patch should be called "hot
> standby",

No.  AIUI, hot standby means that when your primary falls over, the
secondary automatically promotes itself and takes over.  It requires
things like heartbeat monitoring and STONITH and is unrelated to
anything we currently have under consideration.

> and the "hot standby" patch should be called "read only slaves"?

Yes.

> And *why* can't we call it log-based replication?

Well, we can call it anything we want.  For example, up until now
we've been calling it "hot standby", even though that's clearly wrong.
:-)

But on the merits: log-based replication is, I think, what we already
have.  Both of these patches do things that make it better.  Streaming
replication (fka synch rep) makes it more nearly real-time, and
read-only slaves (fka hot standby) makes the standby server more
useful.  But neither is ADDING log-based replication, they're both
IMPROVING it, in different ways.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Peter Eisentraut]

On Tuesday 11 August 2009 18:16:04 Gianni Ciolli wrote:
> As for "warm/hot", it depends on what you exactly mean with "get
> ready":
>
> (A) If you mean "it is possible to connect to the second node", then
> Simon's patch is "hot".

Yeah, but by that definiton doing a pg_dump/pg_restore every hour is also 
"hot". ;-)

Incidentally, we billed pg_dump as hot backup at some point.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Gianni Ciolli]

On Tue, Aug 11, 2009 at 01:14:56PM -0400, Robert Haas wrote:
> On Tue, Aug 11, 2009 at 1:08 PM, Josh Berkus wrote:
> > I believe we're just copying Oracle's terminology.  While that
> > terminology is not consistent, it is understood by the industry.  Oracle
> > defined their Hot Standby to have both asynchronous and synchronous modes:
> > http://www.oracle.com/technology/products/rdb/htdocs/dbms/hotstandby.html
> 
> I think that you are wrong.  Hot Standby is described in that document
> as a way of replication the database and providing failover in the
> event that the master dies.  This feature has to do with allowing
> queries to be executed on the slave, which is not discussed in that
> document.

That web page contains only a brief summary of Oracle's Hot Standby;
by looking at the attached PDF guide you can see that the capability
of allowing read-only queries on the standby node is actually part of
their architecture.

Since we are comparing the features with Oracle's Hot Standby, it is
perhaps useful to mention two of the major differences:

* Oracle's Hot Standby allows only one Standby node, while on
  PostgreSQL we can have as many as we want, since the Primary is
  totally unaware of them.

  Allowing N clones instead of only one may be a crucial advantage in
  these use cases where read-only queries represent the vast majority
  of the load.

* Oracle's Hot Standby allows cloning hash indexes, which on
  PostgreSQL are not WAL safe (anyway, they are currently discouraged
  by the documentation).

Best regards,
Dr. Gianni Ciolli - 2ndQuadrant Italia
PostgreSQL Training, Services and Support
gianni.cio...@2ndquadrant.it | www.2ndquadrant.it


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Shipping documentation untarred

[Peter Eisentraut]

On Tuesday 11 August 2009 17:02:01 Tom Lane wrote:
> Having all the derived files in the build directory definitely seems
> to me to reduce the complexity and surprise factor, so +1 for changing.

I've looked at that briefly, and it's a bit more complicated than it would 
appear.  I will figure this out later, but it's not going to happen this week.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] autogenerating headers & bki stuff

[Peter Eisentraut]

On Tuesday 30 June 2009 06:59:51 Robert Haas wrote:
> The attached patch merges all of the logic currently in genbki.sh and
> Gen_fmgrtab.{sh,pl} into a single script called gen_catalog.pl

I can't really convince myself to like this change.  I think there is some 
merit that these scripts are separate.  I'm not sure what the combined script 
buys us except that it is a lot bigger and does everything at once instead of 
in two or three steps.

That together with the big makefile moving around makes me think that this 
would cause more confusion and not much advantage.

Btw., is this stamp-h business really necessary?  I guess you copied that from 
pg_config.h, but the reason there is that everything depends on pg_config.h, 
and rerunning configure would normally cause everything to be rebuilt.  The 
files we are looking at here should only change when something is edited.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] machine-readable explain output v4

[Andres Freund]

On Tuesday 11 August 2009 21:59:48 Andrew Dunstan wrote:
> Tom Lane wrote:
> > Mike  writes:
> >> Have any tool authors stepped up and committed resources to utilizing
> >> this feature in the near term?
> >
> > I don't think anyone's promised much.  If you want to have a go at using
> > it, we'd be very happy.
> >
> >> I'm guessing that my vision likely exceeds the scope of this feature in
> >> its initial form at least, but assuming no one else has stepped up, I'm
> >> more than willing to start committing resources as early as this
> >> weekend with the understanding that this feature is still in
> >> development and likely will change several times before or if its
> >> finally committed for 8.5.
> >
> > It's definitely committed for 8.5, but the exact format of the output
> > is (obviously) still subject to change.
> Good. I had a look at this for a little while yesterday. I built it, did
> an install, loaded auto_explain and then ran the regression tests. I
> didn't like the output much. It looks like the XML has been dumbed down
> to fit a data model that works for JSON as well, particularly in the
> lack of use of attributes. An XML processor won't care that much, but
> humans will certainly find it more tiresome to read. In effect we are
> swapping horizontal expansion for vertical expansion. It would be nicer
> to be able to fit a plan into a screen.
The problem is that nobody yet came up with one that is easy to generate and 
liked by many people...
Aesthetically I do not like the current schema as well, but personally I don't 
think it matters that much.

Everything fancy has the problem of requiring relatively much code... 

> I also took the last relaxng spec I could find and used a nice little
> tool called Trang to translate it into an XML Schema spec. The good news
> is that that worked. The bad news is that the spec almost certainly
> needs some tightening, especially around those elements that probably
> should be XML attributes.
Unrelated to the other issues the relaxng schema has some missing pieces if I 
remember it correctly (constraint => constraint-name, trigger => trigger-name) 
I think). It is also by far not strict enough yet...

Andres

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] dependencies for generated header files

[Peter Eisentraut]

On Sunday 28 June 2009 21:21:35 Robert Haas wrote:
> I think that our dependencies for generated header files (gram.h,
> fmgroids.h, probes.h) are not as good as they could be.  What we do
> right now is make src/backend/Makefile rebuild these before recursing
> through its subdirectories.  This works OK for a top-level make, but
> if you run make further down in the tree (like under
> src/backend/commands) it won't necessarily rebuild everything that it
> should.
>
> The attached patch moves some of this logic from src/backend/Makefile
> to src/Makefile.global.in.  That way, if you --enable-depend and then
> do something like "touch src/include/catalog/pg_proc.h" and then "cd
> src/backend/commands; make vacuum.o", it rebuilds fmgroids.h and then
> recompiles vacuum.c.  Under HEAD, it just tells you that vacuum.o is
> up to date.

I'm not convinced by this use case.  Why do you want to rebuild vacuum.o 
anyway?  If you change something, then you probably want to rebuild postgres, 
and that works fine without this change.

I'm concerned how this change moves random make rules into a global makefile.  
This is exacerbated by your next patch, which among other things moves the 
entire catalog list to build the various output files into Makefile.global, 
far away from its home.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Merlin Moncure]

On Tue, Aug 11, 2009 at 4:07 PM, Josh Berkus wrote:
> All,
>
> So really, the "streaming replication" patch should be called "hot
> standby", and the "hot standby" patch should be called "read only slaves"?
>
> And *why* can't we call it log-based replication?

+1

*) it _is_ used to replicate a database (replicate means make a copy!)
*) our target market will perceive it that way
*) sounds cool

'synchronous log-based replication'
'asynchronous log-based replication'
or,
'log-based replication', in both synchronous and asynchronous modes

merlin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Josh Berkus]

All,

So really, the "streaming replication" patch should be called "hot
standby", and the "hot standby" patch should be called "read only slaves"?

And *why* can't we call it log-based replication?

-- 
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Any tutorial or FAQ on building an extension?

[Kevin Grittner]

Matt Culbreth  wrote: 
 
> My new component intercepts this, and decides if it wants to do
> something
 
> If it does, it passes the request over to my new server (via
> sockets), does its work, and pass back the results
 
That's still too vague to allow people to give very specific advice. 
For example, I don't have a clue yet whether the query rewrite rules
might satisfy your needs:
 
http://www.postgresql.org/docs/8.4/interactive/sql-createrule.html
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] machine-readable explain output v4

[Andrew Dunstan]

Tom Lane wrote:

Mike  writes:
  

Have any tool authors stepped up and committed resources to utilizing
this feature in the near term?



I don't think anyone's promised much.  If you want to have a go at using
it, we'd be very happy.

  

I'm guessing that my vision likely exceeds the scope of this feature in
its initial form at least, but assuming no one else has stepped up, I'm
more than willing to start committing resources as early as this
weekend with the understanding that this feature is still in
development and likely will change several times before or if its
finally committed for 8.5. 



It's definitely committed for 8.5, but the exact format of the output
is (obviously) still subject to change.


  


Good. I had a look at this for a little while yesterday. I built it, did 
an install, loaded auto_explain and then ran the regression tests. I 
didn't like the output much. It looks like the XML has been dumbed down 
to fit a data model that works for JSON as well, particularly in the 
lack of use of attributes. An XML processor won't care that much, but 
humans will certainly find it more tiresome to read. In effect we are 
swapping horizontal expansion for vertical expansion. It would be nicer 
to be able to fit a plan into a screen.


I also took the last relaxng spec I could find and used a nice little 
tool called Trang to translate it into an XML Schema spec. The good news 
is that that worked. The bad news is that the spec almost certainly 
needs some tightening, especially around those elements that probably 
should be XML attributes.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Tom Lane]

Alvaro Herrera  writes:
> Andrew Dunstan escribió:
>> Here's the extract attached.  I replace tabs with a literal '\t' so
>> I could see what it was doing. I can't make much head or tail of it
>> either.

> pgindent uses entab/detab, which counts spaces and replaces them with
> tabs.  It is wildly undocumented.  See src/tools/entab

What surprises me is that it seems to be changing lines that have been
there for quite some time.  Bruce, have you been changing pgindent
underneath us?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Kevin Grittner]

Mark Mielke  wrote:
 
> This patch does not provide streaming replication?
 
There's a separate effort to provide asynchronous and synchronous
streaming replication.  Different patch.
 
> "Hot standby" to me means "the slave is as close to up-to-date as
> possible and can potentially take over at any time in a near
> instance." This *implies* some sort of streaming replication (byte
> level rather than file-by-file) rather than waiting for WAL logs to
> become full and shipped.
 
Most of use would expect that from something called "hot standby".
That's why so many of us have been saying that the name is misleading.
This patch, as I understand it, would allow you to use the warm
standby to run read-only queries -- for reports and such, to take some
load off the primary database.  No more; no less.
 
So it's still warm, not hot, and it's still usable as a warm standby.
It just lets you squeeze a little extra benefit from the copy while it
sits there periodically updating itself.
 
-Kevin

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Alvaro Herrera]

Mark Mielke escribió:

> I don't think I was confused before - but I am confused now. :-)
> 
> This patch does not provide streaming replication?

No.  What it does is allow you to query the slave while it's still
replaying transactions.  There's another patch allowing you to do
"stream replication".  Hopefully both things could be used
simultaneously.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Mark Mielke]

On 08/11/2009 02:52 PM, Robert Haas wrote:

On Tue, Aug 11, 2009 at 2:48 PM, Mark Mielke  wrote:
   

I remember this debate from 6 months ago. :-)

I prefer not to try and fit square pegs into round holes. Streaming
replication sounds like the best description. It may not be the keywords
that newbies are looking for, but too bad for them. Calling it something
different than what it is, just so that people who don't understand why it
is wrong will have something that approximates the right understanding, is
not a just cause. :-)
 


Uhm, I think you are confused.

Hot Standby = Allow read-only queries on a PostgreSQL server during
archive recovery
Synchronous (or Streaming) Replication = Allow WAL to be streamed on a
byte level rather than copied file-by-file

Hot Standby is not any sort of replication.


I don't think I was confused before - but I am confused now. :-)

This patch does not provide streaming replication?

Having the standby allow for read-only queries is a "would be nice" for 
me, but it's not very useful. I've been monitoring these threads (and 
wishing they were at a level I could participate and contribute on), 
because I want the ability to have near real time updates such that the 
standby can become live. "Hot standby" to me means "the slave is as 
close to up-to-date as possible and can potentially take over at any 
time in a near instance." This *implies* some sort of streaming 
replication (byte level rather than file-by-file) rather than waiting 
for WAL logs to become full and shipped.


If this patch doesn't give me near real time replication, then I am 
confused about why I would want it at all. pg_standby already gives the 
ability to do replication on a per completed WAL log file basis.


Cheers,
mark

--
Mark Mielke

Re: [HACKERS] Any tutorial or FAQ on building an extension?

[Matt Culbreth]

On Aug 11, 1:11 pm, j...@agliodbs.com (Josh Berkus) wrote:
> > Is there an easier way of going about this other than replacing the
> > postmaster / postgres components?
>
> I'd start with creating my own extended version to psql (the client
> library), I suppose.  But since I don't really know what kind of
> "transformations" you have in mind, any advice is going to be purely
> speculative.
>

Thanks for the response Josh.

I'm not sure that psql is the right thing for me to do though, since I
want to build a back-end component that takes the place of the
existing postmaster.  Very possible I misunderstood you though.

To clarify, essentially what I want to do is this:

Client [ psql | JDBC driver | pgAdmin | etc. ] issues a Query
[ "Select * from sales" ]
|
|
\/
My new component intercepts this, and decides if it wants to do
something
|
|
\/
If it does not, it simply passes this on to the real PostgreSQL server
running somewhere
|
|
\/
If it does, it passes the request over to my new server (via sockets),
does its work, and pass back the results
|
|
\/
The client gets the results back, either from PostgreSQL or from my
new server, and goes about its way.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Alvaro Herrera]

Andrew Dunstan escribió:
> 
> 
> Tom Lane wrote:
> >Andrew Dunstan  writes:
> >>Robert Haas wrote:
> >>>Where it really bit me as when it reindented the DATA() statements
> >>>that were touched by ALTER TABLE ... SET STATISTICS DISTINCT.  It's
> >>>not so hard to compare code, but comparing DATA() lines is the pits.
> >
> >>Oh? Maybe that's a problem we need to address more directly. I
> >>just looked at what it did to the DATA lines - it seems to have
> >>changed 501 of them, and all the changes seem to be to do with
> >>tabbing.
> >
> >That's interesting --- the whitespace in those macros has always been
> >wildly inconsistent, so I assumed pgindent wasn't touching them at all.
> >I wonder what it thinks it's doing...
> 
> Here's the extract attached.  I replace tabs with a literal '\t' so
> I could see what it was doing. I can't make much head or tail of it
> either.

pgindent uses entab/detab, which counts spaces and replaces them with
tabs.  It is wildly undocumented.  See src/tools/entab

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Robert Haas]

On Tue, Aug 11, 2009 at 2:10 PM, Ron Mayer wrote:
> David Fetter wrote:
>> On Tue, Aug 11, 2009 at 08:56:38AM -0500, Kevin Grittner wrote:
>>> Bruce Momjian  wrote:
>>>
 OK, so it is "warm slave".
>
> Why isn't it just a "read only slave".  Do some systems
> have read-only slave databases that can't serve as a warm
> standby system as well as this one could?

I think that's about right.  What we have now via pg_standby or
similar tools is a warm standby.  What this tool does is makes the
warm-standby also serve as a read-only slave.  It doesn't make
failover any simpler so it's not making the standby any hotter - it
instead makes the standby be able to do more useful work when no
failover has occurred.

The technical description for the commit message is probably something
like this:

"Allow read-only queries to be processed during archive recovery."

The P/R version is probably something like this:

"Warm standby servers now function as read-only slaves."

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Tom Lane]

Andrew Dunstan  writes:
> Robert Haas wrote:
>> Where it really bit me as when it reindented the DATA() statements
>> that were touched by ALTER TABLE ... SET STATISTICS DISTINCT.  It's
>> not so hard to compare code, but comparing DATA() lines is the pits.

> Oh? Maybe that's a problem we need to address more directly. I just 
> looked at what it did to the DATA lines - it seems to have changed 501 
> of them, and all the changes seem to be to do with tabbing.

That's interesting --- the whitespace in those macros has always been
wildly inconsistent, so I assumed pgindent wasn't touching them at all.
I wonder what it thinks it's doing...

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Robert Haas]

On Tue, Aug 11, 2009 at 2:48 PM, Mark Mielke wrote:
> On 08/11/2009 09:56 AM, Kevin Grittner wrote:
>
> Bruce Momjian  wrote:
>
>
>
> OK, so it is "warm slave".
>
>
>
> That is technically accurate, given the preceding definitions, but it
> has disturbing connotations.  Enough so, in my view, to merit getting
> a little more creative in the naming.  How about "warm replica"?
> Other ideas?
>
> I agree that the present moniker misleads.
>
> I remember this debate from 6 months ago. :-)
>
> I prefer not to try and fit square pegs into round holes. Streaming
> replication sounds like the best description. It may not be the keywords
> that newbies are looking for, but too bad for them. Calling it something
> different than what it is, just so that people who don't understand why it
> is wrong will have something that approximates the right understanding, is
> not a just cause. :-)

Uhm, I think you are confused.

Hot Standby = Allow read-only queries on a PostgreSQL server during
archive recovery
Synchronous (or Streaming) Replication = Allow WAL to be streamed on a
byte level rather than copied file-by-file

Hot Standby is not any sort of replication.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Mark Mielke]

On 08/11/2009 09:56 AM, Kevin Grittner wrote:

Bruce Momjian  wrote:

   

OK, so it is "warm slave".
 


That is technically accurate, given the preceding definitions, but it
has disturbing connotations.  Enough so, in my view, to merit getting
a little more creative in the naming.  How about "warm replica"?
Other ideas?

I agree that the present moniker misleads.


I remember this debate from 6 months ago. :-)

I prefer not to try and fit square pegs into round holes. Streaming 
replication sounds like the best description. It may not be the keywords 
that newbies are looking for, but too bad for them. Calling it something 
different than what it is, just so that people who don't understand why 
it is wrong will have something that approximates the right 
understanding, is not a just cause. :-)


Cheers,
mark

--
Mark Mielke

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Andrew Dunstan]

Robert Haas wrote:

On Tue, Aug 11, 2009 at 12:52 PM, Andrew Dunstan wrote:
  

Robert Haas wrote:


I'm not sure there's a
good solution to this problem short of making pgindent easy enough
that we can make it a requirement for patch submission, and I'm not
sure that's practical.

But in any case, I think running pgindent immediately after the last
CommitFest rather than after a longish delay would be a good idea.
  

Frankly, fixing up patch bitrot caused by pgindent is not terribly difficult
in my experience - bitrot caused by code drift is a much harder problem (and
yes, git fans, I know git can help with that).



Where it really bit me as when it reindented the DATA() statements
that were touched by ALTER TABLE ... SET STATISTICS DISTINCT.  It's
not so hard to compare code, but comparing DATA() lines is the pits.


  



Oh? Maybe that's a problem we need to address more directly. I just 
looked at what it did to the DATA lines - it seems to have changed 501 
of them, and all the changes seem to be to do with tabbing.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] machine-readable explain output v4

[Tom Lane]

Mike  writes:
> Have any tool authors stepped up and committed resources to utilizing
> this feature in the near term?

I don't think anyone's promised much.  If you want to have a go at using
it, we'd be very happy.

> I'm guessing that my vision likely exceeds the scope of this feature in
> its initial form at least, but assuming no one else has stepped up, I'm
> more than willing to start committing resources as early as this
> weekend with the understanding that this feature is still in
> development and likely will change several times before or if its
> finally committed for 8.5. 

It's definitely committed for 8.5, but the exact format of the output
is (obviously) still subject to change.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] machine-readable explain output v4

[Mike]

On Tue, 11 Aug 2009 13:11:47 -0400
Robert Haas  wrote:
 
> Unfortunately, I have to admit to total confusion. The idea in the
> last paragraph seems reasonable to me, but since I don't understand
> the other alternative, I can't say whether it's better or worse.  I
> wonder if we would be better off waiting for feedback from actual tool
> authors.  I have no illusions that the format is perfect...

Have any tool authors stepped up and committed resources to utilizing
this feature in the near term?

This is one of the features I have been most interested in, and I
believe it has the potential to greatly improve PostgreSQL's
"reputation" of being hard to optimize if used correctly.

I envision a web site where users can paste the XML explain
output and have it return some pretty specific information and fancy
graphs describing exactly whats going on with their query and possible
ideas on how to further optimize the query itself and even PostgreSQL
configuration settings. 

The reason I would like to provide this tool in a web-based form is
that no additional software installation would be necessary for the
user, reducing any hurdles to using it to zero.

I'm guessing that my vision likely exceeds the scope of this feature in
its initial form at least, but assuming no one else has stepped up, I'm
more than willing to start committing resources as early as this
weekend with the understanding that this feature is still in
development and likely will change several times before or if its
finally committed for 8.5. 

-- 
Mike

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Ron Mayer]

David Fetter wrote:
> On Tue, Aug 11, 2009 at 08:56:38AM -0500, Kevin Grittner wrote:
>> Bruce Momjian  wrote: 
>>  
>>> OK, so it is "warm slave".

Why isn't it just a "read only slave".  Do some systems
have read-only slave databases that can't serve as a warm
standby system as well as this one could?

>> That is technically accurate, given the preceding definitions, but
>> it has disturbing connotations.  Enough so, in my view, to merit
>> getting a little more creative in the naming.  How about "warm
>> replica"?  Other ideas?
> 
> Warm Read
> Streamed Copy

Master/Slave Replication and Warm Standby systems are common
enough terms that I can google them or look them up in many
computer science books.

While coming up with creative politically correct euphemisms
might be fun, I hope we stick near terms that other DBAs could
already be familiar with.

ISTM the best way to refer to it formally would be a
   "Read Only Slave / Warm Standby"
system, even if informally we might have informal
discussions of "just how hot our slaves are" when hot-standby
features get added down the road.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Robert Haas]

On Tue, Aug 11, 2009 at 12:52 PM, Andrew Dunstan wrote:
> Robert Haas wrote:
>>
>> I'm not sure there's a
>> good solution to this problem short of making pgindent easy enough
>> that we can make it a requirement for patch submission, and I'm not
>> sure that's practical.
>>
>> But in any case, I think running pgindent immediately after the last
>> CommitFest rather than after a longish delay would be a good idea.
>
> Frankly, fixing up patch bitrot caused by pgindent is not terribly difficult
> in my experience - bitrot caused by code drift is a much harder problem (and
> yes, git fans, I know git can help with that).

Where it really bit me as when it reindented the DATA() statements
that were touched by ALTER TABLE ... SET STATISTICS DISTINCT.  It's
not so hard to compare code, but comparing DATA() lines is the pits.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Robert Haas]

On Tue, Aug 11, 2009 at 1:08 PM, Josh Berkus wrote:
> I believe we're just copying Oracle's terminology.  While that
> terminology is not consistent, it is understood by the industry.  Oracle
> defined their Hot Standby to have both asynchronous and synchronous modes:
> http://www.oracle.com/technology/products/rdb/htdocs/dbms/hotstandby.html

I think that you are wrong.  Hot Standby is described in that document
as a way of replication the database and providing failover in the
event that the master dies.  This feature has to do with allowing
queries to be executed on the slave, which is not discussed in that
document.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Any tutorial or FAQ on building an extension?

[Josh Berkus]

> Is there an easier way of going about this other than replacing the
> postmaster / postgres components?

I'd start with creating my own extended version to psql (the client
library), I suppose.  But since I don't really know what kind of
"transformations" you have in mind, any advice is going to be purely
speculative.

-- 
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] machine-readable explain output v4

[Robert Haas]

On Tue, Aug 11, 2009 at 12:11 PM, Tom Lane wrote:
> Robert Haas  writes:
>> I think I might be starting to understand what you're getting at here.
>>  Let me check: I think what you're saying is that the Expr node is
>> potentially useful to clients for identifying where in the tree the
>> Exprs are, even without specific knowledge about the different types
>> of filter nodes, much as the Plans node is useful for finding all of
>> the subplan objects.
>
> Well, my point is that there are a bunch of different places that
> expressions can appear in a plan tree --- filters, join filters, sort
> keys, hash clauses, merge clauses, index clauses, output expressions,
> and I'm not sure I remembered them all, and I fully expect that some
> more might appear in future.  I don't think it's appropriate to require
> clients to keep track of exactly which tree properties are expressions,
> especially not when we can easily label them.
>
>> If that is correct, then let me make two comments.  First, I don't
>> think it should be a goal of this format to be self-documenting.
>
> It isn't.  But I think it should be a goal that a client can, say,
> find all the references to a given variable without a huge amount of
> hard-wired knowledge about specific properties of specific node types.
> So it should be able to find all the expressions with a relatively
> dumb search of the tree (eg, XPath?).
>
>> But, second, there might be a way that we can have our cake and eat it
>> too.  Instead of decorating each node with properties Filter,
>> Join-Filter, One-Time-Filter, and so on, what we could do is decorate
>> each node with a single property called Filters, which would be a list
>> of hashes:
>
>> 'Filters' : [ { 'Filter-Type' => 'Join', 'Text-Expr' => 'a.foo < b.bar' }]
>
> I still say that's a bad idea for child Plans; and it's a worse one for
> expressions.  The point about retaining knowledge of child order is
> absolutely critical for many of the places where exprs appear, such as
> sort keys and output columns.  You'd have to seriously uglify the
> notation in order to do it like this.
>
> One way that we could handle this, I guess, is to embed more information
> in the property names --- eg use "Expr Text" where I just had "Text".
> But it's not apparent to me why that should be considered better style
> than the other way.

Unfortunately, I have to admit to total confusion. The idea in the
last paragraph seems reasonable to me, but since I don't understand
the other alternative, I can't say whether it's better or worse.  I
wonder if we would be better off waiting for feedback from actual tool
authors.  I have no illusions that the format is perfect...

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "Hot standby"?

[Josh Berkus]

Peter,

I believe we're just copying Oracle's terminology.  While that
terminology is not consistent, it is understood by the industry.  Oracle
defined their Hot Standby to have both asynchronous and synchronous modes:
http://www.oracle.com/technology/products/rdb/htdocs/dbms/hotstandby.html

The other industry term which would be understood would be "log-based
replication".   Terms we invent would be less likely to be understood,
and users would not get excited about them.

Otherwise, I say we go with "simmering follower".  ;-)

-- 
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)

[Andrew Dunstan]

Robert Haas wrote:

I'm not sure there's a
good solution to this problem short of making pgindent easy enough
that we can make it a requirement for patch submission, and I'm not
sure that's practical.

But in any case, I think running pgindent immediately after the last
CommitFest rather than after a longish delay would be a good idea.

  



Frankly, fixing up patch bitrot caused by pgindent is not terribly 
difficult in my experience - bitrot caused by code drift is a much 
harder problem (and yes, git fans, I know git can help with that).


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers