Re: [HACKERS] typedefs for indent

[Tom Lane]

Bruce Momjian  writes:
> Frankly, I don't remember anyone complaining we didn't find any typedefs
> in pgindent,

There are lots and lots of places where it's obvious that pgindent
was misinformed on the subject, because it puts in a space where there
should not be one, eg "typename * ptr" instead of "typename *ptr".
Maybe I'm just being too anal in worrying about a space here or a space
there ... but then why do we run the thing at all?

This is the first time I've heard anyone suggest that false positives
could be a problem.  What exactly would be the results of a false
match?  Would it look worse than the false negatives do?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 release notes proof reading 1/2

[Guillaume Smet]

On Fri, Mar 27, 2009 at 2:44 AM, Bruce Momjian  wrote:
> Guillaume Smet wrote:
>> - "Add -M (query mode) to /contrib/pgbench (ITAGAKI Takahiro)"
>> ->Itagaki san's name inconsistent with other mentions of his name
>
> Above all fixed, thanks.

I think you fixed this one the wrong way.

It should be s/ITAGAKI Takahiro/Takahiro Itagaki/g, shouldn't it?

-- 
Guillaume

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 open items list

[Guillaume Smet]

On Fri, Mar 27, 2009 at 2:58 AM, Robert Haas  wrote:
> That includes a whole slough of patches that weren't submitted until
> after November 1st and which I think should probably be bumped en
> masse to 8.5:
>
> postgresql.conf: patch to have ParseConfigFile report all parsing
> errors, then bail

Not sure about this one. A similar patch for the pg_hba.conf file
submitted after the commit fest (IIRC) has already been commited.

> pg_standby trigger behavior is dangerous

This one has to be fixed IMHO. I'm not sure how but we have to take a
decision. The current behaviour is really dangerous for most of our
users.

-- 
Guillaume

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 open items list

[Tom Lane]

Robert Haas  writes:
> On Thu, Mar 26, 2009 at 10:11 PM, Bruce Momjian  wrote:
>> I think pushing "pre-existing bugs" to 8.5 is a mistake,

> What is the threshold for "has to be fixed before we can go to beta"
> versus "has to be fixed before release"?

I did not by any means intend that those things should wait until 8.5.
The point was that we needn't be ashamed to release an 8.4 *beta*
that has such bugs.  Or at least no more than we are ashamed to have
released existing stable versions with 'em.

Beta is about getting a reasonably stable version into the hands of
testers.  It's not about achieving perfection before the testers
ever see it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] typedefs for indent

[Bruce Momjian]

Andrew Dunstan wrote:
> >> Andrew, this is disappointing news.  When you talked about generating an
> >> typedef list from the buildfarm, you were saying how great it would be
> >> --- now a year later you post:
> >>
> >>It'd be nice to get that dealt with before we run pg_indent, but it's
> >>not like we'd be any worse off than before if we don't. In any case it's
> >>surely no blocker for 8.4beta.
> >> 
> >
> > My apologies;  the above are Tom's words, not Andrews.
> >
> >   
> 
> Apology accepted.
> 
> What I promised was a list that was more comprehensive than what we were 
> using. I think I've already delivered on that, but I would like to do 
> better by including some other Operating Systems: particularly some BSD 
> flavors.  Buildfarm owners with non-Linux non-Windows members please 
> take note. Email me if you need help with this.
> 
> Unless we come up with some tolerably correct and maintainable code 
> analysis tool for identifying typedefs, using the current heuristic 
> methods is apparently the best we can do. Nobody has suggested even an 
> outline for such a tool. I don't think using the buildfarm for this 
> heuristic method is great, and never suggested it would be. I do think 
> it's an improvement, which is what I promised. I'm sorry if you find the 
> result disappointing.

Well, as you, I was hoping for a clear solution, and it seems we don't
have one.  I think the false-positives problem is real and might make
the greater code coverage of the buildfarm worse than what we did for
8.3.

I think our only fallback is to find places that our BSD items miss,
perhaps Win32 cases, see what is different with those lists, and just
hard-code them in, because then we aren't importing a huge number of
additional typedefs that have uncertain consequences.

Frankly, I don't remember anyone complaining we didn't find any typedefs
in pgindent, though I think there might have been a few EXEC_BACKEND
cases, and maybe we can just hardcode those.

Frankly, pgindent has larger problems than an imcomplete typedef list. :-(

When I am ready to run pgindent I will ask for your typedef list and see
what the diff shows when I use your list and we can figure something out
then.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] tuplestore API problem

[Hitoshi Harada]

2009/3/27 Tom Lane :
> By chance I discovered that this query in the regression tests
>
> SELECT ntile(NULL) OVER (ORDER BY ten, four), ten, four FROM tenk1 LIMIT 2;
>
> stops working if work_mem is small enough: it either dumps core or
> delivers wrong answers depending on platform.
>
> After some tracing I found out the reason.  ExecWindowAgg() does this:
>
>    if (!tuplestore_gettupleslot(winstate->buffer, true,
>                                 winstate->ss.ss_ScanTupleSlot))
>        elog(ERROR, "unexpected end of tuplestore");
>
> and then goes off and calls the window functions (ntile() here), and
> expects the ScanTupleSlot to still be valid afterwards.  However,
> ntile() forces us to read to the end of the input to find out the number
> of rows.  If work_mem is small enough, that means the tuplestore is
> forced into dump-to-disk mode, which means it releases all its in-memory
> tuples.  And guess what: the ScanTupleSlot is pointing at one of those,
> it doesn't have its own copy of the tuple.  So we wind up trying to read
> from a trashed bit of memory.
>
> A brute-force solution is to change tuplestore_gettupleslot() so that it
> always copies the tuple, but this would be wasted cycles for most uses
> of tuplestores.  I'm thinking of changing tuplestore_gettupleslot's API
> to add a bool parameter specifying whether the caller wants to force
> a copy.
>
> Comments, better ideas?

Is this tuplestore API problem? ISTM this is window function's
problem. I think my early code was holding heaptuple instead of
tupleslot for the current row. At a glance, the issue appears in only
current row in window function, which fetches row and uses it later
after storing following rows in some cases. So a brute-force solution
might be that ExecWindowAgg() copies the current row from tuplestore
instead of pointing directly to inside tuplestore memory, not changing
tuplestore API.

Regards,


-- 
Hitoshi Harada

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] typedefs for indent

[Andrew Dunstan]

Bruce Momjian wrote:

bruce wrote:
  

Andrew Dunstan wrote:

This URL  gives a 
typedef list that is (currently) the combined result from three fairly 
different buildfarm members:


dungbeetle | 2009-03-22 06:44:01
brown_bat | 2009-03-21 13:00:58
dawn_bat | 2009-03-21 14:23:40

These are respectively my Linux, Cygwin and MinGW buildfarm members.

I don't have a BSD machine of any flavor to test on, and I don't know 
how to extract the typedefs on OSX.


Anyone running a buildfarm member should be able to do this and add to 
the results, if they are up to date with release 3.2. I have my linux 
crontab set up to do one typedefs run on the HEAD branch each day.
  

[ Discussion deleted.]

Andrew, this is disappointing news.  When you talked about generating an
typedef list from the buildfarm, you were saying how great it would be
--- now a year later you post:

It'd be nice to get that dealt with before we run pg_indent, but it's
not like we'd be any worse off than before if we don't. In any case it's
surely no blocker for 8.4beta.



My apologies;  the above are Tom's words, not Andrews.

  


Apology accepted.

What I promised was a list that was more comprehensive than what we were 
using. I think I've already delivered on that, but I would like to do 
better by including some other Operating Systems: particularly some BSD 
flavors.  Buildfarm owners with non-Linux non-Windows members please 
take note. Email me if you need help with this.


Unless we come up with some tolerably correct and maintainable code 
analysis tool for identifying typedefs, using the current heuristic 
methods is apparently the best we can do. Nobody has suggested even an 
outline for such a tool. I don't think using the buildfarm for this 
heuristic method is great, and never suggested it would be. I do think 
it's an improvement, which is what I promised. I'm sorry if you find the 
result disappointing.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New trigger option of pg_standby

[Fujii Masao]

Hi,

On Thu, Mar 26, 2009 at 8:54 PM, Guillaume Smet
 wrote:
> Hi Simon.
>
> On Thu, Mar 26, 2009 at 11:50 AM, Simon Riggs  wrote:
>> Earlier, we discussed having a single trigger file that contains an
>> option rather than two distinct trigger files. That design is better
>> because it allows the user to choose at failover time, rather than
>> making a binding decision at config time. That solution would be the
>> ideal one, IMHO, because it gives user more choice - and would allow us
>> to keep the -t option meaningfully. In that case the default should be
>> patience.
>
> Or you can define both files in your command line to have the choice.

Personally I like this.

> I like the idea of removing -t and adding 2 new options so that people
> are warned about the intended behavior.

OK, I'll change the patch as Simon suggested; removing -t and adding
two new options: -f = fast failover (existing behavior), -p patient failover.
Also I'll default the patient failover, so it's performed when the signal
(SIGINT or SIGUSR1) is received.

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] typedefs for indent

[Bruce Momjian]

bruce wrote:
> Andrew Dunstan wrote:
> > This URL  gives a 
> > typedef list that is (currently) the combined result from three fairly 
> > different buildfarm members:
> > 
> > dungbeetle | 2009-03-22 06:44:01
> > brown_bat | 2009-03-21 13:00:58
> > dawn_bat | 2009-03-21 14:23:40
> > 
> > These are respectively my Linux, Cygwin and MinGW buildfarm members.
> > 
> > I don't have a BSD machine of any flavor to test on, and I don't know 
> > how to extract the typedefs on OSX.
> > 
> > Anyone running a buildfarm member should be able to do this and add to 
> > the results, if they are up to date with release 3.2. I have my linux 
> > crontab set up to do one typedefs run on the HEAD branch each day.
> 
> [ Discussion deleted.]
> 
> Andrew, this is disappointing news.  When you talked about generating an
> typedef list from the buildfarm, you were saying how great it would be
> --- now a year later you post:
> 
>   It'd be nice to get that dealt with before we run pg_indent, but it's
>   not like we'd be any worse off than before if we don't. In any case it's
>   surely no blocker for 8.4beta.

My apologies;  the above are Tom's words, not Andrews.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Any reason not to return row_count in cursor of plpgsql?

[Bruce Momjian]

laser wrote:
> hi all,
> 
>  I read the code that it seems easy for the cursor in plpgsql to return 
> ROW_COUNT after
>  MOVE LAST etc. The SPI_processed variable already there, but didn't put 
> it into estate
>  structure, any reason for that?
> 
>  thanks and best regards

Sorry, we have decided against this change because it might break
existing applications.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 release notes proof reading 1/2

[Bruce Momjian]

Greg Stark wrote:
> On Fri, Mar 27, 2009 at 1:44 AM, Bruce Momjian  wrote:
> >
> >> - "Previously EXPLAIN VERBOSE output an internal representation of the
> >> query plan" -> s/output/outputs/ ?
> >
> > The existing wording seems correct.
> 
> I think Bruce's phrasing was in the past tense. It's a bit weird
> because the verb form of "output" is a relatively recent invention and
> the past tense isn't well settled. Dictionaries list both "outputted"
> and "output" as past tense forms. Personally I think Bruce's "output"
> sounds better than the alternative "outputted".
> 
> Perhaps "had output" would be clearer?

Excellent idea;  done.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] typedefs for indent

[Bruce Momjian]

Andrew Dunstan wrote:
> This URL  gives a 
> typedef list that is (currently) the combined result from three fairly 
> different buildfarm members:
> 
> dungbeetle | 2009-03-22 06:44:01
> brown_bat | 2009-03-21 13:00:58
> dawn_bat | 2009-03-21 14:23:40
> 
> These are respectively my Linux, Cygwin and MinGW buildfarm members.
> 
> I don't have a BSD machine of any flavor to test on, and I don't know 
> how to extract the typedefs on OSX.
> 
> Anyone running a buildfarm member should be able to do this and add to 
> the results, if they are up to date with release 3.2. I have my linux 
> crontab set up to do one typedefs run on the HEAD branch each day.

[ Discussion deleted.]

Andrew, this is disappointing news.  When you talked about generating an
typedef list from the buildfarm, you were saying how great it would be
--- now a year later you post:

It'd be nice to get that dealt with before we run pg_indent, but it's
not like we'd be any worse off than before if we don't. In any case it's
surely no blocker for 8.4beta.

We can't have the system-supplied typedef list changing from release to
release because that affects the indenting from release to release,
which affects backpatching and other stuff.  And even if you get a more
complete list then we have used in the past, what are the odds you are
going to supply a typedef that is a typedef on some operating system
that matches an identifier in our code that _isn't_ used as a typedef by
us?

We only have a few weeks until I have to run pgindent so I would like
this resolved one way or another soon.  Unless I hear otherwise I assume
we are going to just use the an updated list of our defined typedefs
that gets generated from our code, which includes my BSD typedefs.

One other approach would be to include in pg_indent a hard-coded list of
non-BSD system-defined typedefs that we reference from our code.  One
way to find those would be to run pg_indent with and without Andrew's
list of typedefs and see how the formatting changes.

Or just use a Linux list of system typedefs from now on.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 release notes proof reading 1/2

[Greg Stark]

On Fri, Mar 27, 2009 at 1:44 AM, Bruce Momjian  wrote:
>
>> - "Previously EXPLAIN VERBOSE output an internal representation of the
>> query plan" -> s/output/outputs/ ?
>
> The existing wording seems correct.

I think Bruce's phrasing was in the past tense. It's a bit weird
because the verb form of "output" is a relatively recent invention and
the past tense isn't well settled. Dictionaries list both "outputted"
and "output" as past tense forms. Personally I think Bruce's "output"
sounds better than the alternative "outputted".

Perhaps "had output" would be clearer?

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 open items list

[Bruce Momjian]

Robert Haas wrote:
> On Thu, Mar 26, 2009 at 9:36 PM, Bruce Momjian  wrote:
> >> I'm sure they will. ?But the current problem is getting beta released
> >> in the first place, and AFAICS we're all waiting for you.
> >
> > As Tom said, it is more the open items that we are waiting on, not the
> > release notes, but if if you are waiting for me to generate a list of
> > open items, I already published in inaccurate list that at least
> > _includes_ all the open items, plus lots of stuff that isn't open.
> 
> Hmm, well, Tom dropped a filtered version of your list into the open
> items wiki page.
> 
> http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
> 
> That includes a whole slough of patches that weren't submitted until
> after November 1st and which I think should probably be bumped en
> masse to 8.5:
> 
> Change behavior of statement-level triggers for inheritance cases?
> GetCurrentVirtualXIDs() (is this patch safe?)
> PQinitSSL broken in some use cases
> postgresql.conf: patch to have ParseConfigFile report all parsing
> errors, then bail
> small but useful patches for text search
> Additional DTrace Probes
> pg_standby trigger behavior is dangerous
> psql \d commands and information_schema (already in CommitFest 2009-First)
> Have \d show child tables that inherit from the specified parent
> (already in CommitFest 2009-First)

Wow, that is a large list.  Getting this all on a wiki is really what
needed to happen.  I can't keep an open list current enough to be
useful.

> I think we should also boot everything in the "pre-existing bugs"
> category, and the first two items from the "questions" category, which
> don't seem important enough to worry about at this stage of the game.
> That would leave us with 14 items, all of which look reasonably
> relevant and 8.4-related.

I think pushing "pre-existing bugs" to 8.5 is a mistake, first from a
software quality standpoint, and second because we are going to have a
lots of downtime during beta while we wait for feedback, so we can work
on some of these issues then.  These things are not going to be any
easier to fix during 8.5 than now so let's make 8.4 as good as we can
without overly-delaying it.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 open items list

[Robert Haas]

On Thu, Mar 26, 2009 at 9:36 PM, Bruce Momjian  wrote:
>> I'm sure they will.  But the current problem is getting beta released
>> in the first place, and AFAICS we're all waiting for you.
>
> As Tom said, it is more the open items that we are waiting on, not the
> release notes, but if if you are waiting for me to generate a list of
> open items, I already published in inaccurate list that at least
> _includes_ all the open items, plus lots of stuff that isn't open.

Hmm, well, Tom dropped a filtered version of your list into the open
items wiki page.

http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items

That includes a whole slough of patches that weren't submitted until
after November 1st and which I think should probably be bumped en
masse to 8.5:

Change behavior of statement-level triggers for inheritance cases?
GetCurrentVirtualXIDs() (is this patch safe?)
PQinitSSL broken in some use cases
postgresql.conf: patch to have ParseConfigFile report all parsing
errors, then bail
small but useful patches for text search
Additional DTrace Probes
pg_standby trigger behavior is dangerous
psql \d commands and information_schema (already in CommitFest 2009-First)
Have \d show child tables that inherit from the specified parent
(already in CommitFest 2009-First)

I think we should also boot everything in the "pre-existing bugs"
category, and the first two items from the "questions" category, which
don't seem important enough to worry about at this stage of the game.
That would leave us with 14 items, all of which look reasonably
relevant and 8.4-related.

Comments?

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Robert Haas]

On Thu, Mar 26, 2009 at 8:30 PM, Bruce Momjian  wrote:
> Tom Lane wrote:
>> Robert Haas  writes:
>> > OK, I am all wet.  I now understand why the editing is the
>> > time-consuming part of this job.  On the plus side it is probably
>> > possible to parallelize it to some degree by splitting the list into N
>> > pieces after the "remove insignificant items" step.
>>
>> The advantage of having one person do it (and do it over a short period
>> of time) is that you end up with a fairly uniform "voice" across the
>> whole set of notes.  Since we lack a professional copy editor, we'd have
>> a hard time coming out with something that wasn't pretty obviously a
>> patchwork if several people did bits of it.
>>
>> In any case, the release notes aren't normally a bottleneck.  I still
>> think that Bruce had his priorities out of whack in not cleaning up
>> his open-items list before doing this.  If he had done so, nobody
>> would have noticed how long the notes took.
>
> Ah, but the open items list is never done;  it is always in flux and
> will be probably until final release.  Also, you can't just put out the
> open items list becuase then there is a flurry of activity and people
> want you to keep the list current.

At this point I think we are just trying to get a list of items that
need to be done before we can release beta.  Very little, if anything,
should be getting added to that list at this point.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 release notes proof reading 1/2

[Bruce Momjian]

Guillaume Smet wrote:
> Bruce,
> 
> Here is the second set of comments:
> 
> - pg_hba.conf: it seems to me the format has changed which may break
> existing pg_hba.conf (it broke the default one of the RPM packaging).
> We should make it very visible as the format hasn't changed for a
> while. I suppose we'll put it at the top but I just don't want us to
> forget it so I mention it here.

Well we have this line:

pg_hba.conf also has extensive changes related to
migration. bjm: add markup

Do we need more?

> - "Allow recovery.conf boolean variables to take the same range of
> string alues as postgresql.conf (Bruce)" -> s/alues/values/
> - "Add support for Unicode sting and identifier specifications using
> code points, e.g. U&'d\0061t\+61' (Peter)"  -> s/sting/string/

Above fixed.

> - "Allow CREATE OR REPLACE VIEW to add columns to the _end_ of the
> view" -> formatting problem with the underscores?

I added markup, .

> - "Previously EXPLAIN VERBOSE output an internal representation of the
> query plan" -> s/output/outputs/ ?

The existing wording seems correct.

> - "Autovacuum options uses to be stored in a system table. " -> s/uses/used/ ?
> - "Allows dumps to fail if unable to acquire shared table within the
> specified amount of time. " -> shared lock instead of shared table?
> - "Add configure option --disable-float4-byval to use the old
> behavior. tgl Pass float8 and int8 by value in the server where
> possible (Zoltan Boszormenyi) " -> tgl + there's a carriage return
> missing (and a bullet)
> - "Add -M (query mode) to /contrib/pgbench (ITAGAKI Takahiro)"
> ->Itagaki san's name inconsistent with other mentions of his name

Above all fixed, thanks.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Robert Haas]

On Thu, Mar 26, 2009 at 9:28 PM, Bruce Momjian  wrote:
>> At this point I think we are just trying to get a list of items that
>> need to be done before we can release beta.  Very little, if anything,
>> should be getting added to that list at this point.
>
> You can say that, but things are going to be uncovered during beta
> regularly.

I'm sure they will.  But the current problem is getting beta released
in the first place, and AFAICS we're all waiting for you.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 release notes proof reading 1/2

[Bruce Momjian]

Guillaume Smet wrote:
> Hi,
> 
> Just to warn people that I'm making a comprehensive proof reading of
> the release notes.
> 
> Here are the first comments:
> 
> - "This was available previously via a configure
> --enable-integer-datetimes (Neil Conway)" -> I don't think we need
> Neil's name in the details
> - "New semi- and ansi-joins (Tom)" -> s/ansi/anti/
> - "While semi-joins merely replace existing IN joins, anti-joins are a
> new capability for NOT IN and NOT EXIST clauses (Tom) This improves
> optimization possibilities." -> remove the "(Tom)", s/EXIST/EXISTS/
> - "IN and EXISTs now provide similar performance for equivalent
> clauses. " -> s/EXISTs/EXISTS/

Above fixed.

> - "A new constraint_exclusion setting, "partition", which matches this
> behavior." -> this sentence is weird

Yep, new wording:

   A new constraint_exclusion setting, "partition", was added to match this
   behavior.

> - "Move that server statistics file into the subdirectory pg_stat_tmp"
> -> s/that/the/ ? I think this item can be grouped with the following
> one "Allow the location of the new pg_stat_tmp to be configured via
> the stats_temp_directory setting"

Agreed, new merged text:

 
  
   Move the server statistics file into the subdirectory pg_stat_tmp
   and allow its location to be specified via stats_temp_directory (Magnus)
  

  
   This allows the statistics file to be placed in a RAM-resident
   directory to reduce I/O requirements.  On startup/shutdown, the
   file is copied to the top-level $PGDATA directory so it is preserved
   between restarts.
  
 

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Bruce Momjian]

Robert Haas wrote:
> On Thu, Mar 26, 2009 at 8:30 PM, Bruce Momjian  wrote:
> > Tom Lane wrote:
> >> Robert Haas  writes:
> >> > OK, I am all wet. ?I now understand why the editing is the
> >> > time-consuming part of this job. ?On the plus side it is probably
> >> > possible to parallelize it to some degree by splitting the list into N
> >> > pieces after the "remove insignificant items" step.
> >>
> >> The advantage of having one person do it (and do it over a short period
> >> of time) is that you end up with a fairly uniform "voice" across the
> >> whole set of notes. ?Since we lack a professional copy editor, we'd have
> >> a hard time coming out with something that wasn't pretty obviously a
> >> patchwork if several people did bits of it.
> >>
> >> In any case, the release notes aren't normally a bottleneck. ?I still
> >> think that Bruce had his priorities out of whack in not cleaning up
> >> his open-items list before doing this. ?If he had done so, nobody
> >> would have noticed how long the notes took.
> >
> > Ah, but the open items list is never done; ?it is always in flux and
> > will be probably until final release. ?Also, you can't just put out the
> > open items list becuase then there is a flurry of activity and people
> > want you to keep the list current.
> 
> At this point I think we are just trying to get a list of items that
> need to be done before we can release beta.  Very little, if anything,
> should be getting added to that list at this point.

You can say that, but things are going to be uncovered during beta
regularly.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Robert Haas]

On Thu, Mar 26, 2009 at 9:35 PM, Bruce Momjian  wrote:
> Robert Haas wrote:
>> On Thu, Mar 26, 2009 at 8:31 PM, Bruce Momjian  wrote:
>> > Josh Berkus wrote:
>> >> All,
>> >>
>> >> > In any case, the release notes aren't normally a bottleneck. ?I still
>> >> > think that Bruce had his priorities out of whack in not cleaning up
>> >> > his open-items list before doing this. ?If he had done so, nobody
>> >> > would have noticed how long the notes took.
>> >>
>> >> Yes, although Bruce *has* asked for help in cleaning up the open-items 
>> >> list.
>> >
>> > Uh, not now; ?I put up the list only so people wouldn't think I was
>> > hiding things, and I said it wasn't cleaned up. ?What is so hard for
>> > people to understand about that. ?This is quite annoying.
>>
>> If I understand correctly, you're saying that you didn't want to have
>> help with the release notes, you don't want help cleaning up your
>> mailbox, but you do want beta to wait until you are done doing those
>> things and any resulting action items have been completed.  Is that
>> right?
>>
>> I don't think anyone thinks you're hiding anything, but I think there
>> is a general desire to move this thing along as quickly as reasonably
>> possible.
>
> And to answer your question about the release notes, Tom is right that
> it probably has to be done by one person because it needs a consistent
> voice, and someone has to get the entire release notes in their head so
> they can see logical sections/groupings, etc.  Once it is done, as it is
> now, people can jump in and add improvements.

Fair enough.  I'm just looking for a way to help out so we can get
this done.  I am not coming up with much that seems to be helpful.  I
was planning to do some comments on your mailbox file but Tom beat me
to the punch.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Bruce Momjian]

Robert Haas wrote:
> On Thu, Mar 26, 2009 at 9:28 PM, Bruce Momjian  wrote:
> >> At this point I think we are just trying to get a list of items that
> >> need to be done before we can release beta. ?Very little, if anything,
> >> should be getting added to that list at this point.
> >
> > You can say that, but things are going to be uncovered during beta
> > regularly.
> 
> I'm sure they will.  But the current problem is getting beta released
> in the first place, and AFAICS we're all waiting for you.

As Tom said, it is more the open items that we are waiting on, not the
release notes, but if if you are waiting for me to generate a list of
open items, I already published in inaccurate list that at least
_includes_ all the open items, plus lots of stuff that isn't open.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Bruce Momjian]

Robert Haas wrote:
> On Thu, Mar 26, 2009 at 8:31 PM, Bruce Momjian  wrote:
> > Josh Berkus wrote:
> >> All,
> >>
> >> > In any case, the release notes aren't normally a bottleneck. ?I still
> >> > think that Bruce had his priorities out of whack in not cleaning up
> >> > his open-items list before doing this. ?If he had done so, nobody
> >> > would have noticed how long the notes took.
> >>
> >> Yes, although Bruce *has* asked for help in cleaning up the open-items 
> >> list.
> >
> > Uh, not now; ?I put up the list only so people wouldn't think I was
> > hiding things, and I said it wasn't cleaned up. ?What is so hard for
> > people to understand about that. ?This is quite annoying.
> 
> If I understand correctly, you're saying that you didn't want to have
> help with the release notes, you don't want help cleaning up your
> mailbox, but you do want beta to wait until you are done doing those
> things and any resulting action items have been completed.  Is that
> right?
> 
> I don't think anyone thinks you're hiding anything, but I think there
> is a general desire to move this thing along as quickly as reasonably
> possible.

And to answer your question about the release notes, Tom is right that
it probably has to be done by one person because it needs a consistent
voice, and someone has to get the entire release notes in their head so
they can see logical sections/groupings, etc.  Once it is done, as it is
now, people can jump in and add improvements.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] psql: Make tab completion work for ANALYZE VERBOSE ...

[Greg Sabino Mullane]

Quick patch to fix the fact that the EXPLAIN ANALYZE VERBOSE is clobbering
tab-completion for ANALYZE VERBOSE.

-- 
Greg Sabino Mullane g...@endpoint.com
End Point Corporation
PGP Key: 0x14964AC8
Index: tab-complete.c
===
RCS file: /projects/cvsroot/pgsql/src/bin/psql/tab-complete.c,v
retrieving revision 1.180
diff -c -r1.180 tab-complete.c
*** tab-complete.c	24 Feb 2009 10:06:34 -	1.180
--- tab-complete.c	27 Mar 2009 01:29:06 -
***
*** 1627,1633 
  	else if (pg_strcasecmp(prev_wd, "VERBOSE") == 0 &&
  			 pg_strcasecmp(prev3_wd, "VACUUM") != 0 &&
  			 pg_strcasecmp(prev4_wd, "VACUUM") != 0 &&
! 			 (pg_strcasecmp(prev2_wd, "ANALYZE") == 0 ||
  			  pg_strcasecmp(prev2_wd, "EXPLAIN") == 0))
  	{
  		static const char *const list_EXPLAIN[] =
--- 1627,1634 
  	else if (pg_strcasecmp(prev_wd, "VERBOSE") == 0 &&
  			 pg_strcasecmp(prev3_wd, "VACUUM") != 0 &&
  			 pg_strcasecmp(prev4_wd, "VACUUM") != 0 &&
! 			 ((pg_strcasecmp(prev2_wd, "ANALYZE") == 0 && 
! 			   pg_strcasecmp(prev3_wd, "EXPLAIN") == 0) ||
  			  pg_strcasecmp(prev2_wd, "EXPLAIN") == 0))
  	{
  		static const char *const list_EXPLAIN[] =


signature.asc
Description: OpenPGP digital signature

Re: [HACKERS] small but useful patches for text search

[Bruce Momjian]

Robert Haas wrote:
> On Thu, Mar 26, 2009 at 8:31 PM, Bruce Momjian  wrote:
> > Josh Berkus wrote:
> >> All,
> >>
> >> > In any case, the release notes aren't normally a bottleneck. ?I still
> >> > think that Bruce had his priorities out of whack in not cleaning up
> >> > his open-items list before doing this. ?If he had done so, nobody
> >> > would have noticed how long the notes took.
> >>
> >> Yes, although Bruce *has* asked for help in cleaning up the open-items 
> >> list.
> >
> > Uh, not now; ?I put up the list only so people wouldn't think I was
> > hiding things, and I said it wasn't cleaned up. ?What is so hard for
> > people to understand about that. ?This is quite annoying.
> 
> If I understand correctly, you're saying that you didn't want to have
> help with the release notes, you don't want help cleaning up your
> mailbox, but you do want beta to wait until you are done doing those
> things and any resulting action items have been completed.  Is that
> right?
> 
> I don't think anyone thinks you're hiding anything, but I think there
> is a general desire to move this thing along as quickly as reasonably
> possible.

I can't start on the release notes, stop to update an open items list,
jump back to the release notes, update open items as people do stuff,
etc.  The release notes have to be done in one big chunk of time, pretty
much.

I will need help cleaning out the open items list once I get a final
list together, though Tom's comments on the existing list has helped
greatly.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Robert Haas]

On Thu, Mar 26, 2009 at 8:31 PM, Bruce Momjian  wrote:
> Josh Berkus wrote:
>> All,
>>
>> > In any case, the release notes aren't normally a bottleneck.  I still
>> > think that Bruce had his priorities out of whack in not cleaning up
>> > his open-items list before doing this.  If he had done so, nobody
>> > would have noticed how long the notes took.
>>
>> Yes, although Bruce *has* asked for help in cleaning up the open-items list.
>
> Uh, not now;  I put up the list only so people wouldn't think I was
> hiding things, and I said it wasn't cleaned up.  What is so hard for
> people to understand about that.  This is quite annoying.

If I understand correctly, you're saying that you didn't want to have
help with the release notes, you don't want help cleaning up your
mailbox, but you do want beta to wait until you are done doing those
things and any resulting action items have been completed.  Is that
right?

I don't think anyone thinks you're hiding anything, but I think there
is a general desire to move this thing along as quickly as reasonably
possible.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Proper subject lines (was: small but useful patches for text search)

[Greg Sabino Mullane]

-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160


> In any case, the release notes aren't normally a bottleneck.
...

Could I respectfully request people make an effort to change the
subject lines when the thread radically moves away from its
original purpose? Modern mail systems don't thread by subject
lines anyway, so no worries there. Thanks.

- --
Greg Sabino Mullane g...@turnstep.com
PGP Key: 0x14964AC8 200903262040
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-BEGIN PGP SIGNATURE-

iEYEAREDAAYFAknMIKYACgkQvJuQZxSWSsi3TgCfQJn+kbfWEJOMbj9WfPpXpgPN
GHUAoIcCnekhlDrDxaqpfJ9lAnMtxQDj
=icok
-END PGP SIGNATURE-


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSL over Unix-domain sockets

[Bruce Momjian]

Peter Eisentraut wrote:
> Bruce Momjian wrote:
> > I thought the logical solution to this was to place the socket in a
> > secure directory and not bother with SSL at all.
> 
> How would a client algorithmically determine whether the server socket 
> was in a "secure" directory?

You have to configure your client to know that, but don't you need to
configure your client for SSL too?

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Bruce Momjian]

Tom Lane wrote:
> Josh Berkus  writes:
> > Yes, although Bruce *has* asked for help in cleaning up the open-items list.
> 
> I spent several hours on that on Saturday, and more or less got the bird
> in response... the way Bruce has that page set up, only he can do any
> actual item removal, the rest of us can only comment.

Sorry you felt that way.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Bruce Momjian]

Josh Berkus wrote:
> All,
> 
> > In any case, the release notes aren't normally a bottleneck.  I still
> > think that Bruce had his priorities out of whack in not cleaning up
> > his open-items list before doing this.  If he had done so, nobody
> > would have noticed how long the notes took.
> 
> Yes, although Bruce *has* asked for help in cleaning up the open-items list.

Uh, not now;  I put up the list only so people wouldn't think I was
hiding things, and I said it wasn't cleaned up.  What is so hard for
people to understand about that.  This is quite annoying.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Bruce Momjian]

Tom Lane wrote:
> Robert Haas  writes:
> > OK, I am all wet.  I now understand why the editing is the
> > time-consuming part of this job.  On the plus side it is probably
> > possible to parallelize it to some degree by splitting the list into N
> > pieces after the "remove insignificant items" step.
> 
> The advantage of having one person do it (and do it over a short period
> of time) is that you end up with a fairly uniform "voice" across the
> whole set of notes.  Since we lack a professional copy editor, we'd have
> a hard time coming out with something that wasn't pretty obviously a
> patchwork if several people did bits of it.
> 
> In any case, the release notes aren't normally a bottleneck.  I still
> think that Bruce had his priorities out of whack in not cleaning up
> his open-items list before doing this.  If he had done so, nobody
> would have noticed how long the notes took.

Ah, but the open items list is never done;  it is always in flux and
will be probably until final release.  Also, you can't just put out the
open items list becuase then there is a flurry of activity and people
want you to keep the list current.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Bruce Momjian]

Robert Haas wrote:
> OK, I am all wet.  I now understand why the editing is the
> time-consuming part of this job.  On the plus side it is probably
> possible to parallelize it to some degree by splitting the list into N
> pieces after the "remove insignificant items" step.
> 
> With respect to this item:
> Disable appending of the epoch date/time when '%' escapes are missing
> in log_filename (Robert Haas)
> I might suggest explaining it this way:
> This change makes it easier to use PostgreSQL in conjunction with an
> external log rotation tool.
> 
> The following item uses "of" where it should say "if":
> Throw an error of an escape character is the last character in a LIKE
> pattern (nothing to escape) (Tom)

Thanks, I think Tom fixed that one already.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Error message and infinite date and timestamp conversion in XML

[Bernd Helmle]

map_sql_value_to_xml_value() currently errors out with a more or less vague 
error message, when a date or timestamp datatype with an infinite value is 
converted to XML. This is likely to create some confusion, especially when 
you have to debug some complex procedures and involved XML conversions. I 
propose to add the attached DETAIL to this error message, so people will 
get an idea what's currently going wrong.



--
 Thanks

   Bernd*** a/src/backend/utils/adt/xml.c
--- b/src/backend/utils/adt/xml.c
***
*** 1627,1632  map_sql_value_to_xml_value(Datum value, Oid type)
--- 1627,1633 
  	if (DATE_NOT_FINITE(date))
  		ereport(ERROR,
  (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE),
+  errdetail("XML does not support infinite date values."),
   errmsg("date out of range")));
  	j2date(date + POSTGRES_EPOCH_JDATE,
  		   &(tm.tm_year), &(tm.tm_mon), &(tm.tm_mday));
***
*** 1649,1654  map_sql_value_to_xml_value(Datum value, Oid type)
--- 1650,1656 
  	if (TIMESTAMP_NOT_FINITE(timestamp))
  		ereport(ERROR,
  (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE),
+  errdetail("XML does not support infinite timestamp values."),
   errmsg("timestamp out of range")));
  	else if (timestamp2tm(timestamp, NULL, &tm, &fsec, NULL, NULL) == 0)
  		EncodeDateTime(&tm, fsec, NULL, &tzn, USE_XSD_DATES, buf);
***
*** 1675,1680  map_sql_value_to_xml_value(Datum value, Oid type)
--- 1677,1683 
  	if (TIMESTAMP_NOT_FINITE(timestamp))
  		ereport(ERROR,
  (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE),
+  errdetail("XML does not support infinite timestamp values."),
   errmsg("timestamp out of range")));
  	else if (timestamp2tm(timestamp, &tz, &tm, &fsec, &tzn, NULL) == 0)
  		EncodeDateTime(&tm, fsec, &tz, &tzn, USE_XSD_DATES, buf);

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] maintenance_work_mem and autovacuum

[Josh Berkus]

On 3/26/09 4:10 PM, Tom Lane wrote:

Simon Riggs  writes:

On Thu, 2009-03-26 at 13:43 -0700, Josh Berkus wrote:

That said, it would be unnecessary if I could use ROLES to set
parameters more reliably 



Hmmm, perhaps the right way to do this is to have a user called
"autovacuum" that is used to perform autovacuums.


Interesting idea, but maybe we should wait on it until we see what
(if any) changes we are going to put into the relationship between
roles and parameters.  Josh seems not to be happy with that ...


Well, it would work at present for this, because it's a login role, 
presumably.


--Josh


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] maintenance_work_mem and autovacuum

[Tom Lane]

Simon Riggs  writes:
> On Thu, 2009-03-26 at 13:43 -0700, Josh Berkus wrote:
>> That said, it would be unnecessary if I could use ROLES to set 
>> parameters more reliably 

> Hmmm, perhaps the right way to do this is to have a user called
> "autovacuum" that is used to perform autovacuums. 

Interesting idea, but maybe we should wait on it until we see what
(if any) changes we are going to put into the relationship between
roles and parameters.  Josh seems not to be happy with that ...

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: B-Tree emulation for GIN

[Tom Lane]

Jeff Davis  writes:
> Also, if extractQuery is non-strict, shouldn't we call it and see if it
> returns some useful keys?

Perhaps.  One risk factor for approaching it that way is that there are
probably a lot of opclasses out there that haven't bothered to mark
these functions strict, since it's never mattered before.  (A handy
example is that the brand new btree_gin opclasses did not bother, as
submitted; though in a fit of paranoia I made them do so before
committing.)  If the extractQuery function isn't actually guarding
against this, you'll get a crash.

That's not a showstopper reason not to change, of course, but it does
mean that I'd like to see an actual use case for a non-strict GIN index
operator before taking any risk.  Note that IS NULL isn't an operator,
so even if we were to try to support it in GIN, that would be a
different code path (just as it is in btree).

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "maxretries" option of pg_standby doesn't work as expected

[Tom Lane]

Fujii Masao  writes:
>> Set the maximum number of times to retry the copy or link command

> maxretries option of pg_standby is documented as above, but actually
> indicates the maximum number of times to *try* the copy or link command.
> So, if "-r 0" is specified, pg_standby always fails.

> Attached is the patch which adjusted the code so that maxretries actually
> indicates the number of retry.

Looks correct to me; applied to HEAD and 8.3.  Thanks!

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: B-Tree emulation for GIN

[Jeff Davis]

On Wed, 2009-03-25 at 19:31 -0400, Tom Lane wrote:
> * I'd also like to come to some agreement about getting rid of the
> fail-on-NULL-scankey problem in newScanKey().  As I noted in the
> comment there, we could make that work cleanly if we are willing to
> assume that all GIN-indexable operators are strict.  We already assume
> the same for hash and btree operators, so it doesn't seem like a big
> problem to do this, but I wonder if there are any objections.

"IS NULL" is indexable in a btree and non-strict, so there is at least
some precedent.

Also, if extractQuery is non-strict, shouldn't we call it and see if it
returns some useful keys? If so, I don't see a reason to assume that
nothing matches.

If the opclass author wants a search against NULL to mean "matches
nothing", they can just make extractQuery non-strict and return -1.

However, if extractQuery is strict or returns NULL, I'm fine with either
an error or assuming "nothing matches". I don't see a functionality
difference either way, so we should just document whatever seems to make
the most sense.

Regards,
Jeff Davis



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] gettext, plural form and translation

[Peter Eisentraut]

On Saturday 21 March 2009 01:01:57 Sergey Burladyan wrote:
> Alvaro Herrera  writes:
> > Care to submit a patch?
>
> this is it, i divide it into two, first is change source and second is
> change ru.po file for psql.

I have now committed a more extensive pluralization, but your case is included 
there.

As for the ru.po file, please see http://babel.postgresql.org/.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Partitioning feature ...

[Emmanuel Cecchet]

Hi Kedar,

First of all, congratulations for the excellent work.
I have some comments and questions.

In get_relevent_partition (btw, relevant is spelled with an a) you are 
maintaining 2 lists. I guess this is only useful for multi-column 
partitions, right?
If you have a single column partition (without subpartitions), I think 
you could directly return on the first match (without maintaining any 
list) since you guarantee that there is no overlap between partitions.
A simple but effective optimization for inserts consists of caching the 
last partition used (consecutive inserts often go to the same partition) 
and try it first before going through the whole loop.


The update trigger should first check if the tuple needs to be moved. If 
the updated tuple still matches the constraints of the partitions it 
will not have to be moved and will save a lot of overhead.


The COPY operation should probably be optimized to use the same code as 
the one in the insert trigger for partitioned tables. I guess some code 
could be factorized in COPY to make the inserts more efficient.


The current trigger approach should prevent other triggers to be added 
to the table, or you should make sure that the partition trigger is 
always the one to execute last.


As we don't have automatic partition creation, it would be interesting 
to have an optional mechanism to deal with tuples that don't match any 
partition (very useful when you do bulk insert and some new data require 
a new partition). Having a simple overflow partition or an error logging 
mechanism would definitely help to identify these tuples and prevent 
things like large COPY operations to fail.



Looking forward to your responses,
Emmanuel



We are implementing table partitioning feature to support Range and 
Hash partitions. Please find attached, the WIP patch and test-cases.


The syntax used conforms to most of the suggestions mentioned in 
http://archives.postgresql.org/pgsql-hackers/2008-01/msg00413.php, 
barring the following:
-- Specification of partition names is optional. System will be able 
to generate partition names in such cases.

-- Sub partitioning

We are maintaining a system catalog(pg_partition) for partition 
meta-data. System will look-up this table to find appropriate 
partition to operate on.

System internally uses low-level 'C' triggers to row-movement.

Regards,
--
Kedar.






  



--
Emmanuel Cecchet
Aster Data Systems
Web: http://www.asterdata.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] maintenance_work_mem and autovacuum

[Bernd Helmle]

--On Donnerstag, März 26, 2009 13:43:45 -0700 Josh Berkus 
 wrote:



I actually have a client who does both automated and manual vacuums.
Having two settings would definitely be convenient for them.


I often found people doing this running within a) their own superuser with 
special GUCs set or b) using their own scripts which are setting 
maintenance_work_mem accordingly.


Often people are very confused about the number of *_vaccum_* settings 
already present: they sometimes have a really hard time to learn the 
difference between manual and auto vaccum and the interaction of 
default/inherited settings.


That said, having a separate setting would be consequent, but it seems to 
me it's enough there to satisfy all needs, isn't it?


--
 Thanks

   Bernd

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] maintenance_work_mem and autovacuum

[Josh Berkus]

Simon,


Hmmm, perhaps the right way to do this is to have a user called
"autovacuum" that is used to perform autovacuums.


This makes sense, depending on which autovac params actually get picked 
up from the session.



Seems like a nice small change for 8.4?


Hmmm.  Maybe not small enough.

--Josh


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] maintenance_work_mem and autovacuum

[Simon Riggs]

On Thu, 2009-03-26 at 13:43 -0700, Josh Berkus wrote:
> > I agree with Magnus' original reasoning: we can have more than one
> > autovacuum process, so we may have autovacuum_max_workers active and so
> > the work mem they use must be smaller. For maintenance_work_mem we would
> > typically only have one session using it at any time, so we either have
> > to start hardcoding the value in scripts or accept the fact it has been
> > set lower.
> 
> I actually have a client who does both automated and manual vacuums. 
> Having two settings would definitely be convenient for them.
> 
> That said, it would be unnecessary if I could use ROLES to set 
> parameters more reliably 

Hmmm, perhaps the right way to do this is to have a user called
"autovacuum" that is used to perform autovacuums. 

That way we can actually get rid of a few autovacuum_* parameters
without losing function, and yet add the capability to change
maintenance_work_mem just for autovacuum. Avoid some special case code
also, like setting of zero_damaged_pages.

Seems like a nice small change for 8.4?

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GIN versus zero-key queries

[Jeff Davis]

On Wed, 2009-03-25 at 13:25 -0400, Tom Lane wrote:
> I am not sure whether the statement in 52.5 is still accurate, though.
> We have an API definition by which extractQuery can distinguish "all
> match" from "no match".  If we just legislate that "some match" isn't
> a valid behavior for zero-key queries, then the code is correct and the
> documentation is wrong.  However, if the above quote is correct, then
> I think newScanKey() is buggy.

Legislating that "some match" is invalid for zero-key queries seems
reasonable to me.

If the operator class author wants it to throw an error for zero keys
(as the documentation says should happen), they can do that easily
enough without being forced. However, if the opclass author finds "all
match" to be useful behavior (which seems reasonable), I don't see a
reason to stop them.

Regards,
Jeff Davis


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] maintenance_work_mem and autovacuum

[Josh Berkus]

I agree with Magnus' original reasoning: we can have more than one
autovacuum process, so we may have autovacuum_max_workers active and so
the work mem they use must be smaller. For maintenance_work_mem we would
typically only have one session using it at any time, so we either have
to start hardcoding the value in scripts or accept the fact it has been
set lower.


I actually have a client who does both automated and manual vacuums. 
Having two settings would definitely be convenient for them.


That said, it would be unnecessary if I could use ROLES to set 
parameters more reliably 


;-)

--Josh


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Potential problem with HOT and indexes?

[Tom Lane]

Greg Stark  writes:
> On Thu, Mar 26, 2009 at 7:41 PM, Tom Lane  wrote:
>> I realized what was bothering me about that patch: it could reset
>> indcheckxmin too soon, ie, while there are still transactions that
>> shouldn't use the index.

> That doesn't sound like the right solution. What we want to do is use
> the indcheckxmin for the newly built index if any -- ignoring any
> indcheckxmin from the previous index. The old value is completely
> irrelevant.

> What you describe would be right for the ALTER INDEX commands like
> RENAME or SET/RESET which might update the xmin without rebuilding the
> index contents. Likewise for CLUSTER/SET WITHOUT CLUSTER.

Hmm.  You're right, I was thinking about the case of a generic update on
the pg_index row, but this particular patch is focused on REINDEX which
has just rebuilt the index contents.  So never mind that.

However, I'm still unconvinced that it's safe to clear indcheckxmin
unconditionally.  Isn't it possible for such a REINDEX to encounter
HOT chains that are (still) broken?  Consider a REINDEX launched
right after a CREATE INDEX CONCURRENTLY (possibly a failed one),
while there are still open transactions older than the C.I.C.

We could reorganize the function slightly so that the IndexInfo is
still available after the PG_TRY block, and then do something like
"if (!indexInfo->ii_BrokenHotChain) reset indcheckxmin".  This would
ensure that we didn't wipe the flag if index_build had just set it.
If I'm wrong and this can never happen, no harm done.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Potential problem with HOT and indexes?

[Greg Stark]

On Thu, Mar 26, 2009 at 7:41 PM, Tom Lane  wrote:
> Gregory Stark  writes:
>>> Another thought now though. What if someone updates the pg_index entry --
>>> since we never reset indcheckxmin then the new tuple will have a new xmin 
>>> and
>>> will suddenly become invisible again for no reason.
>
>> Fixing this for REINDEX is fairly straightforward I think. It already updates
>> the pg_index line to fix indisvalid and indisready. see:
>
> I realized what was bothering me about that patch: it could reset
> indcheckxmin too soon, ie, while there are still transactions that
> shouldn't use the index.
>
> I propose that we modify it slightly: if we are updating a pg_index
> row, and indcheckxmin is set, *and the old xmin is below the GlobalXmin
> horizon*, then reset indcheckxmin.  Otherwise leave it set, which will
> mean that we postpone the time when the index becomes usable to
> everyone, but it won't risk breaking anything.

That doesn't sound like the right solution. What we want to do is use
the indcheckxmin for the newly built index if any -- ignoring any
indcheckxmin from the previous index. The old value is completely
irrelevant.

What you describe would be right for the ALTER INDEX commands like
RENAME or SET/RESET which might update the xmin without rebuilding the
index contents. Likewise for CLUSTER/SET WITHOUT CLUSTER.

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Potential problem with HOT and indexes?

[Tom Lane]

Gregory Stark  writes:
>> Another thought now though. What if someone updates the pg_index entry --
>> since we never reset indcheckxmin then the new tuple will have a new xmin and
>> will suddenly become invisible again for no reason.

> Fixing this for REINDEX is fairly straightforward I think. It already updates
> the pg_index line to fix indisvalid and indisready. see:

I realized what was bothering me about that patch: it could reset
indcheckxmin too soon, ie, while there are still transactions that
shouldn't use the index.

I propose that we modify it slightly: if we are updating a pg_index
row, and indcheckxmin is set, *and the old xmin is below the GlobalXmin
horizon*, then reset indcheckxmin.  Otherwise leave it set, which will
mean that we postpone the time when the index becomes usable to
everyone, but it won't risk breaking anything.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] maintenance_work_mem and autovacuum

[Simon Riggs]

On Thu, 2009-03-26 at 19:46 +0100, Guillaume Smet wrote:
> On Thu, Mar 26, 2009 at 7:34 PM, Simon Riggs  wrote:
> > Why do we have separate parameters for autovacuum and vacuum, except for
> > maintenance_work_mem?
> >
> > Should we also have autovacuum_work_mem?
> 
> We already discussed it here:
> http://archives.postgresql.org/message-id/49353a69.20...@hagander.net
> 
> It resulted in a doc patch - not sure it's sufficient but it's
> interesting to read this thread before discussing further.

Hmmm, OK, read that, thanks. Must have missed that thread earlier.

Tom was asking for evidence of a need for them to be different. I don't
see it as a case that requires performance results.

I agree with Magnus' original reasoning: we can have more than one
autovacuum process, so we may have autovacuum_max_workers active and so
the work mem they use must be smaller. For maintenance_work_mem we would
typically only have one session using it at any time, so we either have
to start hardcoding the value in scripts or accept the fact it has been
set lower.

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 open items list updated

[Tom Lane]

Magnus Hagander  writes:
> Tom Lane wrote:
>> http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
>> Anybody who wants to start cleaning these things up, have at it.

> We were in agreement to move the Win32 namespace issue to the TODO list,
> right? Unless anybody objects, I'll go ahead and do that. So if you
> think it should really be on that list, please speak up :-)

Sure.  I think a fair number of these items will end up being punted to
TODO; they're just here because the decision isn't clear yet.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 open items list updated

[Magnus Hagander]

Tom Lane wrote:
> Since Bruce seems not to be in a hurry to update his open-items mailbox,
> I've taken the liberty of adding entries for all the items that I think
> are relevant for 8.4 to the wiki page:
> http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items
> 
> Anybody who wants to start cleaning these things up, have at it.

We were in agreement to move the Win32 namespace issue to the TODO list,
right? Unless anybody objects, I'll go ahead and do that. So if you
think it should really be on that list, please speak up :-)

//Magnus


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] tuplestore API problem

[Tom Lane]

Simon Riggs  writes:
> Sounds very similar to the solution that you just removed from the hash
> join code for performance reasons. Flushing memory when we overflow
> sounds like an artifact from the time when tuplestore split from
> tuplesort. Can't we keep the appropriate rows in memory and scroll
> through them?

We're not doing a fundamental redesign of the window function support
right now.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] 8.4 open items list updated

[Tom Lane]

Since Bruce seems not to be in a hurry to update his open-items mailbox,
I've taken the liberty of adding entries for all the items that I think
are relevant for 8.4 to the wiki page:
http://wiki.postgresql.org/wiki/PostgreSQL_8.4_Open_Items

Anybody who wants to start cleaning these things up, have at it.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] maintenance_work_mem and autovacuum

[Guillaume Smet]

On Thu, Mar 26, 2009 at 7:34 PM, Simon Riggs  wrote:
> Why do we have separate parameters for autovacuum and vacuum, except for
> maintenance_work_mem?
>
> Should we also have autovacuum_work_mem?

We already discussed it here:
http://archives.postgresql.org/message-id/49353a69.20...@hagander.net

It resulted in a doc patch - not sure it's sufficient but it's
interesting to read this thread before discussing further.

-- 
Guillaume

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] maintenance_work_mem and autovacuum

[Simon Riggs]

Why do we have separate parameters for autovacuum and vacuum, except for
maintenance_work_mem?

Should we also have autovacuum_work_mem?

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] display previous query string of idle-in-transaction

[Simon Riggs]

On Wed, 2009-03-25 at 18:08 +0900, Tatsuhito Kasahara wrote:

> If we can also check previous query_string of idle-in-transaction,
> it is useful for analysis of long transaction problem.

I'm more interested in the problem itself. Why do you think there is a
problem and why does knowing this help you? I had similar problems
recently, so I'm interested in thoughts around this.

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] tuplestore API problem

[Simon Riggs]

On Thu, 2009-03-26 at 12:57 -0400, Tom Lane wrote:
> If work_mem is small enough, that means the tuplestore is
> forced into dump-to-disk mode, which means it releases all its
> in-memory tuples.  And guess what: the ScanTupleSlot is pointing at
> one of those, it doesn't have its own copy of the tuple.  So we wind
> up trying to read from a trashed bit of memory.
> 
> A brute-force solution is to change tuplestore_gettupleslot() so that
> it always copies the tuple, but this would be wasted cycles for most
> uses of tuplestores.  I'm thinking of changing
> tuplestore_gettupleslot's API
> to add a bool parameter specifying whether the caller wants to force
> a copy.
> 
> Comments, better ideas?

Sounds very similar to the solution that you just removed from the hash
join code for performance reasons. Flushing memory when we overflow
sounds like an artifact from the time when tuplestore split from
tuplesort. Can't we keep the appropriate rows in memory and scroll
through them?

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Potential Join Performance Issue

[Tom Lane]

"Lawrence, Ramon"  writes:
> Attached is a patch that will disable the physical-tlist optimization
> for hash join if the number of batches is greater than 1.  The patch and
> performance results were created by Michael Henderson (graduate
> student).

I've applied the attached modified version of this patch.

regards, tom lane

Index: src/backend/nodes/outfuncs.c
===
RCS file: /cvsroot/pgsql/src/backend/nodes/outfuncs.c,v
retrieving revision 1.355
diff -c -r1.355 outfuncs.c
*** src/backend/nodes/outfuncs.c21 Mar 2009 00:04:39 -  1.355
--- src/backend/nodes/outfuncs.c26 Mar 2009 15:19:43 -
***
*** 1448,1453 
--- 1448,1454 
_outJoinPathInfo(str, (JoinPath *) node);
  
WRITE_NODE_FIELD(path_hashclauses);
+   WRITE_INT_FIELD(num_batches);
  }
  
  static void
Index: src/backend/optimizer/path/costsize.c
===
RCS file: /cvsroot/pgsql/src/backend/optimizer/path/costsize.c,v
retrieving revision 1.205
diff -c -r1.205 costsize.c
*** src/backend/optimizer/path/costsize.c   21 Mar 2009 00:04:39 -  
1.205
--- src/backend/optimizer/path/costsize.c   26 Mar 2009 15:19:43 -
***
*** 1880,1885 
--- 1880,1887 
&numbatches,
&num_skew_mcvs);
virtualbuckets = (double) numbuckets *(double) numbatches;
+   /* mark the path with estimated # of batches */
+   path->num_batches = numbatches;
  
/*
 * Determine bucketsize fraction for inner relation.  We use the 
smallest
Index: src/backend/optimizer/plan/createplan.c
===
RCS file: /cvsroot/pgsql/src/backend/optimizer/plan/createplan.c,v
retrieving revision 1.256
diff -c -r1.256 createplan.c
*** src/backend/optimizer/plan/createplan.c 21 Mar 2009 00:04:39 -  
1.256
--- src/backend/optimizer/plan/createplan.c 26 Mar 2009 15:19:44 -
***
*** 1910,1915 
--- 1910,1919 
/* We don't want any excess columns in the hashed tuples */
disuse_physical_tlist(inner_plan, best_path->jpath.innerjoinpath);
  
+   /* If we expect batching, suppress excess columns in outer tuples too */
+   if (best_path->num_batches > 1)
+   disuse_physical_tlist(outer_plan, 
best_path->jpath.outerjoinpath);
+ 
/*
 * If there is a single join clause and we can identify the outer
 * variable as a simple column reference, supply its identity for
Index: src/backend/optimizer/util/pathnode.c
===
RCS file: /cvsroot/pgsql/src/backend/optimizer/util/pathnode.c,v
retrieving revision 1.150
diff -c -r1.150 pathnode.c
*** src/backend/optimizer/util/pathnode.c   27 Feb 2009 00:06:27 -  
1.150
--- src/backend/optimizer/util/pathnode.c   26 Mar 2009 15:19:44 -
***
*** 1480,1488 
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
!   /* A hashjoin never has pathkeys, since its ordering is unpredictable */
pathnode->jpath.path.pathkeys = NIL;
pathnode->path_hashclauses = hashclauses;
  
cost_hashjoin(pathnode, root, sjinfo);
  
--- 1480,1499 
pathnode->jpath.outerjoinpath = outer_path;
pathnode->jpath.innerjoinpath = inner_path;
pathnode->jpath.joinrestrictinfo = restrict_clauses;
!   /*
!* A hashjoin never has pathkeys, since its output ordering is
!* unpredictable due to possible batching.  XXX If the inner relation is
!* small enough, we could instruct the executor that it must not batch,
!* and then we could assume that the output inherits the outer 
relation's
!* ordering, which might save a sort step.  However there is 
considerable
!* downside if our estimate of the inner relation size is badly off.
!* For the moment we don't risk it.  (Note also that if we wanted to 
take
!* this seriously, joinpath.c would have to consider many more paths for
!* the outer rel than it does now.)
!*/
pathnode->jpath.path.pathkeys = NIL;
pathnode->path_hashclauses = hashclauses;
+   /* cost_hashjoin will fill in pathnode->num_batches */
  
cost_hashjoin(pathnode, root, sjinfo);
  
Index: src/include/nodes/relation.h
===
RCS file: /cvsroot/pgsql/src/include/nodes/relation.h,v
retrieving revision 1.170
diff -c -r1.170 relation.h
*** src/include/nodes/relation.h5 Mar 2009 23:06:45 -   1.170
--- src/include/nodes/relation.h26 Mar 

[HACKERS] tuplestore API problem

[Tom Lane]

By chance I discovered that this query in the regression tests

SELECT ntile(NULL) OVER (ORDER BY ten, four), ten, four FROM tenk1 LIMIT 2;

stops working if work_mem is small enough: it either dumps core or
delivers wrong answers depending on platform.

After some tracing I found out the reason.  ExecWindowAgg() does this:

if (!tuplestore_gettupleslot(winstate->buffer, true,
 winstate->ss.ss_ScanTupleSlot))
elog(ERROR, "unexpected end of tuplestore");

and then goes off and calls the window functions (ntile() here), and
expects the ScanTupleSlot to still be valid afterwards.  However,
ntile() forces us to read to the end of the input to find out the number
of rows.  If work_mem is small enough, that means the tuplestore is
forced into dump-to-disk mode, which means it releases all its in-memory
tuples.  And guess what: the ScanTupleSlot is pointing at one of those,
it doesn't have its own copy of the tuple.  So we wind up trying to read
from a trashed bit of memory.

A brute-force solution is to change tuplestore_gettupleslot() so that it
always copies the tuple, but this would be wasted cycles for most uses
of tuplestores.  I'm thinking of changing tuplestore_gettupleslot's API
to add a bool parameter specifying whether the caller wants to force
a copy.

Comments, better ideas?

BTW: this tells me that no one has tried to apply window functions
to nontrivial problems yet ... we'll need to encourage beta testers
to stress that code.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Crash in gist insertion on pathological box data

[Sergey Konoplev]

On Thu, Mar 26, 2009 at 5:39 PM, Andrew Gierth
 wrote:
> A user on IRC reported a crash (backend segfault) in GiST insertion
> (in 8.3.5 but I can reproduce this in today's HEAD) that turns out
> to be due to misbehaviour of gist_box_picksplit.
>

Andrew, thank you for the test case and report.

p.s. The user Andrew mentioned above is me and if you have a question
to me I am ready to answer it.

-- 
Regards,
Sergey Konoplev
--
PostgreSQL articles in english & russian
http://gray-hemp.blogspot.com/search/label/postgresql/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] global index - work in progress patch

[Tom Lane]

"Srinath K"  writes:
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which
> is the property of Persistent Systems Ltd.

If you want to submit patches, you're really going to have to get your
corporate lawyers to let you submit them without this disclaimer.
I'm not sure it's even legal for me to read the patch, let alone put
it into the Postgres sources.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] global index - work in progress patch

[Greg Stark]

On Thu, Mar 26, 2009 at 2:16 PM, Srinath K  wrote:
> I'm implementing "global index" - an index that indexes all tables in an
> inheritance hierarchy. For complete feature description, please refer
> README.user.
>
...
> This e-mail may contain privileged and confidential information which is the 
> property of Persistent Systems Ltd. It is intended only for the use of the 
> individual or entity to which it is addressed. If you are not the intended 
> recipient, you are not authorized to read, retain, copy, print, distribute or 
> use this message.

Am I authorized to refer to this file?

-- 
greg

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Crash in gist insertion on pathological box data

[Andrew Gierth]

A user on IRC reported a crash (backend segfault) in GiST insertion
(in 8.3.5 but I can reproduce this in today's HEAD) that turns out
to be due to misbehaviour of gist_box_picksplit.

The nature of the problem is this: if gist_box_picksplit doesn't find
a good disposition on the first try, then it tries to split the data
again based on the positions of the box centers. But there's a problem
here with floating-point rounding; it's possible for the average of N
floating-point values to be strictly greater (or less) than all of the
values individually, and the function then returns with, for example,
all the entries assigned to the left node, and nothing in the right
node. This causes gistSplit to try and split the left node again, with
predictable results.

Here is a test case:

file of floating-point values here (999 lines):
http://www.rhodiumtoad.org.uk/junk/badfloats.txt

create table floats3(x float8, y float8);
\copy floats3 from 'badfloats.txt'
create table boxes1 (b box);
create index boxes1_idx on boxes1 using gist (b);
insert into boxes1 select box(point(x,x),point(y,y)) as b from floats3;
[crash]

I'm not sure what the best fix is. I would think that it would make
sense for gistUserPickSplit to error out if the user's split function
returned an empty left or right node, since that would seem to
guarantee this problem. Certainly gist_box_picksplit also needs some
sort of fix to try and split sensibly in the presence of data of this
type.

-- 
Andrew (irc:RhodiumToad)

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_dump Add dumping of comments on index columns

[higepon]

Hi.

> Since we are about to go to beta, it may be that no one is up for
> reviewing it right now.  But I've added it to the CommitFest page for
> the next CommitFest.

Thank you.
I wait until the next CommitFest.

-
Taro Minowa(Higepon)

http://www.monaos.org/
http://code.google.com/p/mosh-scheme/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_dump Add dumping of comments on index columns

[Robert Haas]

> Would someone please review this?

Since we are about to go to beta, it may be that no one is up for
reviewing it right now.  But I've added it to the CommitFest page for
the next CommitFest.

http://wiki.postgresql.org/wiki/CommitFest_2009-First

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New trigger option of pg_standby

[Guillaume Smet]

Hi Simon.

On Thu, Mar 26, 2009 at 11:50 AM, Simon Riggs  wrote:
> Earlier, we discussed having a single trigger file that contains an
> option rather than two distinct trigger files. That design is better
> because it allows the user to choose at failover time, rather than
> making a binding decision at config time. That solution would be the
> ideal one, IMHO, because it gives user more choice - and would allow us
> to keep the -t option meaningfully. In that case the default should be
> patience.

Or you can define both files in your command line to have the choice.

I like the idea of removing -t and adding 2 new options so that people
are warned about the intended behavior.

Anyway, I don't have a strong opinion about how we should fix it as I
don't use pg_standby personnally, just that we should. The two options
you mention have their own merits.

-- 
Guillaume

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] 8.4 release notes proof reading 1/2

[Guillaume Smet]

Bruce,

Here is the second set of comments:

- pg_hba.conf: it seems to me the format has changed which may break
existing pg_hba.conf (it broke the default one of the RPM packaging).
We should make it very visible as the format hasn't changed for a
while. I suppose we'll put it at the top but I just don't want us to
forget it so I mention it here.

- "Allow recovery.conf boolean variables to take the same range of
string alues as postgresql.conf (Bruce)" -> s/alues/values/
- "Add support for Unicode sting and identifier specifications using
code points, e.g. U&'d\0061t\+61' (Peter)"  -> s/sting/string/
- "Allow CREATE OR REPLACE VIEW to add columns to the _end_ of the
view" -> formatting problem with the underscores?
- "Previously EXPLAIN VERBOSE output an internal representation of the
query plan" -> s/output/outputs/ ?
- "Autovacuum options uses to be stored in a system table. " -> s/uses/used/ ?
- "Allows dumps to fail if unable to acquire shared table within the
specified amount of time. " -> shared lock instead of shared table?
- "Add configure option --disable-float4-byval to use the old
behavior. tgl Pass float8 and int8 by value in the server where
possible (Zoltan Boszormenyi) " -> tgl + there's a carriage return
missing (and a bullet)
- "Add -M (query mode) to /contrib/pgbench (ITAGAKI Takahiro)"
->Itagaki san's name inconsistent with other mentions of his name

I can provide a patch for all this if needed. I thought you'd prefer
validate the changes.

Really nice work and really interesting reading.

Thanks.

-- 
Guillaume

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] small but useful patches for text search

[Robert Treat]

On Wednesday 25 March 2009 23:17:41 Robert Haas wrote:
> With respect to this item:
> Disable appending of the epoch date/time when '%' escapes are missing
> in log_filename (Robert Haas)
> I might suggest explaining it this way:
> This change makes it easier to use PostgreSQL in conjunction with an
> external log rotation tool.
>

Hey! We were just complaining about this behavior the other day at $dayjob. We 
were considering hacking our build to make it stop doing this ourselves, but 
decided to use syslog in the end.  Nice to see this "feature" disappear. :-)

-- 
Robert Treat
Conjecture: http://www.xzilla.net
Consulting: http://www.omniti.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] display previous query string of idle-in-transaction

[Tatsuhito Kasahara]

ITAGAKI Takahiro wrote:

The feature could be achieved by an extension module using new executor
hooks in 8.4. It is just like contrib/pg_stat_statements; 

Well, it is a good idea.

Displaying last-query-string may be useful, but it is not a feature for
general purpose. So, it may be an external module.

Now, I have two choice.

1. Displaying last-query-string by pg_stat_activity (suggested by Gurjeet):
   Extends the pg_stat_activity to display "previous_query".
   We can check only a previous query.

2. Displaying query-string-history by external module (suggested by Itagaki):
   Using executor hook to get a query string and stores it in
   shared memory.
   We can check specified number of query string history.

I think I will try mainly 2.
# Or this feature may merge to pg_stat_statements.
Any comments welcome.

Best regards,
--
Tatsuhito Kasahara
kasahara.tatsuh...@oss.ntt.co.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New trigger option of pg_standby

[Simon Riggs]

On Thu, 2009-03-26 at 08:32 +0100, Guillaume Smet wrote:
> On Thu, Mar 26, 2009 at 2:51 AM, Fujii Masao  wrote:
> > What does "the default" mean? You mean that new trigger should use
> > the existing trigger option character (-t)?
> 
> Yes, that's my point.
> 
> I understand it seems weird to switch the options but I'm pretty sure
> a lot of persons currently using -t would be surprised by the current
> behaviour. Moreover playing all the remaining WALs before starting up
> should be the most natural option when people are looking in the help.

If the standby has fallen behind then waiting for it to catch up might
take hours to failover if it waits for all files. If you haven't been
monitoring it correctly, you have no clue. That is also a surprising
thing, so let's not jump from one surprising thing into the arms of
another.

If we go with this, I would suggest we make *neither* the default by
removing -t, and adopting two new options: something like -f == fast
failover, -p == patient failover. This then forces people to read and
understand the difference between the two behaviours so they can make an
informed choice of how they would like to act at this critical point in
time. It is justifiable because there is no single thing called a
trigger file any longer and the concept will lead to pain.

Earlier, we discussed having a single trigger file that contains an
option rather than two distinct trigger files. That design is better
because it allows the user to choose at failover time, rather than
making a binding decision at config time. That solution would be the
ideal one, IMHO, because it gives user more choice - and would allow us
to keep the -t option meaningfully. In that case the default should be
patience.

-- 
 Simon Riggs   www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New trigger option of pg_standby

[Matteo Beccati]

Hi,

Guillaume Smet wrote:

On Thu, Mar 26, 2009 at 2:51 AM, Fujii Masao  wrote:

What does "the default" mean? You mean that new trigger should use
the existing trigger option character (-t)?


Yes, that's my point.

I understand it seems weird to switch the options but I'm pretty sure
a lot of persons currently using -t would be surprised by the current
behaviour. Moreover playing all the remaining WALs before starting up
should be the most natural option when people are looking in the help.

That said, it would be nice to hear from people really using
pg_standby to know if they understand how it works now and if it's
what they intended when they set it up.


My fault not RTFM well enough, but I was surprised finding out that -t 
is working like that.


+1 for me to switch -t to the new behaviour.


Cheers

--
Matteo Beccati

OpenX - http://www.openx.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] display previous query string of idle-in-transaction

[ITAGAKI Takahiro]

Tatsuhito Kasahara  wrote:

> So, main purpose of displaying the last query string is ..
> - check whether "idle in transaction (running long time) process
>after SOME SQL" is exists or not.
> - check the content of "SOME SQL".

The feature could be achieved by an extension module using new executor
hooks in 8.4. It is just like contrib/pg_stat_statements; Backends store
their queries history in process-local or shared memory. If you store
queries in local memory, you could dump them at some of callback routines
called at the end of transaction. On the other hand, if you use shared
memory, it might be possitble to define a SRF function which return history
of queries:

CREATE FUNCTION query_history(backend_pid integer) RETURNS SETOF text;

Required memory for query history is not so much.
"1kB of query text * 100 connection * 10 queries" consumes just 1MB.

You can discard old queries at the end of transaction or out of memory.
If there are some interesting queries in it, you can dump them into
server logs. For example, logging configurations something like
'idle_in_transaction_min_duration' and 'total_transaction_min_duration'
will be possible.

I'm not sure this feature should be in the core or not.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] 8.4 release notes proof reading 1/2

[Guillaume Smet]

Hi,

Just to warn people that I'm making a comprehensive proof reading of
the release notes.

Here are the first comments:

- "This was available previously via a configure
--enable-integer-datetimes (Neil Conway)" -> I don't think we need
Neil's name in the details
- "New semi- and ansi-joins (Tom)" -> s/ansi/anti/
- "While semi-joins merely replace existing IN joins, anti-joins are a
new capability for NOT IN and NOT EXIST clauses (Tom) This improves
optimization possibilities." -> remove the "(Tom)", s/EXIST/EXISTS/
- "IN and EXISTs now provide similar performance for equivalent
clauses. " -> s/EXISTs/EXISTS/
- "A new constraint_exclusion setting, "partition", which matches this
behavior." -> this sentence is weird
- "Move that server statistics file into the subdirectory pg_stat_tmp"
-> s/that/the/ ? I think this item can be grouped with the following
one "Allow the location of the new pg_stat_tmp to be configured via
the stats_temp_directory setting"

I'll post the other part in a few hours.

-- 
Guillaume

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] SSL over Unix-domain sockets

[Peter Eisentraut]

Bruce Momjian wrote:

I thought the logical solution to this was to place the socket in a
secure directory and not bother with SSL at all.


How would a client algorithmically determine whether the server socket 
was in a "secure" directory?


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] display previous query string of idle-in-transaction

[Tatsuhito Kasahara]

Guillaume Smet wrote:

On Wed, Mar 25, 2009 at 5:48 PM, hubert depesz lubaczewski
 wrote:

I would love to get it, but when I suggested it some time in the past
Tom shot it down as bad idea.

http://archives.postgresql.org/message-id/20071016132131.ga4...@depesz.com


I agree with Tom here. I tracked a lot of idle in transaction problems
and you need the whole sequence of queries of all the backends to
really understand what happens.
The last query executed is mostly useless to solve this sort of
problem. At least, it was for me in every case I had to deal with.

I think so too. But last-query-string may be a useful *hint*.
It is a hard work to trace the whole sequence of queries.

In most cases, last query string is enough information to solve the
long transaction problem. And I want a easy way to get the informarion.
# last query string is also a key to trace queries.

"COMMIT;BEGIN;idle in transaction" is not a problem on 8.3 or later, but
"(BEGIN;)SOME SQL;idle in transaction" is a problem.

So, main purpose of displaying the last query string is ..
- check whether "idle in transaction (running long time) process
  after SOME SQL" is exists or not.
- check the content of "SOME SQL".

best regards,

--
Tatsuhito Kasahara
kasahara.tatsuh...@oss.ntt.co.jp

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_dump Add dumping of comments on index columns

[higepon]

Hi.

Here is a patch for pg_dump "Commenting on a composite-type column".
This patch is for Todo item named "Add dumping of comments on index
columns and composite type columns".
As Tom Lane said, this patch is not for dumping "comments on index columns",
but only for "comment on composite-type column".

With this patch, pg_dump can dump comments on composite-type column.

 --
 -- Name: COLUMN bar.b1; Type: COMMENT; Schema: public; Owner: taro
 --

 COMMENT ON COLUMN bar.b1 IS 'column of a composite type b1';


 --
 -- Name: COLUMN bar.b3; Type: COMMENT; Schema: public; Owner: taro
 --

 COMMENT ON COLUMN bar.b3 IS 'column of a composite type b3';

Would someone please review this?

Cheers.

-
Taro Minowa(Higepon)

Cybozu Labs, Inc.

http://www.monaos.org/
http://code.google.com/p/mosh-scheme/


On Tue, Mar 24, 2009 at 11:56 PM, Tom Lane  wrote:
> Bruce Momjian  writes:
>> Wow, I have no idea what that means.  I am wondering if we should just
>> remove this TODO item.  We don't even support comments on indexed
>> columns, so why would pg_dump need to dump it?
>
> The system will let you do it, both cases:
>
> regression=# create type foo as (f1 int, f2 text);
> CREATE TYPE
> regression=# comment on column foo.f2 is 'column of a composite type';
> COMMENT
> regression=# create table tt (f1 int primary key);
> NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "tt_pkey" for 
> table "tt"
> CREATE TABLE
> regression=# comment on column tt_pkey.f1 is 'column of an index';
> COMMENT
>
> and pg_dump fails to dump both cases.
>
> Commenting on a composite-type column seems reasonable.  I'm less happy
> about the other because it depends on the names assigned to index
> columns, which are implementation artifacts.  I'd rather see us forbid
> the case.
>
>                        regards, tom lane
>


pg_dump_composite_type_v1.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] New trigger option of pg_standby

[Guillaume Smet]

On Thu, Mar 26, 2009 at 2:51 AM, Fujii Masao  wrote:
> What does "the default" mean? You mean that new trigger should use
> the existing trigger option character (-t)?

Yes, that's my point.

I understand it seems weird to switch the options but I'm pretty sure
a lot of persons currently using -t would be surprised by the current
behaviour. Moreover playing all the remaining WALs before starting up
should be the most natural option when people are looking in the help.

That said, it would be nice to hear from people really using
pg_standby to know if they understand how it works now and if it's
what they intended when they set it up.

-- 
Guillaume

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers