[HACKERS] pg_upgrade version check improvements and small fixes

2011-06-22 Thread Dan McGee
Not sure what the normal process is for patches, but I put together a few small patches for pg_upgrade after trying to use it earlier today and staring a non-helpful error message before I finally figured out what was going on. 0001 is just a simple typo fix, but didn't want to mix it in with the

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Alvaro Herrera
Excerpts from Greg Sabino Mullane's message of mié jun 22 03:24:34 UTC 2011: Hash: RIPEMD160 Do we do enough to show which tables are db shared, e.g. pg_database? I don't see any indication from psql \dS. Are our docs clear enough? I don't think \dS should be indicating such a thing.

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread David Christensen
# Avoid bug that converts 'x =- 1' to 'x = -1' $source =~ s!=- !-= !g; I haven't looked at the shell script this replaces, but is that the correct substitution pattern? (BTW, I'm not seeing the token =- anywhere except in the Makefile, which wouldn't be run against, no? Am I

Re: [HACKERS] Coding style point: const in function parameter declarations

2011-06-22 Thread Heikki Linnakangas
On 22.06.2011 01:51, Tom Lane wrote: I notice that the SSI code is rather heavily invested in function declarations like this: extern bool PageIsPredicateLocked(const Relation relation, const BlockNumber blkno); I find this to be poor style, and would like to see if there's any support for

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Heikki Linnakangas
On 22.06.2011 07:58, Dan Ports wrote: I was looking at ExecSeqScan today and noticed that it invokes PredicateLockRelation each time it's called, i.e. for each tuple returned. Any reason we shouldn't skip that call if rs_relpredicatelocked is already set, as in the attached patch? That would

Re: [HACKERS] Coding style point: const in function parameter declarations

2011-06-22 Thread Heikki Linnakangas
On 22.06.2011 02:58, Dan Ports wrote: On Tue, Jun 21, 2011 at 06:51:20PM -0400, Tom Lane wrote: I find this to be poor style, and would like to see if there's any support for getting rid of the const keywords. I'm in favor of removing them too. Ok, I've removed all the useless const

[HACKERS] Re: [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Noah Misch
On Tue, Jun 21, 2011 at 11:11:41PM -0400, Robert Haas wrote: On Tue, Jun 21, 2011 at 11:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Some of the refactoring you've done here seems likely to break things, because you're basically making the relation

Re: [HACKERS] [COMMITTERS] pgsql: Make the visibility map crash-safe.

2011-06-22 Thread Heikki Linnakangas
On 22.06.2011 06:05, Robert Haas wrote: Second, when inserting, updating, or deleting a tuple, we can no longer get away with clearing the visibility map bit after releasing the lock on the corresponding heap page, because an intervening crash might leave the visibility map bit set and the

Re: [HACKERS] Hugetables question

2011-06-22 Thread Marti Raudsepp
On Sun, Jun 19, 2011 at 12:56, Radosław Smogura rsmog...@softperience.eu wrote: I want to implement hugepages for shared memory Hi, Have you read this post by Tom Lane about the performance estimation and a proof-of-concept patch with hugepages?

Re: [HACKERS] Latch implementation that wakes on postmaster death on both win32 and Unix

2011-06-22 Thread Peter Geoghegan
Attached patch addresses Fujii's more recent concerns. On 22 June 2011 04:54, Fujii Masao masao.fu...@gmail.com wrote: +WaitLatch(volatile Latch *latch, int wakeEvents, long timeout) +WaitLatchOrSocket(volatile Latch *latch, int wakeEvents, pgsocket sock, long timeout) If 'wakeEvent' is

Re: [HACKERS] Hugetables question

2011-06-22 Thread Radosław Smogura
On Wed, 22 Jun 2011 14:24:17 +0300, Marti Raudsepp wrote: On Sun, Jun 19, 2011 at 12:56, Radosław Smogura rsmog...@softperience.eu wrote: I want to implement hugepages for shared memory Hi, Have you read this post by Tom Lane about the performance estimation and a proof-of-concept patch with

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Andrew Dunstan
On 06/22/2011 02:03 AM, David Christensen wrote: # Avoid bug that converts 'x =- 1' to 'x = -1' $source =~ s!=- !-= !g; I haven't looked at the shell script this replaces, but is that the correct substitution pattern? (BTW, I'm not seeing the token =- anywhere except in the

Re: [HACKERS] [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 6:18 AM, Noah Misch n...@leadboat.com wrote: On Tue, Jun 21, 2011 at 11:11:41PM -0400, Robert Haas wrote: On Tue, Jun 21, 2011 at 11:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Some of the refactoring you've done here seems

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Andrew Dunstan
On 06/22/2011 08:35 AM, Andrew Dunstan wrote: On 06/22/2011 02:03 AM, David Christensen wrote: # Avoid bug that converts 'x =- 1' to 'x = -1' $source =~ s!=- !-= !g; I haven't looked at the shell script this replaces, but is that the correct substitution pattern? (BTW, I'm not

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Bruce Momjian
Andrew Dunstan wrote: On 06/22/2011 08:35 AM, Andrew Dunstan wrote: On 06/22/2011 02:03 AM, David Christensen wrote: # Avoid bug that converts 'x =- 1' to 'x = -1' $source =~ s!=- !-= !g; I haven't looked at the shell script this replaces, but is that the correct

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Bruce Momjian
Alvaro Herrera wrote: Excerpts from Greg Sabino Mullane's message of mi?? jun 22 03:24:34 UTC 2011: Hash: RIPEMD160 Do we do enough to show which tables are db shared, e.g. pg_database? I don't see any indication from psql \dS. Are our docs clear enough? I don't think \dS

Re: [HACKERS] smallserial / serial2

2011-06-22 Thread Josh Kupershmidt
On Tue, Jun 21, 2011 at 10:58 PM, Robert Haas robertmh...@gmail.com wrote: Committed the main patch, and your regression tests. Hmph, looks like buildfarm members koi and jaguar are failing make check now: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=koidt=2011-06-22%2008%3A06%3A00 due

Re: [HACKERS] smallserial / serial2

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 9:14 AM, Josh Kupershmidt schmi...@gmail.com wrote: On Tue, Jun 21, 2011 at 10:58 PM, Robert Haas robertmh...@gmail.com wrote: Committed the main patch, and your regression tests. Hmph, looks like buildfarm members koi and jaguar are failing make check now:  

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Jeff MacDonald
Greetings, On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote: I assumed it was important to indicate if someone was looking at per-database or per-cluster data, like pg_tablespace. The issue comes up when I do admin training about the system tables. +1 I favor features that make

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Andrew Dunstan
On 06/22/2011 09:08 AM, Bruce Momjian wrote: Andrew Dunstan wrote: On 06/22/2011 08:35 AM, Andrew Dunstan wrote: On 06/22/2011 02:03 AM, David Christensen wrote: # Avoid bug that converts 'x =- 1' to 'x = -1' $source =~ s!=- !-= !g; I haven't looked at the shell script this

Re: [HACKERS] WIP pgindent replacement

2011-06-22 Thread Bruce Momjian
Andrew Dunstan wrote: Further research shows that C89 explicitly dropped support for the old KR =- operator, so we probably *should* remove this in case it introduces an unintended bug. Well, the point is if someone does use that, it isn't going to generate a pgindent error, but rather

Re: [HACKERS] [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Another option might be to leave heap_openrv() and relation_openrv() alone and add a missing_ok argument to try_heap_openrv() and try_relation_openrv(). +1 for that, although the try_ prefix might be inappropriate naming now; how about

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Tom Lane
Dan Ports d...@csail.mit.edu writes: I was looking at ExecSeqScan today and noticed that it invokes PredicateLockRelation each time it's called, i.e. for each tuple returned. Any reason we shouldn't skip that call if rs_relpredicatelocked is already set, as in the attached patch? Why is the

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Magnus Hagander
On Fri, Jun 10, 2011 at 21:07, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: I came across a situation today with a pretty bad crash of pg_dump, due to not checking the return code from malloc(). When looking through the code, it seems there are a *lot* of

Re: [HACKERS] smallserial / serial2

2011-06-22 Thread Tom Lane
Josh Kupershmidt schmi...@gmail.com writes: Hmph, looks like buildfarm members koi and jaguar are failing make check now: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=koidt=2011-06-22%2008%3A06%3A00 due to a difference in sequence.out. I didn't muck with the part about SELECT *

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2011-06-22 Thread Simon Riggs
On Tue, Jun 21, 2011 at 4:12 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Mon, Jun 20, 2011 at 10:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: The ALTER TABLE patch has greatly expanded the scope of the issue, and that *is* a regression compared to prior

Re: [HACKERS] smallserial / serial2

2011-06-22 Thread Tom Lane
I wrote: That previous approach of adding extra expected files isn't going to scale nicely if there are multiple places at risk ... but do we need multiple places selecting the sequence contents? I remain of the opinion that just omitting the value isn't good testing policy. Actually, on

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Something along the line of this? I think this is a seriously, seriously bad idea: +#define strdup(x) pg_strdup(x) +#define malloc(x) pg_malloc(x) +#define calloc(x,y) pg_calloc(x, y) +#define realloc(x,y) pg_realloc(x, y) as it will render the

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Peter Geoghegan
On 22 June 2011 16:25, Magnus Hagander mag...@hagander.net wrote: Something along the line of this? IMHO the redefinition of malloc() looks a bit hairy...can't you just make the callers use the functions directly? -- Peter Geoghegan       http://www.2ndQuadrant.com/ PostgreSQL Development,

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Magnus Hagander
On Wed, Jun 22, 2011 at 17:48, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Something along the line of this? I think this is a seriously, seriously bad idea: +#define strdup(x) pg_strdup(x) +#define malloc(x) pg_malloc(x) +#define calloc(x,y) pg_calloc(x,

[HACKERS] Re: Coding style point: const in function parameter declarations

2011-06-22 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: 2. In cases such as const Relation foo where the parameter type is a typedeffed pointer, it is easy for readers to arrive at the false conclusion that this guarantees the function doesn't change the pointed-to structure. So easy that in fact that was my

Re: [HACKERS] Coding style point: const in function parameter declarations

2011-06-22 Thread Merlin Moncure
On Tue, Jun 21, 2011 at 5:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: Declarations like const structtype *param are fine, because those create a real, enforced contract on what the function can do to data that is visible to its caller.  But I don't see any value at all in const-ifying the

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Why is the call in ExecSeqScan at all, and not in the node startup function? Because when I asked about the placement of such calls in January of 2010 I didn't get any advice which suggested that, and this was a place I was able to find which worked

Re: [HACKERS] Coding style point: const in function parameter declarations

2011-06-22 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: On Tue, Jun 21, 2011 at 5:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: Declarations like const structtype *param are fine, because those create a real, enforced contract on what the function can do to data that is visible to its caller. But I don't see

Re: [HACKERS] [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié jun 22 08:56:02 -0400 2011: Another option might be to leave heap_openrv() and relation_openrv() alone and add a missing_ok argument to try_heap_openrv() and try_relation_openrv(). Passing true would give the same behavior as presently; passing

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Alvaro Herrera
Excerpts from Jeff MacDonald's message of mié jun 22 09:27:36 -0400 2011: Greetings, On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote: I assumed it was important to indicate if someone was looking at per-database or per-cluster data, like pg_tablespace. The issue comes up

Re: [HACKERS] [v9.2] DROP Reworks Part.0 - 'missing_ok' support of get_object_address

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 12:51 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Robert Haas's message of mié jun 22 08:56:02 -0400 2011: Another option might be to leave heap_openrv() and relation_openrv() alone and add a missing_ok argument to try_heap_openrv() and

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Heikki Linnakangas
On 22.06.2011 17:28, Tom Lane wrote: Dan Portsd...@csail.mit.edu writes: I was looking at ExecSeqScan today and noticed that it invokes PredicateLockRelation each time it's called, i.e. for each tuple returned. Any reason we shouldn't skip that call if rs_relpredicatelocked is already set, as

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Bruce Momjian
Alvaro Herrera wrote: Excerpts from Jeff MacDonald's message of mi?? jun 22 09:27:36 -0400 2011: Greetings, On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote: I assumed it was important to indicate if someone was looking at per-database or per-cluster data, like

Re: [HACKERS] Coding style point: const in function parameter declarations

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 12:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: On Tue, Jun 21, 2011 at 5:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: Declarations like const structtype *param are fine, because those create a real, enforced contract on what the

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Kevin Grittner
Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 22.06.2011 17:28, Tom Lane wrote: Why is the call in ExecSeqScan at all, and not in the node startup function? It makes sense to delay it until the scan is actually started, so that you don't get unnecessary serialization

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mié jun 22 14:31:51 -0400 2011: Alvaro Herrera wrote: Excerpts from Jeff MacDonald's message of mi jun 22 09:27:36 -0400 2011: Greetings, On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote: I assumed it was important to indicate if

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Bruce Momjian
Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mi?? jun 22 14:31:51 -0400 2011: Alvaro Herrera wrote: Excerpts from Jeff MacDonald's message of mi jun 22 09:27:36 -0400 2011: Greetings, On Wednesday, June 22, 2011 09:10:02 AM Bruce Momjian wrote: I assumed

Re: [HACKERS] pg_dump vs malloc

2011-06-22 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mié jun 22 11:25:43 -0400 2011: On Fri, Jun 10, 2011 at 21:07, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: I came across a situation today with a pretty bad crash of pg_dump, due to not checking the return code

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mié jun 22 15:24:40 -0400 2011: Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mi jun 22 14:31:51 -0400 2011: I am going to try to add descriptions for system tables for \d+ so maybe I can put it there. That would be OK with

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Andrew Dunstan
On 06/22/2011 03:24 PM, Bruce Momjian wrote: I am going to try to add descriptions for system tables for \d+ so maybe I can put it there. That would be OK with me, I guess -- something like shared system catalog for databases, you mean? I was thinking (shared) as part of the description.

Re: [HACKERS] Indication of db-shared tables

2011-06-22 Thread Bruce Momjian
Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mi?? jun 22 15:24:40 -0400 2011: Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of mi jun 22 14:31:51 -0400 2011: I am going to try to add descriptions for system tables for \d+ so maybe I can put it there.

[HACKERS] SYNONYMS (again)

2011-06-22 Thread Joshua D. Drake
Per: http://archives.postgresql.org/pgsql-hackers/2010-11/msg02043.php It seems we did come up with a use case in the procpid discussion. The ability to change the names of columns/databases etc, to handle the fixing of bad decision decisions during development over time. Thoughts? JD --

[HACKERS] fixing PQsetvalue()

2011-06-22 Thread Merlin Moncure
On Jun 6 (http://archives.postgresql.org/pgsql-hackers/2011-06/msg00272.php), Pavel discovered an issue with PQsetvalue that could cause libpq to wander off into unallocated memory that was present in 9.0.x. A fairly uninteresting fix was quickly produced, but Tom indicated during subsequent

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Dan Ports
On Wed, Jun 22, 2011 at 12:07:04PM +0300, Heikki Linnakangas wrote: Hmm, I wonder if we should move this logic to heapam.c. The optimization to acquire a relation lock straight away should apply to all heap scans, not only those coming from ExecSeqScan. The distinction is academic at the

Re: [HACKERS] lazy vxid locks, v1

2011-06-22 Thread Florian Pflug
On Jun12, 2011, at 23:39 , Robert Haas wrote: So, the majority (60%) of the excess spinning appears to be due to SInvalReadLock. A good chunk are due to ProcArrayLock (25%). Hm, sizeof(LWLock) is 24 on X86-64, making sizeof(LWLockPadded) 32. However, cache lines are 64 bytes large on recent

Re: [HACKERS] Repeated PredicateLockRelation calls during seqscan

2011-06-22 Thread Kevin Grittner
Dan Ports d...@csail.mit.edu wrote: Note that this hadn't been a reasonable option until last week when we added the check for non-MVCC snapshots, since there are lots of things that use heap scans but SeqScan is the only (currently-existing) one we want to lock. That is the sort of thing

Re: [HACKERS] SYNONYMS (again)

2011-06-22 Thread Alvaro Herrera
Excerpts from Joshua D. Drake's message of mié jun 22 15:37:17 -0400 2011: Per: http://archives.postgresql.org/pgsql-hackers/2010-11/msg02043.php It seems we did come up with a use case in the procpid discussion. The ability to change the names of columns/databases etc, to handle the

Re: [HACKERS] Coding style point: const in function parameter declarations

2011-06-22 Thread Greg Stark
On Wed, Jun 22, 2011 at 5:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: If we were thinking of moving in that direction, I would argue that we should get rid of typedef'd pointers altogether, ie, change Relation to be a typedef for the struct and write Relation *rel not Relation rel. Hm. I have

Re: [HACKERS] Coding style point: const in function parameter declarations

2011-06-22 Thread Tom Lane
Greg Stark st...@mit.edu writes: On Wed, Jun 22, 2011 at 5:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: If we were thinking of moving in that direction, I would argue that we should get rid of typedef'd pointers altogether, ie, change Relation to be a typedef for the struct and write Relation

Re: [HACKERS] Coding style point: const in function parameter declarations

2011-06-22 Thread Peter Geoghegan
On 23 June 2011 00:37, Greg Stark st...@mit.edu wrote: Hm. I have to say the single most confusing thing about the Postgres source that took me a *long* time to get over was remembering that some of the typedefs were already pointers and some weren't. It seems silly now but when I was trying

Re: [HACKERS] crash-safe visibility map, take five

2011-06-22 Thread Jeff Davis
On Thu, 2011-06-16 at 23:17 -0400, Noah Misch wrote: 2. In the words of a comment added by the patch: * The critical integrity requirement here is that we must never end up with * a situation where the visibility map bit is set, and the page-level * PD_ALL_VISIBLE bit is clear. If that

Re: [HACKERS] crash-safe visibility map, take five

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 8:53 PM, Jeff Davis pg...@j-davis.com wrote: On Thu, 2011-06-16 at 23:17 -0400, Noah Misch wrote: 2. In the words of a comment added by the patch:  * The critical integrity requirement here is that we must never end up with  * a situation where the visibility map bit is

Re: [HACKERS] pg_upgrade version check improvements and small fixes

2011-06-22 Thread Bruce Momjian
Dan McGee wrote: Not sure what the normal process is for patches, but I put together a few small patches for pg_upgrade after trying to use it earlier today and staring a non-helpful error message before I finally figured out what was going on. Thanks for the detailed report and patches. Let

Re: [HACKERS] [COMMITTERS] pgsql: Make the visibility map crash-safe.

2011-06-22 Thread Robert Haas
On Wed, Jun 22, 2011 at 9:12 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 22, 2011 at 6:55 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 22.06.2011 06:05, Robert Haas wrote: Second, when inserting, updating, or deleting a tuple, we can no longer get away

Re: [HACKERS] crash-safe visibility map, take five

2011-06-22 Thread Jeff Davis
On Wed, 2011-06-22 at 21:53 -0400, Robert Haas wrote: On Wed, Jun 22, 2011 at 8:53 PM, Jeff Davis pg...@j-davis.com wrote: On Thu, 2011-06-16 at 23:17 -0400, Noah Misch wrote: 2. In the words of a comment added by the patch: * The critical integrity requirement here is that we must never