[HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Kyotaro HORIGUCHI
Hello. This message is a proposal of a pair of patches that enables the memory allocator for PGresult in libpq to be replaced. The comment at the the begging of pqexpbuffer.c says that libpq should not rely on palloc(). Besides, Tom Lane said that palloc should not be visible outside the backend(

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Heikki Linnakangas
On 11.11.2011 11:18, Kyotaro HORIGUCHI wrote: The comment at the the begging of pqexpbuffer.c says that libpq should not rely on palloc(). Besides, Tom Lane said that palloc should not be visible outside the backend(*1) and I agree with it. *1: http://archives.postgresql.org/pgsql-hackers/1999-0

Re: [HACKERS] Adding Node support in outfuncs.c and readfuncs.c

2011-11-11 Thread Dimitri Fontaine
Dimitri Fontaine writes: > What about adding that into src/tools/editors/pgsrc.el? Of course after having used it some more, I fixed a bug and added a function to start from the readfunc node name rather than the enum node number in the interactive calls. Please find attached a newer version of

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread Dimitri Fontaine
"David E. Wheeler" writes: > EXTENSION= pgtap pgtap-core pgtap-schema See contrib/spi/Makefile for an exemple of that. > > make > Makefile:79: *** multiple target patterns. Stop. > > So is that not supported? If not, why not? Can we see your Makefile? Regards, -- Dimitri Font

[HACKERS] SQLDA fix for ECPG

2011-11-11 Thread Boszormenyi Zoltan
Hi, I had a report about ECPG code crashing which involved a query using a date field. Attached is a one liner fix to make the date type's offset computed consistently across sqlda_common_total_size(), sqlda_compat_total_size() and sqlda_native_total_size(). This must have been a cut and paste bu

Re: [HACKERS] type privileges and default privileges

2011-11-11 Thread Robert Haas
On Thu, Nov 10, 2011 at 11:17 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Thu, Nov 10, 2011 at 10:52 PM, Stephen Frost wrote: >> > Certainly a big one that people get caught by is our default of execute >> > to public on functions..  Most of our privileges are se

[HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Robert Haas
I noticed while playing around this morning that, in read committed mode, the following query - or any other straightforward query - calls GetSnapshotData() twice: SELECT 1; What happens is this: exec_simple_query() calls analyze_requires_snapshot() on the query. Since the query is a SELECT, thi

Re: [HACKERS] type privileges and default privileges

2011-11-11 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > Oh, I didn't realize that you were proposing a database-wide setting; > my point was just that the way the feature looks to the user doesn't > have to dictate the catalog representation. I agree that the features don't have to define the catalog repre

Rename trace_userlocks? WAS Re: [HACKERS] LOCK_DEBUG is busted

2011-11-11 Thread Bruce Momjian
Robert Haas wrote: > On Thu, Nov 10, 2011 at 5:07 PM, Bruce Momjian wrote: > > Tom Lane wrote: > >> Robert Haas writes: > >> > It's possible to compile the source tree with LOCK_DEBUG defined, but > >> > the resulting postgres promptly dumps core, due to the fact that > >> > user_lockmethod doesn

Re: Rename trace_userlocks? WAS Re: [HACKERS] LOCK_DEBUG is busted

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 9:40 AM, Bruce Momjian wrote: > Robert Haas wrote: >> On Thu, Nov 10, 2011 at 5:07 PM, Bruce Momjian wrote: >> > Tom Lane wrote: >> >> Robert Haas writes: >> >> > It's possible to compile the source tree with LOCK_DEBUG defined, but >> >> > the resulting postgres promptly

[HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Tom Lane
In another thread, Alvaro quoted from the manual: > If for some reason autovacuum fails to clear old XIDs from a table, the > system will begin to emit warning messages like this when the > database's oldest > XIDs reach ten million transactions from the wraparound point: >

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Tom Lane
Robert Haas writes: > Can't we arrange to retain the snapshot used for parse > analysis / planning and reuse it for the portal that we create just > afterwards? Possibly, but what if planning takes a long time? Also, I think you're ignoring the extended-query-protocol scenario, where it would be

[HACKERS] Hotel reservation for FOSDEM 2012 - Deadline: December 31th, 2011

2011-11-11 Thread Andreas 'ads' Scherbaum
Hi all, like the last years we will have a devroom at FOSDEM 2012. We also look forward to have a booth. We made a group reservation in the Agenda Louise hotel: Hotel Agenda Louise rue de Florence 6 B-1000 Brussels Tel: + 32.2.539.00.31 Fax: + 32.2.539.00.63 www.hotel-agenda.com This time,

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 10:01 AM, Tom Lane wrote: > Robert Haas writes: >> Can't we arrange to retain the snapshot used for parse >> analysis / planning and reuse it for the portal that we create just >> afterwards? > > Possibly, but what if planning takes a long time?  Also, I think you're > ign

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane wrote: > In another thread, Alvaro quoted from the manual: >>       If for some reason autovacuum fails to clear old XIDs from a table, the >>       system will begin to emit warning messages like this when the >> database's oldest >>       XIDs reach ten

Re: [HACKERS] Disable OpenSSL compression

2011-11-11 Thread Albe Laurenz
Tom Lane wrote: > But in any case, my objection is that there's no adequate use-case > for this GUC, because it's much more sensible to set it from the client > side. We have too many GUCs already --- Josh B regularly goes on the > warpath looking for ones we can remove. This one should never get

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Alvaro Herrera
Excerpts from Robert Haas's message of vie nov 11 12:29:42 -0300 2011: > On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane wrote: > > In another thread, Alvaro quoted from the manual: > >>       If for some reason autovacuum fails to clear old XIDs from a table, > >> the > >>       system will begin to

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Tom Lane
Robert Haas writes: > On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane wrote: >> In another thread, Alvaro quoted from the manual: > If for some reason autovacuum fails to clear old XIDs from a table, the > system will begin to emit warning messages like this when the > database's oldest >

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Florian Pflug
On Nov11, 2011, at 16:18 , Robert Haas wrote: > In the extend query protocol scenario, it seems to me that keeping the > snapshot would be both wrong and a bad idea. It would be wrong > because the user will (I think) expect the query can see all rows that > were marked as committed prior to Execu

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Tom Lane
Florian Pflug writes: > On Nov11, 2011, at 16:18 , Robert Haas wrote: >> In the extend query protocol scenario, it seems to me that keeping the >> snapshot would be both wrong and a bad idea. > Hm, but that'd penalize clients who use the extended query protocol, which > they have to if they want

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Florian Pflug
On Nov11, 2011, at 17:06 , Tom Lane wrote: > Florian Pflug writes: >> On Nov11, 2011, at 16:18 , Robert Haas wrote: >>> In the extend query protocol scenario, it seems to me that keeping the >>> snapshot would be both wrong and a bad idea. > >> Hm, but that'd penalize clients who use the extended

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Robert Haas
On Nov 11, 2011, at 10:47 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane wrote: >>> In another thread, Alvaro quoted from the manual: >> If for some reason autovacuum fails to clear old XIDs from a table, the >> system will begin to emit warning

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Heikki Linnakangas
On 11.11.2011 17:47, Tom Lane wrote: Robert Haas writes: On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane wrote: It occurs to me to wonder how this scenario will interact with the recent changes to let VACUUM skip pages. AFAIR there is not a way for a manual VACUUM to set the anti-wraparound mode,

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread David E. Wheeler
On Nov 11, 2011, at 4:27 AM, Dimitri Fontaine wrote: >>EXTENSION= pgtap pgtap-core pgtap-schema > > See contrib/spi/Makefile for an exemple of that. Okay, good, it *should* work then. >>> make >>Makefile:79: *** multiple target patterns. Stop. >> >> So is that not supported? If no

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Heikki Linnakangas
On 11.11.2011 19:15, Heikki Linnakangas wrote: On 11.11.2011 17:47, Tom Lane wrote: (3) In disaster recovery scenarios, the last thing we want is to be imposing extra conditions on what an already-stressed DBA has to do to fix things; especially extra conditions that are different from the way i

Re: [HACKERS] foreign key locks, 2nd attempt

2011-11-11 Thread David Kerr
On Thu, Nov 10, 2011 at 03:17:59PM -0500, Christopher Browne wrote: - On Sun, Nov 6, 2011 at 2:28 AM, Jeroen Vermeulen wrote: - > On 2011-11-04 01:12, Alvaro Herrera wrote: - > - >> I would like some opinions on the ideas on this patch, and on the patch - >> itself.  If someone wants more discussi

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Tom Lane
Florian Pflug writes: > Still, optimizing only the simple protocol seems weird. Would it be sane to decree that the "statement snapshot" lasts until Sync is received, in extended query mode? But frankly I do not like any of these proposals. Making fundamental changes in long-established semanti

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Tom Lane
Heikki Linnakangas writes: > On second thought, if XID wraparound is close enough that the DBA has to > log in to do manual vacuums to avoid it, relfrozenxid of the > trouble-making tables are surely older than default > vacuum_freeze_table_age, so plain VACUUM is enough to scan the whole table

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread David E. Wheeler
On Nov 11, 2011, at 9:20 AM, David E. Wheeler wrote: >> Can we see your Makefile? > > Right here: > > https://github.com/theory/pgtap/blob/master/Makefile Oy, this is stupidly my fault. I blame the late hours and the second stout. Thanks, David -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread Dimitri Fontaine
"David E. Wheeler" writes: >>>EXTENSION= pgtap pgtap-core pgtap-schema > https://github.com/theory/pgtap/blob/master/Makefile I don't see the line above in the file you linked. The content of the Makefile I just read is not using multiple extensions at all, it's re implementing all of

Re: [HACKERS] Multiple Extensions

2011-11-11 Thread David E. Wheeler
On Nov 11, 2011, at 11:04 AM, Dimitri Fontaine wrote: EXTENSION= pgtap pgtap-core pgtap-schema >> https://github.com/theory/pgtap/blob/master/Makefile > > I don't see the line above in the file you linked. The content of the > Makefile I just read is not using multiple extensions at

[HACKERS] Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Daniel Farina
On Thu, Nov 10, 2011 at 11:57 PM, Nikhil Sontakke wrote: > Hi, > >> >> But if it's deemed to be a >> problem, I want to see a solution that's actually watertight.) >> > > After Daniel's hunch about pg_dump barfing due to such leftover entries > proving out to be true, we have one credible explanat

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Dimitri Fontaine
Robert Haas writes: > Considering that GetSnapshotData() is the number-one consumer of CPU > time on many profiling runs I've done, this seems needlessly > inefficient. Can't we arrange to retain the snapshot used for parse > analysis / planning and reuse it for the portal that we create just > a

Re: [HACKERS] Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Alvaro Herrera
Excerpts from Daniel Farina's message of vie nov 11 16:08:01 -0300 2011: > On Thu, Nov 10, 2011 at 11:57 PM, Nikhil Sontakke wrote: > > Hi, > > > >> > >> But if it's deemed to be a > >> problem, I want to see a solution that's actually watertight.) > >> > > > > After Daniel's hunch about pg_dump

Re: [HACKERS] Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Tom Lane
Alvaro Herrera writes: > Excerpts from Daniel Farina's message of vie nov 11 16:08:01 -0300 2011: >> The 2004 report was in version 7.3, released in 2002. So it's Very >> Nearly a ten year old bug, and may very well go back further back in >> time. > Wasn't 7.3 the release that introduced schema

Re: [HACKERS] reduce null bitmap size

2011-11-11 Thread YAMAMOTO Takashi
hi, > y...@mwd.biglobe.ne.jp (YAMAMOTO Takashi) writes: >> how about making heap_form_tuple and its variants use smaller natts for >> tuples whose trailing columns are NULL? > > This idea has been proposed before, and rejected on the basis that it's > unlikely to save enough to be worth the cycle

Re: [HACKERS] Disable OpenSSL compression

2011-11-11 Thread Bruce Momjian
Albe Laurenz wrote: > Tom Lane wrote: > > But in any case, my objection is that there's no adequate use-case > > for this GUC, because it's much more sensible to set it from the > client > > side. We have too many GUCs already --- Josh B regularly goes on the > > warpath looking for ones we can re

Re: [HACKERS] Manual anti-wraparound vacuums

2011-11-11 Thread Bruce Momjian
Heikki Linnakangas wrote: > On 11.11.2011 17:47, Tom Lane wrote: > > Robert Haas writes: > >> On Fri, Nov 11, 2011 at 9:59 AM, Tom Lane wrote: > >>> It occurs to me to wonder how this scenario will interact with the > >>> recent changes to let VACUUM skip pages. AFAIR there is not a way for a >

[HACKERS] Working with git repo tagged versions

2011-11-11 Thread Robert Treat
Howdy folks, Occasionally I need to grab an older release from git based on a tag rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE branch. I used to know how to do this in CVS, but I find I tend to revert to grabbing tarballs now that we're on git. So, I'm wondering if anyone knows

Re: [HACKERS] Working with git repo tagged versions

2011-11-11 Thread Bruce Momjian
Robert Treat wrote: > Howdy folks, > > Occasionally I need to grab an older release from git based on a tag > rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE > branch. I used to know how to do this in CVS, but I find I tend to > revert to grabbing tarballs now that we're on git. So

Re: [HACKERS] Working with git repo tagged versions

2011-11-11 Thread Alvaro Herrera
Excerpts from Robert Treat's message of vie nov 11 17:18:43 -0300 2011: > Howdy folks, > > Occasionally I need to grab an older release from git based on a tag > rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE > branch. I used to know how to do this in CVS, but I find I tend to >

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 2:21 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> Considering that GetSnapshotData() is the number-one consumer of CPU >> time on many profiling runs I've done, this seems needlessly >> inefficient.  Can't we arrange to retain the snapshot used for parse >> analysi

Re: [HACKERS] Working with git repo tagged versions

2011-11-11 Thread Andrew Dunstan
On 11/11/2011 03:53 PM, Bruce Momjian wrote: Robert Treat wrote: Howdy folks, Occasionally I need to grab an older release from git based on a tag rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE branch. I used to know how to do this in CVS, but I find I tend to revert to grabb

Re: [HACKERS] Working with git repo tagged versions

2011-11-11 Thread Tom Lane
Robert Treat writes: > Occasionally I need to grab an older release from git based on a tag > rather than a branch, eg the REL8_3_10 tag vs the REL8_3_STABLE > branch. I used to know how to do this in CVS, but I find I tend to > revert to grabbing tarballs now that we're on git. So, I'm wondering

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Simon Riggs
On Fri, Nov 11, 2011 at 7:21 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> Considering that GetSnapshotData() is the number-one consumer of CPU >> time on many profiling runs I've done, this seems needlessly >> inefficient.  Can't we arrange to retain the snapshot used for parse >> analysi

Re: [HACKERS] foreign key locks, 2nd attempt

2011-11-11 Thread Josh Berkus
>> An UPDATE that modifies the key columns will be blocked, just as now. >> Same with a DELETE. > > OK, so it prevents non-key data modifications from spilling to the > referred rows --- nice. Yes. Eliminates the leading cause of deadlocks in Postgres applications. -- Josh Berkus PostgreSQL E

Re: [HACKERS] why do we need two snapshots per query?

2011-11-11 Thread Tom Lane
Simon Riggs writes: > Tom, in that earlier thread you said you'd be doing something in this > release about that. Can you say more about what that was, and will you > be doing it still? http://git.postgresql.org/gitweb/?p=postgresql.git&a=commitdiff&h=e6faf910d75027bdce7cd0f2033db4e912592bcc I t

Re: [HACKERS] [GENERAL] VACUUM touching file but not updating relation

2011-11-11 Thread Tom Lane
Thom Brown writes: > On 11 November 2011 00:55, Tom Lane wrote: >> Thom Brown writes: >>> I just noticed that the VACUUM process touches a lot of relations >>> (affects mtime) but for one file I looked at, it didn't change. This >>> doesn't always happen, and many relations aren't touched at al

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Tom Lane
Heikki Linnakangas writes: > On 11.11.2011 11:18, Kyotaro HORIGUCHI wrote: >> The comment at the the begging of pqexpbuffer.c says that libpq >> should not rely on palloc(). Besides, Tom Lane said that palloc >> should not be visible outside the backend(*1) and I agree with >> it. >> >> *1: http:

[HACKERS] fix for psql's \dd version check

2011-11-11 Thread Josh Kupershmidt
Someone (me) didn't get the version check for part of psql's \dd command quite right. I was using a 9.2 client on a 9.1 server and got this when I ran \dd: ERROR: function pg_catalog.pg_opfamily_is_visible(oid) does not exist LINE 33: AND pg_catalog.pg_opfamily_is_visible(opf.oid) since pg_opf

Re: [HACKERS] [GENERAL] VACUUM touching file but not updating relation

2011-11-11 Thread Thom Brown
On 11 November 2011 23:28, Tom Lane wrote: > Thom Brown writes: >> On 11 November 2011 00:55, Tom Lane wrote: >>> Thom Brown writes: I just noticed that the VACUUM process touches a lot of relations (affects mtime) but for one file I looked at, it didn't change.  This doesn't alw

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Heikki's idea is probably superior so far as PG backend usage is > concerned in isolation, but I wonder if there are scenarios where a > client application would like to be able to manage libpq's allocations. The answer to that is certainly 'yes'. It was o

Re: [HACKERS] foreign key locks, 2nd attempt

2011-11-11 Thread Jeroen Vermeulen
On 2011-11-12 00:30, David Kerr wrote: Is this being suggested in lieu of Alvaro's patch? because it seems to be adding complexity to the system (multiple types of primary key definitions) instead of just fixing an obvious problem (over-aggressive locking done on FK checks). It wouldn't be a n

Re: [HACKERS] fix for psql's \dd version check

2011-11-11 Thread Robert Haas
On Fri, Nov 11, 2011 at 6:59 PM, Josh Kupershmidt wrote: > Someone (me) didn't get the version check for part of psql's \dd > command quite right. I was using a 9.2 client on a 9.1 server and got > this when I ran \dd: > > ERROR:  function pg_catalog.pg_opfamily_is_visible(oid) does not exist > LI

Re: [HACKERS] Re: pg_dump: schema with OID XXXXX does not exist - was Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Nikhil Sontakke
> > Wasn't 7.3 the release that introduced schemas in the first place? > > I think there's a very good chance that the older reports with similar > symptoms are completely unrelated, anyhow. > > Tom Lane is reluctant and that should tell me something :) So unless the list feels that this should be

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Heikki's idea is probably superior so far as PG backend usage is >> concerned in isolation, but I wonder if there are scenarios where a >> client application would like to be able to manage libpq's allocations. > The answer to that

Re: [HACKERS] Allow substitute allocators for PGresult.

2011-11-11 Thread Robert Haas
On Sat, Nov 12, 2011 at 12:48 AM, Tom Lane wrote: > AIUI Kyotaro-san is just suggesting that the app should be able to > provide a substitute malloc function for use in allocating PGresult > space (and not, I think, anything else that libpq allocates internally). > Basically this would allow PGres

Re: [HACKERS] Concurrent CREATE TABLE/DROP SCHEMA leaves inconsistent leftovers

2011-11-11 Thread Robert Haas
On Thu, Nov 10, 2011 at 7:00 AM, Nikhil Sontakke wrote: > PFA, a patch against git head. We take the AccessShareLock lock on the > schema in DefineRelation now. Note that we do not want to interlock with > other concurrent creations in the schema. We only want to interlock with > deletion activity