Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: It's just ridiculous to assert that it doesn't matter if all the anti-wraparound vacuums start simultaneously. It does matter. For one thing, once every single autovacuum worker is pinned down doing an anti-wraparound vacuum of some table, then a

Re: [HACKERS] WIP patch for Todo Item : Provide fallback_application_name in contrib/pgbench, oid2name, and dblink

2012-06-28 Thread Amit Kapila
From: Robert Haas [mailto:robertmh...@gmail.com] Sent: Thursday, June 28, 2012 7:46 AM On Wed, Jun 27, 2012 at 9:57 PM, Shigeru HANADA shigeru.han...@gmail.com wrote: On Thu, Jun 14, 2012 at 10:55 PM, Robert Haas robertmh...@gmail.com wrote: Indeed reparsing connection string is not cheap, but

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Simon Riggs
On 27 June 2012 23:01, Robert Haas robertmh...@gmail.com wrote: 1. Are there any call sites from which this should be disabled, either because the optimization won't help, or because the caller is holding a lock that we don't want them sitting on for a moment longer than necessary?  I think

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Daniel Farina
On Wed, Jun 27, 2012 at 7:00 PM, Josh Berkus j...@agliodbs.com wrote: I've seen this at two sites now, and my conclusion is that a single autovacuum_max_workers isn't sufficient if to cover the case of wraparound vacuum. Nor can we just single-thread the wraparound vacuum (i.e. just one

Re: [HACKERS] pg_signal_backend() asymmetry

2012-06-28 Thread Daniel Farina
On Wed, Jun 27, 2012 at 5:38 PM, Josh Kupershmidt schmi...@gmail.com wrote: Hi all, I have one nitpick related to the recent changes for pg_cancel_backend() and pg_terminate_backend(). If you use these functions as an unprivileged user, and try to signal a nonexistent PID, you get: I think

[HACKERS] Patch: Fix for a small tipo (space lost)

2012-06-28 Thread Alexander Lakhin
Fix for a small tipo (space lost) From be61035d21512324aafd41074511625d97d17256 Mon Sep 17 00:00:00 2001 From: Alexander Lakhin exclus...@gmail.com Date: Thu, 28 Jun 2012 12:10:25 +0400 Subject: Fix for a small tipo (space lost). --- src/backend/utils/misc/guc.c |2 +- 1 file changed, 1

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Magnus Hagander
On Thu, Jun 28, 2012 at 7:00 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 27, 2012 at 9:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jun 27, 2012 at 12:00 AM, Tom Lane t...@sss.pgh.pa.us wrote: Would Posix shmem help with that at all?  

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 7:05 AM, Magnus Hagander mag...@hagander.net wrote: Do we really need a runtime check for that? Isn't a configure check enough? If they *do* deploy postgresql 9.3 on something that old, they're building from source anyway... [...] Could we actually turn *that* into a

Re: [HACKERS] Patch: Fix for a small tipo (space lost)

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 4:44 AM, Alexander Lakhin exclus...@gmail.com wrote: Fix for a small tipo (space lost) Thanks! Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

[HACKERS] Covering Indexes

2012-06-28 Thread David E. Wheeler
Hackers, Very interesting design document for SQLite 4: http://www.sqlite.org/src4/doc/trunk/www/design.wiki I'm particularly intrigued by covering indexes. For example: CREATE INDEX cover1 ON table1(a,b) COVERING(c,d); This allows the following query to do an index-only scan:

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 4:21 AM, Simon Riggs si...@2ndquadrant.com wrote: Let's put this in now, without too much fiddling. There is already a GUC to disable it, so measurements can be made to see if this causes any problems. If there are problems, we fix them. We can't second guess

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 2:02 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: It's just ridiculous to assert that it doesn't matter if all the anti-wraparound vacuums start simultaneously.  It does matter.  For one thing, once every single autovacuum worker is

Re: [HACKERS] pg_signal_backend() asymmetry

2012-06-28 Thread Magnus Hagander
On Thu, Jun 28, 2012 at 10:36 AM, Daniel Farina dan...@heroku.com wrote: On Wed, Jun 27, 2012 at 5:38 PM, Josh Kupershmidt schmi...@gmail.com wrote: Hi all, I have one nitpick related to the recent changes for pg_cancel_backend() and pg_terminate_backend(). If you use these functions as an

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Andreas Joseph Krogh
On 06/28/2012 02:16 PM, David E. Wheeler wrote: Hackers, Very interesting design document for SQLite 4: http://www.sqlite.org/src4/doc/trunk/www/design.wiki I'm particularly intrigued by covering indexes. For example: CREATE INDEX cover1 ON table1(a,b) COVERING(c,d); This allows the

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Rob Wultsch
On Thu, Jun 28, 2012 at 8:16 AM, David E. Wheeler da...@justatheory.com wrote: Hackers, Very interesting design document for SQLite 4:  http://www.sqlite.org/src4/doc/trunk/www/design.wiki I'm particularly intrigued by covering indexes. For example:    CREATE INDEX cover1 ON table1(a,b)

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Cédric Villemain
Parallelism is not free, ever, and particularly not here, where it has the potential to yank the disk head around between five different files, seeking like crazy, instead of a nice sequential I/O pattern on each file in turn. Interesting point. Maybe what's going on here is that

Re: [HACKERS] Event Triggers reduced, v1

2012-06-28 Thread Robert Haas
On Sun, Jun 24, 2012 at 5:46 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Here's an early revision 2 of the patch, not yet ready for commit, so including the PL stuff still. What's missing is mainly a cache reference leak to fix at DROP EVENT TRIGGER, but I failed to spot where it comes

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Jon Nelson
On Thu, Jun 28, 2012 at 6:05 AM, Magnus Hagander mag...@hagander.net wrote: On Thu, Jun 28, 2012 at 7:00 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 27, 2012 at 9:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jun 27, 2012 at 12:00 AM,

Re: [HACKERS] pg_signal_backend() asymmetry

2012-06-28 Thread Noah Misch
On Thu, Jun 28, 2012 at 01:36:49AM -0700, Daniel Farina wrote: On Wed, Jun 27, 2012 at 5:38 PM, Josh Kupershmidt schmi...@gmail.com wrote: I have one nitpick related to the recent changes for pg_cancel_backend() and pg_terminate_backend(). If you use these functions as an unprivileged user,

Re: [HACKERS] [Review] Add SPI_gettypmod() to return a field's typemod from a TupleDesc

2012-06-28 Thread Robert Haas
On Mon, Jun 18, 2012 at 3:29 PM, Amit Kapila amit.kap...@huawei.com wrote: [ review ] Chetan, this patch is waiting for an update from you. If you'd like this to get committed this CommitFest, we'll need an updated patch soon. Thanks, -- Robert Haas EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jun 28, 2012 at 2:02 AM, Tom Lane t...@sss.pgh.pa.us wrote: Well, that's a fair point, but I don't think it has anything to do with Josh's complaint --- which AFAICT is about imposed load, not about failure to vacuum things that need vacuumed.

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 9:51 AM, Tom Lane t...@sss.pgh.pa.us wrote:  You are inventing problem details to fit your solution. Well, what I'm actually doing is assuming that Josh's customers have the same problem that our customers do. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 9:47 AM, Jon Nelson jnelson+pg...@jamponi.net wrote: Why not just mmap /dev/zero (MAP_SHARED but not MAP_ANONYMOUS)?  I seem to think that's what I did when I needed this functionality oh so many moons ago. From the reading I've done on this topic, that seems to be a

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Thu, Jun 28, 2012 at 7:00 AM, Robert Haas robertmh...@gmail.com wrote: A related question is - if we do this - should we enable it only on ports where we've verified that it works, or should we just turn it on everywhere and fix breakage if/when

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Jon Nelson
On Thu, Jun 28, 2012 at 8:57 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 28, 2012 at 9:47 AM, Jon Nelson jnelson+pg...@jamponi.net wrote: Why not just mmap /dev/zero (MAP_SHARED but not MAP_ANONYMOUS)?  I seem to think that's what I did when I needed this functionality oh so many

Re: [HACKERS] Event Triggers reduced, v1

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 9:46 AM, Robert Haas robertmh...@gmail.com wrote: [ review ] Also: ../../../src/include/catalog/pg_event_trigger.h:34: error: expected specifier-qualifier-list before ‘int2’ This needs to be changed to int16 as a result of commit

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
... btw, I rather imagine that Robert has already noticed this, but OS X (and presumably other BSDen) spells the flag MAP_ANON not MAP_ANONYMOUS. I also find this rather interesting flag there: MAP_HASSEMAPHORE Notify the kernel that the region may contain sema-

[HACKERS] Patch-2 (2-move-continuation-record-to-page-header.patch) WAL Format Changes

2012-06-28 Thread Amit Kapila
While reading patch-2 (2-move-continuation-record-to-page-header.patch) of WAL Format Changes(http://archives.postgresql.org/message-id/4FDA5136.6080206@enterpris edb.com), I had few observations which are summarized below: 1. @@ -693,7 +693,6 @@ XLogInsert(RmgrId rmid, uint8 info,

Re: [HACKERS] [v9.3] Row-Level Security

2012-06-28 Thread Kohei KaiGai
2012/6/27 Florian Pflug f...@phlo.org: On Jun27, 2012, at 15:07 , Kohei KaiGai wrote: Probably, PlannedStmt-invalItems allows to handle invalidation of plan-cache without big code changes. I'll try to put a flag of user-id to track the query plan with RLS assumed, or InvalidOid if no RLS was

Re: [HACKERS] experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-06-28 Thread Jeff Janes
On Tue, Jun 26, 2012 at 3:58 PM, Nils Goroll sl...@schokola.de wrote: It's still unproven whether it'd be an improvement, but you could expect to prove it one way or the other with a well-defined amount of testing. I've hacked the code to use adaptive pthread mutexes instead of spinlocks. see

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 10:11 AM, Tom Lane t...@sss.pgh.pa.us wrote: ... btw, I rather imagine that Robert has already noticed this, but OS X (and presumably other BSDen) spells the flag MAP_ANON not MAP_ANONYMOUS.  I also find this rather interesting flag there:     MAP_HASSEMAPHORE  Notify

Re: [HACKERS] experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 11:21 AM, Jeff Janes jeff.ja...@gmail.com wrote: Also, 20 transactions per connection is not enough of a run to make any evaluation on. FWIW, I kicked off a looong benchmarking run on this a couple of days ago on the IBM POWER7 box, testing pgbench -S, regular pgbench,

Re: [HACKERS] [v9.3] Row-Level Security

2012-06-28 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: 2012/6/27 Florian Pflug f...@phlo.org: Hm, what happens if a SECURITY DEFINER functions returns a refcursor? My impression is, here is no matter even if SECURITY DEFINER function returns refcursor. I think Florian has a point: it *should* work, but

Re: [HACKERS] [v9.3] Row-Level Security

2012-06-28 Thread Florian Pflug
On Jun28, 2012, at 17:29 , Tom Lane wrote: Kohei KaiGai kai...@kaigai.gr.jp writes: 2012/6/27 Florian Pflug f...@phlo.org: Hm, what happens if a SECURITY DEFINER functions returns a refcursor? My impression is, here is no matter even if SECURITY DEFINER function returns refcursor. I

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Jeff Janes
On Thu, Jun 28, 2012 at 5:16 AM, David E. Wheeler da...@justatheory.com wrote: Hackers, Very interesting design document for SQLite 4:  http://www.sqlite.org/src4/doc/trunk/www/design.wiki I'm particularly intrigued by covering indexes. For example:    CREATE INDEX cover1 ON table1(a,b)

Re: [HACKERS] [v9.3] Row-Level Security

2012-06-28 Thread Tom Lane
Florian Pflug f...@phlo.org writes: On Jun28, 2012, at 17:29 , Tom Lane wrote: I believe it works today, because the executor only applies permissions checks during query startup. So those checks are executed while still within the SECURITY DEFINER context, and should behave as expected.

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Andrew Dunstan
On 06/28/2012 11:37 AM, Jeff Janes wrote: On Thu, Jun 28, 2012 at 5:16 AM, David E. Wheelerda...@justatheory.com wrote: Hackers, Very interesting design document for SQLite 4: http://www.sqlite.org/src4/doc/trunk/www/design.wiki I'm particularly intrigued by covering indexes. For

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-28 Thread Robert Haas
On Tue, Jun 26, 2012 at 8:13 PM, Andres Freund and...@2ndquadrant.com wrote: It even can be significantly higher than max_connections because subtransactions are only recognizable as part of their parent transaction uppon commit. I've been wondering whether sub-XID assignment was going to end

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 06:01:10 PM Robert Haas wrote: On Tue, Jun 26, 2012 at 8:13 PM, Andres Freund and...@2ndquadrant.com wrote: It even can be significantly higher than max_connections because subtransactions are only recognizable as part of their parent transaction uppon commit.

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 06/28/2012 11:37 AM, Jeff Janes wrote: On Thu, Jun 28, 2012 at 5:16 AM, David E. Wheelerda...@justatheory.com wrote: I'm particularly intrigued by covering indexes. For example: CREATE INDEX cover1 ON table1(a,b) COVERING(c,d); I don't see the

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Alvaro Herrera
Excerpts from Tom Lane's message of jue jun 28 12:07:58 -0400 2012: When this came up a couple weeks ago, the argument that was made for it was that you could attach non-significant columns to an index that *is* unique. That might or might not be a wide enough use-case to justify adding

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Thom Brown
On 28 June 2012 16:26, Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 28, 2012 at 10:11 AM, Tom Lane t...@sss.pgh.pa.us wrote: ... btw, I rather imagine that Robert has already noticed this, but OS X (and presumably other BSDen) spells the flag MAP_ANON not MAP_ANONYMOUS.  I also find

Re: [HACKERS] embedded list v2

2012-06-28 Thread Robert Haas
On Tue, Jun 26, 2012 at 7:26 PM, Andres Freund and...@2ndquadrant.com wrote: Attached are three patches: 1. embedded list implementation 2. make the list implementation usable without USE_INLINE 3. convert all callers to ilist.h away from dllist.h This code doesn't follow PostgreSQL coding

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Aidan Van Dyk
On Thu, Jun 28, 2012 at 12:12 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: The other question is whether such an index would prevent an update from being HOT when the non-indexed values are touched.  That could be a significant difference. I don't see Index-Only-Scans being something

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: The other question is whether such an index would prevent an update from being HOT when the non-indexed values are touched. Surely it would *have* to, whether the columns are significant or not for uniqueness purposes. Else an index-only scan

Re: [HACKERS] pg_signal_backend() asymmetry

2012-06-28 Thread Josh Kupershmidt
On Thu, Jun 28, 2012 at 6:48 AM, Noah Misch n...@leadboat.com wrote: On Thu, Jun 28, 2012 at 01:36:49AM -0700, Daniel Farina wrote: On Wed, Jun 27, 2012 at 5:38 PM, Josh Kupershmidt schmi...@gmail.com wrote: I have one nitpick related to the recent changes for pg_cancel_backend() and

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Jeff Janes
On Thu, Jun 28, 2012 at 8:26 AM, Robert Haas robertmh...@gmail.com wrote: 3. Consider adjusting the logic inside initdb.  If this works everywhere, the code for determining how to set shared_buffers should become pretty much irrelevant.  Even if it only works some places, we could add 64MB or

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Jeff Janes
On Thu, Jun 28, 2012 at 9:12 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Tom Lane's message of jue jun 28 12:07:58 -0400 2012: When this came up a couple weeks ago, the argument that was made for it was that you could attach non-significant columns to an index that *is*

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 12:13 PM, Thom Brown t...@linux.com wrote: On 64-bit Linux, if I allocate more shared buffers than the system is capable of reserving, it doesn't start.  This is expected, but there's no error logged anywhere (actually, nothing logged at all), and the postmaster.pid

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Magnus Hagander
On Thu, Jun 28, 2012 at 7:15 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 28, 2012 at 12:13 PM, Thom Brown t...@linux.com wrote: On 64-bit Linux, if I allocate more shared buffers than the system is capable of reserving, it doesn't start.  This is expected, but there's no error

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: On Thu, Jun 28, 2012 at 7:15 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 28, 2012 at 12:13 PM, Thom Brown t...@linux.com wrote: On 64-bit Linux, if I allocate more shared buffers than the system is capable of

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Magnus Hagander
On Thu, Jun 28, 2012 at 7:27 PM, Andres Freund and...@2ndquadrant.com wrote: On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: On Thu, Jun 28, 2012 at 7:15 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 28, 2012 at 12:13 PM, Thom Brown t...@linux.com wrote: On 64-bit

[HACKERS] Re: Patch-2 (2-move-continuation-record-to-page-header.patch) WAL Format Changes

2012-06-28 Thread Heikki Linnakangas
On 28.06.2012 17:40, Amit Kapila wrote: 1. @@ -693,7 +693,6 @@ XLogInsert(RmgrId rmid, uint8 info, XLogRecData *rdata) { XLogCtlInsert *Insert =XLogCtl-Insert; XLogRecord *record; -XLogContRecord *contrecord; XLogRecPtrRecPtr; XLogRecPtr

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Thu, Jun 28, 2012 at 7:27 PM, Andres Freund and...@2ndquadrant.com wrote: On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: What happens if you mlock() it into memory - does that fail quickly? Is that not something we might want to do

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 07:43:16 PM Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Thu, Jun 28, 2012 at 7:27 PM, Andres Freund and...@2ndquadrant.com wrote: On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: What happens if you mlock() it into memory - does

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Heikki Linnakangas
On 28.06.2012 15:18, Robert Haas wrote: On Thu, Jun 28, 2012 at 4:21 AM, Simon Riggssi...@2ndquadrant.com wrote: 2. Should we rename the GUCs, since this patch will cause them to control WAL flush in general, as opposed to commit specifically? Peter Geoghegan and Simon were arguing that we

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On Thursday, June 28, 2012 07:43:16 PM Tom Lane wrote: I think it *would* be a good idea to mlock if we could. Setting shmem large enough that it swaps has always been horrible for performance, and in sysv-land there's no way to prevent that. But

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 08:00:06 PM Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On Thursday, June 28, 2012 07:43:16 PM Tom Lane wrote: I think it *would* be a good idea to mlock if we could. Setting shmem large enough that it swaps has always been horrible for

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On Thursday, June 28, 2012 08:00:06 PM Tom Lane wrote: Well, the permissions angle is actually a good thing here. There is pretty much no risk of the mlock succeeding on a box that hasn't been specially configured --- and, in most cases, I think

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 18:57, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: FWIW, I think commit_delay is just fine. In practice, it's mostly commits that are affected, anyway. If we try to be more exact, I think it's just going to be more confusing to users. You think it will confuse

Re: [HACKERS] embedded list v2

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 06:23:05 PM Robert Haas wrote: On Tue, Jun 26, 2012 at 7:26 PM, Andres Freund and...@2ndquadrant.com wrote: Attached are three patches: 1. embedded list implementation 2. make the list implementation usable without USE_INLINE 3. convert all callers to ilist.h

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Kevin Grittner
Peter Geoghegan pe...@2ndquadrant.com wrote: Is anyone aware of a non-zero commit_delay in the wild today? I personally am not. http://archives.postgresql.org/pgsql-performance/2011-11/msg00083.php -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 19:25, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Peter Geoghegan pe...@2ndquadrant.com wrote: Is anyone aware of a non-zero commit_delay in the wild today? I personally am not. http://archives.postgresql.org/pgsql-performance/2011-11/msg00083.php In that thread,

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 1:43 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Thu, Jun 28, 2012 at 7:27 PM, Andres Freund and...@2ndquadrant.com wrote: On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: What happens if you mlock() it into

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I tried this. At least on my fairly vanilla MacOS X desktop, an mlock for a larger amount of memory than was conveniently on hand (4GB, on a 4GB box) neither succeeded nor failed in a timely fashion but instead progressively hung the machine,

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 2:18 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: You think it will confuse users less if we start telling them to use something that we have a very long history of telling them not to use? I don't buy this line of reasoning at all. If we're going to rename the GUC,

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 19:55, Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 28, 2012 at 2:18 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: You think it will confuse users less if we start telling them to use something that we have a very long history of telling them not to use? I don't

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jun 28, 2012 at 2:18 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: You think it will confuse users less if we start telling them to use something that we have a very long history of telling them not to use? I don't buy this line of

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Josh Berkus
Robert, Tom, Stephen, So, first, a description of the specific problem I've encountered at two sites. I'm working on another email suggesting workarounds and solutions, but that's going to take a bit longer. Observation --- This problem occured on two database systems which shared the

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 20:00, Tom Lane t...@sss.pgh.pa.us wrote: See VACUUM FULL for a recent counterexample --- we basically jacked it up and drove a new implementation underneath, but we didn't change the name, despite the fact that we were obsoleting a whole lot more folk knowledge than exists

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 2:51 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I tried this.  At least on my fairly vanilla MacOS X desktop, an mlock for a larger amount of memory than was conveniently on hand (4GB, on a 4GB box) neither succeeded nor failed in a

Re: [HACKERS] embedded list v2

2012-06-28 Thread Alvaro Herrera
Excerpts from Andres Freund's message of jue jun 28 14:20:59 -0400 2012: Looks good now? The one thing I dislike about this code is the names you've chosen. I mean, ilist_s_stuff and ilist_d_stuff. I mean, why not just Slist_foo and Dlist_bar, say? As far as I can tell, you've chosen the i

[HACKERS] initdb check_need_password fix

2012-06-28 Thread Peter Eisentraut
If you run initdb -A md5, you get an error initdb: must specify a password for the superuser to enable md5 authentication In 9.2, when you run something like initdb --auth-local=peer --auth-host=md5, you still get that error, which I think is a mistake. The error should only be shown if both

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Alvaro Herrera
Excerpts from Josh Berkus's message of jue jun 28 15:03:15 -0400 2012: 2) They have large partitioned tables, in which the partitions are time-based and do not receive UPDATES after a certain date. Each partition was larger than RAM. I think the solution to this problem has nothing to do

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 2:58 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 28 June 2012 19:55, Robert Haas robertmh...@gmail.com wrote: On Thu, Jun 28, 2012 at 2:18 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: You think it will confuse users less if we start telling them to use

Re: [HACKERS] embedded list v2

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 3:47 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Andres Freund's message of jue jun 28 14:20:59 -0400 2012: Looks good now? The one thing I dislike about this code is the names you've chosen.  I mean, ilist_s_stuff and ilist_d_stuff.  I mean,

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Christopher Browne
On Thu, Jun 28, 2012 at 3:03 PM, Josh Berkus j...@agliodbs.com wrote: 1. Databases can inadvertently get to the state where many tables need wraparound vacuuming at exactly the same time, especially if they have many cold data partition tables. This suggests that this should be handled rather

Re: [HACKERS] embedded list v2

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 09:47:05 PM Alvaro Herrera wrote: Excerpts from Andres Freund's message of jue jun 28 14:20:59 -0400 2012: Looks good now? The one thing I dislike about this code is the names you've chosen. I mean, ilist_s_stuff and ilist_d_stuff. I mean, why not just Slist_foo

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 20:55, Robert Haas robertmh...@gmail.com wrote: I don't buy this line of reasoning at all.  If we're going to rename the GUC, it should be for accuracy, not PR value.  If we start renaming something every time we improve it, we're going to go nuts. We improved lots of things in

Re: [HACKERS] embedded list v2

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 10:03:26 PM Andres Freund wrote: What I wonder is how hard it would be to remove catcache.h's structs into the implementation. Thats the reason why the old and new list implementation currently is included all over the backend... Moving them into the implementation

Re: [HACKERS] embedded list v2

2012-06-28 Thread Alvaro Herrera
Excerpts from Andres Freund's message of jue jun 28 16:03:26 -0400 2012: On Thursday, June 28, 2012 09:47:05 PM Alvaro Herrera wrote: Excerpts from Andres Freund's message of jue jun 28 14:20:59 -0400 2012: Looks good now? The one thing I dislike about this code is the names you've

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Daniel Farina
On Thu, Jun 28, 2012 at 1:32 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: Anyway, it seems that no one other than you and I is very excited about renaming this for whatever reason, so maybe we should leave it at that. I think not changing the name is a really bad decision, and I am

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 22:22, Daniel Farina dan...@heroku.com wrote: All in all, I don't think this can be a very productive discussion unless someone just pitches a equal or better name overall in terms of conciseness and descriptiveness.  I'd rather optimize for those attributes.  Old advice is

Re: [HACKERS] embedded list v2

2012-06-28 Thread Alvaro Herrera
Excerpts from Andres Freund's message of jue jun 28 17:06:49 -0400 2012: On Thursday, June 28, 2012 10:03:26 PM Andres Freund wrote: What I wonder is how hard it would be to remove catcache.h's structs into the implementation. Thats the reason why the old and new list implementation

Re: [HACKERS] embedded list v2

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 11:45:08 PM Alvaro Herrera wrote: Excerpts from Andres Freund's message of jue jun 28 17:06:49 -0400 2012: On Thursday, June 28, 2012 10:03:26 PM Andres Freund wrote: What I wonder is how hard it would be to remove catcache.h's structs into the implementation.

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Daniel Farina
On Thu, Jun 28, 2012 at 2:32 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 28 June 2012 22:22, Daniel Farina dan...@heroku.com wrote: All in all, I don't think this can be a very productive discussion unless someone just pitches a equal or better name overall in terms of conciseness and

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-28 Thread Jan Urbański
On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. CREATE PROCEDURAL LANGUAGE 'plpython3u'; CREATE OR REPLACE FUNCTION pymax (a integer, b integer) RETURNS integer AS $$ if a

Re: [HACKERS] initdb check_need_password fix

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 3:48 PM, Peter Eisentraut pete...@gmx.net wrote: If you run initdb -A md5, you get an error initdb: must specify a password for the superuser to enable md5 authentication In 9.2, when you run something like initdb --auth-local=peer --auth-host=md5, you still get that

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: So there are two parts to this problem, each of which needs a different solution: 1. Databases can inadvertently get to the state where many tables need wraparound vacuuming at exactly the same time, especially if they have many cold data partition

Re: [HACKERS] Notify system doesn't recover from No space error

2012-06-28 Thread Tom Lane
Christoph Berg christoph.b...@credativ.de writes: Warming up an old thread here - we ran into the same problem. Thanks for the example proving it is repeatable. I dug into this a bit and found the problem. When a page of pg_notify is filled, asyncQueueAddEntries() advances the global

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Josh Berkus
I'm not especially sold on your theory that there's some behavior that forces such convergence, but it's certainly plausible that there was, say, a schema alteration applied to all of those partitions at about the same time. In any case, as Robert has been saying, it seems like it would be

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: I don't find Stephen's proposal of goal-based solutions to be practical. A goal-based approach makes the assumption that database activity is predictable, and IME most databases are anything but. We're talking about over the entire transaction space,

Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregation is needed

2012-06-28 Thread Etsuro Fujita
Hi Ants, -Original Message- From: Ants Aasma [mailto:a...@cybertec.at] Sent: Wednesday, June 27, 2012 9:23 PM To: Robert Haas Cc: Etsuro Fujita; Jay Levitt; Tom Lane; PostgreSQL-development; Francois Deliege Subject: Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregation is

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: Well, I think it's plausible but wrong under at least some common circumstances. In addition to seeking, it ignores FS cache effects (not that I have any idea how to account for these mathematically). It also makes the assumption that 3 autovacuum

[HACKERS] pg_upgrade log files

2012-06-28 Thread Alvaro Herrera
Right now, the pg_upgrade output files contain the output of the commands it runs. However, it does not contain the actual commands being run ... so if you want to figure out what is happening, you have to trace the pg_upgrade source while reading the log file. This is, to say the least,

Re: [HACKERS] pg_upgrade log files

2012-06-28 Thread Tom Lane
Alvaro Herrera alvhe...@alvh.no-ip.org writes: I propose this patch which echoes the commands to the respective log files. I would backpatch this to 9.2. OK, but the fflush just before fclose seems a bit pointless; fclose will do that anyway no? regards, tom lane --