Re: [HACKERS] pg_upgrade log files

2012-06-28 Thread Tom Lane
Alvaro Herrera writes: > I propose this patch which echoes the commands to the respective log > files. I would backpatch this to 9.2. OK, but the fflush just before fclose seems a bit pointless; fclose will do that anyway no? regards, tom lane -- Sent via pgsql-hackers

[HACKERS] pg_upgrade log files

2012-06-28 Thread Alvaro Herrera
Right now, the pg_upgrade output files contain the output of the commands it runs. However, it does not contain the actual commands being run ... so if you want to figure out what is happening, you have to trace the pg_upgrade source while reading the log file. This is, to say the least, suboptim

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Tom Lane
Josh Berkus writes: > Well, I think it's "plausible but wrong under at least some common > circumstances". In addition to seeking, it ignores FS cache effects > (not that I have any idea how to account for these mathematically). It > also makes the assumption that 3 autovacuum workers running at

Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregation is needed

2012-06-28 Thread Etsuro Fujita
Hi Ants, > -Original Message- > From: Ants Aasma [mailto:a...@cybertec.at] > Sent: Wednesday, June 27, 2012 9:23 PM > To: Robert Haas > Cc: Etsuro Fujita; Jay Levitt; Tom Lane; PostgreSQL-development; Francois > Deliege > Subject: Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregatio

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: > I don't find Stephen's proposal of goal-based solutions to be practical. > A goal-based approach makes the assumption that database activity is > predictable, and IME most databases are anything but. We're talking about over the entire transaction space,

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Josh Berkus
> I'm not especially sold on your theory that there's some behavior that > forces such convergence, but it's certainly plausible that there was, > say, a schema alteration applied to all of those partitions at about the > same time. In any case, as Robert has been saying, it seems like it > would

Re: [HACKERS] Notify system doesn't recover from "No space" error

2012-06-28 Thread Tom Lane
Christoph Berg writes: > Warming up an old thread here - we ran into the same problem. Thanks for the example proving it is repeatable. I dug into this a bit and found the problem. When a page of pg_notify is filled, asyncQueueAddEntries() advances the global QUEUE_HEAD to point to the next pag

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Tom Lane
Josh Berkus writes: > So there are two parts to this problem, each of which needs a different > solution: > 1. Databases can inadvertently get to the state where many tables need > wraparound vacuuming at exactly the same time, especially if they have > many "cold" data partition tables. I'm not

Re: [HACKERS] initdb check_need_password fix

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 3:48 PM, Peter Eisentraut wrote: > If you run initdb -A md5, you get an error > > initdb: must specify a password for the superuser to enable md5 authentication > > In 9.2, when you run something like initdb --auth-local=peer > --auth-host=md5, you still get that error, whi

Re: [HACKERS] plpython issue with Win64 (PG 9.2)

2012-06-28 Thread Jan Urbański
On 27/06/12 13:57, Jan Urbański wrote: On 27/06/12 11:51, Asif Naeem wrote: Hi, On Windows 7 64bit, plpython is causing server crash with the following test case i.e. CREATE PROCEDURAL LANGUAGE 'plpython3u'; CREATE OR REPLACE FUNCTION pymax (a integer, b integer) RETURNS integer AS $$ if a> b

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Daniel Farina
On Thu, Jun 28, 2012 at 2:32 PM, Peter Geoghegan wrote: > On 28 June 2012 22:22, Daniel Farina wrote: >> All in all, I don't think this can be a very productive discussion >> unless someone just pitches a equal or better name overall in terms of >> conciseness and descriptiveness. I'd rather opt

Re: [HACKERS] embedded list v2

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 11:45:08 PM Alvaro Herrera wrote: > Excerpts from Andres Freund's message of jue jun 28 17:06:49 -0400 2012: > > On Thursday, June 28, 2012 10:03:26 PM Andres Freund wrote: > > > What I wonder is how hard it would be to remove catcache.h's structs > > > into the implemen

Re: [HACKERS] embedded list v2

2012-06-28 Thread Alvaro Herrera
Excerpts from Andres Freund's message of jue jun 28 17:06:49 -0400 2012: > On Thursday, June 28, 2012 10:03:26 PM Andres Freund wrote: > > What I wonder is how hard it would be to remove catcache.h's structs into > > the implementation. Thats the reason why the old and new list > > implementation

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 22:22, Daniel Farina wrote: > All in all, I don't think this can be a very productive discussion > unless someone just pitches a equal or better name overall in terms of > conciseness and descriptiveness.  I'd rather optimize for those > attributes.  Old advice is old; that's the n

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Daniel Farina
On Thu, Jun 28, 2012 at 1:32 PM, Peter Geoghegan wrote: >> Anyway, it seems that no one other than you and I is very excited >> about renaming this for whatever reason, so maybe we should leave it >> at that. > > I think not changing the name is a really bad decision, and I am > personally unhappy

Re: [HACKERS] embedded list v2

2012-06-28 Thread Alvaro Herrera
Excerpts from Andres Freund's message of jue jun 28 16:03:26 -0400 2012: > > On Thursday, June 28, 2012 09:47:05 PM Alvaro Herrera wrote: > > Excerpts from Andres Freund's message of jue jun 28 14:20:59 -0400 2012: > > > Looks good now? > > > > The one thing I dislike about this code is the name

Re: [HACKERS] embedded list v2

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 10:03:26 PM Andres Freund wrote: > What I wonder is how hard it would be to remove catcache.h's structs into > the implementation. Thats the reason why the old and new list > implementation currently is included all over the backend... Moving them into the implementation

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 20:55, Robert Haas wrote: >>> I don't buy this line of reasoning at all.  If we're going to rename >>> the GUC, it should be for accuracy, not PR value.  If we start >>> renaming something every time we improve it, we're going to go nuts. >>> We improved lots of things in 9.2; they

Re: [HACKERS] embedded list v2

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 09:47:05 PM Alvaro Herrera wrote: > Excerpts from Andres Freund's message of jue jun 28 14:20:59 -0400 2012: > > Looks good now? > > The one thing I dislike about this code is the names you've chosen. I > mean, ilist_s_stuff and ilist_d_stuff. I mean, why not just Slis

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Christopher Browne
On Thu, Jun 28, 2012 at 3:03 PM, Josh Berkus wrote: > 1. Databases can inadvertently get to the state where many tables need > wraparound vacuuming at exactly the same time, especially if they have > many "cold" data partition tables. This suggests that this should be handled rather earlier, and

Re: [HACKERS] embedded list v2

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 3:47 PM, Alvaro Herrera wrote: > > Excerpts from Andres Freund's message of jue jun 28 14:20:59 -0400 2012: > >> Looks good now? > > The one thing I dislike about this code is the names you've chosen.  I > mean, ilist_s_stuff and ilist_d_stuff.  I mean, why not just Slist_f

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 2:58 PM, Peter Geoghegan wrote: > On 28 June 2012 19:55, Robert Haas wrote: >> On Thu, Jun 28, 2012 at 2:18 PM, Peter Geoghegan >> wrote: >>> You think it will confuse users less if we start telling them to use >>> something that we have a very long history of telling th

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Alvaro Herrera
Excerpts from Josh Berkus's message of jue jun 28 15:03:15 -0400 2012: > 2) They have large partitioned tables, in which the partitions are > time-based and do not receive UPDATES after a certain date. Each > partition was larger than RAM. I think the solution to this problem has nothing to do

[HACKERS] initdb check_need_password fix

2012-06-28 Thread Peter Eisentraut
If you run initdb -A md5, you get an error initdb: must specify a password for the superuser to enable md5 authentication In 9.2, when you run something like initdb --auth-local=peer --auth-host=md5, you still get that error, which I think is a mistake. The error should only be shown if both auth

Re: [HACKERS] embedded list v2

2012-06-28 Thread Alvaro Herrera
Excerpts from Andres Freund's message of jue jun 28 14:20:59 -0400 2012: > Looks good now? The one thing I dislike about this code is the names you've chosen. I mean, ilist_s_stuff and ilist_d_stuff. I mean, why not just Slist_foo and Dlist_bar, say? As far as I can tell, you've chosen the "i

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 2:51 PM, Tom Lane wrote: > Robert Haas writes: >> I tried this.  At least on my fairly vanilla MacOS X desktop, an mlock >> for a larger amount of memory than was conveniently on hand (4GB, on a >> 4GB box) neither succeeded nor failed in a timely fashion but instead >> pr

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 20:00, Tom Lane wrote: > See VACUUM FULL for a recent counterexample --- we basically jacked it > up and drove a new implementation underneath, but we didn't change the > name, despite the fact that we were obsoleting a whole lot more folk > knowledge than exists around commit_dela

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Josh Berkus
Robert, Tom, Stephen, So, first, a description of the specific problem I've encountered at two sites. I'm working on another email suggesting workarounds and solutions, but that's going to take a bit longer. Observation --- This problem occured on two database systems which shared the f

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Tom Lane
Robert Haas writes: > On Thu, Jun 28, 2012 at 2:18 PM, Peter Geoghegan > wrote: >> You think it will confuse users less if we start telling them to use >> something that we have a very long history of telling them not to use? > I don't buy this line of reasoning at all. If we're going to renam

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 19:55, Robert Haas wrote: > On Thu, Jun 28, 2012 at 2:18 PM, Peter Geoghegan > wrote: >> You think it will confuse users less if we start telling them to use >> something that we have a very long history of telling them not to use? > > I don't buy this line of reasoning at all.  

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 2:18 PM, Peter Geoghegan wrote: > You think it will confuse users less if we start telling them to use > something that we have a very long history of telling them not to use? I don't buy this line of reasoning at all. If we're going to rename the GUC, it should be for ac

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Robert Haas writes: > I tried this. At least on my fairly vanilla MacOS X desktop, an mlock > for a larger amount of memory than was conveniently on hand (4GB, on a > 4GB box) neither succeeded nor failed in a timely fashion but instead > progressively hung the machine, apparently trying to progr

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 1:43 PM, Tom Lane wrote: > Magnus Hagander writes: >> On Thu, Jun 28, 2012 at 7:27 PM, Andres Freund >> wrote: >>> On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: What happens if you mlock() it into memory - does that fail quickly? Is that not som

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 19:25, Kevin Grittner wrote: > Peter Geoghegan wrote: > >> Is anyone aware of a non-zero commit_delay in the wild today? I >> personally am not. > > http://archives.postgresql.org/pgsql-performance/2011-11/msg00083.php In that thread, Robert goes on to say to the OP that has set

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Kevin Grittner
Peter Geoghegan wrote: > Is anyone aware of a non-zero commit_delay in the wild today? I > personally am not. http://archives.postgresql.org/pgsql-performance/2011-11/msg00083.php -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] embedded list v2

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 06:23:05 PM Robert Haas wrote: > On Tue, Jun 26, 2012 at 7:26 PM, Andres Freund wrote: > > Attached are three patches: > > 1. embedded list implementation > > 2. make the list implementation usable without USE_INLINE > > 3. convert all callers to ilist.h away from dllis

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Peter Geoghegan
On 28 June 2012 18:57, Heikki Linnakangas wrote: > FWIW, I think commit_delay is just fine. In practice, it's mostly commits > that are affected, anyway. If we try to be more exact, I think it's just > going to be more confusing to users. You think it will confuse users less if we start telling t

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Andres Freund writes: > On Thursday, June 28, 2012 08:00:06 PM Tom Lane wrote: >> Well, the permissions angle is actually a good thing here. There is >> pretty much no risk of the mlock succeeding on a box that hasn't been >> specially configured --- and, in most cases, I think you'd need root >>

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 08:00:06 PM Tom Lane wrote: > Andres Freund writes: > > On Thursday, June 28, 2012 07:43:16 PM Tom Lane wrote: > >> I think it *would* be a good idea to mlock if we could. Setting shmem > >> large enough that it swaps has always been horrible for performance, > >> and i

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Andres Freund writes: > On Thursday, June 28, 2012 07:43:16 PM Tom Lane wrote: >> I think it *would* be a good idea to mlock if we could. Setting shmem >> large enough that it swaps has always been horrible for performance, >> and in sysv-land there's no way to prevent that. But we can't error >

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Heikki Linnakangas
On 28.06.2012 15:18, Robert Haas wrote: On Thu, Jun 28, 2012 at 4:21 AM, Simon Riggs wrote: 2. Should we rename the GUCs, since this patch will cause them to control WAL flush in general, as opposed to commit specifically? Peter Geoghegan and Simon were arguing that we should retitle it to grou

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 07:43:16 PM Tom Lane wrote: > Magnus Hagander writes: > > On Thu, Jun 28, 2012 at 7:27 PM, Andres Freund wrote: > >> On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: > >>> What happens if you mlock() it into memory - does that fail quickly? > >>> Is that n

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Magnus Hagander writes: > On Thu, Jun 28, 2012 at 7:27 PM, Andres Freund wrote: >> On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: >>> What happens if you mlock() it into memory - does that fail quickly? >>> Is that not something we might want to do *anyway*? >> You normally can on

[HACKERS] Re: Patch-2 (2-move-continuation-record-to-page-header.patch) WAL Format Changes

2012-06-28 Thread Heikki Linnakangas
On 28.06.2012 17:40, Amit Kapila wrote: 1. @@ -693,7 +693,6 @@ XLogInsert(RmgrId rmid, uint8 info, XLogRecData *rdata) { XLogCtlInsert *Insert =&XLogCtl->Insert; XLogRecord *record; -XLogContRecord *contrecord; XLogRecPtrRecPtr; XLogRecPtr

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Magnus Hagander
On Thu, Jun 28, 2012 at 7:27 PM, Andres Freund wrote: > On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: >> On Thu, Jun 28, 2012 at 7:15 PM, Robert Haas wrote: >> > On Thu, Jun 28, 2012 at 12:13 PM, Thom Brown wrote: >> >> On 64-bit Linux, if I allocate more shared buffers than the

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 07:19:46 PM Magnus Hagander wrote: > On Thu, Jun 28, 2012 at 7:15 PM, Robert Haas wrote: > > On Thu, Jun 28, 2012 at 12:13 PM, Thom Brown wrote: > >> On 64-bit Linux, if I allocate more shared buffers than the system is > >> capable of reserving, it doesn't start. This

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Magnus Hagander
On Thu, Jun 28, 2012 at 7:15 PM, Robert Haas wrote: > On Thu, Jun 28, 2012 at 12:13 PM, Thom Brown wrote: >> On 64-bit Linux, if I allocate more shared buffers than the system is >> capable of reserving, it doesn't start.  This is expected, but there's >> no error logged anywhere (actually, nothi

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 12:13 PM, Thom Brown wrote: > On 64-bit Linux, if I allocate more shared buffers than the system is > capable of reserving, it doesn't start.  This is expected, but there's > no error logged anywhere (actually, nothing logged at all), and the > postmaster.pid file is left b

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Jeff Janes
On Thu, Jun 28, 2012 at 9:12 AM, Alvaro Herrera wrote: > > Excerpts from Tom Lane's message of jue jun 28 12:07:58 -0400 2012: > >> When this came up a couple weeks ago, the argument that was made for it >> was that you could attach non-significant columns to an index that *is* >> unique.  That mi

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Jeff Janes
On Thu, Jun 28, 2012 at 8:26 AM, Robert Haas wrote: > 3. Consider adjusting the logic inside initdb.  If this works > everywhere, the code for determining how to set shared_buffers should > become pretty much irrelevant.  Even if it only works some places, we > could add 64MB or 128MB or whatever

Re: [HACKERS] pg_signal_backend() asymmetry

2012-06-28 Thread Josh Kupershmidt
On Thu, Jun 28, 2012 at 6:48 AM, Noah Misch wrote: > On Thu, Jun 28, 2012 at 01:36:49AM -0700, Daniel Farina wrote: >> On Wed, Jun 27, 2012 at 5:38 PM, Josh Kupershmidt wrote: >> > I have one nitpick related to the recent changes for >> > pg_cancel_backend() and pg_terminate_backend(). If you use

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Tom Lane
Alvaro Herrera writes: > The other question is whether such an index would prevent an update from > being HOT when the non-indexed values are touched. Surely it would *have* to, whether the columns are significant or not for uniqueness purposes. Else an index-only scan gives the wrong value afte

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Aidan Van Dyk
On Thu, Jun 28, 2012 at 12:12 PM, Alvaro Herrera wrote: > The other question is whether such an index would prevent an update from > being HOT when the non-indexed values are touched.  That could be a > significant difference. I don't see Index-Only-Scans being something that will be used in "hi

Re: [HACKERS] embedded list v2

2012-06-28 Thread Robert Haas
On Tue, Jun 26, 2012 at 7:26 PM, Andres Freund wrote: > Attached are three patches: > 1. embedded list implementation > 2. make the list implementation usable without USE_INLINE > 3. convert all callers to ilist.h away from dllist.h This code doesn't follow PostgreSQL coding style guidelines; in

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Thom Brown
On 28 June 2012 16:26, Robert Haas wrote: > On Thu, Jun 28, 2012 at 10:11 AM, Tom Lane wrote: >> ... btw, I rather imagine that Robert has already noticed this, but OS X >> (and presumably other BSDen) spells the flag "MAP_ANON" not >> "MAP_ANONYMOUS".  I also find this rather interesting flag th

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Alvaro Herrera
Excerpts from Tom Lane's message of jue jun 28 12:07:58 -0400 2012: > When this came up a couple weeks ago, the argument that was made for it > was that you could attach non-significant columns to an index that *is* > unique. That might or might not be a wide enough use-case to justify > adding

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Tom Lane
Andrew Dunstan writes: > On 06/28/2012 11:37 AM, Jeff Janes wrote: >> On Thu, Jun 28, 2012 at 5:16 AM, David E. Wheeler >> wrote: >>> I'm particularly intrigued by "covering indexes". For example: >>> CREATE INDEX cover1 ON table1(a,b) COVERING(c,d); >> I don't see the virtue of this in this ca

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-28 Thread Andres Freund
On Thursday, June 28, 2012 06:01:10 PM Robert Haas wrote: > On Tue, Jun 26, 2012 at 8:13 PM, Andres Freund wrote: > > It even can be significantly higher than max_connections because > > subtransactions are only recognizable as part of their parent transaction > > uppon commit. > > I've been won

Re: [HACKERS] [PATCH 08/16] Introduce the ApplyCache module which can reassemble transactions from a stream of interspersed changes

2012-06-28 Thread Robert Haas
On Tue, Jun 26, 2012 at 8:13 PM, Andres Freund wrote: > It even can be significantly higher than max_connections because > subtransactions are only recognizable as part of their parent transaction > uppon commit. I've been wondering whether sub-XID assignment was going to end up on the list of th

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Andrew Dunstan
On 06/28/2012 11:37 AM, Jeff Janes wrote: On Thu, Jun 28, 2012 at 5:16 AM, David E. Wheeler wrote: Hackers, Very interesting design document for SQLite 4: http://www.sqlite.org/src4/doc/trunk/www/design.wiki I'm particularly intrigued by "covering indexes". For example: CREATE INDEX

Re: [HACKERS] [v9.3] Row-Level Security

2012-06-28 Thread Tom Lane
Florian Pflug writes: > On Jun28, 2012, at 17:29 , Tom Lane wrote: >> I believe it works today, because the executor only applies permissions >> checks during query startup. So those checks are executed while still >> within the SECURITY DEFINER context, and should behave as expected. >> Subseque

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Jeff Janes
On Thu, Jun 28, 2012 at 5:16 AM, David E. Wheeler wrote: > Hackers, > > Very interesting design document for SQLite 4: > >  http://www.sqlite.org/src4/doc/trunk/www/design.wiki > > I'm particularly intrigued by "covering indexes". For example: > >    CREATE INDEX cover1 ON table1(a,b) COVERING(c,d

Re: [HACKERS] [v9.3] Row-Level Security

2012-06-28 Thread Florian Pflug
On Jun28, 2012, at 17:29 , Tom Lane wrote: > Kohei KaiGai writes: >> 2012/6/27 Florian Pflug : >>> Hm, what happens if a SECURITY DEFINER functions returns a refcursor? > >> My impression is, here is no matter even if SECURITY DEFINER function >> returns refcursor. > > I think Florian has a poin

Re: [HACKERS] [v9.3] Row-Level Security

2012-06-28 Thread Tom Lane
Kohei KaiGai writes: > 2012/6/27 Florian Pflug : >> Hm, what happens if a SECURITY DEFINER functions returns a refcursor? > My impression is, here is no matter even if SECURITY DEFINER function > returns refcursor. I think Florian has a point: it *should* work, but *will* it? I believe it works

Re: [HACKERS] experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 11:21 AM, Jeff Janes wrote: > Also, 20 transactions per connection is not enough of a run to make > any evaluation on. FWIW, I kicked off a looong benchmarking run on this a couple of days ago on the IBM POWER7 box, testing pgbench -S, regular pgbench, and pgbench --unlogg

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 10:11 AM, Tom Lane wrote: > ... btw, I rather imagine that Robert has already noticed this, but OS X > (and presumably other BSDen) spells the flag "MAP_ANON" not > "MAP_ANONYMOUS".  I also find this rather interesting flag there: > >     MAP_HASSEMAPHORE  Notify the kernel

Re: [HACKERS] experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-06-28 Thread Jeff Janes
On Tue, Jun 26, 2012 at 3:58 PM, Nils Goroll wrote: >> It's >> still unproven whether it'd be an improvement, but you could expect to >> prove it one way or the other with a well-defined amount of testing. > > I've hacked the code to use adaptive pthread mutexes instead of spinlocks. see > attache

Re: [HACKERS] [v9.3] Row-Level Security

2012-06-28 Thread Kohei KaiGai
2012/6/27 Florian Pflug : > On Jun27, 2012, at 15:07 , Kohei KaiGai wrote: >> Probably, PlannedStmt->invalItems allows to handle invalidation of >> plan-cache without big code changes. I'll try to put a flag of user-id >> to track the query plan with RLS assumed, or InvalidOid if no RLS >> was appl

[HACKERS] Patch-2 (2-move-continuation-record-to-page-header.patch) WAL Format Changes

2012-06-28 Thread Amit Kapila
While reading patch-2 (2-move-continuation-record-to-page-header.patch) of WAL Format Changes(http://archives.postgresql.org/message-id/4FDA5136.6080206@enterpris edb.com), I had few observations which are summarized below: 1. @@ -693,7 +693,6 @@ XLogInsert(RmgrId rmid, uint8 info, XLogRecDat

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
... btw, I rather imagine that Robert has already noticed this, but OS X (and presumably other BSDen) spells the flag "MAP_ANON" not "MAP_ANONYMOUS". I also find this rather interesting flag there: MAP_HASSEMAPHORE Notify the kernel that the region may contain sema- p

Re: [HACKERS] Event Triggers reduced, v1

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 9:46 AM, Robert Haas wrote: > [ review ] Also: ../../../src/include/catalog/pg_event_trigger.h:34: error: expected specifier-qualifier-list before ‘int2’ This needs to be changed to int16 as a result of commit b8b2e3b2deeaab19715af063fc009b7c230b2336. alter.c:73: warnin

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Jon Nelson
On Thu, Jun 28, 2012 at 8:57 AM, Robert Haas wrote: > On Thu, Jun 28, 2012 at 9:47 AM, Jon Nelson wrote: >> Why not just mmap /dev/zero (MAP_SHARED but not MAP_ANONYMOUS)?  I >> seem to think that's what I did when I needed this functionality oh so >> many moons ago. > > From the reading I've don

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Tom Lane
Magnus Hagander writes: > On Thu, Jun 28, 2012 at 7:00 AM, Robert Haas wrote: >> A related question is - if we do this - should we enable it only on >> ports where we've verified that it works, or should we just turn it on >> everywhere and fix breakage if/when it's reported?  I lean toward the >

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 9:47 AM, Jon Nelson wrote: > Why not just mmap /dev/zero (MAP_SHARED but not MAP_ANONYMOUS)?  I > seem to think that's what I did when I needed this functionality oh so > many moons ago. From the reading I've done on this topic, that seems to be a trick invented on Solaris

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 9:51 AM, Tom Lane wrote: >  You are inventing problem details to fit > your solution. Well, what I'm actually doing is assuming that Josh's customers have the same problem that our customers do. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Post

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Tom Lane
Robert Haas writes: > On Thu, Jun 28, 2012 at 2:02 AM, Tom Lane wrote: >> Well, that's a fair point, but I don't think it has anything to do with >> Josh's complaint --- which AFAICT is about imposed load, not about >> failure to vacuum things that need vacuumed. > I think it's got everything to

Re: [HACKERS] [Review] Add SPI_gettypmod() to return a field's typemod from a TupleDesc

2012-06-28 Thread Robert Haas
On Mon, Jun 18, 2012 at 3:29 PM, Amit Kapila wrote: > [ review ] Chetan, this patch is waiting for an update from you. If you'd like this to get committed this CommitFest, we'll need an updated patch soon. Thanks, -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgre

Re: [HACKERS] pg_signal_backend() asymmetry

2012-06-28 Thread Noah Misch
On Thu, Jun 28, 2012 at 01:36:49AM -0700, Daniel Farina wrote: > On Wed, Jun 27, 2012 at 5:38 PM, Josh Kupershmidt wrote: > > I have one nitpick related to the recent changes for > > pg_cancel_backend() and pg_terminate_backend(). If you use these > > functions as an unprivileged user, and try to

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Jon Nelson
On Thu, Jun 28, 2012 at 6:05 AM, Magnus Hagander wrote: > On Thu, Jun 28, 2012 at 7:00 AM, Robert Haas wrote: >> On Wed, Jun 27, 2012 at 9:44 AM, Tom Lane wrote: >>> Robert Haas writes: On Wed, Jun 27, 2012 at 12:00 AM, Tom Lane wrote: > Would Posix shmem help with that at all?  Why d

Re: [HACKERS] Event Triggers reduced, v1

2012-06-28 Thread Robert Haas
On Sun, Jun 24, 2012 at 5:46 PM, Dimitri Fontaine wrote: > Here's an early revision 2 of the patch, not yet ready for commit, so > including the PL stuff still. What's missing is mainly a cache reference > leak to fix at DROP EVENT TRIGGER, but I failed to spot where it comes > from. > > As I fixe

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Cédric Villemain
> >> Parallelism is not free, ever, and particularly not here, where it has > >> the potential to yank the disk head around between five different > >> files, seeking like crazy, instead of a nice sequential I/O pattern on > >> each file in turn. > > > > Interesting point. Maybe what's going on h

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Rob Wultsch
On Thu, Jun 28, 2012 at 8:16 AM, David E. Wheeler wrote: > Hackers, > > Very interesting design document for SQLite 4: > >  http://www.sqlite.org/src4/doc/trunk/www/design.wiki > > I'm particularly intrigued by "covering indexes". For example: > >    CREATE INDEX cover1 ON table1(a,b) COVERING(c,d

Re: [HACKERS] Covering Indexes

2012-06-28 Thread Andreas Joseph Krogh
On 06/28/2012 02:16 PM, David E. Wheeler wrote: Hackers, Very interesting design document for SQLite 4: http://www.sqlite.org/src4/doc/trunk/www/design.wiki I'm particularly intrigued by "covering indexes". For example: CREATE INDEX cover1 ON table1(a,b) COVERING(c,d); This allows th

Re: [HACKERS] pg_signal_backend() asymmetry

2012-06-28 Thread Magnus Hagander
On Thu, Jun 28, 2012 at 10:36 AM, Daniel Farina wrote: > On Wed, Jun 27, 2012 at 5:38 PM, Josh Kupershmidt wrote: >> Hi all, >> >> I have one nitpick related to the recent changes for >> pg_cancel_backend() and pg_terminate_backend(). If you use these >> functions as an unprivileged user, and try

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 2:02 AM, Tom Lane wrote: > Robert Haas writes: >> It's just ridiculous to assert that it doesn't matter if all the >> anti-wraparound vacuums start simultaneously.  It does matter.  For >> one thing, once every single autovacuum worker is pinned down doing an >> anti-wrapa

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 4:21 AM, Simon Riggs wrote: > Let's put this in now, without too much fiddling. There is already a > GUC to disable it, so measurements can be made to see if this causes > any problems. > > If there are problems, we fix them. We can't second guess everything. Fair enough.

[HACKERS] Covering Indexes

2012-06-28 Thread David E. Wheeler
Hackers, Very interesting design document for SQLite 4: http://www.sqlite.org/src4/doc/trunk/www/design.wiki I'm particularly intrigued by "covering indexes". For example: CREATE INDEX cover1 ON table1(a,b) COVERING(c,d); This allows the following query to do an index-only scan: SEL

Re: [HACKERS] Patch: Fix for a small tipo (space lost)

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 4:44 AM, Alexander Lakhin wrote: > Fix for a small tipo (space lost) Thanks! Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Robert Haas
On Thu, Jun 28, 2012 at 7:05 AM, Magnus Hagander wrote: > Do we really need a runtime check for that? Isn't a configure check > enough? If they *do* deploy postgresql 9.3 on something that old, > they're building from source anyway... [...] > > Could we actually turn *that* into a configure test,

Re: [HACKERS] Posix Shared Mem patch

2012-06-28 Thread Magnus Hagander
On Thu, Jun 28, 2012 at 7:00 AM, Robert Haas wrote: > On Wed, Jun 27, 2012 at 9:44 AM, Tom Lane wrote: >> Robert Haas writes: >>> On Wed, Jun 27, 2012 at 12:00 AM, Tom Lane wrote: Would Posix shmem help with that at all?  Why did you choose not to use the Posix API, anyway? >> >>> It

[HACKERS] Patch: Fix for a small tipo (space lost)

2012-06-28 Thread Alexander Lakhin
Fix for a small tipo (space lost) >From be61035d21512324aafd41074511625d97d17256 Mon Sep 17 00:00:00 2001 From: Alexander Lakhin Date: Thu, 28 Jun 2012 12:10:25 +0400 Subject: Fix for a small tipo (space lost). --- src/backend/utils/misc/guc.c |2 +- 1 file changed, 1 insertion(+), 1 deleti

Re: [HACKERS] pg_signal_backend() asymmetry

2012-06-28 Thread Daniel Farina
On Wed, Jun 27, 2012 at 5:38 PM, Josh Kupershmidt wrote: > Hi all, > > I have one nitpick related to the recent changes for > pg_cancel_backend() and pg_terminate_backend(). If you use these > functions as an unprivileged user, and try to signal a nonexistent > PID, you get: I think the goal ther

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-06-28 Thread Daniel Farina
On Wed, Jun 27, 2012 at 7:00 PM, Josh Berkus wrote: > I've seen this at two sites now, and my conclusion is that a single > autovacuum_max_workers isn't sufficient if to cover the case of > wraparound vacuum. Nor can we just single-thread the wraparound vacuum > (i.e. just one worker) since that

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-28 Thread Simon Riggs
On 27 June 2012 23:01, Robert Haas wrote: > 1. Are there any call sites from which this should be disabled, either > because the optimization won't help, or because the caller is holding > a lock that we don't want them sitting on for a moment longer than > necessary?  I think the worst case is t