[HACKERS] relation complex types

2012-06-01 Thread Jaime Casanova
Hi, I knew that we create an entry in pg_type for every table we create, what i didn't know is that we actually create 2 entries. for example CREATE TABLE foo (i int); will create types foo and _foo. so, any reason to create 2 entries? anyway, what really kept my attention is that CREATE SEQUENCE

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-06-01 Thread Kohei KaiGai
2012/6/1 Robert Haas : > On Thu, May 31, 2012 at 3:52 PM, Kohei KaiGai wrote: >> It may be an option to separate the case into two; a situation to execute >> the given query immediately just after optimization and never reused, >> and others. > > Yes.  Simon suggested exactly this a while back, an

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Ants Aasma
On Sat, Jun 2, 2012 at 1:48 AM, Merlin Moncure wrote: > Buffer pins aren't a cache: with a cache you are trying to mask a slow > operation (like a disk i/o) with a faster such that the amount of slow > operations are minimized.  Buffer pins however are very different in > that we only care about c

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Merlin Moncure
On Fri, Jun 1, 2012 at 3:40 PM, Robert Haas wrote: > On Fri, Jun 1, 2012 at 3:16 PM, Florian Pflug wrote: >> Ok, now you've lost me. If the read() blocks, how on earth can a few >> additional pins/unpins ever account for any meaningful execution time? >> >> It seems to me that once read() blocks

Re: [HACKERS] Re: [COMMITTERS] pgsql: Send new protocol keepalive messages to standby servers.

2012-06-01 Thread Robert Haas
On Thu, May 31, 2012 at 11:46 PM, Tom Lane wrote: > Hmm ... first question is do we actually care whether the clocks are > synced to the millisecond level, ie what is it you'd do differently > if you know that the master and slave clocks are synced more closely > than you can measure at the protoc

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Robert Haas
On Fri, Jun 1, 2012 at 3:16 PM, Florian Pflug wrote: > Ok, now you've lost me. If the read() blocks, how on earth can a few > additional pins/unpins ever account for any meaningful execution time? > > It seems to me that once read() blocks we're talking about a delay in the > order of the scheduli

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Florian Pflug
On Jun1, 2012, at 21:07 , Robert Haas wrote: > On Fri, Jun 1, 2012 at 2:54 PM, Florian Pflug wrote: >> On Jun1, 2012, at 19:51 , Robert Haas wrote: >>> On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug wrote: We'd drain the unpin queue whenever we don't expect a PinBuffer() request to happe

Re: [HACKERS] Uh, I change my mind about commit_delay + commit_siblings (sort of)

2012-06-01 Thread Peter Geoghegan
This is a benchmark I performed on the same hardware, for tpc-b.sql, with a commit_delay of 0 and 4000 for the patch: http://tpcbdelay.staticloud.com/ There is a rather large improvement in throughput here. Robert previously complained that our group commit implementation didn't do very well on t

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Tom Lane
Robert Haas writes: > Another thought is that if the problem is limited to the root index > block, then we could consider less general solutions, like allowing > backends to cache the root index block and sending some kind of > invalidation when it gets split. Possibly worth noting here that we a

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Robert Haas
On Fri, Jun 1, 2012 at 2:54 PM, Florian Pflug wrote: > On Jun1, 2012, at 19:51 , Robert Haas wrote: >> On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug wrote: >>> A simpler idea would be to collapse UnpinBuffer() / PinBuffer() pairs >>> by queing UnpinBuffer() requests for a while before actually up

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Robert Haas
On Fri, Jun 1, 2012 at 2:06 PM, Jeff Janes wrote: > On Fri, Jun 1, 2012 at 10:51 AM, Robert Haas wrote: >> On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug wrote: >> >>> We'd drain the unpin queue whenever we don't expect a PinBuffer() request >>> to happen for a while. Returning to the main loop i

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Florian Pflug
On Jun1, 2012, at 19:51 , Robert Haas wrote: > On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug wrote: >> A simpler idea would be to collapse UnpinBuffer() / PinBuffer() pairs >> by queing UnpinBuffer() requests for a while before actually updating >> shared state. > > So, what happens when somebody

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Jeff Janes
On Fri, Jun 1, 2012 at 10:51 AM, Robert Haas wrote: > On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug wrote: > >> We'd drain the unpin queue whenever we don't expect a PinBuffer() request >> to happen for a while. Returning to the main loop is an obvious such place, >> but there might be others. >

Re: [HACKERS] [RFC] Interface of Row Level Security

2012-06-01 Thread Robert Haas
On Thu, May 31, 2012 at 3:52 PM, Kohei KaiGai wrote: > It may be an option to separate the case into two; a situation to execute > the given query immediately just after optimization and never reused, > and others. Yes. Simon suggested exactly this a while back, and I agree with him that we ough

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Robert Haas
On Fri, Jun 1, 2012 at 8:47 AM, Florian Pflug wrote: > A simpler idea would be to collapse UnpinBuffer() / PinBuffer() pairs > by queing UnpinBuffer() requests for a while before actually updating > shared state. So, what happens when somebody wants a cleanup lock on the buffer you've decided to

Re: [HACKERS] Schema version management

2012-06-01 Thread Kevin Grittner
Joel Jacobson wrote: > hm, is it even possible to have function names with slashes? test=# create function "is/""it""\even possible?"() returns void language plpgsql as $$begin end;$$; CREATE FUNCTION test=# select "is/""it""\even possible?"(); is/"it"\even+ possible? -- (1

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Ants Aasma
On Fri, Jun 1, 2012 at 5:57 PM, Florian Pflug wrote: > My proposed algorithm could be made to use exactly that criterion > by tracking a little bit more state. We'd have to tag queue entries > with a flag indicating whether they are > >  Unpinned (COLD) > >  Pinned, and unpinning should be delayed

Re: [HACKERS] Create collation incorrect error code

2012-06-01 Thread Tom Lane
I wrote: > Magnus Hagander writes: >> On Thu, May 31, 2012 at 8:16 PM, Tom Lane wrote: >>> What platform? >> Ubuntu 12.04. > FWIW, I get the same on Fedora 16. This seems to be a glibc bug: it won't set errno correctly if you've already asked about the same wrong locale name. Filed at https:/

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Merlin Moncure
On Fri, Jun 1, 2012 at 8:45 AM, Tom Lane wrote: > Merlin Moncure writes: >> A potential issue with this line of thinking is that your pin delay >> queue could get highly pressured by outer portions of the query (as in >> the OP's case)  that will get little or no benefit from the delayed >> pin.

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Florian Pflug
On Jun1, 2012, at 15:45 , Tom Lane wrote: > Merlin Moncure writes: >> A potential issue with this line of thinking is that your pin delay >> queue could get highly pressured by outer portions of the query (as in >> the OP's case) that will get little or no benefit from the delayed >> pin. But ch

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Tom Lane
Simon Riggs writes: > On 1 June 2012 14:59, Tom Lane wrote: >> Ah.  Well, as long as the overflowed fsyncs do get handled on the >> requesting side, I see no bug here.  No objection to changing the order >> in which we launch the processes, but as Heikki says, it's not clear >> that that is reall

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Simon Riggs
On 1 June 2012 14:59, Tom Lane wrote: > Simon Riggs writes: >> On 1 June 2012 14:29, Tom Lane wrote: >>> Surely that commit is useless.  Fsync requests go into a queue in shared >>> memory, which had better have been set up by the postmaster.  There is >>> no requirement that the receiving proce

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Tom Lane
Simon Riggs writes: > On 1 June 2012 14:29, Tom Lane wrote: >> Surely that commit is useless.  Fsync requests go into a queue in shared >> memory, which had better have been set up by the postmaster.  There is >> no requirement that the receiving process exist before somebody can put >> a request

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Simon Riggs
On 1 June 2012 14:29, Tom Lane wrote: > Surely that commit is useless.  Fsync requests go into a queue in shared > memory, which had better have been set up by the postmaster.  There is > no requirement that the receiving process exist before somebody can put > a request into the queue.  If the q

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Tom Lane
Merlin Moncure writes: > A potential issue with this line of thinking is that your pin delay > queue could get highly pressured by outer portions of the query (as in > the OP's case) that will get little or no benefit from the delayed > pin. But choosing a sufficiently sized drain queue would wo

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Merlin Moncure
On Fri, Jun 1, 2012 at 7:47 AM, Florian Pflug wrote: > On May31, 2012, at 20:50 , Robert Haas wrote: >> Suppose we introduce two new buffer flags, >> BUF_NAILED and BUF_NAIL_REMOVAL.  When we detect excessive contention >> on the buffer header spinlock, we set BUF_NAILED.  Once we do that, >> the

Re: [HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Tom Lane
Simon Riggs writes: > On 1 June 2012 08:55, Heikki Linnakangas > wrote: >> On 01.06.2012 10:28, Simon Riggs wrote: >>> Checkpointer starts before bgwriter to avoid missing fsync requests. >>> Noted while testing Hot Standby startup. >> The processes are just forked and it will take some time for

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Florian Pflug
On May31, 2012, at 20:50 , Robert Haas wrote: > Suppose we introduce two new buffer flags, > BUF_NAILED and BUF_NAIL_REMOVAL. When we detect excessive contention > on the buffer header spinlock, we set BUF_NAILED. Once we do that, > the buffer can't be evicted until that flag is removed, and back

Re: [HACKERS] slow dropping of tables, DropRelFileNodeBuffers, tas

2012-06-01 Thread Simon Riggs
On 1 June 2012 12:34, Sergey Koposov wrote: > On Fri, 1 Jun 2012, Simon Riggs wrote: > >> >> Why do you have 10,000 tables and why is it important to drop them so >> quickly? > > > 1 tables are there, because that's the number of partitions. And I'm > dropping them at the moment, because I'm d

Re: [HACKERS] 9.2beta1, parallel queries, ReleasePredicateLocks, CheckForSerializableConflictIn in the oprofile

2012-06-01 Thread Sergey Koposov
On Thu, 31 May 2012, Jeff Janes wrote: No, idt_match is getting filled by multi-threaded copy() and then joined with 4 other big tables like idt_phot. The result is then split into partitions. That does make things more complicated. But you could you partition it at that level and then do th

Re: [HACKERS] slow dropping of tables, DropRelFileNodeBuffers, tas

2012-06-01 Thread Sergey Koposov
On Fri, 1 Jun 2012, Simon Riggs wrote: Why do you have 10,000 tables and why is it important to drop them so quickly? 1 tables are there, because that's the number of partitions. And I'm dropping them at the moment, because I'm doing testing. So it won't be really crucial for production.

[HACKERS] Re: [COMMITTERS] pgsql: Checkpointer starts before bgwriter to avoid missing fsync reque

2012-06-01 Thread Simon Riggs
On 1 June 2012 08:55, Heikki Linnakangas wrote: > On 01.06.2012 10:28, Simon Riggs wrote: >> >> Checkpointer starts before bgwriter to avoid missing fsync requests. >> Noted while testing Hot Standby startup. > > > The processes are just forked and it will take some time for them to > initialize.