Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-19 Thread Robert Haas
On Mon, Jul 15, 2013 at 1:41 PM, Jeff Davis wrote: >> I'm of the opinion that we ought to extract the parts of the patch >> that hold the VM pin for longer, review those separately, and if >> they're good and desirable, apply them. > > I'm confused. My patch holds a VM page pinned for those cases

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-15 Thread Jeff Davis
On Sun, 2013-07-14 at 23:06 -0400, Robert Haas wrote: > > Of course, there's a reason that PD_ALL_VISIBLE is not like a normal > > hint: we need to make sure that inserts/updates/deletes clear the VM > > bit. But my patch already addresses that by keeping the VM page pinned. > > I'm of the opinion

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-14 Thread Robert Haas
On Fri, Jul 5, 2013 at 4:18 PM, Jeff Davis wrote: > On Tue, 2013-07-02 at 10:12 -0700, Jeff Davis wrote: >> Regardless, this is at least a concrete issue that I can focus on, and I >> appreciate that. Are scans of small tables the primary objection to this >> patch, or are there others? If I solve

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-05 Thread Jeff Davis
On Tue, 2013-07-02 at 10:12 -0700, Jeff Davis wrote: > Regardless, this is at least a concrete issue that I can focus on, and I > appreciate that. Are scans of small tables the primary objection to this > patch, or are there others? If I solve it, will this patch make real > progress? I had an ide

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-02 Thread Jeff Davis
On Tue, 2013-07-02 at 19:34 +0200, Andres Freund wrote: > Well, I still have my doubts that it's a good idea to remove this > knowledge from the page level. And that's not primarily related to > performance. Unfortunately a good part of judging architectural > questions is gut feeling... > The only

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-02 Thread Andres Freund
On 2013-07-02 10:12:31 -0700, Jeff Davis wrote: > On Tue, 2013-07-02 at 14:02 +0200, Andres Freund wrote: > > Ok, so you want some benchmark results. I spent 20 minutes concocting some > > quick tests. Here you go: > > > > master (384f933046dc9e9a2b416f5f7b3be30b93587c63): > > tps = 155075.448341

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-02 Thread Jeff Davis
On Tue, 2013-07-02 at 14:02 +0200, Andres Freund wrote: > Ok, so you want some benchmark results. I spent 20 minutes concocting some > quick tests. Here you go: > > master (384f933046dc9e9a2b416f5f7b3be30b93587c63): > tps = 155075.448341 (including connections establishing) > tps = 155259.752267 (

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-02 Thread Andres Freund
On 2013-07-02 14:02:22 +0200, Andres Freund wrote: > Data loaded: load.sql. Attached... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services DROP TABLE IF EXISTS kv; CREATE TABLE kv( id serial

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-02 Thread Andres Freund
On 2013-07-01 19:52:57 -0700, Jeff Davis wrote: > 2. Can you be more specific about the scenarios that you *are* concerned > about? Preferably in a form that could be tested on a 64-core box; but > at least some kind of analysis involving numbers. "More page accesses" > is scary, but completely mea

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-01 Thread Jeff Davis
On Mon, 2013-07-01 at 16:05 -0400, Robert Haas wrote: > Well, I don't believe there's any way to really eliminate the > contention concern completely. There's no way around the fact that it > means more access to the visibility map, and I've seen recent (albeit > circumstantial thus far) evidence

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-01 Thread Jeff Davis
On Mon, 2013-07-01 at 20:59 -0400, Robert Haas wrote: > One of several relevant emails is at: > > http://www.postgresql.org/message-id/51a7473c.6070...@vmware.com > > It is definitely possible that I am mixing up two different things. > But if I am, I don't know what the other one is. I believ

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-01 Thread Robert Haas
On Mon, Jul 1, 2013 at 8:23 PM, Jeff Davis wrote: > Can you point me to that criticism? Why can't you just drop the VM > completely if it becomes corrupted? > > (You might be referring to another idea of mine that was related to > Andres's proposal for "getting rid of freezing".) One of several r

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-01 Thread Jeff Davis
On Mon, 2013-07-01 at 16:05 -0400, Robert Haas wrote: > The other concern I remember being expressed (and not just by me, but > by a number of people) is that your patch turns loss of a visibility > map bit into a data corruption scenario, which it currently isn't. > Right now, if your visibility m

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-01 Thread Robert Haas
On Mon, Jul 1, 2013 at 1:21 PM, Jeff Davis wrote: > On Sun, 2013-06-30 at 22:58 -0400, Robert Haas wrote: >> I thought that Jeff withdrew this patch. > > No -- was there a reason you thought that? I thought I remembered you saying you were going to abandon it in the face of objections. > I know

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-07-01 Thread Jeff Davis
On Sun, 2013-06-30 at 22:58 -0400, Robert Haas wrote: > I thought that Jeff withdrew this patch. No -- was there a reason you thought that? I know it could use another round of testing before commit, and there may be a couple other things to clear up. But I don't want to invest a lot of time there

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-06-30 Thread Josh Berkus
> I thought that Jeff withdrew this patch. > He did, but nobody removed it from the commitfest --- partly because of a change of subject line breaking the thread. Bounced to "returned with feedback" now. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-06-30 Thread Robert Haas
On Sat, Jun 29, 2013 at 11:24 AM, Robins wrote: > On 10 June 2013 00:17, Jeff Davis wrote: >> >> On Thu, 2013-05-30 at 10:07 -0700, Jeff Davis wrote: >> > > Come to think of it, even without the torn page & checksum issue, do >> > > we >> > > really want to actively clear the all-visible flags af

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-06-29 Thread Robins
On 10 June 2013 00:17, Jeff Davis wrote: > On Thu, 2013-05-30 at 10:07 -0700, Jeff Davis wrote: > > > Come to think of it, even without the torn page & checksum issue, do we > > > really want to actively clear the all-visible flags after upgrade? > > Removed that from the patch and rebased. I thi

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-05-30 Thread Jeff Davis
On Thu, 2013-05-30 at 11:32 +0300, Heikki Linnakangas wrote: > > That could cause a torn page and checksum failure if checksums are > > enabled. Thank you, I missed that in the rebase; it should be MarkBufferDirtyHint(). > Come to think of it, even without the torn page & checksum issue, do we >

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-05-30 Thread Heikki Linnakangas
On 30.05.2013 11:26, Heikki Linnakangas wrote: On 30.05.2013 06:54, Jeff Davis wrote: Continuation of: http://www.postgresql.org/message-id/1353551097.11440.128.camel@sussancws0025 Rebased patch attached; no other changes. @@ -675,6 +675,16 @@ lazy_scan_heap(Relation onerel, LVRelStats *va

Re: [HACKERS] Eliminating PD_ALL_VISIBLE, take 2

2013-05-30 Thread Heikki Linnakangas
On 30.05.2013 06:54, Jeff Davis wrote: Continuation of: http://www.postgresql.org/message-id/1353551097.11440.128.camel@sussancws0025 Rebased patch attached; no other changes. @@ -675,6 +675,16 @@ lazy_scan_heap(Relation onerel, LVRelStats *vacrelstats, } /*