Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-23 Thread Teodor Sigaev
Thank you, pushed Andrew Borodin wrote: 2017-03-22 22:48 GMT+05:00 Teodor Sigaev : hasEmptyChild? and hasNonEmptyChild (BTW, isAnyNonempy has missed 't') Yes, I think this naming is good. It's clear what's in common in these flags and what's different. And if the whole posting tree is empty

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-22 Thread Andrew Borodin
2017-03-22 22:48 GMT+05:00 Teodor Sigaev : > hasEmptyChild? and hasNonEmptyChild (BTW, isAnyNonempy has missed 't') Yes, I think this naming is good. It's clear what's in common in these flags and what's different. > And if the whole posting tree is empty,then we could mark root page as leaf > an

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-22 Thread Teodor Sigaev
No, second conditional code will be called for any subtree, which contains totally empty subtree. That check !isRoot covers case when the entire posting tree should be erased: we cannot just quit out of recursive cleanup, we have to make a scan here, starting from root. Oh, I see Probably, vari

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-21 Thread Andrew Borodin
Hi, Teodor! 2017-03-21 20:32 GMT+05:00 Teodor Sigaev : > I had a look on patch That's great, thanks! > > /* > * All subtree is empty - just return TRUE to indicate that parent > must > * do a cleanup. Unless we are ROOT an there is way to go upper. > */ > >

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-21 Thread Teodor Sigaev
Thank you for your suggestions, do not hesitate to ask any questions, concurrency and GIN both are very interesting topics. I had a look on patch and found some issue. Look at ginvacuum.c around line 387, function ginVacuumPostingTreeLeaves(): /* * All subtree is empty - just

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-16 Thread Andrew Borodin
2017-03-16 23:55 GMT+05:00 Peter Geoghegan : > On Thu, Mar 16, 2017 at 11:53 AM, Andrew Borodin wrote: >> 2. Thus, L&S fully concurrent vacuum is possible, indeed, and >> furthermore Theodor suggested that I should implement not only page >> eviction, but also page merge and tree condence algorit

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-16 Thread Peter Geoghegan
On Thu, Mar 16, 2017 at 11:53 AM, Andrew Borodin wrote: > 2. Thus, L&S fully concurrent vacuum is possible, indeed, and > furthermore Theodor suggested that I should implement not only page > eviction, but also page merge and tree condence algorithm. I think that it's very hard to make merging o

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-16 Thread Andrew Borodin
2017-03-16 21:27 GMT+05:00 David Steele : > This patch applies cleanly and compiles at cccbdde. > > Jeff, any thoughts on Andrew's responses? Hi, David! I've got some updates on the matter of this patch, since the understanding of the B-tree bothered me much. Currently, I'm at PgConf.Russia, wher

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-03-16 Thread David Steele
On 2/5/17 11:04 AM, Andrew Borodin wrote: > Hi, Jeff! > > 2017-02-05 3:45 GMT+05:00 Jeff Davis : >> On Sun, Jan 22, 2017 at 10:32 PM, Jeff Davis wrote: >>> On Sat, Jan 21, 2017 at 4:25 AM, Andrew Borodin >>> wrote: >>> One idea I had that might be simpler is to use a two-stage page >>> delete.

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-02-05 Thread Andrew Borodin
Hi, Jeff! 2017-02-05 3:45 GMT+05:00 Jeff Davis : > On Sun, Jan 22, 2017 at 10:32 PM, Jeff Davis wrote: >> On Sat, Jan 21, 2017 at 4:25 AM, Andrew Borodin wrote: >> One idea I had that might be simpler is to use a two-stage page >> delete. The first stage would remove the link from the parent and

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-02-04 Thread Jeff Davis
On Sun, Jan 22, 2017 at 10:32 PM, Jeff Davis wrote: > On Sat, Jan 21, 2017 at 4:25 AM, Andrew Borodin wrote: > One idea I had that might be simpler is to use a two-stage page > delete. The first stage would remove the link from the parent and mark > the page deleted, but leave the right link inta

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-31 Thread Michael Paquier
On Tue, Jan 31, 2017 at 4:31 PM, Vladimir Borodin wrote: > 31 янв. 2017 г., в 9:50, Michael Paquier > написал(а): > >> I am marking this patch as returned with feedback. > > Michael, sorry, but why? Because I have been through many patches today. > If I understood everything right, the main que

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-30 Thread Vladimir Borodin
> 31 янв. 2017 г., в 9:50, Michael Paquier > написал(а): > > On Mon, Jan 30, 2017 at 4:28 PM, Andrew Borodin wrote: >> I'll summarize here my state of studying concurrent methods of page >> unlinking. >> >> GIN B-tree does not have "high key". That means, that rightmost key on >> a page is m

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-30 Thread Michael Paquier
On Mon, Jan 30, 2017 at 4:28 PM, Andrew Borodin wrote: > I'll summarize here my state of studying concurrent methods of page unlinking. > > GIN B-tree does not have "high key". That means, that rightmost key on > a page is maximal for the non-leaf page. > But I do not see anything theoretical in a

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-29 Thread Andrew Borodin
2017-01-25 12:09 GMT+05:00 Andrew Borodin : > 2017-01-24 22:29 GMT+05:00 Jeff Davis : >> By the way, can you show me where the Lehman and Yao paper addresses >> page recycling? > > Here J. Hellerstein comments L&Y paper [1] saying that effectively > there is no deletion algorithm provided. > Here [

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-24 Thread Andrew Borodin
2017-01-24 22:29 GMT+05:00 Jeff Davis : > On Tue, Jan 24, 2017 at 3:18 AM, Andrew Borodin wrote: >> Technically, approach of locking a subtree is not novel. Lehman and >> Yao focused on "that any process for manipulating the tree uses only a >> small (constant) number of locks at any time." We are

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-24 Thread Jeff Davis
On Tue, Jan 24, 2017 at 3:18 AM, Andrew Borodin wrote: > Technically, approach of locking a subtree is not novel. Lehman and > Yao focused on "that any process for manipulating the tree uses only a > small (constant) number of locks at any time." We are locking unknown > and possibly large amount

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-24 Thread Andrew Borodin
Hello! I see your point on alternatives. I will do my best to generate more ideas on how vacuum can be done withing existing index structure, this will take a day or two. In this message I'll answer some questions. 2017-01-23 22:45 GMT+05:00 Jeff Davis : > 1. I haven't seen the approach before of

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-23 Thread Jeff Davis
On Mon, Jan 23, 2017 at 1:55 AM, Andrew Borodin wrote: > Proposed patch, on a contrary, simplifies code. There is more code > deleted than added. It does not affect WAL, changes nothing in page > layout. I appreciate that it is fewer lines of code. My hesitation comes from a couple areas: 1. I h

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-23 Thread Andrew Borodin
Hi! 2017-01-23 11:32 GMT+05:00 Jeff Davis : > I have some reservations about the complexity and novelty of the > approach. I don't think I've seen an approach quite like this one > before. It seems like the main reason you are using this approach is > because the btree structure was too simple (no

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-22 Thread Jeff Davis
On Sat, Jan 21, 2017 at 4:25 AM, Andrew Borodin wrote: > Hello Jeff! > >>Review of the code itself: > How do you think, is there anything else to improve in that patch or > we can mark it as 'Ready for committer'? > > I've checked the concurrency algorithm with original work of Lehman > and Yao on

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-21 Thread Andrew Borodin
Hello Jeff! >Review of the code itself: How do you think, is there anything else to improve in that patch or we can mark it as 'Ready for committer'? I've checked the concurrency algorithm with original work of Lehman and Yao on B-link-tree. For me everything looks fine (safe, deadlock free and n

Re: [HACKERS] Review: GIN non-intrusive vacuum of posting tree

2017-01-19 Thread Andrew Borodin
Hi, Jeff! Thanks for review! Here's a patch corrected according to your suggestions. 2017-01-19 11:48 GMT+05:00 Jeff Davis : > https://www.postgresql.org/message-id/CAJEAwVE4UAmm8fr%2BNW8XTnKV6M--ACoNhL3ES8yoKL2sKhbaiw%40mail.gmail.com > > Let me re-summarize what's been done here to make sure I