Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On 02/04/2019 08:58, Andrey Lepikhov wrote: On 25/03/2019 15:21, Heikki Linnakangas wrote: I had another quick look. I still think using the "generic xlog AM" for this is a wrong level of abstraction, and we should use the XLOG_FPI records for this directly. We can extend XLOG_FPI so that it

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On 25/03/2019 15:21, Heikki Linnakangas wrote: I had another quick look. I still think using the "generic xlog AM" for this is a wrong level of abstraction, and we should use the XLOG_FPI records for this directly. We can extend XLOG_FPI so that it can store multiple pages in a single

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On 26/03/2019 15:59, Heikki Linnakangas wrote: On 26/03/2019 11:29, Andrey Lepikhov wrote: On 25/03/2019 15:21, Heikki Linnakangas wrote: Hmm. When do we create all-zero pages during index build? That seems pretty surprising. GIST uses buffered pages. During GIST build it is possible (very

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On 26/03/2019 11:29, Andrey Lepikhov wrote: On 25/03/2019 15:21, Heikki Linnakangas wrote: Hmm. When do we create all-zero pages during index build? That seems pretty surprising. GIST uses buffered pages. During GIST build it is possible (very rarely) what no one index tuple was written to

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On 25/03/2019 15:21, Heikki Linnakangas wrote: On 25/03/2019 09:57, David Steele wrote: On 2/6/19 2:08 PM, Andrey Lepikhov wrote: The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So  lsn field

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On 25/03/2019 09:57, David Steele wrote: On 2/6/19 2:08 PM, Andrey Lepikhov wrote: The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So lsn field at all-zero page was initialized and the

Re: Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On 2/6/19 2:08 PM, Andrey Lepikhov wrote: The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So lsn field at all-zero page was initialized and the PageIsVerified() routine detects it as a bad page.

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

The patchset had a problem with all-zero pages, has appeared at index build stage: the generic_log_relation() routine sends all pages into the WAL. So lsn field at all-zero page was initialized and the PageIsVerified() routine detects it as a bad page. The solution may be: 1. To improve index

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On 04.02.2019 10:04, Michael Paquier wrote: > On Tue, Dec 18, 2018 at 10:41:48AM +0500, Andrey Lepikhov wrote: >> Ok. It is used only for demonstration. > > The latest patch set needs a rebase, so moved to next CF, waiting on > author as this got no reviews. The new version in attachment. > -- >

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On Tue, Dec 18, 2018 at 10:41:48AM +0500, Andrey Lepikhov wrote: > Ok. It is used only for demonstration. The latest patch set needs a rebase, so moved to next CF, waiting on author as this got no reviews. -- Michael signature.asc Description: PGP signature

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On 30.11.2018 15:10, Dmitry Dolgov wrote: Thank you. I played a bit with this patch, and can confirm visible WAL size reduction (it's rather obvious, but still). Although, probably due to lack of my knowledge here, I wonder how does it work when an index is build concurrently? Routine

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

> On Tue, Jul 31, 2018 at 6:36 AM Andrey Lepikhov > wrote: > > With the consent of Anastasia I will improving this patch further. > Attachment contains next version of the patch set. Thank you. I played a bit with this patch, and can confirm visible WAL size reduction (it's rather obvious, but

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

With the consent of Anastasia I will improving this patch further. Attachment contains next version of the patch set. 11.07.2018 00:03, Heikki Linnakangas пишет: On 28/02/18 18:03, Anastasia Lubennikova wrote: Implementation is based on generic_xlog. Why? I think we should just add a

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

On Tue, Jul 10, 2018 at 10:03 PM Heikki Linnakangas wrote: > > On 28/02/18 18:03, Anastasia Lubennikova wrote: > > I want to propose a bunch of patches which allow to reduce WAL traffic > > generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree > > and RUM, we can now log index

Re: Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

Hi Anastasia, On 2/28/18 11:03 AM, Anastasia Lubennikova wrote: > I want to propose a bunch of patches which allow to reduce WAL traffic > generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree > and RUM, we can now log index pages of other access methods only once > in the end

Reduce amount of WAL generated by CREATE INDEX for gist, gin and sp-gist

Hi, I want to propose a bunch of patches which allow to reduce WAL traffic generated by CREATE INDEX for GiST, GIN and SP-GiST. Similarly to b-tree and RUM, we can now log index pages of other access methods only once in the end of indexbuild process. Implementation is based on generic_xlog.