Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Alexander Korotkov
On Tue, Jul 21, 2015 at 7:20 PM, Heikki Linnakangas wrote: > On 07/21/2015 04:14 PM, Alexander Korotkov wrote: > >> On Tue, Jul 21, 2015 at 3:52 PM, Heikki Linnakangas >> wrote: >> >> On 07/21/2015 02:56 PM, Alexander Korotkov wrote: >>> >>> Probably, but currently we are in quite unlogical si

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 04:14 PM, Alexander Korotkov wrote: On Tue, Jul 21, 2015 at 3:52 PM, Heikki Linnakangas wrote: On 07/21/2015 02:56 PM, Alexander Korotkov wrote: Probably, but currently we are in quite unlogical situation. We have default fillfactor = 90 for GiST where it has no use cases at al

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Alexander Korotkov
On Tue, Jul 21, 2015 at 3:52 PM, Heikki Linnakangas wrote: > On 07/21/2015 02:56 PM, Alexander Korotkov wrote: > >> Probably, but currently we are in quite unlogical situation. We have >> default fillfactor = 90 for GiST where it has no use cases at all and >> effectively is just a waste of space

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 02:56 PM, Alexander Korotkov wrote: Probably, but currently we are in quite unlogical situation. We have default fillfactor = 90 for GiST where it has no use cases at all and effectively is just a waste of space. Why is it useless for GiST? - Heikki -- Sent via pgsql-hackers ma

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Alexander Korotkov
On Tue, Jul 21, 2015 at 2:49 PM, Heikki Linnakangas wrote: > Hmm. That's slightly different from the test case I used, in that the > update is changing the indexed value, which means that the new value goes > to different posting tree than the old one. I'm not sure if that makes any > difference.

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 02:00 PM, Alexander Korotkov wrote: On Tue, Jul 21, 2015 at 12:40 PM, Heikki Linnakangas wrote: Has anyone done any performance testing of this? The purpose of a fillfactor is to avoid the storm of page splits right after building the index, when there are some random updates to

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Alexander Korotkov
On Tue, Jul 21, 2015 at 12:40 PM, Heikki Linnakangas wrote: > Has anyone done any performance testing of this? > > The purpose of a fillfactor is to avoid the storm of page splits right > after building the index, when there are some random updates to the table. > It causes the index to bloat, as

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
Has anyone done any performance testing of this? The purpose of a fillfactor is to avoid the storm of page splits right after building the index, when there are some random updates to the table. It causes the index to bloat, as every full page is split to two half-full pages, and also adds lat

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Michael Paquier
On Fri, Jul 10, 2015 at 7:13 PM, Alexander Korotkov wrote: > On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier < > michael.paqu...@gmail.com> wrote: > >> >> >> On Thu, Jul 9, 2015 at 10:33 PM, Alexander Korotkov wrote: >> > [...] >> >> + /* Caclculate max data size on page according to fillfac

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Heikki Linnakangas
On 07/10/2015 01:13 PM, Alexander Korotkov wrote: On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier wrote: + #define GIN_MIN_FILLFACTOR20 + #define GIN_DEFAULT_FILLFACTOR90 I am still worrying about using a default fillfactor at 90, as we did a lot of promotion about compress

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Alexander Korotkov
On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier wrote: > > > On Thu, Jul 9, 2015 at 10:33 PM, Alexander Korotkov wrote: > > [...] > > + /* Caclculate max data size on page according to fillfactor */ > s/Caclculate/Calculate > > When creating a simple gin index, I am seeing that GinGetMaxData

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-09 Thread Michael Paquier
On Thu, Jul 9, 2015 at 10:33 PM, Alexander Korotkov wrote: > [...] + /* Caclculate max data size on page according to fillfactor */ s/Caclculate/Calculate When creating a simple gin index, I am seeing that GinGetMaxDataSize gets called with ginEntryInsert: * frame #0: 0x00010a49d72e pos

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-09 Thread Alexander Korotkov
Hi! On Wed, Jul 8, 2015 at 10:27 PM, Heikki Linnakangas wrote: > In dataPlaceToPageLeaf-function: > > if (append) >> { >> /* >> * Even when appending, trying to append more items than >> will fit is >> * not completely free, beca

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-08 Thread Heikki Linnakangas
In dataPlaceToPageLeaf-function: if (append) { /* * Even when appending, trying to append more items than will fit is * not completely free, because we will merge the new items and old * items into an array belo

Re: [HACKERS] Fillfactor for GIN indexes

2015-02-25 Thread Tomas Vondra
On 25.2.2015 10:20, Alexander Korotkov wrote: > On Tue, Feb 24, 2015 at 5:15 PM, Tomas Vondra > mailto:tomas.von...@2ndquadrant.com>> wrote: > >> I hoped lowering the fillfactor will improve this, but >> fillfactor=75 had pretty much no effect in this case. Is that >> expected for this kind of wor

Re: [HACKERS] Fillfactor for GIN indexes

2015-02-25 Thread Alexander Korotkov
On Tue, Feb 24, 2015 at 5:15 PM, Tomas Vondra wrote: > I hoped lowering the fillfactor will improve this, but fillfactor=75 had > pretty much no effect in this case. Is that expected for this kind of > workload? I see the previous discussion talked about random updates, not > inserts, so maybe th

Re: [HACKERS] Fillfactor for GIN indexes

2015-02-24 Thread Tomas Vondra
Hi, I've been wondering whether this might improve behavior with one of my workloads, suffering by GIN bloat - the same one I used to test GIN fastscan, for example. It's a batch process that loads a mailing list archive into a table with a GIN index on message body, by doing something like this:

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Michael Paquier
On Tue, Jan 20, 2015 at 12:06 AM, Cédric Villemain wrote: > Michael, I first didn't understood how GiN can benefits from the > patch...thus my question. > > There were no noise for me, and I learn some more about GiN. So I thank you > for your work! Kicking back the ball, I haven't done as much wo

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Cédric Villemain
Le samedi 17 janvier 2015 08:23:44 Michael Paquier a écrit : > On Sat, Jan 17, 2015 at 2:40 AM, Robert Haas wrote: > > There's only value in adding a fillfactor parameter to GIN indexes > > if > > it improves performance. There are no benchmarks showing it does. > > So, why are we still talking a

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Alexander Korotkov
On Mon, Jan 19, 2015 at 5:46 PM, Cédric Villemain wrote: > Le lundi 19 janvier 2015 08:24:08 Robert Haas a écrit : > > > On Sat, Jan 17, 2015 at 4:49 AM, Alexander Korotkov > > > > > > wrote: > > > > I already wrote quite detailed explanation of subject. Let mel try > > > > to > > > > explain i

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Cédric Villemain
Le lundi 19 janvier 2015 08:24:08 Robert Haas a écrit : > On Sat, Jan 17, 2015 at 4:49 AM, Alexander Korotkov > > wrote: > > I already wrote quite detailed explanation of subject. Let mel try > > to > > explain in shortly. GIN is two level nested btree. Thus, GIN would > > have absolutely same ben

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Robert Haas
On Sat, Jan 17, 2015 at 4:49 AM, Alexander Korotkov wrote: > I already wrote quite detailed explanation of subject. Let mel try to > explain in shortly. GIN is two level nested btree. Thus, GIN would have > absolutely same benefits from fillfactor as btree. Lack of tests showing it > is, for sure,

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-17 Thread Alexander Korotkov
On Sat, Jan 17, 2015 at 12:49 PM, Alexander Korotkov wrote: > BTW, previous version of patch contained some bugs. Revised version is > attached. > Ooops, it's here. -- With best regards, Alexander Korotkov. gin_fillfactor_5.patch Description: Binary data -- Sent via pgsql-hackers mailin

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-17 Thread Alexander Korotkov
On Fri, Jan 16, 2015 at 8:40 PM, Robert Haas wrote: > On Thu, Jan 15, 2015 at 7:06 AM, Michael Paquier > wrote: > > Alexander Korotkov wrote: > >> I'm not sure. On the one hand it's unclear why fillfactor should be > >> different from 9.4. > >> On the other hand it's unclear why it should be dif

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-16 Thread Michael Paquier
On Sat, Jan 17, 2015 at 2:40 AM, Robert Haas wrote: > There's only value in adding a fillfactor parameter to GIN indexes if > it improves performance. There are no benchmarks showing it does. > So, why are we still talking about this? Indeed. There is no such benchmark as of now, and I am not sur

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-16 Thread Robert Haas
On Thu, Jan 15, 2015 at 7:06 AM, Michael Paquier wrote: > Alexander Korotkov wrote: >> I'm not sure. On the one hand it's unclear why fillfactor should be >> different from 9.4. >> On the other hand it's unclear why it should be different from btree. >> I propose marking this "ready for committer"

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-15 Thread Michael Paquier
Alexander Korotkov wrote: > I'm not sure. On the one hand it's unclear why fillfactor should be > different from 9.4. > On the other hand it's unclear why it should be different from btree. > I propose marking this "ready for committer". So, committer can make a final > decision. OK let's do so the

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-15 Thread Alexander Korotkov
On Thu, Jan 15, 2015 at 10:19 AM, Michael Paquier wrote: > On Thu, Jan 8, 2015 at 2:03 PM, Michael Paquier > wrote: > > On Thu, Jan 8, 2015 at 6:31 AM, Alexander Korotkov > wrote: > >> On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier < > michael.paqu...@gmail.com> > >>> I am attaching an updated

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-14 Thread Michael Paquier
On Thu, Jan 8, 2015 at 2:03 PM, Michael Paquier wrote: > On Thu, Jan 8, 2015 at 6:31 AM, Alexander Korotkov > wrote: >> On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier >>> I am attaching an updated patch, with the default fillfactor value at >>> 75%, and with the page split code using the fillf

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Michael Paquier
On Thu, Jan 8, 2015 at 6:31 AM, Alexander Korotkov wrote: > On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier >> I am attaching an updated patch, with the default fillfactor value at >> 75%, and with the page split code using the fillfactor rate. >> Thoughts? > Rewritten version of patch is attache

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Alexander Korotkov
On Thu, Jan 8, 2015 at 12:31 AM, Alexander Korotkov wrote: > > Rewritten version of patch is attached. I made following changes: > > 1) I removed fillfactor handling from entry tree. Because in this case > fillfactor parameter would be only upper bound for actual page fullness. > It's very like Gi

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Alexander Korotkov
On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier wrote: > On Wed, Dec 3, 2014 at 2:37 AM, Robert Haas wrote: > > On Fri, Nov 28, 2014 at 4:27 AM, Alexander Korotkov > > wrote: > >> On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier < > michael.paqu...@gmail.com> > >> wrote: > >>> Please find attac

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Michael Paquier
On Wed, Dec 3, 2014 at 2:37 AM, Robert Haas wrote: > On Fri, Nov 28, 2014 at 4:27 AM, Alexander Korotkov > wrote: >> On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier >> wrote: >>> Please find attached a simple patch adding fillfactor as storage parameter >>> for GIN indexes. The default value is

Re: [HACKERS] Fillfactor for GIN indexes

2014-12-02 Thread Robert Haas
On Fri, Nov 28, 2014 at 4:27 AM, Alexander Korotkov wrote: > On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier > wrote: >> Please find attached a simple patch adding fillfactor as storage parameter >> for GIN indexes. The default value is the same as the one currently aka 100 >> to have the pages

Re: [HACKERS] Fillfactor for GIN indexes

2014-11-28 Thread Alexander Korotkov
Hi! On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier wrote: > Please find attached a simple patch adding fillfactor as storage parameter > for GIN indexes. The default value is the same as the one currently aka 100 > to have the pages completely packed when a GIN index is created. > That's not

Re: [HACKERS] Fillfactor for GIN indexes

2014-11-27 Thread Cédric Villemain
Le jeudi 27 novembre 2014 23:33:11 Michael Paquier a écrit : > On Fri, Nov 21, 2014 at 2:12 PM, Michael Paquier > > wrote: > > I am adding that to the commit fest of December. > > Here are updated patches. Alvaro notified me about an inconsistent > comment. what are the benefits of this patch ?

Re: [HACKERS] Fillfactor for GIN indexes

2014-11-27 Thread Michael Paquier
On Fri, Nov 21, 2014 at 2:12 PM, Michael Paquier wrote: > I am adding that to the commit fest of December. Here are updated patches. Alvaro notified me about an inconsistent comment. -- Michael From eda0730d991f8b4dfbacc4d7a953ec5bff8b2ffe Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Fri

[HACKERS] Fillfactor for GIN indexes

2014-11-20 Thread Michael Paquier
Hi all, Please find attached a simple patch adding fillfactor as storage parameter for GIN indexes. The default value is the same as the one currently aka 100 to have the pages completely packed when a GIN index is created. Note that to have this feature correctly working, the fix I sent yesterda