Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 04:14 PM, Alexander Korotkov wrote: On Tue, Jul 21, 2015 at 3:52 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/21/2015 02:56 PM, Alexander Korotkov wrote: Probably, but currently we are in quite unlogical situation. We have default fillfactor = 90 for GiST where it has no

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Alexander Korotkov
On Tue, Jul 21, 2015 at 7:20 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/21/2015 04:14 PM, Alexander Korotkov wrote: On Tue, Jul 21, 2015 at 3:52 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/21/2015 02:56 PM, Alexander Korotkov wrote: Probably, but currently we are in

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Alexander Korotkov
On Tue, Jul 21, 2015 at 12:40 PM, Heikki Linnakangas hlinn...@iki.fi wrote: Has anyone done any performance testing of this? The purpose of a fillfactor is to avoid the storm of page splits right after building the index, when there are some random updates to the table. It causes the index

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Alexander Korotkov
On Tue, Jul 21, 2015 at 2:49 PM, Heikki Linnakangas hlinn...@iki.fi wrote: Hmm. That's slightly different from the test case I used, in that the update is changing the indexed value, which means that the new value goes to different posting tree than the old one. I'm not sure if that makes any

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 02:00 PM, Alexander Korotkov wrote: On Tue, Jul 21, 2015 at 12:40 PM, Heikki Linnakangas hlinn...@iki.fi wrote: Has anyone done any performance testing of this? The purpose of a fillfactor is to avoid the storm of page splits right after building the index, when there are some

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
Has anyone done any performance testing of this? The purpose of a fillfactor is to avoid the storm of page splits right after building the index, when there are some random updates to the table. It causes the index to bloat, as every full page is split to two half-full pages, and also adds

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Heikki Linnakangas
On 07/21/2015 02:56 PM, Alexander Korotkov wrote: Probably, but currently we are in quite unlogical situation. We have default fillfactor = 90 for GiST where it has no use cases at all and effectively is just a waste of space. Why is it useless for GiST? - Heikki -- Sent via pgsql-hackers

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-21 Thread Alexander Korotkov
On Tue, Jul 21, 2015 at 3:52 PM, Heikki Linnakangas hlinn...@iki.fi wrote: On 07/21/2015 02:56 PM, Alexander Korotkov wrote: Probably, but currently we are in quite unlogical situation. We have default fillfactor = 90 for GiST where it has no use cases at all and effectively is just a waste

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Michael Paquier
On Fri, Jul 10, 2015 at 7:13 PM, Alexander Korotkov aekorot...@gmail.com wrote: On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jul 9, 2015 at 10:33 PM, Alexander Korotkov wrote: [...] + /* Caclculate max data size on page according to

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Heikki Linnakangas
On 07/10/2015 01:13 PM, Alexander Korotkov wrote: On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier michael.paqu...@gmail.com wrote: + #define GIN_MIN_FILLFACTOR20 + #define GIN_DEFAULT_FILLFACTOR90 I am still worrying about using a default fillfactor at 90, as we did a lot

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-10 Thread Alexander Korotkov
On Fri, Jul 10, 2015 at 4:54 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jul 9, 2015 at 10:33 PM, Alexander Korotkov wrote: [...] + /* Caclculate max data size on page according to fillfactor */ s/Caclculate/Calculate When creating a simple gin index, I am seeing

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-09 Thread Michael Paquier
On Thu, Jul 9, 2015 at 10:33 PM, Alexander Korotkov wrote: [...] + /* Caclculate max data size on page according to fillfactor */ s/Caclculate/Calculate When creating a simple gin index, I am seeing that GinGetMaxDataSize gets called with ginEntryInsert: * frame #0: 0x00010a49d72e

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-09 Thread Alexander Korotkov
Hi! On Wed, Jul 8, 2015 at 10:27 PM, Heikki Linnakangas hlinn...@iki.fi wrote: In dataPlaceToPageLeaf-function: if (append) { /* * Even when appending, trying to append more items than will fit is * not completely free,

Re: [HACKERS] Fillfactor for GIN indexes

2015-07-08 Thread Heikki Linnakangas
In dataPlaceToPageLeaf-function: if (append) { /* * Even when appending, trying to append more items than will fit is * not completely free, because we will merge the new items and old * items into an array

Re: [HACKERS] Fillfactor for GIN indexes

2015-02-25 Thread Tomas Vondra
On 25.2.2015 10:20, Alexander Korotkov wrote: On Tue, Feb 24, 2015 at 5:15 PM, Tomas Vondra tomas.von...@2ndquadrant.com mailto:tomas.von...@2ndquadrant.com wrote: I hoped lowering the fillfactor will improve this, but fillfactor=75 had pretty much no effect in this case. Is that expected

Re: [HACKERS] Fillfactor for GIN indexes

2015-02-25 Thread Alexander Korotkov
On Tue, Feb 24, 2015 at 5:15 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: I hoped lowering the fillfactor will improve this, but fillfactor=75 had pretty much no effect in this case. Is that expected for this kind of workload? I see the previous discussion talked about random updates,

Re: [HACKERS] Fillfactor for GIN indexes

2015-02-24 Thread Tomas Vondra
Hi, I've been wondering whether this might improve behavior with one of my workloads, suffering by GIN bloat - the same one I used to test GIN fastscan, for example. It's a batch process that loads a mailing list archive into a table with a GIN index on message body, by doing something like

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Michael Paquier
On Tue, Jan 20, 2015 at 12:06 AM, Cédric Villemain ced...@2ndquadrant.com wrote: Michael, I first didn't understood how GiN can benefits from the patch...thus my question. There were no noise for me, and I learn some more about GiN. So I thank you for your work! Kicking back the ball, I

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Robert Haas
On Sat, Jan 17, 2015 at 4:49 AM, Alexander Korotkov aekorot...@gmail.com wrote: I already wrote quite detailed explanation of subject. Let mel try to explain in shortly. GIN is two level nested btree. Thus, GIN would have absolutely same benefits from fillfactor as btree. Lack of tests showing

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Alexander Korotkov
On Mon, Jan 19, 2015 at 5:46 PM, Cédric Villemain ced...@2ndquadrant.com wrote: Le lundi 19 janvier 2015 08:24:08 Robert Haas a écrit : On Sat, Jan 17, 2015 at 4:49 AM, Alexander Korotkov aekorot...@gmail.com wrote: I already wrote quite detailed explanation of subject. Let mel try

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Cédric Villemain
Le lundi 19 janvier 2015 08:24:08 Robert Haas a écrit : On Sat, Jan 17, 2015 at 4:49 AM, Alexander Korotkov aekorot...@gmail.com wrote: I already wrote quite detailed explanation of subject. Let mel try to explain in shortly. GIN is two level nested btree. Thus, GIN would have

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-19 Thread Cédric Villemain
Le samedi 17 janvier 2015 08:23:44 Michael Paquier a écrit : On Sat, Jan 17, 2015 at 2:40 AM, Robert Haas robertmh...@gmail.com wrote: There's only value in adding a fillfactor parameter to GIN indexes if it improves performance. There are no benchmarks showing it does. So, why are we

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-17 Thread Alexander Korotkov
On Fri, Jan 16, 2015 at 8:40 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jan 15, 2015 at 7:06 AM, Michael Paquier michael.paqu...@gmail.com wrote: Alexander Korotkov wrote: I'm not sure. On the one hand it's unclear why fillfactor should be different from 9.4. On the other hand

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-17 Thread Alexander Korotkov
On Sat, Jan 17, 2015 at 12:49 PM, Alexander Korotkov aekorot...@gmail.com wrote: BTW, previous version of patch contained some bugs. Revised version is attached. Ooops, it's here. -- With best regards, Alexander Korotkov. gin_fillfactor_5.patch Description: Binary data -- Sent via

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-16 Thread Robert Haas
On Thu, Jan 15, 2015 at 7:06 AM, Michael Paquier michael.paqu...@gmail.com wrote: Alexander Korotkov wrote: I'm not sure. On the one hand it's unclear why fillfactor should be different from 9.4. On the other hand it's unclear why it should be different from btree. I propose marking this

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-16 Thread Michael Paquier
On Sat, Jan 17, 2015 at 2:40 AM, Robert Haas robertmh...@gmail.com wrote: There's only value in adding a fillfactor parameter to GIN indexes if it improves performance. There are no benchmarks showing it does. So, why are we still talking about this? Indeed. There is no such benchmark as of

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-15 Thread Alexander Korotkov
On Thu, Jan 15, 2015 at 10:19 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jan 8, 2015 at 2:03 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jan 8, 2015 at 6:31 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Wed, Jan 7, 2015 at 4:11 PM, Michael

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-15 Thread Michael Paquier
Alexander Korotkov wrote: I'm not sure. On the one hand it's unclear why fillfactor should be different from 9.4. On the other hand it's unclear why it should be different from btree. I propose marking this ready for committer. So, committer can make a final decision. OK let's do so then. My

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-14 Thread Michael Paquier
On Thu, Jan 8, 2015 at 2:03 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Thu, Jan 8, 2015 at 6:31 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier michael.paqu...@gmail.com I am attaching an updated patch, with the default

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Michael Paquier
On Wed, Dec 3, 2014 at 2:37 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Nov 28, 2014 at 4:27 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier michael.paqu...@gmail.com wrote: Please find attached a simple patch adding fillfactor as

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Michael Paquier
On Thu, Jan 8, 2015 at 6:31 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier michael.paqu...@gmail.com I am attaching an updated patch, with the default fillfactor value at 75%, and with the page split code using the fillfactor rate. Thoughts?

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Alexander Korotkov
On Wed, Jan 7, 2015 at 4:11 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Dec 3, 2014 at 2:37 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Nov 28, 2014 at 4:27 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier

Re: [HACKERS] Fillfactor for GIN indexes

2015-01-07 Thread Alexander Korotkov
On Thu, Jan 8, 2015 at 12:31 AM, Alexander Korotkov aekorot...@gmail.com wrote: Rewritten version of patch is attached. I made following changes: 1) I removed fillfactor handling from entry tree. Because in this case fillfactor parameter would be only upper bound for actual page fullness.

Re: [HACKERS] Fillfactor for GIN indexes

2014-12-02 Thread Robert Haas
On Fri, Nov 28, 2014 at 4:27 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier michael.paqu...@gmail.com wrote: Please find attached a simple patch adding fillfactor as storage parameter for GIN indexes. The default value is the same as the one

Re: [HACKERS] Fillfactor for GIN indexes

2014-11-28 Thread Alexander Korotkov
Hi! On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier michael.paqu...@gmail.com wrote: Please find attached a simple patch adding fillfactor as storage parameter for GIN indexes. The default value is the same as the one currently aka 100 to have the pages completely packed when a GIN index is

Re: [HACKERS] Fillfactor for GIN indexes

2014-11-27 Thread Michael Paquier
On Fri, Nov 21, 2014 at 2:12 PM, Michael Paquier michael.paqu...@gmail.com wrote: I am adding that to the commit fest of December. Here are updated patches. Alvaro notified me about an inconsistent comment. -- Michael From eda0730d991f8b4dfbacc4d7a953ec5bff8b2ffe Mon Sep 17 00:00:00 2001 From:

Re: [HACKERS] Fillfactor for GIN indexes

2014-11-27 Thread Cédric Villemain
Le jeudi 27 novembre 2014 23:33:11 Michael Paquier a écrit : On Fri, Nov 21, 2014 at 2:12 PM, Michael Paquier michael.paqu...@gmail.com wrote: I am adding that to the commit fest of December. Here are updated patches. Alvaro notified me about an inconsistent comment. what are the

[HACKERS] Fillfactor for GIN indexes

2014-11-20 Thread Michael Paquier
Hi all, Please find attached a simple patch adding fillfactor as storage parameter for GIN indexes. The default value is the same as the one currently aka 100 to have the pages completely packed when a GIN index is created. Note that to have this feature correctly working, the fix I sent