Re: [HACKERS] WARM and indirect indexes

2017-02-22 Thread Bruce Momjian
On Wed, Jan 11, 2017 at 04:38:10PM -0500, Robert Haas wrote: > More broadly, I don't share Bruce's negativity about indirect indexes. > My estimate of what needs to be done for them to be really useful is - > I think - higher than your estimate of what needs to be done, but I > think the concept is

Re: [HACKERS] WARM and indirect indexes

2017-01-12 Thread Jim Nasby
On 1/11/17 8:09 PM, Pavan Deolasee wrote: The other thing the patch changes is how update-chain is maintained. In order to quickly find the root offset while updating a tuple, we now store the root offset in the t_ctid field of the last tuple in the chain and use a separate bit to mark end-of-the

Re: [HACKERS] WARM and indirect indexes

2017-01-12 Thread Robert Haas
On Wed, Jan 11, 2017 at 11:09 PM, Pavan Deolasee wrote: > I think as a developer of the patch, what I would like to know is what can > we do address concerns raised by you? What kind of tests you would like to > do to get confidence in the patch? Well, off the top of my head, I'd say that there a

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Pavan Deolasee
On Thu, Jan 12, 2017 at 3:08 AM, Robert Haas wrote: > On Tue, Jan 10, 2017 at 2:24 PM, Alvaro Herrera > wrote: > > The big advantage of WARM is that it works automatically, like HOT: the > > user doesn't need to do anything different than today to get the > > benefit. With indirect indexes, the

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Robert Haas
On Tue, Jan 10, 2017 at 2:24 PM, Alvaro Herrera wrote: > The big advantage of WARM is that it works automatically, like HOT: the > user doesn't need to do anything different than today to get the > benefit. With indirect indexes, the user needs to create the index as > indirect explicitely. Howe

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Bruce Momjian
On Wed, Jan 11, 2017 at 12:24:55PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Therefore, I think we need WARM done first, then we can test indirect > > indexes to see if they are a sufficient win to add it for the small > > percentage of users who will use it. > > Agreed -- that's

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Bruce Momjian
On Wed, Jan 11, 2017 at 08:28:28AM -0500, Andrew Dunstan wrote: > > > On 01/10/2017 09:25 PM, Bruce Momjian wrote: > >I am not saying we shouldn't do it, but I am afraid that the complexity > >in figuring out when to use indirect indexes, combined with the number > >of users who will try them, re

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Bruce Momjian
On Wed, Jan 11, 2017 at 12:36:24PM +0530, Pavan Deolasee wrote: > That could also be seen as an advantage to indirect indexes. While I haven't > seen the code, I believe indirect index code will only be hit if someone > actually uses them. So there won't be any overhead for other users who do not >

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Alvaro Herrera
Bruce Momjian wrote: > Therefore, I think we need WARM done first, then we can test indirect > indexes to see if they are a sufficient win to add it for the small > percentage of users who will use it. Agreed -- that's my plan. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ Postg

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Bruce Momjian
On Tue, Jan 10, 2017 at 09:25:05PM -0500, Bruce Momjian wrote: > Thank you for the summary. I think we have to consider two things with > indirect indexes: > > 1. What percentage speedup is the _average_ user going to get? You > have to consider people who will use indirect indexes who get no b

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Craig Ringer
On 11 Jan. 2017 21:29, "Andrew Dunstan" wrote: On 01/10/2017 09:25 PM, Bruce Momjian wrote: > I am not saying we shouldn't do it, but I am afraid that the complexity > in figuring out when to use indirect indexes, combined with the number > of users who will try them, really hurts its inclusio

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Andrew Dunstan
On 01/10/2017 09:25 PM, Bruce Momjian wrote: I am not saying we shouldn't do it, but I am afraid that the complexity in figuring out when to use indirect indexes, combined with the number of users who will try them, really hurts its inclusion. I think you're making this out to be far more c

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Alvaro Herrera
Bruce Momjian wrote: > My point is that anything you add must be weighed against the value it > gives to users who use it, and the percentage of users who will use it. > Against that benefit, you have to look at the cost of exposing that API > to users, code complexity, maintenance, etc. I agree.

Re: [HACKERS] WARM and indirect indexes

2017-01-11 Thread Alvaro Herrera
Pavan Deolasee wrote: > I was going to ask if we could implement indirect indexes as a separate > IndexAM. But I re-read this thread and found that you'd in fact done it > that way in the first version but then discarded it for performance > reasons. Is there a merit in evaluating that path for in

Re: [HACKERS] WARM and indirect indexes

2017-01-10 Thread Pavan Deolasee
On Wed, Jan 11, 2017 at 7:55 AM, Bruce Momjian wrote: > On Tue, Jan 10, 2017 at 04:24:42PM -0300, Alvaro Herrera wrote: > > Two options are on the table to attack the problem of updates causing > > write amplification: WARM and indirect indexes. They are completely > > different approaches but h

Re: [HACKERS] WARM and indirect indexes

2017-01-10 Thread Amit Kapila
On Wed, Jan 11, 2017 at 12:54 AM, Alvaro Herrera wrote: > Two options are on the table to attack the problem of updates causing > write amplification: WARM and indirect indexes. They are completely > different approaches but have overlapping effects on what scenarios are > improved. Here's a rec

Re: [HACKERS] WARM and indirect indexes

2017-01-10 Thread Bruce Momjian
On Tue, Jan 10, 2017 at 11:36:24PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > 1. What percentage speedup is the _average_ user going to get? You > > have to consider people who will use indirect indexes who get no benefit > > or a net slowdown, and users who will get a benefit. >

Re: [HACKERS] WARM and indirect indexes

2017-01-10 Thread Alvaro Herrera
Bruce Momjian wrote: > 1. What percentage speedup is the _average_ user going to get? You > have to consider people who will use indirect indexes who get no benefit > or a net slowdown, and users who will get a benefit. > > 2. What percentage of users are going to use indirect indexes? > > So

Re: [HACKERS] WARM and indirect indexes

2017-01-10 Thread Bruce Momjian
On Tue, Jan 10, 2017 at 04:24:42PM -0300, Alvaro Herrera wrote: > Two options are on the table to attack the problem of updates causing > write amplification: WARM and indirect indexes. They are completely > different approaches but have overlapping effects on what scenarios are > improved. Here'

[HACKERS] WARM and indirect indexes

2017-01-10 Thread Alvaro Herrera
Two options are on the table to attack the problem of updates causing write amplification: WARM and indirect indexes. They are completely different approaches but have overlapping effects on what scenarios are improved. Here's a recap of both features, with the intent that we make a well-consider