Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-11 Thread Claudio Freire
On Thu, Aug 11, 2016 at 11:07 AM, Jim Nasby wrote: > On 8/10/16 12:48 PM, Claudio Freire wrote: >> >> On Tue, Aug 9, 2016 at 11:39 PM, Jim Nasby >> wrote: >>> >>> On 8/9/16 6:44 PM, Claudio Freire wrote: Since we can lookup all

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-11 Thread Jim Nasby
On 8/10/16 12:48 PM, Claudio Freire wrote: On Tue, Aug 9, 2016 at 11:39 PM, Jim Nasby wrote: On 8/9/16 6:44 PM, Claudio Freire wrote: Since we can lookup all occurrences of k1=a index=0 and k2=a index=0, and in fact we probably did so already as part of the update

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Amit Kapila
On Mon, Aug 8, 2016 at 9:56 PM, Bruce Momjian wrote: > On Mon, Aug 8, 2016 at 06:34:46PM +0530, Amit Kapila wrote: >> I think here expensive part would be recheck for the cases where the >> index value is changed to a different value (value which doesn't exist >> in WARM

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Claudio Freire
On Wed, Aug 10, 2016 at 4:37 PM, Simon Riggs wrote: > On 10 August 2016 at 03:45, Pavan Deolasee wrote: >> >> >> On Tue, Aug 9, 2016 at 12:06 AM, Claudio Freire >> wrote: >>> >>> On Mon, Aug 8, 2016 at 2:52 PM, Pavan

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Simon Riggs
On 10 August 2016 at 03:45, Pavan Deolasee wrote: > > > On Tue, Aug 9, 2016 at 12:06 AM, Claudio Freire > wrote: >> >> On Mon, Aug 8, 2016 at 2:52 PM, Pavan Deolasee >> wrote: >> >> > Some heuristics and limits on

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-10 Thread Claudio Freire
On Tue, Aug 9, 2016 at 11:39 PM, Jim Nasby wrote: > On 8/9/16 6:44 PM, Claudio Freire wrote: >> >> Since we can lookup all occurrences of k1=a index=0 and k2=a index=0, >> and in fact we probably did so already as part of the update logic > > > That's a change from what

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-09 Thread Pavan Deolasee
On Tue, Aug 9, 2016 at 12:06 AM, Claudio Freire wrote: > On Mon, Aug 8, 2016 at 2:52 PM, Pavan Deolasee > wrote: > > > Some heuristics and limits on amount of work done to detect duplicate > index > > entries will help too. > > I think I prefer

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-09 Thread Jim Nasby
On 8/9/16 6:44 PM, Claudio Freire wrote: Since we can lookup all occurrences of k1=a index=0 and k2=a index=0, and in fact we probably did so already as part of the update logic That's a change from what currently happens, right? The reason I think that's important is that dropping the

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-09 Thread Jim Nasby
On 8/9/16 6:46 PM, Claudio Freire wrote: On Tue, Aug 9, 2016 at 8:44 PM, Claudio Freire wrote: index 0 1 2 3 4 k1 .. a a a k2 .. b a a i1 ^ i2 ^ ^ lp u u r3 hot* Sorry, that

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-09 Thread Bruce Momjian
On Tue, Aug 9, 2016 at 06:19:57PM -0500, Jim Nasby wrote: > On 8/8/16 3:19 PM, Bruce Momjian wrote: > >>What will help, and something I haven't yet applied any thoughts, is when we > >>> can turn WARM chains back to HOT by removing stale index entries. > >I can't see how we can ever do that

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-09 Thread Claudio Freire
On Tue, Aug 9, 2016 at 8:44 PM, Claudio Freire wrote: > index 0 1 2 3 4 > k1 .. a a a > k2 .. b a a > i1 ^ > i2 ^ ^ > lp u u r3 > hot* Sorry, that should read: index 0 1 2 3

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-09 Thread Claudio Freire
On Tue, Aug 9, 2016 at 8:19 PM, Jim Nasby wrote: > On 8/8/16 3:19 PM, Bruce Momjian wrote: >>> >>> What will help, and something I haven't yet applied any thoughts, is when >>> we >>> > can turn WARM chains back to HOT by removing stale index entries. >> >> I can't see

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-09 Thread Jim Nasby
On 8/8/16 3:19 PM, Bruce Momjian wrote: What will help, and something I haven't yet applied any thoughts, is when we > can turn WARM chains back to HOT by removing stale index entries. I can't see how we can ever do that because we have multiple indexes pointing to the chain, and keys that

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-08 Thread Claudio Freire
On Mon, Aug 8, 2016 at 5:24 PM, Bruce Momjian wrote: > On Mon, Aug 8, 2016 at 03:36:12PM -0300, Claudio Freire wrote: >> I think I prefer a more thorough approach. >> >> Increment/decrement may get very complicated with custom opclasses, >> for instance. A column-change bitmap

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-08 Thread Bruce Momjian
On Mon, Aug 8, 2016 at 03:36:12PM -0300, Claudio Freire wrote: > I think I prefer a more thorough approach. > > Increment/decrement may get very complicated with custom opclasses, > for instance. A column-change bitmap won't know how to handle > funcional indexes, etc. Hot does HOT handle them?

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-08 Thread Bruce Momjian
On Mon, Aug 8, 2016 at 11:22:49PM +0530, Pavan Deolasee wrote: > What I am currently trying to do is to reuse at least the BlockNumber field in > t_ctid. For HOT/WARM chains, that field is really unused (except the last > tuple > when regular update needs to store block number of the new block).

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-08 Thread Claudio Freire
On Mon, Aug 8, 2016 at 2:52 PM, Pavan Deolasee wrote: >> >> My guess is we would need one bit to mark a WARM chain, and perhaps >> reuse obsolete pre-9.0 HEAP_MOVED_OFF to indicate increment-only or >> decrement-only. > > > I am not convinced that the checking for

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-08 Thread Pavan Deolasee
On Mon, Aug 8, 2016 at 11:08 PM, Bruce Momjian wrote: > On Sun, Aug 7, 2016 at 12:55:01PM -0400, Bruce Momjian wrote: > > On Sun, Aug 7, 2016 at 10:49:45AM -0400, Bruce Momjian wrote: > > > OK, crazy idea time --- what if we only do WARM chain additions when > all > > >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-08 Thread Bruce Momjian
On Sun, Aug 7, 2016 at 12:55:01PM -0400, Bruce Momjian wrote: > On Sun, Aug 7, 2016 at 10:49:45AM -0400, Bruce Momjian wrote: > > OK, crazy idea time --- what if we only do WARM chain additions when all > > indexed values are increasing (with NULLs higher than all values)? (If > > a key is

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-08 Thread Bruce Momjian
On Mon, Aug 8, 2016 at 06:34:46PM +0530, Amit Kapila wrote: > I think here expensive part would be recheck for the cases where the > index value is changed to a different value (value which doesn't exist > in WARM chain). You anyway have to add the new entry (key,TID) in > index, but each time

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-08 Thread Amit Kapila
On Fri, Aug 5, 2016 at 9:57 AM, Pavan Deolasee wrote: > > > On Fri, Aug 5, 2016 at 8:23 AM, Claudio Freire > wrote: >> >> On Thu, Aug 4, 2016 at 11:15 PM, Bruce Momjian wrote: >> >> > >> > OK, that's a lot of text, and I am

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-07 Thread Bruce Momjian
On Sun, Aug 7, 2016 at 10:49:45AM -0400, Bruce Momjian wrote: > OK, crazy idea time --- what if we only do WARM chain additions when all > indexed values are increasing (with NULLs higher than all values)? (If > a key is always-increasing, it can't match a previous value in the > chain.) That

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-07 Thread Bruce Momjian
On Sat, Aug 6, 2016 at 10:51:21AM -0400, Bruce Momjian wrote: > > If we need to find an efficient way to convert WARM chains back to HOT, > > which > > will happen soon when the old index tuple retires, the system can attain a > > stable state, not for all but many use cases. > > I don't see

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-06 Thread Bruce Momjian
On Sat, Aug 6, 2016 at 10:08:47AM +0530, Pavan Deolasee wrote: > So, we are only checking the index if the WARM chain was pruned, and we > can bail out if there is only one index changed.  This is looking more > doable. > > > The duplicate tuples problem that we are focusing on,

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Pavan Deolasee
On Sat, Aug 6, 2016 at 8:34 AM, Bruce Momjian wrote: > On Fri, Aug 5, 2016 at 09:40:35PM -0400, Bruce Momjian wrote: > > So to summarize again: > > > > o chains start out as HOT > > o they become WARM when some indexes change and others don't > > o for multiple index

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Bruce Momjian
On Fri, Aug 5, 2016 at 09:40:35PM -0400, Bruce Momjian wrote: > So to summarize again: > > o chains start out as HOT > o they become WARM when some indexes change and others don't > o for multiple index changes, we have to check all indexes >for key/ctid matches > o for single index

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Bruce Momjian
On Fri, Aug 5, 2016 at 09:02:39PM -0400, Bruce Momjian wrote: > Yes, it seems we either find the entry fast and avoid the index > addition, or don't find it quickly and use a non-HOT, non-WARM update. > The problem is that you have to do this for multiple indexes, and if you > quickly find you

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Bruce Momjian
On Fri, Aug 5, 2016 at 09:21:49PM -0300, Claudio Freire wrote: > On Fri, Aug 5, 2016 at 9:07 PM, Bruce Momjian wrote: > > On Fri, Aug 5, 2016 at 07:51:05PM -0300, Claudio Freire wrote: > >> > This does create more HOT chains where the root ctid cannot be removed, > >> > but it

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Claudio Freire
On Fri, Aug 5, 2016 at 9:21 PM, Claudio Freire wrote: > On Fri, Aug 5, 2016 at 9:07 PM, Bruce Momjian wrote: >> On Fri, Aug 5, 2016 at 07:51:05PM -0300, Claudio Freire wrote: >>> > This does create more HOT chains where the root ctid cannot be removed,

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Claudio Freire
On Fri, Aug 5, 2016 at 9:07 PM, Bruce Momjian wrote: > On Fri, Aug 5, 2016 at 07:51:05PM -0300, Claudio Freire wrote: >> > This does create more HOT chains where the root ctid cannot be removed, >> > but it does avoid the index key/ctid check because any entry in the HOT >> >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Bruce Momjian
On Fri, Aug 5, 2016 at 07:51:05PM -0300, Claudio Freire wrote: > > This does create more HOT chains where the root ctid cannot be removed, > > but it does avoid the index key/ctid check because any entry in the HOT > > chain has identical keys. What this would not handle is when an entire > >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Claudio Freire
On Fri, Aug 5, 2016 at 4:26 PM, Bruce Momjian wrote: > On Fri, Aug 5, 2016 at 11:30:26PM +0530, Pavan Deolasee wrote: >> On Fri, Aug 5, 2016 at 11:26 PM, Bruce Momjian wrote: >> >> On Fri, Aug 5, 2016 at 02:43:37PM -0300, Claudio Freire wrote: >> >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Bruce Momjian
On Fri, Aug 5, 2016 at 03:26:15PM -0400, Bruce Momjian wrote: > > I just don't know how would you do that without delaying/not-doing HOT chain > > prune. As soon as root and intermediate HOT tuples are gone, all > > information is > > lost. You may delay that, but that will defeat the whole

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Bruce Momjian
On Fri, Aug 5, 2016 at 11:30:26PM +0530, Pavan Deolasee wrote: > On Fri, Aug 5, 2016 at 11:26 PM, Bruce Momjian wrote: > > On Fri, Aug  5, 2016 at 02:43:37PM -0300, Claudio Freire wrote: > > But doing the WARM chain backtracking and checking for previous > >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Pavan Deolasee
On Fri, Aug 5, 2016 at 11:26 PM, Bruce Momjian wrote: > On Fri, Aug 5, 2016 at 02:43:37PM -0300, Claudio Freire wrote: > > But doing the WARM chain backtracking and checking for previous > > versions with appropriate keys should be enough to support this use > > case too, it

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Bruce Momjian
On Fri, Aug 5, 2016 at 02:43:37PM -0300, Claudio Freire wrote: > But doing the WARM chain backtracking and checking for previous > versions with appropriate keys should be enough to support this use > case too, it just needs to be well optimized to avoid seriously > impacting performance on the

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Claudio Freire
On Fri, Aug 5, 2016 at 2:26 PM, Pavan Deolasee wrote: > On Fri, Aug 5, 2016 at 8:55 PM, Claudio Freire > wrote: >> >> On Fri, Aug 5, 2016 at 1:27 AM, Pavan Deolasee >> wrote: >> > >> > >> > I don't see why it is hard.

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Pavan Deolasee
On Fri, Aug 5, 2016 at 8:55 PM, Claudio Freire wrote: > On Fri, Aug 5, 2016 at 1:27 AM, Pavan Deolasee > wrote: > > > > > > I don't see why it is hard. Trying to find the index entry for the old > > update row seems odd, and updating an index

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Bruce Momjian
On Fri, Aug 5, 2016 at 12:25:39PM -0300, Claudio Freire wrote: > > 2. The existence of index pointers to intermediate tuples will lead to index > > bloat. This is the same problem that we found in Simon's original proposal > > (discussed internally). None of the intermediate line pointers can be

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Claudio Freire
On Fri, Aug 5, 2016 at 1:27 AM, Pavan Deolasee wrote: > > > On Fri, Aug 5, 2016 at 8:23 AM, Claudio Freire > wrote: >> >> On Thu, Aug 4, 2016 at 11:15 PM, Bruce Momjian wrote: >> >> > >> > OK, that's a lot of text, and I am

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-05 Thread Bruce Momjian
On Fri, Aug 5, 2016 at 09:57:19AM +0530, Pavan Deolasee wrote: > Hmm. That seems like a real problem. The only way to address that is to ensure > that for a given WARM chain and given index key, there must exists only a > single index entry. There can and will be multiple entries if the index key

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
On Fri, Aug 5, 2016 at 8:23 AM, Claudio Freire wrote: > On Thu, Aug 4, 2016 at 11:15 PM, Bruce Momjian wrote: > > > > > OK, that's a lot of text, and I am confused. Please tell me the > > advantage of having an index point to a string of HOT chains,

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Fri, Aug 5, 2016 at 12:44 AM, Bruce Momjian wrote: >> UPDATE t SET dat = dat + 1, id = 3, someid = 3 where id = 2; > > This is ends the WARM chain, and creates new index entries because all > indexes are changed. > >> UPDATE t SET dat = dat + 1, id = 1, someid = 2 where id =

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 11:53:05PM -0300, Claudio Freire wrote: > The point is avoiding duplicate rows in the output of index scans. > > I don't think you can avoid it simply by applying index condition > rechecks as the original proposal implies, in this case: > > CREATE TABLE t (id integer not

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 11:15 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 09:53:31PM -0300, Claudio Freire wrote: >> It's the other way around. A WARM chain has to span whole HOT chains. >> The WARM chain always begins in the head of a HOT chain (or non-HOT >> tuple of

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:53:31PM -0300, Claudio Freire wrote: > It's the other way around. A WARM chain has to span whole HOT chains. > The WARM chain always begins in the head of a HOT chain (or non-HOT > tuple of course), and cannot end before the HOT chain ends. > > That is, all within a

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 7:59 AM, Pavan Deolasee wrote: > So marking the index would require us to mark both old and new index tuples > as requiring re-check. That requires an additional index scan to locate the > old row and then an additional write to force it to

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
>On Thu, Aug 4, 2016 at 06:21:48PM -0400, Bruce Momjian wrote: > Another approach would be to keep updates on the same page in a single > WARM chain, even if all indexes have changed columns. We will already > have code to walk the HOT chain looking for matches, and at most one > tuple in the

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 7:21 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 03:23:10PM -0300, Claudio Freire wrote: >> INSERT INTO TABLE t (id,val,dat) VALUES (32,'a','blabla'); >> UPDATE t SET dat='blabla2' where id = 32; >> UPDATE t SET dat='blabla3', id=33 where id = 32;

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 10:58:33PM +0530, Pavan Deolasee wrote: > 2. I also think that most often the tuple that will be updated will have its > t_ctid pointing to itself. This sounds more invasive, but can we somehow Agreed! I think it has to. Great idea. > utilise the t_ctid field to store

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 03:33:16PM -0300, Claudio Freire wrote: > On Thu, Aug 4, 2016 at 3:23 PM, Claudio Freire wrote: > >> That is a new HOT chain given our current code, but > >> would the new tuple addition realize it needs to create a new index > >> tuple? The new

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 03:23:10PM -0300, Claudio Freire wrote: > INSERT INTO TABLE t (id,val,dat) VALUES (32,'a','blabla'); > UPDATE t SET dat='blabla2' where id = 32; > UPDATE t SET dat='blabla3', id=33 where id = 32; > UPDATE t SET val='b' where id = 33; > UPDATE t SET dat='blabla4' where id =

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 3:23 PM, Claudio Freire wrote: >> That is a new HOT chain given our current code, but >> would the new tuple addition realize it needs to create a new index >> tuple? The new tuple code needs to check the index's root HOT tid for a >> non-match. > >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 3:07 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 02:35:32PM -0300, Claudio Freire wrote: >> > Vacuum will need to be taught not to break the invariant, but I >> > believe this is viable and efficient. >> >> >> And I didn't mention the invariant. >>

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 02:35:32PM -0300, Claudio Freire wrote: > > Vacuum will need to be taught not to break the invariant, but I > > believe this is viable and efficient. > > > And I didn't mention the invariant. > > The invariant would be that all WARM chains: > > * contain entire HOT

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 01:05:53PM -0400, Bruce Momjian wrote: > > Approach 2 seems more reasonable and simple.  > > > > There are only 2 bits for lp_flags and all combinations are already used. > > But > > for LP_REDIRECT root line pointer, we could use the lp_len field to store > > this > >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 11:04:49PM +0530, Pavan Deolasee wrote: > If the root tuple still exists, we store the WARM flag (or > HEAP_RECHECK_REQUIRED as used in the original post) in the tuple header > itself. > When the root tuple becomes dead and HOT prune decides to replace it with a >

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
On Thu, Aug 4, 2016 at 10:49 PM, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 06:16:02PM +0100, Simon Riggs wrote: > > On 4 August 2016 at 18:05, Bruce Momjian wrote: > > > > >> Approach 2 seems more reasonable and simple. > > >> > > >> There are only 2

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 2:14 PM, Claudio Freire wrote: > To fix this, what I was pondering, and I believe it would be viable, > is to teach heap_hot_search_buffer about the WARM invariant. It will > make correctness heavily depend on the invariant being true, which > means

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Pavan Deolasee
On Thu, Aug 4, 2016 at 10:41 PM, Simon Riggs wrote: > On 4 August 2016 at 17:31, Andres Freund wrote: > > Hi, > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > >> Indexes whose values do not change do not require new index pointers. > Only

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 06:18:47PM +0100, Simon Riggs wrote: > > Have you considered what it'd take to allow index pointers to allow to > > point to "intermediate" root tuples? I.e. have some indexes point into > > the middle of an update-chain, whereas others still point to the > > beginning?

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
On 2016-08-04 18:18:47 +0100, Simon Riggs wrote: > On 4 August 2016 at 18:17, Andres Freund wrote: > > On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: > > I'm less afraid of the fiddlyness of finding the root tuple, than the > > cost. It's not cheap to walk through,

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 06:16:02PM +0100, Simon Riggs wrote: > On 4 August 2016 at 18:05, Bruce Momjian wrote: > > >> Approach 2 seems more reasonable and simple. > >> > >> There are only 2 bits for lp_flags and all combinations are already used. > >> But > >> for LP_REDIRECT

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:17, Andres Freund wrote: > On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: >> On 4 August 2016 at 17:31, Andres Freund wrote: >> > Hi, >> > >> > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: >> >> Indexes whose values do not

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
On 2016-08-04 18:11:17 +0100, Simon Riggs wrote: > On 4 August 2016 at 17:31, Andres Freund wrote: > > Hi, > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > >> Indexes whose values do not change do not require new index pointers. Only > >> the index whose key is

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:58:29AM -0700, Andres Freund wrote: > On 2016-08-04 12:55:25 -0400, Bruce Momjian wrote: > > On Thu, Aug 4, 2016 at 09:31:18AM -0700, Andres Freund wrote: > > > Hi, > > > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > > > > Indexes whose values do not

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:05, Bruce Momjian wrote: >> Approach 2 seems more reasonable and simple. >> >> There are only 2 bits for lp_flags and all combinations are already used. But >> for LP_REDIRECT root line pointer, we could use the lp_len field to store >> this >> special

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 7:59 AM, Pavan Deolasee wrote: > We don’t want to force a recheck for every index fetch because that will > slow everything down. So we would like a simple and efficient way of knowing > about the existence of a WARM tuple in a chain and do a

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:58:29AM -0700, Andres Freund wrote: > On 2016-08-04 12:55:25 -0400, Bruce Momjian wrote: > > On Thu, Aug 4, 2016 at 09:31:18AM -0700, Andres Freund wrote: > > > Hi, > > > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > > > > Indexes whose values do not

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 17:31, Andres Freund wrote: > Hi, > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: >> Indexes whose values do not change do not require new index pointers. Only >> the index whose key is being changed will need a new index entry. The new >> index

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 04:29:09PM +0530, Pavan Deolasee wrote: > Write Amplification Reduction Method (WARM) > > > A few years back, we developed HOT to address the problem associated with MVCC > and frequent updates and it has served us very well. But in the

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
On 2016-08-04 12:55:25 -0400, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 09:31:18AM -0700, Andres Freund wrote: > > Hi, > > > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > > > Indexes whose values do not change do not require new index pointers. Only > > > the index whose key is

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 09:31:18AM -0700, Andres Freund wrote: > Hi, > > On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > > Indexes whose values do not change do not require new index pointers. Only > > the index whose key is being changed will need a new index entry. The new > > index entry

Re: [HACKERS] Heap WARM Tuples - Design Draft

2016-08-04 Thread Andres Freund
Hi, On 2016-08-04 16:29:09 +0530, Pavan Deolasee wrote: > Indexes whose values do not change do not require new index pointers. Only > the index whose key is being changed will need a new index entry. The new > index entry will be set to the CTID of the root line pointer. That seems to require