Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 18:27, Bruce Momjian wrote: >> > Also, why not use this bitmap for all indexes, not just update chains? >> >> I don't understand where you get this update chains thing from. >> >> The bitmap can apply to multiple tuples on one page, which is described. > > I am asking if we co

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Claudio Freire
On Thu, Aug 4, 2016 at 1:58 PM, Simon Riggs wrote: > On 3 August 2016 at 20:37, Claudio Freire wrote: >> On Wed, Aug 3, 2016 at 4:20 AM, Simon Riggs wrote: >>> == IndexScan == >>> >>> Note that the executor code for IndexScan appears identical between >>> the two optimizations. The difference be

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 06:03:34PM +0100, Simon Riggs wrote: > On 4 August 2016 at 02:13, Bruce Momjian wrote: > > > How do plan to clear the bitmask so it, over time, doesn't end up being > > all-set? > > I don't have a plan, though thoughts welcome. > > Similar situation that our current inde

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
On 4 August 2016 at 02:13, Bruce Momjian wrote: > How do plan to clear the bitmask so it, over time, doesn't end up being > all-set? I don't have a plan, though thoughts welcome. Similar situation that our current indexes don't recover from bloat, a situation made worse by non-HOT updates. So,

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-04 Thread Simon Riggs
On 3 August 2016 at 20:37, Claudio Freire wrote: > On Wed, Aug 3, 2016 at 4:20 AM, Simon Riggs wrote: >> == IndexScan == >> >> Note that the executor code for IndexScan appears identical between >> the two optimizations. The difference between duplicate and range LITE >> tuples is needed only at

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Claudio Freire
On Wed, Aug 3, 2016 at 4:37 PM, Claudio Freire wrote: > On Wed, Aug 3, 2016 at 4:20 AM, Simon Riggs wrote: >> == IndexScan == >> >> Note that the executor code for IndexScan appears identical between >> the two optimizations. The difference between duplicate and range LITE >> tuples is needed onl

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Bruce Momjian
On Wed, Aug 3, 2016 at 08:34:02PM -0400, Bruce Momjian wrote: > On Thu, Aug 4, 2016 at 01:16:20AM +0100, Simon Riggs wrote: > > > Would you only add a LITE index entry when there isn't an > > > existing index entry for the same values and heap page? That seems > > > quite complicated. > > > > T

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 01:16:20AM +0100, Simon Riggs wrote: > > Would you only add a LITE index entry when there isn't an > > existing index entry for the same values and heap page? That seems > > quite complicated. > > The insertion algorithm is described. Doesn't seem complicated to me. Ah, I

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Bruce Momjian
On Thu, Aug 4, 2016 at 01:16:20AM +0100, Simon Riggs wrote: > On 4 August 2016 at 00:56, Bruce Momjian wrote: > > On Wed, Aug 3, 2016 at 07:28:52PM -0400, Bruce Momjian wrote: > >> With LITE, you can avoid the creation of duplicate-value index entries > >> for indexes without changed column valu

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Simon Riggs
On 4 August 2016 at 00:56, Bruce Momjian wrote: > On Wed, Aug 3, 2016 at 07:28:52PM -0400, Bruce Momjian wrote: >> With LITE, you can avoid the creation of duplicate-value index entries >> for indexes without changed column values by using a bitmap in place of >> the tid item number (16 bits). I

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Bruce Momjian
On Wed, Aug 3, 2016 at 07:28:52PM -0400, Bruce Momjian wrote: > With LITE, you can avoid the creation of duplicate-value index entries > for indexes without changed column values by using a bitmap in place of > the tid item number (16 bits). It can't remove dead tids. How would you handle the ca

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Bruce Momjian
On Wed, Aug 3, 2016 at 08:20:49AM +0100, Simon Riggs wrote: > == Update Duplicate Removal == > > We want an optimization that reduces the effects of multiple UPDATEs > on the same block that have duplicate values caused because another > index column has been updated and a non-HOT index insert ha

Re: [HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Claudio Freire
On Wed, Aug 3, 2016 at 4:20 AM, Simon Riggs wrote: > == IndexScan == > > Note that the executor code for IndexScan appears identical between > the two optimizations. The difference between duplicate and range LITE > tuples is needed only at INSERT time (or UPDATE indexed column to a > new value).

[HACKERS] Lossy Index Tuple Enhancement (LITE)

2016-08-03 Thread Simon Riggs
This proposal introduces the idea of "lossy index tuples", an idea that provides two important optimizations for btrees and potentially other index types stimulated by recent blog complaints. The two optimizations are Clustered Indexes (mostly for INSERTs) and Update Duplicate Removal (for non-HOT