Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Amit Kapila
On Sun, Jun 7, 2015 at 3:02 PM, Simon Riggs si...@2ndquadrant.com wrote: On 23 April 2015 at 17:24, Andres Freund and...@anarazel.de wrote: It's hard to see how to save space there without reference to a specific use case. I see different solutions depending upon whether we assume a low

Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Simon Riggs
On 23 April 2015 at 17:24, Andres Freund and...@anarazel.de wrote: Split into a new thread, the other one is already growing fast enough. This discussion started at http://archives.postgresql.org/message-id/55391469.5010506%40iki.fi On April 23, 2015 6:48:57 PM GMT+03:00, Heikki Linnakangas

Re: [HACKERS] Reducing tuple overhead

2015-06-07 Thread Peter Geoghegan
On Thu, Apr 30, 2015 at 6:54 AM, Robert Haas robertmh...@gmail.com wrote: The other, related problem is that the ordering operator might start to return different results than it did at index creation time. For example, consider a btree index built on a text column. Now consider 'yum

Re: [HACKERS] Reducing tuple overhead

2015-06-06 Thread Amit Kapila
On Thu, Apr 23, 2015 at 9:54 PM, Andres Freund and...@anarazel.de wrote: Split into a new thread, the other one is already growing fast enough. This discussion started at http://archives.postgresql.org/message-id/55391469.5010506%40iki.fi On April 23, 2015 6:48:57 PM GMT+03:00, Heikki

Re: [HACKERS] Reducing tuple overhead

2015-05-01 Thread Jim Nasby
On 4/30/15 7:37 AM, Robert Haas wrote: On Thu, Apr 30, 2015 at 8:05 AM, Simon Riggs si...@2ndquadrant.com wrote: A much better idea is to work out how to avoid index bloat at cause. If we are running an UPDATE and we cannot get a cleanup lock, we give up and do a non-HOT update, causing the

Re: [HACKERS] Reducing tuple overhead

2015-05-01 Thread Amit Kapila
On Thu, Apr 30, 2015 at 5:35 PM, Simon Riggs si...@2ndquadrant.com wrote: On 25 April 2015 at 01:12, Amit Kapila amit.kapil...@gmail.com wrote: On Sat, Apr 25, 2015 at 1:58 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/23/15 10:40 PM, Amit Kapila wrote: I agree with you and what I

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Simon Riggs
On 25 April 2015 at 01:12, Amit Kapila amit.kapil...@gmail.com wrote: On Sat, Apr 25, 2015 at 1:58 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/23/15 10:40 PM, Amit Kapila wrote: I agree with you and what I think one of the major reasons of bloat is that Index segment doesn't

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Robert Haas
On Thu, Apr 30, 2015 at 8:05 AM, Simon Riggs si...@2ndquadrant.com wrote: A much better idea is to work out how to avoid index bloat at cause. If we are running an UPDATE and we cannot get a cleanup lock, we give up and do a non-HOT update, causing the index to bloat. It seems better to wait

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Robert Haas
On Thu, Apr 30, 2015 at 12:31 AM, Amit Kapila amit.kapil...@gmail.com wrote: I think I am missing something here, but when this second evaluation is needed. Basically what I understand from index insertion is that it evaluates the value to be inserted in index before calling nbtree module and

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Amit Kapila
On Thu, Apr 30, 2015 at 5:03 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Apr 30, 2015 at 12:31 AM, Amit Kapila amit.kapil...@gmail.com wrote: I think I am missing something here, but when this second evaluation is needed. Basically what I understand from index insertion is that

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Robert Haas
On Thu, Apr 30, 2015 at 9:46 AM, Amit Kapila amit.kapil...@gmail.com wrote: As the index expression contain table columns and all the functions or operators used in expression must be IMMUTABLE, won't that guarantee to avoid such a situation? The concern is that they might be labeled as

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Amit Kapila
On Thu, Apr 30, 2015 at 7:24 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Apr 30, 2015 at 9:46 AM, Amit Kapila amit.kapil...@gmail.com wrote: As the index expression contain table columns and all the functions or operators used in expression must be IMMUTABLE, won't that guarantee

Re: [HACKERS] Reducing tuple overhead

2015-04-30 Thread Alvaro Herrera
Robert Haas wrote: On Mon, Apr 27, 2015 at 5:01 PM, Jim Nasby jim.na...@bluetreble.com wrote: The problem with just having the value is that if *anything* changes between how you evaluated the value when you created the index tuple and when you evaluate it a second time you'll corrupt your

Re: [HACKERS] Reducing tuple overhead

2015-04-29 Thread Robert Haas
On Mon, Apr 27, 2015 at 5:01 PM, Jim Nasby jim.na...@bluetreble.com wrote: The problem with just having the value is that if *anything* changes between how you evaluated the value when you created the index tuple and when you evaluate it a second time you'll corrupt your index. This is actually

Re: [HACKERS] Reducing tuple overhead

2015-04-29 Thread Jim Nasby
On 4/29/15 12:18 PM, Robert Haas wrote: On Mon, Apr 27, 2015 at 5:01 PM, Jim Nasby jim.na...@bluetreble.com wrote: The problem with just having the value is that if *anything* changes between how you evaluated the value when you created the index tuple and when you evaluate it a second time

Re: [HACKERS] Reducing tuple overhead

2015-04-29 Thread Amit Kapila
On Tue, Apr 28, 2015 at 2:31 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/25/15 12:12 AM, Amit Kapila wrote: ... which isn't possible. You can not go from a heap tuple to an index tuple. We will have the access to index value during delete, so why do you think that we need linkage

Re: [HACKERS] Reducing tuple overhead

2015-04-27 Thread Jim Nasby
On 4/25/15 12:12 AM, Amit Kapila wrote: ... which isn't possible. You can not go from a heap tuple to an index tuple. We will have the access to index value during delete, so why do you think that we need linkage between heap and index tuple to perform Delete operation? I think we need to

Re: [HACKERS] Reducing tuple overhead

2015-04-24 Thread Amit Kapila
On Sat, Apr 25, 2015 at 1:58 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/23/15 10:40 PM, Amit Kapila wrote: I agree with you and what I think one of the major reasons of bloat is that Index segment doesn't have visibility information due to which clearing of Index needs to be tied

Re: [HACKERS] Reducing tuple overhead

2015-04-24 Thread Jim Nasby
On 4/23/15 10:40 PM, Amit Kapila wrote: I agree with you and what I think one of the major reasons of bloat is that Index segment doesn't have visibility information due to which clearing of Index needs to be tied along with heap. Now if we can move transaction information at page level, then

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Joshua D. Drake
On 04/23/2015 09:42 AM, Jim Nasby wrote: On 4/23/15 11:24 AM, Andres Freund wrote: I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we liberally add in many cases... Assuming you're talking about padding between

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Petr Jelinek
On 23/04/15 18:24, Andres Freund wrote: Whether that's feasible complexity wise is debatable, but it's certainly possible. I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we liberally add in many cases... The logical

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Alvaro Herrera
Thanks for posting this. Joshua D. Drake wrote: First of all I should note that I discussed the approach mentioned above to pgsql-hackers and got a very interesting comment from Tom Lane that adding sorting info to Var and TargetEntry nodes was not a very good idea because it'd break stored

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Amit Kapila
On Fri, Apr 24, 2015 at 1:03 AM, Jim Nasby jim.na...@bluetreble.com wrote: On 4/23/15 11:45 AM, Petr Jelinek wrote: On 23/04/15 18:24, Andres Freund wrote: Whether that's feasible complexity wise is debatable, but it's certainly possible. I do wonder what, in realistic cases, is actually

Re: [HACKERS] Reducing tuple overhead

2015-04-23 Thread Jim Nasby
On 4/23/15 11:45 AM, Petr Jelinek wrote: On 23/04/15 18:24, Andres Freund wrote: Whether that's feasible complexity wise is debatable, but it's certainly possible. I do wonder what, in realistic cases, is actually the bigger contributor to the overhead. The tuple header or the padding we