Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-07-08 Thread Josh Berkus
On 06/26/2013 07:05 AM, Jamie Martin wrote: FYI I submitted a slightly modified patch since Amit's measurements that is slightly faster. Yes. My perspective is that this is a worthwhile optimization for a minority, but fairly well-known, use case, provided that it doesn't negatively impact

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-27 Thread Josh Berkus
All, So, is this patch currently depending on performance testing, or not? Like I said, it'll be a chunk of time to set up what I beleive is a realistic performance test, so I don't want to do it if the patch is likely to be bounced for other reasons. -- Josh Berkus PostgreSQL Experts Inc.

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-27 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: So, is this patch currently depending on performance testing, or not? Like I said, it'll be a chunk of time to set up what I beleive is a realistic performance test, so I don't want to do it if the patch is likely to be bounced for other reasons. AFAICS,

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-27 Thread Josh Berkus
On 06/27/2013 11:11 AM, Tom Lane wrote: AFAICS, the threshold question here is whether the patch helps usefully for tables with typical numbers of columns (ie, not 800 ;-)), and I wouldn't expect it to help at all for 50 columns, and probably not measurably below 200. doesn't hurt materially

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-26 Thread Jamie Martin
FYI I submitted a slightly modified patch since Amit's measurements that is slightly faster. On Jun 25, 2013, at 1:26 PM, Amit Kapila amit.kap...@huawei.com wrote: On Monday, June 24, 2013 8:20 PM Tom Lane wrote: Amit Kapila amit.kap...@huawei.com writes: I will summarize the results, and

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-25 Thread Amit Kapila
On Monday, June 24, 2013 8:20 PM Tom Lane wrote: Amit Kapila amit.kap...@huawei.com writes: I will summarize the results, and if most of us feel that they are not good enough, then we can return this patch. Aside from the question of whether there's really any generally-useful

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Amit Kapila
On Monday, June 24, 2013 5:48 PM Andres Freund wrote: On 2013-01-25 09:06:12 +0530, Amit Kapila wrote: On 01/25/2013 03:43 AM, Jameison Martin wrote: there have been a lot of different threads on this patch. i'm going to take a stab at teasing them out, summarizing them, and addressing

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Tom Lane
Amit Kapila amit.kap...@huawei.com writes: I will summarize the results, and if most of us feel that they are not good enough, then we can return this patch. Aside from the question of whether there's really any generally-useful performance improvement from this patch, it strikes me that this

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Simon Riggs
On 24 June 2013 15:49, Tom Lane t...@sss.pgh.pa.us wrote: Amit Kapila amit.kap...@huawei.com writes: I will summarize the results, and if most of us feel that they are not good enough, then we can return this patch. Aside from the question of whether there's really any generally-useful

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Robert Haas
On Mon, Jun 24, 2013 at 4:05 PM, Simon Riggs si...@2ndquadrant.com wrote: I think its rather a shame that the proponents of this patch have tried so hard to push this through without working variations on the theme. Please guys, go away and get creative; rethink the approach so that you can

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Josh Berkus
Simon, I think its rather a shame that the proponents of this patch have tried so hard to push this through without working variations on the theme. Please guys, go away and get creative; rethink the approach so that you can have your lunch without anybody else losing theirs. I would add

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: If there's an actual performance consequence of applying this patch, then I think that's a good reason for rejecting it. But if the best argument we can come up with is that we might someday try to do even more clever things with the tuple's natts

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Simon Riggs
On 24 June 2013 21:50, Tom Lane t...@sss.pgh.pa.us wrote: So, Tom, how's that pluggable storage format coming? :-) Well, actually, it's looking to me like heap_form_tuple will be underneath the API cut, because alternate storage managers will probably have other tuple storage formats ---

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Jameison Martin
i believe the last submission of the patch had no negative performance impact on any of the tested use cases, but you'd have to go back and see the last exchange on that.  i think it was the concern about potentially regressing the codeline in unforeseen ways without a clear benefit to all but

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Josh Berkus
On 06/24/2013 01:50 PM, Tom Lane wrote: The point of what I was suggesting isn't to conserve storage, but to reduce downtime during a schema change. Remember that a rewriting ALTER TABLE locks everyone out of that table for a long time. Right, but I'm worried about the surprise! factor. That

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: On 06/24/2013 01:50 PM, Tom Lane wrote: The point of what I was suggesting isn't to conserve storage, but to reduce downtime during a schema change. Remember that a rewriting ALTER TABLE locks everyone out of that table for a long time. Right, but I'm

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Josh Berkus
On 06/24/2013 02:21 PM, Tom Lane wrote: Right, but I'm worried about the surprise! factor. That is, if we make the first default free by using a magic value, then a SET DEFAULT on that column is going to have some very surprising results as suddenly the whole table needs to get written out

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Robert Haas
On Mon, Jun 24, 2013 at 4:50 PM, Tom Lane t...@sss.pgh.pa.us wrote: The point of what I was suggesting isn't to conserve storage, but to reduce downtime during a schema change. Remember that a rewriting ALTER TABLE locks everyone out of that table for a long time. Sure, I understand. As Josh

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-06-24 Thread Noah Misch
On Mon, Jun 24, 2013 at 01:32:41PM -0700, Josh Berkus wrote: Do we really have ironclad numbers which show that the patch affects performance negatively? I didn't think the previous performance test was comprehensive; I was planning to develop one myself this week. Not ironclad, no:

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-01-24 Thread Amit Kapila
On Thursday, January 24, 2013 7:42 AM Noah Misch wrote: On Wed, Jan 09, 2013 at 11:22:06AM +, Simon Riggs wrote: On 24 December 2012 16:57, Amit Kapila amit.kap...@huawei.com wrote: Performance: Average of 3 runs of pgbench in tps 9.3devel | with trailing null patch

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-01-24 Thread Tom Lane
Greg Stark st...@mit.edu writes: That said I don't know just how common that usage pattern is. And I'm not sure how many of those people would be able to arrange for the null columns to land at the end of the row. Obviously, they can't arrange that all the time (else their trailing columns are