On 15 February 2012 15:27, Robert Haas wrote:
>> I am inclined to agree that given that we already use Perl to generate
>> source code like this, it seems natural that we should prefer to do
>> that, if only to avoid paranoia about the inclusion of a dial-a-bloat
>> knob. I am at a disadvantage he
On Wed, Feb 15, 2012 at 8:29 AM, Peter Geoghegan wrote:
> Cool. I agree that we should do this. It doesn't need to be justified
> as a performance optimisation - it makes sense to refactor in this
> way. If that makes things faster, then so much the better.
Well, maybe so, but I think if the perf
On 15 February 2012 06:16, Robert Haas wrote:
> On Fri, Feb 10, 2012 at 10:30 AM, Peter Geoghegan
> wrote:
>> [ new patch ]
>
> I spent quite a bit of time looking at this today - the patch
> specifically, and the issue of making quicksort fast more generally.
> It seemed to me that if we're goi
On Fri, Feb 10, 2012 at 10:30 AM, Peter Geoghegan wrote:
> [ new patch ]
I spent quite a bit of time looking at this today - the patch
specifically, and the issue of making quicksort fast more generally.
It seemed to me that if we're going to have separate copies of the
quicksort code for tuple s
On 9 February 2012 14:51, Robert Haas wrote:
> I'm not sure I entirely follow all this, but I'll look at the code
> once you have it.
I have attached a revision of the patch, with the adjustments already
described. Note that I have not made this support btree tuplesorting
yet, as there is an impe
On 9 February 2012 17:16, Bruce Momjian wrote:
>> Yes, I am. The main reason that the loops exist in those functions
>> (which is the only way that they substantially differ) is because they
>> each have to get the other keys through various ways that characterise
>> the tuple class that they enca
On Thu, Feb 09, 2012 at 03:36:23PM +, Peter Geoghegan wrote:
> On 9 February 2012 14:51, Robert Haas wrote:
> > I'm not sure I entirely follow all this, but I'll look at the code
> > once you have it. Are you saying that all the comparetup_yadda
> > functions are redundant to each other in th
On 9 February 2012 14:51, Robert Haas wrote:
> I'm not sure I entirely follow all this, but I'll look at the code
> once you have it. Are you saying that all the comparetup_yadda
> functions are redundant to each other in the single-key case?
Yes, I am. The main reason that the loops exist in th
On Thu, Feb 09, 2012 at 07:24:49AM -0500, Noah Misch wrote:
> This patch has gotten more than its fair share of attention for bloat, and I
> think that's mostly because there's a dial-a-bloat-level knob sticking out and
> begging to be frobbed.
I already emailed Peter privately stating that he sho
On Thu, Feb 9, 2012 at 9:37 AM, Peter Geoghegan wrote:
> On 9 February 2012 13:50, Robert Haas wrote:
>> I'm also more than slightly concerned that we are losing sight of the
>> forest for the trees. I have heard reports that sorting large amounts
>> of data is many TIMES slower for us than for
On 9 February 2012 13:50, Robert Haas wrote:
> I'm also more than slightly concerned that we are losing sight of the
> forest for the trees. I have heard reports that sorting large amounts
> of data is many TIMES slower for us than for a certain other database
> product. I frankly wonder if this
On Thu, Feb 9, 2012 at 7:24 AM, Noah Misch wrote:
> On Tue, Feb 07, 2012 at 09:38:39PM -0500, Robert Haas wrote:
>> Second, there's a concern about binary bloat: duplicating lots of code
>> with different comparators inlined generates, well, a lot of machine
>> code. Of course, an 0.8% increase i
On Tue, Feb 07, 2012 at 09:38:39PM -0500, Robert Haas wrote:
> Second, there's a concern about binary bloat: duplicating lots of code
> with different comparators inlined generates, well, a lot of machine
> code. Of course, an 0.8% increase in the size of the resulting binary
> is very unlikely to
On 8 February 2012 23:33, Robert Haas wrote:
> On Wed, Feb 8, 2012 at 1:48 PM, Peter Geoghegan wrote:
>> That was clear from an early stage, and is something that I
>> acknowledged way back in September
>
> OK, so why didn't/don't we do and commit that part first, and then
> proceed to argue abou
On Wed, Feb 8, 2012 at 1:48 PM, Peter Geoghegan wrote:
> That was clear from an early stage, and is something that I
> acknowledged way back in September
OK, so why didn't/don't we do and commit that part first, and then
proceed to argue about the remainder once it's in?
> I think that there may
On 8 February 2012 18:48, Peter Geoghegan wrote:
> I think that there may be additional benefits from making the
> qsort_arg specialisation look less like a c stdlib one, like refining
> the swap logic to have compile-time knowledge of the type it is
> sorting. I'm thinking that we could usefully
On 8 February 2012 17:58, Robert Haas wrote:
> It seems clear that the single sort-key optimizations are a much
> better value per byte of code than the type-specific optimizations.
> Ignoring client overhead, we get between half and two-thirds of the
> benefit, and it costs us just one extra copy
Robert Haas writes:
> [ lots of numbers ]
> ... I just can't get excited about that. However, I
> find the single-key optimizations much more compelling, for the
> reasons stated above, and feel we ought to include those.
This conclusion seems sound to me, for the reasons you stated and one
mor
On Wed, Feb 8, 2012 at 10:59 AM, Bruce Momjian wrote:
> On Wed, Feb 08, 2012 at 10:17:36AM -0500, Robert Haas wrote:
>> On Wed, Feb 8, 2012 at 9:51 AM, Tom Lane wrote:
>> > IMO this patch is already well past the point of diminishing returns in
>> > value-per-byte-added. I'd like to see it trimm
On 8 February 2012 15:17, Robert Haas wrote:
> On Wed, Feb 8, 2012 at 9:51 AM, Tom Lane wrote:
>> IMO this patch is already well past the point of diminishing returns in
>> value-per-byte-added. I'd like to see it trimmed back to provide a fast
>> path for just single-column int4/int8/float4/flo
On Wed, Feb 08, 2012 at 11:35:46AM -0500, Tom Lane wrote:
> Bruce Momjian writes:
> > Yes, please. That would be a big help. Is there no optimization for
> > strings? I assume they are sorted a lot.
>
> It seems unlikely that it'd be worth including strings, especially if
> your locale is n
Bruce Momjian writes:
> Yes, please. That would be a big help. Is there no optimization for
> strings? I assume they are sorted a lot.
It seems unlikely that it'd be worth including strings, especially if
your locale is not C. This whole thing only makes sense for datatypes
that are compar
On Wed, Feb 08, 2012 at 10:17:36AM -0500, Robert Haas wrote:
> On Wed, Feb 8, 2012 at 9:51 AM, Tom Lane wrote:
> > IMO this patch is already well past the point of diminishing returns in
> > value-per-byte-added. I'd like to see it trimmed back to provide a fast
> > path for just single-column in
On Wed, Feb 08, 2012 at 01:33:30PM +, Peter Geoghegan wrote:
> It doesn't necessarily matter if we increase the size of the postgres
> binary by 10%, precisely because most of that is not going to be in
> play from one instant to the next. I'm thinking, in particular, of
> btree index specialis
On Wed, Feb 8, 2012 at 9:51 AM, Tom Lane wrote:
> IMO this patch is already well past the point of diminishing returns in
> value-per-byte-added. I'd like to see it trimmed back to provide a fast
> path for just single-column int4/int8/float4/float8 sorts. The other
> cases aren't going to offer
On Wed, Feb 8, 2012 at 8:33 AM, Peter Geoghegan wrote:
> It doesn't necessarily matter if we increase the size of the postgres
> binary by 10%, precisely because most of that is not going to be in
> play from one instant to the next.
As Tom says, that doesn't jive with my experience. If you add
Peter Geoghegan writes:
> It doesn't necessarily matter if we increase the size of the postgres
> binary by 10%, precisely because most of that is not going to be in
> play from one instant to the next.
You've heard of swapping, no? Basically what I'm hearing from you is
total denial that binary
It doesn't necessarily matter if we increase the size of the postgres
binary by 10%, precisely because most of that is not going to be in
play from one instant to the next. I'm thinking, in particular, of
btree index specialisations, where it could make perfect sense to "go
crazy". You cannot have
On Tue, Feb 07, 2012 at 09:38:39PM -0500, Robert Haas wrote:
> So we need some principled way of deciding how much inlining is
> reasonable, because I am 100% certain this is not going to be the last
> time someone discovers that a massive exercise in inlining can yield a
> nifty performance benefi
On Tue, Feb 7, 2012 at 2:39 PM, Jay Levitt wrote:
> Jim "Decibel!" Nasby wrote:
>>
>> I agree that it's probably pretty unusual to index floats.
>
> FWIW: Cubes and points are floats, right? So would spatial indexes benefit
> from this optimization, or is it only raw floats?
Cubes are not floats,
Jim "Decibel!" Nasby wrote:
I agree that it's probably pretty unusual to index floats.
FWIW: Cubes and points are floats, right? So would spatial indexes benefit
from this optimization, or is it only raw floats?
Jay Levitt
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org
On 2/6/12 3:19 PM, Bruce Momjian wrote:
> While we're waiting for anyone else to weigh in with an opinion on the
> right place to draw the line here, do you want to post an updated
> patch with the changes previously discussed?
Well, I think we have to ask not only how many people are using
f
On Mon, Feb 06, 2012 at 06:43:04PM -0500, Bruce Momjian wrote:
> On Mon, Feb 06, 2012 at 10:49:10PM +, Peter Geoghegan wrote:
> > On 6 February 2012 21:19, Bruce Momjian wrote:
> > > Peter Geoghegan obviously has done some serious work in improving
> > > sorting, and worked well with the commu
On Mon, Feb 06, 2012 at 10:49:10PM +, Peter Geoghegan wrote:
> On 6 February 2012 21:19, Bruce Momjian wrote:
> > Peter Geoghegan obviously has done some serious work in improving
> > sorting, and worked well with the community process.
>
> Thank you for acknowledging that.
>
> It's unfortun
On 6 February 2012 21:19, Bruce Momjian wrote:
> Peter Geoghegan obviously has done some serious work in improving
> sorting, and worked well with the community process.
Thank you for acknowledging that.
It's unfortunate that C does not support expressing these kinds of
ideas in a more natural w
On Mon, Feb 06, 2012 at 04:19:07PM -0500, Bruce Momjian wrote:
> Peter Geoghegan obviously has done some serious work in improving
> sorting, and worked well with the community process. He has done enough
> analysis that I am hard-pressed to see how we would get similar
> improvement using a diffe
On Fri, Jan 27, 2012 at 09:37:37AM -0500, Robert Haas wrote:
> On Fri, Jan 27, 2012 at 9:27 AM, Peter Geoghegan
> wrote:
> > Well, I don't think it's all that subjective - it's more the case that
> > it is just difficult, or it gets that way as you consider more
> > specialisations.
>
> Sure it'
On Wed, Feb 01, 2012 at 04:12:58PM -0600, Jim Nasby wrote:
> On Jan 26, 2012, at 9:32 PM, Robert Haas wrote:
> > But if we want to put it on a diet, the first thing I'd probably be
> > inclined to lose is the float4 specialization. Some members of the
> > audience will recall that I take dim view
Excerpts from Jim Nasby's message of mié feb 01 19:12:58 -0300 2012:
> On Jan 26, 2012, at 9:32 PM, Robert Haas wrote:
> > But if we want to put it on a diet, the first thing I'd probably be
> > inclined to lose is the float4 specialization. Some members of the
> > audience will recall that I tak
On Jan 26, 2012, at 9:32 PM, Robert Haas wrote:
> But if we want to put it on a diet, the first thing I'd probably be
> inclined to lose is the float4 specialization. Some members of the
> audience will recall that I take dim view of floating point arithmetic
> generally, but I'll concede that the
On 31 January 2012 19:47, Robert Haas wrote:
> On Fri, Jan 27, 2012 at 3:33 PM, Peter Geoghegan
> wrote:
>> Patch is attached. I have not changed the duplicate functions. This is
>> because I concluded that it was the lesser of two evils to have to get
>> the template to generate both declaratio
On Fri, Jan 27, 2012 at 3:33 PM, Peter Geoghegan wrote:
> Patch is attached. I have not changed the duplicate functions. This is
> because I concluded that it was the lesser of two evils to have to get
> the template to generate both declarations in the header file, and
> definitions in the .c fil
On Fri, Jan 27, 2012 at 9:27 AM, Peter Geoghegan wrote:
> Well, I don't think it's all that subjective - it's more the case that
> it is just difficult, or it gets that way as you consider more
> specialisations.
Sure it's subjective. Two well-meaning people could have different
opinions without
Uh, obviously I meant causal relationship and not correlation.
On 27 January 2012 13:37, Robert Haas wrote:
> I completely agree. So the point is that, when faced a patch that
> adds an atypically large number of CPU instructions, we ought to ask
> ourselves whether those instructions are pullin
On Thu, Jan 26, 2012 at 11:36 PM, Peter Geoghegan wrote:
>> I'm not surprised that you weren't able to measure a performance
>> regression from the binary bloat. Any such regression is bound to be
>> very small and probably quite difficult to notice most of the time;
>> it's really the cumulative
On 27 January 2012 03:32, Robert Haas wrote:
> But if we want to put it on a diet, the first thing I'd probably be
> inclined to lose is the float4 specialization. Some members of the
> audience will recall that I take dim view of floating point arithmetic
> generally, but I'll concede that there
On Thu, Jan 26, 2012 at 5:10 PM, Peter Geoghegan wrote:
> Alright, so while I agree with everything you've asked for, the fact
> is that there is a controversy in relation to binary bloat, and that's
> the blocker here. How can we satisfactorily resolve that, or is that
> question adequately addre
Alright, so while I agree with everything you've asked for, the fact
is that there is a controversy in relation to binary bloat, and that's
the blocker here. How can we satisfactorily resolve that, or is that
question adequately addressed by the benchmark that I produced?
What if third party modul
On Thu, Jan 26, 2012 at 4:09 PM, Peter Geoghegan wrote:
> On 26 January 2012 19:45, Robert Haas wrote:
>> The patch came out about 28% faster than master. Admittedly, that's
>> with no client overhead, but still: not bad.
>
> Thanks. There was a 28% reduction in the time it took to execute the
>
On 26 January 2012 19:45, Robert Haas wrote:
> The patch came out about 28% faster than master. Admittedly, that's
> with no client overhead, but still: not bad.
Thanks. There was a 28% reduction in the time it took to execute the
query, but there would have also been a larger reduction in the t
On Thu, Jan 19, 2012 at 1:47 PM, Peter Geoghegan wrote:
> Thoughts?
I generated some random data using this query:
create table nodups (g) as select (g%10)*1000+g/10 from
generate_series(1,1) g;
And then used pgbench to repeatedly sort it using this query:
select * from nodups order by g o
I decided to take advantage of my ongoing access to a benchmarking
server to see how I could get on with a query with an especially large
sort. Now, that box has 16GB of ram, and some people have that much
memory in their laptops these days, so I was somewhat limited in how
far I could push things.
On 9 January 2012 19:45, Josh Berkus wrote:
>> Obviously, many indexes are unique and thus won't have duplicates at
>> all. But if someone creates an index and doesn't make it unique, odds
>> are very high that it has some duplicates. Not sure how many we
>> typically expect to see, but more tha
> Obviously, many indexes are unique and thus won't have duplicates at
> all. But if someone creates an index and doesn't make it unique, odds
> are very high that it has some duplicates. Not sure how many we
> typically expect to see, but more than zero...
Peter may not, but I personally admin
On 6 January 2012 21:14, Tom Lane wrote:
> When there are lots of duplicates of a particular indexed value, the
> existing code will cause an indexscan to search them in physical order,
> whereas if we remove the existing logic it'll be random --- in
> particular, accesses to the same heap page ca
On Fri, Jan 6, 2012 at 4:14 PM, Tom Lane wrote:
> Admittedly, I don't have any numbers quantifying just how useful that
> might be, but on the other hand you've not presented any evidence
> justifying removing the behavior either. If we believe your position
> that indexes don't generally have lo
Peter Geoghegan writes:
> On 6 January 2012 18:45, Tom Lane wrote:
>> Actually, I'm going to object to reverting that commit, as I believe
>> that having equal-keyed index entries in physical table order may offer
>> some performance benefits at access time. If you don't like the
>> comment, we
On 6 January 2012 18:45, Tom Lane wrote:
> Peter Geoghegan writes:
>> I didn't bother isolating that, because it doesn't really make sense
>> to (not having it is probably only of particular value when doing what
>> I'm doing anyway, but who knows). Go ahead and commit something to
>> remove that
Peter Geoghegan writes:
> I didn't bother isolating that, because it doesn't really make sense
> to (not having it is probably only of particular value when doing what
> I'm doing anyway, but who knows). Go ahead and commit something to
> remove that code (get it in both comparetup_index_btree and
On 6 January 2012 17:29, Robert Haas wrote:
> On Fri, Jan 6, 2012 at 12:10 PM, Peter Geoghegan
> wrote:
>> As you know, all queries tested have lots and lots of duplicates (a
>> ~1.5GB table that contains the same number of distinct elements as a
>> ~10MB table once did), and removing the "dupli
On Fri, Jan 6, 2012 at 12:10 PM, Peter Geoghegan wrote:
> As you know, all queries tested have lots and lots of duplicates (a
> ~1.5GB table that contains the same number of distinct elements as a
> ~10MB table once did), and removing the "duplicate protection" for
> btrees, on top of everything e
On 5 January 2012 20:23, Robert Haas wrote:
> I don't have a problem with the idea of a pg_always_inline, but I'm
> wondering what sort of fallback mechanism you propose. It seems to me
> that if the performance improvement is conditioned on inlining be done
> and we're not confident that we can
On Thu, Jan 5, 2012 at 5:27 PM, Tom Lane wrote:
> There is no compiler anywhere that implements "always inline", unless
> you are talking about a macro. "inline" is a hint and nothing more,
> and if you think you can force it you are mistaken. So this controversy
> is easily resolved: we do not
On 5 January 2012 22:27, Tom Lane wrote:
> There is no compiler anywhere that implements "always inline", unless
> you are talking about a macro. "inline" is a hint and nothing more,
> and if you think you can force it you are mistaken. So this controversy
> is easily resolved: we do not need an
Robert Haas writes:
> On Thu, Dec 29, 2011 at 9:03 PM, Peter Geoghegan
> wrote:
>> The first (controversy A) is that I have added a new
>> piece of infrastructure, pg_always_inline, which, as the name
>> suggests, is a portable way of insisting that a function should be
>> invariably inlined.
>
On Thu, Dec 29, 2011 at 9:03 PM, Peter Geoghegan wrote:
> 3. Resolve two anticipated controversies that are, respectively,
> somewhat orthogonal and completely orthogonal to the binary bloat
> controversy. The first (controversy A) is that I have added a new
> piece of infrastructure, pg_always_in
On Fri, Dec 30, 2011 at 2:30 PM, Peter Geoghegan wrote:
> On 30 December 2011 19:46, Merlin Moncure wrote:
>> On Thu, Dec 29, 2011 at 8:03 PM, Peter Geoghegan
>> wrote:
>>> * A spreadsheet that shows the results of re-running my earlier heap
>>> tuple sorting benchmark with this new patch. The
On 30 December 2011 19:46, Merlin Moncure wrote:
> On Thu, Dec 29, 2011 at 8:03 PM, Peter Geoghegan
> wrote:
>> * A spreadsheet that shows the results of re-running my earlier heap
>> tuple sorting benchmark with this new patch. The improvement in the
>> query that orders by 2 columns is all tha
On Thu, Dec 29, 2011 at 8:03 PM, Peter Geoghegan wrote:
> * A spreadsheet that shows the results of re-running my earlier heap
> tuple sorting benchmark with this new patch. The improvement in the
> query that orders by 2 columns is all that is pertinent there, when
> considering the value of (1)
69 matches
Mail list logo
