Re: [HACKERS] Minor performance improvement in transition to external sort

2014-04-17 Thread Robert Haas
On Wed, Apr 16, 2014 at 7:38 PM, Bruce Momjian br...@momjian.us wrote: On Thu, Apr 10, 2014 at 06:03:15PM +0100, Simon Riggs wrote: On 6 February 2014 18:21, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Feb 4, 2014 at 2:22 PM, Jeremy Harris j...@wizmail.org wrote: The attached patch

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-04-16 Thread Bruce Momjian
On Thu, Apr 10, 2014 at 06:03:15PM +0100, Simon Riggs wrote: On 6 February 2014 18:21, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Feb 4, 2014 at 2:22 PM, Jeremy Harris j...@wizmail.org wrote: The attached patch replaces the existing siftup method for heapify with a siftdown method.

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-04-10 Thread Simon Riggs
On 6 February 2014 18:21, Jeff Janes jeff.ja...@gmail.com wrote: On Tue, Feb 4, 2014 at 2:22 PM, Jeremy Harris j...@wizmail.org wrote: The attached patch replaces the existing siftup method for heapify with a siftdown method. Tested with random integers it does 18% fewer compares and takes

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-26 Thread Jeremy Harris
On 26/02/14 03:08, Robert Haas wrote: On Tue, Feb 25, 2014 at 2:55 PM, Jeremy Harris j...@wizmail.org wrote: On 24/02/14 17:38, Robert Haas wrote: On Thu, Feb 20, 2014 at 7:27 PM, Jeremy Harris j...@wizmail.org wrote: Run under cachegrind, it takes about N/10 last-level cache misses, all

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-25 Thread Jeremy Harris
On 24/02/14 17:38, Robert Haas wrote: On Thu, Feb 20, 2014 at 7:27 PM, Jeremy Harris j...@wizmail.org wrote: Run under cachegrind, it takes about N/10 last-level cache misses, all for the new item being introduced to the heap. The existing code takes none at all. Can you explain this

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-25 Thread Robert Haas
On Tue, Feb 25, 2014 at 2:55 PM, Jeremy Harris j...@wizmail.org wrote: On 24/02/14 17:38, Robert Haas wrote: On Thu, Feb 20, 2014 at 7:27 PM, Jeremy Harris j...@wizmail.org wrote: Run under cachegrind, it takes about N/10 last-level cache misses, all for the new item being introduced to the

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-24 Thread Robert Haas
On Thu, Feb 20, 2014 at 7:27 PM, Jeremy Harris j...@wizmail.org wrote: On 09/02/14 17:11, Jeremy Harris wrote: On 06/02/14 18:21, Jeff Janes wrote: Did you try sorting already-sorted, reverse sorted, or pipe-organ shaped data sets? We will also need to test it on strings. I usually use

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-20 Thread Jeremy Harris
On 09/02/14 17:11, Jeremy Harris wrote: On 06/02/14 18:21, Jeff Janes wrote: Did you try sorting already-sorted, reverse sorted, or pipe-organ shaped data sets? We will also need to test it on strings. I usually use md5(random()::text) to generate strings for such purposes, at least for a

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-09 Thread Robert Haas
On Fri, Feb 7, 2014 at 4:28 PM, Jeremy Harris j...@wizmail.org wrote: On 06/02/14 22:12, Jeremy Harris wrote: Did you try sorting already-sorted, reverse sorted, or pipe-organ shaped data sets? Summary (low numbers better): Random ints: 83% compares, level on time. Sorted ints:

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-07 Thread Jeremy Harris
On 06/02/14 22:12, Jeremy Harris wrote: Did you try sorting already-sorted, reverse sorted, or pipe-organ shaped data sets? Summary (low numbers better): Random ints: 83% compares, level on time. Sorted ints: level compares, 70% time. Reverse-sorted ints: 10% compares, 15%

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-06 Thread Jeremy Harris
On 05/02/14 21:56, Robert Haas wrote: On Tue, Feb 4, 2014 at 5:22 PM, Jeremy Harris j...@wizmail.org wrote: The attached patch replaces the existing siftup method for heapify with a siftdown method. Tested with random integers it does 18% fewer compares and takes 10% less time for the heapify,

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-06 Thread Robert Haas
On Thu, Feb 6, 2014 at 4:22 AM, Jeremy Harris j...@wizmail.org wrote: On 05/02/14 21:56, Robert Haas wrote: On Tue, Feb 4, 2014 at 5:22 PM, Jeremy Harris j...@wizmail.org wrote: The attached patch replaces the existing siftup method for heapify with a siftdown method. Tested with random

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-06 Thread Jeff Janes
On Tue, Feb 4, 2014 at 2:22 PM, Jeremy Harris j...@wizmail.org wrote: The attached patch replaces the existing siftup method for heapify with a siftdown method. Tested with random integers it does 18% fewer compares and takes 10% less time for the heapify, over the work_mem range 1024 to

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-06 Thread Jeremy Harris
On 06/02/14 18:21, Jeff Janes wrote: How big of sets were you sorting in each case? Big enough to go external. The timings and compare-counts given are purely for the heapify stage not the total for the sort, so are constrained by the work_mem not by the sort size per se. I'm limited to

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-06 Thread Jeremy Harris
On 06/02/14 14:22, Robert Haas wrote: On Thu, Feb 6, 2014 at 4:22 AM, Jeremy Harris j...@wizmail.org wrote: On 05/02/14 21:56, Robert Haas wrote: On Tue, Feb 4, 2014 at 5:22 PM, Jeremy Harris j...@wizmail.org wrote: The attached patch replaces the existing siftup method for heapify with a

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-05 Thread Robert Haas
On Tue, Feb 4, 2014 at 5:22 PM, Jeremy Harris j...@wizmail.org wrote: The attached patch replaces the existing siftup method for heapify with a siftdown method. Tested with random integers it does 18% fewer compares and takes 10% less time for the heapify, over the work_mem range 1024 to

[HACKERS] Minor performance improvement in transition to external sort

2014-02-04 Thread Jeremy Harris
The attached patch replaces the existing siftup method for heapify with a siftdown method. Tested with random integers it does 18% fewer compares and takes 10% less time for the heapify, over the work_mem range 1024 to 1048576. Both algorithms appear to be O(n) (contradicting Wikipedia's claim

Re: [HACKERS] Minor performance improvement in transition to external sort

2014-02-04 Thread Michael Paquier
On Wed, Feb 5, 2014 at 7:22 AM, Jeremy Harris j...@wizmail.org wrote: The attached patch replaces the existing siftup method for heapify with a siftdown method. Tested with random integers it does 18% fewer compares and takes 10% less time for the heapify, over the work_mem range 1024 to