Re: [HACKERS] Performance degradation in TPC-H Q18

On 2017-03-06 18:59:02 -0500, Robert Haas wrote: > On Mon, Mar 6, 2017 at 5:19 PM, Andres Freund wrote: > > The whole performance issue trigger this thread only exists when the > > hashtable sizes are mis-estimated, right? Turns out that after applying > > the just-committed

Re: [HACKERS] Performance degradation in TPC-H Q18

On Mon, Mar 6, 2017 at 5:19 PM, Andres Freund wrote: > The whole performance issue trigger this thread only exists when the > hashtable sizes are mis-estimated, right? Turns out that after applying > the just-committed changes, that "fixing" the bad-mixing and/or doing >

Re: [HACKERS] Performance degradation in TPC-H Q18

On 2017-03-06 16:58:23 -0500, Robert Haas wrote: > On Mon, Mar 6, 2017 at 3:32 PM, Andres Freund wrote: > >> I think DEBUG1 is far too high for something that could occur with > >> some frequency on a busy system; I'm fairly strongly of the opinion > >> that you ought to

Re: [HACKERS] Performance degradation in TPC-H Q18

On Mon, Mar 6, 2017 at 3:32 PM, Andres Freund wrote: >> I think DEBUG1 is far too high for something that could occur with >> some frequency on a busy system; I'm fairly strongly of the opinion >> that you ought to downgrade that by a couple of levels, say to DEBUG3 >> or so.

Re: [HACKERS] Performance degradation in TPC-H Q18

On 2017-03-04 11:09:40 +0530, Robert Haas wrote: > On Sat, Mar 4, 2017 at 5:56 AM, Andres Freund wrote: > > attached is a patch to address this problem, and the one reported by > > Dilip. I ran a lot of TPC-H and other benchmarks, and so far this > > addresses all the

Re: [HACKERS] Performance degradation in TPC-H Q18

On Sat, Mar 4, 2017 at 11:09 AM, Robert Haas wrote: > On Sat, Mar 4, 2017 at 5:56 AM, Andres Freund wrote: >> attached is a patch to address this problem, and the one reported by >> Dilip. I ran a lot of TPC-H and other benchmarks, and so far this >>

Re: [HACKERS] Performance degradation in TPC-H Q18

On Sat, Mar 4, 2017 at 5:56 AM, Andres Freund wrote: > attached is a patch to address this problem, and the one reported by > Dilip. I ran a lot of TPC-H and other benchmarks, and so far this > addresses all the performance issues, often being noticeably faster than > with

Re: [HACKERS] Performance degradation in TPC-H Q18

Hi, attached is a patch to address this problem, and the one reported by Dilip. I ran a lot of TPC-H and other benchmarks, and so far this addresses all the performance issues, often being noticeably faster than with the dynahash code. Comments? On 2017-03-03 11:23:00 +0530, Kuntal Ghosh

Re: [HACKERS] Performance degradation in TPC-H Q18

On Fri, Mar 3, 2017 at 8:41 AM, Robert Haas wrote: > On Fri, Mar 3, 2017 at 1:22 AM, Andres Freund wrote: >> the resulting hash-values aren't actually meaningfully influenced by the >> IV. Because we just xor with the IV, most hash-value that without

Re: [HACKERS] Performance degradation in TPC-H Q18

On Fri, Mar 3, 2017 at 1:22 AM, Andres Freund wrote: > the resulting hash-values aren't actually meaningfully influenced by the > IV. Because we just xor with the IV, most hash-value that without the IV > would have fallen into a single hash-bucket, fall into a single >

Re: [HACKERS] Performance degradation in TPC-H Q18

On 2017-03-01 11:05:33 +0530, Kuntal Ghosh wrote: > On Wed, Mar 1, 2017 at 10:53 AM, Andres Freund wrote: > > On 2017-03-01 10:47:45 +0530, Kuntal Ghosh wrote: > >> if (insertdist > curdist) > >> { > >> swap the entry to be inserted with the current entry. > >> Try to insert

Re: [HACKERS] Performance degradation in TPC-H Q18

Hi, On 2017-03-01 10:39:11 +0530, Kuntal Ghosh wrote: > On Wed, Mar 1, 2017 at 9:33 AM, Kuntal Ghosh > wrote: > > On Wed, Mar 1, 2017 at 9:19 AM, Andres Freund wrote: > >> That's without the patch in > >>

Re: [HACKERS] Performance degradation in TPC-H Q18

On Wed, Mar 1, 2017 at 10:53 AM, Andres Freund wrote: > On 2017-03-01 10:47:45 +0530, Kuntal Ghosh wrote: >> if (insertdist > curdist) >> { >> swap the entry to be inserted with the current entry. >> Try to insert the current entry in the same logic. >> } >> >> So, the second

Re: [HACKERS] Performance degradation in TPC-H Q18

On 2017-03-01 10:47:45 +0530, Kuntal Ghosh wrote: > if (insertdist > curdist) > { > swap the entry to be inserted with the current entry. > Try to insert the current entry in the same logic. > } > > So, the second approach will not cause all the followers to be shifted by 1. How not? You'll have

Re: [HACKERS] Performance degradation in TPC-H Q18

On Wed, Mar 1, 2017 at 9:38 AM, Andres Freund wrote: > Hi, > > On 2017-03-01 09:33:07 +0530, Kuntal Ghosh wrote: >> On Wed, Mar 1, 2017 at 9:19 AM, Andres Freund wrote: >> >> So, I was looking for other alternatives and I've found one called >> >>

Re: [HACKERS] Performance degradation in TPC-H Q18

On Wed, Mar 1, 2017 at 9:33 AM, Kuntal Ghosh wrote: > On Wed, Mar 1, 2017 at 9:19 AM, Andres Freund wrote: >> That's without the patch in >> http://archives.postgresql.org/message-id/20161123083351.5vramz52nmdokhzz%40alap3.anarazel.de >> ? With

Re: [HACKERS] Performance degradation in TPC-H Q18

Hi, On 2017-03-01 09:33:07 +0530, Kuntal Ghosh wrote: > On Wed, Mar 1, 2017 at 9:19 AM, Andres Freund wrote: > >> So, I was looking for other alternatives and I've found one called > >> RobinHood hashing. > > > > simplehash.h implements robin hood hashing. > But, it doesn't

Re: [HACKERS] Performance degradation in TPC-H Q18

On Wed, Mar 1, 2017 at 9:19 AM, Andres Freund wrote: > On 2017-03-01 09:13:15 +0530, Kuntal Ghosh wrote: >> On Wed, Mar 1, 2017 at 8:48 AM, Andres Freund wrote: >> >> BTW, another option to consider is lowering the target fillfactor. >> >> IIRC, Kuntal

Re: [HACKERS] Performance degradation in TPC-H Q18

On 2017-03-01 09:21:47 +0530, Robert Haas wrote: > On Wed, Mar 1, 2017 at 8:48 AM, Andres Freund wrote: > >> BTW, another option to consider is lowering the target fillfactor. > >> IIRC, Kuntal mentioned to me that cranking it down seemed to fix the > >> issue. Obviously,

Re: [HACKERS] Performance degradation in TPC-H Q18

On Wed, Mar 1, 2017 at 8:48 AM, Andres Freund wrote: >> BTW, another option to consider is lowering the target fillfactor. >> IIRC, Kuntal mentioned to me that cranking it down seemed to fix the >> issue. Obviously, it's better to detect when we need a lower >> fillfactor

Re: [HACKERS] Performance degradation in TPC-H Q18

On 2017-03-01 09:13:15 +0530, Kuntal Ghosh wrote: > On Wed, Mar 1, 2017 at 8:48 AM, Andres Freund wrote: > >> BTW, another option to consider is lowering the target fillfactor. > >> IIRC, Kuntal mentioned to me that cranking it down seemed to fix the > >> issue. Obviously,

Re: [HACKERS] Performance degradation in TPC-H Q18

On Wed, Mar 1, 2017 at 8:48 AM, Andres Freund wrote: >> BTW, another option to consider is lowering the target fillfactor. >> IIRC, Kuntal mentioned to me that cranking it down seemed to fix the >> issue. Obviously, it's better to detect when we need a lower >> fillfactor

Re: [HACKERS] Performance degradation in TPC-H Q18

On 2017-03-01 08:42:35 +0530, Robert Haas wrote: > On Tue, Feb 28, 2017 at 11:16 PM, Andres Freund wrote: > >> To me, it seems like a big problem that we have large, unfixed > >> performance regressions with simplehash four months after it went in. > > > > Yea, I agree. I'm

Re: [HACKERS] Performance degradation in TPC-H Q18

On Tue, Feb 28, 2017 at 11:16 PM, Andres Freund wrote: >> To me, it seems like a big problem that we have large, unfixed >> performance regressions with simplehash four months after it went in. > > Yea, I agree. I'm fairly sure that the patch I posted in that thread >

Re: [HACKERS] Performance degradation in TPC-H Q18

Hi, On 2017-02-26 19:30:32 +0530, Robert Haas wrote: > On Wed, Feb 22, 2017 at 11:23 AM, Kuntal Ghosh > wrote: > > While conducting the experiments for parallelism, Rafia came across a > > hang in Q18 when plan uses partial and finalize hash aggregate. This > > could

Re: [HACKERS] Performance degradation in TPC-H Q18

On Wed, Feb 22, 2017 at 11:23 AM, Kuntal Ghosh wrote: > While conducting the experiments for parallelism, Rafia came across a > hang in Q18 when plan uses partial and finalize hash aggregate. This > could be seen on both scale factors - 20 and 300, on setting work_mem

[HACKERS] Performance degradation in TPC-H Q18

Hello everyone, While conducting the experiments for parallelism, Rafia came across a hang in Q18 when plan uses partial and finalize hash aggregate. This could be seen on both scale factors - 20 and 300, on setting work_mem high enough so that the query uses hash aggregate. It seems that commit