Re: Extremely slow HashAggregate in simple UNION query

2019-08-24 Thread Andres Freund
Hi, On August 24, 2019 12:41:03 PM PDT, Tom Lane wrote: >Jeff Janes writes: >> Most of the time is not after the clock stops, but before the >stepwise >> ANALYZE clock starts. If you just do an EXPLAIN rather than EXPLAIN >> ANALYZE, that is also slow. The giant hash table is created during

Re: Extremely slow HashAggregate in simple UNION query

2019-08-24 Thread Jeff Janes
On Thu, Aug 22, 2019 at 1:09 AM Pavel Stehule wrote: > čt 22. 8. 2019 v 3:11 odesílatel Jeff Janes napsal: > >> ... > But the same advance in v12 which makes it harder to fool with your test >> case also opens the possibility of fixing your real case. >> > > I think so much more interesting

Re: Extremely slow HashAggregate in simple UNION query

2019-08-22 Thread Felix Geisendörfer
> On 21. Aug 2019, at 20:26, Jeff Janes wrote: > > As noted elsewhere, v12 thwarts your attempts to deliberately design the bad > estimates. You can still get them, you just have to work a bit harder at it: > > CREATE FUNCTION j (bigint, bigint) returns setof bigint as $$ select >

Re: Extremely slow HashAggregate in simple UNION query

2019-08-22 Thread Pavel Stehule
čt 22. 8. 2019 v 3:11 odesílatel Jeff Janes napsal: > On Tue, Aug 20, 2019 at 11:12 AM Felix Geisendörfer > wrote: > ... > > >> [1] My actual query had bad estimates for other reasons (GIN Index), but >> that's another story. The query above was of course deliberately designed >> to have bad

Re: Extremely slow HashAggregate in simple UNION query

2019-08-21 Thread Jeff Janes
On Tue, Aug 20, 2019 at 11:12 AM Felix Geisendörfer wrote: ... > [1] My actual query had bad estimates for other reasons (GIN Index), but > that's another story. The query above was of course deliberately designed > to have bad estimates. > As noted elsewhere, v12 thwarts your attempts to

Re: Extremely slow HashAggregate in simple UNION query

2019-08-20 Thread Michael Lewis
I believe this would be relevant- https://www.cybertec-postgresql.com/en/optimizer-support-functions/ It seems there is hope down the road to improve those estimates.

Re: Extremely slow HashAggregate in simple UNION query

2019-08-20 Thread Andres Freund
Hi, On 2019-08-20 19:55:56 +0200, Felix Geisendörfer wrote: > > On 20. Aug 2019, at 19:32, Andres Freund wrote: > > FWIW, that's not a mis-estimate I'm getting on master ;). Obviously > > that doesn't actually address your concern... > > I suppose this is thanks to the new optimizer support

Re: Extremely slow HashAggregate in simple UNION query

2019-08-20 Thread Felix Geisendörfer
Hi, > On 20. Aug 2019, at 19:32, Andres Freund wrote: > > Hi, > > On 2019-08-20 17:11:58 +0200, Felix Geisendörfer wrote: >> >> HashAggregate (cost=80020.01..100020.01 rows=200 width=8) (actual >> time=19.349..23.123 rows=1 loops=1) > > FWIW, that's not a mis-estimate I'm getting on

Re: Extremely slow HashAggregate in simple UNION query

2019-08-20 Thread Andres Freund
Hi, On 2019-08-20 17:11:58 +0200, Felix Geisendörfer wrote: > today I debugged a query that was executing about 100x slower than expected, > and was very surprised by what I found. > > I'm posting to this list to see if this might be an issue that should be > fixed in PostgreSQL itself. > >

Re: Extremely slow HashAggregate in simple UNION query

2019-08-20 Thread Pavel Stehule
út 20. 8. 2019 v 17:12 odesílatel Felix Geisendörfer napsal: > Hi all, > > today I debugged a query that was executing about 100x slower than > expected, and was very surprised by what I found. > > I'm posting to this list to see if this might be an issue that should be > fixed in PostgreSQL

Extremely slow HashAggregate in simple UNION query

2019-08-20 Thread Felix Geisendörfer
Hi all, today I debugged a query that was executing about 100x slower than expected, and was very surprised by what I found. I'm posting to this list to see if this might be an issue that should be fixed in PostgreSQL itself. Below is a simplified version of the query in question: SET