Re: select distinct runs slow on pg 10.6

2019-09-12 Thread Dinesh Somani
Thanks a lot, Merlin. Yes, it could appear kinda gross to some ;-) On Thu, Sep 12, 2019 at 7:19 AM Merlin Moncure wrote: > On Wed, Sep 11, 2019 at 12:57 PM Rick Otten > wrote: > > > > On Wed, Sep 11, 2019 at 12:38 PM Dinesh Somani > wrote: > >> > >> I think Merlin has outlined pretty much all

Re: select distinct runs slow on pg 10.6

2019-09-12 Thread Merlin Moncure
On Wed, Sep 11, 2019 at 12:57 PM Rick Otten wrote: > > On Wed, Sep 11, 2019 at 12:38 PM Dinesh Somani wrote: >> >> I think Merlin has outlined pretty much all the options and very neatly. (As >> an asides Merlin could you possibly elaborate on the "C Hack" how that might >> be accomplished.) >>

Re: select distinct runs slow on pg 10.6

2019-09-11 Thread Rick Otten
On Wed, Sep 11, 2019 at 12:38 PM Dinesh Somani wrote: > I think Merlin has outlined pretty much all the options and very neatly. > (As an asides Merlin could you possibly elaborate on the "C Hack" how that > might be accomplished.) > > To OP, I am curious if the performance changes were the query

Re: select distinct runs slow on pg 10.6

2019-09-11 Thread Dinesh Somani
I think Merlin has outlined pretty much all the options and very neatly. (As an asides Merlin could you possibly elaborate on the "C Hack" how that might be accomplished.) To OP, I am curious if the performance changes were the query rewritten such that all timestamp columns were listed first in t

Re: select distinct runs slow on pg 10.6

2019-09-11 Thread Merlin Moncure
On Mon, Sep 9, 2019 at 3:55 AM yash mehta wrote: > > We have a query that takes 1min to execute in postgres 10.6 and the same > executes in 4 sec in Oracle database. The query is doing 'select distinct'. > If I add a 'group by' clause, performance in postgres improves significantly > and fetche

Re: select distinct runs slow on pg 10.6

2019-09-10 Thread Rick Otten
On Tue, Sep 10, 2019 at 12:53 AM yash mehta wrote: > Hi Michael/Justin/Flo, > > Thank you all for your assistance. As Michael said, looks like there are > no more tricks left. > > On Mon, Sep 9, 2019 at 9:09 PM Michael Lewis wrote: > >> If you can't modify the query, then there is nothing more t

Re: select distinct runs slow on pg 10.6

2019-09-09 Thread yash mehta
Hi Michael/Justin/Flo, Thank you all for your assistance. As Michael said, looks like there are no more tricks left. On Mon, Sep 9, 2019 at 9:09 PM Michael Lewis wrote: > If you can't modify the query, then there is nothing more to be done to > optimize the execution afaik. Distinct is much slo

Re: select distinct runs slow on pg 10.6

2019-09-09 Thread Michael Lewis
If you can't modify the query, then there is nothing more to be done to optimize the execution afaik. Distinct is much slower than group by in scenarios like this with many columns. You already identified the disk sort and increased work mem to get it faster by 3x. There are not any other tricks of

Re: select distinct runs slow on pg 10.6

2019-09-09 Thread Flo Rance
There are few things to consider: - you don't need to use distinct on all columns (and therefore sort all columns) - you should try to sort in memory, better than on-disk - it seems that the planner doesn't predict the good number of rows Regards, Florian On Mon, Sep 9, 2019 at 12:46 PM Justin Pr

Re: select distinct runs slow on pg 10.6

2019-09-09 Thread Justin Pryzby
On Mon, Sep 09, 2019 at 02:00:01PM +0530, yash mehta wrote: > We have a query that takes 1min to execute in postgres 10.6 and the same > executes in 4 sec in Oracle database. The query is doing 'select distinct'. > If I add a 'group by' clause, performance in postgres improves > significantly and f

Re: select distinct runs slow on pg 10.6

2019-09-09 Thread yash mehta
Hi Flo, PFB the explain plan: "Limit (cost=5925.59..5944.03 rows=25 width=6994) (actual time=57997.219..58002.451 rows=25 loops=1)" " -> Unique (cost=5925.59..5969.10 rows=59 width=6994) (actual time=57997.218..58002.416 rows=25 loops=1)" "-> Sort (cost=5925.59..5925.74 rows=59

Re: select distinct runs slow on pg 10.6

2019-09-09 Thread Flo Rance
On Mon, Sep 9, 2019 at 10:38 AM yash mehta wrote: > In addition to below mail, we have used btree indexes for primary key > columns. Below is the query: > > select distinct shipmentre0_.FIN_IDas FIN1_53_0_, > workflowst10_.FIN_ID as FIN1_57_1_, > car

select distinct runs slow on pg 10.6

2019-09-09 Thread yash mehta
We have a query that takes 1min to execute in postgres 10.6 and the same executes in 4 sec in Oracle database. The query is doing 'select distinct'. If I add a 'group by' clause, performance in postgres improves significantly and fetches results in 2 sec (better than oracle). But unfortunately, we

Re: select distinct runs slow on pg 10.6

2019-09-09 Thread yash mehta
In addition to below mail, we have used btree indexes for primary key columns. Below is the query: select distinct shipmentre0_.FIN_IDas FIN1_53_0_, workflowst10_.FIN_ID as FIN1_57_1_, carriers3_.FIN_ID as FIN1_40_2_, shipment