Re: Specific query taking time to process

2020-01-31 Thread Duncan Whitham
Thanks for the reply Tom - you've been a great help. I had been looking at changing - default_statistics_target in a broadbrush fashion - changing it to 1000 on alf_node, alf_node_aspects and alf_node_properties and that gets the query to run fine irrespective of work_mem settings. I then set the

Re: Specific query taking time to process

2020-01-30 Thread Tom Lane
Duncan Whitham writes: > We now only need 1 environment as we can replicate the performance problem > on a copy of live – snapshot/restore from AWS of live. We now have a vacuum > analyse running every night on the 3 tables in question on live – to > eliminate bloat and inaccurate stats as the ro

Re: Specific query taking time to process

2020-01-30 Thread Duncan Whitham
I have read through your comments so far – and they tally with the checks we have been making to an extent – thanks for them. We now only need 1 environment as we can replicate the performance problem on a copy of live – snapshot/restore from AWS of live. We now have a vacuum analyse running ev

Re: Specific query taking time to process

2020-01-30 Thread Fahiz Mohamed
Hi Jeff, Thank you for your email, Sorry I couldn’t respond back to you. I am not working on this project at the moment. I have copied one of my colleague who working on this. He has some progress on this, he will update the email thread with those findings Appreciate your support. Thank you,

Re: Specific query taking time to process

2019-12-11 Thread Jeff Janes
On Wed, Dec 11, 2019 at 5:21 PM Fahiz Mohamed wrote: > There is a slight different in both instance’s data. Inastanbce 1 contains > latest data and instance 2 consists of data which is 3 weeks older than > instance 1. > In knowing where to look for differences in performance, there is a big diff

Re: Specific query taking time to process

2019-12-11 Thread Fahiz Mohamed
There is a slight different in both instance’s data. Inastanbce 1 contains latest data and instance 2 consists of data which is 3 weeks older than instance 1. Following are the number of rows in each table in both instances Instance 1 alf_node : 96493129 rows alf_node_properties : 455599288 row

Re: Specific query taking time to process

2019-12-11 Thread Jeff Janes
On Tue, Dec 10, 2019 at 3:40 AM Fahiz Mohamed wrote: > Thank you very much for your prompt responses. > > I have analysed more regarding this and found the long running query. > > I ran "explain analyse" on this query and I got following result. (We have > 2 identical DB instances and they consis

Re: Specific query taking time to process

2019-12-11 Thread Michael Lewis
This seems beyond me at this point, but I am curious if you also vacuumed alf_node_properties and alf_node tables and checked when they last got (auto)vacuumed/analyzed. With default configs for autovacuum parameters and tables with that many rows, they don't qualify for autovacuum very often. I do

Re: Specific query taking time to process

2019-12-10 Thread Justin Pryzby
On Mon, Dec 09, 2019 at 10:39:38PM +, Fahiz Mohamed wrote: > Thank you very much for your prompt responses. > > I have analysed more regarding this and found the long running query. > > I ran "explain analyse" on this query and I got following result. (We have 2 > identical DB instances and

Re: Specific query taking time to process

2019-12-10 Thread Michael Lewis
On Mon, Dec 9, 2019 at 3:39 PM Fahiz Mohamed wrote: > I ran "explain analyse" on this query and I got following result. (We have > 2 identical DB instances and they consist of same data. Instane 1 took 20+ > second to process and instance 2 took less than a second) > > Instance 1: (This is used b

Re: Specific query taking time to process

2019-12-10 Thread Fahiz Mohamed
Thank you very much for your prompt responses. I have analysed more regarding this and found the long running query. I ran "explain analyse" on this query and I got following result. (We have 2 identical DB instances and they consist of same data. Instane 1 took 20+ second to process and instan

Re: Specific query taking time to process

2019-12-09 Thread Michael Lewis
> > There is a specific search query I am running to get list of Documents and > their metadata from several table in the DB. > We are running Postgres 9.6.9 on Amazon RDS (db.m5.4xlarge instance) > > Our current DB consists of 500GB of data and indexes. Most of the rows in > table are consist of 4

Re: Specific query taking time to process

2019-12-08 Thread Justin Pryzby
On Sat, Dec 07, 2019 at 08:05:59PM +, Fahiz Mohamed wrote: > There is a specific search query I am running to get list of Documents and > their metadata from several table in the DB. > We are running Postgres 9.6.9 on Amazon RDS (db.m5.4xlarge instance) > > Our current DB consists of 500GB of

Specific query taking time to process

2019-12-08 Thread Fahiz Mohamed
Hi team, Could you please help me with this strange issue I am facing in my current live server I am maintaining. There is a specific search query I am running to get list of Documents and their metadata from several table in the DB. We are running Postgres 9.6.9 on Amazon RDS (db.m5.4xlarge in