Re: 600 million rows of data. Bad hardware or need partitioning?

2020-05-03 Thread David Rowley
On Mon, 4 May 2020 at 15:52, Arya F wrote: > > On Sun, May 3, 2020 at 11:46 PM Michael Lewis wrote: > > > > What kinds of storage (ssd or old 5400 rpm)? What else is this machine > > running? > > Not an SSD, but an old 1TB 7200 RPM HDD > > > What configs have been customized such as work_mem or

Re: 600 million rows of data. Bad hardware or need partitioning?

2020-05-03 Thread Arya F
On Sun, May 3, 2020 at 11:46 PM Michael Lewis wrote: > > What kinds of storage (ssd or old 5400 rpm)? What else is this machine > running? Not an SSD, but an old 1TB 7200 RPM HDD > What configs have been customized such as work_mem or random_page_cost? work_mem = 2403kB random_page_cost = 1.1

Re: 600 million rows of data. Bad hardware or need partitioning?

2020-05-03 Thread Michael Lewis
What kinds of storage (ssd or old 5400 rpm)? What else is this machine running? What configs have been customized such as work_mem or random_page_cost?

600 million rows of data. Bad hardware or need partitioning?

2020-05-03 Thread Arya F
I have created the following table to duplicate my performance numbers, but I have simplified the table for this question. I'm running PostgreSQL 12 on the following hardware. Dual Xeon Quad-Core E5320 1.86GHz 4GB of RAM The table structure is id uuid address_api_url text check_timestamp tim

Re: Please help! Query jumps from 1s -> 4m

2020-05-03 Thread Justin Pryzby
On Sun, May 03, 2020 at 09:58:27AM +0100, James Thompson wrote: > Hi, > > Hoping someone can help with this performance issue that's been driving a > few of us crazy :-) Any guidance greatly appreciated. > > A description of what you are trying to achieve and what results you > expect.: > - I'd

Please help! Query jumps from 1s -> 4m

2020-05-03 Thread James Thompson
Hi, Hoping someone can help with this performance issue that's been driving a few of us crazy :-) Any guidance greatly appreciated. A description of what you are trying to achieve and what results you expect.: - I'd like to get an understanding of why the following query (presented in full, but

Re: Duplicate WHERE condition changes performance and plan

2020-05-03 Thread singh...@gmail.com
> I don't *think* we are using SSDs but I'll need to confirm that though. Confirmed we are not using SSDs but '10K RPM SAS in RAID-10.' I've also been hunt for other queries that show this behaviour too, and I've found one. The PG settings/versions will be different in this example due to the ea