Re: [SOLVED?] Re: Disk wait problem... not hardware...

2023-10-30 Thread Peter J. Holzer
On 2023-10-29 12:45:08 -0400, p...@pfortin.com wrote: > On Sun, 29 Oct 2023 16:16:05 +0100 Peter J. Holzer wrote: > >However, the table statistics contain an estimate for the number of > >rows: > > > >hjp=> select schemaname, relname, n_live_tup from pg_stat_user_tables order > >by 3 desc; >

Re: [SOLVED?] Re: Disk wait problem... not hardware...

2023-10-29 Thread Adrian Klaver
On 10/29/23 09:45, p...@pfortin.com wrote: On Sun, 29 Oct 2023 16:16:05 +0100 Peter J. Holzer wrote: On 2023-10-29 09:21:46 -0400, p...@pfortin.com wrote: These are all static tables. Does PG maintain a table row count so as to avoid having to count each time? No. To count the rows in a

Re: [SOLVED?] Re: Disk wait problem... not hardware...

2023-10-29 Thread Ron
On 10/29/23 11:45, p...@pfortin.com wrote: On Sun, 29 Oct 2023 16:16:05 +0100 Peter J. Holzer wrote: On 2023-10-29 09:21:46 -0400, p...@pfortin.com wrote: These are all static tables. Does PG maintain a table row count so as to avoid having to count each time? No. To count the rows in a

Re: [SOLVED?] Re: Disk wait problem... not hardware...

2023-10-29 Thread pf
On Sun, 29 Oct 2023 16:16:05 +0100 Peter J. Holzer wrote: >On 2023-10-29 09:21:46 -0400, p...@pfortin.com wrote: >> These are all static tables. Does PG maintain a table row count so as to >> avoid having to count each time? > >No. To count the rows in a table, Postgres has to actually read the

Re: Disk wait problem... may not be hardware...

2023-10-29 Thread pf
On Sun, 29 Oct 2023 16:00:46 +0100 Peter J. Holzer wrote: >On 2023-10-27 19:46:09 -0400, p...@pfortin.com wrote: >> On Fri, 27 Oct 2023 19:07:11 +0200 Peter J. Holzer wrote: >> >Have you looked at the query plans as I recommended? (You might also >> >want to enable track_io_timing to get extra

Re: [SOLVED?] Re: Disk wait problem... not hardware...

2023-10-29 Thread Peter J. Holzer
On 2023-10-29 09:21:46 -0400, p...@pfortin.com wrote: > These are all static tables. Does PG maintain a table row count so as to > avoid having to count each time? No. To count the rows in a table, Postgres has to actually read the whole table (or an index, if a suitable index (e.g. a primary

Re: Disk wait problem... may not be hardware...

2023-10-29 Thread Peter J. Holzer
On 2023-10-27 19:46:09 -0400, p...@pfortin.com wrote: > On Fri, 27 Oct 2023 19:07:11 +0200 Peter J. Holzer wrote: > >Have you looked at the query plans as I recommended? (You might also > >want to enable track_io_timing to get extra information, but comparing > >just the query plans of fast and

[SOLVED?] Re: Disk wait problem... not hardware...

2023-10-29 Thread pf
On Sat, 28 Oct 2023 18:34:50 -0400 Jim Mlodgenski wrote: Looking like a GOLD star for Jim... >On Fri, Oct 27, 2023 at 7:46 PM wrote: > >> Memory: 125.5 GiB of RAM >> >It looks like you have a large amount of memory allocated to the server > >But your plans are doing reads instead of pulling

Re: Disk wait problem... may not be hardware...

2023-10-28 Thread Jim Mlodgenski
On Fri, Oct 27, 2023 at 7:46 PM wrote: > Memory: 125.5 GiB of RAM > It looks like you have a large amount of memory allocated to the server But your plans are doing reads instead of pulling things from shared buffers >explain (analyze, buffers) select count(*) from ncvhis_2016_12_03; >

Re: Disk wait problem... may not be hardware...

2023-10-28 Thread pf
On Fri, 27 Oct 2023 21:21:18 -0700 Adrian Klaver wrote: >On 10/27/23 16:46, p...@pfortin.com wrote: >> Peter, >> >> Thanks for your patience; I've been feeling pressure to get this >> resolved; so have been lax in providing info here.. Hope the following >> helps... >> > > >> Something I

Re: Disk wait problem... may not be hardware...

2023-10-27 Thread Adrian Klaver
On 10/27/23 16:46, p...@pfortin.com wrote: Peter, Thanks for your patience; I've been feeling pressure to get this resolved; so have been lax in providing info here.. Hope the following helps... Something I hadn't noticed before: SQL-workbench/J (build 129.6) displays an execution timer

Re: Disk wait problem... may not be hardware...

2023-10-27 Thread pf
Peter, Thanks for your patience; I've been feeling pressure to get this resolved; so have been lax in providing info here.. Hope the following helps... On Fri, 27 Oct 2023 19:07:11 +0200 Peter J. Holzer wrote: >On 2023-10-26 22:03:25 -0400, p...@pfortin.com wrote: >> Are there any extra PG

Re: Disk wait problem... may not be hardware...

2023-10-27 Thread Peter J. Holzer
On 2023-10-26 22:03:25 -0400, p...@pfortin.com wrote: > Are there any extra PG low level logs that can be turned on? Have you looked at the query plans as I recommended? (You might also want to enable track_io_timing to get extra information, but comparing just the query plans of fast and slow

Re: Disk wait problem... may not be hardware...

2023-10-27 Thread Justin Clift
On 2023-10-27 12:03, p...@pfortin.com wrote: I can't think of a common hardware bus/other that would only affect PostgreSQL disk accesses. Which file system is PostgreSQL being run on? Asking because I remember seeing weirdness reported with *some* SSD drives when used with ZFS:

Re: Disk wait problem... may not be hardware...

2023-10-26 Thread pf
On Thu, 26 Oct 2023 15:50:16 -0400 p...@pfortin.com wrote: >Hi Peter, > >All of the following is based on using SQL_workbench/J (WB) (Mageia Linux >has not provided a viable pgAdmin4 package); WB is setup to autoload the >table row count and 10 rows. I'm sticking to one set of files where they

Re: Disk wait problem...

2023-10-26 Thread pf
Hi Peter, All of the following is based on using SQL_workbench/J (WB) (Mageia Linux has not provided a viable pgAdmin4 package); WB is setup to autoload the table row count and 10 rows. I'm sticking to one set of files where they are all roughly 33.6M rows. I've been doing a lot of digging and

Re: Disk wait problem...

2023-10-26 Thread Peter J. Holzer
On 2023-10-23 16:31:30 -0700, Adrian Klaver wrote: > On 10/23/23 14:55, p...@pfortin.com wrote: > > Next, I made a query list for 30 tables, turned on timing and pasted the > > queries. Pasting results inline: > > ncsbe=# select count(*) from ncvhis_2012_12_06; > > select count(*) from

Re: Disk wait problem... 15.4

2023-10-24 Thread pf
After much searching, I eventually discovered that there's a firmware update for my Samsung Firecuda NVMe drives: https://www.seagate.com/support/downloads/ Also Seagate: https://www.dell.com/en-us/shop/workstations-isv-certified/sr/workstations/linux?appliedRefinements=39332 Not sure about the

Re: Disk wait problem...

2023-10-23 Thread pf
On Mon, 23 Oct 2023 16:31:30 -0700 Adrian Klaver wrote: >Please reply to list also. >Ccing the list for this post. Sorry, got the list's message and one directly from you; looks like I picked the wrong one to reply to... I just heard from a remote team member who wrote this: !! Eventually

Re: Disk wait problem...

2023-10-23 Thread Adrian Klaver
On 10/23/23 14:55, p...@pfortin.com wrote: Please reply to list also. Ccing the list for this post. On Mon, 23 Oct 2023 12:44:56 -0700 Adrian Klaver wrote: On 10/23/23 11:54, p...@pfortin.com wrote: Hi, I have a 1.6TB database with over 330 tables on a 4TB NVMe SSD. All tables are static

Re: Disk wait problem... 15.4

2023-10-23 Thread pf
Forgot to mention version: PostgreSQL 15.4 on x86_64-mageia-linux-gnu, compiled by gcc (Mageia 12.3.0-3.mga9) 12.3.0, 64-bit Sorry, Pierre

Re: Disk wait problem...

2023-10-23 Thread pf
On Mon, 23 Oct 2023 15:09:16 -0500 Ken Marshall wrote: >> Hi, >> >> I have a 1.6TB database with over 330 tables on a 4TB NVMe SSD. All >> tables are static (no updates); most in 8M and 33M row sizes. Queries have >> been great, until recently. > >> Also attached is the relevant system journal

Re: Disk wait problem...

2023-10-23 Thread Ken Marshall
> Hi, > > I have a 1.6TB database with over 330 tables on a 4TB NVMe SSD. All > tables are static (no updates); most in 8M and 33M row sizes. Queries have > been great, until recently. > Also attached is the relevant system journal entries for one query that > took 2 seconds and two more that

Re: Disk wait problem...

2023-10-23 Thread Adrian Klaver
On 10/23/23 11:54, p...@pfortin.com wrote: Hi, I have a 1.6TB database with over 330 tables on a 4TB NVMe SSD. All tables are static (no updates); most in 8M and 33M row sizes. Queries have been great, until recently. Also attached is the relevant system journal entries for one query that

Disk wait problem...

2023-10-23 Thread pf
Hi, I have a 1.6TB database with over 330 tables on a 4TB NVMe SSD. All tables are static (no updates); most in 8M and 33M row sizes. Queries have been great, until recently. I use SQL-workbench/J (WB) and starting at any table, if I use up/down arrow to switch to another table, all that happens