Re: Bypassing shared_buffers

2023-06-19 Thread Andrey M . Borodin
Hi! > On 15 Jun 2023, at 03:57, Vladimir Churyukin wrote: > > Hello, > > There is often a need to test particular queries executed in the worst-case > scenario, i.e. right after a server restart or with no or minimal amount of > data in shared buffers. In Postgres it's currently hard to

Re: Bypassing shared_buffers

2023-06-17 Thread Greg Sabino Mullane
On Thu, Jun 15, 2023 at 4:16 AM Vladimir Churyukin wrote: > We're trying to see what is the worst performance in terms of I/O, i.e. >> when the database just started up or the data/indexes being queried are not >> cached at all. > > You could create new tables that are copies of the existing

Re: Bypassing shared_buffers

2023-06-15 Thread Vladimir Churyukin
On Thu, Jun 15, 2023 at 12:32 AM Konstantin Knizhnik wrote: > > > On 15.06.2023 4:37 AM, Vladimir Churyukin wrote: > > Ok, got it, thanks. > > Is there any alternative approach to measuring the performance as if > > the cache was empty? > > The goal is basically to calculate the max possible I/O

Re: Bypassing shared_buffers

2023-06-15 Thread Konstantin Knizhnik
On 15.06.2023 4:37 AM, Vladimir Churyukin wrote: Ok, got it, thanks. Is there any alternative approach to measuring the performance as if the cache was empty? The goal is basically to calculate the max possible I/O time for a query, to get a range between min and max timing. It's ok if it's

Re: Bypassing shared_buffers

2023-06-14 Thread Thomas Munro
On Thu, Jun 15, 2023 at 2:51 PM Vladimir Churyukin wrote: > Do you foresee any difficulties in implementation of the "unwarm" operation? > It requires a cache flush operation, > so I'm curious how complicated that is (probably there is a reason this is > not supported by Postgres by now? mssql

Re: Bypassing shared_buffers

2023-06-14 Thread Vladimir Churyukin
It could be cheaper, if the testing is done for many SELECT queries sequentially - you need to flush dirty buffers just once pretty much. -Vladimir Churyukin On Wed, Jun 14, 2023 at 7:43 PM Tom Lane wrote: > Thomas Munro writes: > > There are two levels of cache. If you're on Linux you can

Re: Bypassing shared_buffers

2023-06-14 Thread Vladimir Churyukin
Do you foresee any difficulties in implementation of the "unwarm" operation? It requires a cache flush operation, so I'm curious how complicated that is (probably there is a reason this is not supported by Postgres by now? mssql and oracle support stuff like that for a long time) Cluster restart

Re: Bypassing shared_buffers

2023-06-14 Thread Tom Lane
Thomas Munro writes: > There are two levels of cache. If you're on Linux you can ask it to > drop its caches by writing certain values to /proc/sys/vm/drop_caches. > For PostgreSQL's own buffer pool, it would be nice if someone would > extend the pg_prewarm extension to have a similar 'unwarm'

Re: Bypassing shared_buffers

2023-06-14 Thread Thomas Munro
On Thu, Jun 15, 2023 at 1:37 PM Vladimir Churyukin wrote: > Ok, got it, thanks. > Is there any alternative approach to measuring the performance as if the > cache was empty? There are two levels of cache. If you're on Linux you can ask it to drop its caches by writing certain values to

Re: Bypassing shared_buffers

2023-06-14 Thread Vladimir Churyukin
Ok, got it, thanks. Is there any alternative approach to measuring the performance as if the cache was empty? The goal is basically to calculate the max possible I/O time for a query, to get a range between min and max timing. It's ok if it's done during EXPLAIN ANALYZE call only, not for regular

Re: Bypassing shared_buffers

2023-06-14 Thread Tom Lane
Vladimir Churyukin writes: > There is often a need to test particular queries executed in the worst-case > scenario, i.e. right after a server restart or with no or minimal amount of > data in shared buffers. In Postgres it's currently hard to achieve (other > than to restart the server

Re: Bypassing shared_buffers

2023-06-14 Thread Vladimir Churyukin
To be clear, I'm talking about bypassing shared buffers for reading data / indexes only, not about disabling it completely (which I guess is impossible anyway). -Vladimir Churyukin On Wed, Jun 14, 2023 at 5:57 PM Vladimir Churyukin wrote: > Hello, > > There is often a need to test particular

Bypassing shared_buffers

2023-06-14 Thread Vladimir Churyukin
Hello, There is often a need to test particular queries executed in the worst-case scenario, i.e. right after a server restart or with no or minimal amount of data in shared buffers. In Postgres it's currently hard to achieve (other than to restart the server completely to run a single query,