Re: High QPS, random index writes and vacuum

2023-04-17 Thread Peter Geoghegan
On Mon, Apr 17, 2023 at 7:43 PM peter plachta wrote: > Version: I sheepishly admit it's 9.6, 10 and 11 (it's Azure Single Server, > that's another story). If you can upgrade to 14, you'll find that there is much improved management of index updates on that version:

Re: High QPS, random index writes and vacuum

2023-04-17 Thread Thomas Munro
On Tue, Apr 18, 2023 at 2:43 PM peter plachta wrote: > I was trying to understand whether there are any known workarounds for random > access + index vacuums. Are my vacuum times 'normal' ? Ah, it's not going to help on the old versions you mentioned, but for what it's worth: I remember

Re: High QPS, random index writes and vacuum

2023-04-17 Thread peter plachta
Thank you David -- I increased this to 1GB as well (seeing as that was the max). We are doing mostly single passes now. On Mon, Apr 17, 2023 at 7:40 PM David Rowley wrote: > On Tue, 18 Apr 2023 at 12:35, peter plachta wrote: > > I increased work_mem to 2Gb > > maintenance_work_mem is the

Re: High QPS, random index writes and vacuum

2023-04-17 Thread peter plachta
Thank you Tom. Version: I sheepishly admit it's 9.6, 10 and 11 (it's Azure Single Server, that's another story). I am definitely looking at redoing the way we do UUIDs... but that' s not a trivial change given the volume of data we have + 24/7 workload. I was trying to understand whether there

Re: High QPS, random index writes and vacuum

2023-04-17 Thread David Rowley
On Tue, 18 Apr 2023 at 12:35, peter plachta wrote: > I increased work_mem to 2Gb maintenance_work_mem is the configuration option that vacuum uses to control how much memory it'll make available for storage of dead tuples. I believe 1GB would allow 178,956,970 tuples to be stored before multiple

Re: High QPS, random index writes and vacuum

2023-04-17 Thread Tom Lane
peter plachta writes: > The company I work for has a large (50+ instances, 2-4 TB each) Postgres > install. One of the key problems we are facing in vanilla Postgres is > vacuum behavior on high QPS (20K writes/s), random index access on UUIDs. Indexing on a UUID column is an antipattern,

Re: High QPS, random index writes and vacuum

2023-04-17 Thread Peter Geoghegan
On Mon, Apr 17, 2023 at 5:35 PM peter plachta wrote: > My question is: what is the recommended strategy to deal with such cases in > Postgres? You didn't say what version of Postgres you're using... -- Peter Geoghegan

High QPS, random index writes and vacuum

2023-04-17 Thread peter plachta
Hi all The company I work for has a large (50+ instances, 2-4 TB each) Postgres install. One of the key problems we are facing in vanilla Postgres is vacuum behavior on high QPS (20K writes/s), random index access on UUIDs. In one case the table is 50Gb and has 3 indexes which are also 50Gb

Re: time sorted UUIDs

2023-04-17 Thread peter plachta
Hi Tim -- I am looking at the issue of random IDs (ie, UUIDs) as well. Did you have a chance to try time sorted UUIDs as was suggested in one of the responses? On Mon, Apr 17, 2023 at 5:23 PM Tim Jones wrote: > Hi, > > could someone please comment on this article >

Re: speeding up grafana sensor-data query on raspberry pi 3

2023-04-17 Thread Luiz Felipph
Is an option partitioning the table by month? If your report is month based, you can improve performance by partitioning. Felipph Em dom., 16 de abr. de 2023 às 19:10, Andres Freund escreveu: > Hi, > > On 2023-04-16 19:00:33 +0200, Clemens Eisserer wrote: > > I am currently trying to migrate