Re: Performance degradation with CTEs, switching from PG 11 to PG 15

2023-11-22 Thread Andreas Kretschmer
PG and I guess blindly adding the "materialized" keyword will cause errors. yeah, prior to 11 CTEs are a optimizer barrier. You can try to rewrite the queries to not using CTEs - or upgrade. If i were you i would upgrade. Regards, Andreas -- Andreas Kretschmer - currently still (gard

Re: simple query running long time within a long transaction.

2023-11-17 Thread Andreas Kretschmer
to force_custom_plan, default is auto and with that and bind variables pg will use a generic plan. Regards, Andreas -- Andreas Kretschmer - currently still (garden leave) Technical Account Manager (TAM) www.enterprisedb.com

Re: [EXTERNAL] Re: Performance down with JDBC 42

2023-11-05 Thread Andreas Kretschmer
the statement is run by the JDBC driver. Then you can compare the execution plans and spot the difference. Yours, Laurenz Albe -- Andreas Kretschmer - currently still (garden leave) Technical Account Manager (TAM) www.enterprisedb.com

Re: Planning time is time-consuming

2023-09-10 Thread Andreas Kretschmer
On 11 September 2023 03:15:43 CEST, Laurenz Albe wrote: >On Fri, 2023-09-08 at 18:51 +0800, Mikhail Balayan wrote: >> I have three tables: >>     - test_db_bench_1 >>     - test_db_bench_tenants >>     - test_db_bench_tenant_closure >> >> And the query to join them: >> SELECT

Re: Selecting RAM and CPU based on max_connections

2022-05-20 Thread Andreas Kretschmer
On 20 May 2022 10:27:50 CEST, aditya desai wrote: >Hi, >One of our applications needs 3000 max_connections to the database. >Connection pooler like pgbouncer or pgpool is not certified within the >organization yet. So they are looking for setting up high configuration >Hardware with CPU and

Re: Postgresql 12, 512 partition by hash. Slow select

2020-04-05 Thread Andreas Kretschmer
Am 05.04.20 um 19:48 schrieb Arya F: Am I missing something in my setup? Or is this expected? I do know having more than 100 partitions in prior versions of PostgreSQL 12 would cause a major slow down, but from what I read PostgreSQL 12 addresses that now? to say more about your problem we

Re: Slow planning time when public schema included (12 vs. 9.4)

2020-03-21 Thread Andreas Kretschmer
Am 21.03.20 um 13:02 schrieb Anders Steinlein: default_statistics_target = 1000 not sure if this be the culprit here, but i think this is way too high. Leave it at the normal value of 100 and raise it only for particular tables and columns. Regards, Andreas -- 2ndQuadrant - The

Re: Query optimization advice for beginners

2020-01-27 Thread Andreas Kretschmer
Am 27.01.20 um 14:15 schrieb Kemal Ortanca: https://explain.depesz.com/s/G4vq the estimates and the real values are very different, seems like problems with autoanalyze. which version? Andreas -- 2ndQuadrant - The PostgreSQL Support Company. www.2ndQuadrant.com

Re: Delete huge Table under XFS

2019-09-19 Thread Andreas Kretschmer
Am 19.09.19 um 17:59 schrieb Joao Junior: I have a table that Is not being use anymore, I want to drop it. The table is huge, around 800GB and it has some index on it. When I execute the drop table command it goes very slow, I realised that the problem is the filesystem. It seems that XFS

Re: improving windows functions performance

2019-08-05 Thread Andreas Kretschmer
Am 05.08.19 um 22:47 schrieb Mariel Cherkassky: Hey, I have a very big query that consist from 3-4 subqueries that use windows functions. There is a chance that I'll need to rewrite the query but first I'm trying to search for other ways to improve it and I'll be happy to hear if one of u

Re: Strange runtime partition pruning behaviour with 11.4

2019-08-03 Thread Andreas Kretschmer
Am 03.08.19 um 16:06 schrieb Thomas Kellerer: it's posible to rewrite the query to: test=# explain analyse select count(*) from foo_bar_baz as fbb where foo_id = (select foo_id from foo where foo_name = 'eeny'); I know, that's not a solution, but a workaround. :-( Yes, I discovered

Re: Strange runtime partition pruning behaviour with 11.4

2019-08-03 Thread Andreas Kretschmer
Hi, Am 03.08.19 um 15:16 schrieb MichaelDBA: I too am a bit perplexed by why runtime partition pruning does not seem to work with this example.  Anybody got any ideas of this? please don't top-posting. it's posible to rewrite the query to: test=# explain analyse select count(*) from

Re: Searching in varchar column having 100M records

2019-07-17 Thread Andreas Kretschmer
Am 17.07.19 um 14:48 schrieb Tomas Vondra: Either that, or try creating a covering index, so that the query can do an index-only scan. That might reduce the amount of IO against the table, and in the index the data should be located close to each other (same page or pages close to each

Re: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Andreas Kretschmer
Am 17.04.19 um 11:51 schrieb laurent.decha...@orange.com: Here are the logs (with log_error_verbosity = verbose) : 2019-04-17 11:30:42 CEST;35895;thedbuser;thedb;0;LOG:  0: execute : SELECT COUNT(1) FROM big_table 2019-04-17 11:30:42 CEST;35895;thedbuser;thedb;0;LOCATION:

Re: Pg10 : Client Configuration for Parallelism ?

2019-04-17 Thread Andreas Kretschmer
Am 17.04.19 um 08:30 schrieb laurent.decha...@orange.com: SELECT current_setting('max_parallel_workers_per_gather') gives 10 from my session. Is there a client configuration that prevents from using parallelism ? unlikely. if i were you, i would compare all settings, using the different

Re: PostgreSQL upgrade.

2019-04-10 Thread Andreas Kretschmer
Am 10.04.19 um 07:40 schrieb Daulat Ram: We have two node postgresql database version 9.6 with streaming replication which is running on docker environment, os Linux (Ubuntu) and we have to migrate on PostgresQL11. I need your suggestions & steps to compete the upgrade  process

Re: endless quere when upsert with ON CONFLICT clause

2019-03-29 Thread Andreas Kretschmer
Am 29.03.19 um 15:29 schrieb Stephan Schmidt: PostgreSQL version: 11.2 Operating system:   Linux Description: We have a wuite complex CTE which collects data fast enough for us and has a ok execution plan. When we insert the result into a table like With _/some/_data AS ( SELECT…. ),

Re: Zero throughput on a query on a very large table.

2019-01-24 Thread Andreas Kretschmer
Am 25.01.19 um 06:20 schrieb l...@laurent-hasson.com: Hello, We have been stuck for the past week on a query that simply won’t “execute”. We have a table with 1.2B rows that took around 14h to load, but a simple select takes forever and after 10h, no records are coming through still.

Re: Multi-second pauses blocking even trivial activity

2018-09-22 Thread Andreas Kretschmer
Am 21.09.2018 um 21:07 schrieb Patrick Molgaard: Andreas -- just following up to say that this was indeed the root cause. Thanks again. glad i could help you. Regards, Andreas -- 2ndQuadrant - The PostgreSQL Support Company. www.2ndQuadrant.com

Re: Multi-second pauses blocking even trivial activity

2018-09-07 Thread Andreas Kretschmer
> >Intermittently (one or two times a week), all queries on that host are >simultaneously blocked for extended periods (10s of seconds). > >The blocked queries are trivial & not related to locking - I'm seeing >slowlogs of the form: > please check if THP are enabled. Regards, Andreas --

Re: trying to delete most of the table by range of date col

2018-09-03 Thread Andreas Kretschmer
Am 03.09.2018 um 09:06 schrieb Justin Pryzby: Note, I believe it's planned in the future for foreign keys to support referenes to partitioned tables, at which point you could just DROP the monthly partition...but not supported right now. the future is close, that's possible in 11 ;-)

Re: Extremely slow when query uses GIST exclusion index

2018-08-29 Thread Andreas Kretschmer
Am 29.08.2018 um 20:10 schrieb David: On Wed, Aug 29, 2018 at 7:25 AM, Andreas Kretschmer mailto:andr...@a-kretschmer.de>> wrote: Okay, other solution. The problem is the nested loop, we can disable that: test=*# set enable_nestloop to false; Is it OK t

Re: Extremely slow when query uses GIST exclusion index

2018-08-29 Thread Andreas Kretschmer
Am 29.08.2018 um 05:31 schrieb David: For now, I can bypass the GIST index by avoiding range operators in my queries. But why is the GIST index so slow? your GiST-Index contains (member_id,group_id,valid_period), but your query is only on the latter 2 fields. test=*# create index

Re: Performance

2018-02-23 Thread Andreas Kretschmer
Am 23.02.2018 um 20:29 schrieb Daulat Ram: We have the following requirements in single query or any proper solution. Please help on this. How many sessions are currently opened. ask pg_stat_activity, via select * from pg_stat_activity -and if opened then how many queries have

Re: OT: Performance of VM

2018-02-05 Thread Andreas Kretschmer
Am 05.02.2018 um 17:22 schrieb Andrew Kerber: Oracle has a problem with transparent hugepages, postgres may well have the same problem, so consider disabling transparent hugepages. yes, that's true. Regards, Andreas -- 2ndQuadrant - The PostgreSQL Support Company. www.2ndQuadrant.com

Re: OT: Performance of VM

2018-02-05 Thread Andreas Kretschmer
Am 05.02.2018 um 14:14 schrieb Thomas Güttler: What do you suggest to get some reliable figures? sar is often recommended, see https://blog.2ndquadrant.com/in-the-defense-of-sar/. Can you exclude other reasons like vacuum / vacuum freeze? Regards, Andreas -- 2ndQuadrant - The

Re: postgresql 10.1 wrong plan in when using partitions bug

2018-02-04 Thread Andreas Kretschmer
Am 04.02.2018 um 13:19 schrieb Mariel Cherkassky: I checked the plan of the next query : explain select count(*) from log_full where end_date between to_date('2017/12/03','YY/MM/DD') and to_date('2017/12/03','YY/MM/DD'); can you rewrite the query to ... where end_date between

Re: pg_xlog unbounded growth

2018-01-26 Thread Andreas Kretschmer
Hi, Am 24.01.2018 um 12:48 schrieb Stefan Petrea: We're using PostgreSQL 9.6.6 on a Ubuntu 16.04.3 LTS. During some database imports(using pg_restore), we're noticing fast and unbounded growth of pg_xlog up to the point where the partition(280G in size for us) that stores it fills up and

Re: 8.2 Autovacuum BUG ?

2018-01-23 Thread Andreas Kretschmer
Am 23.01.2018 um 16:20 schrieb Pavan Teja: Hi David, If it's yes what needs to be done in order to stabilize this issue?? Don't top-post ;-) You can't prevent the generation of wal's (apart from using unlogged tables, but i'm sure, that will be not your solution.) Regards, Andreas --

Re: Table with large number of int columns, very slow COPY FROM

2017-12-07 Thread Andreas Kretschmer
On 08.12.2017 05:21, Alex Tokarev wrote: I have made a minimally reproducible test case consisting of a table with 848 columns Such a high number of columns is maybe a sign of a wrong table / database design, why do you have such a lot of columns? How many indexes do you have? Regards,