Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Andrei Lepikhov
On 5/12/25 20:07, Tom Lane wrote: Maxim Boguk writes: Reading the code - probably the lowest hanging fruit is to make 'The current multiplier of 1000 * cpu_operator_cost' configurable in the future versions. I'm wondering whether we should try to make the planner not expend the effort in the

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Tom Lane
Maxim Boguk writes: > Reading the code - probably the lowest hanging fruit is to make > 'The current multiplier of 1000 * cpu_operator_cost' configurable in the > future versions. I'm wondering whether we should try to make the planner not expend the effort in the first place, but leave partition

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Maxim Boguk
On Mon, May 12, 2025 at 6:33 PM Tom Lane wrote: > Maxim Boguk writes: > > And the problem is that the cost of a custom plan ignores the cost of > > planning itself (which is like 2x orders of magnitude worse than the cost > > of real time partition pruning of a generic plan). > > False. The est

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Andrei Lepikhov
On 12/5/2025 16:04, Maxim Boguk wrote: On Mon, May 12, 2025 at 4:48 PM Andrei Lepikhov It is not hard to write such a tiny extension. As I see, the only extra stored "C" procedure is needed to set up force-plan-type flag employing FetchPreparedStatement(). The rest of the code - query

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Tom Lane
Maxim Boguk writes: > And the problem is that the cost of a custom plan ignores the cost of > planning itself (which is like 2x orders of magnitude worse than the cost > of real time partition pruning of a generic plan). False. The estimate is evidently pretty wrong, but it's not that there is n

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Maxim Boguk
On Mon, May 12, 2025 at 6:01 PM David Rowley wrote: > On Mon, 12 May 2025, 05:08 Andrei Lepikhov, wrote: > >> Thanks for this puzzle! >> I suppose, in case generic planning is much faster than custom one, >> there are two candidates exist: >> 1. Touching the index during planning causes too much

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread David Rowley
On Mon, 12 May 2025, 05:08 Andrei Lepikhov, wrote: > Thanks for this puzzle! > I suppose, in case generic planning is much faster than custom one, > there are two candidates exist: > 1. Touching the index during planning causes too much overhead - see > get_actual_variable_range > 2. You have a m

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Maxim Boguk
On Mon, May 12, 2025 at 4:48 PM Andrei Lepikhov wrote: > On 5/12/25 15:08, Maxim Boguk wrote: > > PS: problem not with difference between custom and generic planning time > > but with prepared statements > > generic plan plans only once, but custom plan plan every call (and plan > > time cost 95%

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Maxim Boguk
On Mon, May 12, 2025 at 4:48 PM Andrei Lepikhov wrote: > On 5/12/25 15:08, Maxim Boguk wrote: > > PS: problem not with difference between custom and generic planning time > > but with prepared statements > > generic plan plans only once, but custom plan plan every call (and plan > > time cost 95%

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Andrei Lepikhov
On 5/12/25 15:08, Maxim Boguk wrote: PS: problem not with difference between custom and generic planning time but with prepared statements generic plan plans only once, but custom plan plan every call (and plan time cost 95% on total query runtime). Ah, now I got it. I'm aware of this problem f

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Maxim Boguk
On Mon, May 12, 2025 at 3:08 PM Andrei Lepikhov wrote: > On 5/12/25 13:49, Maxim Boguk wrote: > > I suspect this situation should be quite common with queries over > > partitioned tables (where planning time is usually quite a high). > > > > Any suggestions what could be done there outside of usi

Re: inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Andrei Lepikhov
On 5/12/25 13:49, Maxim Boguk wrote: I suspect this situation should be quite common with queries over partitioned tables (where planning time is usually quite a high). Any suggestions what could be done there outside of using force_generic_plan for a particular db user (which will kill perfor

inefficient/wrong plan cache mode selection for queries with partitioned tables (postgresql 17)

2025-05-12 Thread Maxim Boguk
Hi, I found a case where plan cache all time switching to custom plans forces query replan each call (and thus slows down the whole query for 10x or more). What makes the situation intriguing - that both custom and generic plans are the same. job_stats_master - partitioned table with 24 partitions