Attached is a redesigned version. While working on the previous design, I grew increasingly uncomfortable with CachedPlanPrepData -- it was smuggling executor state out of GetCachedPlan() through an out-parameter, which papered over the real problem: GetCachedPlan() was doing too much. The main change in this version is architectural: GetCachedPlan() no longer acquires execution locks. Callers now own that responsibility, which is natural because each call site iterates stmt_list differently and manages execution state in its own way -- and it lets them choose between conservative lock-all and pruning-aware locking where appropriate.
Non-portal call sites remain on the conservative path for now. _SPI_execute_plan requires care around snapshot setup, which happens after plan fetch rather than before. SQL functions have a different issue: init_execution_state() fetches the plan while postquel_start() handles execution, with execution_state containers in between, making it harder to thread a prepped QueryDesc through. The portal path and EXPLAIN EXECUTE cover the most common prepared-statement-with-partitions workloads; the remaining sites can be converted incrementally. This is now starting to feel closer to what Tom suggested back in January 2023 [1], where he proposed getting rid of AcquireExecutorLocks() inside GetCachedPlan() entirely and pushing lock acquisition out to callers. He noted that "we'd be pushing the responsibility for looping back and re-planning out to fairly high-level calling code" and that "we'd definitely be changing some fundamental APIs." That is the direction I came around to over the last couple of weeks while wrestling with CachedPlanPrepData. The reverted approach also tried to follow Tom's direction but moved locking into ExecutorStart(), which forced it to handle plan invalidation from inside the executor by mutating the CachedPlan in-place. This version moves locking out to the callers instead, so the executor and plan cache never reach into each other. The series is now four patches: 0001: Move execution lock acquisition out of GetCachedPlan(). Adds AcquireExecutorLocks() as a caller-facing function with validity check and retry. Adds PortalLockCachedPlan() in pquery.c to centralize the portal retry logic. All callers are converted. No behavioral change. 0002: Refactor executor's initial partition pruning setup. Cleanup only, no behavioral change. 0003: Introduce ExecutorPrep() and refactor executor startup. Factors range table init, permission checks, and initial pruning out of InitPlan(). Scaffolding for 0004; all callers still go through the normal ExecutorStart() path. 0004: Use pruning-aware locking for single-statement cached plans. Adds ExecutorPrepAndLock() which locks unprunable relations, runs ExecutorPrep() to determine surviving partitions, then locks only those. Extends PortalLockCachedPlan() with a pruning-aware path for eligible plans. Multi-statement CachedPlans (from rule rewriting) always use conservative locking. In principle, this could be relaxed if the planner can prove that no pruning expression reads state modified by an earlier statement, but that is left for a future patch. Includes regression tests. In case it's not clear, I'm not targeting v19 at this point. I'd like to get this into v20 CF1 and would welcome review from anyone interested. -- Thanks, Amit Langote [1] https://www.postgresql.org/message-id/4191508.1674157166%40sss.pgh.pa.us
v11-0004-Use-pruning-aware-locking-for-single-statement-c.patch
Description: Binary data
v11-0003-Introduce-ExecutorPrep-and-refactor-executor-sta.patch
Description: Binary data
v11-0001-Move-execution-lock-acquisition-out-of-GetCached.patch
Description: Binary data
v11-0002-Refactor-executor-s-initial-partition-pruning-se.patch
Description: Binary data
