Hi all,
All phases of AIP-5 (Dynamic Resource Allocation for Optimizer) have now been merged, so I’d like to close out this thread. AIP-5: https://cwiki.apache.org/confluence/display/AMORO/AIP-5%3A+Dynamic+Resource+Allocation+for+Optimizer Issue: https://github.com/apache/amoro/issues/4191 #4254 (AMORO-4253) — Phase 1: configuration and validation #4272 (AMORO-4271) — Phase 2: demand-driven scale-up #4296 (AMORO-4295) — Phase 3+4: idle-driven scale-down with graceful drain #4314 (AMORO-4313) — Phase 5: observability metrics As proposed, DRA is opt-in: with dynamic-allocation.enabled=false (the default), behavior is unchanged and no migration is needed. The wiki page has been updated to Implemented, with the PR list recorded there. User documentation shipped alongside the corresponding PRs. Many thanks to Jinsong Zhou, ConradJam, and Xu Bai for the reviews and design feedback throughout, and to everyone who commented on the original proposal. Feedback from production use is welcome — please open an issue if you hit anything unexpected. Thanks, Jiwon Park > 2026. 6. 5. 오후 9:43, Jiwon Park <[email protected]> 작성: > > Hi all, > > I’d like to open discussion on AIP-5, which proposes opt-in Dynamic Resource > Allocation (DRA) for optimizer resource groups. > > AIP-5: > https://cwiki.apache.org/confluence/display/AMORO/AIP-5%3A+Dynamic+Resource+Allocation+for+Optimizer > Issue: https://github.com/apache/amoro/issues/4191 > > Optimizer groups today run at a fixed parallelism: min-parallelism scales up > to a floor but never down, so pods sit idle during low load and operators > must manually rescale for spikes. AIP-5 adds an opt-in DRA mode (modeled on > Spark’s SPARK-3174) that scales optimizers up and down with demand, with > graceful drain, all inside AMS. With dynamic-allocation.enabled=false > (default), behavior is unchanged and no migration is needed. > > The wiki page has the full design — algorithm, config properties, metrics, > diagrams, phased plan, and rejected alternatives. > > Feedback and questions welcome. I’d love to hear your thoughts on this > direction. > > Thanks, > Jiwon Park
