Hi all, I’d like to open discussion on AIP-5, which proposes opt-in Dynamic Resource Allocation (DRA) for optimizer resource groups.
AIP-5: https://cwiki.apache.org/confluence/display/AMORO/AIP-5%3A+Dynamic+Resource+Allocation+for+Optimizer Issue: https://github.com/apache/amoro/issues/4191 Optimizer groups today run at a fixed parallelism: min-parallelism scales up to a floor but never down, so pods sit idle during low load and operators must manually rescale for spikes. AIP-5 adds an opt-in DRA mode (modeled on Spark’s SPARK-3174) that scales optimizers up and down with demand, with graceful drain, all inside AMS. With dynamic-allocation.enabled=false (default), behavior is unchanged and no migration is needed. The wiki page has the full design — algorithm, config properties, metrics, diagrams, phased plan, and rejected alternatives. Feedback and questions welcome. I’d love to hear your thoughts on this direction. Thanks, Jiwon Park
