Hi everyone, I would love to start discussion on the initial dynamic memory resource broker prototype. AsterixDB currently assigns fixed memory budgets to memory-intensive operators (join, sort, group-by) at compile time, and these budgets never change during execution. If one operator is holding unused memory while another is spilling to disk, there is no mechanism to redistribute it.
This proposal introduces a per-NC memory resource broker that dynamically reallocates memory between operators at runtime. The design covers the broker interface, a two-stage victimization system (query and operator level), and integration plans for existing operators. APE: https://cwiki.apache.org/confluence/spaces/ASTERIXDB/pages/449282523/APE+35+Dynamic+Operator+Memory+Broker Looking forward to your feedback! Regards, Alexander Anokhin
