Thanks for your suggestion. Indeed, if we set the default quota for tables to a fixed value, it is still possible to lead to a situation where a single table monopolizes all optimizer resources when there are very few optimizer resources. Conversely, when optimizer resources are abundant, it may result in consistently low optimizing efficiency for the table having multiple self-optimizing tasks, and cause inefficient utilization of optimizer resources. Therefore, setting the quota as a percentage to all available optimizer resources is a more appropriate approach. I will proceed with the corresponding modifications accordingly.
Best, XixiChen Jinsong Zhou <jinsongz...@apache.org> 于2025年7月22日周二 18:01写道： > Hi, > > Thanks for bringing up this improvement. > > This improvement is indeed valuable. In my production practice, we once > encountered a situation where a single table suddenly consumed all > resources, causing all other tables to enter a pending status. We need the > capability to limit the maximum resources a single table can use. > > However, I'd like to discuss how to set the default quota for tables. I > believe in most cases, users won't configure individual quotas for each > table, making default quotas particularly important. Rather than using a > fixed value, a percentage might be more appropriate - for example, 50%, > indicating a table can only consume half of the entire group's resources. > > Best, > Jinsong > > > > On Tue, Jul 22, 2025 at 5:52 PM 陈xx <cxxiii8...@gmail.com> wrote: > > > Hi devs: > > > > We would like to start a discussion about AIP-1: Optimizing Allocation > and > > Schedule Priority of Optimizer resources for Tables[1]. > > > > An optimizer group comprises a collection of optimizers, where each > > optimizer instance typically contains multiple threads, with each > optimizer > > thread responsible for executing a single optimizing task. When multiple > > self-optimizing tasks are pending and optimizer resources are limited, > > tasks originating from the same table may monopolize all available > > resources in the absence of proper constraints. > > So, we propose to optimize allocation and schedule priority of > > optimizer resources for tables. > > Looking forward to hearing from you. > > > > [1] https://cwiki.apache.org/confluence/x/bQ5JFg > > > > Best > > XixiChen > > >
