+1 -Murtadha ________________________________ From: Hongyu Shi <h...@scu.edu.INVALID> Sent: Tuesday, October 29, 2024 2:58:20 AM To: dev@asterixdb.apache.org <dev@asterixdb.apache.org> Subject: Dynamic Workload Management
Hi all, Initiating the discussion thread proposing new dynamic workload management for query scheduling. *Feature:* New query scheduler with dynamic workload management *Details:* The old scheduler can lead resource-intensive jobs blocking shorter ones under the basic FIFO policy. Users also have limited control over the execution order of queries while need for queries with different levels of importance could be common. The new scheduler allows the system to prioritize queries according to user-defined classifications and priorities. Additionally it also ensures that short, low-resource queries are prioritized, while also maintaining fair scheduling by balancing wait times relative to execution times. An openly typed metadata collection may be used for configuring query groups and their priorities. A DDL/DML syntax to manipulate the metadata collection and statement syntax indicating groups and hints in queries are under discussion. They will be described in another APE proposal. *APE*: https://cwiki.apache.org/confluence/display/ASTERIXDB/APE+13%3A+Dynamic+Workload+Management
