Thanks Zhuojun for introducing this improvement! This proposal brings many benefits to the AMS service and improves its throughput and performance. I strongly support moving forward with its implementation, so here’s a huge +1 from me.
Best Regards, Xu Jzjsnow <5797...@qq.com.invalid> 于2025年8月14日周四 20:56写道： > Hi devs, > > > We would like to start a discussion about AIP-3: Event-Triggered > Optimization of Iceberg Tables in Amoro [1]. > > > Currently, Amoro uses a periodic scan mechanism to evaluate and optimize > Iceberg tables. However, with the increasing size of tables and frequent > data changes, this periodic approach can lead to inefficiencies, including > redundant scans and resource wastage. These inefficiencies can ultimately > cause system delays, reduced performance, and scalability limitations, > negatively impacting the user experience. > > > In this proposal, we introduce an event-driven optimization mechanism that > triggers optimization evaluation based on the loaded table metadata/metrics > or Iceberg commit operations. > > > The key goals of this proposal include: > > Reducing unnecessary full-table scans. > > > Improving scan efficiency by triggering scans for tables that need > optimization based on table metadata. > > > Enhancing system performance and reducing resource > consumption for large-scale tables. > > > > For ease of discussion, the Google Doc for this AIP is provided in > [2]. We are excited to hear your thoughts, feedback, and any concerns you > may have about this new approach. > > > Looking forward to your input! > > > [1] > https://cwiki.apache.org/confluence/display/AMORO/AIP-3%3A+Event-Triggered+Optimization+of+Iceberg+Tables+in+Amoro > [2] > https://docs.google.com/document/d/1vNdMPjnKZeGukEFBrJLoIdXILT7HXflkBYSJpEqzkM4/edit?usp=sharing > > > Best regards, > Zhuojun > > > Jzjsnow > 5797...@qq.com
