dataroaring opened a new issue, #55664: URL: https://github.com/apache/doris/issues/55664
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description Currently, Doris supports a lightweight delete mechanism where delete operations are recorded as predicates. - During query execution, delete predicates are applied for filtering. - The actual physical deletion is performed asynchronously by background compaction. This works well in most cases, but there is a performance and space utilization issue when a delete predicate effectively removes all data in a tablet/partition/table: - Queries still need to evaluate the delete predicate until compaction is triggered. - Storage space is not reclaimed until background compaction eventually runs. ### Solution When Doris detects that a delete predicate matches all data (e.g., DELETE FROM t WHERE 1=1 or a partition-level full delete), Doris could: - Trigger compaction immediately for the affected tablet(s). - Skip predicate filtering for subsequent queries (since all rows are logically deleted). - Reclaim disk space earlier to optimize storage usage. Benefits: - Improve query performance by avoiding unnecessary predicate evaluation. - Reduce storage space overhead by reclaiming space immediately. - Provide a better user experience for full-table or full-partition deletes. Use cases: - Partition lifecycle management (drop or refresh partitions by delete). - Large-scale data cleanup scenarios. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
