Phoenix500526 commented on PR #24923: URL: https://github.com/apache/datafusion/pull/24923#issuecomment-5570239768
Hi, @jayzhan211 , after rebasing this PR, I've pushed three follow-up commits: * `29bab9cb0a` applies your suggestion: removes the timeout from drop cleanup, keeps the concurrency limit, and docs that backends should bound their own abort latency. It also declares the `tokio::time` feature. * `7b7a396343` is a very simple modification that fixes a clippy warning in the PostgreSQL test helper by throwing `BigDecimal` instead of taking ownership. * `33f4ab2c8e8` adapts the async spill path to the retained workspace introduced in PR #24740. It replaces unchecked reservation growth with borrowing from available workspace, while keeping the output batch accounted for throughout the async write. It also writes each batch before requesting the next one and releases fully consumed merge inputs so their budget can be reused. The tests cover insufficient memory, reservation retention during pending writes, and cleanup after cancellation. The remaining CI check, `cargo test hash collisions(amd64)`, was canceled after reaching the six-hour limit. This appears unrelated to the spill changes: I found the same count-distinct tests taking hours and hitting the timeout on `main` and other branches. I've opened #25011 with the logs and reproduction steps to track this and ask about the intended test coverage before proposing any changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
