andygrove commented on code in PR #6698: URL: https://github.com/apache/datafusion-comet/pull/6698#discussion_r4195262237
########## docs/source/user-guide/latest/tuning/memory.md: ########## @@ -282,3 +282,20 @@ use more than `N` times this value on shared local disks. If the limit is reache fail and the query errors out. Raise this on workloads with large sort/aggregate/shuffle spills, or lower it to protect executors on shared disks, remembering that the total across an executor is a multiple of this value. + +## Compressing Spill Files + +Native sorts, aggregations, and sort-merge joins spill through DataFusion, which writes spill files Review Comment: DataFusion 55's `NestedLoopJoinExec` also spills through the same `SpillManager` when its build side doesn't fit, and Comet's native broadcast nested loop join is on by default. Could this list nested loop joins too? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
