pantShrey commented on PR #21882: URL: https://github.com/apache/datafusion/pull/21882#issuecomment-4488665457
Hey @alamb, quick update! I've reworked both PRs to make them easier to review independently: - **#22230 (SMJ refactor)** no longer depends on this PR, it now migrates SortMergeJoin to async spill abstractions while keeping the concrete `RefCountedTempFile` type, so it can be reviewed and merged standalone. - **This PR** now only does one focused thing: introduces the `SpillFile` trait + `TempFileFactory` and swaps the internal type from `RefCountedTempFile` → `Arc<dyn SpillFile>` and internal changes in the spill module. I've also removed the `open_sync` tech debt that was here before, and have added the skip validation in streamdecoder The plan is for #22230 to merge first, then I'll rebase this on top of it. Would be grateful if you could take a look whenever you get the chance! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
