2010YOUY01 opened a new issue, #26361: URL: https://github.com/apache/datafusion/issues/26361
### Is your feature request related to a problem or challenge? Part of - https://github.com/apache/datafusion/issues/26225 See issue for general background, below is HJ-specific rationales. Hash Join operator has accumulated certain complexity, it's easier to proceed with future feature/improvements after a refactor. Specifically, when we want to add spilling, or existence join fast path - https://github.com/apache/datafusion/issues/24768 Then regular joins and existence joins have sufficiently different optimization opportunities, or implementation logic, so that splitting them is simpler than reusing the same state machine. ### Describe the solution you'd like Incremental rewrite HJ into 2 dedicated paths 1. https://github.com/apache/datafusion/pull/26227 2. Implement the existence join path ### Describe alternatives you've considered _No response_ ### Additional context In the existing implementation: - NLJ and HJ: reused execution state machine for existence and regular join. NLJ has implemented spilling and some existence join fast path, I think that causes unnecessary complexity -- lots of local flag are used to route the execution for different join logic - SMJ: already split due to the same concern ### Willingness to contribute None -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
