2010YOUY01 commented on issue #24157: URL: https://github.com/apache/datafusion/issues/24157#issuecomment-5224269894
Thank you for driving this feature! I'm also curious how many people find this algorithm easy to reason about. I tried to understand it before (with the paper + duckdb blog), but after spending several hours on it, I still didn't fully understand it, and don't have the confidence to implement it in datafusion. In particular, some of the bitmask construction/manipulation steps were difficult for me to follow. This is probably the only operator I've worked with that I still find hard to fully understand. If the algorithm is inherently this tricky, I think there may be a real long-term maintenance cost. Piecewise merge join might be fast enough and also way easier to understand. That said, it's also possible that IEJoin is easy if explained from the right angle, I'm wondering do you have a clear explanation, and I'm interested in seeing it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
