alamb commented on issue #26225: URL: https://github.com/apache/datafusion/issues/26225#issuecomment-6142937591
Thank you for this writeup @2010YOUY01 -- your heuristics 1-5 read to me like a very concise summary of how to write understandable Rust code in general 💯 The biggest danger for refactoring projects like this in the past is that they get mostly implemented, but then are not quite completed (typically because the last bits are the most complicated and the author gets pulled away on something else that is higher priority) I believe the most effective way to avoid this fate is to structure the project with a non-refactoring goal and the refactoring becomes a step towards that ultimate goal. Also the refactoring serves a double purpose of getting the author more familiar with the code before adding the new feature. For example, in my mind the refactoring of Hash aggregates is driven by the desire to get blocked memory management May I suggest that the refactoring on Hash Join be driven by trying to implement Splling Hash Join (e.g. https://github.com/apache/datafusion/issues/24768) ? > My next two targets for similar refactoring are HJ and NLJ. If anyone has some review capacity and is interested in this idea for a specific operator, please let me know. I'd be happy to implement the main structural changes myself, and then help review further refinements if others are interested in contributing. I think @jayzhan211 was looking for someone to help with Join related work -- perhaps he is interested in helping this project -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
