alamb commented on PR #8991: URL: https://github.com/apache/arrow-datafusion/pull/8991#issuecomment-1913244756
> Note that "investigate removing projection before join input" is not a necessary step for join support in Comet, because Comet doesn't depend on DataFusion's query planning. But definitely we have other steps to work on DataFusion Join. That makes sense. I think having DataFusion planned joins and Spark planned joins going through different codepaths not only potentially duplicates code it makes it more lkely there are discrepancies that may lead to bugs (e.g. it may be hard to test the spark path as part of the datafusion tests) For that reason I suggest we make a ticket listing the overall project where we can collect other follow on steps as we come on tehem (not that you or Comet people will actually do the work, other contributors are likely to be able to help too I suspect). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
