hvanhovell commented on PR #41342: URL: https://github.com/apache/spark/pull/41342#issuecomment-1568041469
@beliefer thanks for doing this. I will take a look :)... Architecturally I think we need to move to a situation where we do not create Dataset/Dataframes in the planner at all. It should all be logical plans. The only place where we may need a Dataset/Dataframe is for execution. The reason for this is that I want the two implementations share as much code as possible by making connect the primary API, and have the current API an extension of that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
