grundprinzip commented on PR #38793: URL: https://github.com/apache/spark/pull/38793#issuecomment-1328671269
> > Maybe we can do Project[*, add columns]? > > The semantic is different. `df.withColumns` can replace columns if the given name(s) conflict with the existing columns. `Project[*, added columns]` can't do it. Yes `Project[*, added columns]` cant do that, but you're able to replace the columns on the client side by resolving the schema first. What I'm trying to figure out is where we draw the line between adding another input relation type to avoid become a different Py4j. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
