nchammas commented on PR #57313: URL: https://github.com/apache/spark/pull/57313#issuecomment-5029639134
> I believe this requires an SPIP because it's a pretty large new feature. That's what I expected, but wanted to confirm with a committer. I will write one up. Is there a PMC member you think might be interested in being the [shepherd](https://spark.apache.org/improvement-proposals.html)? > If we do this, we need to eventually replace the old scripts, instead of having two sets of code that do the same thing. I agree, and that is something I mentioned in my [initial email to the dev list][email] about this idea. [email]: https://lists.apache.org/thread/hby7fw6kxbzk6szzmgyocw662415ls6r > If we can't do that, then a fallback option would be having a CLI just for spark developers. Users can still use the shell scripts. My motivation for proposing a unified CLI is specifically to help users and make various features of Spark more discoverable. I am not as attracted to the idea of a new CLI just for Spark developers. > Unless we are creating a CLI without replacing the old one - then I'm just against the idea because it does not make sense. I think having the new CLI be a wrapper around the existing scripts will be a transitional phase and will allow us to merge the various scripts into the CLI bit by bit, rather than having to do everything all at once. I expect at least some committers will express hesitation about rewriting old and stable scripts. This will be part of the SPIP discussion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
