gaogaotiantian commented on PR #57313: URL: https://github.com/apache/spark/pull/57313#issuecomment-5028032445
I did express interest in this idea and I think it should be bigger than the original thought. I believe this requires an SPIP because it's a pretty large new feature. If we do this, we need to eventually replace the old scripts, instead of having two sets of code that do the same thing. That will raise a problem - making python a hard requirement for using spark. Admittedly, python is so popular and almost every developer has some version of python on their computer, but making it a requirement for using spark is another story. If we can move past this blocker, then we should be able to make a much better CLI for spark usage. I don't like shell scripts in general. If we can't do that, then a fallback option would be having a CLI just for spark developers. Users can still use the shell scripts. That should be an easier push. In any case, I believe features like this need an SPIP and a vote from the community. Unless we are creating a CLI without replacing the old one - then I'm just against the idea because it does not make sense. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
