Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/1715#issuecomment-50982344
@andrewor14 Interesting, actually this is almost exactly the same solution
I came across at the very beginning :)
The only difference is that we chose `--` rather than some more intuitive
option name like `--spark-application-args`. And `--` was chosen because it's
an idiomatic way among UNIX-like systems to pass this kind of "user application
options".
The reason that we (@pwendell and me) gave it up after discussion is that
this solution is actually not fully downward compatible, it breaks existing
user applications which already recognize `--` as a valid option. Turning `--`
into something more specific like `--spark-application-args` does reduce the
probability of name collision. Especially, after this change, we won't have
similar compatibility issue whenever we add any new options to `spark-submit`
in the future. @pwendell Maybe this is acceptable?
And I agree with your arguments about the drawbacks of putting application
jar into `--jars`. Similar arguments applies to Python application. That is
also an important reason that I introduced `--primary` at the first place.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
