Github user CodingCat commented on the pull request:
https://github.com/apache/spark/pull/8610#issuecomment-180629018
Hi, @andrewor14, when I had a second think about your suggestion that we
shall put parameter checking in SparkSubmit, I found that it may not be the
right way to do this
Because the users are always able to set the parameters by code, instead of
through spark-submit. Reading the parameters set in a programmable way happens
after reading the parameters set through spark-submit. In this case, we have to
check whether the parameters are making sense right before we use them to
create the application descriptor, e.g. ApplicationDescription in standalone
mode. Otherwise, we may miss the parameters which are set after spark-submit
parameter reading...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
