Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/6052#issuecomment-100917675
See the constructors:
=> first calls this one:
public Job(Configuration conf) throws IOException {
this(new JobConf(conf));
} => calls below constructor
Job(JobConf conf) throws IOException {
super(conf, null);
// propagate existing user credentials to job
this.credentials.mergeAll(this.ugi.getCredentials());
this.cluster = null;
}
It merges what is in the current ugi credentials into the Job. The Job
this.credentials is just pointing to the confs credentials (
this.credentials = this.conf.getCredentials();) so when we call the
job.getConfiguration it should return the conf populated with the credentials.
This should be exactly the same as what you have coded up. Well except for
its doing the merge twice now.
I would really like this person to respond with the exact conditions this
happened and stack trace. Have you reproduced it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
