Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/2126#issuecomment-53575210
thanks for investigating. I don't think LOGNAME should really affect it.
I thought it would use the Subject that has been created when you do the
createRemoteuser and the doAs. When you say "I'm printing logs at the first of
security manager." what do you mean?
One other note is that this patch is fragile because is someone creates a
UGI before the HADOOP_USER_NAME is set then it will be ignored. I believe it
only does the commit() routine at the first time the UGI is created.
Can you please clarify exactly how things are running:
What user are the spark processes running on mesos running as?
What user is the Hadoop cluster running as?
The reason I want this clarified is above you said "14/08/27 01:11:01 DEBUG
UserGroupInformation: using local user:UnixPrincipal: hdfs" which seems to
indicate that the spark processes are running as user 'hdfs' not the user
'mesos' like I thought.
Can you put some log statements in various places to see what the user is?
For instance inside of the doAS in runAsUser can you log what the
UserGroupInformation.getCurrentUser() is?
Could you also perhaps put some log statements in
Executor.createClassLoader() to make sure the classloader isn't messing things
up. Also perhaps in a few places like in MesosExecutorBackend.launchTask,
Executor.launchTask, and TaskRunner.run.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
