[
https://issues.apache.org/jira/browse/HADOOP-15722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625579#comment-16625579
]
Brahma Reddy Battula commented on HADOOP-15722:
-----------------------------------------------
bq. Looks like in the following log snippets for the good and bad runs, the
user.name system property is not getting expanded in the bad run because the
conf resource is untrusted:
Yes,Our Spark Team also faced the same issue and analysed the issue. Please
check following for same.
if *hive hive.exec.scratchdir* is set to */tmp/hive-${user.name}* when we can
see the effect
hive dosAs is default true, so spark will execute hive as is if its proxy user
First export proxy user to user_a
when *user_a* creates a scratch dir, he will create
{color:#205081}*/tmp/hive-*{color}{color:#d04437}*$%7Buser.name%7D*{color}
(failure to replace system property in path in 2.7.7) instead of
/tmp/hive-user_a (replace system property in path in 2.7.3)
Second export proxy user to user_b
when *user_b* tries to create scratch dir,{color:#FF0000} *he will fail as he
will also try to create in
{color:#205081}/tmp/hive-{color}$%7Buser.name%7D*{color} instead of
{color:#14892c}*/tmp/hive-user_b* {color}
IMO,Usually we'll configure system properties in xml's. So I feel,we should
allow system properties(option2 should be fine?).
Verified with "allowing system properties", it was working fine.
This should be blocker for 3.2 release..? If you all agree, we can mark this
as blocker.
> regression: Hadoop 2.7.7 release breaks spark submit
> ----------------------------------------------------
>
> Key: HADOOP-15722
> URL: https://issues.apache.org/jira/browse/HADOOP-15722
> Project: Hadoop Common
> Issue Type: Bug
> Components: build, conf, security
> Affects Versions: 2.7.7
> Reporter: Steve Loughran
> Priority: Major
>
> SPARK-25330 highlights that upgrading spark to hadoop 2.7.7 is causing a
> regression in client setup, with things only working when
> {{Configuration.getRestrictParserDefault(Object resource)}} = false.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
