[
https://issues.apache.org/jira/browse/YARN-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wang, Xinglong updated YARN-10735:
----------------------------------
Description:
With kerberos enabled, NPE will be reported when launching UnmanagedAMLauncher.
It is due to there is no AMRMToken is returned in ApplicationReport. After a
while investigation, it turns out that RMAppImpl has a bad if condition inside
createAndGetApplicationReport
{code:java}
21/04/14 02:46:01 INFO unmanagedamlauncher.UnmanagedAMLauncher: Initializing
Client
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Starting Client
21/04/14 02:46:02 INFO client.AHSProxy: Connecting to Application History
server at /0.0.0.0:10200
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting up
application submission context for ASM
21/04/14 02:46:02 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
to rm2
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting
unmanaged AM
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Submitting
application to ASM
21/04/14 02:46:03 INFO impl.YarnClientImpl: Submitted application
application_1618393442264_0002
21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Got application
report from ASM for, appId=2,
appAttemptId=appattempt_1618393442264_0002_000001, clientToAMToken=Token {
kind: YARN_CLIENT_TOKEN, service: }, appDiagnostics=AM container is launched,
waiting for AM container to Register with RM, appMasterHost=N/A, appQueue=abc,
appMasterRpcPort=-1, appStartTime=1618393562917, yarnAppState=ACCEPTED,
distributedFinalState=UNDEFINED, appTrackingUrl=N/A, appUser=abc
21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Launching AM
with application attempt id appattempt_1618393442264_0002_000001
21/04/14 02:46:04 FATAL unmanagedamlauncher.UnmanagedAMLauncher: Error running
Client
java.lang.NullPointerException
at
org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.launchAM(UnmanagedAMLauncher.java:186)
at
org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.run(UnmanagedAMLauncher.java:354)
at
org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.main(UnmanagedAMLauncher.java:111)
{code}
{code:java}
public ApplicationReport createAndGetApplicationReport(String clientUserName,
boolean allowAccess) {
......
if (currentAttempt != null &&
currentAttempt.getAppAttemptState() == RMAppAttemptState.LAUNCHED) {
if (getApplicationSubmissionContext().getUnmanagedAM() &&
clientUserName != null && getUser().equals(clientUserName)) {
Token<AMRMTokenIdentifier> token = currentAttempt.getAMRMToken();
if (token != null) {
amrmToken = BuilderUtils.newAMRMToken(token.getIdentifier(),
token.getKind().toString(), token.getPassword(),
token.getService().toString());
}
}
}
{code}
clientUserName is fullName of a kerberos principle like [email protected] whereas
getUser() will return the username recorded in RMAppImpl which is short name.
was:
With kerberos enabled, NPE will be reported when launching UnmanagedAMLauncher.
It is due to there is no AMRMToken is returned in ApplicationReport. After a
while investigation, it turns out that RMAppImpl has a bad if condition inside
createAndGetApplicationReport
{code:java}
21/04/14 02:46:01 INFO unmanagedamlauncher.UnmanagedAMLauncher: Initializing
Client
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Starting Client
21/04/14 02:46:02 INFO client.AHSProxy: Connecting to Application History
server at /0.0.0.0:10200
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting up
application submission context for ASM
21/04/14 02:46:02 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
to rm2
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting
unmanaged AM
21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Submitting
application to ASM
21/04/14 02:46:03 INFO impl.YarnClientImpl: Submitted application
application_1618393442264_0002
21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Got application
report from ASM for, appId=2,
appAttemptId=appattempt_1618393442264_0002_000001, clientToAMToken=Token {
kind: YARN_CLIENT_TOKEN, service: }, appDiagnostics=AM container is launched,
waiting for AM container to Register with RM, appMasterHost=N/A,
appQueue=hdmi-default, appMasterRpcPort=-1, appStartTime=1618393562917,
yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A,
appUser=b_carmel
21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Launching AM
with application attempt id appattempt_1618393442264_0002_000001
21/04/14 02:46:04 FATAL unmanagedamlauncher.UnmanagedAMLauncher: Error running
Client
java.lang.NullPointerException
at
org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.launchAM(UnmanagedAMLauncher.java:186)
at
org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.run(UnmanagedAMLauncher.java:354)
at
org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.main(UnmanagedAMLauncher.java:111)
{code}
{code:java}
public ApplicationReport createAndGetApplicationReport(String clientUserName,
boolean allowAccess) {
......
if (currentAttempt != null &&
currentAttempt.getAppAttemptState() == RMAppAttemptState.LAUNCHED) {
if (getApplicationSubmissionContext().getUnmanagedAM() &&
clientUserName != null && getUser().equals(clientUserName)) {
Token<AMRMTokenIdentifier> token = currentAttempt.getAMRMToken();
if (token != null) {
amrmToken = BuilderUtils.newAMRMToken(token.getIdentifier(),
token.getKind().toString(), token.getPassword(),
token.getService().toString());
}
}
}
{code}
clientUserName is fullName of a kerberos principle like [email protected] whereas
getUser() will return the username recorded in RMAppImpl which is short name.
> Unmanaged AM is won't populate AMRMToken to ApplicationReport in secure
> cluster
> -------------------------------------------------------------------------------
>
> Key: YARN-10735
> URL: https://issues.apache.org/jira/browse/YARN-10735
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Wang, Xinglong
> Assignee: Wang, Xinglong
> Priority: Minor
>
> With kerberos enabled, NPE will be reported when launching
> UnmanagedAMLauncher.
> It is due to there is no AMRMToken is returned in ApplicationReport. After a
> while investigation, it turns out that RMAppImpl has a bad if condition
> inside createAndGetApplicationReport
> {code:java}
> 21/04/14 02:46:01 INFO unmanagedamlauncher.UnmanagedAMLauncher: Initializing
> Client
> 21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Starting
> Client
> 21/04/14 02:46:02 INFO client.AHSProxy: Connecting to Application History
> server at /0.0.0.0:10200
> 21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting up
> application submission context for ASM
> 21/04/14 02:46:02 INFO client.ConfiguredRMFailoverProxyProvider: Failing over
> to rm2
> 21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Setting
> unmanaged AM
> 21/04/14 02:46:02 INFO unmanagedamlauncher.UnmanagedAMLauncher: Submitting
> application to ASM
> 21/04/14 02:46:03 INFO impl.YarnClientImpl: Submitted application
> application_1618393442264_0002
> 21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Got
> application report from ASM for, appId=2,
> appAttemptId=appattempt_1618393442264_0002_000001, clientToAMToken=Token {
> kind: YARN_CLIENT_TOKEN, service: }, appDiagnostics=AM container is
> launched, waiting for AM container to Register with RM, appMasterHost=N/A,
> appQueue=abc, appMasterRpcPort=-1, appStartTime=1618393562917,
> yarnAppState=ACCEPTED, distributedFinalState=UNDEFINED, appTrackingUrl=N/A,
> appUser=abc
> 21/04/14 02:46:04 INFO unmanagedamlauncher.UnmanagedAMLauncher: Launching AM
> with application attempt id appattempt_1618393442264_0002_000001
> 21/04/14 02:46:04 FATAL unmanagedamlauncher.UnmanagedAMLauncher: Error
> running Client
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.launchAM(UnmanagedAMLauncher.java:186)
> at
> org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.run(UnmanagedAMLauncher.java:354)
> at
> org.apache.hadoop.yarn.applications.unmanagedamlauncher.UnmanagedAMLauncher.main(UnmanagedAMLauncher.java:111)
> {code}
>
> {code:java}
> public ApplicationReport createAndGetApplicationReport(String clientUserName,
> boolean allowAccess) {
> ......
> if (currentAttempt != null &&
> currentAttempt.getAppAttemptState() ==
> RMAppAttemptState.LAUNCHED) {
> if (getApplicationSubmissionContext().getUnmanagedAM() &&
> clientUserName != null && getUser().equals(clientUserName)) {
> Token<AMRMTokenIdentifier> token = currentAttempt.getAMRMToken();
> if (token != null) {
> amrmToken = BuilderUtils.newAMRMToken(token.getIdentifier(),
> token.getKind().toString(), token.getPassword(),
> token.getService().toString());
> }
> }
> }
> {code}
> clientUserName is fullName of a kerberos principle like [email protected]
> whereas getUser() will return the username recorded in RMAppImpl which is
> short name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
