[
https://issues.apache.org/jira/browse/HDDS-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943818#comment-16943818
]
Attila Doroszlai commented on HDDS-2230:
----------------------------------------
Thanks [~xyao] for checking and finding the cause of YarnException. With patch
002 the test fails for me with a different root cause:
{noformat}
rm_1 | 2019-10-03 17:57:00 WARN DelegationTokenRenewer:935 - Unable to
add the application to the delegation token renewer.
rm_1 | java.io.IOException: Failed to renew token: Kind:
TIMELINE_DELEGATION_TOKEN, Service: 192.168.32.9:8188, Ident:
(TIMELINE_DELEGATION_TOKEN owner=hadoop, renewer=rm, realUser=,
issueDate=1570125419747, maxDate=1570730219747, sequenceNumber=2, masterKeyId=2)
rm_1 | at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:503)
rm_1 | at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.access$900(DelegationTokenRenewer.java:79)
rm_1 | at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.handleDTRenewerAppSubmitEvent(DelegationTokenRenewer.java:931)
rm_1 | at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.run(DelegationTokenRenewer.java:908)
rm_1 | at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
rm_1 | at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
rm_1 | at java.lang.Thread.run(Thread.java:748)
rm_1 | Caused by: java.io.IOException: java.net.URISyntaxException:
Illegal character in hostname at index 21:
http://ozonesecure-mr_jhs_1.ozonesecure-mr_default:8188/ws/v1/timeline/
rm_1 | at
org.apache.hadoop.yarn.client.api.impl.TimelineConnector$TimelineClientRetryOpForOperateDelegationToken.run(TimelineConnector.java:433)
rm_1 | at
org.apache.hadoop.yarn.client.api.impl.TimelineConnector$TimelineClientConnectionRetry.retryOn(TimelineConnector.java:334)
rm_1 | at
org.apache.hadoop.yarn.client.api.impl.TimelineConnector.operateDelegationToken(TimelineConnector.java:218)
rm_1 | at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.renewDelegationToken(TimelineClientImpl.java:250)
rm_1 | at
org.apache.hadoop.yarn.security.client.TimelineDelegationTokenIdentifier$Renewer.renew(TimelineDelegationTokenIdentifier.java:81)
rm_1 | at org.apache.hadoop.security.token.Token.renew(Token.java:495)
rm_1 | at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$1.run(DelegationTokenRenewer.java:629)
rm_1 | at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$1.run(DelegationTokenRenewer.java:626)
rm_1 | at java.security.AccessController.doPrivileged(Native Method)
rm_1 | at javax.security.auth.Subject.doAs(Subject.java:422)
rm_1 | at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
rm_1 | at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.renewToken(DelegationTokenRenewer.java:625)
rm_1 | at
org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:489)
rm_1 | ... 6 more
rm_1 | Caused by: java.net.URISyntaxException: Illegal character in
hostname at index 21:
http://ozonesecure-mr_jhs_1.ozonesecure-mr_default:8188/ws/v1/timeline/
rm_1 | at java.net.URI$Parser.fail(URI.java:2848)
rm_1 | at java.net.URI$Parser.parseHostname(URI.java:3387)
rm_1 | at java.net.URI$Parser.parseServer(URI.java:3236)
rm_1 | at java.net.URI$Parser.parseAuthority(URI.java:3155)
rm_1 | at java.net.URI$Parser.parseHierarchical(URI.java:3097)
rm_1 | at java.net.URI$Parser.parse(URI.java:3053)
rm_1 | at java.net.URI.<init>(URI.java:673)
rm_1 | at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$2.run(TimelineClientImpl.java:245)
rm_1 | at
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$2.run(TimelineClientImpl.java:227)
rm_1 | at java.security.AccessController.doPrivileged(Native Method)
rm_1 | at javax.security.auth.Subject.doAs(Subject.java:422)
rm_1 | at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
rm_1 | at
org.apache.hadoop.yarn.client.api.impl.TimelineConnector$TimelineClientRetryOpForOperateDelegationToken.run(TimelineConnector.java:431)
rm_1 | ... 18 more
rm_1 | 2019-10-03 17:57:00 INFO RMAppImpl:1310 - Updating application
application_1570125337120_0002 with final state: FAILED
{noformat}
This can be fixed by avoiding {{_}} in hostname for Timeline History Server.
I'll post a new patch soon.
> Invalid entries in ozonesecure-mr config
> ----------------------------------------
>
> Key: HDDS-2230
> URL: https://issues.apache.org/jira/browse/HDDS-2230
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: docker
> Reporter: Attila Doroszlai
> Assignee: Attila Doroszlai
> Priority: Minor
> Attachments: HDDS-2230.001.patch, HDDS-2230.002.patch
>
>
> Some of the entries in {{ozonesecure-mr/docker-config}} are in invalid
> format, thus they end up missing from the generated config files.
> {noformat}
> $ cd hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozonesecure-mr
> $ ./test.sh # configs are generated during container startup
> $ cd ../..
> $ grep -c 'ozone.administrators' compose/ozonesecure-mr/docker-config
> 1
> $ grep -c 'ozone.administrators' etc/hadoop/ozone-site.xml
> 0
> $ grep -c 'yarn.timeline-service' compose/ozonesecure-mr/docker-config
> 5
> $ grep -c 'yarn.timeline-service' etc/hadoop/yarn-site.xml
> 2
> $ grep -c 'container-executor' compose/ozonesecure-mr/docker-config
> 3
> $ grep -c 'container-executor' etc/hadoop/yarn-site.xml
> 0
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
