[
https://issues.apache.org/jira/browse/FLINK-14207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Lin updated FLINK-14207:
-----------------------------
Description:
I built flink with hadoop 2.6.5 and docker image `{{maven:3-jdk-8-alpine`,
which failed multiple times with the same exception that YarnMiniCluster failed
to start in secure mode. I've successfully run the tests with the previous
versions including 1.5-1.8, and it's wired that with 1.9 it failed with an
error that probrably relates to Hadoop setup. }}
{{The stack trace is as follow:}}
{{```}}
06:36:37,275 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
- Transitioned to standby state Exception in thread "RM-0"
java.lang.RuntimeException: Could not read HTTP signature secret file:
/tmp/junit4738785782616411228/junit7110292436248761811/hadoop-http-auth-signature-secret
at
org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.createFilterConfig(RMAuthenticationFilterInitializer.java:100)
at
org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.initFilter(RMAuthenticationFilterInitializer.java:130)
at
org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:395) at
org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:346) at
org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:105) at
org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:293) at
org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:260) at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:990)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1090)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.MiniYARNCluster$2.run(MiniYARNCluster.java:310)
06:36:38,401 INFO org.apache.hadoop.service.AbstractService - Service
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 failed
in state STARTED; cause:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
ResourceManager failed to start. Final state is STOPPED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
ResourceManager failed to start. Final state is STOPPED at
org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:328)
at
org.apache.hadoop.yarn.server.MiniYARNCluster.access$500(MiniYARNCluster.java:94)
at
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceStart(MiniYARNCluster.java:448)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.flink.yarn.YarnTestBase.start(YarnTestBase.java:621) at
org.apache.flink.yarn.YarnTestBase.startYARNSecureMode(YarnTestBase.java:577)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:89)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:86)
at java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase.setup(YARNSessionFIFOSecuredITCase.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
org.junit.rules.RunRules.evaluate(RunRules.java:20) at
org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: java.io.IOException: ResourceManager failed to start. Final state is
STOPPED at
org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:323)
... 34 more
```
was:
I built flink with hadoop 2.6.5 and docker image `{{maven:3-jdk-8-alpine`,
which failed multiple times with the same exception that YarnMiniCluster failed
to start in secure mode. I've successfully run the tests with the previous
versions including 1.5-1.8, and it's wired that with 1.9 it failed with an
error that probrably relates to Hadoop setup. }}
{{}}
{{The stack trace is as follow:}}
{{}}
{{```}}
06:36:37,275 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
- Transitioned to standby state Exception in thread "RM-0"
java.lang.RuntimeException: Could not read HTTP signature secret file:
/tmp/junit4738785782616411228/junit7110292436248761811/hadoop-http-auth-signature-secret
at
org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.createFilterConfig(RMAuthenticationFilterInitializer.java:100)
at
org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.initFilter(RMAuthenticationFilterInitializer.java:130)
at
org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:395) at
org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:346) at
org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:105) at
org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:293) at
org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:260) at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:990)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1090)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.MiniYARNCluster$2.run(MiniYARNCluster.java:310)
06:36:38,401 INFO org.apache.hadoop.service.AbstractService - Service
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 failed
in state STARTED; cause:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
ResourceManager failed to start. Final state is STOPPED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
ResourceManager failed to start. Final state is STOPPED at
org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:328)
at
org.apache.hadoop.yarn.server.MiniYARNCluster.access$500(MiniYARNCluster.java:94)
at
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceStart(MiniYARNCluster.java:448)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.flink.yarn.YarnTestBase.start(YarnTestBase.java:621) at
org.apache.flink.yarn.YarnTestBase.startYARNSecureMode(YarnTestBase.java:577)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:89)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:86)
at java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase.setup(YARNSessionFIFOSecuredITCase.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
org.junit.rules.RunRules.evaluate(RunRules.java:20) at
org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: java.io.IOException: ResourceManager failed to start. Final state is
STOPPED at
org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:323)
... 34 more
```
{{}}
{{```}}
{{}}
{{}}
> YarnMiniCluster failed to start in YARNSessionFIFOSecuredITCase
> ---------------------------------------------------------------
>
> Key: FLINK-14207
> URL: https://issues.apache.org/jira/browse/FLINK-14207
> Project: Flink
> Issue Type: Task
> Components: Tests
> Affects Versions: 1.9.0
> Reporter: Paul Lin
> Priority: Minor
>
> I built flink with hadoop 2.6.5 and docker image `{{maven:3-jdk-8-alpine`,
> which failed multiple times with the same exception that YarnMiniCluster
> failed to start in secure mode. I've successfully run the tests with the
> previous versions including 1.5-1.8, and it's wired that with 1.9 it failed
> with an error that probrably relates to Hadoop setup. }}
>
> {{The stack trace is as follow:}}
> {{```}}
> 06:36:37,275 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager - Transitioned
> to standby state Exception in thread "RM-0" java.lang.RuntimeException: Could
> not read HTTP signature secret file:
> /tmp/junit4738785782616411228/junit7110292436248761811/hadoop-http-auth-signature-secret
> at
> org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.createFilterConfig(RMAuthenticationFilterInitializer.java:100)
> at
> org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.initFilter(RMAuthenticationFilterInitializer.java:130)
> at
> org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:395)
> at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:346) at
> org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:105) at
> org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:293) at
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:260) at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:990)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1090)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$2.run(MiniYARNCluster.java:310)
> 06:36:38,401 INFO org.apache.hadoop.service.AbstractService - Service
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 failed
> in state STARTED; cause:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
> ResourceManager failed to start. Final state is STOPPED
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
> ResourceManager failed to start. Final state is STOPPED at
> org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:328)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$500(MiniYARNCluster.java:94)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceStart(MiniYARNCluster.java:448)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.flink.yarn.YarnTestBase.start(YarnTestBase.java:621) at
> org.apache.flink.yarn.YarnTestBase.startYARNSecureMode(YarnTestBase.java:577)
> at
> org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:89)
> at
> org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:86)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at
> org.apache.flink.yarn.YARNSessionFIFOSecuredITCase.setup(YARNSessionFIFOSecuredITCase.java:86)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
> org.junit.rules.RunRules.evaluate(RunRules.java:20) at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Caused by: java.io.IOException: ResourceManager failed to start. Final state
> is STOPPED at
> org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:323)
> ... 34 more
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
