[
https://issues.apache.org/jira/browse/FLINK-14207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Lin updated FLINK-14207:
-----------------------------
Description:
I built flink with Hadoop 2.6.5 and docker image `{{maven:3-jdk-8-alpine`,
which failed multiple times with the same exception that YarnMiniCluster failed
to start in the secure mode. I've successfully run the tests with the previous
versions including 1.5-1.8, and it's wired that with 1.9 it failed with an
error that probably relates to Hadoop setup.
The stack trace is as follow (for the full log please see the attachment):
```
06:36:37,275 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
- Transitioned to standby state Exception in thread "RM-0"
java.lang.RuntimeException: Could not read HTTP signature secret file:
/tmp/junit4738785782616411228/junit7110292436248761811/hadoop-http-auth-signature-secret
at
org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.createFilterConfig(RMAuthenticationFilterInitializer.java:100)
at
org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.initFilter(RMAuthenticationFilterInitializer.java:130)
at
org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:395) at
org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:346) at
org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:105) at
org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:293) at
org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:260) at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:990)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1090)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.MiniYARNCluster$2.run(MiniYARNCluster.java:310)
06:36:38,401 INFO org.apache.hadoop.service.AbstractService - Service
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 failed
in state STARTED; cause:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
ResourceManager failed to start. Final state is STOPPED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
ResourceManager failed to start. Final state is STOPPED at
org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:328)
at
org.apache.hadoop.yarn.server.MiniYARNCluster.access$500(MiniYARNCluster.java:94)
at
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceStart(MiniYARNCluster.java:448)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.flink.yarn.YarnTestBase.start(YarnTestBase.java:621) at
org.apache.flink.yarn.YarnTestBase.startYARNSecureMode(YarnTestBase.java:577)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:89)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:86)
at java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase.setup(YARNSessionFIFOSecuredITCase.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
org.junit.rules.RunRules.evaluate(RunRules.java:20) at
org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: java.io.IOException: ResourceManager failed to start. Final state is
STOPPED at
org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:323)
... 34 more
```
was:
I built flink with hadoop 2.6.5 and docker image `{{maven:3-jdk-8-alpine`,
which failed multiple times with the same exception that YarnMiniCluster failed
to start in secure mode. I've successfully run the tests with the previous
versions including 1.5-1.8, and it's wired that with 1.9 it failed with an
error that probrably relates to Hadoop setup. }}
{{The stack trace is as follow:}}
{{```}}
06:36:37,275 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager
- Transitioned to standby state Exception in thread "RM-0"
java.lang.RuntimeException: Could not read HTTP signature secret file:
/tmp/junit4738785782616411228/junit7110292436248761811/hadoop-http-auth-signature-secret
at
org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.createFilterConfig(RMAuthenticationFilterInitializer.java:100)
at
org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.initFilter(RMAuthenticationFilterInitializer.java:130)
at
org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:395) at
org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:346) at
org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:105) at
org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:293) at
org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:260) at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:990)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1090)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.hadoop.yarn.server.MiniYARNCluster$2.run(MiniYARNCluster.java:310)
06:36:38,401 INFO org.apache.hadoop.service.AbstractService - Service
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 failed
in state STARTED; cause:
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
ResourceManager failed to start. Final state is STOPPED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
ResourceManager failed to start. Final state is STOPPED at
org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:328)
at
org.apache.hadoop.yarn.server.MiniYARNCluster.access$500(MiniYARNCluster.java:94)
at
org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceStart(MiniYARNCluster.java:448)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at
org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.flink.yarn.YarnTestBase.start(YarnTestBase.java:621) at
org.apache.flink.yarn.YarnTestBase.startYARNSecureMode(YarnTestBase.java:577)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:89)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:86)
at java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
at
org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at
org.apache.flink.yarn.YARNSessionFIFOSecuredITCase.setup(YARNSessionFIFOSecuredITCase.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
org.junit.rules.RunRules.evaluate(RunRules.java:20) at
org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: java.io.IOException: ResourceManager failed to start. Final state is
STOPPED at
org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:323)
... 34 more
```
> YarnMiniCluster failed to start in YARNSessionFIFOSecuredITCase
> ---------------------------------------------------------------
>
> Key: FLINK-14207
> URL: https://issues.apache.org/jira/browse/FLINK-14207
> Project: Flink
> Issue Type: Task
> Components: Tests
> Affects Versions: 1.9.0
> Reporter: Paul Lin
> Priority: Minor
> Attachments: FLINK-14207-logs.md
>
>
> I built flink with Hadoop 2.6.5 and docker image `{{maven:3-jdk-8-alpine`,
> which failed multiple times with the same exception that YarnMiniCluster
> failed to start in the secure mode. I've successfully run the tests with the
> previous versions including 1.5-1.8, and it's wired that with 1.9 it failed
> with an error that probably relates to Hadoop setup.
> The stack trace is as follow (for the full log please see the attachment):
> ```
> 06:36:37,275 INFO
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager - Transitioned
> to standby state Exception in thread "RM-0" java.lang.RuntimeException: Could
> not read HTTP signature secret file:
> /tmp/junit4738785782616411228/junit7110292436248761811/hadoop-http-auth-signature-secret
> at
> org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.createFilterConfig(RMAuthenticationFilterInitializer.java:100)
> at
> org.apache.hadoop.yarn.server.security.http.RMAuthenticationFilterInitializer.initFilter(RMAuthenticationFilterInitializer.java:130)
> at
> org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:395)
> at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:346) at
> org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:105) at
> org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:293) at
> org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:260) at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:990)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1090)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$2.run(MiniYARNCluster.java:310)
> 06:36:38,401 INFO org.apache.hadoop.service.AbstractService - Service
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper_0 failed
> in state STARTED; cause:
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
> ResourceManager failed to start. Final state is STOPPED
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException:
> ResourceManager failed to start. Final state is STOPPED at
> org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:328)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$500(MiniYARNCluster.java:94)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceStart(MiniYARNCluster.java:448)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at org.apache.flink.yarn.YarnTestBase.start(YarnTestBase.java:621) at
> org.apache.flink.yarn.YarnTestBase.startYARNSecureMode(YarnTestBase.java:577)
> at
> org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:89)
> at
> org.apache.flink.yarn.YARNSessionFIFOSecuredITCase$1.call(YARNSessionFIFOSecuredITCase.java:86)
> at java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at
> org.apache.flink.yarn.YARNSessionFIFOSecuredITCase.setup(YARNSessionFIFOSecuredITCase.java:86)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498) at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at
> org.junit.rules.RunRules.evaluate(RunRules.java:20) at
> org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
> Caused by: java.io.IOException: ResourceManager failed to start. Final state
> is STOPPED at
> org.apache.hadoop.yarn.server.MiniYARNCluster.startResourceManager(MiniYARNCluster.java:323)
> ... 34 more
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
