[jira] [Commented] (YARN-9452) Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures
[
https://issues.apache.org/jira/browse/YARN-9452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17449724#comment-17449724
]
Aki Tanaka commented on YARN-9452:
--
Hi. Looks like this was already backported by YARN-9338
> Fix TestDistributedShell and TestTimelineAuthFilterForV2 failures
> -
>
> Key: YARN-9452
> URL: https://issues.apache.org/jira/browse/YARN-9452
> Project: Hadoop YARN
> Issue Type: Bug
> Components: ATSv2, distributed-shell, test
>Affects Versions: 3.2.0
>Reporter: Prabhu Joseph
>Assignee: Prabhu Joseph
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: YARN-9452-001.patch, YARN-9452-002.patch,
> YARN-9452-003.patch, YARN-9452-004.patch
>
>
> *TestDistributedShell#testDSShellWithoutDomainV2CustomizedFlow*
> {code}
> [ERROR]
> testDSShellWithoutDomainV2CustomizedFlow(org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell)
> Time elapsed: 72.14 s <<< FAILURE!
> java.lang.AssertionError: Entity ID prefix should be same across each publish
> of same entity expected:<9223372036854775806> but was:<9223370482298585580>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:645)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.verifyEntityForTimelineV2(TestDistributedShell.java:695)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.checkTimelineV2(TestDistributedShell.java:588)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShell(TestDistributedShell.java:459)
> at
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithoutDomainV2CustomizedFlow(TestDistributedShell.java:330)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> *TestTimelineAuthFilterForV2#testPutTimelineEntities*
> {code}
> [ERROR]
> testPutTimelineEntities[3](org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2)
> Time elapsed: 1.047 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertNotNull(Assert.java:712)
> at org.junit.Assert.assertNotNull(Assert.java:722)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.verifyEntity(TestTimelineAuthFilterForV2.java:282)
> at
> org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.testPutTimelineEntities(TestTimelineAuthFilterForV2.java:421)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at
[jira] [Commented] (YARN-8840) Add missing cleanupSSLConfig() call for TestTimelineClient test
[
https://issues.apache.org/jira/browse/YARN-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637265#comment-16637265
]
Aki Tanaka commented on YARN-8840:
--
Fixed styles issues. Regarding the unit test failure, I think this is not
related to my patch. I confirmed the error without applying my patch.
[^YARN-8840.003.patch]
> Add missing cleanupSSLConfig() call for TestTimelineClient test
> ---
>
> Key: YARN-8840
> URL: https://issues.apache.org/jira/browse/YARN-8840
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, timelineclient
>Reporter: Aki Tanaka
>Priority: Minor
> Attachments: YARN-8840.001.patch, YARN-8840.002.patch,
> YARN-8840.003.patch
>
>
> Tests that setup SSLConfigs can leave conf-files lingering unless they are
> cleaned up via {{KeyStoreTestUtil.cleanupSSLConfig}} call. TestTimelineClient
> test is missing this call.
> If the cleanup method is not called explicitly, a modified ssl-client.xml is
> left in {{test-classes}}, might affect subsequent test cases.
>
> There was a similar report in HDFS-11042, but looks that we need to fix
> TestTimelineClient test too.
>
> {code:java}
> $ mvn test -Dtest=TestTimelineClient
> $ find .|grep ssl-client.xml$
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
> $ cat
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
>
> ssl.client.truststore.reload.interval1000falseprogrammatically
> ssl.client.truststore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/trustKS.jksfalseprogrammatically
> ssl.client.keystore.keypasswordclientPfalseprogrammatically
> ssl.client.keystore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/clientKS.jksfalseprogrammatically
> ssl.client.truststore.passwordtrustPfalseprogrammatically
> ssl.client.keystore.passwordclientPfalseprogrammatically
> {code}
>
> After applying this patch, the ssl-client.xml is not generated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8840) Add missing cleanupSSLConfig() call for TestTimelineClient test
[
https://issues.apache.org/jira/browse/YARN-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aki Tanaka updated YARN-8840:
-
Attachment: YARN-8840.003.patch
> Add missing cleanupSSLConfig() call for TestTimelineClient test
> ---
>
> Key: YARN-8840
> URL: https://issues.apache.org/jira/browse/YARN-8840
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, timelineclient
>Reporter: Aki Tanaka
>Priority: Minor
> Attachments: YARN-8840.001.patch, YARN-8840.002.patch,
> YARN-8840.003.patch
>
>
> Tests that setup SSLConfigs can leave conf-files lingering unless they are
> cleaned up via {{KeyStoreTestUtil.cleanupSSLConfig}} call. TestTimelineClient
> test is missing this call.
> If the cleanup method is not called explicitly, a modified ssl-client.xml is
> left in {{test-classes}}, might affect subsequent test cases.
>
> There was a similar report in HDFS-11042, but looks that we need to fix
> TestTimelineClient test too.
>
> {code:java}
> $ mvn test -Dtest=TestTimelineClient
> $ find .|grep ssl-client.xml$
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
> $ cat
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
>
> ssl.client.truststore.reload.interval1000falseprogrammatically
> ssl.client.truststore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/trustKS.jksfalseprogrammatically
> ssl.client.keystore.keypasswordclientPfalseprogrammatically
> ssl.client.keystore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/clientKS.jksfalseprogrammatically
> ssl.client.truststore.passwordtrustPfalseprogrammatically
> ssl.client.keystore.passwordclientPfalseprogrammatically
> {code}
>
> After applying this patch, the ssl-client.xml is not generated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8840) Add missing cleanupSSLConfig() call for TestTimelineClient test
[
https://issues.apache.org/jira/browse/YARN-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16636501#comment-16636501
]
Aki Tanaka commented on YARN-8840:
--
Thank you for looking and your comment! I moved the cleanup method in @After
class.
[^YARN-8840.002.patch]
> Add missing cleanupSSLConfig() call for TestTimelineClient test
> ---
>
> Key: YARN-8840
> URL: https://issues.apache.org/jira/browse/YARN-8840
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, timelineclient
>Reporter: Aki Tanaka
>Priority: Minor
> Attachments: YARN-8840.001.patch, YARN-8840.002.patch
>
>
> Tests that setup SSLConfigs can leave conf-files lingering unless they are
> cleaned up via {{KeyStoreTestUtil.cleanupSSLConfig}} call. TestTimelineClient
> test is missing this call.
> If the cleanup method is not called explicitly, a modified ssl-client.xml is
> left in {{test-classes}}, might affect subsequent test cases.
>
> There was a similar report in HDFS-11042, but looks that we need to fix
> TestTimelineClient test too.
>
> {code:java}
> $ mvn test -Dtest=TestTimelineClient
> $ find .|grep ssl-client.xml$
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
> $ cat
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
>
> ssl.client.truststore.reload.interval1000falseprogrammatically
> ssl.client.truststore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/trustKS.jksfalseprogrammatically
> ssl.client.keystore.keypasswordclientPfalseprogrammatically
> ssl.client.keystore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/clientKS.jksfalseprogrammatically
> ssl.client.truststore.passwordtrustPfalseprogrammatically
> ssl.client.keystore.passwordclientPfalseprogrammatically
> {code}
>
> After applying this patch, the ssl-client.xml is not generated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8840) Add missing cleanupSSLConfig() call for TestTimelineClient test
[
https://issues.apache.org/jira/browse/YARN-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aki Tanaka updated YARN-8840:
-
Attachment: YARN-8840.002.patch
> Add missing cleanupSSLConfig() call for TestTimelineClient test
> ---
>
> Key: YARN-8840
> URL: https://issues.apache.org/jira/browse/YARN-8840
> Project: Hadoop YARN
> Issue Type: Bug
> Components: test, timelineclient
>Reporter: Aki Tanaka
>Priority: Minor
> Attachments: YARN-8840.001.patch, YARN-8840.002.patch
>
>
> Tests that setup SSLConfigs can leave conf-files lingering unless they are
> cleaned up via {{KeyStoreTestUtil.cleanupSSLConfig}} call. TestTimelineClient
> test is missing this call.
> If the cleanup method is not called explicitly, a modified ssl-client.xml is
> left in {{test-classes}}, might affect subsequent test cases.
>
> There was a similar report in HDFS-11042, but looks that we need to fix
> TestTimelineClient test too.
>
> {code:java}
> $ mvn test -Dtest=TestTimelineClient
> $ find .|grep ssl-client.xml$
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
> $ cat
> ./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
>
> ssl.client.truststore.reload.interval1000falseprogrammatically
> ssl.client.truststore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/trustKS.jksfalseprogrammatically
> ssl.client.keystore.keypasswordclientPfalseprogrammatically
> ssl.client.keystore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/clientKS.jksfalseprogrammatically
> ssl.client.truststore.passwordtrustPfalseprogrammatically
> ssl.client.keystore.passwordclientPfalseprogrammatically
> {code}
>
> After applying this patch, the ssl-client.xml is not generated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8840) Add missing cleanupSSLConfig() call for TestTimelineClient test
Aki Tanaka created YARN-8840:
Summary: Add missing cleanupSSLConfig() call for
TestTimelineClient test
Key: YARN-8840
URL: https://issues.apache.org/jira/browse/YARN-8840
Project: Hadoop YARN
Issue Type: Bug
Components: test, timelineclient
Reporter: Aki Tanaka
Tests that setup SSLConfigs can leave conf-files lingering unless they are
cleaned up via {{KeyStoreTestUtil.cleanupSSLConfig}} call. TestTimelineClient
test is missing this call.
If the cleanup method is not called explicitly, a modified ssl-client.xml is
left in {{test-classes}}, might affect subsequent test cases.
There was a similar report in HDFS-11042, but looks that we need to fix
TestTimelineClient test too.
{code:java}
$ mvn test -Dtest=TestTimelineClient
$ find .|grep ssl-client.xml$
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
$ cat
./hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-classes/ssl-client.xml
ssl.client.truststore.reload.interval1000falseprogrammatically
ssl.client.truststore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/trustKS.jksfalseprogrammatically
ssl.client.keystore.keypasswordclientPfalseprogrammatically
ssl.client.keystore.location/Users/tanaka/work/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/target/test-dir/clientKS.jksfalseprogrammatically
ssl.client.truststore.passwordtrustPfalseprogrammatically
ssl.client.keystore.passwordclientPfalseprogrammatically
{code}
After applying this patch, the ssl-client.xml is not generated.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8019) Resource manager webproxy fails to validate backend server's SSL cert
[
https://issues.apache.org/jira/browse/YARN-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aki Tanaka updated YARN-8019:
-
Summary: Resource manager webproxy fails to validate backend server's SSL
cert (was: Resource manager webproxy uses the client truststore specified in
ssl-client.xml)
> Resource manager webproxy fails to validate backend server's SSL cert
> -
>
> Key: YARN-8019
> URL: https://issues.apache.org/jira/browse/YARN-8019
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Aki Tanaka
>Priority: Major
> Attachments: YARN-8019.001.patch
>
>
> A Yarn ResourceManager's web proxy launches with Java default SSL
> certificate. Due to this behavior, the web proxy failed to validate a backend
> server's SSL certificate when the backend server listens with HTTPS using
> custom SSL certificate.
>
> For example, Spark launches Spark context web UI with custom SSL certificate
> when we enable SSL with "spark.ssl.trustStore" and "spark.ssl.keyStore"
> properties. In this case, Yarn web proxy cannot connect the Spark context web
> UI since the web proxy cannot verify the SSL cert
> ("javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed" error
> is returned).
>
> We should add an option to set SSL trust store to Yarn RM web proxy. Attached
> a patch to Yarn web proxy, and this patch lets web proxy use an SSL custom
> trust-store if it is configured in ssl-client.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2554) Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
[ https://issues.apache.org/jira/browse/YARN-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393864#comment-16393864 ] Aki Tanaka commented on YARN-2554: -- Created YARN-8019 for visibility. > Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy > - > > Key: YARN-2554 > URL: https://issues.apache.org/jira/browse/YARN-2554 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Affects Versions: 2.6.0 >Reporter: Jonathan Maron >Priority: Major > Labels: BB2015-05-TBR > Attachments: YARN-2554.1.patch, YARN-2554.2.patch, YARN-2554.3.patch, > YARN-2554.3.patch > > > If the HTTP policy to enable HTTPS is specified, the RM and AM are > initialized with SSL listeners. The RM has a web app proxy servlet that acts > as a proxy for incoming AM requests. In order to forward the requests to the > AM the proxy servlet makes use of HttpClient. However, the HttpClient > utilized is not initialized correctly with the necessary certs to allow for > successful one way SSL invocations to the other nodes in the cluster (it is > not configured to access/load the client truststore specified in > ssl-client.xml). I imagine SSLFactory.createSSLSocketFactory() could be > utilized to create an instance that can be assigned to the HttpClient. > The symptoms of this issue are: > AM: Displays "unknown_certificate" exception > RM: Displays an exception such as "javax.net.ssl.SSLHandshakeException: > sun.security.validator.ValidatorException: PKIX path building failed: > sun.security.provider.certpath.SunCertPathBuilderException: unable to find > valid certification path to requested target" -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8019) Resource manager webproxy uses the client truststore specified in ssl-client.xml
[
https://issues.apache.org/jira/browse/YARN-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aki Tanaka updated YARN-8019:
-
Attachment: YARN-8019.001.patch
> Resource manager webproxy uses the client truststore specified in
> ssl-client.xml
>
>
> Key: YARN-8019
> URL: https://issues.apache.org/jira/browse/YARN-8019
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Aki Tanaka
>Priority: Major
> Attachments: YARN-8019.001.patch
>
>
> A Yarn ResourceManager's web proxy launches with Java default SSL
> certificate. Due to this behavior, the web proxy failed to validate a backend
> server's SSL certificate when the backend server listens with HTTPS using
> custom SSL certificate.
>
> For example, Spark launches Spark context web UI with custom SSL certificate
> when we enable SSL with "spark.ssl.trustStore" and "spark.ssl.keyStore"
> properties. In this case, Yarn web proxy cannot connect the Spark context web
> UI since the web proxy cannot verify the SSL cert
> ("javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed" error
> is returned).
>
> We should add an option to set SSL trust store to Yarn RM web proxy. Attached
> a patch to Yarn web proxy, and this patch lets web proxy use an SSL custom
> trust-store if it is configured in ssl-client.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8019) Resource manager webproxy uses the client truststore specified in ssl-client.xml
[
https://issues.apache.org/jira/browse/YARN-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aki Tanaka updated YARN-8019:
-
Summary: Resource manager webproxy uses the client truststore specified in
ssl-client.xml (was: RM webproxy uses the client truststore specified in
ssl-client.xml)
> Resource manager webproxy uses the client truststore specified in
> ssl-client.xml
>
>
> Key: YARN-8019
> URL: https://issues.apache.org/jira/browse/YARN-8019
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.0.0
>Reporter: Aki Tanaka
>Priority: Major
>
> A Yarn ResourceManager's web proxy launches with Java default SSL
> certificate. Due to this behavior, the web proxy failed to validate a backend
> server's SSL certificate when the backend server listens with HTTPS using
> custom SSL certificate.
>
> For example, Spark launches Spark context web UI with custom SSL certificate
> when we enable SSL with "spark.ssl.trustStore" and "spark.ssl.keyStore"
> properties. In this case, Yarn web proxy cannot connect the Spark context web
> UI since the web proxy cannot verify the SSL cert
> ("javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed" error
> is returned).
>
> We should add an option to set SSL trust store to Yarn RM web proxy. Attached
> a patch to Yarn web proxy, and this patch lets web proxy use an SSL custom
> trust-store if it is configured in ssl-client.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-8019) RM webproxy uses the client truststore specified in ssl-client.xml
Aki Tanaka created YARN-8019:
Summary: RM webproxy uses the client truststore specified in
ssl-client.xml
Key: YARN-8019
URL: https://issues.apache.org/jira/browse/YARN-8019
Project: Hadoop YARN
Issue Type: Bug
Components: yarn
Affects Versions: 3.0.0
Reporter: Aki Tanaka
A Yarn ResourceManager's web proxy launches with Java default SSL certificate.
Due to this behavior, the web proxy failed to validate a backend server's SSL
certificate when the backend server listens with HTTPS using custom SSL
certificate.
For example, Spark launches Spark context web UI with custom SSL certificate
when we enable SSL with "spark.ssl.trustStore" and "spark.ssl.keyStore"
properties. In this case, Yarn web proxy cannot connect the Spark context web
UI since the web proxy cannot verify the SSL cert
("javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed" error is
returned).
We should add an option to set SSL trust store to Yarn RM web proxy. Attached a
patch to Yarn web proxy, and this patch lets web proxy use an SSL custom
trust-store if it is configured in ssl-client.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2554) Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
[ https://issues.apache.org/jira/browse/YARN-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196249#comment-16196249 ] Aki Tanaka commented on YARN-2554: -- Ping, this issue has been open for entirely too long. Can someone please give me some comments on this? > Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy > - > > Key: YARN-2554 > URL: https://issues.apache.org/jira/browse/YARN-2554 > Project: Hadoop YARN > Issue Type: Bug > Components: webapp >Affects Versions: 2.6.0 >Reporter: Jonathan Maron > Labels: BB2015-05-TBR > Attachments: YARN-2554.1.patch, YARN-2554.2.patch, YARN-2554.3.patch, > YARN-2554.3.patch > > > If the HTTP policy to enable HTTPS is specified, the RM and AM are > initialized with SSL listeners. The RM has a web app proxy servlet that acts > as a proxy for incoming AM requests. In order to forward the requests to the > AM the proxy servlet makes use of HttpClient. However, the HttpClient > utilized is not initialized correctly with the necessary certs to allow for > successful one way SSL invocations to the other nodes in the cluster (it is > not configured to access/load the client truststore specified in > ssl-client.xml). I imagine SSLFactory.createSSLSocketFactory() could be > utilized to create an instance that can be assigned to the HttpClient. > The symptoms of this issue are: > AM: Displays "unknown_certificate" exception > RM: Displays an exception such as "javax.net.ssl.SSLHandshakeException: > sun.security.validator.ValidatorException: PKIX path building failed: > sun.security.provider.certpath.SunCertPathBuilderException: unable to find > valid certification path to requested target" -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-2554) Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
[
https://issues.apache.org/jira/browse/YARN-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157544#comment-16157544
]
Aki Tanaka edited comment on YARN-2554 at 9/7/17 9:35 PM:
--
I want to raise the issue again since the issue affects other application which
runs on YARN. Actually, I see this problem when we run Spark job on Yarn.
Spark launches Spark context web UI with custom SSL certificate when we enable
SSL with "spark.ssl.trustStore" and "spark.ssl.keyStore" properties. In this
case, Yarn web proxy cannot connect the Spark context web UI since the web
proxy cannot verify the SSL cert ("javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed" error is
returned).
We should add an option to set SSL trust store to Yarn RM web proxy. I added
the updated patch, and this patch lets web proxy use an SSL custom trust-store
if it is configured in ssl-client.xml
Pull Request: https://github.com/apache/hadoop/pull/271
was (Author: tanakahda):
I want to raise the issue again since the issue affects other application which
runs on YARN. Actually, I see this problem when we run Spark job on Yarn.
Spark launches Spark context web UI with custom SSL certificate when we enable
SSL with "spark.ssl.trustStore" and "spark.ssl.keyStore" properties. In this
case, Yarn web proxy cannot connect the Spark context web UI since the web
proxy cannot verify the SSL cert ("javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed" error is
returned).
We should add an option to set SSL trust store to Yarn RM web proxy. I added
the updated patch, and this patch lets web proxy use an SSL custom trust-store
if it is configured in ssl-client.xml
Pull Request: https://github.com/apache/hadoop/pull/270
> Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
> -
>
> Key: YARN-2554
> URL: https://issues.apache.org/jira/browse/YARN-2554
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Affects Versions: 2.6.0
>Reporter: Jonathan Maron
> Labels: BB2015-05-TBR
> Attachments: YARN-2554.1.patch, YARN-2554.2.patch, YARN-2554.3.patch,
> YARN-2554.3.patch
>
>
> If the HTTP policy to enable HTTPS is specified, the RM and AM are
> initialized with SSL listeners. The RM has a web app proxy servlet that acts
> as a proxy for incoming AM requests. In order to forward the requests to the
> AM the proxy servlet makes use of HttpClient. However, the HttpClient
> utilized is not initialized correctly with the necessary certs to allow for
> successful one way SSL invocations to the other nodes in the cluster (it is
> not configured to access/load the client truststore specified in
> ssl-client.xml). I imagine SSLFactory.createSSLSocketFactory() could be
> utilized to create an instance that can be assigned to the HttpClient.
> The symptoms of this issue are:
> AM: Displays "unknown_certificate" exception
> RM: Displays an exception such as "javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2554) Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
[
https://issues.apache.org/jira/browse/YARN-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157544#comment-16157544
]
Aki Tanaka commented on YARN-2554:
--
I want to raise the issue again since the issue affects other application which
runs on YARN. Actually, I see this problem when we run Spark job on Yarn.
Spark launches Spark context web UI with custom SSL certificate when we enable
SSL with "spark.ssl.trustStore" and "spark.ssl.keyStore" properties. In this
case, Yarn web proxy cannot connect the Spark context web UI since the web
proxy cannot verify the SSL cert ("javax.net.ssl.SSLHandshakeException:
sun.security.validator.ValidatorException: PKIX path building failed" error is
returned).
We should add an option to set SSL trust store to Yarn RM web proxy. I added
the updated patch, and this patch lets web proxy use an SSL custom trust-store
if it is configured in ssl-client.xml
Pull Request: https://github.com/apache/hadoop/pull/270
> Slider AM Web UI is inaccessible if HTTPS/SSL is specified as the HTTP policy
> -
>
> Key: YARN-2554
> URL: https://issues.apache.org/jira/browse/YARN-2554
> Project: Hadoop YARN
> Issue Type: Bug
> Components: webapp
>Affects Versions: 2.6.0
>Reporter: Jonathan Maron
> Labels: BB2015-05-TBR
> Attachments: YARN-2554.1.patch, YARN-2554.2.patch, YARN-2554.3.patch,
> YARN-2554.3.patch
>
>
> If the HTTP policy to enable HTTPS is specified, the RM and AM are
> initialized with SSL listeners. The RM has a web app proxy servlet that acts
> as a proxy for incoming AM requests. In order to forward the requests to the
> AM the proxy servlet makes use of HttpClient. However, the HttpClient
> utilized is not initialized correctly with the necessary certs to allow for
> successful one way SSL invocations to the other nodes in the cluster (it is
> not configured to access/load the client truststore specified in
> ssl-client.xml). I imagine SSLFactory.createSSLSocketFactory() could be
> utilized to create an instance that can be assigned to the HttpClient.
> The symptoms of this issue are:
> AM: Displays "unknown_certificate" exception
> RM: Displays an exception such as "javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
