[jira] [Commented] (APEXCORE-712) Support distribution of custom SSL material to the Stram node while launch the app

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052397#comment-16052397
 ] 

ASF GitHub Bot commented on APEXCORE-712:
-

GitHub user sanjaypujare opened a pull request:

https://github.com/apache/apex-core/pull/547

APEXCORE-712 custom keystore at launch. This closes #526

@PramodSSImmaneni @devtagare pls review and merge

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sanjaypujare/apex-core APEXCORE-712.sanjay1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/547.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #547


commit ccdad2f924dc6b255945383de28e68b45a2de484
Author: devtagare 
Date:   2017-05-12T22:04:22Z

APEXCORE-712 custom keystore at launch. This closes #526

commit f1bc35c06c87b64423820265e71f32cdced79cfd
Author: Sanjay Pujare 
Date:   2017-06-16T07:55:22Z

incorporate comments




> Support distribution of custom SSL material to the Stram node while launch 
> the app
> --
>
> Key: APEXCORE-712
> URL: https://issues.apache.org/jira/browse/APEXCORE-712
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Sanjay M Pujare
>Assignee: devendra tagare
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> This JIRA is dependent on APEXCORE-711. APEXCORE-711 talks about using a 
> custom SSL configuration but assumes the SSL files (ssl-server.xml and the 
> keystore) are already available on any cluster node so when the Stram starts 
> it is able to find them. There are cases where users don't want to do this 
> and they expect the Apex client to package these files so that they are 
> copied to the App master node so when Stram starts it will find them in the 
> expected location.
> Enhance the Apex client/launcher to distribute the custom SSL files (XML and 
> the keystore) along with the application jars/resources so the user does not 
> need to pre-distribute the custom SSL files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] apex-core pull request #547: APEXCORE-712 custom keystore at launch. This cl...

[sanjaypujare]

GitHub user sanjaypujare opened a pull request:

https://github.com/apache/apex-core/pull/547

APEXCORE-712 custom keystore at launch. This closes #526

@PramodSSImmaneni @devtagare pls review and merge

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sanjaypujare/apex-core APEXCORE-712.sanjay1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/547.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #547


commit ccdad2f924dc6b255945383de28e68b45a2de484
Author: devtagare 
Date:   2017-05-12T22:04:22Z

APEXCORE-712 custom keystore at launch. This closes #526

commit f1bc35c06c87b64423820265e71f32cdced79cfd
Author: Sanjay Pujare 
Date:   2017-06-16T07:55:22Z

incorporate comments




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-712) Support distribution of custom SSL material to the Stram node while launch the app

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052391#comment-16052391
 ] 

ASF GitHub Bot commented on APEXCORE-712:
-

Github user sanjaypujare closed the pull request at:

https://github.com/apache/apex-core/pull/546


> Support distribution of custom SSL material to the Stram node while launch 
> the app
> --
>
> Key: APEXCORE-712
> URL: https://issues.apache.org/jira/browse/APEXCORE-712
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Sanjay M Pujare
>Assignee: devendra tagare
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> This JIRA is dependent on APEXCORE-711. APEXCORE-711 talks about using a 
> custom SSL configuration but assumes the SSL files (ssl-server.xml and the 
> keystore) are already available on any cluster node so when the Stram starts 
> it is able to find them. There are cases where users don't want to do this 
> and they expect the Apex client to package these files so that they are 
> copied to the App master node so when Stram starts it will find them in the 
> expected location.
> Enhance the Apex client/launcher to distribute the custom SSL files (XML and 
> the keystore) along with the application jars/resources so the user does not 
> need to pre-distribute the custom SSL files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] apex-core pull request #546: APEXCORE-712 custom keystore at launch

[sanjaypujare]

Github user sanjaypujare closed the pull request at:

https://github.com/apache/apex-core/pull/546


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] apex-core pull request #546: APEXCORE-712 custom keystore at launch

[sanjaypujare]

GitHub user sanjaypujare opened a pull request:

https://github.com/apache/apex-core/pull/546

APEXCORE-712 custom keystore at launch

This is meant to replace PR https://github.com/apache/apex-core/pull/526 . 
Pls see the comments in there for the reasons.  

@PramodSSImmaneni @devtagare @tweise let me know.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sanjaypujare/apex-core APEXCORE-712.sanjay

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/546.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #546


commit c583cb5d88c0418d787076937c99d677d73653e5
Author: Sanjay Pujare 
Date:   2017-06-16T17:31:38Z

APEXCORE-712 custom keystore at launch




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-712) Support distribution of custom SSL material to the Stram node while launch the app

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXCORE-712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052197#comment-16052197
 ] 

ASF GitHub Bot commented on APEXCORE-712:
-

GitHub user sanjaypujare opened a pull request:

https://github.com/apache/apex-core/pull/546

APEXCORE-712 custom keystore at launch

This is meant to replace PR https://github.com/apache/apex-core/pull/526 . 
Pls see the comments in there for the reasons.  

@PramodSSImmaneni @devtagare @tweise let me know.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sanjaypujare/apex-core APEXCORE-712.sanjay

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-core/pull/546.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #546


commit c583cb5d88c0418d787076937c99d677d73653e5
Author: Sanjay Pujare 
Date:   2017-06-16T17:31:38Z

APEXCORE-712 custom keystore at launch




> Support distribution of custom SSL material to the Stram node while launch 
> the app
> --
>
> Key: APEXCORE-712
> URL: https://issues.apache.org/jira/browse/APEXCORE-712
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Sanjay M Pujare
>Assignee: devendra tagare
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> This JIRA is dependent on APEXCORE-711. APEXCORE-711 talks about using a 
> custom SSL configuration but assumes the SSL files (ssl-server.xml and the 
> keystore) are already available on any cluster node so when the Stram starts 
> it is able to find them. There are cases where users don't want to do this 
> and they expect the Apex client to package these files so that they are 
> copied to the App master node so when Stram starts it will find them in the 
> expected location.
> Enhance the Apex client/launcher to distribute the custom SSL files (XML and 
> the keystore) along with the application jars/resources so the user does not 
> need to pre-distribute the custom SSL files.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (APEXMALHAR-2507) Example for inner join functionality using Windowed merge operator

[Thomas Weise (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052156#comment-16052156
 ] 

Thomas Weise commented on APEXMALHAR-2507:
--

I don't see the point in removing the example, it needs to be changed/replaced 
to use the appropriate operator.

> Example for inner join functionality using Windowed merge operator
> --
>
> Key: APEXMALHAR-2507
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2507
> Project: Apache Apex Malhar
>  Issue Type: Improvement
>Reporter: Bhupesh Chawda
>Assignee: Shunxin Lu
>
> With removal of the inner join operator (based on managed state), we need to 
> present windowed merge operator as the means of doing an inner join. An 
> example showing how to do an inner join using the windowed merge operator is 
> needed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (APEXMALHAR-2507) Example for inner join functionality using Windowed merge operator

[Shunxin Lu (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16052137#comment-16052137
 ] 

Shunxin Lu commented on APEXMALHAR-2507:


Hi all,
Hitesh, could you elaborate on what you meant by "lost its way"? I will take a 
closer look at the app.
Thomas, I agree [APEXMALHAR-2300] should be closed. I think the current example 
in innjoin folder will be removed in [APEXMALHAR-2488]

> Example for inner join functionality using Windowed merge operator
> --
>
> Key: APEXMALHAR-2507
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2507
> Project: Apache Apex Malhar
>  Issue Type: Improvement
>Reporter: Bhupesh Chawda
>Assignee: Shunxin Lu
>
> With removal of the inner join operator (based on managed state), we need to 
> present windowed merge operator as the means of doing an inner join. An 
> example showing how to do an inner join using the windowed merge operator is 
> needed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (APEXCORE-746) Application fails to run in TLS/SSL enabled cluster because of a stram web service error

[Velineni Lakshmi Prasanna (JIRA)]

Velineni Lakshmi Prasanna created APEXCORE-746:
--

 Summary: Application fails to run in TLS/SSL enabled cluster 
because of a stram web service error
 Key: APEXCORE-746
 URL: https://issues.apache.org/jira/browse/APEXCORE-746
 Project: Apache Apex Core
  Issue Type: Bug
 Environment: CDH 5.11.0 + TLS/SSL enabled
Reporter: Velineni Lakshmi Prasanna


In the specified environment with TLS/SSL enabled, an application this is 
launched fails to get to a running state. It gets stuck in accepted state.

On further debugging, the app master starts but gets stuck failing to 
initialize the stram web service and eventually fails with the following 
exception

java.io.IOException: !JsseListener: java.lang.NullPointerException
at 
org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:531)
at 
org.apache.hadoop.security.ssl.SslSocketConnectorSecure.newServerSocket(SslSocketConnectorSecure.java:46)
at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
at 
org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:953)
at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:894)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:274)
at 
com.datatorrent.stram.StreamingAppMasterService.serviceStart(StreamingAppMasterService.java:636)
at 
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at 
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:103)
2017-06-16 07:46:48,530 ERROR com.datatorrent.stram.StreamingAppMasterService: 
Webapps failed to start. Ignoring for now:
org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:279)
at 
com.datatorrent.stram.StreamingAppMasterService.serviceStart(StreamingAppMasterService.java:636)
at 
org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at 
com.datatorrent.stram.StreamingAppMaster.main(StreamingAppMaster.java:103)
Caused by: java.io.IOException: !JsseListener: java.lang.NullPointerException
at 
org.mortbay.jetty.security.SslSocketConnector.newServerSocket(SslSocketConnector.java:531)
at 
org.apache.hadoop.security.ssl.SslSocketConnectorSecure.newServerSocket(SslSocketConnectorSecure.java:46)
at org.mortbay.jetty.bio.SocketConnector.open(SocketConnector.java:73)
at 
org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:953)
at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:894)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:274)

Debugging deeper, this is because the SSL credentials don't get passed on to 
the underlying jetty subsystem that is being used by hadoop for the web server 
functionality. These credentials were configured in CDH and are being used by 
the hadoop web services without any problem.

The workaround to get past this and get the application running is to pass the 
credentials explicitly when launching the application by specifying the 
following attribute 

dt.attr.CONTAINER_JVM_OPTIONS : -Djetty.ssl.keypassword= 
-Djetty.ssl.password=



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)