[jira] [Updated] (APEXCORE-474) Unifier placement during M*1 deployment

2016-09-01 Thread Sandesh (JIRA) 

 [ 
https://issues.apache.org/jira/browse/APEXCORE-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandesh updated APEXCORE-474:
-
Summary: Unifier placement during M*1 deployment  (was: Default unifier 
placement during M*1 deployment)

> Unifier placement during M*1 deployment
> ---
>
> Key: APEXCORE-474
> URL: https://issues.apache.org/jira/browse/APEXCORE-474
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Sandesh
>Assignee: Sandesh
>
> During M*1 deployment, unifier was deployed in the separate container. But 
> there is no advantage in doing that. 
> It is better to make the unifier THREAD_LOCAL with the downstream operator.
> ( https://issues.apache.org/jira/browse/APEXCORE-482 )
> Note:
> Recently saw one Kafka ETL app, that had a total of 18 containers allocated, 
> but out of that 5 containers were allocated for default unifiers. It also 
> means, lots of time is spent in SerDe. 
> Implementing this feature will improve the performance greatly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (APEXCORE-474) Default unifier placement during M*1 deployment

2016-09-01 Thread Sandesh (JIRA) 

 [ 
https://issues.apache.org/jira/browse/APEXCORE-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sandesh reassigned APEXCORE-474:


Assignee: Sandesh

> Default unifier placement during M*1 deployment
> ---
>
> Key: APEXCORE-474
> URL: https://issues.apache.org/jira/browse/APEXCORE-474
> Project: Apache Apex Core
>  Issue Type: Improvement
>Reporter: Sandesh
>Assignee: Sandesh
>
> During M*1 deployment, unifier was deployed in the separate container. But 
> there is no advantage in doing that. 
> It is better to make the unifier THREAD_LOCAL with the downstream operator.
> ( https://issues.apache.org/jira/browse/APEXCORE-482 )
> Note:
> Recently saw one Kafka ETL app, that had a total of 18 containers allocated, 
> but out of that 5 containers were allocated for default unifiers. It also 
> means, lots of time is spent in SerDe. 
> Implementing this feature will improve the performance greatly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2001) Fix all the checkstyle violations in kafka module

2016-09-01 Thread Sandesh (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457014#comment-15457014
 ] 

Sandesh commented on APEXMALHAR-2001:
-

That fell off the radar, add it to my todo list.

Will pick that up once I resolve the 
https://issues.apache.org/jira/browse/APEXCORE-474



> Fix all the checkstyle violations in kafka module
> -
>
> Key: APEXMALHAR-2001
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2001
> Project: Apache Apex Malhar
>  Issue Type: Improvement
>  Components: adapters message bus
>Reporter: Chandni Singh
>Assignee: Sandesh
>
> Please fix all the violations in Kafka module 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (APEXMALHAR-2001) Fix all the checkstyle violations in kafka module

2016-09-01 Thread Sandesh (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457014#comment-15457014
 ] 

Sandesh edited comment on APEXMALHAR-2001 at 9/2/16 12:07 AM:
--

That fell off the radar, added it to my todo list.

Will pick that up once I resolve the 
https://issues.apache.org/jira/browse/APEXCORE-474




was (Author: sandesh):
That fell off the radar, add it to my todo list.

Will pick that up once I resolve the 
https://issues.apache.org/jira/browse/APEXCORE-474



> Fix all the checkstyle violations in kafka module
> -
>
> Key: APEXMALHAR-2001
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2001
> Project: Apache Apex Malhar
>  Issue Type: Improvement
>  Components: adapters message bus
>Reporter: Chandni Singh
>Assignee: Sandesh
>
> Please fix all the violations in Kafka module 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (APEXMALHAR-2221) Fix checkstyle violations in benchmark module

2016-09-01 Thread Thomas Weise (JIRA) 
Thomas Weise created APEXMALHAR-2221:


 Summary: Fix checkstyle violations in benchmark module
 Key: APEXMALHAR-2221
 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2221
 Project: Apache Apex Malhar
  Issue Type: Sub-task
Reporter: Thomas Weise
Assignee: bright chen






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2001) Fix all the checkstyle violations in kafka module

2016-09-01 Thread Thomas Weise (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456940#comment-15456940
 ] 

Thomas Weise commented on APEXMALHAR-2001:
--

[~sandesh] any update?

> Fix all the checkstyle violations in kafka module
> -
>
> Key: APEXMALHAR-2001
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2001
> Project: Apache Apex Malhar
>  Issue Type: Improvement
>  Components: adapters message bus
>Reporter: Chandni Singh
>Assignee: Sandesh
>
> Please fix all the violations in Kafka module 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2205) State management benchmark

2016-09-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456928#comment-15456928
 ] 

ASF GitHub Bot commented on APEXMALHAR-2205:


Github user asfgit closed the pull request at:

https://github.com/apache/apex-malhar/pull/398


> State management benchmark
> --
>
> Key: APEXMALHAR-2205
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: bright chen
>Assignee: bright chen
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-malhar pull request #398: APEXMALHAR-2205 #resolve #comment State manag...

2016-09-01 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/apex-malhar/pull/398


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXMALHAR-2205) State management benchmark

2016-09-01 Thread Thomas Weise (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456921#comment-15456921
 ] 

Thomas Weise commented on APEXMALHAR-2205:
--

Is it just writing to the WAL or also writing the data files? Any information 
about key cardinality?


> State management benchmark
> --
>
> Key: APEXMALHAR-2205
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: bright chen
>Assignee: bright chen
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2205) State management benchmark

2016-09-01 Thread bright chen (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456794#comment-15456794
 ] 

bright chen commented on APEXMALHAR-2205:
-

The throughput ( almost same in cluster and local machine ) is around 
40k/second in peek and 28k/second at bottom. I ran around one hour and pretty 
stable.


> State management benchmark
> --
>
> Key: APEXMALHAR-2205
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: bright chen
>Assignee: bright chen
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2205) State management benchmark

2016-09-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456780#comment-15456780
 ] 

ASF GitHub Bot commented on APEXMALHAR-2205:


GitHub user brightchen opened a pull request:

https://github.com/apache/apex-malhar/pull/398

APEXMALHAR-2205 #resolve #comment State management benchmark

previous PR closed by accident. 
https://github.com/apache/apex-malhar/pull/386

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2205

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/398.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #398


commit c5a12e4e747c5be840a16e4c932cbc1dbff79894
Author: brightchen 
Date:   2016-08-26T23:09:12Z

APEXMALHAR-2205 #resolve #comment State management benchmark




> State management benchmark
> --
>
> Key: APEXMALHAR-2205
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: bright chen
>Assignee: bright chen
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-malhar pull request #398: APEXMALHAR-2205 #resolve #comment State manag...

2016-09-01 Thread brightchen 
GitHub user brightchen opened a pull request:

https://github.com/apache/apex-malhar/pull/398

APEXMALHAR-2205 #resolve #comment State management benchmark

previous PR closed by accident. 
https://github.com/apache/apex-malhar/pull/386

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/brightchen/apex-malhar APEXMALHAR-2205

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/398.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #398


commit c5a12e4e747c5be840a16e4c932cbc1dbff79894
Author: brightchen 
Date:   2016-08-26T23:09:12Z

APEXMALHAR-2205 #resolve #comment State management benchmark




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXMALHAR-2205) State management benchmark

2016-09-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456736#comment-15456736
 ] 

ASF GitHub Bot commented on APEXMALHAR-2205:


Github user brightchen closed the pull request at:

https://github.com/apache/apex-malhar/pull/386


> State management benchmark
> --
>
> Key: APEXMALHAR-2205
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2205
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: bright chen
>Assignee: bright chen
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-malhar pull request #386: APEXMALHAR-2205 State management benchmark

2016-09-01 Thread brightchen 
Github user brightchen closed the pull request at:

https://github.com/apache/apex-malhar/pull/386


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (APEXMALHAR-2220) Move the FunctionOperator to Malhar library

2016-09-01 Thread Siyuan Hua (JIRA) 
Siyuan Hua created APEXMALHAR-2220:
--

 Summary: Move the FunctionOperator to Malhar library
 Key: APEXMALHAR-2220
 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2220
 Project: Apache Apex Malhar
  Issue Type: Improvement
Reporter: Siyuan Hua
Assignee: Siyuan Hua


FunctionOperator initially is just designed for high-level API and we think it 
can also useful if people want to build stateless transformation and work with 
other operator directly. FunctionOperator can be reused. Thus we should move FO 
to malhar library



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (APEXMALHAR-2142) High-level API window support

2016-09-01 Thread Siyuan Hua (JIRA) 

 [ 
https://issues.apache.org/jira/browse/APEXMALHAR-2142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siyuan Hua updated APEXMALHAR-2142:
---
Attachment: Java-High-level-API-Siyuan.pdf
Apache Beam AutoComplete Example with Apex High-Level API.pdf

Some write-up about high-level API

> High-level API window support
> -
>
> Key: APEXMALHAR-2142
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2142
> Project: Apache Apex Malhar
>  Issue Type: New Feature
>Reporter: Siyuan Hua
>Assignee: Siyuan Hua
>  Labels: roadmap
> Fix For: 3.5.0
>
> Attachments: Apache Beam AutoComplete Example with Apex High-Level 
> API.pdf, Java-High-level-API-Siyuan.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (APEXMALHAR-1939) Stream API

2016-09-01 Thread Siyuan Hua (JIRA) 

 [ 
https://issues.apache.org/jira/browse/APEXMALHAR-1939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siyuan Hua updated APEXMALHAR-1939:
---
Attachment: Java-High-level-API-Siyuan.pdf
Apache Beam AutoComplete Example with Apex High-Level API.pdf

Some writeups about high-level API

> Stream API
> --
>
> Key: APEXMALHAR-1939
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-1939
> Project: Apache Apex Malhar
>  Issue Type: New Feature
>Reporter: Siyuan Hua
>Priority: Critical
>  Labels: roadmap
> Attachments: Apache Beam AutoComplete Example with Apex High-Level 
> API.pdf, Java-High-level-API-Siyuan.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-core pull request #374: APEXCORE-515 Providing principal for token refr...

2016-09-01 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/apex-core/pull/374


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXCORE-515) Refresh tokens failing in some scenarios with a login failure message

2016-09-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXCORE-515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456539#comment-15456539
 ] 

ASF GitHub Bot commented on APEXCORE-515:
-

Github user asfgit closed the pull request at:

https://github.com/apache/apex-core/pull/374


> Refresh tokens failing in some scenarios with a login failure message
> -
>
> Key: APEXCORE-515
> URL: https://issues.apache.org/jira/browse/APEXCORE-515
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pramod Immaneni
>Assignee: Pramod Immaneni
> Fix For: 3.5.0
>
>
> In some scenarios the token refresh to allow applications to run without 
> shutting down is failing with the following exception
> java.io.IOException: Login failure for x from keytab
> at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1146)
> at 
> com.datatorrent.stram.security.StramUserLogin.refreshTokens(StramUserLogin.java:96)
> at 
> com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:623)
> at 
> com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:313)
> Caused by: javax.security.auth.login.LoginException: Unable to obtain 
> password from user
>  
> at 
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897)
> at 
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
> at 
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at 
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
> at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
> at java.security.AccessController.doPrivileged(Native Method)
> at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
> at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1135)
> ... 3 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (APEXMALHAR-2201) Suppress console output in Stream API tests

2016-09-01 Thread Thomas Weise (JIRA) 

 [ 
https://issues.apache.org/jira/browse/APEXMALHAR-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Weise resolved APEXMALHAR-2201.
--
   Resolution: Fixed
Fix Version/s: 3.6.0

> Suppress console output in Stream API tests
> ---
>
> Key: APEXMALHAR-2201
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2201
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: Thomas Weise
>Assignee: Shunxin Lu
> Fix For: 3.6.0
>
>
> There is quite a bit of console output in the CI. It would be nice to 
> suppress that when running as unit test, we are not interested what gets 
> printed to the console but what the test asserts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXCORE-515) Refresh tokens failing in some scenarios with a login failure message

2016-09-01 Thread Thomas Weise (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXCORE-515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456467#comment-15456467
 ] 

Thomas Weise commented on APEXCORE-515:
---

I don't see the change in master, instead I see a branch "APEXCORE-515" ?

> Refresh tokens failing in some scenarios with a login failure message
> -
>
> Key: APEXCORE-515
> URL: https://issues.apache.org/jira/browse/APEXCORE-515
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pramod Immaneni
>Assignee: Pramod Immaneni
> Fix For: 3.5.0
>
>
> In some scenarios the token refresh to allow applications to run without 
> shutting down is failing with the following exception
> java.io.IOException: Login failure for x from keytab
> at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1146)
> at 
> com.datatorrent.stram.security.StramUserLogin.refreshTokens(StramUserLogin.java:96)
> at 
> com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:623)
> at 
> com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:313)
> Caused by: javax.security.auth.login.LoginException: Unable to obtain 
> password from user
>  
> at 
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897)
> at 
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
> at 
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at 
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
> at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
> at java.security.AccessController.doPrivileged(Native Method)
> at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
> at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1135)
> ... 3 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (APEXCORE-515) Refresh tokens failing in some scenarios with a login failure message

2016-09-01 Thread Vlad Rozov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/APEXCORE-515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vlad Rozov resolved APEXCORE-515.
-
   Resolution: Fixed
Fix Version/s: 3.5.0

> Refresh tokens failing in some scenarios with a login failure message
> -
>
> Key: APEXCORE-515
> URL: https://issues.apache.org/jira/browse/APEXCORE-515
> Project: Apache Apex Core
>  Issue Type: Bug
>Reporter: Pramod Immaneni
>Assignee: Pramod Immaneni
> Fix For: 3.5.0
>
>
> In some scenarios the token refresh to allow applications to run without 
> shutting down is failing with the following exception
> java.io.IOException: Login failure for x from keytab
> at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1146)
> at 
> com.datatorrent.stram.security.StramUserLogin.refreshTokens(StramUserLogin.java:96)
> at 
> com.datatorrent.stram.engine.StreamingContainer.heartbeatLoop(StreamingContainer.java:623)
> at 
> com.datatorrent.stram.engine.StreamingContainer.main(StreamingContainer.java:313)
> Caused by: javax.security.auth.login.LoginException: Unable to obtain 
> password from user
>  
> at 
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897)
> at 
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
> at 
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at 
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
> at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
> at java.security.AccessController.doPrivileged(Native Method)
> at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
> at 
> org.apache.hadoop.security.UserGroupInformation.loginUserFromKeytabAndReturnUGI(UserGroupInformation.java:1135)
> ... 3 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (APEXMALHAR-2143) Evaluate and retire lib/math, lib/algo, and lib/streamquery operators

2016-09-01 Thread Thomas Weise (JIRA) 

 [ 
https://issues.apache.org/jira/browse/APEXMALHAR-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Weise resolved APEXMALHAR-2143.
--
   Resolution: Fixed
Fix Version/s: 3.6.0

> Evaluate and retire lib/math, lib/algo, and lib/streamquery operators
> -
>
> Key: APEXMALHAR-2143
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2143
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: David Yan
>Assignee: Velineni Lakshmi Prasanna
> Fix For: 3.6.0
>
>
> The operators in these 3 packages were developed in the early stages of the 
> product and we need to evaluate and assess them to see if there are 
> redundancies, bugs and look at the usability.
> Pramod's suggestion is that we go through the operators in those packages on 
> an
> individual basis and grade them into 3 buckets, those that meet the level
> we expect from the operators (could be few of them), those that are
> potentially useful but need additional work and those that we don't think
> would be useful. The ones in the first bucket can remain in place, the
> second set can be moved to malhar-misc and third set moved to malhar-misc and 
> deprecated.
> Please also track the individual operator in a shared Google spreadsheet.
> Note that the operators in lib/streamquery does not need to be individually 
> evaluated since it is really one unit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2143) Evaluate and retire lib/math, lib/algo, and lib/streamquery operators

2016-09-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456254#comment-15456254
 ] 

ASF GitHub Bot commented on APEXMALHAR-2143:


Github user asfgit closed the pull request at:

https://github.com/apache/apex-malhar/pull/389


> Evaluate and retire lib/math, lib/algo, and lib/streamquery operators
> -
>
> Key: APEXMALHAR-2143
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2143
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: David Yan
>Assignee: Velineni Lakshmi Prasanna
>
> The operators in these 3 packages were developed in the early stages of the 
> product and we need to evaluate and assess them to see if there are 
> redundancies, bugs and look at the usability.
> Pramod's suggestion is that we go through the operators in those packages on 
> an
> individual basis and grade them into 3 buckets, those that meet the level
> we expect from the operators (could be few of them), those that are
> potentially useful but need additional work and those that we don't think
> would be useful. The ones in the first bucket can remain in place, the
> second set can be moved to malhar-misc and third set moved to malhar-misc and 
> deprecated.
> Please also track the individual operator in a shared Google spreadsheet.
> Note that the operators in lib/streamquery does not need to be individually 
> evaluated since it is really one unit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-malhar pull request #389: APEXMALHAR-2143 - Evaluate and retire lib/mat...

2016-09-01 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/apex-malhar/pull/389


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXMALHAR-2143) Evaluate and retire lib/math, lib/algo, and lib/streamquery operators

2016-09-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456108#comment-15456108
 ] 

ASF GitHub Bot commented on APEXMALHAR-2143:


GitHub user prasannapramod reopened a pull request:

https://github.com/apache/apex-malhar/pull/389

APEXMALHAR-2143 - Evaluate and retire lib/math, lib/algo, and 
lib/streamquery operators

@davidyan74 @PramodSSImmaneni @tweise  please review . I am working on 
fixing the functionality of the operators like "BottomNUniqueMap" , 
"InsertSort", "TopN", "UniqueCounter", "MarginKeyVal" . Also, I am planning to 
reimplement some operators like "BottomNMap" using the windows operator and 
managed state .

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/prasannapramod/apex-malhar ApexMalhar-2143

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/389.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #389


commit 8f00cefa2a14756a65e0baad48db0d52e2fe66a3
Author: Lakshmi Prasanna Velineni 
Date:   2016-08-25T03:54:53Z

Updated algo & working on math operators




> Evaluate and retire lib/math, lib/algo, and lib/streamquery operators
> -
>
> Key: APEXMALHAR-2143
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2143
> Project: Apache Apex Malhar
>  Issue Type: Task
>Reporter: David Yan
>Assignee: Velineni Lakshmi Prasanna
>
> The operators in these 3 packages were developed in the early stages of the 
> product and we need to evaluate and assess them to see if there are 
> redundancies, bugs and look at the usability.
> Pramod's suggestion is that we go through the operators in those packages on 
> an
> individual basis and grade them into 3 buckets, those that meet the level
> we expect from the operators (could be few of them), those that are
> potentially useful but need additional work and those that we don't think
> would be useful. The ones in the first bucket can remain in place, the
> second set can be moved to malhar-misc and third set moved to malhar-misc and 
> deprecated.
> Please also track the individual operator in a shared Google spreadsheet.
> Note that the operators in lib/streamquery does not need to be individually 
> evaluated since it is really one unit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] apex-malhar pull request #389: APEXMALHAR-2143 - Evaluate and retire lib/mat...

2016-09-01 Thread prasannapramod 
GitHub user prasannapramod reopened a pull request:

https://github.com/apache/apex-malhar/pull/389

APEXMALHAR-2143 - Evaluate and retire lib/math, lib/algo, and 
lib/streamquery operators

@davidyan74 @PramodSSImmaneni @tweise  please review . I am working on 
fixing the functionality of the operators like "BottomNUniqueMap" , 
"InsertSort", "TopN", "UniqueCounter", "MarginKeyVal" . Also, I am planning to 
reimplement some operators like "BottomNMap" using the windows operator and 
managed state .

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/prasannapramod/apex-malhar ApexMalhar-2143

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/389.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #389


commit 8f00cefa2a14756a65e0baad48db0d52e2fe66a3
Author: Lakshmi Prasanna Velineni 
Date:   2016-08-25T03:54:53Z

Updated algo & working on math operators




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Flaky unit tests

2016-09-01 Thread Thomas Weise 
One more:

Failed tests:
  FileSplitterTest.testRecursive:426 window 2: files expected:<2> but was:<3>

Tests run: 1001, Failures: 1, Errors: 0, Skipped: 3

https://s3.amazonaws.com/archive.travis-ci.org/jobs/156794047/log.txt


On Fri, Aug 26, 2016 at 10:09 AM, Thomas Weise 
wrote:

> I see occasional failures with these two tests in the last few days:
>
> Failed tests:
>   JMSOutputOperatorTest.testBatch:279 Batch should not be written
> expected:<20> but was:<15>
>   POJOTimeBasedJoinOperatorTest.testFullOuterJoinOperator:370 Number of
> tuple emitted  expected:<3> but was:<1>
>
> Can respective authors please have a look? Please file JIRA if applicable.
>
> Thanks
>
>

Re: Malhar 3.5.0 release

2016-09-01 Thread Bhupesh Chawda 
I have a documentation for Deduper:
https://issues.apache.org/jira/browse/APEXMALHAR-2219
Here is the PR: https://github.com/apache/apex-malhar/pull/397

~ Bhupesh

On Thu, Sep 1, 2016 at 4:26 AM, Thomas Weise  wrote:

> Anyone want to contribute something for the other new features:
>
> https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20APEXMALHAR%20AND%20fixVersion%20%3D%203.5.0%
> 20AND%20type%20%3D%20%22New%20Feature%22%20ORDER%20BY%
> 20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>
> With link to example, documentation perhaps ;-)
>
> Thanks,
> Thomas
>
>
> On Sat, Aug 27, 2016 at 4:46 AM, Chinmay Kolhatkar <
> chin...@datatorrent.com>
> wrote:
>
> > https://issues.apache.org/jira/browse/APEXMALHAR-2085 - Windows
> operator -
> > Good to highlight in release announcement.
> >
> >
> > On Sat, Aug 27, 2016 at 4:59 AM, Sanjay Pujare 
> > wrote:
> >
> > > https://issues.apache.org/jira/browse/APEXMALHAR-2156
> > >
> > > The JMS input operator has been enhanced to support Amazon SQS. There
> is
> > a
> > > unit test (SQSStringInputOperatorTest) that demonstrates how to use the
> > > operator for that use case.
> > >
> > > On 8/26/16, 2:04 PM, "Thomas Weise"  wrote:
> > >
> > > There are 2 issues left which unless they get resolved today I
> > suggest
> > > we
> > > move to 3.6
> > >
> > > There is a lot of good stuff going into this release that users may
> > > want to
> > > know about. Can everyone please have a look at resolved issues and
> > > reply if
> > > you see items that you think we should highlight in the release
> > > announcement.
> > >
> > > https://issues.apache.org/jira/issues/?jql=fixVersion%
> > > 20%3D%203.5.0%20AND%20project%20%3D%20APEXMALHAR
> > >
> > >
> > >
> > > On Fri, Aug 26, 2016 at 7:18 AM, Thomas Weise <
> > tho...@datatorrent.com>
> > > wrote:
> > >
> > > > See comments on PR.
> > > >
> > > > On Fri, Aug 26, 2016 at 7:16 AM, Yogi Devendra <
> > > > devendra.vyavah...@gmail.com> wrote:
> > > >
> > > >> Fix for APEXMALHAR-2206 has been reviewed and merged into asf
> > > master.
> > > >>
> > > >> ~ Yogi
> > > >>
> > > >> On 26 August 2016 at 19:04, Yogi Devendra <
> > > devendra.vyavah...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > We noticed that lc.runAsync() is called without corresponding
> > > >> > lc.shutdown() at couple of places in unit tests.
> > > >> >
> > > >> > This caused:
> > > >> > 1. Async thread continuing.
> > > >> > 2. Build not terminating.
> > > >> > 3. target dir size incresing indefinately.
> > > >> >
> > > >> > @All reviewers
> > > >> > Please make sure to check for lc.shutdown() for every
> > > lc.runAsync().
> > > >> >
> > > >> > ~ Yogi
> > > >> >
> > > >> > On 26 August 2016 at 17:07, Bhupesh Chawda <
> > > bhup...@datatorrent.com>
> > > >> > wrote:
> > > >> >
> > > >> >> Hi Thomas,
> > > >> >>
> > > >> >> https://issues.apache.org/jira/browse/APEXMALHAR-2206 is one
> > > small bug
> > > >> >> which Yogi pointed out. It was intermittently taking too long
> > to
> > > >> complete
> > > >> >> the travis builds. We thought it would be a good idea to get
> it
> > > in for
> > > >> the
> > > >> >> release.
> > > >> >>
> > > >> >> I have just opened a PR: https://github.com/apache/apex
> > > >> -malhar/pull/383
> > > >> >>
> > > >> >> ~ Bhupesh
> > > >> >>
> > > >> >>
> > > >> >> On Fri, Aug 26, 2016 at 9:45 AM, Yogi Devendra <
> > > >> >> devendra.vyavah...@gmail.com
> > > >> >> > wrote:
> > > >> >>
> > > >> >> > https://github.com/apache/apex-malhar/pull/372 is open for
> > > >> >> > APEXMALHAR-2195.
> > > >> >> >
> > > >> >> > I have incorporated the review comments from
> @gauravgopi123,
> > > @tweise
> > > >> >> > I have updated fix version to 3.5.0 to reflect this ticket
> in
> > > the
> > > >> query.
> > > >> >> >
> > > >> >> > Can someone please review and merge this for 3.5.0?
> > > >> >> >
> > > >> >> > ~ Yogi
> > > >> >> >
> > > >> >> > On 25 August 2016 at 21:41, Chaitanya Chebolu <
> > > >> >> chaita...@datatorrent.com>
> > > >> >> > wrote:
> > > >> >> >
> > > >> >> > > Hi Thomas,
> > > >> >> > >
> > > >> >> > >I will fix APEXMALHAR-2134.
> > > >> >> > >
> > > >> >> > > Regards,
> > > >> >> > > Chaitanya
> > > >> >> > >
> > > >> >> > > On Thu, Aug 25, 2016 at 9:13 PM, Thomas Weise <
> > > >> tho...@datatorrent.com
> > > >> >> >
> > > >> >> > > wrote:
> > > >> >> > >
> > > >> >> > > > We are almost ready to get the RC out.
> > > >> >> > > >
> > > >> >> > > > Can someone fix APEXMALHAR-2134 for the release?
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > On Wed, Aug 17, 2016 at 12:56 PM, Thomas

[GitHub] apex-malhar pull request #397: APEXMALHAR-2219: Added documentation for Dedu...

2016-09-01 Thread bhupeshchawda 
GitHub user bhupeshchawda opened a pull request:

https://github.com/apache/apex-malhar/pull/397

APEXMALHAR-2219: Added documentation for Deduper



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bhupeshchawda/apex-malhar 
APEXMALHAR-2219-deduper-documentation

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/397.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #397


commit 0062edc6e10deb6a1ec99a2115042cbd67d830e0
Author: bhupeshchawda 
Date:   2016-09-01T12:24:08Z

APEXMALHAR-2219: Added documentation for Deduper




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (APEXMALHAR-2219) Add documentation for Deduper

2016-09-01 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455228#comment-15455228
 ] 

ASF GitHub Bot commented on APEXMALHAR-2219:


GitHub user bhupeshchawda opened a pull request:

https://github.com/apache/apex-malhar/pull/397

APEXMALHAR-2219: Added documentation for Deduper



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/bhupeshchawda/apex-malhar 
APEXMALHAR-2219-deduper-documentation

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/397.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #397


commit 0062edc6e10deb6a1ec99a2115042cbd67d830e0
Author: bhupeshchawda 
Date:   2016-09-01T12:24:08Z

APEXMALHAR-2219: Added documentation for Deduper




> Add documentation for Deduper
> -
>
> Key: APEXMALHAR-2219
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2219
> Project: Apache Apex Malhar
>  Issue Type: Documentation
>Reporter: Bhupesh Chawda
>Assignee: Bhupesh Chawda
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)