Re: [DISCUSSION] Custom Control Tuples

2016-06-24 Thread Vlad Rozov
+1 for option 1. Thank you, Vlad On 6/24/16 14:35, Bright Chen wrote: +1 It also can help to Shutdown the application gracefully. Bright On Jun 24, 2016, at 1:35 PM, Siyuan Hua wrote: +1 I think it's good to have custom control tuple and I prefer the 1 option.

[jira] [Commented] (APEXMALHAR-2085) Implement Windowed Operators

2016-06-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348844#comment-15348844 ] ASF GitHub Bot commented on APEXMALHAR-2085: Github user brightchen commented on a diff

[GitHub] apex-malhar pull request #319: APEXMALHAR-2085: REVIEW ONLY: Operator suppor...

2016-06-24 Thread brightchen
Github user brightchen commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/319#discussion_r68475212 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/Window.java --- @@ -0,0 +1,179 @@ +/** + * Licensed to the Apache Software

[GitHub] apex-core issue #351: APEXCORE-405 Common API to launch on local mode or clu...

2016-06-24 Thread PramodSSImmaneni
Github user PramodSSImmaneni commented on the issue: https://github.com/apache/apex-core/pull/351 @siyuanh @tushargosavi please see as you have expressed interest in JIRA --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] apex-malhar pull request #319: APEXMALHAR-2085: REVIEW ONLY: Operator suppor...

2016-06-24 Thread chinmaykolhatkar
Github user chinmaykolhatkar commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/319#discussion_r68474418 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/AbstractWindowedOperator.java --- @@ -0,0 +1,486 @@ +/** + *

[jira] [Commented] (APEXCORE-405) Provide an API to launch DAG on the cluster

2016-06-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348824#comment-15348824 ] ASF GitHub Bot commented on APEXCORE-405: - Github user PramodSSImmaneni commented on the issue:

[jira] [Commented] (APEXCORE-475) change the YARN_APPLICATION_TYPE from DataTorrent to ApacheApex

2016-06-24 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348800#comment-15348800 ] David Yan commented on APEXCORE-475: In a development environment, yes, that can be an issue. >

[jira] [Commented] (APEXCORE-475) change the YARN_APPLICATION_TYPE from DataTorrent to ApacheApex

2016-06-24 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348793#comment-15348793 ] Sandesh commented on APEXCORE-475: -- They can be using RTS 3.4, but launching the apps using 3.5 Apex

[jira] [Commented] (APEXMALHAR-2085) Implement Windowed Operators

2016-06-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348758#comment-15348758 ] ASF GitHub Bot commented on APEXMALHAR-2085: Github user chinmaykolhatkar commented on a

[jira] [Commented] (APEXCORE-475) change the YARN_APPLICATION_TYPE from DataTorrent to ApacheApex

2016-06-24 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348743#comment-15348743 ] David Yan commented on APEXCORE-475: We don't expect users to have multiple Apex installations with

[jira] [Resolved] (APEXMALHAR-2120) Fix bugs on KafkaInputOperatorTest and AbstractKafkaInputOperator

2016-06-24 Thread bright chen (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bright chen resolved APEXMALHAR-2120. - Resolution: Fixed > Fix bugs on KafkaInputOperatorTest and

Re: [DISCUSSION] Custom Control Tuples

2016-06-24 Thread Bright Chen
+1 It also can help to Shutdown the application gracefully. Bright > On Jun 24, 2016, at 1:35 PM, Siyuan Hua wrote: > > +1 > > I think it's good to have custom control tuple and I prefer the 1 option. > > Also I think we should think about couple different callbacks,

[jira] [Commented] (APEXMALHAR-2120) Fix bugs on KafkaInputOperatorTest and AbstractKafkaInputOperator

2016-06-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348699#comment-15348699 ] ASF GitHub Bot commented on APEXMALHAR-2120: Github user asfgit closed the pull request

[GitHub] apex-malhar pull request #321: APEXMALHAR-2120 #resolve #comment solve probl...

2016-06-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/321 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (APEXCORE-475) change the YARN_APPLICATION_TYPE from DataTorrent to ApacheApex

2016-06-24 Thread David Yan (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348653#comment-15348653 ] David Yan commented on APEXCORE-475: If we change the application type to ApacheApex for all newly

Re: [DISCUSSION] Custom Control Tuples

2016-06-24 Thread David Yan
My initial thinking is that the custom control tuples, just like the existing control tuples, will only be generated from the input operators and will be propagated downstream to all operators in the DAG. So the NxM partitioning scenario works just like how other control tuples work, i.e. the

[jira] [Commented] (APEXMALHAR-2085) Implement Windowed Operators

2016-06-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348583#comment-15348583 ] ASF GitHub Bot commented on APEXMALHAR-2085: Github user davidyan74 commented on a diff

[jira] [Commented] (APEXMALHAR-2085) Implement Windowed Operators

2016-06-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15348582#comment-15348582 ] ASF GitHub Bot commented on APEXMALHAR-2085: Github user davidyan74 commented on a diff

[GitHub] apex-malhar pull request #319: APEXMALHAR-2085: REVIEW ONLY: Operator suppor...

2016-06-24 Thread davidyan74
Github user davidyan74 commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/319#discussion_r68450198 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/window/impl/KeyedWindowedOperatorImpl.java --- @@ -0,0 +1,159 @@ +/** + *

[GitHub] apex-malhar pull request #321: APEXMALHAR-2120 #resolve #comment solve probl...

2016-06-24 Thread siyuanh
Github user siyuanh commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/321#discussion_r68420530 --- Diff: kafka/src/test/java/org/apache/apex/malhar/kafka/KafkaInputOperatorTest.java --- @@ -264,49 +336,57 @@ public void testInputOperator(boolean

[GitHub] apex-core pull request #351: APEXCORE-405 Common API to launch on local mode...

2016-06-24 Thread PramodSSImmaneni
GitHub user PramodSSImmaneni opened a pull request: https://github.com/apache/apex-core/pull/351 APEXCORE-405 Common API to launch on local mode or cluster @tweise please take a look. Common API to launch on local mode or cluster. No functionality lost in local mode but

[jira] [Commented] (APEXCORE-405) Provide an API to launch DAG on the cluster

2016-06-24 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15347935#comment-15347935 ] ASF GitHub Bot commented on APEXCORE-405: - GitHub user PramodSSImmaneni opened a pull request: