[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349957#comment-15349957 ] ASF GitHub Bot commented on APEXCORE-222: - Github user sandeshh commented on a d

[jira] [Created] (APEXCORE-476) Window Id calculation in ToString method in SubscribeRequestTuple & GenericRequestTuple is wrong

2016-06-25 Thread Sandesh (JIRA)
Sandesh created APEXCORE-476: Summary: Window Id calculation in ToString method in SubscribeRequestTuple & GenericRequestTuple is wrong Key: APEXCORE-476 URL: https://issues.apache.org/jira/browse/APEXCORE-476

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349910#comment-15349910 ] ASF GitHub Bot commented on APEXCORE-222: - Github user tweise commented on a dif

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-25 Thread tweise
Github user tweise commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r68496167 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java --- @@ -769,7 +769,11 @@ public void processHeartbeatResponse(Container

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349909#comment-15349909 ] ASF GitHub Bot commented on APEXCORE-222: - Github user sandeshh commented on a d

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-25 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r68496137 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java --- @@ -769,7 +769,11 @@ public void processHeartbeatResponse(Contain

[jira] [Commented] (APEXCORE-148) Auto metric: metric aggregations test is failing

2016-06-25 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349897#comment-15349897 ] Sandesh commented on APEXCORE-148: -- I have already planned to resolve few of the core i

[jira] [Updated] (APEXCORE-148) Auto metric: metric aggregations test is failing

2016-06-25 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXCORE-148: -- Assignee: (was: Chandni Singh) > Auto metric: metric aggregations test is failing > --

[jira] [Commented] (APEXCORE-148) Auto metric: metric aggregations test is failing

2016-06-25 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349867#comment-15349867 ] Thomas Weise commented on APEXCORE-148: --- I think we need to devote some time to ha

[jira] [Commented] (APEXCORE-148) Auto metric: metric aggregations test is failing

2016-06-25 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349843#comment-15349843 ] Sandesh commented on APEXCORE-148: -- Is it time to reduce the priority of this jira? >

[jira] [Commented] (APEXCORE-475) change the YARN_APPLICATION_TYPE from DataTorrent to ApacheApex

2016-06-25 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349834#comment-15349834 ] Sandesh commented on APEXCORE-475: -- Agree, this is about forward compatibility. It wil

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349824#comment-15349824 ] ASF GitHub Bot commented on APEXCORE-222: - Github user sandeshh commented on a d

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-25 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r68493910 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java --- @@ -769,7 +769,11 @@ public void processHeartbeatResponse(Contain

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-25 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r68493801 --- Diff: bufferserver/src/main/java/com/datatorrent/bufferserver/internal/DataList.java --- @@ -176,21 +176,20 @@ public void reset() numberOfI

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349820#comment-15349820 ] ASF GitHub Bot commented on APEXCORE-222: - Github user sandeshh commented on a d

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349819#comment-15349819 ] ASF GitHub Bot commented on APEXCORE-222: - Github user sandeshh commented on a d

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-25 Thread sandeshh
Github user sandeshh commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r68493761 --- Diff: engine/src/main/java/com/datatorrent/stram/StramLocalCluster.java --- @@ -72,7 +72,7 @@ protected final StreamingContainerManager dnmgr;

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349773#comment-15349773 ] ASF GitHub Bot commented on APEXCORE-222: - Github user tweise commented on a dif

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-25 Thread tweise
Github user tweise commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r68493464 --- Diff: engine/src/main/java/com/datatorrent/stram/engine/StreamingContainer.java --- @@ -769,7 +769,11 @@ public void processHeartbeatResponse(Container

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349771#comment-15349771 ] ASF GitHub Bot commented on APEXCORE-222: - Github user tweise commented on a dif

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-25 Thread tweise
Github user tweise commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r68493431 --- Diff: engine/src/main/java/com/datatorrent/stram/StramLocalCluster.java --- @@ -72,7 +72,7 @@ protected final StreamingContainerManager dnmgr;

[jira] [Commented] (APEXCORE-222) Delegate Buffer Server purge to StreamingContainer

2016-06-25 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349770#comment-15349770 ] ASF GitHub Bot commented on APEXCORE-222: - Github user tweise commented on a dif

[GitHub] apex-core pull request #350: APEXCORE-222 Purging the stale data present in ...

2016-06-25 Thread tweise
Github user tweise commented on a diff in the pull request: https://github.com/apache/apex-core/pull/350#discussion_r68493398 --- Diff: bufferserver/src/main/java/com/datatorrent/bufferserver/internal/DataList.java --- @@ -176,21 +176,20 @@ public void reset() numberOfInM

[jira] [Commented] (APEXCORE-475) change the YARN_APPLICATION_TYPE from DataTorrent to ApacheApex

2016-06-25 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349768#comment-15349768 ] Thomas Weise commented on APEXCORE-475: --- The suggestion from [~davidyan] is backwa

Re: [DISCUSSION] Custom Control Tuples

2016-06-25 Thread Thomas Weise
It should not matter from where the control tuple is triggered. It will be good to have a generic mechanism to propagate it and other things can be accomplished outside the engine. For example, the new comprehensive support for windowing will all be in Malhar, nothing that the engine needs to know

Re: [DISCUSSION] Custom Control Tuples

2016-06-25 Thread Amol Kekre
I did not say that "notably" does not mean "exclusive" Thks Amol On Sat, Jun 25, 2016 at 9:29 AM, Sandesh Hegde wrote: > Why restrict the control tuples to input operators? > > On Sat, Jun 25, 2016 at 9:07 AM Amol Kekre wrote: > > > David, > > We should avoid control tuple within the window b

Re: [DISCUSSION] Custom Control Tuples

2016-06-25 Thread Sandesh Hegde
Why restrict the control tuples to input operators? On Sat, Jun 25, 2016 at 9:07 AM Amol Kekre wrote: > David, > We should avoid control tuple within the window by simply restricting it > through API. This can be done by calling something like "sendControlTuple" > between windows, notably in inp

Re: [DISCUSSION] Custom Control Tuples

2016-06-25 Thread Amol Kekre
David, We should avoid control tuple within the window by simply restricting it through API. This can be done by calling something like "sendControlTuple" between windows, notably in input operators. Thks Amol On Sat, Jun 25, 2016 at 7:32 AM, Munagala Ramanath wrote: > What would the API look

[jira] [Commented] (APEXCORE-475) change the YARN_APPLICATION_TYPE from DataTorrent to ApacheApex

2016-06-25 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15349688#comment-15349688 ] Sandesh commented on APEXCORE-475: -- We can implement part of your suggestion now, that

Re: [DISCUSSION] Custom Control Tuples

2016-06-25 Thread Munagala Ramanath
What would the API look like for option 1 ? Another operator callback called controlTuple() or does the operator code have to check each incoming tuple to see if it was data or control ? Ram On Fri, Jun 24, 2016 at 11:42 PM, David Yan wrote: > It looks like option 1 is preferred by the communit

Re: [DISCUSSION] Custom Control Tuples

2016-06-25 Thread Sasha Parfenov
+1 I think this is a great feature, and is needed for all the reasons stated above, as well as supporting integrations with frameworks like Apache Beam. Giving users more control by supporting custom serialized objects, Option 1, would be my preference. Thanks, Sasha On Sat, Jun 25, 2016 at 6:1

[jira] [Resolved] (APEXCORE-214) Containers Fail With RPC Timeout

2016-06-25 Thread Pramod Immaneni (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pramod Immaneni resolved APEXCORE-214. -- Resolution: Implemented Already implemented with a setting to change timeout value >

Re: [DISCUSSION] Custom Control Tuples

2016-06-25 Thread Pramod Immaneni
For the use cases you mentioned, I think 1) and 2) are more likely to be controlled directly by the application, 3) and 4) are more likely going to be triggered externally and directly handled by the engine and 3) is already being implemented that way (apexcore-163). The control tuples emitted by