Improvement: Collecting metric

2017-03-14 Thread Chinmay Kolhatkar
Dear Community, While going through the code, I found that the AutoMetrics are retrieved from operators using java reflection. This adds a lot of processing overhead. Hence to improve the performance of containers and stram, I want to suggest following appraoch: 1. In Node, try to collect the met

[jira] [Commented] (APEXCORE-660) Documentation for Control tuple support changes

2017-03-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925530#comment-15925530 ] ASF GitHub Bot commented on APEXCORE-660: - GitHub user bhupeshchawda opened a pu

[GitHub] apex-core pull request #484: [REVIEW ONLY] APEXCORE-660 Added documentation ...

2017-03-14 Thread bhupeshchawda
GitHub user bhupeshchawda opened a pull request: https://github.com/apache/apex-core/pull/484 [REVIEW ONLY] APEXCORE-660 Added documentation for custom control tuple support @tweise Please see. Currently just opened PR for review. Waiting for any other comments over the re

[jira] [Assigned] (APEXCORE-580) Interface for processing and emitting control tuples

2017-03-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reassigned APEXCORE-580: - Assignee: Bhupesh Chawda (was: Bhupesh Chawda) > Interface for processing and emitting

[jira] [Assigned] (APEXCORE-579) Custom control tuple support

2017-03-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reassigned APEXCORE-579: - Assignee: Bhupesh Chawda (was: Bhupesh Chawda) > Custom control tuple support > --

[jira] [Assigned] (APEXCORE-581) Delivery of Custom Control Tuples

2017-03-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise reassigned APEXCORE-581: - Assignee: Bhupesh Chawda (was: Bhupesh Chawda) > Delivery of Custom Control Tuples > -

[jira] [Updated] (APEXMALHAR-2206) Some Application tests taking too long

2017-03-14 Thread Gavin (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gavin updated APEXMALHAR-2206: -- Reporter: Bhupesh Chawda (was: Bhupesh Chawda) > Some Application tests taking too long >

[GitHub] apex-site pull request #69: Updating rpm & deb packages for apex from bigtop

2017-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-site/pull/69 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] apex-site pull request #69: Updating rpm & deb packages for apex from bigtop

2017-03-14 Thread chinmaykolhatkar
GitHub user chinmaykolhatkar opened a pull request: https://github.com/apache/apex-site/pull/69 Updating rpm & deb packages for apex from bigtop @tweise Please review and merge. These are changes related to bigtop upgrade of apex to 3.5.0: 1. Bigtop upgrade of apex has happe

[jira] [Updated] (APEXMALHAR-2444) Documentation for Highlevel API

2017-03-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXMALHAR-2444: - Priority: Major (was: Minor) > Documentation for Highlevel API > ---

[jira] [Moved] (APEXMALHAR-2444) Documentation for Highlevel API

2017-03-14 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise moved APEXCORE-666 to APEXMALHAR-2444: --- Component/s: (was: Documentation) do

[jira] [Created] (APEXCORE-667) Documentation for Modules.

2017-03-14 Thread Tushar Gosavi (JIRA)
Tushar Gosavi created APEXCORE-667: -- Summary: Documentation for Modules. Key: APEXCORE-667 URL: https://issues.apache.org/jira/browse/APEXCORE-667 Project: Apache Apex Core Issue Type: Docum

[jira] [Created] (APEXCORE-666) Documentation for Highlevel API

2017-03-14 Thread Tushar Gosavi (JIRA)
Tushar Gosavi created APEXCORE-666: -- Summary: Documentation for Highlevel API Key: APEXCORE-666 URL: https://issues.apache.org/jira/browse/APEXCORE-666 Project: Apache Apex Core Issue Type:

Re: Sort Accumulation

2017-03-14 Thread Bright Chen
Hi Ajay, My feeling is you assume handle all tuples in memory. How to handle the case if memory is not enough to hold all tuples? thanks -Bright On Mon, Mar 13, 2017 at 11:00 PM, AJAY GUPTA wrote: > Hi Apex Dev community, > > Kindly let me know if implementing this accumulation using TreeMultiS

Re: Apex project description

2017-03-14 Thread Nathan Trueblood
+1 to the suggestion about linking to what we mean by scalable and fault tolerant. Many projects falsely claim these capabilities, but of course your "your mileage may vary." ;) -Nathan On Sat, Mar 11, 2017 at 2:13 PM David Yan wrote: > Here's my proposal. I added one more sentence in the de

[jira] [Commented] (APEXCORE-577) Visitor API for DAG

2017-03-14 Thread Pramod Immaneni (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924402#comment-15924402 ] Pramod Immaneni commented on APEXCORE-577: -- As a follow up please add versionin

[jira] [Commented] (APEXCORE-577) Visitor API for DAG

2017-03-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924400#comment-15924400 ] ASF GitHub Bot commented on APEXCORE-577: - Github user asfgit closed the pull re

[GitHub] apex-core pull request #456: APEXCORE-577 Add visitor API to examine the DAG...

2017-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-core/pull/456 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

Re: APEXCORE-619 Recovery windowId in future during application relaunch.

2017-03-14 Thread Thomas Weise
The JDBC operator was an example of a valid use case where no state needs to be saved in a checkpoint (it is saved to the database instead). We cannot break such use cases and neither should the application developer be exposed to it, especially not in backward incompatible fashion. I think that t

Re: Improving Apex relaunch time.

2017-03-14 Thread Bhupesh Chawda
Hi All, The PR: https://github.com/apache/apex-core/pull/422 to solve this issue looks good to me. If there are no other comments, will merge this PR soon. ~ Bhupesh ___ Bhupesh Chawda E: bhup...@datatorrent.com | Twitter: @bhupeshsc www.da

[jira] [Commented] (APEXCORE-575) Improve application relaunch time.

2017-03-14 Thread Bhupesh Chawda (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924130#comment-15924130 ] Bhupesh Chawda commented on APEXCORE-575: - [~tushargosavi] Can you please update

[jira] [Resolved] (APEXCORE-617) InputNodeTest intermittently fails with ConcurrentModificationException

2017-03-14 Thread Tushar Gosavi (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tushar Gosavi resolved APEXCORE-617. Resolution: Fixed Fix Version/s: 3.6.0 > InputNodeTest intermittently fails with Co

[jira] [Commented] (APEXCORE-617) InputNodeTest intermittently fails with ConcurrentModificationException

2017-03-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924089#comment-15924089 ] ASF GitHub Bot commented on APEXCORE-617: - Github user asfgit closed the pull re

[GitHub] apex-core pull request #451: APEXCORE-617 InputNodeTest intermittently fails...

2017-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-core/pull/451 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

Re: APEXCORE-619 Recovery windowId in future during application relaunch.

2017-03-14 Thread Tushar Gosavi
Hi Vlad, I am more worried about backward compatibility. Most of the new users will be not able to launch simple applications which are starting point while learning Apex such as PiDemo and WordCount. We could provide a way for user to not use empty file storage agent for leaf stateless operators

[jira] [Updated] (APEXMALHAR-2408) Issues in correctness of get() for key search in ManagedTimeStateImpl

2017-03-14 Thread Bhupesh Chawda (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhupesh Chawda updated APEXMALHAR-2408: --- Summary: Issues in correctness of get() for key search in ManagedTimeStateImpl (

[jira] [Commented] (APEXMALHAR-2408) Validate correctness of get() for key search in ManagedTimeStateImpl

2017-03-14 Thread Bhupesh Chawda (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924001#comment-15924001 ] Bhupesh Chawda commented on APEXMALHAR-2408: Fixes included in PR: https:

[jira] [Commented] (APEXMALHAR-2407) Time buckets are not purging after expiry

2017-03-14 Thread Bhupesh Chawda (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923999#comment-15923999 ] Bhupesh Chawda commented on APEXMALHAR-2407: Fixes included in PR: https:

[jira] [Updated] (APEXMALHAR-2406) ManagedState Issues

2017-03-14 Thread Bhupesh Chawda (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhupesh Chawda updated APEXMALHAR-2406: --- Description: Few bugs encountered while using ManagedState. Specific issues descr

[jira] [Commented] (APEXMALHAR-2400) In PojoInnerJoin accumulation same field names are emitted as single field

2017-03-14 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923943#comment-15923943 ] ASF GitHub Bot commented on APEXMALHAR-2400: GitHub user Hitesh-Scorpio o

[GitHub] apex-malhar pull request #581: APEXMALHAR-2400 removing dependency of output...

2017-03-14 Thread Hitesh-Scorpio
GitHub user Hitesh-Scorpio opened a pull request: https://github.com/apache/apex-malhar/pull/581 APEXMALHAR-2400 removing dependency of output fieldnames to input fie… …ld names @chinmaykolhatkar please review. You can merge this pull request into a Git repository by running:

[jira] [Commented] (APEXCORE-593) apex cli get-app-package-info could not retrieve properties defined in properties.xml

2017-03-14 Thread Vikram Patil (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923840#comment-15923840 ] Vikram Patil commented on APEXCORE-593: --- Following steps are executed when get-app

[jira] [Commented] (APEXCORE-593) apex cli get-app-package-info could not retrieve properties defined in properties.xml

2017-03-14 Thread Bhupesh Chawda (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923801#comment-15923801 ] Bhupesh Chawda commented on APEXCORE-593: - [~vikram] Can you please add more det

[jira] [Commented] (APEXMALHAR-2406) ManagedState Issues

2017-03-14 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923717#comment-15923717 ] Chaitanya commented on APEXMALHAR-2406: --- By mistake, I added it under comment.

[jira] [Issue Comment Deleted] (APEXMALHAR-2408) Validate correctness of get() for key search in ManagedTimeStateImpl

2017-03-14 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaitanya updated APEXMALHAR-2408: -- Comment: was deleted (was: Below are the issues: 1) When the key searches all time buckets,

[jira] [Updated] (APEXMALHAR-2408) Validate correctness of get() for key search in ManagedTimeStateImpl

2017-03-14 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaitanya updated APEXMALHAR-2408: -- Description: Below are the issues: 1) When the key searches all time buckets, not checking

[jira] [Updated] (APEXMALHAR-2407) Time buckets are not purging after expiry

2017-03-14 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaitanya updated APEXMALHAR-2407: -- Description: Below are the observations: 1) purgeListner is not triggering purgeTimeBuckets

[jira] [Issue Comment Deleted] (APEXMALHAR-2407) Time buckets are not purging after expiry

2017-03-14 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaitanya updated APEXMALHAR-2407: -- Comment: was deleted (was: Below are the observations: 1) purgeListner is not triggering p