[GitHub] apex-malhar pull request #367: SPOI-8830: Adding documentation for File Inpu...

GitHub user DT-Priyanka opened a pull request: https://github.com/apache/apex-malhar/pull/367 SPOI-8830: Adding documentation for File Input Operator @yogidevendra please review. You can merge this pull request into a Git repository by running: $ git pull

[jira] [Commented] (APEXMALHAR-2063) Integrate WAL to FS WindowDataManager

[ https://issues.apache.org/jira/browse/APEXMALHAR-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416592#comment-15416592 ] ASF GitHub Bot commented on APEXMALHAR-2063: Github user ilooner commented on a diff in

[GitHub] apex-malhar pull request #322: APEXMALHAR-2063 Made window data manager use ...

Github user ilooner commented on a diff in the pull request: https://github.com/apache/apex-malhar/pull/322#discussion_r74370558 --- Diff: library/src/main/java/org/apache/apex/malhar/lib/wal/FSWindowDataManager.java --- @@ -51,291 +66,607 @@ public class FSWindowDataManager

[jira] [Commented] (APEXCORE-494) Window id of downstream operator is not moving after dynamic partition of upstream operator.

[ https://issues.apache.org/jira/browse/APEXCORE-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416584#comment-15416584 ] ASF GitHub Bot commented on APEXCORE-494: - Github user tweise commented on the issue:

[GitHub] apex-core issue #362: APEXCORE-494 Fixed the dynamic partition issue in case...

Github user tweise commented on the issue: https://github.com/apache/apex-core/pull/362 See #365 for correct fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[jira] [Commented] (APEXCORE-494) Window id of downstream operator is not moving after dynamic partition of upstream operator.

[ https://issues.apache.org/jira/browse/APEXCORE-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416583#comment-15416583 ] ASF GitHub Bot commented on APEXCORE-494: - GitHub user tweise opened a pull request:

[GitHub] apex-core pull request #365: APEXCORE-494 Fix scale up from single partition...

GitHub user tweise opened a pull request: https://github.com/apache/apex-core/pull/365 APEXCORE-494 Fix scale up from single partition when partition remain… …s unchanged. You can merge this pull request into a Git repository by running: $ git pull

Re: [ANNOUNCE] New Apache Apex Committer: Devendra Tagare

Congratulations Dev ! Thanks, Shubham On Thu, Aug 11, 2016 at 10:43 AM, Bhupesh Chawda wrote: > Congrats Dev!! > > ~ Bhupesh > > On Thu, Aug 11, 2016 at 2:01 AM, Pradeep A. Dalvi wrote: > > > Congratulations Dev. Welcome aboard. > > > > --prad > > > >

[jira] [Resolved] (APEXMALHAR-2075) Support fields of type Date,Time and Timestamp in Pojo Class for JdbcPOJOInputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shubham pathak resolved APEXMALHAR-2075. Resolution: Fixed > Support fields of type Date,Time and Timestamp in Pojo

[jira] [Resolved] (APEXMALHAR-2074) Support fields of type Date,Time and Timestamp in Pojo Class for JdbcPOJOInputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shubham pathak resolved APEXMALHAR-2074. Resolution: Fixed > Support fields of type Date,Time and Timestamp in Pojo

Re: [ANNOUNCE] New Apache Apex Committer: Devendra Tagare

Congrats Dev!! ~ Bhupesh On Thu, Aug 11, 2016 at 2:01 AM, Pradeep A. Dalvi wrote: > Congratulations Dev. Welcome aboard. > > --prad > > On Wed, Aug 10, 2016 at 1:28 PM, Bright Chen > wrote: > > > Devendra, Congratulations > > > > Best, > > Bright > > >

[jira] [Created] (APEXMALHAR-2183) Add user document for CsvFormatter operator

Venkatesh Kottapalli created APEXMALHAR-2183: Summary: Add user document for CsvFormatter operator Key: APEXMALHAR-2183 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2183 Project:

Re: can operators emit on a different from the operator itself thread?

+1 on debug proposal. Even if tuples lands up within the window, it breaks all guarantees. A rerun (after restart from a checkpoint) can have tuples in different windows from this thread. A separate thread simply exposes users to unwarranted risks. Thks Amol On Wed, Aug 10, 2016 at 6:05 PM,

[jira] [Commented] (APEXMALHAR-2182) benchmark for spillable data structure and storage

[ https://issues.apache.org/jira/browse/APEXMALHAR-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416313#comment-15416313 ] David Yan commented on APEXMALHAR-2182: --- [~brightchen] Thanks for creating this JIRA to track

[jira] [Created] (APEXMALHAR-2182) benchmark for spillable data structure and storage

bright chen created APEXMALHAR-2182: --- Summary: benchmark for spillable data structure and storage Key: APEXMALHAR-2182 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2182 Project: Apache

[jira] [Resolved] (APEXMALHAR-2156) JMS Input operator enhancements

[ https://issues.apache.org/jira/browse/APEXMALHAR-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjay M Pujare resolved APEXMALHAR-2156. - Resolution: Fixed Fix Version/s: 3.5.0 > JMS Input operator

[jira] [Created] (APEXMALHAR-2181) Non-Transactional Prepared Statement Based Cassandra Output Operator

Ananth created APEXMALHAR-2181: -- Summary: Non-Transactional Prepared Statement Based Cassandra Output Operator Key: APEXMALHAR-2181 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2181 Project:

Re: can operators emit on a different from the operator itself thread?

If the goal is to do this validation through static analysis of operator code, I guess it is possible but is going to be non-trivial. And there could be false positives and false negatives. Also I suppose this discussion applies to processor operators (those having both in and out ports) so

Re: can operators emit on a different from the operator itself thread?

Sanjay just reminded me of my typo -> I meant between end_window and start_window :) Thks Amol On Wed, Aug 10, 2016 at 2:36 PM, Sanjay Pujare wrote: > If the goal is to do this validation through static analysis of operator > code, I guess it is possible but is going to

Re: can operators emit on a different from the operator itself thread?

Send too soon. A quicker way would be to catch emit happening between start_window and end_window and flag an error. Catching "another thread" for every tuple may have a huge performance hit. Thks Amol On Wed, Aug 10, 2016 at 2:31 PM, Amol Kekre wrote: > > Currently user

Re: can operators emit on a different from the operator itself thread?

Currently user can code it that way. IMHO Apex should catch this and flag error. Thks Amol On Wed, Aug 10, 2016 at 2:04 PM, Ashwin Chandra Putta < ashwinchand...@gmail.com> wrote: > In a separate thread I mean. > > Regards, > Ashwin. > > On Wed, Aug 10, 2016 at 2:01 PM, Ashwin Chandra Putta <

[jira] [Commented] (APEXMALHAR-2129) ManagedState: Add a disable purging option

[ https://issues.apache.org/jira/browse/APEXMALHAR-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416044#comment-15416044 ] ASF GitHub Bot commented on APEXMALHAR-2129: GitHub user chandnisingh opened a pull

[GitHub] apex-malhar pull request #366: APEXMALHAR-2129 removed moving time boundarie...

GitHub user chandnisingh opened a pull request: https://github.com/apache/apex-malhar/pull/366 APEXMALHAR-2129 removed moving time boundaries periodically and async… …hrnously based on system time You can merge this pull request into a Git repository by running: $ git pull

Re: can operators emit on a different from the operator itself thread?

In a separate thread I mean. Regards, Ashwin. On Wed, Aug 10, 2016 at 2:01 PM, Ashwin Chandra Putta < ashwinchand...@gmail.com> wrote: > + dev@apex.apache.org > - us...@apex.apache.org > > This is one of those best practices that we learn by experience during > operator development. It will

Re: can operators emit on a different from the operator itself thread?

+ dev@apex.apache.org - us...@apex.apache.org This is one of those best practices that we learn by experience during operator development. It will save a lot of time during operator development if we can catch and throw validation error when someone emits tuples in a non separate thread.

Re: anti-affinity - parameter to control the containers of an operator on a node

I don't think we have such a feature, it will be a good addition. A configuration parameter for number of operators in a container would be useful as well. Regards, Ashwin. On Wed, Aug 10, 2016 at 11:53 AM, Amol Kekre wrote: > Good idea. > > Thks > Amol > > > On Wed, Aug

[jira] [Updated] (APEXMALHAR-2129) ManagedState: Add a disable purging option

[ https://issues.apache.org/jira/browse/APEXMALHAR-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated APEXMALHAR-2129: -- Description: Have an option that can disable purging of data. Currently the

[jira] [Updated] (APEXMALHAR-2129) ManagedState: Add a disable purging option

[ https://issues.apache.org/jira/browse/APEXMALHAR-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandni Singh updated APEXMALHAR-2129: -- Description: Have an option that can disable purging of data. Currently the

[jira] [Updated] (APEXMALHAR-2168) Bug Found in JdbcPOJOInputOperator.

[ https://issues.apache.org/jira/browse/APEXMALHAR-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyuan Hua updated APEXMALHAR-2168: --- Fix Version/s: 3.5.0 > Bug Found in JdbcPOJOInputOperator. >

[jira] [Commented] (APEXMALHAR-2168) Bug Found in JdbcPOJOInputOperator.

[ https://issues.apache.org/jira/browse/APEXMALHAR-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415956#comment-15415956 ] ASF GitHub Bot commented on APEXMALHAR-2168: Github user asfgit closed the pull request

[GitHub] apex-malhar pull request #352: APEXMALHAR-2168 #resolve #comment Bug fixed i...

Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/352 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (APEXMALHAR-2028) Add System.err to ConsoleOutputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415953#comment-15415953 ] ASF GitHub Bot commented on APEXMALHAR-2028: GitHub user siyuanh opened a pull request:

[GitHub] apex-malhar pull request #365: APEXMALHAR-2028 #resolve #comment add stderr ...

GitHub user siyuanh opened a pull request: https://github.com/apache/apex-malhar/pull/365 APEXMALHAR-2028 #resolve #comment add stderr for ConsoleOperator Add stderr to ConsleOutputOperator You can merge this pull request into a Git repository by running: $ git pull

Re: [ANNOUNCE] New Apache Apex Committer: Devendra Tagare

Congratulations Dev. Welcome aboard. --prad On Wed, Aug 10, 2016 at 1:28 PM, Bright Chen wrote: > Devendra, Congratulations > > Best, > Bright > > > On Aug 10, 2016, at 1:13 PM, Siyuan Hua wrote: > > > > Welcome, Devendra! > > > > On Wed, Aug

Re: [ANNOUNCE] New Apache Apex Committer: Devendra Tagare

Devendra, Congratulations Best, Bright > On Aug 10, 2016, at 1:13 PM, Siyuan Hua wrote: > > Welcome, Devendra! > > On Wed, Aug 10, 2016 at 12:28 PM, Thomas Weise wrote: > >> The Project Management Committee (PMC) for Apache Apex has asked Devendra >>

Re: [ANNOUNCE] New Apache Apex Committer: Devendra Tagare

Dev, Welcome aboard Thks Amol On Wed, Aug 10, 2016 at 1:13 PM, Siyuan Hua wrote: > Welcome, Devendra! > > On Wed, Aug 10, 2016 at 12:28 PM, Thomas Weise wrote: > > > The Project Management Committee (PMC) for Apache Apex has asked Devendra > > Tagare

[jira] [Commented] (APEXMALHAR-2028) Add System.err to ConsoleOutputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415943#comment-15415943 ] Siyuan Hua commented on APEXMALHAR-2028: For high-level API as well > Add System.err to

Re: [ANNOUNCE] New Apache Apex Committer: Devendra Tagare

Congratulations Devendra and welcome! Regards, Ashwin. On Wed, Aug 10, 2016 at 12:28 PM, Thomas Weise wrote: > The Project Management Committee (PMC) for Apache Apex has asked Devendra > Tagare to become a committer and we are pleased to announce that he has > accepted. > >

[ANNOUNCE] New Apache Apex Committer: Devendra Tagare

The Project Management Committee (PMC) for Apache Apex has asked Devendra Tagare to become a committer and we are pleased to announce that he has accepted. Devendra has been contributing to Apex for several months now, for example the Avro support and JDBC poll. He also did a few Apex meetup

Re: anti-affinity - parameter to control the containers of an operator on a node

Good idea. Thks Amol On Wed, Aug 10, 2016 at 11:27 AM, Venkatesh Kottapalli < venkat...@datatorrent.com> wrote: > Hi team, > > On the anti-affinity rules while deploying containers, do we have a > feature which can control the number of containers of the same operator > that get deployed on

[jira] [Comment Edited] (APEXMALHAR-2129) ManagedState: Add a disable purging option

[ https://issues.apache.org/jira/browse/APEXMALHAR-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415717#comment-15415717 ] Chandni Singh edited comment on APEXMALHAR-2129 at 8/10/16 6:47 PM:

anti-affinity - parameter to control the containers of an operator on a node

Hi team, On the anti-affinity rules while deploying containers, do we have a feature which can control the number of containers of the same operator that get deployed on the same node. If the environment is capable of handling, then this will be a good feature to have as it is possible that

[jira] [Commented] (APEXMALHAR-2129) ManagedState: Add a disable purging option

[ https://issues.apache.org/jira/browse/APEXMALHAR-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415717#comment-15415717 ] Chandni Singh commented on APEXMALHAR-2129: --- [~bhup...@apache.org] In TimeBucketAssigner,

[jira] [Commented] (APEXCORE-494) Window id of downstream operator is not moving after dynamic partition of upstream operator.

[ https://issues.apache.org/jira/browse/APEXCORE-494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415474#comment-15415474 ] Thomas Weise commented on APEXCORE-494: --- There is an existing unit test for a similar scenario

Re: [Proposal] DAG listener

Any operators can subscribe to Stram Events affecting the DAG. Implementation will most probably use heartbeat. On Tue, Aug 9, 2016 at 11:43 PM Tushar Gosavi wrote: > Hi Sandesh, > > Dag changes are handled in Stram, and operators are running in > different containers.

[jira] [Commented] (APEXCORE-496) Provide Operator name to StatsListener

[ https://issues.apache.org/jira/browse/APEXCORE-496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415147#comment-15415147 ] ASF GitHub Bot commented on APEXCORE-496: - Github user tushargosavi commented on a diff in the

[GitHub] apex-core pull request #361: APEXCORE-496 make operator name available to St...

Github user tushargosavi commented on a diff in the pull request: https://github.com/apache/apex-core/pull/361#discussion_r74229007 --- Diff: api/src/main/java/com/datatorrent/api/StatsListener.java --- @@ -115,6 +115,28 @@ List getOperatorResponse(); } +

Re: Apex website update proposal

Hi Kartavya, This is a good initiative. +1 Regards, Mohit On Wed, Aug 10, 2016 at 5:22 AM, Kartavya Jain wrote: > Hi Pramod, > > +1 for "Initiatives". And yes, I would appreciate some help to make these > changes on the website. > > Thanks, > Kartavya > > On Tue,

[GitHub] apex-malhar pull request #364: APEXMALHAR-2151 delimited file format support...

GitHub user shubham-pathak22 opened a pull request: https://github.com/apache/apex-malhar/pull/364 APEXMALHAR-2151 delimited file format support to FSLoader @chinmaykolhatkar Could you please review You can merge this pull request into a Git repository by running: $ git pull

Re: [Proposal] DAG listener

Hi Sandesh, Dag changes are handled in Stram, and operators are running in different containers. Are you suggesting an RPC interface or operator request for sending this information from Stram to all partitions of the interested operator? - Tushar. On Wed, Aug 10, 2016 at 11:28 AM, Sandesh

[jira] [Updated] (APEXMALHAR-2169) KafkaInputoperator: Remove the stuff related to Partition Based on throughput.

[ https://issues.apache.org/jira/browse/APEXMALHAR-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaitanya updated APEXMALHAR-2169: -- Summary: KafkaInputoperator: Remove the stuff related to Partition Based on throughput.

[jira] [Commented] (APEXMALHAR-2169) KafkaInputOperator: Dynamic partition is not working in case of ONE_TO_MANY partition strategy

[ https://issues.apache.org/jira/browse/APEXMALHAR-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15414802#comment-15414802 ] Chaitanya commented on APEXMALHAR-2169: --- Based on comments, Changing the title of this JIRA to