[GitHub] apex-malhar pull request #557: Changing the version of twitter4j

2017-02-22 Thread Shubham0411
Github user Shubham0411 closed the pull request at: https://github.com/apache/apex-malhar/pull/557 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

[jira] [Updated] (APEXCORE-652) Error in document for PI example

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXCORE-652: -- Labels: newbie (was: ) > Error in document for PI example >

[jira] [Moved] (APEXCORE-652) Error in document for PI example

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise moved APEXMALHAR-2423 to APEXCORE-652: --- Workflow: jira (was: Default workflow, editable Closed stat

[jira] [Deleted] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise deleted APEXMALHAR-2420: - > Error in document for PI example > - > >

[jira] [Moved] (APEXMALHAR-2423) Error in document for PI example

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise moved APEXCORE-651 to APEXMALHAR-2423: --- Component/s: (was: Documentation) Workflow: De

[jira] [Updated] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh updated APEXMALHAR-2422: Description: In WindowDataManager getLargestWindow during recovery is not reflecting the saved

[jira] [Created] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-02-22 Thread Sandesh (JIRA)
Sandesh created APEXMALHAR-2422: --- Summary: WindowDataManager not recovering as expected on HDFS Key: APEXMALHAR-2422 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2422 Project: Apache Apex Malhar

[jira] [Closed] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongming Liang closed APEXMALHAR-2420. -- Resolution: Duplicate > Error in document for PI example > ---

[jira] [Commented] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879556#comment-15879556 ] Dongming Liang commented on APEXMALHAR-2420: Moved to https://issues.apac

[jira] [Created] (APEXCORE-651) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
Dongming Liang created APEXCORE-651: --- Summary: Error in document for PI example Key: APEXCORE-651 URL: https://issues.apache.org/jira/browse/APEXCORE-651 Project: Apache Apex Core Issue Ty

[jira] [Closed] (APEXMALHAR-2421) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongming Liang closed APEXMALHAR-2421. -- Resolution: Duplicate > Error in document for PI example > ---

[jira] [Updated] (APEXCORE-650) Remove ResetWindowTuple

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh updated APEXCORE-650: - Description: ResetWindowTuple raises the complexity of the engine and development, without much benefit.

[jira] [Created] (APEXMALHAR-2421) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
Dongming Liang created APEXMALHAR-2421: -- Summary: Error in document for PI example Key: APEXMALHAR-2421 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2421 Project: Apache Apex Malhar

[jira] [Created] (APEXCORE-650) Remove ResetWindowTuple

2017-02-22 Thread Sandesh (JIRA)
Sandesh created APEXCORE-650: Summary: Remove ResetWindowTuple Key: APEXCORE-650 URL: https://issues.apache.org/jira/browse/APEXCORE-650 Project: Apache Apex Core Issue Type: Improvement

[jira] [Commented] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879541#comment-15879541 ] Dongming Liang commented on APEXMALHAR-2420: This should be filed under a

[jira] [Created] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
Dongming Liang created APEXMALHAR-2420: -- Summary: Error in document for PI example Key: APEXMALHAR-2420 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2420 Project: Apache Apex Malhar

[jira] [Assigned] (APEXMALHAR-2373) JdbcOutputOperator does not handle NUMERIC type

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongming Liang reassigned APEXMALHAR-2373: -- Assignee: Dongming Liang > JdbcOutputOperator does not handle NUMERIC type

[jira] [Issue Comment Deleted] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh updated APEXMALHAR-2419: Comment: was deleted (was: Fix is here, until it is merged. https://github.com/sandeshh/apex-ma

[jira] [Commented] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879182#comment-15879182 ] Thomas Weise commented on APEXMALHAR-2419: -- Completed windows (endWindow) ar

[jira] [Comment Edited] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878951#comment-15878951 ] Sandesh edited comment on APEXMALHAR-2419 at 2/22/17 6:46 PM: -

[jira] [Commented] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878951#comment-15878951 ] Sandesh commented on APEXMALHAR-2419: - Largest completed window is moving only af

[jira] [Commented] (APEXMALHAR-2414) Improve performance of PojoInnerJoin accum by using PojoUtils

2017-02-22 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878926#comment-15878926 ] ASF GitHub Bot commented on APEXMALHAR-2414: GitHub user Hitesh-Scorpio o

[GitHub] apex-malhar pull request #559: APEXMALHAR-2414 PojoInner Join accumulation w...

2017-02-22 Thread Hitesh-Scorpio
GitHub user Hitesh-Scorpio opened a pull request: https://github.com/apache/apex-malhar/pull/559 APEXMALHAR-2414 PojoInner Join accumulation was using java reflection @chinmaykolhatkar please review You can merge this pull request into a Git repository by running: $ git pull ht

[jira] [Assigned] (APEXMALHAR-2414) Improve performance of PojoInnerJoin accum by using PojoUtils

2017-02-22 Thread Hitesh Kapoor (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Kapoor reassigned APEXMALHAR-2414: - Assignee: Hitesh Kapoor > Improve performance of PojoInnerJoin accum by usin

[jira] [Commented] (APEXMALHAR-2239) Null pointer exception in JDBCPojoInputOperator is thrown when we set a query to fetch rows and have additional columns in fieldInfos

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878743#comment-15878743 ] Dongming Liang commented on APEXMALHAR-2239: [~Hitesh_]I'll work on the c

Re: [DISCUSS] Proposal for adapting Malhar operators for batch use cases

2017-02-22 Thread Thomas Weise
Hi Bhupesh, This looks great. You use the watermark as measure of completeness and the window to isolate the state, which is how it should work. Questions/comments: Why does the count operator have a 2ms window when this should be driven by the watermark from the input operator? I don't think t

[GitHub] apex-core pull request #477: APEXCORE-648 Unnecessary byte array copy in Def...

2017-02-22 Thread vrozov
GitHub user vrozov opened a pull request: https://github.com/apache/apex-core/pull/477 APEXCORE-648 Unnecessary byte array copy in DefaultStatefulStreamCodec.toDataStatePair() @sandeshh Please review and merge You can merge this pull request into a Git repository by running: $

[jira] [Commented] (APEXCORE-648) Unnecessary byte array copy in DefaultStatefulStreamCodec.toDataStatePair()

2017-02-22 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878618#comment-15878618 ] ASF GitHub Bot commented on APEXCORE-648: - GitHub user vrozov opened a pull requ

[jira] [Commented] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Munagala V. Ramanath (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878506#comment-15878506 ] Munagala V. Ramanath commented on APEXMALHAR-2419: -- To clarify, the

[jira] [Commented] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Munagala V. Ramanath (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878463#comment-15878463 ] Munagala V. Ramanath commented on APEXMALHAR-2419: -- [~sandesh] Does

Re: [Community Opinion] Use of Lombok Library

2017-02-22 Thread Bhupesh Chawda
I think this could be a good way to improve readability of code in Malhar operators as well. This one however is MIT license. Any alternatives? ~ Bhupesh ___ Bhupesh Chawda Software Engineer E: bhup...@datatorrent.com | Twitter: @bhupeshsc

Re: [DISCUSS] Proposal for adapting Malhar operators for batch use cases

2017-02-22 Thread Bhupesh Chawda
Yes Amol, Watermarks, in general solve a different issue and we should not mix it with data association. If needed, the data association can be solved by the user on the operator/ application layer. The engine should not worry about it. Regarding your point on event time based watermarks, I think

Re: [DISCUSS] Proposal for adapting Malhar operators for batch use cases

2017-02-22 Thread Bhupesh Chawda
Hi Thomas, Sorry for the delay. I agree that the watermark concept is general and is understood by intermediate transformations. File name is some additional information in the watermark which helps the start and end operators do stuff related to batch. As suggested, I have created a wordcount app

[jira] [Resolved] (APEXMALHAR-2380) Add MutablePair for Kinensis Operator for Recovery State

2017-02-22 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaitanya resolved APEXMALHAR-2380. --- Resolution: Fixed Fix Version/s: 3.7.0 > Add MutablePair for Kinensis Operator for

[jira] [Resolved] (APEXCORE-634) Apex Platform unable to set unifier attributes for modules in DAG

2017-02-22 Thread Tushar Gosavi (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tushar Gosavi resolved APEXCORE-634. Resolution: Fixed Fix Version/s: 3.6.0 > Apex Platform unable to set unifier attrib

[jira] [Commented] (APEXCORE-634) Apex Platform unable to set unifier attributes for modules in DAG

2017-02-22 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878031#comment-15878031 ] ASF GitHub Bot commented on APEXCORE-634: - Github user asfgit closed the pull re

[GitHub] apex-core pull request #466: APEXCORE-634 Apex Platform unable to set unifie...

2017-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-core/pull/466 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

Re: [Community Opinion] Use of Lombok Library

2017-02-22 Thread Chinmay Kolhatkar
Certainly good and convenient for application/operator developers to use. But could someone include this dependency in apex application project and use it? If yes, then I would prefer to keep this outside of Apex in general and let the user use it for convenience. -Chinmay. On Wed, Feb 22, 2017

[Community Opinion] Use of Lombok Library

2017-02-22 Thread AJAY GUPTA
The Java operators we write have a lot of code which is less related to the business logic of the operator, eg. getters, setters, constructors, log variables, etc. Lombok library can be used to greatly avoid such code by using annotations, eg. @Getter, @Setter. The library generates this code durin

[jira] [Commented] (APEXMALHAR-2239) Null pointer exception in JDBCPojoInputOperator is thrown when we set a query to fetch rows and have additional columns in fieldInfos

2017-02-22 Thread Hitesh Kapoor (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1585#comment-1585 ] Hitesh Kapoor commented on APEXMALHAR-2239: --- [~d9liang] Have you had a chan