[GitHub] apex-malhar pull request #557: Changing the version of twitter4j

2017-02-22 Thread Shubham0411
Github user Shubham0411 closed the pull request at: https://github.com/apache/apex-malhar/pull/557 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Updated] (APEXCORE-652) Error in document for PI example

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXCORE-652: -- Labels: newbie (was: ) > Error in document for PI example >

[jira] [Moved] (APEXCORE-652) Error in document for PI example

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise moved APEXMALHAR-2423 to APEXCORE-652: --- Workflow: jira (was: Default workflow, editable Closed

[jira] [Deleted] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise deleted APEXMALHAR-2420: - > Error in document for PI example > - > >

[jira] [Moved] (APEXMALHAR-2423) Error in document for PI example

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise moved APEXCORE-651 to APEXMALHAR-2423: --- Component/s: (was: Documentation) Workflow:

[jira] [Updated] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh updated APEXMALHAR-2422: Description: In WindowDataManager getLargestWindow during recovery is not reflecting the

[jira] [Created] (APEXMALHAR-2422) WindowDataManager not recovering as expected on HDFS

2017-02-22 Thread Sandesh (JIRA)
Sandesh created APEXMALHAR-2422: --- Summary: WindowDataManager not recovering as expected on HDFS Key: APEXMALHAR-2422 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2422 Project: Apache Apex

[jira] [Closed] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongming Liang closed APEXMALHAR-2420. -- Resolution: Duplicate > Error in document for PI example >

[jira] [Commented] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879556#comment-15879556 ] Dongming Liang commented on APEXMALHAR-2420: Moved to

[jira] [Created] (APEXCORE-651) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
Dongming Liang created APEXCORE-651: --- Summary: Error in document for PI example Key: APEXCORE-651 URL: https://issues.apache.org/jira/browse/APEXCORE-651 Project: Apache Apex Core Issue

[jira] [Closed] (APEXMALHAR-2421) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongming Liang closed APEXMALHAR-2421. -- Resolution: Duplicate > Error in document for PI example >

[jira] [Updated] (APEXCORE-650) Remove ResetWindowTuple

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh updated APEXCORE-650: - Description: ResetWindowTuple raises the complexity of the engine and development, without much

[jira] [Created] (APEXMALHAR-2421) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
Dongming Liang created APEXMALHAR-2421: -- Summary: Error in document for PI example Key: APEXMALHAR-2421 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2421 Project: Apache Apex Malhar

[jira] [Created] (APEXCORE-650) Remove ResetWindowTuple

2017-02-22 Thread Sandesh (JIRA)
Sandesh created APEXCORE-650: Summary: Remove ResetWindowTuple Key: APEXCORE-650 URL: https://issues.apache.org/jira/browse/APEXCORE-650 Project: Apache Apex Core Issue Type: Improvement

[jira] [Commented] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879541#comment-15879541 ] Dongming Liang commented on APEXMALHAR-2420: This should be filed under apex-core. Will

[jira] [Created] (APEXMALHAR-2420) Error in document for PI example

2017-02-22 Thread Dongming Liang (JIRA)
Dongming Liang created APEXMALHAR-2420: -- Summary: Error in document for PI example Key: APEXMALHAR-2420 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2420 Project: Apache Apex Malhar

[jira] [Assigned] (APEXMALHAR-2373) JdbcOutputOperator does not handle NUMERIC type

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongming Liang reassigned APEXMALHAR-2373: -- Assignee: Dongming Liang > JdbcOutputOperator does not handle NUMERIC

[jira] [Issue Comment Deleted] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandesh updated APEXMALHAR-2419: Comment: was deleted (was: Fix is here, until it is merged.

[jira] [Commented] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879182#comment-15879182 ] Thomas Weise commented on APEXMALHAR-2419: -- Completed windows (endWindow) are supposed to be

[jira] [Comment Edited] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878951#comment-15878951 ] Sandesh edited comment on APEXMALHAR-2419 at 2/22/17 6:46 PM: -- Largest

[jira] [Commented] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Sandesh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878951#comment-15878951 ] Sandesh commented on APEXMALHAR-2419: - Largest completed window is moving only after a committed

[jira] [Commented] (APEXMALHAR-2414) Improve performance of PojoInnerJoin accum by using PojoUtils

2017-02-22 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878926#comment-15878926 ] ASF GitHub Bot commented on APEXMALHAR-2414: GitHub user Hitesh-Scorpio opened a pull

[jira] [Commented] (APEXMALHAR-2239) Null pointer exception in JDBCPojoInputOperator is thrown when we set a query to fetch rows and have additional columns in fieldInfos

2017-02-22 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878743#comment-15878743 ] Dongming Liang commented on APEXMALHAR-2239: [~Hitesh_]I'll work on the comments today.

Re: [DISCUSS] Proposal for adapting Malhar operators for batch use cases

2017-02-22 Thread Thomas Weise
Hi Bhupesh, This looks great. You use the watermark as measure of completeness and the window to isolate the state, which is how it should work. Questions/comments: Why does the count operator have a 2ms window when this should be driven by the watermark from the input operator? I don't think

[GitHub] apex-core pull request #477: APEXCORE-648 Unnecessary byte array copy in Def...

2017-02-22 Thread vrozov
GitHub user vrozov opened a pull request: https://github.com/apache/apex-core/pull/477 APEXCORE-648 Unnecessary byte array copy in DefaultStatefulStreamCodec.toDataStatePair() @sandeshh Please review and merge You can merge this pull request into a Git repository by running:

[jira] [Commented] (APEXCORE-648) Unnecessary byte array copy in DefaultStatefulStreamCodec.toDataStatePair()

2017-02-22 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878618#comment-15878618 ] ASF GitHub Bot commented on APEXCORE-648: - GitHub user vrozov opened a pull request:

[jira] [Commented] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Munagala V. Ramanath (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878506#comment-15878506 ] Munagala V. Ramanath commented on APEXMALHAR-2419: -- To clarify, the point of the

[jira] [Commented] (APEXMALHAR-2419) KafkaSinglePortExactlyOnceOutputOperator fails on recovery

2017-02-22 Thread Munagala V. Ramanath (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878463#comment-15878463 ] Munagala V. Ramanath commented on APEXMALHAR-2419: -- [~sandesh] Does "recovery is not

Re: [Community Opinion] Use of Lombok Library

2017-02-22 Thread Bhupesh Chawda
I think this could be a good way to improve readability of code in Malhar operators as well. This one however is MIT license. Any alternatives? ~ Bhupesh ___ Bhupesh Chawda Software Engineer E: bhup...@datatorrent.com | Twitter: @bhupeshsc

Re: [DISCUSS] Proposal for adapting Malhar operators for batch use cases

2017-02-22 Thread Bhupesh Chawda
Yes Amol, Watermarks, in general solve a different issue and we should not mix it with data association. If needed, the data association can be solved by the user on the operator/ application layer. The engine should not worry about it. Regarding your point on event time based watermarks, I think

Re: [DISCUSS] Proposal for adapting Malhar operators for batch use cases

2017-02-22 Thread Bhupesh Chawda
Hi Thomas, Sorry for the delay. I agree that the watermark concept is general and is understood by intermediate transformations. File name is some additional information in the watermark which helps the start and end operators do stuff related to batch. As suggested, I have created a wordcount

[jira] [Resolved] (APEXMALHAR-2380) Add MutablePair for Kinensis Operator for Recovery State

2017-02-22 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chaitanya resolved APEXMALHAR-2380. --- Resolution: Fixed Fix Version/s: 3.7.0 > Add MutablePair for Kinensis Operator

[jira] [Resolved] (APEXCORE-634) Apex Platform unable to set unifier attributes for modules in DAG

2017-02-22 Thread Tushar Gosavi (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tushar Gosavi resolved APEXCORE-634. Resolution: Fixed Fix Version/s: 3.6.0 > Apex Platform unable to set unifier

[jira] [Commented] (APEXCORE-634) Apex Platform unable to set unifier attributes for modules in DAG

2017-02-22 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878031#comment-15878031 ] ASF GitHub Bot commented on APEXCORE-634: - Github user asfgit closed the pull request at:

[GitHub] apex-core pull request #466: APEXCORE-634 Apex Platform unable to set unifie...

2017-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-core/pull/466 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

Re: [Community Opinion] Use of Lombok Library

2017-02-22 Thread Chinmay Kolhatkar
Certainly good and convenient for application/operator developers to use. But could someone include this dependency in apex application project and use it? If yes, then I would prefer to keep this outside of Apex in general and let the user use it for convenience. -Chinmay. On Wed, Feb 22,

[Community Opinion] Use of Lombok Library

2017-02-22 Thread AJAY GUPTA
The Java operators we write have a lot of code which is less related to the business logic of the operator, eg. getters, setters, constructors, log variables, etc. Lombok library can be used to greatly avoid such code by using annotations, eg. @Getter, @Setter. The library generates this code

[jira] [Commented] (APEXMALHAR-2239) Null pointer exception in JDBCPojoInputOperator is thrown when we set a query to fetch rows and have additional columns in fieldInfos

2017-02-22 Thread Hitesh Kapoor (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1585#comment-1585 ] Hitesh Kapoor commented on APEXMALHAR-2239: --- [~d9liang] Have you had a chance to go over