Re: Malhar release 3.6

+1. On Thu, Oct 27, 2016 at 1:41 AM, Thomas Weise wrote: > Hi, > > I'm proposing another release of Malhar in November. There are 49 issues > marked for the release, including important bug fixes, new documentation, > SQL support and the work for windowed operator state

Suggestion to Compositional API of Apex

Hi All, With a lot of Apache projects making use of inner class in Java 7 and Decarative programming feature of Java 8 to define stages in the pipeline, I would like to propose a small additional to our existing compositional APIs allowing it to define processing stages inline. I've created a

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610758#comment-15610758 ] Bhupesh Chawda commented on APEXMALHAR-2284: [~chaithu] I think we should go ahead with

Re: Suggestion to Compositional API of Apex

I think this is the right idea to eliminate boilerplate code, but it also overlaps with the stream API. There was an earlier discussion/JIRA for making respective underlying operators from that API available for use in other applications as well. I think we should pursue that path and enhance

[jira] [Commented] (APEXMALHAR-2316) Cannot register tuple class in XmlParser Operator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15608767#comment-15608767 ] Munagala V. Ramanath commented on APEXMALHAR-2316: -- Please see APEXMALHAR-2083 for

[jira] [Updated] (APEXMALHAR-2244) Optimize WindowedStorage and Spillable data structures for time series

[ https://issues.apache.org/jira/browse/APEXMALHAR-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Weise updated APEXMALHAR-2244: - Fix Version/s: 3.6.0 > Optimize WindowedStorage and Spillable data structures for

[jira] [Created] (APEXCORE-565) Ability to activate and de-activate operators in the DAG

Vlad Rozov created APEXCORE-565: --- Summary: Ability to activate and de-activate operators in the DAG Key: APEXCORE-565 URL: https://issues.apache.org/jira/browse/APEXCORE-565 Project: Apache Apex Core

[jira] [Resolved] (APEXMALHAR-2317) Change SpillableBenchmarkApp to adapt the change on Spillable Data Structure

[ https://issues.apache.org/jira/browse/APEXMALHAR-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Yan resolved APEXMALHAR-2317. --- Resolution: Fixed Fix Version/s: 3.6.0 > Change SpillableBenchmarkApp to adapt

[jira] [Commented] (APEXCORE-560) Logical plan is not changed when all physical partitions of operator are removed from DAG

[ https://issues.apache.org/jira/browse/APEXCORE-560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610156#comment-15610156 ] Thomas Weise commented on APEXCORE-560: --- It depends on the use case. As suggested earlier, there

[jira] [Commented] (APEXMALHAR-2317) Change SpillableBenchmarkApp to adapt the change on Spillable Data Structure

[ https://issues.apache.org/jira/browse/APEXMALHAR-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609973#comment-15609973 ] ASF GitHub Bot commented on APEXMALHAR-2317: GitHub user brightchen reopened a pull

[jira] [Commented] (APEXMALHAR-2317) Change SpillableBenchmarkApp to adapt the change on Spillable Data Structure

[ https://issues.apache.org/jira/browse/APEXMALHAR-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15609972#comment-15609972 ] ASF GitHub Bot commented on APEXMALHAR-2317: Github user brightchen closed the pull

[jira] [Commented] (APEXMALHAR-2317) Change SpillableBenchmarkApp to adapt the change on Spillable Data Structure

[ https://issues.apache.org/jira/browse/APEXMALHAR-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610086#comment-15610086 ] ASF GitHub Bot commented on APEXMALHAR-2317: Github user asfgit closed the pull request

[jira] [Commented] (APEXCORE-565) Ability to activate and de-activate operators in the DAG

[ https://issues.apache.org/jira/browse/APEXCORE-565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15610081#comment-15610081 ] Sanjay M Pujare commented on APEXCORE-565: -- When will inactive operators be activated? Are

[GitHub] apex-malhar pull request #470: APEXMALHAR-2317 #resolve #comment Change Spil...

Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/470 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Updated] (APEXMALHAR-2314) Improper functioning in partitioning of sequentialFileRead property of FSRecordReader

[ https://issues.apache.org/jira/browse/APEXMALHAR-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Narkhede updated APEXMALHAR-2314: Description: Fix the StreamCodec for FSRecordReader, initially it was hashcode

[jira] [Commented] (APEXMALHAR-2274) AbstractFileInputOperator gets killed when there are a large number of files.

[ https://issues.apache.org/jira/browse/APEXMALHAR-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607573#comment-15607573 ] Tushar Gosavi commented on APEXMALHAR-2274: --- you can take a look at FileSplitterInput which

[jira] [Commented] (APEXCORE-560) Logical plan is not changed when all physical partitions of operator are removed from DAG

[ https://issues.apache.org/jira/browse/APEXCORE-560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607577#comment-15607577 ] Tushar Gosavi commented on APEXCORE-560: I think activate/deactivate will be of same complexity

[jira] [Commented] (APEXMALHAR-2315) Ignore Join Test for because of issues in POJOInnerJoinOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607564#comment-15607564 ] ASF GitHub Bot commented on APEXMALHAR-2315: Github user asfgit closed the pull request

[GitHub] apex-malhar pull request #469: APEXMALHAR-2315 Ignore tests in sql which use...

Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/469 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Created] (APEXMALHAR-2317) Change SpillableBenchmarkApp to adapt the change on Spillable Data Structure

bright chen created APEXMALHAR-2317: --- Summary: Change SpillableBenchmarkApp to adapt the change on Spillable Data Structure Key: APEXMALHAR-2317 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2317

[jira] [Assigned] (APEXMALHAR-2317) Change SpillableBenchmarkApp to adapt the change on Spillable Data Structure

[ https://issues.apache.org/jira/browse/APEXMALHAR-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bright chen reassigned APEXMALHAR-2317: --- Assignee: bright chen > Change SpillableBenchmarkApp to adapt the change on

[GitHub] apex-malhar pull request #470: APEXMALHAR-2317 #resolve #comment Change Spil...

GitHub user brightchen opened a pull request: https://github.com/apache/apex-malhar/pull/470 APEXMALHAR-2317 #resolve #comment Change SpillableBenchmarkApp to ada… …pt the change on Spillable Data Structure You can merge this pull request into a Git repository by running:

[GitHub] apex-malhar pull request #452: APEXMALHAR-2290 Optimization to fetch meta da...

Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/452 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Resolved] (APEXMALHAR-2309) TimeBasedDedupOperator marks new tuples as duplicates if expired tuples exist

[ https://issues.apache.org/jira/browse/APEXMALHAR-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhupesh Chawda resolved APEXMALHAR-2309. Resolution: Fixed Fix Version/s: 3.6.0 > TimeBasedDedupOperator marks

[jira] [Commented] (APEXMALHAR-2309) TimeBasedDedupOperator marks new tuples as duplicates if expired tuples exist

[ https://issues.apache.org/jira/browse/APEXMALHAR-2309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607771#comment-15607771 ] ASF GitHub Bot commented on APEXMALHAR-2309: Github user asfgit closed the pull request

[jira] [Commented] (APEXMALHAR-2314) Improper functioning in partitioning of sequentialFileRead property of FSRecordReader

[ https://issues.apache.org/jira/browse/APEXMALHAR-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607745#comment-15607745 ] ASF GitHub Bot commented on APEXMALHAR-2314: Github user deepak-narkhede closed the pull

[GitHub] apex-malhar pull request #468: APEXMALHAR-2314 Improper functioning in parti...

GitHub user deepak-narkhede reopened a pull request: https://github.com/apache/apex-malhar/pull/468 APEXMALHAR-2314 Improper functioning in partitioning for sequentialFileRead for FSRecord Fix the StreamCodec for FSRecordReader, initially it was hashcode of blockId's mostly always

[jira] [Updated] (APEXMALHAR-2290) JDBCPOJOInsertOutput Operator - Optimization required to populate metadata for columns from database

[ https://issues.apache.org/jira/browse/APEXMALHAR-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar updated APEXMALHAR-2290: -- Summary: JDBCPOJOInsertOutput Operator - Optimization required to populate

[GitHub] apex-malhar pull request #457: APEXMALHAR-2302 Exposing few properties of FS...

GitHub user deepak-narkhede reopened a pull request: https://github.com/apache/apex-malhar/pull/457 APEXMALHAR-2302 Exposing few properties of FSSplitter and BlockReader operators to FSRecordReaderModule This change includes: 1) Expose blockSize property of FileSplitter

[jira] [Commented] (APEXMALHAR-2314) Improper functioning in partitioning of sequentialFileRead property of FSRecordReader

[ https://issues.apache.org/jira/browse/APEXMALHAR-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607854#comment-15607854 ] ASF GitHub Bot commented on APEXMALHAR-2314: Github user deepak-narkhede closed the pull

[jira] [Commented] (APEXMALHAR-2314) Improper functioning in partitioning of sequentialFileRead property of FSRecordReader

[ https://issues.apache.org/jira/browse/APEXMALHAR-2314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607856#comment-15607856 ] ASF GitHub Bot commented on APEXMALHAR-2314: GitHub user deepak-narkhede reopened a pull

[jira] [Resolved] (APEXMALHAR-2315) Ignore Join Test for because of issues in POJOInnerJoinOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhupesh Chawda resolved APEXMALHAR-2315. Resolution: Fixed Fix Version/s: 3.6.0 > Ignore Join Test for because

[jira] [Commented] (APEXMALHAR-2302) Exposing few properties of FSSplitter and BlockReader operators to FSRecordReaderModule to tune Application

[ https://issues.apache.org/jira/browse/APEXMALHAR-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607756#comment-15607756 ] ASF GitHub Bot commented on APEXMALHAR-2302: GitHub user deepak-narkhede reopened a pull

[jira] [Commented] (APEXMALHAR-2302) Exposing few properties of FSSplitter and BlockReader operators to FSRecordReaderModule to tune Application

[ https://issues.apache.org/jira/browse/APEXMALHAR-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15607755#comment-15607755 ] ASF GitHub Bot commented on APEXMALHAR-2302: Github user deepak-narkhede closed the pull

[GitHub] apex-malhar pull request #468: APEXMALHAR-2314 Improper functioning in parti...

GitHub user deepak-narkhede reopened a pull request: https://github.com/apache/apex-malhar/pull/468 APEXMALHAR-2314 Improper functioning in partitioning for sequentialFileRead for FSRecord Fix the StreamCodec for FSRecordReader, initially it was hashcode of blockId's mostly always

[GitHub] apex-malhar pull request #468: APEXMALHAR-2314 Improper functioning in parti...

Github user deepak-narkhede closed the pull request at: https://github.com/apache/apex-malhar/pull/468 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Resolved] (APEXMALHAR-2290) JDBCPOJOInsertOutput Operator - Optimization required to populate metadata for columns from database

[ https://issues.apache.org/jira/browse/APEXMALHAR-2290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar resolved APEXMALHAR-2290. --- Resolution: Fixed > JDBCPOJOInsertOutput Operator - Optimization

[jira] [Created] (APEXMALHAR-2316) Cannot register tuple class in XmlParser Operator

Hitesh Kapoor created APEXMALHAR-2316: - Summary: Cannot register tuple class in XmlParser Operator Key: APEXMALHAR-2316 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2316 Project: Apache

[GitHub] apex-malhar pull request #455: APEXMALHAR-2291 Fix for Exactly-once processi...

Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/455 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Commented] (APEXMALHAR-2291) Exactly-once processing not working correctly for JdbcPOJOInsertOutputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15608440#comment-15608440 ] ASF GitHub Bot commented on APEXMALHAR-2291: Github user asfgit closed the pull request

[jira] [Resolved] (APEXMALHAR-2291) Exactly-once processing not working correctly for JdbcPOJOInsertOutputOperator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bhupesh Chawda resolved APEXMALHAR-2291. Resolution: Fixed Fix Version/s: 3.6.0 > Exactly-once processing not

(APEXMALHAR-2316) Cannot register tuple class in XmlParser Operator

Hi All, The following stack trace is observed when we try to use XmlParser Abandoning deployment due to setup failure. java.lang.IllegalArgumentException at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:635) at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:584) at

[jira] [Updated] (APEXMALHAR-2316) Cannot register tuple class in XmlParser Operator

[ https://issues.apache.org/jira/browse/APEXMALHAR-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Kapoor updated APEXMALHAR-2316: -- Description: Unable to set the tuple class attribute, getting the following stack

Re: [KUDOS] Contributed runner: Apache Apex!

FYI, here is a nice presentation (Beam tutorial): https://github.com/eljefe6a/beamexample/blob/master/BeamTutorial/slides.pdf On Tue, Oct 18, 2016 at 10:45 PM, Chinmay Kolhatkar wrote: > Very nice ..:) > I would like to contribute as well. > > Thanks, > Chinmay. > > On