[
https://issues.apache.org/jira/browse/APEXMALHAR-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313126#comment-15313126
]
ASF GitHub Bot commented on APEXMALHAR-2106:
--------------------------------------------
GitHub user ilganeli opened a pull request:
https://github.com/apache/incubator-apex-malhar/pull/308
[APEXMALHAR-2106][WIP] Support multiple streams in StreamMerger
* Created a module which creates a binary tree of StreamMerger operators to
support merging multiple streams
* StreamMerger operators are presently allocated as CONTAINER_LOCAL. I
would like for them to be THREAD_LOCAL but I was getting the following error
when attempting this:
```Caused by: javax.validation.ValidationException: Locality THREAD_LOCAL
invalid for operator OperatorMeta{name=Merger_Tier_0_#_0,
operator=StreamMerger{name=null}, attributes={}} with multiple input streams as
they origin from different owner OIO operators
```
* An example application is included but i'm still working on further
tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ilganeli/incubator-apex-malhar
APEXMALHAR-2106B
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-apex-malhar/pull/308.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #308
----
commit f991ecb3cd68dc56a1e544503b5bc7eedfe2d015
Author: Ilya Ganelin <[email protected]>
Date: 2016-05-31T23:56:29Z
Updated KeyValPair to override hashcode to support sticky partitining by
key.
commit 7dc1b463ed13502ea6fbb5d4d2588c3a28dd2f09
Author: Ilya Ganelin <[email protected]>
Date: 2016-06-02T21:29:47Z
Created a module to merge multiple streams into a single stream. Created a
test application demonstrating this in action. Still need to add more tests.
commit 429832f91b95169daf9986aa6bc5c965721d9626
Author: Priyanka Gugale <[email protected]>
Date: 2016-05-19T00:52:08Z
APEXMALHAR-2096: Add property blocksThreshold to limit input rate
commit 1352e3f92aef2f6d314230dea81408a409d09466
Author: Tushar R. Gosavi <[email protected]>
Date: 2016-06-01T09:32:27Z
APEXMALHAR-998 extend second type parameter from Object
commit a8122d7ceee14d4c732657250848a1009dc8c14d
Author: Ilya Ganelin <[email protected]>
Date: 2016-06-02T21:36:02Z
Merge branch 'master' into APEXMALHAR-2106B
commit eda0eb4fe47f16cd1c56392f3db879f5e9b1d2bd
Author: Ilya Ganelin <[email protected]>
Date: 2016-06-02T21:37:50Z
Removed extra space.
----
> Support merging multiple streams with StreamMerger
> ---------------------------------------------------
>
> Key: APEXMALHAR-2106
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2106
> Project: Apache Apex Malhar
> Issue Type: New Feature
> Reporter: Ilya Ganelin
> Assignee: Ilya Ganelin
>
> To properly implement the Flatten transformation (and other Stream
> combination operations), Apex must support merging data from multiple
> sources. The StreamMerger operator can be improved to merge multiple streams,
> rather than just the two streams it can handle in the present implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
