[
https://issues.apache.org/jira/browse/APEXCORE-332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303285#comment-15303285
]
ASF GitHub Bot commented on APEXCORE-332:
-----------------------------------------
GitHub user ilganeli opened a pull request:
https://github.com/apache/incubator-apex-core/pull/347
[APEXCORE-332][WIP] Improve partitioning to support partitioning other than
powers of 2
* Updated the Partitioner to store partition definitions as Sets of Ranges
which define which values may be accepted by what ports
* This substantially simplifies both the creation of new partitioning
schemes (e.g. evenly distributing values) and the logic around assigning values
to partitions (no confusing mask, simply a Range.contains() check)
This is still a work in progress but given that it's touching a fairly
intimate piece of the core code it would be great to get some other eyes on
this.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ilganeli/incubator-apex-core APEXCORE-332
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-apex-core/pull/347.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #347
----
commit d95e83ea8d7317cf7faf5e32bd1244787e143084
Author: Ilya Ganelin <ilya.gane...@capitalone.com>
Date: 2016-05-27T00:55:41Z
Updated PartitionKeys to eliminate the use of mask. This class now defines
a Set of Range values which break up the range of possible values for a
partitioning scheme. Some of the updates to the reset of the dependent code
have been made but this is still a work in progress.
----
> Support Even Distribution Of Tuples To A Non Power Of 2 Number Of Partitions
> ----------------------------------------------------------------------------
>
> Key: APEXCORE-332
> URL: https://issues.apache.org/jira/browse/APEXCORE-332
> Project: Apache Apex Core
> Issue Type: Improvement
> Reporter: Timothy Farkas
> Assignee: Ilya Ganelin
> Priority: Minor
>
> Currently partitions masks must be defined as a binary mask. As a result the
> number of partitions must be a power of 2, otherwise the distribution of
> tuples will be uneven. If we support the modulus operation instead of a
> binary mask, we could support an even distribution of tuples to a non-power
> of 2 number of partitions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
