Re: Custom Partitioners Syntax

[Fabian Hueske]

I prefer the first option where partitioning (assigning keys to partitions)
follows key selection.


2014-11-24 19:52 GMT+01:00 Stephan Ewen <se...@apache.org>:

> Hi all!
>
> Custom partitioners allow you manually define the assignment of keys to
> partitions, for cases that have special constraints.
>
> This is a call for opinion on the Syntax for custom partitioners, in the
> case of Join and CoGroup.
>
> Option 1:
>
> input1
>     .join(input2)
>     .where("key1").equalTo("key2")
>
>     .withPartitioner( (key, num) -> key % num )
>
>     .with( (right, left) -> new Tuple3<>(right.key1, right.name,
> left.sizeOfEyebrow) )
>
> Option 2:
>
> input1
>     .join(input2)
>     .where("key1").equalTo("key2")
>     .with( (right, left) -> new Tuple3<>(right.key1, right.name,
> left.sizeOfEyebrow) )
>
>     .withPartitioner( (key, num) -> key % num )
>
>
> (The UDF and the partitioner calls are swapped here) (CoGroup would be
> analogous)
>
> Greetings,
> Stephan
>