The latter version would allow to use the apply method in Scala
without calling it directly, whereas in the first case the user would
have to spell it out.
On Mon, Nov 24, 2014 at 8:56 PM, Fabian Hueske <fhue...@apache.org> wrote:
> I prefer the first option where partitioning (assigning keys to partitions)
> follows key selection.
>
>
> 2014-11-24 19:52 GMT+01:00 Stephan Ewen <se...@apache.org>:
>
>> Hi all!
>>
>> Custom partitioners allow you manually define the assignment of keys to
>> partitions, for cases that have special constraints.
>>
>> This is a call for opinion on the Syntax for custom partitioners, in the
>> case of Join and CoGroup.
>>
>> Option 1:
>>
>> input1
>> .join(input2)
>> .where("key1").equalTo("key2")
>>
>> .withPartitioner( (key, num) -> key % num )
>>
>> .with( (right, left) -> new Tuple3<>(right.key1, right.name,
>> left.sizeOfEyebrow) )
>>
>> Option 2:
>>
>> input1
>> .join(input2)
>> .where("key1").equalTo("key2")
>> .with( (right, left) -> new Tuple3<>(right.key1, right.name,
>> left.sizeOfEyebrow) )
>>
>> .withPartitioner( (key, num) -> key % num )
>>
>>
>> (The UDF and the partitioner calls are swapped here) (CoGroup would be
>> analogous)
>>
>> Greetings,
>> Stephan
>>
