Re: Question about the custom partitioner

[Xingcan Cui]

Hi Aljoscha,

Thanks for your explanation. I'll try what you suggests.

Best,
Xingcan

On Fri, Jun 16, 2017 at 5:19 PM, Aljoscha Krettek 
wrote:

> Hi,
>
> I’m afraid that’s not possible out-of-box with the current APIs. I
> actually don’t know why the user-facing Partitioner only allows returning
> one target because the internal StreamPartitioner (which extends
> ChannelSelector) allows returning multiple target partitions.
>
> You can hack around the API by manually creating your own
> StreamPartitioner and applying it to the DataStream as
> DataStream.partitionCustom() and DataStream.setConnectionType() (the first
> calls the latter) do.
>
> Best,
> Aljoscha
>
> On 14. Jun 2017, at 09:09, Xingcan Cui  wrote:
>
> Hi all,
>
> I want to duplicate records to multiple downstream tasks (not all of them
> thus the
> Broadcasting should not work) in stream environment.
> However, it seems that the current custom partitioner can return only one
> partition index.
> Why this restriction exists or do I miss something?
>
> Thanks,
> Xingcan
>
>
>

Re: Question about the custom partitioner

[Aljoscha Krettek]

Hi,

I’m afraid that’s not possible out-of-box with the current APIs. I actually 
don’t know why the user-facing Partitioner only allows returning one target 
because the internal StreamPartitioner (which extends ChannelSelector) allows 
returning multiple target partitions.

You can hack around the API by manually creating your own StreamPartitioner and 
applying it to the DataStream as DataStream.partitionCustom() and 
DataStream.setConnectionType() (the first calls the latter) do.

Best,
Aljoscha

> On 14. Jun 2017, at 09:09, Xingcan Cui  wrote:
> 
> Hi all,
> 
> I want to duplicate records to multiple downstream tasks (not all of them 
> thus the
> Broadcasting should not work) in stream environment.
> However, it seems that the current custom partitioner can return only one 
> partition index.
> Why this restriction exists or do I miss something?
> 
> Thanks,
> Xingcan

Question about the custom partitioner

[Xingcan Cui]

Hi all,

I want to duplicate records to multiple downstream tasks (not all of them
thus the
Broadcasting should not work) in stream environment.
However, it seems that the current custom partitioner can return only one
partition index.
Why this restriction exists or do I miss something?

Thanks,
Xingcan