Re: Flink kafka producer partitioning scheme

2019-09-14 Thread Eduardo Winpenny Tejedor
Hi Vishwas, Just because a Kafka topic has 40 partitions doesn't mean the 40 partitions need to have messages in them. However, while it is not a requirement it is certainly convenient to populate the 40 partitions for load balancing. What this means if you want to use all partitions is you

Flink kafka producer partitioning scheme

2019-09-13 Thread Vishwas Siravara
Hi guys, >From the flink doc *By default, if a custom partitioner is not specified for the Flink Kafka Producer, the producer will use a FlinkFixedPartitioner that maps each Flink Kafka Producer parallel subtask to a single Kafka partition (i.e., all records received by a sink subtask will end up