if you were to move the partition list fetching out of the constructor
int open(), is there any guarantee that for each fetching subtask the
partition list is identical?
On 05.10.2016 17:17, Tzu-Li (Gordon) Tai wrote:
Hi Timo,
I haven’t had the chance to look at the producer side too much yet, but after a
look in the code,
I think it’s reasonable to remove the instantiation from the producer
constructor.
The instantiation in the constructor is only used for partition list fetching &
eager properties validation
before running up the job. With an alternative to do the eager properties
validation in the constructor without relying on KafkaProducer,
it should be safe to remove it from the constructor.
The consumer side actually has the same problem right now too. I was hoping to
bundle the fix with a bigger task,
but would probably consider moving it up TODO list so it can be resolved sooner
as a standalone fix.
Cheers,
Gordon
On October 5, 2016 at 10:51:05 PM, Timo Walther (twal...@apache.org) wrote:
Hey everyone,
I'm currently rewriting the KafkaTabeSinkTest and discovered something
that doesn't seem to be intended: Is it intended that
FlinkKafkaProducer08 cannot be instantiated without a running Kafka
instance?
The constructor of FlinkKafkaProducerBase calls getKafkaProducer() which
actually should be called in the open() method first. What happens if
the Client has no access to the Kafka properties (e.g. using an remote
execution environment)? Then it is impossible to create a KafkaProducer?
Thanks.
Timo
