Hi Marco,
I'm no expert on the Kafka producer, but I will try to help. [1] seems to
have a decent explanation of possible error causes for the error you
encountered.
Which leads me to two questions:
if (druidProducerTransactionMaxTimeoutMs > 0) {
>
> properties.setProperty("transaction.max.timeout.ms",
> Integer.toString(druidProducerTransactionMaxTimeoutMs));
> }
> if (druidProducerTransactionTimeoutMs > 0) {
> properties.setProperty("transaction.timeout.ms",
> Integer.toString(druidProducerTransactionTimeoutMs));
> }
>
>
Have you tried increasing the timeout settings, to see if transactions
timed out?
> properties.setProperty("transactional.id", "local.druid");
>
>
Do you use multiple producers (parallelism > 1)? It seems you always set
the same transactional.id, which I expect causes problems when you have
multiple producer instances (see "zombie fencing" in [2]). In that case,
just make sure they are unique.
And one additional question: Does the error occur consistently, or only
occasionally?
Best,
Nico
[1]
https://stackoverflow.com/questions/53058715/what-is-reason-for-getting-producerfencedexception-during-producer-send
[2] https://stackoverflow.com/a/52304789
>
