Hello everyone,
We are encountering some issues with NiFi's Kafka Producer restarting frequently. Given that we are utilizing transactions, we have some concerns regarding the logic governing the Transactional ID. Specifically, we're interested in understanding why the transactional.id's trailing UUID, generated by NiFi, changes across restarts, consequently leading to a change in the Producer ID (PID). Our understanding is that for a Kafka producer to maintain idempotence and support exactly-once semantics, it should retain the PID to identify the same producer instance across process restarts. By preserving the transactional producer's unique identifier and incrementing the associated epoch, any producers with the same transactional ID but an older epoch are deemed inactive and subsequently barred, resulting in rejection of future transactional writes from those producers. We fail to see how a continually changing PID aligns with the expected behavior of Kafka transactions. Thus, we are seeking clarification on why the NiFi producer operates in this manner concerning transactions. Best regards, Tirso ----------------------------------------- Tirso del Rey [ https://www.zylk.net/ | ZYLK.net ] :: consultoría.openSource Ribera de Axpe, 11 Edificio A, modulo 201-203 48950 Erandio (Bizkaia) ofic.: +34 944272119 ----------------------------------------- Esta comunicación y su contenido, incluidos los datos personales, son confidenciales, y para uso exclusivo del destinatario/s al/os que se dirige/n. Si ha recibido este mensaje por error, le rogamos nos lo notifique inmediatamente por esta misma vía y proceda a su eliminación. Consulte información detallada sobre el tratamiento de datos en zylk.net
