Hi all, I'd like to start the discussion on KIP-1397, which adds share group support for Kafka Connect sink tasks. https://cwiki.apache.org/confluence/spaces/KAFKA/pages/462817279/KIP-1397+Share+Group+Queue+Support+for+Kafka+Connect+Sink+Connectors+via+ShareSinkTask
It is meant to complement KIP-1302 (cc'ing Shekhar Rajak). KIP-1302 introduces the opt-in share runtime for Connect sinks and a task contract for connectors whose put() returns only once every record is durable. KIP-1397 keeps that runtime and its durability principle. It adds a second task contract for connectors whose durable write completes after put() returns. It also covers poison-pill/DLQ handling, throttling, offset administration and permissions, which KIP-1302's first version defers. It builds on the earlier discussion on the KIP-1302 thread and tries to address some of the points raised there: https://lists.apache.org/thread/p5zkp20rno8zz55mrjlw9brm6os5gfbv One thing to settle early: both KIPs use the name ShareSinkTask. Shekhar, I'm happy to rename mine, or to fold this into KIP-1302 as an additional contract if you'd prefer. Whatever works best for you. Feedback on the design, particularly the per-record acknowledgement API and the DLQ approach, is very welcome. The KIP draft was written with assistance from Claude; I've reviewed it and take responsibility for its content. Thanks, Ashwin
