yashmayya opened a new pull request, #12984:
URL: https://github.com/apache/kafka/pull/12984
- Kafka Connect's `POST /connectors` and `PUT
/connectors/{connector}/config` REST APIs internally simply write a message to
the Connect cluster's internal config topic (which is then processed
asynchronously by the herder).
- However, no callback is passed to the producer's send method and there is
no error handling in place for producer send failures (see
[here](https://github.com/apache/kafka/blob/c1a54671e8fc6c7daec5f5ec3d8c934be96b4989/connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java#L716)
/
[here](https://github.com/apache/kafka/blob/c1a54671e8fc6c7daec5f5ec3d8c934be96b4989/connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java#L726)).
- Consider one such case where the Connect worker's principal doesn't have a
WRITE ACL on the cluster's config topic. Now suppose the user submits a
connector's configs via one of the above two APIs. The producer send
[here](https://github.com/apache/kafka/blob/c1a54671e8fc6c7daec5f5ec3d8c934be96b4989/connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java#L716)
/
[here](https://github.com/apache/kafka/blob/c1a54671e8fc6c7daec5f5ec3d8c934be96b4989/connect/runtime/src/main/java/org/apache/kafka/connect/storage/KafkaConfigBackingStore.java#L726)
won't succeed (due to a TopicAuthorizationException) but the API responses
will be `201 Created` success responses anyway.
- This is a very poor UX because the connector will actually never be
created but the API response indicated success. Furthermore, this failure would
only be detectable if TRACE logs are enabled (via [this
log)](https://github.com/apache/kafka/blob/df29b17fc40f7c15460988d58bc652c3d66b60f8/clients/src/main/java/org/apache/kafka/clients/producer/internals/ProducerBatch.java)
making it near impossible for users to debug
- This PR uses producer callbacks to surface write failures back to the user
via the API response.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
