krishvishal opened a new issue, #4527: URL: https://github.com/apache/iggy/issues/4527
## Summary The gateway caps several APIs below Kafka trunk: | API | Gateway max | Trunk max | What the higher versions add | | --- | --- | --- | --- | | Metadata | 9 | 13 | Topic ids (v10), `REBOOTSTRAP_REQUIRED` (v13) | | Fetch | 12 | 18 | Topic ids (v13) | | Produce | 9 | 13 | Topic ids (v13) | | ListOffsets | 6 | 11 | `MAX_TIMESTAMP` (v7), tiered storage specs | | ApiVersions | 3 | 5 | Cluster and node id test (v5, KIP-1242) | | OffsetCommit, OffsetFetch | 9 | 10 | Topic ids | | CreateTopics, DeleteTopics | 5 | 7, 6 | Topic ids | | FindCoordinator | 4 | 6 | Share group coordinators | Clients negotiate down, so only one failure is known today. `@platformatic/kafka` sets every topic id to zero at Metadata v9, and its consumer keys topics by id. A consumer of two or more topics therefore labels every record with the last topic. This comes from reading its code. A missing Metadata v9 in Redpanda broke the same library. ## Depends on - #4519 for ApiVersions v5 ## Scope - [ ] A stable UUID per Iggy topic, stored with the topic - [ ] Metadata v10+ with real topic ids, then Fetch v13+, OffsetCommit and OffsetFetch v10, and CreateTopics and DeleteTopics by id - [ ] Never advertise Produce v13 before Metadata returns real topic ids. Otherwise the Java producer sends a zero id and fails its batches with `IllegalStateException` - [ ] ListOffsets v7+, with #4516 ## Acceptance criteria - [ ] `@platformatic/kafka` consumes two topics and labels each record with its own topic - [ ] The Java client negotiates the new versions and passes the existing e2e suite ## References - [Sender.java:626-634](https://github.com/apache/kafka/blob/b15258c7398889248d7fbf2a2218cc01346ebf92/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java#L626-L634) - `@platformatic/kafka`: https://github.com/platformatic/kafka/tree/v2.14.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
