alamb opened a new issue #148: URL: https://github.com/apache/arrow-rs/issues/148
*Note*: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-10979 Introduce a basic Kafka reader based on `rdkafka`. Exposes an `Iterator` interface which yields `Result<RecordBatch>`. Columns in the batch are: *key* (Binary, nullable): The key of a message, if present. *payload* (Binary, nullable): The payload bytes of a message, if present. *topic* (Utf8): The topic of the message. *partition* (Int32): The partition of the message. *offset* (Int64): The offset of the message. Note that `rdkafka` has a C++ dependency (`librdkafka`), but we can choose to make this dynamically linked. `rdkafka` provides an `async` Consumer, but I have explicitly chosen the non-`async` Consumer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
