Kevin Flansburg created ARROW-10979:
---------------------------------------
Summary: [Rust] Basic Kafka Reader
Key: ARROW-10979
URL: https://issues.apache.org/jira/browse/ARROW-10979
Project: Apache Arrow
Issue Type: Improvement
Components: Rust
Reporter: Kevin Flansburg
Introduce a basic Kafka reader based on `rdkafka`. Exposes an `Iterator`
interface which yields `Result<RecordBatch>`.
Columns in the batch are:
*key* (Binary, nullable): The key of a message, if present.
*payload* (Binary, nullable): The payload bytes of a message, if present.
*topic* (Utf8): The topic of the message.
*partition* (Int32): The partition of the message.
*offset* (Int64): The offset of the message.
Note that `rdkafka` has a C++ dependency (`librdkafka`), but we can choose to
make this dynamically linked. `rdkafka` provides an `async` Consumer, but I
have explicitly chosen the non-`async` Consumer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
