jecsand838 commented on code in PR #8316:
URL: https://github.com/apache/arrow-rs/pull/8316#discussion_r2345248266
##########
arrow-avro/src/reader/mod.rs:
##########
@@ -138,7 +251,77 @@ fn is_incomplete_data(err: &ArrowError) -> bool {
)
}
-/// A low-level interface for decoding Avro-encoded bytes into Arrow
`RecordBatch`.
+/// A low‑level, push‑based decoder from Avro bytes to Arrow `RecordBatch`.
+///
+/// `Decoder` is designed for **streaming** scenarios:
+///
+/// * You *feed* freshly received bytes using `Self::decode`, potentially
multiple times,
+/// until at least one row is complete.
+/// * You then *drain* completed rows with `Self::flush`, which yields a
`RecordBatch`
+/// if any rows were finished since the last flush.
+///
+/// Unlike `Reader`, which is specialized for Avro **Object Container Files**,
`Decoder`
+/// understands **framed single‑object** inputs and **Confluent Schema
Registry** messages,
+/// switching schemas mid‑stream when the framing indicates a new fingerprint.
+///
+/// ### Supported prefixes
+///
+/// On each new row boundary, `Decoder` tries to match one of the following
"prefixes":
+///
+/// * **Single‑Object encoding**: magic `0xC3 0x01` + schema fingerprint
(length depends on
+/// the configured `FingerprintAlgorithm`); see `SINGLE_OBJECT_MAGIC`.
+/// * **Confluent wire format**: magic `0x00` + 4‑byte big‑endian schema id;
see
+/// `CONFLUENT_MAGIC`.
+///
+/// The active fingerprint determines which cached row decoder is used to
decode the following
+/// record body bytes.
+///
+/// ### Schema switching semantics
+///
+/// When a new fingerprint is observed:
+///
+/// * If the current batch is empty, the decoder switches immediately;
+/// * Otherwise, the current batch is finalized on the next `flush` and only
then
+/// does the decoder switch to the new schema. This guarantees that a single
`RecordBatch`
+/// never mixes rows with different schemas.
+///
+/// ### Examples
+///
+/// Build a `Decoder` for single‑object encoding using a `SchemaStore` with
Rabin fingerprints:
+///
+/// ```no_run
+/// use arrow_avro::schema::{AvroSchema, SchemaStore};
+/// use arrow_avro::reader::ReaderBuilder;
+///
+/// let mut store = SchemaStore::new(); // Rabin by default
+/// let avro = AvroSchema::new(r#""string""#.to_string());
+/// let _fp = store.register(avro).unwrap();
+///
+/// let mut decoder = ReaderBuilder::new()
+/// .with_writer_schema_store(store)
+/// .with_batch_size(512)
+/// .build_decoder()
+/// .unwrap();
+///
+/// // Feed bytes (framed as 0xC3 0x01 + fingerprint and body)
Review Comment:
That's a good idea as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
