jecsand838 commented on code in PR #8006:
URL: https://github.com/apache/arrow-rs/pull/8006#discussion_r2248580794
##########
arrow-avro/src/reader/mod.rs:
##########
@@ -216,34 +365,97 @@ impl ReaderBuilder {
/// - `batch_size` = 1024
/// - `strict_mode` = false
/// - `utf8_view` = false
- /// - `schema` = None
+ /// - `reader_schema` = None
+ /// - `writer_schema_store` = None
+ /// - `active_fingerprint` = None
pub fn new() -> Self {
Self::default()
}
- fn make_record_decoder(&self, schema: &AvroSchema<'_>) ->
Result<RecordDecoder, ArrowError> {
- let root_field = AvroFieldBuilder::new(schema)
- .with_utf8view(self.utf8_view)
- .with_strict_mode(self.strict_mode)
- .build()?;
- RecordDecoder::try_new_with_options(root_field.data_type(),
self.utf8_view)
+ fn make_record_decoder<'a>(
+ &self,
+ writer_schema: &AvroSchema<'a>,
+ reader_schema: Option<&AvroSchema<'a>>,
+ ) -> Result<RecordDecoder, ArrowError> {
+ let field_builder = match reader_schema {
+ Some(rs) if !compare_schemas(writer_schema, rs)? => {
+ AvroFieldBuilder::new(writer_schema).with_reader_schema(rs)
+ }
+ Some(rs) => AvroFieldBuilder::new(rs),
+ None => AvroFieldBuilder::new(writer_schema),
+ }
+ .with_utf8view(self.utf8_view)
+ .with_strict_mode(self.strict_mode);
+ let root = field_builder.build()?;
Review Comment:
@scovich There's some conditional logic that I needed to capture here.
1. If the `reader_schema` is provided and it is different from the
`writer_schema`, then we will have schema resolution. In that scenario we need
to pass in both the `writer_schema` and `reader_schema` into the
`AvroFieldBuilder`, i.e.
`AvroFieldBuilder::new(writer_schema).with_reader_schema(rs)` (I'll rename `rs`
that is pretty bad).
2. If there's a `reader_schema` and it's identical to the `writer_schema`
then we will have no schema resolution, it just becomes added overhead.
3. If there's no `reader_schema` then there's no schema resolution and we
use the `writer_schema`.
That being said I will clean this code up because it's neither clear nor
optimal. For example I could have just merged branches 2 and 3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
