alamb commented on code in PR #8220:
URL: https://github.com/apache/arrow-rs/pull/8220#discussion_r2300634248
##########
arrow-avro/src/reader/record.rs:
##########
@@ -1471,4 +1753,196 @@ mod tests {
assert!(int_array.is_null(0)); // row1 is null
assert_eq!(int_array.value(1), 42); // row3 value is 42
}
+
+ fn make_record_resolved_decoder(
Review Comment:
I didn't fully follow these tests. but I didn't find any coverage for
skipping the nested types (Lists, Maps, Structs).
I ran `llvm-cov` to double check and it seems to imply this code isn't
tested:
```shell
cargo llvm-cov --html -p arrow-avro
```
Report is here:
[coverage.zip](https://github.com/user-attachments/files/21986560/coverage.zip)
For example
`coverage/Users/andrewlamb/Software/arrow-rs/arrow-avro/src/reader/record.rs.html`
<img width="885" height="794" alt="Screenshot 2025-08-26 at 7 05 43 AM"
src="https://github.com/user-attachments/assets/2b5b432a-550a-41a4-b1a1-1b159ff0da01";
/>
##########
arrow-avro/src/reader/record.rs:
##########
@@ -736,6 +858,166 @@ fn sign_extend_to<const N: usize>(raw: &[u8]) ->
Result<[u8; N], ArrowError> {
Ok(arr)
}
+/// Lightweight skipping decoder for writer-only fields
Review Comment:
I found the term "writer only" field somewhat confusing -- I think the same
concept (not decoding fields into arrow that are not requested) is called
"non-projected fields" in the parquet, json, and csv readers.
I think the name `skipper` is quite clear, this is just a high level
comment about the terminology in the comments (I know, 🙄 )
##########
arrow-avro/src/reader/mod.rs:
##########
@@ -1537,6 +1564,57 @@ mod test {
assert!(batch.column(0).as_any().is::<StringViewArray>());
}
+ #[test]
+ fn test_alltypes_skip_writer_fields_keep_double_only() {
+ let file = arrow_test_data("avro/alltypes_plain.avro");
Review Comment:
- It is so cool to me to see the files added by @Igosuki in
https://github.com/apache/arrow-testing/pull/62 keep paying off / are used
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
