c-thiel commented on code in PR #2880:
URL: https://github.com/apache/iceberg-rust/pull/2880#discussion_r4446421751
##########
crates/iceberg/src/arrow/reader/projection.rs:
##########
@@ -2017,4 +2586,353 @@ message schema {
.collect();
assert_eq!(ids, vec![2, 3]);
}
+
+ // ---------------------------------------------------------------------
+ // End-to-end variant read tests (synthetic Parquet).
+ //
+ // Write the unshredded variant layout (a `group { metadata; value }`
tagged with the
+ // `arrow.parquet.variant` extension, so the writer emits the Parquet
variant annotation
Review Comment:
Thanks, both points are fixed in c2835f9c3.
The banner now says what the tests actually do: the files carry a field id
on the variant group and no Parquet `VARIANT` annotation, unless a test says
otherwise.
On the assertions: every end-to-end test that returns a variant now checks
`extension_type_name()` on the returned field, and compares the whole column
against the expected struct in the table's Arrow type, not just the raw bytes.
with the output relabelling removed, the tests fail.
I also added `test_read_variant_with_parquet_annotation`. It reads a file
whose variant group carries the real Parquet `VARIANT` annotation, in the
layout iceberg-java writes. Since the parquet crate's variant feature isn't
enabled here, the reader still identifies that group by its field id. So the
test mainly makes sure files written by Java read correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
