huaxingao commented on code in PR #16292:
URL: https://github.com/apache/iceberg/pull/16292#discussion_r3392821906
##########
spark/v4.0/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestSnapshotTableProcedure.java:
##########
@@ -394,6 +394,11 @@ public void testSnapshotWithVariant() throws IOException {
catalogName, SOURCE_NAME, tableName);
assertThat(result).as("Should have added one file").isEqualTo(1L);
+ // Disable vectorized reads for variant imported from non-Iceberg files
(not yet supported)
Review Comment:
Vectorized reads are on by default, so needing to disable them here points
to an easy-to-hit failure for users: anyone who snapshots/migrates external
Parquet with a variant column (which lacks the VARIANT logical-type annotation)
would hit a crash or wrong reads on a normal query, with no indication they
need to set this property. Could the reader auto-fall back to row-at-a-time
when the annotation is missing, instead of relying on users to know about it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
