claudevdm commented on code in PR #39597:
URL: https://github.com/apache/beam/pull/39597#discussion_r3814237542
##########
sdks/java/io/iceberg/src/test/java/org/apache/beam/sdk/io/iceberg/catalog/IcebergCatalogBaseIT.java:
##########
@@ -738,6 +753,250 @@ public void testWriteRead() throws IOException {
containsInAnyOrder(expectedRows.stream().map(RECORD_FUNC::apply).toArray()));
}
+ /**
+ * Cross-engine consistency: rows written through the Iceberg catalog must
be readable with
+ * BigQueryIO's Storage Read API using the catalog's BigQuery table
reference (see {@link
+ * #bigQueryTableSpec(String)}). Exercises the full read, server-side
projection + filtering
+ * push-down, and a query read with the reference embedded in SQL.
+ *
+ * <p>Rows are compared on a projection of fields whose types survive the
Iceberg-to-BigQuery
+ * mapping losslessly; BigQuery widens e.g. {@code int32} to {@code INT64},
so whole-row equality
+ * against the Iceberg schema does not hold by design.
+ */
+ @Test
+ public void testReadWithBigQueryIO() throws Exception {
Review Comment:
BigQuery has no int32, float32, or micros-DATETIME, so the read-back Rows
have a different Beam schema (INT64/DOUBLE/Joda-millis DATETIME) than what we
wrote. Row.equals compares schemas first, so direct comparison can never pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
