slachiewicz commented on PR #29:
URL:
https://github.com/apache/iceberg-verification/pull/29#issuecomment-7017193668
Prepared fixes for the review comments below. I don't have push access to
this fork, so here is the patch rather than a pushed commit. It applies cleanly
to the current head (`2c8e551`) and `make check` passes.
- @nssalian's timestamp nit: the two columns hold the same micros, so the
check must run outside UTC; the wrong-type/zone-shift claim is now limited to
readers that take the type from the Parquet annotation rather than the Iceberg
schema.
- @nssalian's versions nit: the summary now names PyIceberg 0.12.0, Iceberg
Java 1.11.0 and 1.12.0, iceberg-go a9e2cc3, and iceberg-rust 0.11.0.
- @sungwy's Scope question: added a dedicated `Scope` section to the new
table README, as now required by #30.
```diff
diff --git a/reference-tables/README.md b/reference-tables/README.md
index 951155f..051de05 100644
--- a/reference-tables/README.md
+++ b/reference-tables/README.md
@@ -234,8 +234,9 @@ its `initial-default`, less the positions any deletion
vector that applies to
it deletes. The rows of `v3/deletion-vector-parquet-data` were instead
derived
from its Input, and a PyIceberg 0.12.0 scan returns the same rows. The rows
of
`v2/primitive-types-parquet-data` were also derived from its Input, then
checked
-against the same read with no Iceberg library and against scans with
PyIceberg,
-Iceberg Java, iceberg-go and iceberg-rust.
+against the same read with no Iceberg library and against scans with
PyIceberg
+0.12.0, Iceberg Java 1.11.0 and 1.12.0, iceberg-go a9e2cc3, and iceberg-rust
+0.11.0.
## Procedure
diff --git a/reference-tables/v2/primitive-types-parquet-data/README.md
b/reference-tables/v2/primitive-types-parquet-data/README.md
index 19f3fdf..5bbb324 100644
--- a/reference-tables/v2/primitive-types-parquet-data/README.md
+++ b/reference-tables/v2/primitive-types-parquet-data/README.md
@@ -24,13 +24,21 @@ the spec's JSON forms. The one commit appends one data
file holding its rows,
written in the spec's single-value JSON keyed by field ID, with `long`
values as
strings.
+## Scope
+
+Primitive types read from Parquet: every v1 and v2 primitive type, the three
+decimal physical encodings, the `isAdjustedToUTC` flag on
`TIMESTAMP_MICROS`,
+and three rows of typical, boundary and null values.
+
The table holds one column of every v1 and v2 primitive type. The three
decimal
columns use each of the Parquet physical types the spec maps a decimal to:
`int32` for precision up to 9, `int64` up to 18, and `fixed_len_byte_array`
otherwise. A reader that decodes only one of them fails on the others. The
-`timestamp` and `timestamptz` columns differ only in the `TIMESTAMP_MICROS`
-annotation's `isAdjustedToUTC` flag, so a reader that ignores the flag
returns
-one of them with the wrong type, or shifted by its local zone.
+`timestamp` and `timestamptz` columns hold the same micros and differ only
in
+the `TIMESTAMP_MICROS` annotation's `isAdjustedToUTC` flag, so the check
must run
+in a non-UTC zone: in UTC a reader that ignores the flag still returns the
right
+rows. Outside UTC, a reader that takes the type from the Parquet annotation
+rather than the Iceberg schema returns one of them shifted by its local
zone.
Row 1 holds typical values, except that `int_col` and `long_col` hold their
maxima and `time_col` holds `00:00:00`. Row 2 holds boundary values: the
int and
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]