binarybrain-009 opened a new pull request, #19073: URL: https://github.com/apache/iceberg/pull/19073
`IcebergGenerics` fails with “is not in the store” when reading the list/map columns in [verification fixture #27](https://github.com/apache/iceberg-verification/pull/27). This POC explores the per-file rule proposed in #19037: trust embedded IDs, treat containers with missing child IDs as absent, and use existing missing-field handling. It preserves Variant storage and aligns row-group filter metadata with the resolved schema. **For discussion, not for merge:** unsupported nesting/presence cases fail explicitly; the separate Avro ReadSupport path is unchanged, and Spark/Flink/vectorized integration, container defaults and performance remain unvalidated. The fixture is reused unchanged with attribution to Sylwester Lachiewicz. [Dev-list discussion](https://lists.apache.org/thread/tct2yp45ytsb6dqw1tq7b8gnfnp7jndo). Test plan: Reproduce the original fixture failure with and without name mapping, then verify identified values, null containers, full/single projections, physical column order, fully identified and ID-less controls, required-field rejection, struct-presence safeguards, and pushed null predicates with residual evaluation. All 22 targeted cases pass. On JDK 21, `./gradlew --no-daemon --max-workers=4 -DsparkVersions= -DflinkVersions= :iceberg-parquet:build` passes: 800 tests passed, 9 skipped, including formatting, Checkstyle and API checks. Restoring the original ReadConf reproduces both fixture failures; restoring the POC passes the full build. --- **AI Disclosure** - Model: GPT-6 - Platform/Tool: Codex - Human Oversight: unreviewed (author directed the experiment and requested publication; personal code review is pending) - Prompt Summary: Reproduce the partial-field-ID fixture failure, prototype the proposed per-file behavior, and audit projection, struct-presence and filter behavior. The implementation and tests are generated; the binary fixture is reused from verification PR #27. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
