binarybrain-009 opened a new pull request, #19073:
URL: https://github.com/apache/iceberg/pull/19073

   `IcebergGenerics` fails with “is not in the store” when reading the list/map 
columns in [verification fixture 
#27](https://github.com/apache/iceberg-verification/pull/27). This POC explores 
the per-file rule proposed in #19037: trust embedded IDs, treat containers with 
missing child IDs as absent, and use existing missing-field handling. It 
preserves Variant storage and aligns row-group filter metadata with the 
resolved schema. **For discussion, not for merge:** unsupported 
nesting/presence cases fail explicitly; the separate Avro ReadSupport path is 
unchanged, and Spark/Flink/vectorized integration, container defaults and 
performance remain unvalidated. The fixture is reused unchanged with 
attribution to Sylwester Lachiewicz. [Dev-list 
discussion](https://lists.apache.org/thread/tct2yp45ytsb6dqw1tq7b8gnfnp7jndo).
   
   Test plan: Reproduce the original fixture failure with and without name 
mapping, then verify identified values, null containers, full/single 
projections, physical column order, fully identified and ID-less controls, 
required-field rejection, struct-presence safeguards, and pushed null 
predicates with residual evaluation. All 22 targeted cases pass. On JDK 21, 
`./gradlew --no-daemon --max-workers=4 -DsparkVersions= -DflinkVersions= 
:iceberg-parquet:build` passes: 800 tests passed, 9 skipped, including 
formatting, Checkstyle and API checks. Restoring the original ReadConf 
reproduces both fixture failures; restoring the POC passes the full build.
   
   ---
   **AI Disclosure**
   - Model: GPT-6
   - Platform/Tool: Codex
   - Human Oversight: unreviewed (author directed the experiment and requested 
publication; personal code review is pending)
   - Prompt Summary: Reproduce the partial-field-ID fixture failure, prototype 
the proposed per-file behavior, and audit projection, struct-presence and 
filter behavior. The implementation and tests are generated; the binary fixture 
is reused from verification PR #27.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to