Hi XTable Community,
I wanted to share a summary of open PRs and issues currently queued,
along with a suggested review and merge sequence to help keep review
overhead low and unblock releases.
### Suggested Merge Order:
1. PR #917: [916] Correct three inaccuracies in the REST service spec
- Link: https://github.com/apache/incubator-xtable/pull/917
- Scope: spec/rest-service-open-api.yaml only.
- Details: Documents required source-data-path, clarifies storage
credential scope, removes unused ErrorModel.stack, bumps spec to 0.0.2.
Self-contained and safe to merge quickly.
2. PR #897: [896] Shrink the xtable-utilities bundled jar from 1.1 GB
to 423 MB
- Link: https://github.com/apache/incubator-xtable/pull/897
- Scope: xtable-utilities/pom.xml, root pom.xml.
- Details: Drops software.amazon.awssdk:bundle in favor of granular
S3/STS modules, trims unneeded platform native binaries and unused
transitives. Reduces jar size by ~60%. Verified against live S3, ADLS Gen2,
and GCS.
3. PR #912: [911] Carry Delta's nested field ids into the converted
schema
- Link: https://github.com/apache/incubator-xtable/pull/912
- Scope: Delta schema extractor & unit tests.
- Details: Preserves delta.columnMapping.nested.ids for map and list
children when converting Delta tables with IcebergCompatV2 to Iceberg.
---
### Open Discussions / Issues for Feedback:
- Issue #918: Is rest-service-open-api.yaml a design target or a
description of the service?
(https://github.com/apache/incubator-xtable/issues/918)
Looking for maintainer alignment on whether the OpenAPI specification
is intended to be an aspirational design target (async job resource,
polling endpoints) or strictly mirror the current JAX-RS endpoints.
- Issue #907: Absolute paths in the Delta log treated as relative
(silent row loss)
(https://github.com/apache/incubator-xtable/issues/907)
A fix will be prepared for DeltaActionsConverter.getFullPathToFile.
- Issue #901: Parquet source derives schema from single file footer
(https://github.com/apache/incubator-xtable/issues/901)
Feedback welcome on schema merging across Parquet files and partition
column derivation.
Please review the PRs in the proposed order when you have a moment.
Feedback and approvals are welcome!
Best regards,
Sylwester
