slachiewicz commented on issue #886: URL: https://github.com/apache/incubator-xtable/issues/886#issuecomment-5572669213
Two findings that I think belong in the prerequisite list, because they affect whether the flip is safe rather than whether it's at parity. **The kernel target never writes `delta.columnMapping.physicalName`.** `DeltaKernelSchemaExtractor.getFieldMetadata` (`DeltaKernelSchemaExtractor.java:391`) writes `delta.columnMapping.id`, the XTable logical-type marker and the comment. The read path handles `physicalName` at `:166`, so the asymmetry is in the write path only. Kernel rejects a schema in that state: `IllegalArgumentException: Both columnId and physicalName must be present if one is present.` **Tables the kernel target creates have column mapping disabled.** `getConfigurationsForDeltaSync` (`DeltaKernelConversionTarget.java:483`) sets `XTABLE_METADATA` and `DELTA_LOG_RETENTION_DURATION` and nothing else — never `delta.columnMapping.mode`. Kernel then refuses any commit carrying a schema on such a table, and it treats every `withSchema` call as schema evolution, not only the schema-changing ones. Together these mean schema evolution through the kernel target does not work today, with or without column mapping on the source. That's a behaviour difference from Standalone rather than a missing test, so it belongs alongside the #711 and #713 items above. Scope of what I checked: the two code readings are against current `main`. The Kernel-side errors I reproduced in a standalone harness against `delta-kernel` 4.1.0; this project pins 4.0.0 at `pom.xml:60` and I have not re-run the repro on 4.0.0. I have a local branch exploring a fix and can open it as a draft if that's useful. *This comment was created with AI assistance.* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
