zhang-arvin opened a new pull request, #932: URL: https://github.com/apache/incubator-xtable/pull/932
Closes #907 ## Description The Delta protocol (PROTOCOL.md, Add File and Remove File actions) allows a data file path to be relative to the table root **or absolute**. XTable's delta sources always assume relative paths and concatenate the table base path onto the logged path when it does not lexically start with the base, producing locations like `s3a://bucket/tab/s3://bucket/tab/part-0.parquet`. The resulting manifest entries carry `record_count=0` and are pruned during scan planning, so the sync reports success while silently dropping rows from the converted table. ## Root cause `DeltaActionsConverter.getFullPathToFile` and `DeltaKernelActionsConverter.getFullPathToFile` use `String.startsWith` prefix matching to decide whether the logged path is already resolved. Any absolute path not lexically under the table base (a scheme alias such as `s3://` against an `s3a://` base, a same-scheme different-bucket/container location, or a `file://` URI) fails the prefix test and is treated as relative, mangling the location. ## Changes - `DeltaActionsConverter.getFullPathToFile`: use the logged path as-is when it is absolute. A path is absolute when it has a URI scheme (e.g. `s3://`, `s3a://`, `hdfs://`, `file://`, `abfss://`, `gs://`) or starts with `/`. - `DeltaKernelActionsConverter.getFullPathToFile`: same fix for the Delta Kernel source path. - Added unit tests for relative paths, absolute paths under the table base (unchanged), scheme-alias paths, absolute paths outside the table base, and `file://` URIs. With the fix, all of the failure cases from the issue resolve correctly: | tableBasePath | dataFilePath | result | |---|---|---| | `s3a://bucket/tab` | `part-0.parquet` | `s3a://bucket/tab/part-0.parquet` | | `s3a://bucket/tab` | `s3a://bucket/tab/part-0.parquet` | unchanged | | `s3a://bucket/tab` | `s3://bucket/tab/part-0.parquet` | unchanged | | `s3a://bucket/tab` | `s3://other-bucket/x/part-0.parquet` | unchanged | | `s3a://bucket/tab` | `file:///local/t/part-0.parquet` | unchanged | ## Validation - `./mvnw -pl xtable-core -am -Dtest=TestDeltaActionsConverter -DfailIfNoTests=false -DskipITs test` — 6 tests, all passing - `./mvnw spotless:check -pl xtable-core` — passing -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
