vinishjail97 opened a new issue, #970: URL: https://github.com/apache/incubator-xtable/issues/970
### Describe the bug The Delta protocol stores the `path` of an `add` or `remove` action as a URI, so partition directories with special characters are percent-encoded in the log. `DeltaPathUtils.getFullPathToFile` joins that path to the table base path without decoding it, and both Delta sources use it (`DeltaActionsConverter` and `DeltaKernelActionsConverter`). The resulting `physicalPath` points at a file that does not exist, so any target synced from such a Delta table references missing files. Example from a Delta table partitioned by a string column written with Spark 3.4 / Delta 2.4: | Partition value | Directory on disk | Path XTable passes to the target | |---|---|---| | `level 1` | `level=level 1/part-…parquet` | `level=level%201/part-…parquet` | | `level%2` | `level=level%252/part-…parquet` | `level=level%25252/part-…parquet` | A Delta to Hudi sync of this table fails when Hudi reads the files: ``` java.io.FileNotFoundException: File file:/.../level=level%201/part-00001-....snappy.parquet does not exist ``` ### To reproduce 1. Write a Delta table partitioned by a string column that has a value with a space or a `%`. 2. Sync it to Hudi or Iceberg with XTable. 3. The target metadata references the encoded paths, and reading or committing the target fails. ### Expected behavior The Delta source decodes the log path before it builds the physical path. The Delta target likely needs the inverse: `DeltaPathUtils.getPathForDeltaLog` should encode the path it writes into the log, which is worth checking with a round trip test. ### Context Found while adding a Delta table with these partition values to the secondary index IT in #935. The index itself is not involved, since the sync fails before the index is built. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
