CaptainAni187 opened a new pull request, #986: URL: https://github.com/apache/incubator-xtable/pull/986
Fixes #970 ## What is the purpose of the pull request Delta stores the `path` of add/remove actions as a URI, so partition dirs with special characters are percent-encoded in the log (`level=level 1` → `level=level%201`). `DeltaPathUtils.getFullPathToFile` joined that encoded path to the base path without decoding it, so both Delta sources (`DeltaActionsConverter` and `DeltaKernelActionsConverter`) produced a `physicalPath` that doesn't exist and the synced target pointed at missing files. ## Brief change log - `getFullPathToFile` decodes the log path before resolving it. If the path isn't a valid URI (e.g. a raw space, which is what earlier XTable versions wrote into Delta targets) it's used as is, so those logs still read the same as before. - `getPathForDeltaLog` encodes the path it records, as you suggested in the issue, so a Delta target written by XTable round trips. `:` is encoded too in relative paths, otherwise a partition like `ts=2024-01-01 10:00:00` would read back as an absolute path. - Paths without special characters come out unchanged on both sides. ## Verify this pull request This change added tests and can be verified as follows: - `TestDeltaPathUtils`: encoded relative and absolute paths (`level%201`, the `level%25252` case from the issue), an unencoded legacy path, encoding when writing, and a round trip through `getPathForDeltaLog` + `getFullPathToFile` for spaces, `%`, `+`/`&` and `:`. The new cases fail without the change. - Existing Delta and Kernel unit tests plus `ITDeltaConversionSource` and `ITDeltaKernelConversionSource` still pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
