kotwal-itpro commented on PR #8795: URL: https://github.com/apache/hop/pull/8795#issuecomment-6086067170
Thanks @mattcasters, both fixed in db477871f2 (rebased onto main): **Parent check.** The path is now canonicalized before it's split, and only a parent that is a scheme alone (`file:`, `file://`, `s3a:/`) is rejected. A table in a Windows drive root works (`file:///C:/orders` has warehouse `file:///C:`), and so does `s3a://bucket/lake//orders`, which now resolves to the same table as the single-slash path. `file:///t` and `s3a://bucket` are still rejected. On the read, write and maintenance paths an invalid path now comes back as a `HopException` that names the transform, e.g. `Spark Lake Table 'write orders': Iceberg table path 'file:///t' needs a parent folder ...`. **One folder, one catalog.** `canonicalLocation` lowercases the scheme, turns `file:/x`, `file://localhost/x` and `file:///x` into `file:///x`, collapses empty segments and resolves `.` and `..`. The canonical parent is what gets hashed into `hop_iceberg_<hash>` and what's set as the catalog's warehouse, so the `CachingCatalog` problem you described can't happen through different spellings. `toTableLocationUri` itself is unchanged, since Delta and the other formats use it. New tests in `SparkLakeTableSupportTest`: the drive root, `lake//orders` plus `./` and `../`, five spellings of the same folder mapping to one catalog, and the transform name in the error. Spark tests: 186 (was 182), RAT clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
