vinishjail97 opened a new pull request, #956: URL: https://github.com/apache/incubator-xtable/pull/956
## What is the purpose of the pull request `ParquetConversionSource.getParquetFiles` lists files with `FileSystem.get(hadoopConf)`, which is the default file system, and converts the base path through `java.nio.file.Paths.get(URI)`. `Paths.get` needs an NIO provider for the scheme, so any object store base path fails before listing, for example `s3a://` with `FileSystemNotFoundException: Provider "s3a" not installed`. The parquet source therefore only works on local paths today. ## Brief change log - Resolve the file system from the base path in `getParquetFiles`, the same way `ParquetDataManager` already does. ## Verify this pull request - A new `TestParquetConversionSource` serves the local file system under a non-default scheme and runs `getCurrentSnapshot`. It fails on main with the same `FileSystemNotFoundException` and passes with this change. - `xtable-core` unit tests and `ITParquetConversionSource` pass locally. - Validated end to end on S3, with TPC-DS 100GB `customer` raw parquet converted to Iceberg and read back with matching row counts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
