mbutrovich opened a new pull request, #2895: URL: https://github.com/apache/datafusion-comet/pull/2895
## Which issue does this PR close? N/A. ## Rationale for this change REST catalogs return metadata locations that don't physically exist on disk - the metadata is transferred via HTTP. This caused native Iceberg scans to fall back to Spark. ## What changes are included in this PR? Fixed reflection for protected methods: - Changed `IcebergReflection.getTableMetadata()` and `getFormatVersion()` to use `getDeclaredMethod()` with `setAccessible(true)` to access protected methods in `BaseTable` Handle missing metadata files: - Detect when metadata file doesn't exist (REST catalog case) - Fall back to table location for `FileIO` initialization in iceberg-rust - The location is only used to determine storage backend type (local/S3/GCS) - actual metadata comes from the `FileScanTask`s REST catalog test infrastructure: - Copied `RESTCatalogAdapter` and `RESTCatalogServlet` from Iceberg - Added Jetty dependencies for embedded REST server - Added `withRESTCatalog` test helper ## How are these changes tested? New test "REST catalog with native Iceberg scan" creates an embedded Jetty server with InMemoryCatalog backend, creates a table via REST API, and verifies native Iceberg scan is used with correct results. All Iceberg Java tests pass locally with these changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
