eric-maynard opened a new pull request, #360: URL: https://github.com/apache/polaris/pull/360
# Description In #344, checks within Polaris were relaxed so that Azure [WASB](https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-abfs-driver#prior-capability-the-windows-azure-storage-blob-driver) paths can be used to e.g. create a catalog, and so that WASB paths and ABFS paths that referred to the same object could be compared. This PR introduces a FileIO implementation that Polaris can use to interact with entities that use such a WASB path. The FileIO implementation simply rewrites WASB paths into ABFS paths, and delegations interactions with the ABFS paths to the Iceberg [ADLSFileIO](https://iceberg.apache.org/javadoc/1.5.2/org/apache/iceberg/azure/adlsv2/ADLSFileIO.html). ## Type of change Please delete options that are not relevant. - [ ] Bug fix (non-breaking change which fixes an issue) - [ ] Documentation update - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update # How Has This Been Tested? To use the new FileIO easily in Polaris, you can change the `factoryType` in the server config to `wasb`. With this change, the table creation API succeeds: ``` curl -X POST "http://localhost:8181/api/catalog/v1/wasb_test/namespaces/wasb_test_namespace/tables"; \ > -H "Authorization: Bearer ${token}" \ > -H "Content-Type: application/json" \ > -H "X-Iceberg-Access-Delegation: vended-credentials" \ > -d '{ > "name": "manual-table-1", > "schema": { > "type": "struct", > "fields": [ > {"id": 101, "name": "id", "type": "string", "required": false} > ] > } > }' \ > 2> /dev/null && echo {"metadata-location":"abfss://....","metadata":{"format-version":2,"table-uuid":"...","location":"wasbs://...", ...}} ``` Without this FileIO, it will fail with an error such as: ``` {"error":{"message":"Invalid ADLS URI: wasbs://...","type":"ValidationException","code":400}} ``` # Checklist: Please delete options that are not relevant. - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] My changes generate no new warnings -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@polaris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
