Hi folks,
There is already work underway to return pre-signed URLs from a plan API response, which I believe William and Prashant have scoped to support that user journey [1]. As I was reviewing that work, I began thinking ahead about delegated access patterns for the ongoing FILE type [2] work, and as Dan mentioned in this comment [3], we have yet to have a dedicated discussion on access delegation for FILE. So I wanted to start this thread to begin that discussion. Today access is delegated through remote signing and vended credentials, and I believe both can support FILE type access delegation use cases as long as the object is read by a storage client. With multimodal access, the interesting work often happens in other services, for example an inference service, that need to retrieve the objects to run the inference itself, the result of which is then returned to the calling process for further analytics. While we could require that these functions run in the calling process, or that those services act as storage clients themselves, I think enabling FILE reference URLs to be signed so that they are optimized for sharing with other services will help Iceberg's FILE integration plug readily into the existing multimodal ecosystem. While work is still underway on the FILE type discussions to finalize the shape of the Iceberg type, I'd like to start the discussion on what access enhancements would need to be added to the Iceberg REST Catalog spec and its supporting SDKs. I've put together a spec PR based on recent catalog community discussion [4] on pre-signed urls, along with a draft PR that POCs the integration on the client side [5]. I'd love to get the community's thoughts on whether client initiated pre-signing + sharing URLs is an additional mode we want to introduce to enable FILE access sharing as we look to integrate the Iceberg client into the multimodal ecosystem. I'd also be interested to hear what other consumer patterns people are running into beyond integrating with an inference service, which is the case I've been designing against. Sung [1] https://github.com/apache/iceberg/pull/17457 [2] https://s.apache.org/iceberg-fileref [3] https://github.com/apache/iceberg/pull/17332#discussion_r3992139348 [4] https://github.com/apache/iceberg/pull/18080 [5] https://github.com/apache/iceberg/pull/18110
