Thanks for starting this thread, Sung! I wanted to share some additional background on the work already underway in this area. William and I opened a spec proposal about three months ago focused on supporting file-level access through pre-signed URLs [1]. The initial goal was to enable partition-level sharing (with one file belonging to a single partition).
We went through several POCs [2] and a number of community discussions to evaluate whether the existing remote-signing and batch remote-signing approaches would be sufficient. In particular, the Azure POC [3] helped highlight some of the limitations of remote signing across cloud providers and provided useful evidence for considering pre-signed URLs as an additional access-delegation mechanism. The work has two main pieces: SDK support for the /plan endpoint to return pre-signed URLs [4], and support for obtaining/refreshing pre-signed URLs through the signing path [5][6], along with the associated POC [7]. I noticed that you have opened PRs #18080 and #18110 for FILE-type access. The foundational work from the existing PRs, including returning pre-signed URLs from the plan API and adding client-side support, should be directly reusable for the FILE work. Given the amount of discussion and implementation already underway, I'd love to collaborate on building on these existing tracks rather than creating parallel implementations. Happy to work through this together and make sure the existing work can support the FILE use cases you outlined as well. Best, Prashant [1] https://lists.apache.org/thread/odof6m2npvktwd51cz8qnrxjv95ws4wm [2] https://github.com/williamhyun/iceberg/pulls?q=is%3Apr+state%3Aopen+author%3Awilliamhyun+sign [3] https://github.com/williamhyun/Azure-SAS-Header-POC [4] https://github.com/apache/iceberg/pull/17457 [5] https://github.com/apache/iceberg/pull/17332 [6] https://docs.google.com/document/d/1WyCnD63t4AIOzCgP3dWug17kZ6NnVxfOMwOEKqZx6ss/edit?tab=t.ez1lkov8k91#heading=h.vjmmhe69p5j3 [7] https://github.com/williamhyun/iceberg/pull/206 On Mon, Sep 14, 2026 at 3:53 PM Sung Yun <[email protected]> wrote: > Hi folks, > > > There is already work underway to return pre-signed URLs from a plan API > response, which I believe William and Prashant have scoped to support that > user journey [1]. As I was reviewing that work, I began thinking ahead > about delegated access patterns for the ongoing FILE type [2] work, and as > Dan mentioned in this comment [3], we have yet to have a dedicated > discussion on access delegation for FILE. So I wanted to start this thread > to begin that discussion. > > > Today access is delegated through remote signing and vended credentials, > and I believe both can support FILE type access delegation use cases as > long as the object is read by a storage client. With multimodal access, > the interesting work often happens in other services, for example an > inference service, that need to retrieve the objects to run the inference > itself, the result of which is then returned to the calling process for > further analytics. While we could require that these functions run in the > calling process, or that those services act as storage clients themselves, > I think enabling FILE reference URLs to be signed so that they are > optimized for sharing with other services will help Iceberg's FILE > integration plug readily into the existing multimodal ecosystem. > > > While work is still underway on the FILE type discussions to finalize the > shape of the Iceberg type, I'd like to start the discussion on what access > enhancements would need to be added to the Iceberg REST Catalog spec and > its supporting SDKs. I've put together a spec PR based on recent catalog > community discussion [4] on pre-signed urls, along with a draft PR that > POCs the integration on the client side [5]. > > > I'd love to get the community's thoughts on whether client initiated > pre-signing + sharing URLs is an additional mode we want to introduce to > enable FILE access sharing as we look to integrate the Iceberg client into > the multimodal ecosystem. > > > I'd also be interested to hear what other consumer patterns people are > running into beyond integrating with an inference service, which is the > case I've been designing against. > > > Sung > > > [1] https://github.com/apache/iceberg/pull/17457 > > [2] https://s.apache.org/iceberg-fileref > > [3] https://github.com/apache/iceberg/pull/17332#discussion_r3992139348 > > [4] https://github.com/apache/iceberg/pull/18080 > > [5] https://github.com/apache/iceberg/pull/18110 >
