Thanks Prashant, that makes sense. I’m glad to hear you are carrying William’s work forward.
If the plan is still to introduce /presign for the plan API response, go ahead; I will subscribe to the PR discussion and fold in what comes out of this thread as the use case discussion evolves. I agree that we can pool our efforts here. As noted in the FILE proposal feedback, and in the review comments on William's PR where we began discussing FILE governance, I think it will help to keep this thread on that use case. What file column usage patterns need, and what SDK or spec changes follow from it. Sung On 2026/09/15 21:33:52 Prashant Singh wrote: > Thanks for starting this thread, Sung! > > I wanted to share some additional background on the work already underway > in this area. William and I opened a spec proposal about three months ago > focused on supporting file-level access through pre-signed URLs [1]. The > initial goal was to enable partition-level sharing (with one file belonging > to a single partition). > > We went through several POCs [2] and a number of community discussions to > evaluate whether the existing remote-signing and batch remote-signing > approaches would be sufficient. In particular, the Azure POC [3] helped > highlight some of the limitations of remote signing across cloud providers > and provided useful evidence for considering pre-signed URLs as an > additional access-delegation mechanism. > > The work has two main pieces: SDK support for the /plan endpoint to return > pre-signed URLs [4], and support for obtaining/refreshing pre-signed URLs > through the signing path [5][6], along with the associated POC [7]. > > I noticed that you have opened PRs #18080 and #18110 for FILE-type access. > The foundational work from the existing PRs, including returning pre-signed > URLs from the plan API and adding client-side support, should be directly > reusable for the FILE work. Given the amount of discussion and > implementation already underway, I'd love to collaborate on building on > these existing tracks rather than creating parallel implementations. > > Happy to work through this together and make sure the existing work can > support the FILE use cases you outlined as well. > > Best, > Prashant > > [1] https://lists.apache.org/thread/odof6m2npvktwd51cz8qnrxjv95ws4wm > [2] > https://github.com/williamhyun/iceberg/pulls?q=is%3Apr+state%3Aopen+author%3Awilliamhyun+sign > [3] https://github.com/williamhyun/Azure-SAS-Header-POC > [4] https://github.com/apache/iceberg/pull/17457 > [5] https://github.com/apache/iceberg/pull/17332 > [6] > https://docs.google.com/document/d/1WyCnD63t4AIOzCgP3dWug17kZ6NnVxfOMwOEKqZx6ss/edit?tab=t.ez1lkov8k91#heading=h.vjmmhe69p5j3 > [7] https://github.com/williamhyun/iceberg/pull/206 > > On Mon, Sep 14, 2026 at 3:53 PM Sung Yun <[email protected]> wrote: > > > Hi folks, > > > > > > There is already work underway to return pre-signed URLs from a plan API > > response, which I believe William and Prashant have scoped to support that > > user journey [1]. As I was reviewing that work, I began thinking ahead > > about delegated access patterns for the ongoing FILE type [2] work, and as > > Dan mentioned in this comment [3], we have yet to have a dedicated > > discussion on access delegation for FILE. So I wanted to start this thread > > to begin that discussion. > > > > > > Today access is delegated through remote signing and vended credentials, > > and I believe both can support FILE type access delegation use cases as > > long as the object is read by a storage client. With multimodal access, > > the interesting work often happens in other services, for example an > > inference service, that need to retrieve the objects to run the inference > > itself, the result of which is then returned to the calling process for > > further analytics. While we could require that these functions run in the > > calling process, or that those services act as storage clients themselves, > > I think enabling FILE reference URLs to be signed so that they are > > optimized for sharing with other services will help Iceberg's FILE > > integration plug readily into the existing multimodal ecosystem. > > > > > > While work is still underway on the FILE type discussions to finalize the > > shape of the Iceberg type, I'd like to start the discussion on what access > > enhancements would need to be added to the Iceberg REST Catalog spec and > > its supporting SDKs. I've put together a spec PR based on recent catalog > > community discussion [4] on pre-signed urls, along with a draft PR that > > POCs the integration on the client side [5]. > > > > > > I'd love to get the community's thoughts on whether client initiated > > pre-signing + sharing URLs is an additional mode we want to introduce to > > enable FILE access sharing as we look to integrate the Iceberg client into > > the multimodal ecosystem. > > > > > > I'd also be interested to hear what other consumer patterns people are > > running into beyond integrating with an inference service, which is the > > case I've been designing against. > > > > > > Sung > > > > > > [1] https://github.com/apache/iceberg/pull/17457 > > > > [2] https://s.apache.org/iceberg-fileref > > > > [3] https://github.com/apache/iceberg/pull/17332#discussion_r3992139348 > > > > [4] https://github.com/apache/iceberg/pull/18080 > > > > [5] https://github.com/apache/iceberg/pull/18110 > > >
