I left comments on both sets of PRs and agree we need to align the implementations.
The work for presigned urls via plan/tasks is slightly different from remote pre-signing. Plan/tasks urls are assumed to already be presigned when handed back, so the IO implementations only need to detect and execute the request. Remote pre-signing is a step prior to this where a storage native path (e.g. s3://...) is routed to the catalog for pre-signing prior to opening a stream (via the same path as scan/plan). I think we should get remote-presigning into a state where it's a viable alternative to the current remote signing (largely to address Azure's limitations on remote signing) and this should leverage the same infrastructure for plan/tasks. (Also, this does not require introducing a /presign endpoint). Once pre-signing is in place, we can leverage it as a foundation for some of the file based access delegation. I know there is some PoC work in that area (bulk signing), but I don't agree with the direction that's going and we need to align on how to introduce bulk operations in a way that works for File delegation. We'll discuss more at the File sync, but much of this is common across all of these use cases and we need to introduce the implemention in a way that aligns with all of the various use cases. -Dan On Wed, Sep 16, 2026 at 3:12 AM Sung Yun <[email protected]> wrote: > Thanks Prashant, that makes sense. I’m glad to hear you are carrying > William’s work forward. > > If the plan is still to introduce /presign for the plan API response, go > ahead; I will subscribe to the PR discussion and fold in what comes out of > this thread as the use case discussion evolves. I agree that we can pool > our efforts here. > > As noted in the FILE proposal feedback, and in the review comments on > William's PR where we began discussing FILE governance, I think it will > help to keep this thread on that use case. What file column usage patterns > need, and what SDK or spec changes follow from it. > > Sung > > On 2026/09/15 21:33:52 Prashant Singh wrote: > > Thanks for starting this thread, Sung! > > > > I wanted to share some additional background on the work already underway > > in this area. William and I opened a spec proposal about three months ago > > focused on supporting file-level access through pre-signed URLs [1]. The > > initial goal was to enable partition-level sharing (with one file > belonging > > to a single partition). > > > > We went through several POCs [2] and a number of community discussions to > > evaluate whether the existing remote-signing and batch remote-signing > > approaches would be sufficient. In particular, the Azure POC [3] helped > > highlight some of the limitations of remote signing across cloud > providers > > and provided useful evidence for considering pre-signed URLs as an > > additional access-delegation mechanism. > > > > The work has two main pieces: SDK support for the /plan endpoint to > return > > pre-signed URLs [4], and support for obtaining/refreshing pre-signed URLs > > through the signing path [5][6], along with the associated POC [7]. > > > > I noticed that you have opened PRs #18080 and #18110 for FILE-type > access. > > The foundational work from the existing PRs, including returning > pre-signed > > URLs from the plan API and adding client-side support, should be directly > > reusable for the FILE work. Given the amount of discussion and > > implementation already underway, I'd love to collaborate on building on > > these existing tracks rather than creating parallel implementations. > > > > Happy to work through this together and make sure the existing work can > > support the FILE use cases you outlined as well. > > > > Best, > > Prashant > > > > [1] https://lists.apache.org/thread/odof6m2npvktwd51cz8qnrxjv95ws4wm > > [2] > > > https://github.com/williamhyun/iceberg/pulls?q=is%3Apr+state%3Aopen+author%3Awilliamhyun+sign > > [3] https://github.com/williamhyun/Azure-SAS-Header-POC > > [4] https://github.com/apache/iceberg/pull/17457 > > [5] https://github.com/apache/iceberg/pull/17332 > > [6] > > > https://docs.google.com/document/d/1WyCnD63t4AIOzCgP3dWug17kZ6NnVxfOMwOEKqZx6ss/edit?tab=t.ez1lkov8k91#heading=h.vjmmhe69p5j3 > > [7] https://github.com/williamhyun/iceberg/pull/206 > > > > On Mon, Sep 14, 2026 at 3:53 PM Sung Yun <[email protected]> wrote: > > > > > Hi folks, > > > > > > > > > There is already work underway to return pre-signed URLs from a plan > API > > > response, which I believe William and Prashant have scoped to support > that > > > user journey [1]. As I was reviewing that work, I began thinking ahead > > > about delegated access patterns for the ongoing FILE type [2] work, > and as > > > Dan mentioned in this comment [3], we have yet to have a dedicated > > > discussion on access delegation for FILE. So I wanted to start this > thread > > > to begin that discussion. > > > > > > > > > Today access is delegated through remote signing and vended > credentials, > > > and I believe both can support FILE type access delegation use cases as > > > long as the object is read by a storage client. With multimodal access, > > > the interesting work often happens in other services, for example an > > > inference service, that need to retrieve the objects to run the > inference > > > itself, the result of which is then returned to the calling process for > > > further analytics. While we could require that these functions run in > the > > > calling process, or that those services act as storage clients > themselves, > > > I think enabling FILE reference URLs to be signed so that they are > > > optimized for sharing with other services will help Iceberg's FILE > > > integration plug readily into the existing multimodal ecosystem. > > > > > > > > > While work is still underway on the FILE type discussions to finalize > the > > > shape of the Iceberg type, I'd like to start the discussion on what > access > > > enhancements would need to be added to the Iceberg REST Catalog spec > and > > > its supporting SDKs. I've put together a spec PR based on recent > catalog > > > community discussion [4] on pre-signed urls, along with a draft PR that > > > POCs the integration on the client side [5]. > > > > > > > > > I'd love to get the community's thoughts on whether client initiated > > > pre-signing + sharing URLs is an additional mode we want to introduce > to > > > enable FILE access sharing as we look to integrate the Iceberg client > into > > > the multimodal ecosystem. > > > > > > > > > I'd also be interested to hear what other consumer patterns people are > > > running into beyond integrating with an inference service, which is the > > > case I've been designing against. > > > > > > > > > Sung > > > > > > > > > [1] https://github.com/apache/iceberg/pull/17457 > > > > > > [2] https://s.apache.org/iceberg-fileref > > > > > > [3] > https://github.com/apache/iceberg/pull/17332#discussion_r3992139348 > > > > > > [4] https://github.com/apache/iceberg/pull/18080 > > > > > > [5] https://github.com/apache/iceberg/pull/18110 > > > > > >
