Hi all, I went ahead and asked the Iceberg community:
https://lists.apache.org/thread/5d1v400jbo1t6bp51xpdwkxv6nx418d4 Thanks, Alex On Thu, Jan 8, 2026 at 5:38 PM Alexandre Dutra <[email protected]> wrote: > > Hi Prashant, > > Apologies for the delayed response; I was myself among those who took > some recent time off :-) > > My assertion that this is not a public endpoint is based on its > absence from ResourcePaths [1], S3FileIOProperties [2], and > AwsClientProperties [3] – by contrast, see how other endpoints are > explicitly mentioned there. In fact, the signer endpoint's sole > reference within Iceberg is in S3V4RestSignerClient [4], suggesting it > is an implementation detail of that component, which is indifferent to > the URL's exact format. > > Although technically possible to configure this endpoint via the > catalog, I suspect this was not the original intention of the > feature's creators, who likely meant for it to be communicated in the > LoadTableResponse exclusively. Furthermore, the intent appears to have > been for the endpoint to be customizable and potentially vary per > table. The commit [5] introducing the feature clearly affirms: "the > default endpoint the signer connects to is v1/aws/s3/sign but can be > customized". > > As you've noted, the primary issue with the default endpoint is its > lack of path parameters for namespace, table, and (optionally) prefix, > making it practically unusable since the namespace and table cannot be > reliably inferred from the signing request. > > Before proceeding to a vote, how about we ask the Iceberg Dev ML to > confirm if customizing the signer endpoint, as we are planning to do > in Polaris, is an acceptable practice. Would this clarification step > be useful? > > Thanks, > Alex > > [1]: > https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/ResourcePaths.java > [2]: > https://github.com/apache/iceberg/blob/main/aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIOProperties.java > [3]: > https://github.com/apache/iceberg/blob/main/aws/src/main/java/org/apache/iceberg/aws/AwsClientProperties.java > [4]: > https://github.com/apache/iceberg/blob/main/aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java > [5]: > https://github.com/apache/iceberg/commit/80766723588985c4592ffb336a76eabc046d01a9 > > On Tue, Dec 23, 2025 at 12:39 AM Prashant Singh via dev > <[email protected]> wrote: > > > > Hi Alex, Hello all, > > > > *First, I want to thank you for all the hard work and the detailed response > > on this.* > > > > Regarding the technical approach, I’m not sure I fully follow the rationale > > for these not being public endpoints. What would stop a user from hitting > > Polaris directly via a curl command? > > > > I believe there might be some confusion here. While the Iceberg REST > > Catalog (IRC) can return the URL to connect to the endpoint directly, the > > specification [1] states this should be handled per the signer spec. We are > > essentially overriding those endpoints by introducing new ones. These can > > be configured orthogonally via catalog configurations (e.g., Spark confs), > > meaning there are ways to feed this to the client without the server > > returning them as part of loadTable. This allows the client to connect to > > the server directly to enable the remote signing flow; we just need the > > server to support the endpoint. > > > > > And lastly, we are probably going to leverage the prefix to pass the > > cryptographic payload, in which case, the endpoint template technically > > won't change between M1 and M2. > > > > If we can iron this out, it would be very helpful. I wonder if we should > > also encode the namespace/table inside the prefix as well. If we start with > > a prefix-based approach, and a user attempts to access a namespace or table > > by manually creating a prefix, Polaris can simply return a 403 if they lack > > the necessary permissions in M1. > > > > Finally, while I see some +1s have been mentioned, we have historically > > held formal votes for even minor Polaris API changes, such as updating > > status codes [2]. Since this proposal introduces entirely new Polaris APIs, > > I recommend we start a formal *VOTE thread* to establish general consensus > > and unblock future development. > > > > We should just be cognizant of the fact that many folks are OOO right now, > > so we should ensure the voting window is long enough to account for the > > holiday schedule. > > > > Best, > > > > Prashant > > > > [1] > > https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L3385 > > > > [2] https://lists.apache.org/thread/rv3sqno6orfyx7h3f2llb8lkq00zofg8 > > > > > > > > On Mon, Dec 15, 2025 at 10:08 AM Alexandre Dutra <[email protected]> wrote: > > > > > Hi Prashant, hi all, > > > > > > I wouldn't call the signer endpoint a public-facing API: it's only > > > surfaced in the table configuration and it's meant for programmatic > > > consumption by S3FileIO instances. > > > > > > Also, it's possible that servers might use different endpoints for > > > different clients; it's more of a generated, temporary endpoint, > > > specific to a given client and table. > > > > > > And lastly, we are probably going to leverage the prefix to pass the > > > cryptographic payload, in which case, the endpoint template > > > technically won't change between M1 and M2. > > > > > > Anyways, I wouldn't be too worried about this being a potential > > > blocker for future releases, since the feature is guarded by the beta > > > label and by production readiness checks. > > > > > > Thanks, > > > Alex > > > > > > On Sun, Dec 14, 2025 at 7:20 AM Robert Stupp <[email protected]> wrote: > > > > > > > > Hi all, > > > > > > > > It is awesome that this discussion finally gets traction! > > > > We do have users waiting eagerly for support of S3 remote request > > > > signing in Polaris. > > > > > > > > The proposal leverages existing public APIs/specs. For posterity: an > > > > Iceberg REST catalog announces the S3 remote request signing endpoint > > > > for a particular table in a "load table" response. That endpoint is > > > > not "static", it is chosen by the catalog and can be any URI, it does > > > > not need to be "under" the same service base path URI as the catalog. > > > > All this is defined by the Iceberg REST OpenAPI spec for quite some > > > > time now. > > > > > > > > The effort to get S3 remote signing into Apache Polaris started about > > > > 4 months ago [1] with a very detailed design doc including an > > > > implementation plan: the milestones M1 ("Simple, slow implementation > > > > with basic RBAC", "Not production ready – marked experimental") + M2 > > > > ("Fast implementation with efficient RBAC", "Production-ready"). > > > > > > > > It is great that we have explicit +1 [2] and lazy consensus on this > > > > proposal including the two-milestone approach. This commitment shows > > > > consistency and demonstrates that we are on the same page, which is > > > > great for the Polaris project! > > > > > > > > For me, the phased approach is pretty clear, is sound and allows > > > > smaller, iterative and concise changes to get into the code base. > > > > > > > > Adding everything that is required for M2 all at once is too much for > > > > a single change (PR). Some people raised concerns in the past about > > > > big-ish changes not rendering fine in GitHub UI. > > > > I prefer to do the cryptographic changes being added separately from > > > > the S3 remote signing functionality (this is implied by the proposal). > > > > This cryptographic and security relevant functionality, a prerequisite > > > > for M2, is potentially not a "very quick" task. > > > > > > > > If users explicitly enable a non-production ready feature, they do > > > > this intentionally. The M1 production-readiness check [3] prevents > > > > users from "accidentally" enabling M1. Users who skip/disable > > > > production-readiness checks in production deployments are, sorry to > > > > say, intentionally shooting themselves into the foot. Asking for > > > > "support" of non-production-ready features is quite an ask. > > > > > > > > M1 allows users to try the feature in a test/development environment, > > > > allowing us to gather feedback. An "experimental" feature implies that > > > > things can and likely will change. No change to any experimental > > > > functionality, especially to one that is guarded by a > > > > production-readiness check, can therefore be a "blocker". > > > > > > > > Robert > > > > > > > > [1] > > > https://docs.google.com/document/d/1ygdia7u4bUHUt6n8XhZo48aKoIyyrCvKqan3XP25iB8/edit?tab=t.0#heading=h.76hegagb5gud > > > > [2] > > > https://docs.google.com/document/d/1ygdia7u4bUHUt6n8XhZo48aKoIyyrCvKqan3XP25iB8/edit?disco=AAABpZ-ZQwo > > > > [3] > > > https://github.com/adutra/polaris/blob/2af510459f72e40305ae1c0d0b9de73582912c8f/runtime/service/src/main/java/org/apache/polaris/service/config/ProductionReadinessChecks.java#L394 > > > > > > > > On Sat, Dec 13, 2025 at 6:49 PM Prashant Singh via dev > > > > <[email protected]> wrote: > > > > > > > > > > Thank you Alex for all the hard work ! > > > > > I appreciate you breaking this down to Milestone 1 and MileStone 2 to > > > make > > > > > incremental progress [1] > > > > > > > > > > I wanted to share my concerns / considerations for the same : > > > > > 1. *Perf* : This will now require all the files (which can be easily > > > > > millions) to come to the catalog for getting a signed url and look up > > > > > entities / grants / table metadata. This concern is addressed partly > > > > > by > > > > > adding prod readiness check and feature flag for enable this feature > > > and we > > > > > actually plan to fully address this in MileStone 2 where we send a > > > crypto > > > > > signed metadata from Polaris as part of signed url (i.e table name / > > > table > > > > > metadata required / other auxiliary info) so that signing can be > > > > > lightweight as this will be decrypted in polaris and avoid polaris to > > > do > > > > > additional lookups. > > > > > 2. *Introducing new endpoints in Milestone 1 only to introduce yet > > > another > > > > > new ones in Milestone 2 shortly : *I am fairly concerned about this > > > > > and > > > > > mentioned the same here [2] since we are introducing a set of public > > > facing > > > > > API only to deprecate when M2 is achieved and we already have folks > > > who are > > > > > volunteering working on M2 [3], so why not just go when M2 is ready as > > > we > > > > > plan to deprecate endpoint in M1 in favour of M2. If we wanna merge M1 > > > > > changes to unblock M2 changes (given other folks are on the same page) > > > I > > > > > think that's reasonable, but IMHO it will become a *blocker* to the > > > > > next release unless M2 is completed. > > > > > > > > > > *Note*: These endpoints are custom to Polaris and not what signer spec > > > > > dictates, but it's fine since spec gives wiggle room for that > > > > > > > > > > As folks who have their own Polaris deployment this will be a > > > significant > > > > > burden as to support the endpoint introduced in M1, they will have > > > > > telemetry / alerting / capacity planning made around it and all go in > > > vain > > > > > when M2 comes *very* shortly (since we are actively working on it). I > > > > > understand APIs can be evolving and one should be prepared for it but > > > in > > > > > this scenario since we already know / plan that we want to do M2. > > > > > > > > > > I would recommend going with only 1 set of public Polaris APIs which > > > > > together as a community can stand behind for a reasonable amount of > > > time. > > > > > > > > > > Would love to know what others have to say in this context. > > > > > > > > > > [1] > > > https://github.com/apache/polaris/pull/2280#issuecomment-3487401124 > > > > > [2] > > > https://github.com/apache/polaris/pull/2280#issuecomment-3504202125 > > > > > [3] > > > https://github.com/apache/polaris/pull/2280#issuecomment-3553234640 > > > > > > > > > > > > > > > Best, > > > > > Prashant Singh > > > > > > > > > > On Sat, Dec 13, 2025 at 8:02 AM Artur Rakhmatulin < > > > > > [email protected]> wrote: > > > > > > > > > > > Hello everyone. Thanks Alex for reviving the discussion. > > > > > > I see this PR is struggling a bit with ongoing conflicts, and I'd > > > like > > > > > > to offer my help and share my thought on it. > > > > > > > > > > > > If we decide this feature should move forward, I suggest splitting > > > the > > > > > > PR into 3 smaller parts: > > > > > > > > > > > > **Introduce the API and scaffolding** > > > > > > Add a new s3-sign-service module with the required interfaces/DTOs, > > > but > > > > > > without wiring it into the build/runtime yet. This provides a clean > > > > > > contract for further work. > > > > > > > > > > > > Deliver: > > > > > > - api/s3-sign-service > > > > > > - specs > > > > > > - put s3-sign-service build under a feature flag > > > > > > > > > > > > **Add core changes (config + auth)** > > > > > > Deliver the polaris-core updates needed for storage configuration > > > > > > and > > > > > > authorization, keeping the feature fully opt-in (e.g., > > > > > > storage.s3.signing.enabled=false). No functional signing yet. > > > > > > > > > > > > Deliver: > > > > > > - polaris-core > > > > > > > > > > > > **Add the actual implementation** > > > > > > Provide the concrete S3 signing implementation, register it, and add > > > > > > integration/e2e tests. Enable it only when the feature flag is > > > turned on. > > > > > > > > > > > > Deliver: > > > > > > - runtime > > > > > > - and rest of the tests > > > > > > - weak a feature flag > > > > > > > > > > > > What do you think about splitting the delivery process this way? > > > > > > Do you have alternative suggestions, or do you see this feature > > > > > > being > > > > > > delivered more effectively as a single PR instead? > > > > > > > > > > > > I'd be glad to hear your thoughts. > > > > > > > > > > > > > > > > > > On 12/12/2025 21:20, Alexandre Dutra wrote: > > > > > > > Hi all, > > > > > > > > > > > > > > I'm reviving the discussion regarding remote S3 signing because > > > the PR > > > > > > > [1] is now more than 4 months old, and it's been quite a pain to > > > > > > > rebase it regularly. > > > > > > > > > > > > > > I would like to thank Prashant for his thorough review of the PR > > > > > > > so > > > > > > > far; his feedback did uncover a few issues around table locations > > > that > > > > > > > led to [2], but the resulting PR now aligns with Milestone 1 (M1). > > > And > > > > > > > by the way, Milestone 2 is already underway. As a reminder, in the > > > M1 > > > > > > > PR, remote signing is clearly labeled as beta and disabled by > > > default. > > > > > > > > > > > > > > What is the community's interest and appetite for this, and what > > > > > > > is > > > > > > > the desired timeline? Do we have any outstanding blockers? I know > > > the > > > > > > > PR is big, but maybe it could benefit from more reviews as well. > > > > > > > > > > > > > > Anyways, let me know what's the best way to move forward with > > > remote > > > > > > signing. > > > > > > > > > > > > > > Thanks, > > > > > > > Alex > > > > > > > > > > > > > > [1]: https://github.com/apache/polaris/pull/2280 > > > > > > > [2]: https://github.com/apache/polaris/pull/3226 > > > > > > > > > > > > > > On Tue, Aug 26, 2025 at 3:42 AM Alexandre Dutra <[email protected] > > > > > > > > > > wrote: > > > > > > >> Hi all, > > > > > > >> > > > > > > >> I'm starting a new thread on S3 remote signing to avoid hijacking > > > the > > > > > > >> existing one [1]. > > > > > > >> > > > > > > >> To summarize our current progress: we have a design document [2], > > > a > > > > > > >> Github issue [3] and an initial PR [4]. > > > > > > >> > > > > > > >> This initial PR establishes the foundation for the feature. In > > > that > > > > > > >> PR, remote signing is marked experimental, due to suboptimal > > > > > > >> authorization checks and potential performance bottlenecks. > > > However, a > > > > > > >> clear path for improvements in both areas has been identified. > > > > > > >> > > > > > > >> How should we proceed? Is the community in agreement with the > > > general > > > > > > >> implementation guidelines and the current PR? > > > > > > >> > > > > > > >> Thanks, > > > > > > >> Alex > > > > > > >> > > > > > > >> [1]: > > > https://lists.apache.org/thread/qvzwc3qxlfrk9vr7yfbx6zxfhz9lhlbc > > > > > > >> [2]: > > > > > > > > > https://docs.google.com/document/d/1ygdia7u4bUHUt6n8XhZo48aKoIyyrCvKqan3XP25iB8/edit?usp=sharing > > > > > > >> [3]: https://github.com/apache/polaris/issues/32 > > > > > > >> [4]: https://github.com/apache/polaris/pull/2280 > > > > > > > > >
