I intended to write "marking it deprecated", but... fat-fingers :)
The token endpoint is deprecated, but existing Iceberg clients rely on its existence. Anyone on a release older than a few months will be unable to authenticate to Polaris without that endpoint. I think maintaining compatibility with those clients is important to make the service usable to as much of the Iceberg community as we can. Mike On Tue, Sep 24, 2024 at 7:31 AM Dmitri Bourlatchkov <dmitri.bourlatch...@dremio.com.invalid> wrote: > Sorry, I'm not sure I follow the deprecation argument. > > The token endpoint is already deprecated in the Iceberg REST Catalog spec. > > Polaris cannot "make" it deprecated as the spec is owned by Iceberg. > > Polaris essentially implements a deprecated API. > > Why would we want to implement a deprecated (and marked for removal) API in > the first release of a fresh project? > > Thanks, > Dmitri. > > On Tue, Sep 24, 2024 at 10:22 AM Michael Collado <collado.m...@gmail.com> > wrote: > > > Agree with Yufei 100%. Making it as deprecated, as the Iceberg spec does, > > makes total sense. We should remove it when Iceberg 2.0 is released. > > > > Mike > > > > On Tue, Sep 24, 2024 at 4:46 AM Jean-Baptiste Onofré <j...@nanthrax.net> > > wrote: > > > > > Hi Yufei > > > > > > Yes agree. > > > > > > Regards > > > JB > > > > > > Le mar. 24 sept. 2024 à 01:40, Yufei Gu <flyrain...@gmail.com> a > écrit : > > > > > > > For the token endpoint, I recommend we stay consistent with the > Iceberg > > > > REST spec by marking it as deprecated and aligning its removal with > the > > > > same timeline as outlined in the Iceberg spec. Here are a few key > > points > > > to > > > > consider: > > > > > > > > 1. Since the endpoint is marked as deprecated, and most production > > > > environments already rely on third-party token endpoints, the > security > > > risk > > > > remains minimal. > > > > 2. Per Iceberg REST spec, this endpoint is "Deprecated since Iceberg > > > > (Java) 1.6.0. The endpoint and related types will be removed from > this > > > spec > > > > in Iceberg (Java) 2.0.", Iceberg(Java) 2.0 is still a long way to go. > > > There > > > > would be compatibility issues for dev and test raised between now and > > 2.0 > > > > if Polaris doesn't have it. > > > > 3. Support for third-party token endpoints was introduced about seven > > > > months ago in both Java and Python, which isn't too far back, as > > Michael > > > > pointed out. > > > > > > > > WDYT? > > > > > > > > Yufei > > > > > > > > > > > > On Mon, Sep 23, 2024 at 12:27 PM Dmitri Bourlatchkov > > > > <dmitri.bourlatch...@dremio.com.invalid> wrote: > > > > > > > > > > If we remove the endpoint in Polaris, clients prior to that > release > > > > will > > > > > have no mechanism for generating a token. > > > > > > > > > > Missed to address this in my previous reply :) > > > > > > > > > > Existing Java REST Catalog clients (based on Iceberg's impl.) will > be > > > > able > > > > > to use bearer tokens and the Client Credentials OAuth2 Flow IIRC. > > > > > > > > > > Cheers, > > > > > Dmitri. > > > > > > > > > > On Mon, Sep 23, 2024 at 3:23 PM Dmitri Bourlatchkov < > > > > > dmitri.bourlatch...@dremio.com> wrote: > > > > > > > > > > > > If we remove the endpoint in Polaris, clients prior to that > > release > > > > > will > > > > > > have no mechanism for generating a token. > > > > > > > > > > > > The basic problem is that producing an auth token from the /token > > > > > endpoint > > > > > > in Polaris makes it assume the role of the Authorization Server > > > > according > > > > > > to the OAth2 RFC [2]. > > > > > > > > > > > > Moreover, since Polaris is mainly a Resource Owner, assuming the > > > > > > Authorization Server role goes against the grain of the OAuth2 > > design > > > > > [3]. > > > > > > > > > > > > We can certainly go into more details on this, but maybe it > should > > > be a > > > > > > separate discussion thread. > > > > > > > > > > > > My point about the first release is that such a concern already > > > exists > > > > > and > > > > > > has been discussed in the Iceberg community. I think we ought to > > > > consider > > > > > > it. > > > > > > > > > > > > Cheers, > > > > > > Dmitri. > > > > > > > > > > > > [2] https://www.rfc-editor.org/rfc/rfc6749 > > > > > > [3] > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1Xi5MRk8WdBWFC3N_eSmVcrLhk3yu5nJ9x_wC0ec6kVQ/edit#heading=h.2xju2266i6df > > > > > > > > > > > > > > > > > > On Mon, Sep 23, 2024 at 3:06 PM Michael Collado < > > > > collado.m...@gmail.com> > > > > > > wrote: > > > > > > > > > > > >> +1 on Russell’s comment. > > > > > >> > > > > > >> Re: the OAuth endpoint, it seems to me that compatibility with > > > Iceberg > > > > > >> clients needs to be considered. I think that prior to Iceberg > 1.5 > > or > > > > so, > > > > > >> there was not support for an external oauth tokens endpoint. If > we > > > > > remove > > > > > >> the endpoint in Polaris, clients prior to that release will have > > no > > > > > >> mechanism for generating a token. > > > > > >> > > > > > >> How far back do we want to maintain compatibility for Iceberg > > > clients? > > > > > >> IMO, > > > > > >> Iceberg 1.5 isn’t that old. > > > > > >> > > > > > >> Mike > > > > > >> > > > > > >> On Mon, Sep 23, 2024 at 11:54 AM Russell Spitzer < > > > > > >> russell.spit...@gmail.com> > > > > > >> wrote: > > > > > >> > > > > > >> > My only minor feedback is I'd prefer we do a first release as > > > 1.0. I > > > > > >> think > > > > > >> > there is an allergy to 0.1 software in production so I'd > rather > > we > > > > > just > > > > > >> > start at 1. > > > > > >> > > > > > > >> > On Mon, Sep 23, 2024 at 1:50 PM Jean-Baptiste Onofré < > > > > j...@nanthrax.net > > > > > > > > > > > >> > wrote: > > > > > >> > > > > > > >> > > Hi Dmitri > > > > > >> > > > > > > > >> > > It makes sense to me. > > > > > >> > > > > > > > >> > > Regards > > > > > >> > > JB > > > > > >> > > > > > > > >> > > Le lun. 23 sept. 2024 à 20:01, Dmitri Bourlatchkov > > > > > >> > > <dmitri.bourlatch...@dremio.com.invalid> a écrit : > > > > > >> > > > > > > > >> > > > From my POV, I'd propose to resolve the OAuth token > endpoint > > > > > concern > > > > > >> > [1] > > > > > >> > > > before the initial release. > > > > > >> > > > > > > > > >> > > > I guess it might be a rather big refactoring, but this > issue > > > is > > > > > >> already > > > > > >> > > > generally accepted as a security concern in the Iceberg > > > > community, > > > > > >> so I > > > > > >> > > > think it would be preferable to resolve it before the > first > > > > > release. > > > > > >> > > WDYT? > > > > > >> > > > > > > > > >> > > > Thanks, > > > > > >> > > > Dmitri. > > > > > >> > > > > > > > > >> > > > [1] https://github.com/apache/polaris/issues/12 > > > > > >> > > > > > > > > >> > > > On Mon, Sep 23, 2024 at 1:22 PM Jean-Baptiste Onofré < > > > > > >> j...@nanthrax.net> > > > > > >> > > > wrote: > > > > > >> > > > > > > > > >> > > > > Hi folks, > > > > > >> > > > > > > > > > >> > > > > As we know from experience, that the first release needs > > > some > > > > > >> careful > > > > > >> > > > > preparation steps, I would like to propose aiming for > the > > > > Apache > > > > > >> > > > > Polaris release by the end of October (after CoC NA). > > > > > >> > > > > > > > > > >> > > > > I propose to start from 0.1-incubating (currently we are > > > > > building > > > > > >> > > > > 999-SNAPSHOT :) ). > > > > > >> > > > > I already created 0.1 milestone on GitHub. We can rename > > it > > > if > > > > > >> > needed. > > > > > >> > > > > > > > > > >> > > > > The preparation steps would be: > > > > > >> > > > > - do the triage on the GitHub issues, assigning issues > to > > > the > > > > > 0.1 > > > > > >> > > > milestone > > > > > >> > > > > - check the legal (LICENSE, NOTICE, DISCLAIMER, KEYS, > > etc). > > > I > > > > > will > > > > > >> > > > > start some new checks/updates on that (as I have some > > > > experience > > > > > >> :) > > > > > >> > ). > > > > > >> > > > > - check distribution and artifact publication ( > > > > dist.apache.org, > > > > > >> > > > > repository.apache.org, ...) > > > > > >> > > > > > > > > > >> > > > > I have several PRs in preparation, including the release > > > > > >> preparation > > > > > >> > > > > related PRs. > > > > > >> > > > > > > > > > >> > > > > As reminder about the process > > > > > >> > > > > ( > > https://incubator.apache.org/guides/releasemanagement.html > > > ), > > > > > we > > > > > >> > need > > > > > >> > > > > to do release "internally" to the podling , and then > start > > > > > "again" > > > > > >> > the > > > > > >> > > > > vote on the incubator general mailing list. So it's a > > > "longer" > > > > > >> > process > > > > > >> > > > > comparing to a TLP release. > > > > > >> > > > > > > > > > >> > > > > What do you think ? > > > > > >> > > > > > > > > > >> > > > > Thanks, > > > > > >> > > > > Regards > > > > > >> > > > > JB > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > >
