+1 to the above. I think maintaining backwards compatibility is paramount. I would actually like to see us somewhat couple our release cycle with Iceberg's as the spec lives there and Polaris's clients will be adhering to that spec. When Iceberg goes to 2.0, perhaps Polaris should go to 2.0. I would be in favor of removing this endpoint at that time when it's also no longer available to clients.
I understand that it's a little odd that we are implementing a deprecated API, but to me this deprecation implies clients shouldn't use it, not that implementations must not provide it. --EM On Tue, Sep 24, 2024 at 12:59 PM Michael Collado <collado.m...@gmail.com> wrote: > I intended to write "marking it deprecated", but... fat-fingers :) > > The token endpoint is deprecated, but existing Iceberg clients rely on its > existence. Anyone on a release older than a few months will be unable to > authenticate to Polaris without that endpoint. I think maintaining > compatibility with those clients is important to make the service usable to > as much of the Iceberg community as we can. > > Mike > > On Tue, Sep 24, 2024 at 7:31 AM Dmitri Bourlatchkov > <dmitri.bourlatch...@dremio.com.invalid> wrote: > > > Sorry, I'm not sure I follow the deprecation argument. > > > > The token endpoint is already deprecated in the Iceberg REST Catalog > spec. > > > > Polaris cannot "make" it deprecated as the spec is owned by Iceberg. > > > > Polaris essentially implements a deprecated API. > > > > Why would we want to implement a deprecated (and marked for removal) API > in > > the first release of a fresh project? > > > > Thanks, > > Dmitri. > > > > On Tue, Sep 24, 2024 at 10:22 AM Michael Collado <collado.m...@gmail.com > > > > wrote: > > > > > Agree with Yufei 100%. Making it as deprecated, as the Iceberg spec > does, > > > makes total sense. We should remove it when Iceberg 2.0 is released. > > > > > > Mike > > > > > > On Tue, Sep 24, 2024 at 4:46 AM Jean-Baptiste Onofré <j...@nanthrax.net> > > > wrote: > > > > > > > Hi Yufei > > > > > > > > Yes agree. > > > > > > > > Regards > > > > JB > > > > > > > > Le mar. 24 sept. 2024 à 01:40, Yufei Gu <flyrain...@gmail.com> a > > écrit : > > > > > > > > > For the token endpoint, I recommend we stay consistent with the > > Iceberg > > > > > REST spec by marking it as deprecated and aligning its removal with > > the > > > > > same timeline as outlined in the Iceberg spec. Here are a few key > > > points > > > > to > > > > > consider: > > > > > > > > > > 1. Since the endpoint is marked as deprecated, and most production > > > > > environments already rely on third-party token endpoints, the > > security > > > > risk > > > > > remains minimal. > > > > > 2. Per Iceberg REST spec, this endpoint is "Deprecated since > Iceberg > > > > > (Java) 1.6.0. The endpoint and related types will be removed from > > this > > > > spec > > > > > in Iceberg (Java) 2.0.", Iceberg(Java) 2.0 is still a long way to > go. > > > > There > > > > > would be compatibility issues for dev and test raised between now > and > > > 2.0 > > > > > if Polaris doesn't have it. > > > > > 3. Support for third-party token endpoints was introduced about > seven > > > > > months ago in both Java and Python, which isn't too far back, as > > > Michael > > > > > pointed out. > > > > > > > > > > WDYT? > > > > > > > > > > Yufei > > > > > > > > > > > > > > > On Mon, Sep 23, 2024 at 12:27 PM Dmitri Bourlatchkov > > > > > <dmitri.bourlatch...@dremio.com.invalid> wrote: > > > > > > > > > > > > If we remove the endpoint in Polaris, clients prior to that > > release > > > > > will > > > > > > have no mechanism for generating a token. > > > > > > > > > > > > Missed to address this in my previous reply :) > > > > > > > > > > > > Existing Java REST Catalog clients (based on Iceberg's impl.) > will > > be > > > > > able > > > > > > to use bearer tokens and the Client Credentials OAuth2 Flow IIRC. > > > > > > > > > > > > Cheers, > > > > > > Dmitri. > > > > > > > > > > > > On Mon, Sep 23, 2024 at 3:23 PM Dmitri Bourlatchkov < > > > > > > dmitri.bourlatch...@dremio.com> wrote: > > > > > > > > > > > > > > If we remove the endpoint in Polaris, clients prior to that > > > release > > > > > > will > > > > > > > have no mechanism for generating a token. > > > > > > > > > > > > > > The basic problem is that producing an auth token from the > /token > > > > > > endpoint > > > > > > > in Polaris makes it assume the role of the Authorization Server > > > > > according > > > > > > > to the OAth2 RFC [2]. > > > > > > > > > > > > > > Moreover, since Polaris is mainly a Resource Owner, assuming > the > > > > > > > Authorization Server role goes against the grain of the OAuth2 > > > design > > > > > > [3]. > > > > > > > > > > > > > > We can certainly go into more details on this, but maybe it > > should > > > > be a > > > > > > > separate discussion thread. > > > > > > > > > > > > > > My point about the first release is that such a concern already > > > > exists > > > > > > and > > > > > > > has been discussed in the Iceberg community. I think we ought > to > > > > > consider > > > > > > > it. > > > > > > > > > > > > > > Cheers, > > > > > > > Dmitri. > > > > > > > > > > > > > > [2] https://www.rfc-editor.org/rfc/rfc6749 > > > > > > > [3] > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1Xi5MRk8WdBWFC3N_eSmVcrLhk3yu5nJ9x_wC0ec6kVQ/edit#heading=h.2xju2266i6df > > > > > > > > > > > > > > > > > > > > > On Mon, Sep 23, 2024 at 3:06 PM Michael Collado < > > > > > collado.m...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > >> +1 on Russell’s comment. > > > > > > >> > > > > > > >> Re: the OAuth endpoint, it seems to me that compatibility with > > > > Iceberg > > > > > > >> clients needs to be considered. I think that prior to Iceberg > > 1.5 > > > or > > > > > so, > > > > > > >> there was not support for an external oauth tokens endpoint. > If > > we > > > > > > remove > > > > > > >> the endpoint in Polaris, clients prior to that release will > have > > > no > > > > > > >> mechanism for generating a token. > > > > > > >> > > > > > > >> How far back do we want to maintain compatibility for Iceberg > > > > clients? > > > > > > >> IMO, > > > > > > >> Iceberg 1.5 isn’t that old. > > > > > > >> > > > > > > >> Mike > > > > > > >> > > > > > > >> On Mon, Sep 23, 2024 at 11:54 AM Russell Spitzer < > > > > > > >> russell.spit...@gmail.com> > > > > > > >> wrote: > > > > > > >> > > > > > > >> > My only minor feedback is I'd prefer we do a first release > as > > > > 1.0. I > > > > > > >> think > > > > > > >> > there is an allergy to 0.1 software in production so I'd > > rather > > > we > > > > > > just > > > > > > >> > start at 1. > > > > > > >> > > > > > > > >> > On Mon, Sep 23, 2024 at 1:50 PM Jean-Baptiste Onofré < > > > > > j...@nanthrax.net > > > > > > > > > > > > > >> > wrote: > > > > > > >> > > > > > > > >> > > Hi Dmitri > > > > > > >> > > > > > > > > >> > > It makes sense to me. > > > > > > >> > > > > > > > > >> > > Regards > > > > > > >> > > JB > > > > > > >> > > > > > > > > >> > > Le lun. 23 sept. 2024 à 20:01, Dmitri Bourlatchkov > > > > > > >> > > <dmitri.bourlatch...@dremio.com.invalid> a écrit : > > > > > > >> > > > > > > > > >> > > > From my POV, I'd propose to resolve the OAuth token > > endpoint > > > > > > concern > > > > > > >> > [1] > > > > > > >> > > > before the initial release. > > > > > > >> > > > > > > > > > >> > > > I guess it might be a rather big refactoring, but this > > issue > > > > is > > > > > > >> already > > > > > > >> > > > generally accepted as a security concern in the Iceberg > > > > > community, > > > > > > >> so I > > > > > > >> > > > think it would be preferable to resolve it before the > > first > > > > > > release. > > > > > > >> > > WDYT? > > > > > > >> > > > > > > > > > >> > > > Thanks, > > > > > > >> > > > Dmitri. > > > > > > >> > > > > > > > > > >> > > > [1] https://github.com/apache/polaris/issues/12 > > > > > > >> > > > > > > > > > >> > > > On Mon, Sep 23, 2024 at 1:22 PM Jean-Baptiste Onofré < > > > > > > >> j...@nanthrax.net> > > > > > > >> > > > wrote: > > > > > > >> > > > > > > > > > >> > > > > Hi folks, > > > > > > >> > > > > > > > > > > >> > > > > As we know from experience, that the first release > needs > > > > some > > > > > > >> careful > > > > > > >> > > > > preparation steps, I would like to propose aiming for > > the > > > > > Apache > > > > > > >> > > > > Polaris release by the end of October (after CoC NA). > > > > > > >> > > > > > > > > > > >> > > > > I propose to start from 0.1-incubating (currently we > are > > > > > > building > > > > > > >> > > > > 999-SNAPSHOT :) ). > > > > > > >> > > > > I already created 0.1 milestone on GitHub. We can > rename > > > it > > > > if > > > > > > >> > needed. > > > > > > >> > > > > > > > > > > >> > > > > The preparation steps would be: > > > > > > >> > > > > - do the triage on the GitHub issues, assigning issues > > to > > > > the > > > > > > 0.1 > > > > > > >> > > > milestone > > > > > > >> > > > > - check the legal (LICENSE, NOTICE, DISCLAIMER, KEYS, > > > etc). > > > > I > > > > > > will > > > > > > >> > > > > start some new checks/updates on that (as I have some > > > > > experience > > > > > > >> :) > > > > > > >> > ). > > > > > > >> > > > > - check distribution and artifact publication ( > > > > > dist.apache.org, > > > > > > >> > > > > repository.apache.org, ...) > > > > > > >> > > > > > > > > > > >> > > > > I have several PRs in preparation, including the > release > > > > > > >> preparation > > > > > > >> > > > > related PRs. > > > > > > >> > > > > > > > > > > >> > > > > As reminder about the process > > > > > > >> > > > > ( > > > https://incubator.apache.org/guides/releasemanagement.html > > > > ), > > > > > > we > > > > > > >> > need > > > > > > >> > > > > to do release "internally" to the podling , and then > > start > > > > > > "again" > > > > > > >> > the > > > > > > >> > > > > vote on the incubator general mailing list. So it's a > > > > "longer" > > > > > > >> > process > > > > > > >> > > > > comparing to a TLP release. > > > > > > >> > > > > > > > > > > >> > > > > What do you think ? > > > > > > >> > > > > > > > > > > >> > > > > Thanks, > > > > > > >> > > > > Regards > > > > > > >> > > > > JB > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > >
