+1 to IMPLICIT. Note, I would consider the Hadoop/HDFS one to actually also correctly be following the conventions we described for IMPLICIT, because the Polaris service/system environment can indeed be configured to make the Hadoop FileSystem use whatever auth it wants, even Kerberos-based access to "real" traditional HDFS.
On Wed, Jul 2, 2025 at 3:28 PM Pooja Nilangekar <po...@apache.org> wrote: > Thanks Dimtri and Eric, for now I will update the PR with IMPLICIT. If > others send out suggestions later, I could change it. If not, we can > proceed with IMPLICIT. > > Thanks, > Pooja > > On 2025/07/02 22:21:50 Eric Maynard wrote: > > Yeah I think IMPLICIT seems reasonable -- we could start with that and > then > > expand to NONE if the need arises. > > > > On Wed, Jul 2, 2025 at 2:34 PM Dmitri Bourlatchkov <di...@apache.org> > wrote: > > > > > I'd be fine with supporting both NONE and IMPLICIT. > > > > > > I'd expect NONE to be executed as strictly no authentication in > requests to > > > external catalogs, though, even if the connector (inside Polaris) > allows > > > defaulting to environment or files, etc. > > > > > > If IMPLICIT is specified and the Polaris Server cannot reasonably > leverage > > > any pre-configured (at deployment time) auth mechanisms, then requests > > > should be denied on the Polaris side. > > > > > > As an example, IMPLICIT with AWS SDK is always allowed because the SDK > has > > > well-known file-based configuration / profiling mechanisms. > > > > > > I do not know enough about Hadoop, though. > > > > > > WDYT? > > > > > > Cheers, > > > Dmitri. > > > > > > On Wed, Jul 2, 2025 at 5:24 PM Eric Maynard <eric.w.mayn...@gmail.com> > > > wrote: > > > > > > > Yeah, maybe NONE is misleading and so UNMANAGED or IMPLICIT could be > > > > better. In some cases it's conceivable that there really is no > "auth" as > > > > such -- like with HADOOP -- and so I wonder if IMPLICIT > over-promises a > > > > bit? > > > > > > > > --EM > > > > > > > > On Wed, Jul 2, 2025 at 1:10 PM Dmitri Bourlatchkov <di...@apache.org > > > > > > wrote: > > > > > > > > > How about using the enum name IMPLICIT in this case? > > > > > > > > > > YAML comments will briefly mention runtime env. implications. > > > > Documentation > > > > > will (later) explain how it works in detail. > > > > > > > > > > From my POV, "NONE" means strictly no auth. > > > > > > > > > > Cheers, > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > On Wed, Jul 2, 2025 at 4:04 PM Eric Maynard < > eric.w.mayn...@gmail.com> > > > > > wrote: > > > > > > > > > > > > When the new NONE (or any proposed alternative name) is used > as the > > > > > > authentication type in an External Catalog, what kind of auth > flow > > > will > > > > > > actually happen in runtime? > > > > > > > > > > > > This question really gets to the core of what we are discussing. > From > > > > my > > > > > > perspective in implementing HADOOP, we can interpret NONE in two > > > ways: > > > > > > > > > > > > 1. Polaris does no auth whatsoever > > > > > > 2. The EXTERNAL catalog connection config does not describe any > kind > > > of > > > > > > auth > > > > > > > > > > > > My interpretation of NONE is (2). > > > > > > > > > > > > While it's true that Polaris doesn't explicitly do any kind of > auth > > > for > > > > > > Hadoop and relies on the fact that new Configuration() happens to > > > load > > > > > from > > > > > > some env vars, I do not believe that it's really accurate to say > we > > > are > > > > > in > > > > > > situation (1). Polaris may still be doing some auth, even if > it's not > > > > > > obvious from a quick pass over the code. > > > > > > > > > > > > Rather, NONE indicates that the ConnectionConfigInfo itself does > not > > > > > > contain any authentication credentials or mechanism. Consider > another > > > > > > example -- if the auth type is configured as OAUTH, that doesn't > mean > > > > > that > > > > > > the remote catalog isn't additionally using mTLS. It just means > that > > > > the > > > > > > ConnectionConfigInfo attached to the EXTERNAL catalog in Polaris > > > > contains > > > > > > OAUTH-related information. > > > > > > > > > > > > --EM > > > > > > > > > > > > > > > > > > > > >
