dennishuo commented on PR #77: URL: https://github.com/apache/polaris/pull/77#issuecomment-2316594999
Continuing discussion from https://github.com/apache/polaris/pull/208#issuecomment-2313973645 There are two use cases to consider: 1. How to make the Polaris server itself use APPLICATION_DEFAULT credentials when reading/writing metadata files itself 2. How to vend out credentials to external engines that don't go through the currently-supported subscoping flows It seems the current state of this PR would only provide a way to do (1), by allowing catalog-creators to set per-catalog config values dictating for Polaris to use APPLICATION_DEFAULT behavior when reading/writing files itself. However, this ability poses a problem for situations where the set of admins who run the Polaris server are different from the set of admins who interact with the Polaris server to create catalogs. For this scenario, it's preferable to set `SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION=true` at the *server-config* level to make Polaris use APPLICATION_DEFAULT semantics in its local server environment. For case (2), I don't think there's yet a proposed solution. The APPLICATION_DEFAULT concept itself is probably not sufficiently expressive for this, because by nature APPLICATION_DEFAULT hides a bunch of "convenience" fallthroughs for trying to look for credentials in the local environment, which might include standard credential files (e.g. ~/.awscredentials), environment variables, or local cloud VM "metadata servers" (e.g. http://169.254.169.254). Not all of these are created equal for suitability for credential-vending, if at all. The most plausible use case would be to have a flow that allows simply handing out VM instance metadata-based tokens for credential-vending: - https://cloud.google.com/compute/docs/access/authenticate-workloads#applications - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-metadata-security-credentials.html - https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/how-to-use-vm-token#get-a-token-using-http I believe these are all designed to be "short-lived" credentials where security isn't compromised by handing them out, but they may lack the kinds of "downscoping" semantics normally needed in more advanced Polaris deployments. We could explore an option where these metadata-server-based tokens are returned for credential-vending purposes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
