Hi Dennis, Looking at the existing config ALLOW_EXTERNAL_TABLE_LOCATION, how does this fit into the model you've outlined? It does not support storing tables at UUID locations. The new config added in the linked PR is analogous to this existing config.
If a user wants to store tables at random(ized) locations, I would instead point to DEFAULT_LOCATION_OBJECT_STORAGE_PREFIX_ENABLED as an option. --EM On Tue, Aug 26, 2025 at 10:49 AM Dennis Huo <[email protected]> wrote: > It seems we're very loosely mixing/interchanging three distinct concepts > both here and in the PR, and we need to make sure to clarify which topic is > applicable in the code being added and in the bug report. > > Reposting from my PR comment: > > The original design was always to still securely support a situation where > people might want to remap all their namespaces/tables into UUID paths, as > long as they reside under the ALLOWED_LOCATIONS defined within the > StorageConfig. > > In this PR we need to be very precise about what we mean by "custom > location". As I understand it, "custom" just means it doesn't follow the > "default" where the pathname is equal to the namespace name hierarchically: > ns1.ns2 -> s3://basebucket/basepath1/ns1/ns2/ > StorageConfig.defaultBaseLocation = s3://basebucket/basepath1 > StorageConfig.allowedLocations= [s3://basebucket/basepath1, > s3://basebucket/basepath2] > > If we allow a "custom location", a user might want: > ns1.ns2 -> s3://basebucket/basepath1/6f8af834t/ > StorageConfig.defaultBaseLocation = s3://basebucket/basepath1 > StorageConfig.allowedLocations= [s3://basebucket/basepath1, > s3://basebucket/basepath2] > > In such a case we must still validate that the "custom" location > s3://basebucket/basepath1/6f8af834t/ resides under *one of* the allowed > locations in the StorageConfig, and by default we'd still also perform a > check to prevent overlaps. > > So there's three completely distinct concepts: > > 1. "Custom paths" -> Use supplied paths that might be uuids, rather than > "default" name-friendly paths. Disabling this is fairly commonplace, > and is > more a *stylistic* choice by the user > 2. "Overlapping paths check" -> Whether paths are custom or follow > default name-friendly paths, we perform overlap checks so that RBAC can > "correctly" uniquely identify a storage prefix during > credential-vending. > In theory, if *all* paths only use default name-based paths, we don't > need overlap checks, but since "custom paths" allows someone to already > remap some other namespace to the path before creating a new namespace > using "default name-based paths", in general the overlap check may > still be > needed. Catalog admins may disable this for performance if they have > full > control over path specification - this is a *declarative* security > option, where a catalog admin declares whether paths are fully under > trusted control > 3. "Allowed locations check" -> No matter what mix of custom/default > locations is used for all namespaces and tables, the primary invariant > is > that all credential vending *must* only be under one or more of the > declared ALLOWED_LOCATIONS here. Credential vending is used > *internally* as > well for Polaris to access metadata json, so this also applies for > CreateTable even if the caller isn't asking for vended-credentials > externally. This option should *not* generally be configurable for > disablement. If a catalog admin wishes for a new path to be allowed in a > catalog, they should update the StorageConfig to add that > ALLOWED_LOCATION. > > > On Mon, Aug 25, 2025 at 11:18 PM Jean-Baptiste Onofré <[email protected]> > wrote: > > > Hi Eric > > > > Thanks for the heads up and the fix on namespaces. > > > > I agree that it should be also fixed for tables, for consistency. And > > I think we can consider this as a "bug". > > > > I will test on both namespaces and tables, also evaluating the > > "impacts" for users. > > > > Regards > > JB > > > > On Tue, Aug 26, 2025 at 1:13 AM Eric Maynard <[email protected]> > > wrote: > > > > > > Hi team, > > > > > > I recently filed PR#2422 <https://github.com/apache/polaris/pull/2422> > > and > > > there was an ask there about creating a mailing list thread for > > discussion. > > > > > > The tl;dr of this change is that Polaris currently allows the creation > of > > > namespaces & tables outside of a catalog allowed-locations. The PR > fixes > > > this behavior for namespaces, which in turn should also fix at least > some > > > scenarios where tables can be created outside of allows-locations as > > well. > > > There will be some followup work to fix the behavior for tables more > > > generally, but the PR is scoped just to namespace locations. > > > > > > This does represent a small behavior change OOTB, as namespaces that > were > > > previously OK to create in a given (potentially unsafe) location will > now > > > not be able to be created. > > > > > > If you're interested in this change or the related investigation please > > > feel free to comment here, on the PR, or on the issue linked to the PR. > > > > > > Cheers, > > > > > > Eric M. > > >
