Hi everyone, I’d like to bring up a discussion around the expected behavior of the write.metadata.path config, which is used to set a custom metadata location for tables and views.
Currently, Polaris does *not allow* creating a view with write.metadata.path outside of the allowedLocations defined in the storageConfig. I’ve documented this behavior in #1320 <https://github.com/apache/polaris/pull/1320>, and I’d like to clarify what the intended behavior should be moving forward. Summary of Current Behavior 1. *View Creation with Custom Metadata Path* A view *cannot be created* with a write.metadata.path outside of allowedLocations, *unless*: - The parent namespace defines write.metadata.path, and - The view’s write.metadata.path is a *subpath* of the namespace-defined value. This logic is implemented here <https://github.com/apache/polaris/blob/989e99ae51f6cef0c093a86a4e51796f99a6894d/polaris-core/src/main/java/org/apache/polaris/core/storage/PolarisStorageConfigurationInfo.java#L184-L191>, where the namespace config overrides storage settings. *Example:* If the namespace sets write.metadata.path=file://baseLocation, then the view can be created at file://baseLocation/customLocation. 2. *Editing a View’s Metadata Path to a Sibling Location* Once the view is created, updating its write.metadata.path to a sibling path (e.g., file://baseLocation/customLocation2) is *not allowed*. *Reason:* The validation during update uses the *view’s own* write.metadata.path, not the namespace’s config. 3. *Editing to a Subpath Also Fails* Updating write.metadata.path to a *subpath* (e.g., from file://baseLocation/customLocation to file://baseLocation/customLocation/child) fails. *Reason:* This is rejected because after creation, the view’s write.metadata.path is stored in internalProperties rather than properties, so the previous config override no longer applies. Questions to Discuss: - What is the *intended behavior* of write.metadata.path? - The override logic <https://github.com/apache/polaris/blob/989e99ae51f6cef0c093a86a4e51796f99a6894d/polaris-core/src/main/java/org/apache/polaris/core/storage/PolarisStorageConfigurationInfo.java#L184-L191> doesn't appear to be covered by existing tests. I'm wondering about the original purpose behind it. Looking forward to your feedback! Best, Liam
