dennishuo commented on PR #3699: URL: https://github.com/apache/polaris/pull/3699#issuecomment-4393945081
@tokoko +1 to bundling up SPI changes for a vote. For the locations, I'm okay with refactoring a more advanced way to set different storage actions for different sets of locations. Keeping the muscle under the hood for policy-scoping logic to handle different sets of actions for different locations will be important to avoid that door inadvertently closing with other architectural decisions that start assuming a uniform action-set, even if we failed to properly capitalize on it so far. The original use case in mind was in the case of cloned tables or other tables that originate from some legacy dataset in an "old" location that is no longer the new write-location, it's important to keep the legacy location "read-only" and only provide write access to the new location. It's a layer of protection against misbehaving client-side engines that may unexpectedly delete things from the legacy dataset location, for example. In general with the future of better intent-conveyance in the APIs (e.g. loadTable for a subsequent write vs just for a SELECT), we'd also want to better distinguish whether to include write permissions at all. Some finer-grained operations, such as things that only want to compact a snapshot might only need write access to the metadata/ directory but not the data/ directory. With better specification of intent in the API spec, there's some discussion of providing finer-grained privileges under TABLE_WRITE_PROPERTIES that distinguish between data-writes vs metadata-writes, and even within metadata writes, distinguishing between some sensitive ones like `setLocation` from more innocuous ones like snapshot tags. In some cases you might even want to allow DELETE but not READ on a subdirectory. For example, you might run a shared snapshot-expiry maintenance engine on a highly sensitive dataset. It would need read + write on `metadata/` files but you might even only want to give it DELETE on `data/`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
