GitHub user FANNG1 added a comment to the discussion: [DISCUSS] Long-term architecture for Lance support in Gravitino
Thanks @yuqi1129, this is a very good point. I agree that whether Gravitino should support non-Gravitino namespace backends is the key decision here. I do not have a strong preference yet, and would like to hear more feedback from the community. >From my perspective, the main benefit of supporting `path` and `rest` backends >is compatibility with existing Lance datasets and deployments. If we support `path`, users can expose existing Lance datasets through Lance REST more easily. If we support `rest`, Gravitino Lance REST can integrate with an existing Lance REST namespace deployment. However, from a platform/governance perspective, if a platform chooses to use a managed REST namespace, we probably should not allow or encourage users to access the same datasets through the path namespace. Path namespace requires users or engines to manage storage options, AK/SK, and storage-level configuration directly, which is hard to govern consistently. So I think this is exactly the major tradeoff we need to align on first: whether Lance REST in Gravitino should be only a Gravitino-managed catalog endpoint, or also a gateway for existing path/rest Lance namespaces. GitHub link: https://github.com/apache/gravitino/discussions/11295#discussioncomment-17103083 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
