yuqi1129 commented on PR #5244: URL: https://github.com/apache/gravitino/pull/5244#issuecomment-2443442697
> I think you should think if user implement a custom FS with FS provider, can he integrate into Gravitino server Hadoop catalog, Java and Python gvfs without code change in Gravitino? Take this as a principle, if not, please make it work. In most cases, the current implementation can satisfy users' needs. They can use the `gravitino.bypass` mechanism to pass to keys with prefixes `gravitino.bypass` to the filesystem providers, the file system provider then will remove the prefix and get the real keys. or just directly pass the properties. For example, if we use the following key/value pair as the properties when creating a Hadoop catalog, ``` k1=value1 gravitino.bypass.k2=value2 .... ``` Then, we will pass these configurations on to the file system provider. It's quite flexible now and it depends on the users that custom a new file system provider what a property key stands for. key `k1` may stand for a core configuration or just a useless one. ``` k1=value1 k2=value2 ``` The following three keys are identical and users can use any key that contains endpoint information to the file system provider and then assign it to the key `fs.s3a.endpoint` ``` fs.s3a.endpoint = xxxxx s3-endpoint=xxxx gravitino.bypass.fs.s3a.endpoint = xxxx ``` I'm not very sure whether the kinds of property conversion are acceptable to you. If we need to specify each required property explicitly, like the current S3 implementation, users need to use `s3-endpoint` in the Gravitino server and `s3.endpoint` in the GVFS Java client to pass endpoint information, I believe we need to add a new interface method like `requireConfigurationKey()` to tell users what the properties they required to pass, I'm hesitant to do so as it seems to make thing more complex and not very extensive enough. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
