Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao closed pull request #5400: [#5081] improvement(docs): Add the document about cloud storage fileset. URL: https://github.com/apache/gravitino/pull/5400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324:
URL: https://github.com/apache/gravitino/pull/5324#discussion_r1824026254
##
docs/hadoop-catalog.md:
##
@@ -44,46 +44,43 @@ Apart from the above properties, to access fileset like
HDFS, S3, GCS, OSS or cu
S3 fileset
-| Configuration item | Description
| Default value |
Required | Since version|
-||---|-||--|
-| `filesystem-providers` | The file system providers to add. Set it to
`s3` if it's a s3 fileset or a comma separated string that contains `s3` like
`gs,s3` to support multiple kind of fileset including `s3`. | (none) |
Yes| 0.7.0-incubating |
-| `default-filesystem-provider` | The name default filesystem providers of
this Hadoop catalog if users do not specify the scheme in the URI. Default
value is `builtin-local`, for S3, if we set this value, we can omit the prefix
'oss://' in the location. | `builtin-local` | No |
0.7.0-incubating |
-| `s3-endpoint` | The endpoint of the AWS s3.
| (none) |
Yes if it's a s3 fileset. | 0.7.0-incubating |
-| `s3-access-key-id` | The access key of the AWS s3.
| (none) |
Yes if it's a s3 fileset. | 0.7.0-incubating |
-| `s3-secret-access-key` | The secret key of the AWS s3.
| (none) |
Yes if it's a s3 fileset. | 0.7.0-incubating |
-
-At the same time, you need to place the corresponding bundle jar
[gravitno-aws-bundle-{version}.jar](https://repo1.maven.org/maven2/org/apache/gravitino/aws-bundle/)
in the directory ${GRAVITINO_HOME}/catalogs/hadoop/libs.
+| Configuration item | Description
| Default value | Required | Since version
|
+||--|-|---|--|
+| `filesystem-providers` | The file system providers to add. Set it to
`s3` if it's a s3 fileset, or a comma separated string that contains `s3` like
`gs,s3` to support multiple kinds of fileset including `s3`.
| (none) | Yes |
0.7.0-incubating |
+| `default-filesystem-provider` | The name default filesystem providers of
this Hadoop catalog if users do not specify the scheme in the URI. Default
value is `builtin-local`, for S3, if we set this value, we can omit the prefix
's3a://' in the location. | `builtin-local` | No|
0.7.0-incubating |
+| `s3-endpoint` | The endpoint of the AWS s3.
| (none) | Yes if it's a s3 fileset. |
0.7.0-incubating |
+| `s3-access-key-id` | The access key of the AWS s3.
| (none) | Yes if it's a s3 fileset. |
0.7.0-incubating |
+| `s3-secret-access-key` | The secret key of the AWS s3.
| (none) | Yes if it's a s3 fileset. |
0.7.0-incubating |
+At the same time, you need to place the corresponding bundle jar
[gravitino-aws-bundle-{version}.jar](https://repo1.maven.org/m
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao merged PR #5400: URL: https://github.com/apache/gravitino/pull/5400 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao merged PR #5324: URL: https://github.com/apache/gravitino/pull/5324 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324:
URL: https://github.com/apache/gravitino/pull/5324#discussion_r1824058003
##
docs/hadoop-catalog.md:
##
@@ -44,46 +44,43 @@ Apart from the above properties, to access fileset like
HDFS, S3, GCS, OSS or cu
S3 fileset
-| Configuration item | Description
| Default value |
Required | Since version|
-||---|-||--|
-| `filesystem-providers` | The file system providers to add. Set it to
`s3` if it's a s3 fileset or a comma separated string that contains `s3` like
`gs,s3` to support multiple kind of fileset including `s3`. | (none) |
Yes| 0.7.0-incubating |
-| `default-filesystem-provider` | The name default filesystem providers of
this Hadoop catalog if users do not specify the scheme in the URI. Default
value is `builtin-local`, for S3, if we set this value, we can omit the prefix
'oss://' in the location. | `builtin-local` | No |
0.7.0-incubating |
-| `s3-endpoint` | The endpoint of the AWS s3.
| (none) |
Yes if it's a s3 fileset. | 0.7.0-incubating |
-| `s3-access-key-id` | The access key of the AWS s3.
| (none) |
Yes if it's a s3 fileset. | 0.7.0-incubating |
-| `s3-secret-access-key` | The secret key of the AWS s3.
| (none) |
Yes if it's a s3 fileset. | 0.7.0-incubating |
-
-At the same time, you need to place the corresponding bundle jar
[gravitno-aws-bundle-{version}.jar](https://repo1.maven.org/maven2/org/apache/gravitino/aws-bundle/)
in the directory ${GRAVITINO_HOME}/catalogs/hadoop/libs.
+| Configuration item | Description
| Default value | Required | Since version
|
+||--|-|---|--|
+| `filesystem-providers` | The file system providers to add. Set it to
`s3` if it's a s3 fileset, or a comma separated string that contains `s3` like
`gs,s3` to support multiple kinds of fileset including `s3`.
| (none) | Yes |
0.7.0-incubating |
+| `default-filesystem-provider` | The name default filesystem providers of
this Hadoop catalog if users do not specify the scheme in the URI. Default
value is `builtin-local`, for S3, if we set this value, we can omit the prefix
's3a://' in the location. | `builtin-local` | No|
0.7.0-incubating |
+| `s3-endpoint` | The endpoint of the AWS s3.
| (none) | Yes if it's a s3 fileset. |
0.7.0-incubating |
+| `s3-access-key-id` | The access key of the AWS s3.
| (none) | Yes if it's a s3 fileset. |
0.7.0-incubating |
+| `s3-secret-access-key` | The secret key of the AWS s3.
| (none) | Yes if it's a s3 fileset. |
0.7.0-incubating |
+At the same time, you need to place the corresponding bundle jar
[gravitino-aws-bundle-{version}.jar](https://repo1.maven.org/m
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324:
URL: https://github.com/apache/gravitino/pull/5324#discussion_r1824024139
##
docs/how-to-use-gvfs.md:
##
@@ -76,9 +123,21 @@ You can configure these properties in two ways:
conf.set("fs.gvfs.impl","org.apache.gravitino.filesystem.hadoop.GravitinoVirtualFileSystem");
conf.set("fs.gravitino.server.uri","http://localhost:8090";);
conf.set("fs.gravitino.client.metalake","test_metalake");
+
+// Optional. It's only for s3 catalog, for GCs and OSS catalog, you should
set the corresponding properties.
+conf.set("fs.gvfs.filesystem.providers", "s3");
+conf.set("s3-endpoint", "http://localhost:9000";);
+conf.set("s3-access-key-id", "minio");
+conf.set("s3-secret-access-key", "minio123");
+
Path filesetPath = new
Path("gvfs://fileset/test_catalog/test_schema/test_fileset_1");
FileSystem fs = filesetPath.getFileSystem(conf);
```
+
+:::note
+If you want to access the s3, gcs, oss or custom fileset through GVFS, apart
from the above properties, you need to place the corresponding bundle jar in
the Hadoop environment.
Review Comment:
"S3, GCS, OSS"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1824024744 ## docs/how-to-use-gvfs.md: ## @@ -102,6 +161,24 @@ You can configure these properties in two ways: fs.gravitino.client.metalake test_metalake + + Review Comment: "GCS" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1824022651 ## docs/how-to-use-gvfs.md: ## @@ -49,22 +49,69 @@ the path mapping and convert automatically. ### Configuration -| Configuration item| Description | Default value | Required| Since version | -|---|--|---|-|---| -| `fs.AbstractFileSystem.gvfs.impl` | The Gravitino Virtual File System abstract class, set it to `org.apache.gravitino.filesystem.hadoop.Gvfs`. | (none)| Yes | 0.5.0 | -| `fs.gvfs.impl`| The Gravitino Virtual File System implementation class, set it to `org.apache.gravitino.filesystem.hadoop.GravitinoVirtualFileSystem`. | (none)| Yes | 0.5.0 | -| `fs.gvfs.impl.disable.cache` | Disable the Gravitino Virtual File System cache in the Hadoop environment. If you need to proxy multi-user operations, please set this value to `true` and create a separate File System for each user. | `false` | No | 0.5.0 | -| `fs.gravitino.server.uri` | The Gravitino server URI which GVFS needs to load the fileset metadata. | (none)| Yes | 0.5.0 | -| `fs.gravitino.client.metalake`| The metalake to which the fileset belongs. | (none)| Yes | 0.5.0 | -| `fs.gravitino.client.authType`| The auth type to initialize the Gravitino client to use with the Gravitino Virtual File System. Currently only supports `simple`, `oauth2` and `kerberos` auth types. | `simple` | No | 0.5.0 | -| `fs.gravitino.client.oauth2.serverUri`| The auth server URI for the Gravitino client when using `oauth2` auth type with the Gravitino Virtual File System. | (none)| Yes if you use `oauth2` auth type | 0.5.0 | -| `fs.gravitino.client.oauth2.credential` | The auth credential for the Gravitino client when using `oauth2` auth type in the Gravitino Virtual File System. | (none)| Yes if you use `oauth2` auth type | 0.5.0 | -| `fs.gravitino.client.oauth2.path` | The auth server path for the Gravitino client when using `oauth2` auth type with the Gravitino Virtual File System. Please remove the first slash `/` from the path, for example `oauth/token`. | (none)| Yes if you use `oauth2` auth type | 0.5.0 | -| `fs.gravitino.client.oauth2.scope`| The auth scope for the Gravitino client when using `oauth2` auth type with the Gravitino Virtual File System. | (none)| Yes if you use `oauth2` auth type | 0.5.0 | -| `fs.gravitino.client.kerberos.principal` | The auth principal for the Gravitino client when using `kerberos` auth type with the Gravitino Virtual File System. | (none)| Yes if you use `kerberos` auth type | 0.5.1 | -| `fs.gravitino.client.kerberos.keytabFilePath` | The auth keytab file path for the Gravitino client when using `kerberos` auth type in the Gravitino Virtual File System. | (none)| No | 0.5.1 | -| `fs.gravitino.fileset.cache.maxCapacity`
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
yuqi1129 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1823903168 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
yuqi1129 commented on PR #5324: URL: https://github.com/apache/gravitino/pull/5324#issuecomment-2449197628 all comments have been resolved. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1823823344 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1823828075 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1823825884 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1823823795 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
yuqi1129 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1822313317 ## docs/how-to-use-gvfs.md: ## @@ -335,17 +382,24 @@ to recompile the native libraries like `libhdfs` and others, and completely repl ### Configuration -| Configuration item | Description | Default value | Required | Since version| -|--|--|---|---|--| -| `server_uri` | The Gravitino server uri, e.g. `http://localhost:8090`. | (none)| Yes | 0.6.0-incubating | -| `metalake_name` | The metalake name which the fileset belongs to. | (none)| Yes | 0.6.0-incubating | -| `cache_size` | The cache capacity of the Gravitino Virtual File System. | `20` | No| 0.6.0-incubating | -| `cache_expired_time` | The value of time that the cache expires after accessing in the Gravitino Virtual File System. The value is in `seconds`. | `3600`| No| 0.6.0-incubating | -| `auth_type` | The auth type to initialize the Gravitino client to use with the Gravitino Virtual File System. Currently supports `simple` and `oauth2` auth types. | `simple` | No| 0.6.0-incubating | -| `oauth2_server_uri` | The auth server URI for the Gravitino client when using `oauth2` auth type. | (none)| Yes if you use `oauth2` auth type | 0.7.0-incubating | -| `oauth2_credential` | The auth credential for the Gravitino client when using `oauth2` auth type. | (none)| Yes if you use `oauth2` auth type | 0.7.0-incubating | -| `oauth2_path`| The auth server path for the Gravitino client when using `oauth2` auth type. Please remove the first slash `/` from the path, for example `oauth/token`. | (none)| Yes if you use `oauth2` auth type | 0.7.0-incubating | -| `oauth2_scope` | The auth scope for the Gravitino client when using `oauth2` auth type with the Gravitino Virtual File System. | (none)| Yes if you use `oauth2` auth type | 0.7.0-incubating | +| Configuration item | Description | Default value | Required | Since version| +||--|---|---|--| +| `server_uri` | The Gravitino server uri, e.g. `http://localhost:8090`. | (none)| Yes | 0.6.0-incubating | +| `metalake_name`| The metalake name which the fileset belongs to. | (none)| Yes | 0.6.0-incubating | +| `cache_size` | The cache capacity of the Gravitino Virtual File System. | `20` | No | 0.6.0-incubating | +| `cache_expired_time` | The value of time that the cache expires after accessing in the Gravitino Virtual File System. The value is in `seconds`. | `3600`| No| 0.6.0-incubating | +| `auth_type`| The auth type to initialize the Gravitino client to use with the Gravit
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
yuqi1129 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1822314787 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1822087249 ## docs/how-to-use-gvfs.md: ## @@ -49,22 +49,69 @@ the path mapping and convert automatically. ### Configuration -| Configuration item| Description | Default value | Required| Since version | -|---|--|---|-|---| -| `fs.AbstractFileSystem.gvfs.impl` | The Gravitino Virtual File System abstract class, set it to `org.apache.gravitino.filesystem.hadoop.Gvfs`. | (none)| Yes | 0.5.0 | -| `fs.gvfs.impl`| The Gravitino Virtual File System implementation class, set it to `org.apache.gravitino.filesystem.hadoop.GravitinoVirtualFileSystem`. | (none)| Yes | 0.5.0 | -| `fs.gvfs.impl.disable.cache` | Disable the Gravitino Virtual File System cache in the Hadoop environment. If you need to proxy multi-user operations, please set this value to `true` and create a separate File System for each user. | `false` | No | 0.5.0 | -| `fs.gravitino.server.uri` | The Gravitino server URI which GVFS needs to load the fileset metadata. | (none)| Yes | 0.5.0 | -| `fs.gravitino.client.metalake`| The metalake to which the fileset belongs. | (none)| Yes | 0.5.0 | -| `fs.gravitino.client.authType`| The auth type to initialize the Gravitino client to use with the Gravitino Virtual File System. Currently only supports `simple`, `oauth2` and `kerberos` auth types. | `simple` | No | 0.5.0 | -| `fs.gravitino.client.oauth2.serverUri`| The auth server URI for the Gravitino client when using `oauth2` auth type with the Gravitino Virtual File System. | (none)| Yes if you use `oauth2` auth type | 0.5.0 | -| `fs.gravitino.client.oauth2.credential` | The auth credential for the Gravitino client when using `oauth2` auth type in the Gravitino Virtual File System. | (none)| Yes if you use `oauth2` auth type | 0.5.0 | -| `fs.gravitino.client.oauth2.path` | The auth server path for the Gravitino client when using `oauth2` auth type with the Gravitino Virtual File System. Please remove the first slash `/` from the path, for example `oauth/token`. | (none)| Yes if you use `oauth2` auth type | 0.5.0 | -| `fs.gravitino.client.oauth2.scope`| The auth scope for the Gravitino client when using `oauth2` auth type with the Gravitino Virtual File System. | (none)| Yes if you use `oauth2` auth type | 0.5.0 | -| `fs.gravitino.client.kerberos.principal` | The auth principal for the Gravitino client when using `kerberos` auth type with the Gravitino Virtual File System. | (none)| Yes if you use `kerberos` auth type | 0.5.1 | -| `fs.gravitino.client.kerberos.keytabFilePath` | The auth keytab file path for the Gravitino client when using `kerberos` auth type in the Gravitino Virtual File System. | (none)| No | 0.5.1 | -| `fs.gravitino.fileset.cache.maxCapacity`
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1822061324 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1822090529 ## docs/how-to-use-gvfs.md: ## @@ -335,17 +382,24 @@ to recompile the native libraries like `libhdfs` and others, and completely repl ### Configuration -| Configuration item | Description | Default value | Required | Since version| -|--|--|---|---|--| -| `server_uri` | The Gravitino server uri, e.g. `http://localhost:8090`. | (none)| Yes | 0.6.0-incubating | -| `metalake_name` | The metalake name which the fileset belongs to. | (none)| Yes | 0.6.0-incubating | -| `cache_size` | The cache capacity of the Gravitino Virtual File System. | `20` | No| 0.6.0-incubating | -| `cache_expired_time` | The value of time that the cache expires after accessing in the Gravitino Virtual File System. The value is in `seconds`. | `3600`| No| 0.6.0-incubating | -| `auth_type` | The auth type to initialize the Gravitino client to use with the Gravitino Virtual File System. Currently supports `simple` and `oauth2` auth types. | `simple` | No| 0.6.0-incubating | -| `oauth2_server_uri` | The auth server URI for the Gravitino client when using `oauth2` auth type. | (none)| Yes if you use `oauth2` auth type | 0.7.0-incubating | -| `oauth2_credential` | The auth credential for the Gravitino client when using `oauth2` auth type. | (none)| Yes if you use `oauth2` auth type | 0.7.0-incubating | -| `oauth2_path`| The auth server path for the Gravitino client when using `oauth2` auth type. Please remove the first slash `/` from the path, for example `oauth/token`. | (none)| Yes if you use `oauth2` auth type | 0.7.0-incubating | -| `oauth2_scope` | The auth scope for the Gravitino client when using `oauth2` auth type with the Gravitino Virtual File System. | (none)| Yes if you use `oauth2` auth type | 0.7.0-incubating | +| Configuration item | Description | Default value | Required | Since version| +||--|---|---|--| +| `server_uri` | The Gravitino server uri, e.g. `http://localhost:8090`. | (none)| Yes | 0.6.0-incubating | +| `metalake_name`| The metalake name which the fileset belongs to. | (none)| Yes | 0.6.0-incubating | +| `cache_size` | The cache capacity of the Gravitino Virtual File System. | `20` | No | 0.6.0-incubating | +| `cache_expired_time` | The value of time that the cache expires after accessing in the Gravitino Virtual File System. The value is in `seconds`. | `3600`| No| 0.6.0-incubating | +| `auth_type`| The auth type to initialize the Gravitino client to use with the Gravi
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1822073085 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1822062584 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1822060449 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,88 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
yuqi1129 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1821788712 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,77 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
FANNG1 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1821766890 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,77 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 | No
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
FANNG1 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1821766582 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,77 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 | No
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
FANNG1 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1821765168 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,77 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 | No
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
yuqi1129 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1820924153 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,30 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1820851898 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,30 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
yuqi1129 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1820713863 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,30 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
yuqi1129 commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1820713863 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,30 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
Re: [PR] [#5081] improvement(docs): Add the document about cloud storage fileset. [gravitino]
jerryshao commented on code in PR #5324: URL: https://github.com/apache/gravitino/pull/5324#discussion_r1820165220 ## docs/hadoop-catalog.md: ## @@ -25,19 +25,30 @@ Hadoop 3. If there's any compatibility issue, please create an [issue](https://g Besides the [common catalog properties](./gravitino-server-config.md#gravitino-catalog-properties-configuration), the Hadoop catalog has the following properties: -| Property Name | Description | Default Value | Required | Since Version| -|||-|-|--| -| `location` | The storage location managed by Hadoop catalog. | (none) | No | 0.5.0| -| `filesystem-providers` | The names (split by comma) of filesystem providers for the Hadoop catalog. Gravitino already support built-in `builtin-local`(`local file`) and `builtin-hdfs`(`hdfs`). If users want to support more file system and add it to Gravitino, they custom more file system by implementing `FileSystemProvider`. | (none) | No | 0.7.0-incubating | -| `default-filesystem-provider` | The name default filesystem providers of this Hadoop catalog if users do not specify the scheme in the URI. Default value is `builtin-local` | `builtin-local` | No | 0.7.0-incubating | -| `authentication.impersonation-enable` | Whether to enable impersonation for the Hadoop catalog. | `false` | No | 0.5.1| -| `authentication.type` | The type of authentication for Hadoop catalog, currently we only support `kerberos`, `simple`. | `simple`| No | 0.5.1| -| `authentication.kerberos.principal`| The principal of the Kerberos authentication | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.keytab-uri` | The URI of The keytab for the Kerberos authentication. | (none) | required if the value of `authentication.type` is Kerberos. | 0.5.1| -| `authentication.kerberos.check-interval-sec` | The check interval of Kerberos credential for Hadoop catalog. | 60 |
