yuqi1129 commented on code in PR #8777:
URL: https://github.com/apache/gravitino/pull/8777#discussion_r2522049124
##########
docs/fileset-catalog.md:
##########
@@ -45,14 +159,25 @@ Please refer to [Credential
vending](./security/credential-vending.md) for more
Apart from the above properties, to access fileset like HDFS fileset, you need
to configure the following extra properties.
-| Property Name | Description
|
Default Value | Required |
Since Version |
-|----------------------------------------------------|------------------------------------------------------------------------------------------------|---------------|-------------------------------------------------------------|---------------|
-| `authentication.impersonation-enable` | Whether to enable
impersonation for the Fileset catalog. |
`false` | No |
0.5.1 |
-| `authentication.type` | The type of
authentication for Fileset catalog, currently we only support `kerberos`,
`simple`. | `simple` | No
| 0.5.1 |
-| `authentication.kerberos.principal` | The principal of the
Kerberos authentication |
(none) | required if the value of `authentication.type` is Kerberos. |
0.5.1 |
-| `authentication.kerberos.keytab-uri` | The URI of The keytab
for the Kerberos authentication. |
(none) | required if the value of `authentication.type` is Kerberos. |
0.5.1 |
-| `authentication.kerberos.check-interval-sec` | The check interval of
Kerberos credential for Fileset catalog. | 60
| No | 0.5.1
|
-| `authentication.kerberos.keytab-fetch-timeout-sec` | The fetch timeout of
retrieving Kerberos keytab from `authentication.kerberos.keytab-uri`. | 60
| No | 0.5.1
|
+| Property Name | Description
|
Default Value | Required |
Since Version |
+|----------------------------------------------------|---------------------------------------------------------------------------------------------------|---------------|-------------------------------------------------------------|---------------|
+| `authentication.impersonation-enable` | Whether to enable
impersonation for the Fileset catalog.
| `false` | No |
0.5.1 |
+| `authentication.type` | The type of
authentication for Fileset catalog, currently we only support `kerberos`,
`simple`. | `simple` | No
| 0.5.1 |
+| `authentication.kerberos.principal` | The principal of the
Kerberos authentication |
(none) | required if the value of `authentication.type` is Kerberos. |
0.5.1 |
+| `authentication.kerberos.keytab-uri` | The URI of The keytab
for the Kerberos authentication. |
(none) | required if the value of `authentication.type` is Kerberos. |
0.5.1 |
+| `authentication.kerberos.check-interval-sec` | The check interval of
Kerberos credential for Fileset catalog. |
60 | No |
0.5.1 |
+| `authentication.kerberos.keytab-fetch-timeout-sec` | The fetch timeout of
retrieving Kerberos keytab from `authentication.kerberos.keytab-uri`. |
60 | No |
0.5.1 |
+| `hdfs.config.resources` | The HDFS configuration
resources, separated by comma. For example, `hdfs-site.xml,core-site.xml`. |
(none) | No |
0.5.1 |
+
+The `hdfs.config.resources` property allows users to specify custom HDFS
configuration files.
+
+The Gravitino Fileset extends the following properties in the `xxx-site.xml`:
+
+| Property Name | Description
| Default Value | Required
| Since Version |
+|---------------------------------------------------|-------------------------------------------------------------------------|---------------|-------------------------------------------------------------|---------------|
+| hadoop.security.authentication.kerberos.principal | The principal of the
Kerberos authentication for HDFS client. | (none) | required
if the value of `authentication.type` is Kerberos. | 0.1.1 |
Review Comment:
This has not been resolved.
##########
docs/fileset-catalog.md:
##########
@@ -21,6 +21,120 @@ Note that Gravitino uses Hadoop 3 dependencies to build
Fileset catalog. Theoret
compatible with both Hadoop 2.x and 3.x, since Gravitino doesn't leverage any
new features in
Hadoop 3. If there's any compatibility issue, please create an
[issue](https://github.com/apache/gravitino/issues).
+In general, all schemas and filesets under a Fileset Catalog are stored in the
same Hadoop cluster,
+under the location specified by the catalog itself.
Review Comment:
You may want to express: the locations of all schemas and filesets under a
fileset catalog belong to a single Hadoop cluster if they are HDFS location.
##########
docs/fileset-catalog.md:
##########
@@ -21,6 +21,120 @@ Note that Gravitino uses Hadoop 3 dependencies to build
Fileset catalog. Theoret
compatible with both Hadoop 2.x and 3.x, since Gravitino doesn't leverage any
new features in
Hadoop 3. If there's any compatibility issue, please create an
[issue](https://github.com/apache/gravitino/issues).
+In general, all schemas and filesets under a Fileset Catalog are stored in the
same Hadoop cluster,
+under the location specified by the catalog itself.
Review Comment:
You may want to express: the locations of all schemas and filesets under a
fileset catalog belong to a single Hadoop cluster if they are HDFS location.
##########
docs/fileset-catalog.md:
##########
@@ -21,6 +21,120 @@ Note that Gravitino uses Hadoop 3 dependencies to build
Fileset catalog. Theoret
compatible with both Hadoop 2.x and 3.x, since Gravitino doesn't leverage any
new features in
Hadoop 3. If there's any compatibility issue, please create an
[issue](https://github.com/apache/gravitino/issues).
+In general, all schemas and filesets under a Fileset Catalog are stored in the
same Hadoop cluster,
+under the location specified by the catalog itself.
+
+The example for creating a Fileset is as follows:
+```text
+# create fileset catalog
+curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
+-H "Content-Type: application/json" -d '{
+ "name": "fileset_catalog",
+ "type": "FILESET",
+ "comment": "This is a fileset catalog",
+ "provider": "fileset",
+ "properties": {
+ "location": "hdfs://172.17.0.2:9000/fileset_catalog"
+ }
+}' http://localhost:8090/api/metalakes/test/catalogs
+
+# create a fileset schema under the catalog with inherited properties
+curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
+-H "Content-Type: application/json" -d '{
+ "name": "test_schema",
+ "comment": "This is a schema",
+ "properties": {
+ }
+}' http://localhost:8090/api/metalakes/test/catalogs/fileset_catalog/schemas
+
+# create a fileset under the schema with inherited properties
+curl -X POST -H "Accept: application/vnd.gravitino.v1+json"
+-H "Content-Type: application/json" -d '{
+ "name": "fs1",
+ "comment": "This is an example fileset",
+ "type": "MANAGED",
+ "properties": {
+ }
+}'
http://localhost:8090/api/metalakes/test/catalogs/fileset_catalog/schemas/test_schema/filesets
+```
+
+Within a Fileset Catalog, schemas and filesets can automatically inherit
configuration properties
Review Comment:
No need to capitalize `Fileset Catalog`, please resolve all similar problems.
##########
docs/fileset-catalog.md:
##########
@@ -21,6 +21,120 @@ Note that Gravitino uses Hadoop 3 dependencies to build
Fileset catalog. Theoret
compatible with both Hadoop 2.x and 3.x, since Gravitino doesn't leverage any
new features in
Hadoop 3. If there's any compatibility issue, please create an
[issue](https://github.com/apache/gravitino/issues).
+In general, all schemas and filesets under a Fileset Catalog are stored in the
same Hadoop cluster,
+under the location specified by the catalog itself.
+
+The example for creating a Fileset is as follows:
+```text
+# create fileset catalog
+curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
+-H "Content-Type: application/json" -d '{
+ "name": "fileset_catalog",
+ "type": "FILESET",
+ "comment": "This is a fileset catalog",
+ "provider": "fileset",
+ "properties": {
+ "location": "hdfs://172.17.0.2:9000/fileset_catalog"
+ }
+}' http://localhost:8090/api/metalakes/test/catalogs
+
+# create a fileset schema under the catalog with inherited properties
+curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \
+-H "Content-Type: application/json" -d '{
+ "name": "test_schema",
+ "comment": "This is a schema",
+ "properties": {
+ }
+}' http://localhost:8090/api/metalakes/test/catalogs/fileset_catalog/schemas
+
+# create a fileset under the schema with inherited properties
+curl -X POST -H "Accept: application/vnd.gravitino.v1+json"
+-H "Content-Type: application/json" -d '{
+ "name": "fs1",
+ "comment": "This is an example fileset",
+ "type": "MANAGED",
+ "properties": {
+ }
+}'
http://localhost:8090/api/metalakes/test/catalogs/fileset_catalog/schemas/test_schema/filesets
+```
+
+Within a Fileset Catalog, schemas and filesets can automatically inherit
configuration properties
Review Comment:
No need to capitalize `Fileset Catalog`, please resolve all similar problems.
##########
docs/fileset-catalog.md:
##########
@@ -21,6 +21,120 @@ Note that Gravitino uses Hadoop 3 dependencies to build
Fileset catalog. Theoret
compatible with both Hadoop 2.x and 3.x, since Gravitino doesn't leverage any
new features in
Hadoop 3. If there's any compatibility issue, please create an
[issue](https://github.com/apache/gravitino/issues).
+In general, all schemas and filesets under a Fileset Catalog are stored in the
same Hadoop cluster,
+under the location specified by the catalog itself.
+
+The example for creating a Fileset is as follows:
Review Comment:
creating a Fileset with HDFS location
##########
docs/fileset-catalog.md:
##########
@@ -21,6 +21,120 @@ Note that Gravitino uses Hadoop 3 dependencies to build
Fileset catalog. Theoret
compatible with both Hadoop 2.x and 3.x, since Gravitino doesn't leverage any
new features in
Hadoop 3. If there's any compatibility issue, please create an
[issue](https://github.com/apache/gravitino/issues).
+In general, all schemas and filesets under a Fileset Catalog are stored in the
same Hadoop cluster,
+under the location specified by the catalog itself.
+
+The example for creating a Fileset is as follows:
Review Comment:
`Fileset` -> `fileset`.
##########
docs/fileset-catalog.md:
##########
@@ -21,6 +21,120 @@ Note that Gravitino uses Hadoop 3 dependencies to build
Fileset catalog. Theoret
compatible with both Hadoop 2.x and 3.x, since Gravitino doesn't leverage any
new features in
Hadoop 3. If there's any compatibility issue, please create an
[issue](https://github.com/apache/gravitino/issues).
+In general, all schemas and filesets under a Fileset Catalog are stored in the
same Hadoop cluster,
+under the location specified by the catalog itself.
+
+The example for creating a Fileset is as follows:
Review Comment:
creating a Fileset with HDFS location
##########
docs/fileset-catalog.md:
##########
@@ -45,14 +159,25 @@ Please refer to [Credential
vending](./security/credential-vending.md) for more
Apart from the above properties, to access fileset like HDFS fileset, you need
to configure the following extra properties.
-| Property Name | Description
|
Default Value | Required |
Since Version |
-|----------------------------------------------------|------------------------------------------------------------------------------------------------|---------------|-------------------------------------------------------------|---------------|
-| `authentication.impersonation-enable` | Whether to enable
impersonation for the Fileset catalog. |
`false` | No |
0.5.1 |
-| `authentication.type` | The type of
authentication for Fileset catalog, currently we only support `kerberos`,
`simple`. | `simple` | No
| 0.5.1 |
-| `authentication.kerberos.principal` | The principal of the
Kerberos authentication |
(none) | required if the value of `authentication.type` is Kerberos. |
0.5.1 |
-| `authentication.kerberos.keytab-uri` | The URI of The keytab
for the Kerberos authentication. |
(none) | required if the value of `authentication.type` is Kerberos. |
0.5.1 |
-| `authentication.kerberos.check-interval-sec` | The check interval of
Kerberos credential for Fileset catalog. | 60
| No | 0.5.1
|
-| `authentication.kerberos.keytab-fetch-timeout-sec` | The fetch timeout of
retrieving Kerberos keytab from `authentication.kerberos.keytab-uri`. | 60
| No | 0.5.1
|
+| Property Name | Description
|
Default Value | Required |
Since Version |
+|----------------------------------------------------|---------------------------------------------------------------------------------------------------|---------------|-------------------------------------------------------------|---------------|
+| `authentication.impersonation-enable` | Whether to enable
impersonation for the Fileset catalog.
| `false` | No |
0.5.1 |
+| `authentication.type` | The type of
authentication for Fileset catalog, currently we only support `kerberos`,
`simple`. | `simple` | No
| 0.5.1 |
+| `authentication.kerberos.principal` | The principal of the
Kerberos authentication |
(none) | required if the value of `authentication.type` is Kerberos. |
0.5.1 |
+| `authentication.kerberos.keytab-uri` | The URI of The keytab
for the Kerberos authentication. |
(none) | required if the value of `authentication.type` is Kerberos. |
0.5.1 |
+| `authentication.kerberos.check-interval-sec` | The check interval of
Kerberos credential for Fileset catalog. |
60 | No |
0.5.1 |
+| `authentication.kerberos.keytab-fetch-timeout-sec` | The fetch timeout of
retrieving Kerberos keytab from `authentication.kerberos.keytab-uri`. |
60 | No |
0.5.1 |
+| `hdfs.config.resources` | The HDFS configuration
resources, separated by comma. For example, `hdfs-site.xml,core-site.xml`. |
(none) | No |
0.5.1 |
+
+The `hdfs.config.resources` property allows users to specify custom HDFS
configuration files.
+
+The Gravitino Fileset extends the following properties in the `xxx-site.xml`:
+
+| Property Name | Description
| Default Value | Required
| Since Version |
+|---------------------------------------------------|-------------------------------------------------------------------------|---------------|-------------------------------------------------------------|---------------|
+| hadoop.security.authentication.kerberos.principal | The principal of the
Kerberos authentication for HDFS client. | (none) | required
if the value of `authentication.type` is Kerberos. | 0.1.1 |
Review Comment:
This has not been resolved.
##########
docs/fileset-catalog.md:
##########
@@ -21,6 +21,120 @@ Note that Gravitino uses Hadoop 3 dependencies to build
Fileset catalog. Theoret
compatible with both Hadoop 2.x and 3.x, since Gravitino doesn't leverage any
new features in
Hadoop 3. If there's any compatibility issue, please create an
[issue](https://github.com/apache/gravitino/issues).
+In general, all schemas and filesets under a Fileset Catalog are stored in the
same Hadoop cluster,
+under the location specified by the catalog itself.
+
+The example for creating a Fileset is as follows:
Review Comment:
`Fileset` -> `fileset`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
