yuqi1129 commented on code in PR #6724:
URL: https://github.com/apache/gravitino/pull/6724#discussion_r2011435578
##########
docs/manage-fileset-metadata-using-gravitino.md:
##########
@@ -16,7 +16,7 @@ filesets to manage non-tabular data like training datasets
and other raw data.
Typically, a fileset is mapped to a directory on a file system like HDFS, S3,
ADLS, GCS, etc.
With the fileset managed by Gravitino, the non-tabular data can be managed as
assets together with
tabular data in Gravitino in a unified way. The following operations will use
HDFS as an example, for other
-HCFS like S3, OSS, GCS, etc, please refer to the corresponding operations
[hadoop-with-s3](./hadoop-catalog-with-s3.md),
[hadoop-with-oss](./hadoop-catalog-with-oss.md),
[hadoop-with-gcs](./hadoop-catalog-with-gcs.md) and
+HCFS like S3, OSS, GCS, etc. Please refer to the corresponding operations
[hadoop-with-s3](./hadoop-catalog-with-s3.md),
[hadoop-with-oss](./hadoop-catalog-with-oss.md),
[hadoop-with-gcs](./hadoop-catalog-with-gcs.md) and
Review Comment:
Why do we capitalize the word `please`? It seems that separating it into a
separate sentence is not very good.
##########
docs/manage-fileset-metadata-using-gravitino.md:
##########
@@ -320,24 +320,41 @@ Currently, Gravitino supports two **types** of filesets:
The `storageLocation` is the physical location of the fileset. Users can
specify this location
when creating a fileset, or follow the rules of the catalog/schema location if
not specified.
+The `storageLocation` in each level can contain **placeholders**, format as
`{{name}}`, which will
+be replaced by the corresponding fileset property value when the fileset
object is created. The
+placeholder property in the fileset object is formed as
"placeholder-{{name}}". For example, if
+the `storageLocation` is `file:///tmp/{{schema}}-{{fileset}}-{{verion}}`, and
the fileset object
+named "catalog1.schema1.fileset1" contains the properties
`placeholder-version=v1`,
+the actual `storageLocation` will be `file:///tmp/schema1-fileset1-v1`.
+
The value of `storageLocation` depends on the configuration settings of the
catalog:
- If this is a local fileset catalog, the `storageLocation` should be in the
format of `file:///path/to/fileset`.
- If this is a HDFS fileset catalog, the `storageLocation` should be in the
format of `hdfs://namenode:port/path/to/fileset`.
For a `MANAGED` fileset, the storage location is:
-1. The one specified by the user during the fileset creation.
-2. When the catalog property `location` is specified but the schema property
`location` isn't specified,
- the storage location is `catalog location/schema name/fileset name`.
+1. The one specified by the user during the fileset creation, and the
placeholder will be replaced by the
+ corresponding fileset property value.
+2. When the catalog property `location` is specified but the schema property
`location` isn't specified, the storage location is:
+ 1. `catalog location/schema name/fileset name` if `catalog location` does
not contain any placeholder.
+ 2. `catalog location` - placeholders in the catalog location will be
replaced by the corresponding fileset property value.
+
3. When the catalog property `location` isn't specified but the schema
property `location` is specified,
- the storage location is `schema location/fileset name`.
+ the storage location is:
+ 1. `schema location/fileset name` if `schema location` does not contain any
placeholder.
+ 2. `schema location` - placeholders in the schema location will be replaced
by the corresponding fileset property value.
Review Comment:
What's meaning of `-` that is between `schema location` and `placeholders`.
##########
docs/hadoop-catalog.md:
##########
@@ -129,6 +129,15 @@ Refer to [Schema
operation](./manage-fileset-metadata-using-gravitino.md#schema-
| `authentication.kerberos.principal` | The principal of the Kerberos
authentication for the fileset. | The
parent(schema) value | No | 0.6.0-incubating |
| `authentication.kerberos.keytab-uri` | The URI of The keytab for the
Kerberos authentication for the fileset. | The
parent(schema) value | No | 0.6.0-incubating |
| `credential-providers` | The credential provider types,
separated by comma. |
(none) | No | 0.8.0-incubating |
+| `placeholder-` | Properties that start with
`placeholder-` are used to replace placeholders in the location. |
(none) | No | 0.9.0-incubating |
+
+Here are some properties are reserved and cannot be set by users:
Review Comment:
...properties are reserved .. -> ... reserved properties...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
