jerryshao commented on code in PR #8229:
URL: https://github.com/apache/gravitino/pull/8229#discussion_r2303224455
##########
docs/gravitino-server-config.md:
##########
@@ -51,21 +51,32 @@ If you are going to use H2 in the production environment,
Gravitino will not gua
The following table lists the storage configuration items:
-| Configuration item | Description
| Default value
| Required | Since
version |
-|---------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------|-------------------------------------------------|------------------|
-| `gravitino.entity.store` | Which entity storage
implementation to use. Only`relational` storage is currently supported.
|
`relational` | No
| 0.1.0 |
-| `gravitino.entity.store.maxTransactionSkewTimeMs` | The maximum skew time of
transactions in milliseconds.
| `2000`
| No | 0.3.0
|
-| `gravitino.entity.store.deleteAfterTimeMs` | The maximum time in
milliseconds that deleted and old-version data is kept. Set to at least 10
minutes and no longer than 30 days.
|
`604800000`(7 days) | No
| 0.5.0 |
-| `gravitino.entity.store.versionRetentionCount` | The Count of versions
allowed to be retained, including the current version, used to delete old
versions data. Set to at least 1 and no greater than 10.
| `1`
| No |
0.5.0 |
-| `gravitino.entity.store.relational` | Detailed implementation
of Relational storage. `H2`, `MySQL` and `PostgreSQL` is currently supported,
and the implementation is `JDBCBackend`.
|
`JDBCBackend` | No
| 0.5.0 |
-| `gravitino.entity.store.relational.jdbcUrl` | The database url that
the `JDBCBackend` needs to connect to. If you use `MySQL` or `PostgreSQL`, you
should firstly initialize the database tables yourself by executing the ddl
scripts in the `${GRAVITINO_HOME}/scripts/{DATABASE_TYPE}/` directory. |
`jdbc:h2` | No
| 0.5.0 |
-| `gravitino.entity.store.relational.jdbcDriver` | The jdbc driver name
that the `JDBCBackend` needs to use. You should place the driver Jar package in
the `${GRAVITINO_HOME}/libs/` directory.
|
`org.h2.Driver` | Yes if the jdbc connection url is not
`jdbc:h2` | 0.5.0 |
-| `gravitino.entity.store.relational.jdbcUser` | The username that the
`JDBCBackend` needs to use when connecting the database. It is required for
`MySQL`.
|
`gravitino` | Yes if the jdbc connection url is not
`jdbc:h2` | 0.5.0 |
-| `gravitino.entity.store.relational.jdbcPassword` | The password that the
`JDBCBackend` needs to use when connecting the database. It is required for
`MySQL`.
|
`gravitino` | Yes if the jdbc connection url is not
`jdbc:h2` | 0.5.0 |
-| `gravitino.entity.store.relational.storagePath` | The storage path for
embedded JDBC storage implementation. It supports both absolute and relative
path, if the value is a relative path, the final path is
`${GRAVITINO_HOME}/${PATH_YOU_HAVA_SET}`, default value is
`${GRAVITINO_HOME}/data/jdbc` | `${GRAVITINO_HOME}/data/jdbc` | No
| 0.6.0-incubating |
-| `gravitino.entity.store.relational.maxConnections`| The maximum number of
connections for the JDBC Backend connection pool
| `100`
| No |
0.9.0-incubating |
-| `gravitino.entity.store.relational.maxWaitMillis` | The maximum wait time in
milliseconds for a connection from the JDBC Backend connection pool
| `1000`
| No |
0.9.0-incubating |
-
+| Configuration item | Description
| Default value
| Required
| Since version |
+|----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------|-------------------------------------------------|------------------|
+| `gravitino.entity.store` | Which entity storage
implementation to use. Only`relational` storage is currently supported.
|
`relational` | No
| 0.1.0 |
+| `gravitino.entity.store.maxTransactionSkewTimeMs` | The maximum skew time
of transactions in milliseconds.
| `2000`
| No
| 0.3.0 |
+| `gravitino.entity.store.deleteAfterTimeMs` | The maximum time in
milliseconds that deleted and old-version data is kept. Set to at least 10
minutes and no longer than 30 days.
|
`604800000`(7 days) | No
| 0.5.0 |
+| `gravitino.entity.store.versionRetentionCount` | The Count of versions
allowed to be retained, including the current version, used to delete old
versions data. Set to at least 1 and no greater than 10.
| `1`
| No
| 0.5.0 |
+| `gravitino.entity.store.relational` | Detailed implementation
of Relational storage. `H2`, `MySQL` and `PostgreSQL` is currently supported,
and the implementation is `JDBCBackend`.
|
`JDBCBackend` | No
| 0.5.0 |
+| `gravitino.entity.store.relational.jdbcUrl` | The database url that
the `JDBCBackend` needs to connect to. If you use `MySQL` or `PostgreSQL`, you
should firstly initialize the database tables yourself by executing the ddl
scripts in the `${GRAVITINO_HOME}/scripts/{DATABASE_TYPE}/` directory. |
`jdbc:h2` | No
| 0.5.0 |
+| `gravitino.entity.store.relational.jdbcDriver` | The jdbc driver name
that the `JDBCBackend` needs to use. You should place the driver Jar package in
the `${GRAVITINO_HOME}/libs/` directory.
|
`org.h2.Driver` |
Yes if the jdbc connection url is not `jdbc:h2` | 0.5.0 |
+| `gravitino.entity.store.relational.jdbcUser` | The username that the
`JDBCBackend` needs to use when connecting the database. It is required for
`MySQL`.
|
`gravitino` |
Yes if the jdbc connection url is not `jdbc:h2` | 0.5.0 |
+| `gravitino.entity.store.relational.jdbcPassword` | The password that the
`JDBCBackend` needs to use when connecting the database. It is required for
`MySQL`.
|
`gravitino` |
Yes if the jdbc connection url is not `jdbc:h2` | 0.5.0 |
+| `gravitino.entity.store.relational.storagePath` | The storage path for
embedded JDBC storage implementation. It supports both absolute and relative
path, if the value is a relative path, the final path is
`${GRAVITINO_HOME}/${PATH_YOU_HAVA_SET}`, default value is
`${GRAVITINO_HOME}/data/jdbc` | `${GRAVITINO_HOME}/data/jdbc`
| No
| 0.6.0-incubating |
+| `gravitino.entity.store.relational.maxConnections` | The maximum number of
connections for the JDBC Backend connection pool
| `100`
| No
| 0.9.0-incubating |
+| `gravitino.entity.store.relational.maxWaitMillis` | The maximum wait time
in milliseconds for a connection from the JDBC Backend connection pool
| `1000`
| No
| 0.9.0-incubating |
+| `gravitino.stats.partition.storageFactoryClass` | The storage factory
class for partition statistics, which is used to store partition statistics in
the different storage. The
`org.apache.gravitino.stats.storage.MemoryPartitionStatsStorageFactory` can
only be used for test. |
`org.apache.gravitino.stats.storage.LancePartitionStatisticStorageFactory` |
No | 1.0.0 |
+
+
+If you use [Lance](https://lancedb.github.io/lance/) as the partition stats
storage, you can set below options, if you have other lance storage options,
you can pass it adding prefix `gravitino.stats.partition.lance.`.
+
+| Configuration item | Description
| Default value | Required
| Since version |
+|-----------------------------------------------------------|-------------------------------------------|--------------------------------|-------------------------------------------------|---------------|
+| `gravitino.stats.partition.storageOption.location` | The location of
Lance files | `${GRAVITINO_HOME}/data/lance` | No
| 1.0.0 |
+| `gravitino.stats.partition.storageOption.maxRowsPerFile` | The max rows per
file | `1000000` | No
| 1.0.0 |
+| `gravitino.stats.partition.storageOption.maxBytesPerFile` | The max bytes
per file | `100 * 1024 * 1024` | No
| 1.0.0 |
+| `gravitino.stats.partition.storageOption.maxRowsPerGroup` | The max rows per
group | `1000000` | No
| 1.0.0 |
+| `gravitino.stats.partition.storageOption.readBatchSize` | The read batch
record number when reading | `10000` | No
| 1.0.0 |
Review Comment:
Also, you should explain how to pass the configuration to the specific
storage. Anyway, do you think user can easily use/develop this module by your
doc?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
