This is an automated email from the ASF dual-hosted git repository.
adoroszlai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ozone-site.git
The following commit(s) were added to refs/heads/master by this push:
new 93d195799d HDDS-16088. [Auto] Update configuration documentation (#529)
93d195799d is described below
commit 93d195799d1e58b8218f3645d5f429fe878cdf72
Author: asf-ci-deploy <[email protected]>
AuthorDate: Tue Sep 22 19:07:29 2026 +0200
HDDS-16088. [Auto] Update configuration documentation (#529)
---
.../02-configuration/99-appendix.md | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/docs/05-administrator-guide/02-configuration/99-appendix.md
b/docs/05-administrator-guide/02-configuration/99-appendix.md
index 2ff2ed344f..9f73d7e4db 100644
--- a/docs/05-administrator-guide/02-configuration/99-appendix.md
+++ b/docs/05-administrator-guide/02-configuration/99-appendix.md
@@ -104,6 +104,7 @@ This page provides a comprehensive overview of the
configuration keys available
| `hdds.datanode.container.close.threads.max` | 3 | `DATANODE` | The maximum
number of threads used to close containers on a datanode |
| `hdds.datanode.container.db.dir` | | `OZONE`, `CONTAINER`, `STORAGE`,
`MANAGEMENT` | Determines where the per-disk rocksdb instances will be stored.
This setting is optional. If unspecified, then rocksdb instances are stored on
the same disk as HDDS data. The directories should be tagged with corresponding
storage types ([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for storage policies. The
default storage type will be DISK if the directory does not have a storage type
tagged explicitly. Ideall [...]
| `hdds.datanode.container.delete.threads.max` | 2 | `DATANODE` | The maximum
number of threads used to delete containers on a datanode |
+| `hdds.datanode.container.init.timeout` | 0s | `DATANODE` | Maximum time
allowed for datanode container services to initialize at startup.
Initialization builds the container set and starts the Ratis write channel,
which recovers all Raft groups from disk. If it does not complete within this
time, startup is failed so the datanode shuts down cleanly instead of hanging
indefinitely (for example when Ratis group recovery stalls on a failing
volume). A value of 0 (the default) disables the [...]
| `hdds.datanode.container.schema.v3.enabled` | true | `DATANODE` | Enable use
of container schema v3(one rocksdb per disk). |
| `hdds.datanode.container.schema.v3.key.separator` | \| | `DATANODE` | The
default separator between Container ID and container meta key name. |
| `hdds.datanode.data.dir.permissions` | 700 | | Permissions for the datanode
data directories where actual file blocks are stored. The permissions can be
either octal or symbolic. If the default permissions are not set then the
default value of 700 will be used. |
@@ -527,7 +528,7 @@ This page provides a comprehensive overview of the
configuration keys available
| `ozone.om.open.key.expire.threshold` | 7d | `OZONE`, `OM`, `PERFORMANCE` |
Controls how long an open key operation is considered active. Specifically, if
a key has been open longer than the value of this config entry, that open key
is considered as expired (e.g. due to client crash). Unit could be defined with
postfix (ns,ms,s,m,h,d) |
| `ozone.om.open.mpu.cleanup.service.interval` | 24h | `OZONE`, `OM`,
`PERFORMANCE` | A background job that periodically checks inactive multipart
info send multipart upload abort requests for them. This entry controls the
interval of this cleanup check. Unit could be defined with postfix
(ns,ms,s,m,h,d) |
| `ozone.om.open.mpu.cleanup.service.timeout` | 300s | `OZONE`, `OM`,
`PERFORMANCE` | A timeout value of multipart upload cleanup service. If this is
set greater than 0, the service will stop waiting for the multipart info abort
completion after this time. If timeout happens to a large proportion of
multipart aborts, this value needs to be increased or
`ozone.om.open.key.cleanup.limit.per.task` should be decreased. Unit could be
defined with postfix (ns,ms,s,m,h,d) |
-| `ozone.om.open.mpu.expire.threshold` | 30d | `OZONE`, `OM`, `PERFORMANCE` |
Controls how long multipart upload is considered active. Specifically, if a
multipart info has been ongoing longer than the value of this config entry,
that multipart info is considered as expired (e.g. due to client crash). Unit
could be defined with postfix (ns,ms,s,m,h,d) |
+| `ozone.om.open.mpu.expire.threshold` | 30d | `OZONE`, `OM`, `PERFORMANCE` |
Controls how long a multipart upload is considered active before
MultipartUploadCleanupService treats it as stale and aborts it. If a multipart
upload has been ongoing longer than this threshold (e.g. due to a client
crash), it will be cleaned up automatically. Unit can be specified with a
postfix (ns, ms, s, m, h, d). Interaction with S3 lifecycle
AbortIncompleteMultipartUpload rules: When a bucket lifecycle r [...]
| `ozone.om.open.mpu.parts.cleanup.limit.per.task` | 1000 | `OZONE`, `OM`,
`PERFORMANCE` | The maximum number of parts, rounded up to the nearest number
of expired multipart upload. This property is used to approximately throttle
the number of MPU parts sent to the OM. |
| `ozone.om.ratis.events.max.limit` | 100 | `OZONE`, `RATIS`, `OM` | The
maximum number of events that can be pending in OM Ratis. |
| `ozone.om.ratis.log.appender.queue.byte-limit` | 32MB | `OZONE`, `DEBUG`,
`OM`, `RATIS` | Byte limit for Raft's Log Worker queue. |
@@ -545,7 +546,6 @@ This page provides a comprehensive overview of the
configuration keys available
| `ozone.om.ratis.server.leaderelection.pre-vote` | true | `OZONE`, `OM`,
`RATIS`, `MANAGEMENT` | Enable/disable OM HA leader election pre-vote phase. |
| `ozone.om.ratis.server.pending.write.element-limit` | 4096 | `OZONE`,
`DEBUG`, `OM`, `RATIS` | Maximum number of pending write requests. |
| `ozone.om.ratis.server.request.timeout` | 3s | `OZONE`, `OM`, `RATIS`,
`MANAGEMENT` | The timeout duration for OM's ratis server request . |
-| `ozone.om.ratis.server.retry.cache.timeout` | 600000ms | `OZONE`, `OM`,
`RATIS`, `MANAGEMENT` | Retry Cache entry timeout for OM's ratis server. |
| `ozone.om.ratis.snapshot.dir` | | `OZONE`, `OM`, `STORAGE`, `MANAGEMENT`,
`RATIS` | This directory is used for storing OM's snapshot related files like
the ratisSnapshotIndex and DB checkpoint from leader OM. If undefined, OM
snapshot dir will fallback to `ozone.metadata.dirs`. This fallback approach is
not recommended for production environments. |
| `ozone.om.ratis.snapshot.max.total.sst.size` | 10737418240 | `OZONE`, `OM`,
`RATIS` | Max size of SST files in OM Ratis Snapshot tarball. |
| `ozone.om.ratis.storage.dir` | | `OZONE`, `OM`, `STORAGE`, `MANAGEMENT`,
`RATIS` | This directory is used for storing OM's Ratis metadata like logs. If
this is not set then default metadata dirs is used. A warning will be logged if
this not set. Ideally, this should be mapped to a fast disk like an SSD. If
undefined, OM ratis storage dir will fallback to `ozone.metadata.dirs`. This
fallback approach is not recommended for production environments. |
@@ -570,6 +570,7 @@ This page provides a comprehensive overview of the
configuration keys available
| `ozone.om.snapshot.diff.job.default.wait.time` | 1m | `OZONE`, `OM` |
Default wait time returned to client to wait before retrying snap diff request.
Uses millisecond by default when no time unit is specified. |
| `ozone.om.snapshot.diff.job.report.persistent.time` | 7d | `OZONE`, `OM` |
Maximum time a successful snapshot diff job and its report will be persisted.
Uses millisecond by default when no time unit is specified. |
| `ozone.om.snapshot.diff.max.allowed.keys.changed.per.job` | 1000000000 |
`OZONE`, `OM` | Max numbers of keys changed allowed for a snapshot diff job. |
+| `ozone.om.snapshot.diff.max.in.memory.entries.per.job` | 1000000 | `OZONE`,
`OM` | Maximum number of diff-candidate object IDs a snapshot diff job may
retain in memory before spilling to a temporary RocksDB column family. |
| `ozone.om.snapshot.diff.max.jobs.purge.per.task` | 100 | `OZONE`, `OM` |
Maximum number of snapshot diff jobs to be purged per snapDiff clean up run. |
| `ozone.om.snapshot.diff.max.page.size` | 5000 | `OZONE`, `OM` | Maximum
number of entries to be returned in a single page of snap diff report. |
| `ozone.om.snapshot.diff.thread.pool.size` | 10 | `OZONE`, `OM` | Maximum
numbers of concurrent snapshot diff jobs are allowed. |
@@ -583,6 +584,8 @@ This page provides a comprehensive overview of the
configuration keys available
| `ozone.om.snapshot.prune.compaction.backup.batch.size` | 2000 | `OZONE`,
`OM` | Prune SST files in Compaction backup directory in batches every
`ozone.om.snapshot.compaction.dag.prune.daemon.run.interval`. |
| `ozone.om.snapshot.rename.allowed` | false | `OZONE`, `OM` | Allows the
Ozone snapshot rename operation if set to true on the OM side. Blocks it
otherwise. |
| `ozone.om.snapshot.rocksdb.metrics.enabled` | false | `OZONE`, `OM` | Skip
collecting RocksDBStore metrics for Snapshotted DB. |
+| `ozone.om.sts.token.cleanup.service.interval` | 3h | `OZONE`, `OM`,
`PERFORMANCE`, `SECURITY` | A background service that periodically scans the
s3RevokedStsTokenTable and deletes revocation entries whose cutoff is older
than the maximum STS token lifetime (12 hours). This property controls how
often the cleanup service runs. Unit could be defined with postfix
(ns,ms,s,m,h,d). |
+| `ozone.om.sts.token.cleanup.service.timeout` | 15m | `OZONE`, `OM`,
`PERFORMANCE`, `SECURITY` | A timeout value for the revoked STS token cleanup
service. If this is set greater than 0, the service will stop waiting for the
deletion completion after this time. Unit could be defined with postfix
(ns,ms,s,m,h,d). |
| `ozone.om.transport.class` |
org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory | `OM`,
`MANAGEMENT` | Property to determine the transport protocol for the client to
Ozone Manager channel. |
| `ozone.om.unflushed.transaction.max.count` | 10000 | `OZONE`, `OM` | the
unflushed transactions here are those requests that have been applied to OM
state machine but not been flushed to OM rocksdb. when OM meets high
concurrency-pressure and flushing is not fast enough, too many pending requests
will be hold in memory and will lead to long GC of OM, which will slow down
flushing further. there are some cases that flushing is slow, for example, 1
rocksdb is on a HDD, which has poor IO [...]
| `ozone.om.upgrade.finalization.ratis.based.timeout` | 30s | `OM`, `UPGRADE`
| Maximum time to wait for a slow follower to be finalized through a Ratis
snapshot. This is an advanced config, and needs to be changed only under a
special circumstance when the leader OM has purged the finalize request from
its logs, and a follower OM was down during upgrade finalization. Default is
30s. |
@@ -667,6 +670,9 @@ This page provides a comprehensive overview of the
configuration keys available
| `ozone.s3g.client.buffer.size` | 4MB | `OZONE`, `S3GATEWAY` | The size of
the buffer which is for read block. (4MB by default). |
| `ozone.s3g.default.bucket.layout` | OBJECT_STORE | `OZONE`, `S3GATEWAY` |
The bucket layout that will be used when buckets are created through the S3
API. |
| `ozone.s3g.domain.name` | | `OZONE`, `S3GATEWAY` | List of Ozone S3Gateway
domain names. If multiple domain names to be provided, they should be a ","
separated. This parameter is only required when virtual host style pattern is
followed. |
+| `ozone.s3g.health-check.enabled` | true | `OZONE`, `S3GATEWAY`, `MANAGEMENT`
| If enabled, the S3 Gateway web admin server exposes unauthenticated health
endpoints that a load balancer can poll: a liveness endpoint at /health/live
(process is up) and a readiness endpoint at /health/ready (the gateway can
reach OM). Disable to remove the endpoints entirely; the bundled Kubernetes
manifests and HAProxy configs probe these paths unconditionally, so disabling
here also requires removing th [...]
+| `ozone.s3g.health-check.probe.interval` | 10s | `OZONE`, `S3GATEWAY`,
`MANAGEMENT` | How often the readiness endpoint refreshes its cached view of OM
reachability in the background. The probe runs off the request path so
/health/ready always responds immediately. |
+| `ozone.s3g.health-check.probe.timeout` | 10s | `OZONE`, `S3GATEWAY`,
`MANAGEMENT` | Upper bound on a single background readiness probe to OM. If a
probe does not complete within this time the gateway is reported as not ready,
so a stuck OM cannot leave readiness stuck at ready. |
| `ozone.s3g.http-address` | 0.0.0.0:9878 | `OZONE`, `S3GATEWAY` | The address
and the base port where the Ozone S3Gateway Server will listen on. |
| `ozone.s3g.http-bind-host` | 0.0.0.0 | `OZONE`, `S3GATEWAY` | The actual
address the HTTP server will bind to. If this optional address is set, it
overrides only the hostname portion of `ozone.s3g.http-address`. This is useful
for making the Ozone S3Gateway HTTP server listen on all interfaces by setting
it to 0.0.0.0. |
| `ozone.s3g.http.auth.kerberos.keytab` | /etc/security/keytabs/HTTP.keytab |
`OZONE`, `S3GATEWAY`, `SECURITY`, `KERBEROS` | The keytab file used by the
S3Gateway server to login as its service principal. |
@@ -682,6 +688,11 @@ This page provides a comprehensive overview of the
configuration keys available
| `ozone.s3g.metrics.percentiles.intervals.seconds` | 60 | `S3GATEWAY`,
`PERFORMANCE` | Specifies the interval in seconds for the rollover of
MutableQuantiles metrics. Setting this interval equal to the metrics sampling
time ensures more detailed metrics. |
| `ozone.s3g.secret.http.auth.type` | kerberos | `S3GATEWAY`, `SECURITY`,
`KERBEROS` | simple or kerberos. If kerberos is set, Kerberos SPNEOGO will be
used for http authentication. |
| `ozone.s3g.secret.http.enabled` | false | `OZONE`, `S3GATEWAY` | The boolean
which enables the Ozone S3Gateway Secret endpoint. |
+| `ozone.s3g.sts.http-address` | 0.0.0.0:9880 | `OZONE`, `S3GATEWAY` | The
HTTP address for the S3 Gateway STS endpoint. |
+| `ozone.s3g.sts.http-bind-host` | 0.0.0.0 | `OZONE`, `S3GATEWAY` | The bind
host for the S3 Gateway STS HTTP server. If this optional address is set, it
overrides only the hostname portion of `ozone.s3g.sts.http-address`. If not
set, the value of `ozone.s3g.http-bind-host` is used. |
+| `ozone.s3g.sts.http.enabled` | false | `OZONE`, `S3GATEWAY` | The boolean
which enables the Ozone S3Gateway STS endpoint. |
+| `ozone.s3g.sts.https-address` | 0.0.0.0:9881 | `OZONE`, `S3GATEWAY` | The
HTTPS address for the S3 Gateway STS endpoint. |
+| `ozone.s3g.sts.https-bind-host` | 0.0.0.0 | `OZONE`, `S3GATEWAY` | The bind
host for the S3 Gateway STS HTTPS server. If this optional address is set, it
overrides only the hostname portion of `ozone.s3g.sts.http-address`. If not
set, the value of `ozone.s3g.https-bind-host` is used. |
| `ozone.s3g.volume.name` | s3v | `OZONE`, `S3GATEWAY` | The volume name to
access through the s3gateway. |
| `ozone.s3g.webadmin.http-address` | 0.0.0.0:19878 | `OZONE`, `S3GATEWAY` |
The address and port where Ozone S3Gateway serves web content. |
| `ozone.s3g.webadmin.http-bind-host` | 0.0.0.0 | `OZONE`, `S3GATEWAY` | The
actual address the HTTP server will bind to. If this optional address is set,
it overrides only the hostname portion of `ozone.s3g.webadmin.http-address`.
This is useful for making the Ozone S3Gateway HTTP server listen on all
interfaces by setting it to 0.0.0.0. |
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
