Hello all, We are excited to announce the release of Apache Ozone 2.1.0!
Apache Ozone 2.1.0 adds 805 new features, improvements and bug fixes on top of Ozone 2.0. Notable Changes: HDDS-10239 | Storage Container Reconciliation Introduced a container reconciliation protocol to resolve mismatched container states and verify replica integrity. HDDS-12940 | Ozone Snapshot Phase 3: Scale up Snapshot Operations Enhanced Ozone Snapshot scalability and continued stability improvements. HDDS-8387 | Container and volume scanners phase II Enhanced storage volume failure handling by introducing a 'degraded' state to preserve data availability, along with improved health observability. HDDS-12564 | Handling disk issues in Datanodes - Phase II Enhanced Datanode disk space management to prevent out-of-space failures. New Configuration Options: OZONE_SERVER_OPTS: A new environment variable was added to allow setting common Java options for all Ozone server processes at once. ozone.compaction.service.enabled: Enable or disable a background job that periodically compacts rocksdb tables flagged for compaction. The default is false. ozone.om.compaction.service.run.interval: A background job that periodically compacts rocksdb tables flagged for compaction. The default is 6 hours. ozone.om.compaction.service.timeout: A timeout value of compaction service. If this is set greater than 0, the service will stop waiting for compaction completion after this time. Unit could be defined with postfix (ns,ms,s,m,h,d). The default is 10 min. ozone.om.compaction.service.columnfamilies: A comma separated, no spaces list of all the column families that are compacted by the compaction service. If this is empty, no column families are compacted. The default is "keyTable,fileTable,directoryTable,deletedTable,deletedDirectoryTable,multipartInfoTable". ozone.om.snapshot.prune.compaction.backup.batch.size: This property controls Prune SST files in Compaction backup directory in batches every ozone.om.snapshot.compaction.dag.prune.daemon.run.interval. The default is 2000. These properties provide more control over the Ratis write buffer, which can be adjusted to optimize performance based on workload and hardware. ozone.om.ratis.server.pending.write.byte-limit: Sets the maximum byte size of all pending write requests. The default is 64MB. ozone.om.ratis.server.pending.write.element-limit: Sets the maximum number of pending write requests. The default is 4096. Incompatible change: HDDS-12152 | Ozone now requires the Hadoop 3.4 runtime. Support for Hadoop 3.1.2 is deprecated. Default Value Changes: hdds.scm.block.deletion.per-interval.max: Default is now 500,000 blocks per interval. hdds.datanode.volume.choosing.policy: Default changed from RoundRobinVolumeChoosingPolicy to CapacityVolumeChoosingPolicy. hdds.datanode.volume.min.free.space: Default changed from 5GB to 20GB. hdds.datanode.volume.min.free.space.percent: Default changed to 0.1%. hdds.datanode.block.deleting.limit.per.interval: Default is now 20,000 blocks per interval. hdds.scm.safemode.min.datanode: Default changed from 1 to 3. hdds.secret.key.expiry.duration: Default is now 9 days. This change ensures that tokens remain valid for their full configured duration, preventing premature key invalidation and improving stable authentication. fs.trash.classname: Class name updated to org.apache.hadoop.fs.ozone.OzoneTrashPolicy. ozone.key.deleting.limit.per.task: Default is now 50,000 keys per task. ozone.s3g.https-address: Default changed from None to 0.0.0.0:9879. ozone.om.fs.snapshot.max.limit: Default is now 10,000 snapshots. ozone.om.ratis.segment.size: Default is now 64MB (was 4MB). ozone.scm.ha.dbtransactionbuffer.flush.interval: Default is now 1 min. ozone.scm.ha.ratis.segment.size: Default is now 64MB (was 4MB). Removed Configurations: hdds.rest.http-address: This configuration is removed since HDDS-12761. hdds.scm.safemode.pipeline-availability.check: This configuration is removed since HDDS-11799. Getting Started: Download Apache Ozone 2.1.0 https://ozone.apache.org/downloads/ Documentation https://ozone.apache.org/docs/2.1.0/ Thank you for your continued support, and we look forward to your feedback on this release!
