This is an automated email from the ASF dual-hosted git repository. srichter pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/flink.git
from 5844092408d [FLINK-34000] Remove IncrementalGroupAgg Json Plan & IT tests new 4ddedeb0357 [FLINK-31238] Use IngestDB to speed up Rocksdb rescaling recovery (part 1) new 5495a96a386 [FLINK-31238] Use IngestDB to speed up Rocksdb rescaling recovery (part 2) new 268a30802ed [FLINK-31238] Deactivate parts of the code until new FRocksDB release is available. new 16eae412a12 [FLINK-34199] Add tracing for durations of rescaling/restoring (from local and downloaded remote state). new 6bf97674b66 [FLINK-34199] Add documentation. The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: docs/content.zh/docs/ops/traces.md | 14 +- docs/content/docs/ops/traces.md | 14 +- .../rocksdb_configurable_configuration.html | 12 + .../java/org/apache/flink/util/CollectionUtil.java | 42 + .../org/apache/flink/util/CollectionUtilTest.java | 47 ++ .../apache/flink/runtime/metrics/MetricNames.java | 3 + .../runtime/state/AbstractChannelStateHandle.java | 2 + .../state/AbstractKeyedStateBackendBuilder.java | 2 +- .../flink/runtime/state/DirectoryStateHandle.java | 8 +- .../runtime/state/IncrementalKeyedStateHandle.java | 11 + .../apache/flink/runtime/state/KeyGroupRange.java | 4 + .../flink/runtime/state/StateBackendTestBase.java | 27 +- ...logDelegateEmbeddedRocksDBStateBackendTest.java | 5 + .../flink-statebackend-rocksdb/pom.xml | 19 +- .../state/EmbeddedRocksDBStateBackend.java | 43 +- .../state/RocksDBConfigurableOptions.java | 19 +- .../state/RocksDBIncrementalCheckpointUtils.java | 296 ++++++- .../streaming/state/RocksDBKeyedStateBackend.java | 13 +- .../state/RocksDBKeyedStateBackendBuilder.java | 34 +- .../streaming/state/RocksDBOperationUtils.java | 34 +- .../streaming/state/RocksDBStateDownloader.java | 18 +- .../state/restore/RocksDBFullRestoreOperation.java | 1 + .../streaming/state/restore/RocksDBHandle.java | 29 + .../RocksDBHeapTimersFullRestoreOperation.java | 1 + .../RocksDBIncrementalRestoreOperation.java | 864 ++++++++++++++++----- .../state/restore/RocksDBNoneRestoreOperation.java | 1 + .../state/restore/RocksDBRestoreResult.java | 14 +- .../state/EmbeddedRocksDBStateBackendTest.java | 87 ++- .../streaming/state/RocksDBRecoveryTest.java | 379 +++++++++ .../state/RocksDBStateBackendConfigTest.java | 43 + .../state/RocksDBStateDownloaderTest.java | 17 +- .../contrib/streaming/state/RocksDBTestUtils.java | 35 +- .../RocksIncrementalCheckpointRescalingTest.java | 22 +- .../test/checkpointing/AutoRescalingITCase.java | 14 +- 34 files changed, 1849 insertions(+), 325 deletions(-) create mode 100644 flink-state-backends/flink-statebackend-rocksdb/src/test/java/org/apache/flink/contrib/streaming/state/RocksDBRecoveryTest.java