This is an automated email from the ASF dual-hosted git repository.
pmaheshwari pushed a change to branch state-backend-async-commit
in repository https://gitbox.apache.org/repos/asf/samza.git.
from c2b77f0 Fix StorageConfig bug from newly introduced BlobStoreRestore
factory config (#1503)
new 354620f SnapshotIndexSerde serialization fixes
new 48a8785 Checkpoint-tool changes for checkpoint v2
new 2063155 HOT FIX: KafkaConsumerProxy without any registered
TopicPartitions
new 271ef64 Fix async cleanup TaskMetric
new 7d61a7d Added lastCommitNs and commitTimedOut metrics, updated
default timeout to 30 mins
new 7f5ff3a Updated log line in BlobStoreUtil
new 35c0dbc Checkpoint version config precedence list
new 7676da9 Per store restore manager config precendence
new 5379f97 Job level state backend configs
new 0697457 Fixed case where there are no stores to backup for CSM
new e74c241 Added tests for CSM for non configured stores
new ad408d5 Cleanup of TODOs
new 785d506 checkstype cleanup
new 3c8a462 checkstyle for samza-api
new 65b5df6 Test fix for failing job: only backup persistent stores
new 48699c4 Merge pull request #1512 from
dxichen/checkpoint-restore-precendence-cleanup
The 2358 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../samza/serializers/JsonCheckpointIdMixin.java | 42 ----
.../samza/storage/KafkaChangelogRestoreParams.java | 10 +-
.../apache/samza/storage/StateBackendFactory.java | 2 +
.../apache/samza/storage/TaskRestoreManager.java | 2 +-
.../org/apache/samza/config/StorageConfig.java | 98 +++++++--
.../java/org/apache/samza/config/TaskConfig.java | 18 +-
.../apache/samza/startpoint/StartpointManager.java | 2 +-
.../storage/KafkaChangelogStateBackendFactory.java | 5 +-
.../NonTransactionalStateTaskRestoreManager.java | 1 -
.../apache/samza/storage/StorageManagerUtil.java | 9 +-
.../org/apache/samza/storage/StorageRecovery.java | 9 +-
.../TransactionalStateTaskRestoreManager.java | 2 +-
.../storage/blobstore/BlobStoreBackupManager.java | 2 +-
.../storage/blobstore/BlobStoreRestoreManager.java | 16 +-
.../blobstore/BlobStoreStateBackendFactory.java | 4 +-
.../blobstore/index/serde/SnapshotIndexSerde.java | 8 +-
.../storage/blobstore/util/BlobStoreUtil.java | 4 +-
.../apache/samza/checkpoint/CheckpointTool.scala | 30 ++-
.../file/FileSystemCheckpointManager.scala | 3 +
.../apache/samza/container/SamzaContainer.scala | 40 ++--
.../org/apache/samza/container/TaskInstance.scala | 10 +-
.../samza/container/TaskInstanceMetrics.scala | 3 +-
.../samza/storage/ContainerStorageManager.java | 169 +++++++++++----
.../org/apache/samza/config/TestStorageConfig.java | 191 +++++++++++++++--
.../blobstore/TestBlobStoreRestoreManager.java | 11 +-
.../samza/checkpoint/TestCheckpointTool.scala | 54 ++++-
.../apache/samza/container/TestTaskInstance.scala | 34 ++-
.../samza/storage/TestContainerStorageManager.java | 236 ++++++++++++++++++++-
.../checkpoint/kafka/KafkaCheckpointLogKey.java | 6 +
.../samza/system/kafka/KafkaConsumerProxy.java | 5 +-
.../checkpoint/kafka/KafkaCheckpointManager.scala | 49 ++++-
.../samza/storage/TestTaskStorageManager.scala | 7 +-
.../kafka/TestKafkaCheckpointManager.scala | 55 ++++-
.../kv/BaseKeyValueStorageEngineFactory.java | 4 +-
.../CheckpointVersionIntegrationTest.java | 2 +-
35 files changed, 916 insertions(+), 227 deletions(-)
delete mode 100644
samza-api/src/main/java/org/apache/samza/serializers/JsonCheckpointIdMixin.java
