GitHub user rmatharu opened a pull request:
https://github.com/apache/samza/pull/823
State restore improvements
This PR makes the following changes:
* Consumer consolidation to ensure 1 storeConsumer per system, earlier it
was 1 consumer per SSP per store.
* Refactoring stores to use ContainerStorageManager with parallelization
for restoration, and serial executition of sysConsumers start, stop, register,
etc.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rmatharu/samza consumerConsolidate
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/823.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #823
----
commit 44f7c85e78d12fc7d9c968d881def4b8c1ee3932
Author: Ray Matharu <rmatharu@...>
Date: 2018-11-27T20:03:39Z
Consumer consolidation to ensure 1 storeConsumer per system.
Refactoring stores to use ContainerStorageManager with sequential restore.
Adding parallelization to ContainerStorageManager.
commit 5baa675e89b660597b841f7e3dac8a3323bb07ac
Author: Ray Matharu <rmatharu@...>
Date: 2018-11-27T23:32:41Z
Adding logs and comments to CSM
commit 60d7e30c4e67c9b5a5ee552f2baa238ed4dba6d6
Author: Ray Matharu <rmatharu@...>
Date: 2018-11-27T23:32:55Z
Adding Test for CSM
----
---
