shanthoosh commented on a change in pull request #1090: SAMZA-2260: Standalone
- coordinator stream metadata store lifecycle
URL: https://github.com/apache/samza/pull/1090#discussion_r303577335
##########
File path: samza-core/src/main/java/org/apache/samza/zk/ZkJobCoordinator.java
##########
@@ -308,38 +312,32 @@ JobModel readJobModelFromMetadataStore(String
zkJobModelVersion) {
*/
@VisibleForTesting
void loadMetadataResources(JobModel jobModel) {
- CoordinatorStreamStore coordinatorStreamStore = null;
try {
- // Creates the coordinator stream if it does not exists.
- coordinatorStreamStore = createCoordinatorStreamStore();
- coordinatorStreamStore.init();
-
MetadataResourceUtil metadataResourceUtil =
createMetadataResourceUtil(jobModel, getClass().getClassLoader());
metadataResourceUtil.createResources();
- CoordinatorStreamValueSerde jsonSerde = new
CoordinatorStreamValueSerde(SetConfig.TYPE);
- NamespaceAwareCoordinatorStreamStore configStore =
- new NamespaceAwareCoordinatorStreamStore(coordinatorStreamStore,
SetConfig.TYPE);
- for (Map.Entry<String, String> entry : config.entrySet()) {
- byte[] serializedValue = jsonSerde.toBytes(entry.getValue());
- configStore.put(entry.getKey(), serializedValue);
- }
+ if (coordinatorStreamStore != null) {
+ CoordinatorStreamValueSerde jsonSerde = new
CoordinatorStreamValueSerde(SetConfig.TYPE);
Review comment:
We'd discussed offline that publishing the heavy-weight configuration bag
one-by-one through metadata-store API in leader-startup sequence had
significantly slowed down the processor startup. Please create a JIRA ticket
for it and add a TODO here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
