shanthoosh commented on a change in pull request #1090: SAMZA-2260: Standalone
- coordinator stream metadata store lifecycle
URL: https://github.com/apache/samza/pull/1090#discussion_r304024984
##########
File path:
samza-core/src/main/java/org/apache/samza/runtime/LocalApplicationRunner.java
##########
@@ -240,16 +261,26 @@ CountDownLatch getShutdownLatch() {
}
@VisibleForTesting
- StreamProcessor createStreamProcessor(Config config,
ApplicationDescriptorImpl<? extends ApplicationDescriptor> appDesc,
+ MetadataStore createCoordinatorStreamStore(Config jobConfig) {
+ MetadataStore coordinatorStreamStore =
+ coordinatorMetadataStoreFactory.getMetadataStore("NoOp", jobConfig,
new MetricsRegistryMap());
+ return coordinatorStreamStore;
+ }
+
+ @VisibleForTesting
+ Pair<StreamProcessor, MetadataStore> createStreamProcessor(Config config,
ApplicationDescriptorImpl<? extends ApplicationDescriptor> appDesc,
Review comment:
>> From the looks of the overall code
Take a look at the life-cycle methods in `LocalApplicationRunner`. A
`LocalApplicationRunner` instance can only manage one instance of
`StreamApplication`. Launching multiple `StreamApplication` through a single
`LocalApplicationRunner` will lead to internal-state corruption in
`LocalApplicationRunner`.
Currently samza only supports single stage jobs. Hence the `Set` you're
referring to in `LocalApplicationRunner` will be always of size `1`.
Above point was the motivation behind finding out the rationale behind
`pairing` store with a processor.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
