cameronlee314 commented on a change in pull request #1255: SAMZA-2429: Update
LocalApplicatoinRunner to load full job config from config loader when present.
URL: https://github.com/apache/samza/pull/1255#discussion_r372129266
##########
File path:
samza-core/src/main/java/org/apache/samza/runtime/LocalApplicationRunner.java
##########
@@ -111,29 +112,41 @@ public LocalApplicationRunner(SamzaApplication app,
Config config) {
* @param metadataStoreFactory the instance of {@link MetadataStoreFactory}
to read and write to coordinator stream.
*/
public LocalApplicationRunner(SamzaApplication app, Config config,
MetadataStoreFactory metadataStoreFactory) {
- this(ApplicationDescriptorUtil.getAppDescriptor(app, config),
getCoordinationUtils(config), metadataStoreFactory);
+ this(app, config, getCoordinationUtils(config), metadataStoreFactory);
}
/**
* Constructor only used in unit test to allow injection of {@link
LocalJobPlanner}
*/
@VisibleForTesting
- LocalApplicationRunner(ApplicationDescriptorImpl<? extends
ApplicationDescriptor> appDesc, Optional<CoordinationUtils> coordinationUtils) {
- this(appDesc, coordinationUtils,
getDefaultCoordinatorStreamStoreFactory(new JobConfig(appDesc.getConfig())));
+ LocalApplicationRunner(SamzaApplication app, Config config,
Optional<CoordinationUtils> coordinationUtils) {
+ this(app, config, coordinationUtils,
getDefaultCoordinatorStreamStoreFactory(new JobConfig(config)));
}
private LocalApplicationRunner(
- ApplicationDescriptorImpl<? extends ApplicationDescriptor> appDesc,
+ SamzaApplication app,
+ Config config,
Optional<CoordinationUtils> coordinationUtils,
MetadataStoreFactory metadataStoreFactory) {
- this.appDesc = appDesc;
+ this.appDesc = getApplicationDescriptor(app, config);
this.isAppModeBatch = isAppModeBatch(appDesc.getConfig());
this.coordinationUtils = coordinationUtils;
this.metadataStoreFactory = Optional.ofNullable(metadataStoreFactory);
}
+ @VisibleForTesting
+ static ApplicationDescriptorImpl<? extends ApplicationDescriptor>
getApplicationDescriptor(SamzaApplication app, Config config) {
+ return new JobConfig(config).getConfigLoaderFactory().isPresent()
+ ? ApplicationDescriptorUtil.getAppDescriptor(app,
ConfigUtil.loadConfig(config))
+ : ApplicationDescriptorUtil.getAppDescriptor(app, config);
+ }
+
@VisibleForTesting
static MetadataStoreFactory
getDefaultCoordinatorStreamStoreFactory(JobConfig jobConfig) {
+ if (jobConfig.getConfigLoaderFactory().isPresent()) {
+ jobConfig = new JobConfig(ConfigUtil.loadConfig(jobConfig));
+ }
Review comment:
I do think it is reasonable trade-off.
One of the challenges is that the `LocalApplicationRunner(SamzaApplication
app, Config config)` constructor is overloaded semantically. In one case,
`config` is already the full config, and in another case `config` is a partial
config which needs to be loaded. And then it may seem like we can't use a
single common constructor due to the need to use the full config in the
constructor arguments. I do still think there is a way to share a common
constructor: instead of passing individual arguments, create a
`LocalApplicationRunnerContext` which contains all of the arguments, and then
you can build that other object based on what is given to you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
