cameronlee314 commented on a change in pull request #1255: SAMZA-2429: Update
LocalApplicatoinRunner to load full job config from config loader when present.
URL: https://github.com/apache/samza/pull/1255#discussion_r372033107
##########
File path:
samza-core/src/main/java/org/apache/samza/runtime/LocalApplicationRunner.java
##########
@@ -111,29 +112,41 @@ public LocalApplicationRunner(SamzaApplication app,
Config config) {
* @param metadataStoreFactory the instance of {@link MetadataStoreFactory}
to read and write to coordinator stream.
*/
public LocalApplicationRunner(SamzaApplication app, Config config,
MetadataStoreFactory metadataStoreFactory) {
- this(ApplicationDescriptorUtil.getAppDescriptor(app, config),
getCoordinationUtils(config), metadataStoreFactory);
+ this(app, config, getCoordinationUtils(config), metadataStoreFactory);
}
/**
* Constructor only used in unit test to allow injection of {@link
LocalJobPlanner}
*/
@VisibleForTesting
- LocalApplicationRunner(ApplicationDescriptorImpl<? extends
ApplicationDescriptor> appDesc, Optional<CoordinationUtils> coordinationUtils) {
- this(appDesc, coordinationUtils,
getDefaultCoordinatorStreamStoreFactory(new JobConfig(appDesc.getConfig())));
+ LocalApplicationRunner(SamzaApplication app, Config config,
Optional<CoordinationUtils> coordinationUtils) {
+ this(app, config, coordinationUtils,
getDefaultCoordinatorStreamStoreFactory(new JobConfig(config)));
}
private LocalApplicationRunner(
- ApplicationDescriptorImpl<? extends ApplicationDescriptor> appDesc,
+ SamzaApplication app,
+ Config config,
Optional<CoordinationUtils> coordinationUtils,
MetadataStoreFactory metadataStoreFactory) {
- this.appDesc = appDesc;
+ this.appDesc = getApplicationDescriptor(app, config);
this.isAppModeBatch = isAppModeBatch(appDesc.getConfig());
this.coordinationUtils = coordinationUtils;
this.metadataStoreFactory = Optional.ofNullable(metadataStoreFactory);
}
+ @VisibleForTesting
+ static ApplicationDescriptorImpl<? extends ApplicationDescriptor>
getApplicationDescriptor(SamzaApplication app, Config config) {
+ return new JobConfig(config).getConfigLoaderFactory().isPresent()
+ ? ApplicationDescriptorUtil.getAppDescriptor(app,
ConfigUtil.loadConfig(config))
+ : ApplicationDescriptorUtil.getAppDescriptor(app, config);
+ }
+
@VisibleForTesting
static MetadataStoreFactory
getDefaultCoordinatorStreamStoreFactory(JobConfig jobConfig) {
+ if (jobConfig.getConfigLoaderFactory().isPresent()) {
+ jobConfig = new JobConfig(ConfigUtil.loadConfig(jobConfig));
+ }
Review comment:
Should we avoid multiple calls to `ConfigUtil.loadConfig`? You already call
it in `getApplicationDescriptor`.
If the config loader logic is complex/heavy (e.g. makes remote calls), then
that is going to result in inefficient calls.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
