tedyu commented on code in PR #9904:
URL: https://github.com/apache/gravitino/pull/9904#discussion_r2801630891
##########
maintenance/optimizer/src/main/java/org/apache/gravitino/maintenance/optimizer/recommender/job/GravitinoJobSubmitter.java:
##########
@@ -79,11 +89,67 @@ public void close() throws Exception {
}
}
+ private void ensureClientInitialized() {
+ if (gravitinoClient == null) {
+ if (optimizerEnv == null) {
+ throw new IllegalStateException("Job submitter is not initialized");
+ }
+ this.gravitinoClient = GravitinoClientUtils.createClient(optimizerEnv);
+ }
+ }
+
+ /**
+ * Merge job configs with precedence: optimizer config < adapter config.
+ *
+ * <p>Typical use cases:
+ *
+ * <ul>
+ * <li>Optimizer config: shared engine/runtime defaults (for example,
Spark settings).
+ * <li>Adapter config: adapter-specific parameters (for example, WHERE
filters) required by the
+ * job template.
+ * </ul>
+ */
+ @VisibleForTesting
+ static Map<String, String> buildJobConfig(
+ OptimizerConfig optimizerConfig,
+ JobExecutionContext jobExecutionContext,
+ GravitinoJobAdapter jobAdapter) {
+ Map<String, String> submitterConfigs =
+ optimizerConfig == null ? Map.of() :
optimizerConfig.jobSubmitterConfigs();
+ Map<String, String> adapterConfigs =
+ jobAdapter == null ? Map.of() :
jobAdapter.jobConfig(jobExecutionContext);
+
+ Map<String, String> mergedConfigs = new LinkedHashMap<>();
+ mergedConfigs.putAll(submitterConfigs);
Review Comment:
If either input map is being modified concurrently with the merging (since
`ConcurrentMap` is used, I assume concurreny is possible), we may get
inconsistent results.
If two entries, (key_a->val_a) and (key_b->val_b), are modified (assuming
both keys may appear in both maps).
The update of `key_a` and `key_b` in one input map may only be partially
reflected in the merged map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
