tedyu commented on code in PR #9904:
URL: https://github.com/apache/gravitino/pull/9904#discussion_r2801630891
##########
maintenance/optimizer/src/main/java/org/apache/gravitino/maintenance/optimizer/recommender/job/GravitinoJobSubmitter.java:
##########
@@ -79,11 +89,67 @@ public void close() throws Exception {
}
}
+ private void ensureClientInitialized() {
+ if (gravitinoClient == null) {
+ if (optimizerEnv == null) {
+ throw new IllegalStateException("Job submitter is not initialized");
+ }
+ this.gravitinoClient = GravitinoClientUtils.createClient(optimizerEnv);
+ }
+ }
+
+ /**
+ * Merge job configs with precedence: optimizer config < adapter config.
+ *
+ * <p>Typical use cases:
+ *
+ * <ul>
+ * <li>Optimizer config: shared engine/runtime defaults (for example,
Spark settings).
+ * <li>Adapter config: adapter-specific parameters (for example, WHERE
filters) required by the
+ * job template.
+ * </ul>
+ */
+ @VisibleForTesting
+ static Map<String, String> buildJobConfig(
+ OptimizerConfig optimizerConfig,
+ JobExecutionContext jobExecutionContext,
+ GravitinoJobAdapter jobAdapter) {
+ Map<String, String> submitterConfigs =
+ optimizerConfig == null ? Map.of() :
optimizerConfig.jobSubmitterConfigs();
+ Map<String, String> adapterConfigs =
+ jobAdapter == null ? Map.of() :
jobAdapter.jobConfig(jobExecutionContext);
+
+ Map<String, String> mergedConfigs = new LinkedHashMap<>();
+ mergedConfigs.putAll(submitterConfigs);
Review Comment:
If either input map is being modified concurrently (since `ConcurrentMap` is
used), we may get inconsistent results.
Suppose (key_a->val_a) is related to (key_b->val_b) (assuming both keys may
appear in both maps).
The update of `key_a` and `key_b` in one map may only be partially reflected
in the merged map.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
