[
https://issues.apache.org/jira/browse/GOBBLIN-1919?focusedWorklogId=906306&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-906306
]
ASF GitHub Bot logged work on GOBBLIN-1919:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 21/Feb/24 20:06
Start Date: 21/Feb/24 20:06
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3879:
URL: https://github.com/apache/gobblin/pull/3879#discussion_r1498166313
##########
gobblin-utility/src/main/java/org/apache/gobblin/util/ParallelRunner.java:
##########
@@ -86,6 +87,10 @@ public class ParallelRunner implements Closeable {
public static final String PARALLEL_RUNNER_THREADS_KEY =
"parallel.runner.threads";
public static final int DEFAULT_PARALLEL_RUNNER_THREADS = 10;
+ public static int readConfigNumParallelRunnerThreads(Properties props) {
+ return Integer.parseInt(props.getProperty(PARALLEL_RUNNER_THREADS_KEY,
Integer.toString(DEFAULT_PARALLEL_RUNNER_THREADS)));
+ }
+
Review Comment:
the primary reason I moved it here is that `MRJobRunner` [was referencing
two
configs](https://github.com/apache/gobblin/pull/3879/files#diff-4be3972775f1f53766134530e51058f1efbcb527dae787b37225865dd529c8bbL258)
both defined in this class.
we can certainly rename. I'd prefer not "default", just because the
*default value* is only used as a fallback, in the case of no specific config.
as for multiple classes all leveraging this utility, while wanting their own
setting, I recommend hierarchical naming, a la typesafe `Config`s -
```
Sting thisClassConfigPrefix = getClass().getName(); // potential
convention...
Config thisClassConfigs = ConfigUtils.getConfig(myConfig,
thisClassConfigPrefix, ConfigFactory.empty());
int numThreads = ParallelRunner.getNumThreadsConfig(thisClassConfigs);
```
at the moment I'd continue using `Properties`, since my objective was just
to consolidate cross-class references... but if we find more widespread use, we
could update to take `Config` instead.
Issue Time Tracking
-------------------
Worklog Id: (was: 906306)
Time Spent: 1h 50m (was: 1h 40m)
> Simplify impl related to MR job execution (before reusing code in
> Temporal-based exec)
> --------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1919
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1919
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-core
> Reporter: Kip Kohn
> Assignee: Abhishek Tiwari
> Priority: Trivial
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)