[
https://issues.apache.org/jira/browse/GOBBLIN-1919?focusedWorklogId=906301&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-906301
]
ASF GitHub Bot logged work on GOBBLIN-1919:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 21/Feb/24 18:28
Start Date: 21/Feb/24 18:28
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3879:
URL: https://github.com/apache/gobblin/pull/3879#discussion_r1498076026
##########
gobblin-utility/src/main/java/org/apache/gobblin/util/ParallelRunner.java:
##########
@@ -86,6 +87,10 @@ public class ParallelRunner implements Closeable {
public static final String PARALLEL_RUNNER_THREADS_KEY =
"parallel.runner.threads";
public static final int DEFAULT_PARALLEL_RUNNER_THREADS = 10;
+ public static int readConfigNumParallelRunnerThreads(Properties props) {
+ return Integer.parseInt(props.getProperty(PARALLEL_RUNNER_THREADS_KEY,
Integer.toString(DEFAULT_PARALLEL_RUNNER_THREADS)));
+ }
+
Review Comment:
Does it make sense to have this config/function when a lot of other classes
pass in a different config to control the number of threads in this class?
Example: Metadata writer defines their own config for how many registration
threads it wants when using this class. Seems to me that this class is just a
generic parallel runner class and can be used in a lot of different areas, so
it makes less sense for it to have its own dedicated config for number of
threads unless it's defining a default.
##########
gobblin-utility/src/main/java/org/apache/gobblin/util/ParallelRunner.java:
##########
@@ -86,6 +87,10 @@ public class ParallelRunner implements Closeable {
public static final String PARALLEL_RUNNER_THREADS_KEY =
"parallel.runner.threads";
public static final int DEFAULT_PARALLEL_RUNNER_THREADS = 10;
+ public static int readConfigNumParallelRunnerThreads(Properties props) {
+ return Integer.parseInt(props.getProperty(PARALLEL_RUNNER_THREADS_KEY,
Integer.toString(DEFAULT_PARALLEL_RUNNER_THREADS)));
+ }
+
Review Comment:
We can have it as `getDefaultParallelRunnerConcurrency` ?
Issue Time Tracking
-------------------
Worklog Id: (was: 906301)
Time Spent: 1h 40m (was: 1.5h)
> Simplify impl related to MR job execution (before reusing code in
> Temporal-based exec)
> --------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1919
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1919
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-core
> Reporter: Kip Kohn
> Assignee: Abhishek Tiwari
> Priority: Trivial
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)