[
https://issues.apache.org/jira/browse/GOBBLIN-2042?focusedWorklogId=914714&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-914714
]
ASF GitHub Bot logged work on GOBBLIN-2042:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 15/Apr/24 17:24
Start Date: 15/Apr/24 17:24
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3921:
URL: https://github.com/apache/gobblin/pull/3921#discussion_r1566198718
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/Orchestrator.java:
##########
@@ -117,26 +114,14 @@ public Orchestrator(Config config, TopologyCatalog
topologyCatalog, DagManager d
Optional<DagManagementStateStore> dagManagementStateStore,
FlowCompilationValidationHelper flowCompilationValidationHelper) throws
IOException {
_log = log.isPresent() ? log.get() : LoggerFactory.getLogger(getClass());
- ClassAliasResolver<SpecCompiler> aliasResolver = new
ClassAliasResolver<>(SpecCompiler.class);
this.topologyCatalog = topologyCatalog;
this.dagManager = dagManager;
this.flowStatusGenerator = flowStatusGenerator;
this.flowTriggerDecorator = flowTriggerDecorator;
this.sharedFlowMetricsSingleton = sharedFlowMetricsSingleton;
this.flowCatalog = flowCatalog;
- try {
- String specCompilerClassName =
ServiceConfigKeys.DEFAULT_GOBBLIN_SERVICE_FLOWCOMPILER_CLASS;
- if
(config.hasPath(ServiceConfigKeys.GOBBLIN_SERVICE_FLOWCOMPILER_CLASS_KEY)) {
- specCompilerClassName =
config.getString(ServiceConfigKeys.GOBBLIN_SERVICE_FLOWCOMPILER_CLASS_KEY);
- }
- _log.info("Using specCompiler class name/alias " +
specCompilerClassName);
-
- this.specCompiler = (SpecCompiler)
ConstructorUtils.invokeConstructor(Class.forName(aliasResolver.resolve(specCompilerClassName)),
config);
Review Comment:
Before it would share the same compiler as the orchestrator as it was
initialized differently
Issue Time Tracking
-------------------
Worklog Id: (was: 914714)
Time Spent: 40m (was: 0.5h)
> Fix spec compilation freezing issue on DagActionChangeStoreMonitor
> ------------------------------------------------------------------
>
> Key: GOBBLIN-2042
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2042
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: William Lo
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> This PR [https://github.com/apache/gobblin/pull/3892/files] introduced a bug
> where duplicate flow compilers were being initialized. Each flow compiler
> must be explicitly set to active before it can process any specs otherwise it
> will freeze the service startup.
> The service was experiencing this as only the orchestrator compiler is set to
> being active. We want to consolidate spec compilers (as they can be
> heavyweight) and ensure only one is being initialized.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
