[
https://issues.apache.org/jira/browse/GOBBLIN-2042?focusedWorklogId=914703&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-914703
]
ASF GitHub Bot logged work on GOBBLIN-2042:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 15/Apr/24 17:03
Start Date: 15/Apr/24 17:03
Worklog Time Spent: 10m
Work Description: umustafi commented on code in PR #3921:
URL: https://github.com/apache/gobblin/pull/3921#discussion_r1566169250
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/orchestration/Orchestrator.java:
##########
@@ -117,26 +114,14 @@ public Orchestrator(Config config, TopologyCatalog
topologyCatalog, DagManager d
Optional<DagManagementStateStore> dagManagementStateStore,
FlowCompilationValidationHelper flowCompilationValidationHelper) throws
IOException {
_log = log.isPresent() ? log.get() : LoggerFactory.getLogger(getClass());
- ClassAliasResolver<SpecCompiler> aliasResolver = new
ClassAliasResolver<>(SpecCompiler.class);
this.topologyCatalog = topologyCatalog;
this.dagManager = dagManager;
this.flowStatusGenerator = flowStatusGenerator;
this.flowTriggerDecorator = flowTriggerDecorator;
this.sharedFlowMetricsSingleton = sharedFlowMetricsSingleton;
this.flowCatalog = flowCatalog;
- try {
- String specCompilerClassName =
ServiceConfigKeys.DEFAULT_GOBBLIN_SERVICE_FLOWCOMPILER_CLASS;
- if
(config.hasPath(ServiceConfigKeys.GOBBLIN_SERVICE_FLOWCOMPILER_CLASS_KEY)) {
- specCompilerClassName =
config.getString(ServiceConfigKeys.GOBBLIN_SERVICE_FLOWCOMPILER_CLASS_KEY);
- }
- _log.info("Using specCompiler class name/alias " +
specCompilerClassName);
-
- this.specCompiler = (SpecCompiler)
ConstructorUtils.invokeConstructor(Class.forName(aliasResolver.resolve(specCompilerClassName)),
config);
Review Comment:
were there two specCompiler's being initialized before? One in the
orchestrator and one in validation helper?
Issue Time Tracking
-------------------
Worklog Id: (was: 914703)
Remaining Estimate: 0h
Time Spent: 10m
> Fix spec compilation freezing issue on DagActionChangeStoreMonitor
> ------------------------------------------------------------------
>
> Key: GOBBLIN-2042
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2042
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: William Lo
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This PR [https://github.com/apache/gobblin/pull/3892/files] introduced a bug
> where duplicate flow compilers were being initialized. Each flow compiler
> must be explicitly set to active before it can process any specs otherwise it
> will freeze the service startup.
> The service was experiencing this as only the orchestrator compiler is set to
> being active. We want to consolidate spec compilers (as they can be
> heavyweight) and ensure only one is being initialized.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
