[
https://issues.apache.org/jira/browse/GOBBLIN-2107?focusedWorklogId=925170&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-925170
]
ASF GitHub Bot logged work on GOBBLIN-2107:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 10/Jul/24 00:27
Start Date: 10/Jul/24 00:27
Worklog Time Spent: 10m
Work Description: umustafi commented on PR #3996:
URL: https://github.com/apache/gobblin/pull/3996#issuecomment-2219154425
> [I pointed
out](https://github.com/apache/gobblin/pull/3944/files#r1591517872) in my
earlier PR (you linked to) that we're missing test cases. I had added ones
relating to my change. what would it take to exercise any of the bug fixes that
motivate these latest mods?
I added a unit test `testDeleteFlowSpecCalledForMultiActivePath` that
validates that the `compileAndSubmitFlowToDagManager` used by the
`DagActionStoreChangeMonitor` attempts deleting adhoc `flowSpecs` from the
catalog. Note that this test does not definitively ensure that this method is
called by the `DagActionStoreChangeMonitor` which proves more complex but
verifies if that method is used then the flow will be deleted after compilation.
Issue Time Tracking
-------------------
Worklog Id: (was: 925170)
Time Spent: 1h (was: 50m)
> Delete adhoc flowSpecs from flowCatalog
> ----------------------------------------
>
> Key: GOBBLIN-2107
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2107
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Delete adhoc flowSpecs from flowCatalog to avoid build up of adhoc flowSpecs
> in catalog even when job compilation fails by adding the deletion in a
> finally block in non-MA scheduler case.
> # Previous PR: [https://github.com/apache/gobblin/pull/3944/files] causes a
> bug where adhoc flows are not deleted when multi-active scheduler is enabled.
> This PR re-adds deletion of flowSpec to the dagManager to be called by the
> active host for the MA scheduler case (there will only be one active host if
> dagManager is enabled otherwise code will use dagProcessingEngine).
> # It also ensures quota is released and failed flow compilation event sent
> by the FlowCompilationValidationHelper in all compilation failure cases. This
> was being incorrectly handled before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
