[ https://issues.apache.org/jira/browse/GOBBLIN-2173?focusedWorklogId=944292&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-944292 ]
ASF GitHub Bot logged work on GOBBLIN-2173: ------------------------------------------- Author: ASF GitHub Bot Created on: 19/Nov/24 09:54 Start Date: 19/Nov/24 09:54 Worklog Time Spent: 10m Work Description: vsinghal85 commented on code in PR #4076: URL: https://github.com/apache/gobblin/pull/4076#discussion_r1848018884 ########## gobblin-service/src/test/java/org/apache/gobblin/service/modules/orchestration/OrchestratorTest.java: ########## @@ -370,16 +376,18 @@ public void onAddSpecForScheduledFlow() throws IOException { FlowSpec flowSpec = FlowSpec.builder().withConfig(config).build(); AddSpecResponse response = new AddSpecResponse<>(new Object()); Mockito.when(specCompiler.onAddSpec(flowSpec)).thenReturn(response); - AddSpecResponse addSpecResponse = dagMgrNotFlowLaunchHandlerBasedOrchestrator.onAddSpec(flowSpec); + AddSpecResponse addSpecResponse = orchestrator.onAddSpec(flowSpec); Assert.assertNotNull(addSpecResponse); + // Verifying that for scheduled flow isLeaseAcquirable is not called + Mockito.verify(dagManagementStateStore, Mockito.times(0)).existsCurrentlyLaunchingSimilarFlow(anyString(), anyString(), anyLong()); Review Comment: Sure, Thanks for pointing out, updated Issue Time Tracking ------------------- Worklog Id: (was: 944292) Time Spent: 3h (was: 2h 50m) > Adhoc flows are not being deleted from GaaS FlowSpec store > ---------------------------------------------------------- > > Key: GOBBLIN-2173 > URL: https://issues.apache.org/jira/browse/GOBBLIN-2173 > Project: Apache Gobblin > Issue Type: Bug > Components: gobblin-service > Reporter: Abhishek Jain > Assignee: Abhishek Tiwari > Priority: Critical > Time Spent: 3h > Remaining Estimate: 0h > > In GaaS, we store adhoc flows temporarily in our flowspec DB in order to > persist them in service restart/failover scenarios. However, it is expected > that once these flows are kicked off/ forwarded to the DagProcEngine, they > need to be removed from our flowspec db. > This is currently not consistently happening, there seems to be some edge > case(s) where they are persisted in the db. This can be fatal for users such > as DIL that run adhoc flows using the same flowgroup/flowname consistently, > which will lead to their flows being stuck. We need to find which edge cases > are not handling the flow spec deletion properly. -- This message was sent by Atlassian Jira (v8.20.10#820010)