alirezazamani commented on a change in pull request #741: Fix ConcurrentModification exception in Workflow Garbage Collection URL: https://github.com/apache/helix/pull/741#discussion_r377436725
########## File path: helix-core/src/main/java/org/apache/helix/task/TaskUtil.java ########## @@ -1043,23 +1043,33 @@ public static void purgeExpiredJobs(String workflow, WorkflowConfig workflowConf * @param dataProvider * @param manager */ - public static void workflowGarbageCollection(WorkflowControllerDataProvider dataProvider, + public static void workflowGarbageCollection(final WorkflowControllerDataProvider dataProvider, final HelixManager manager) { // Garbage collections for conditions where workflow context exists but config is missing. - Map<String, ZNRecord> contexts = dataProvider.getContexts(); - HelixDataAccessor accessor = manager.getHelixDataAccessor(); - HelixPropertyStore<ZNRecord> propertyStore = manager.getHelixPropertyStore(); + // toBeDeletedWorkflows is a set that contains the name of the workflows that their contexts + // should be deleted. Set<String> toBeDeletedWorkflows = new HashSet<>(); - for (Map.Entry<String, ZNRecord> entry : contexts.entrySet()) { - if (entry.getValue() != null - && entry.getValue().getId().equals(TaskUtil.WORKFLOW_CONTEXT_KW)) { - if (dataProvider.getWorkflowConfig(entry.getKey()) == null) { - toBeDeletedWorkflows.add(entry.getKey()); + try { + Set<String> existingWorkflowContexts = new HashSet<>(dataProvider.getContexts().keySet()); + for (String entry : existingWorkflowContexts) { + if (entry != null) { + WorkflowConfig cfg = dataProvider.getWorkflowConfig(entry); + WorkflowContext ctx = dataProvider.getWorkflowContext(entry); + if (ctx != null && ctx.getId().equals(TaskUtil.WORKFLOW_CONTEXT_KW) && cfg == null) { + toBeDeletedWorkflows.add(entry); + } } } + } catch (Exception e) { + LOG.warn( + "Exception occurred while creating a list of all existing contexts with missing config!", + e); } Review comment: I don't believe deep copy will help in this case (it would help in the single threaded case where you want to remove elements from the map or list). The code generate concurrent modification in this function: map.keySet(). We cannot avoid this. How you are proposing to get the keys in the map in this scenario? We need to get the keys no matter what. Even the implementation of deep copying involves to loop over the elements and copy them one by one to the new map. A deep copy is merely done by iterating through the elements (keys and values) and cloning those too. Right? In this case if original map has been changed while copy operation is happening, we might still get concurrent modification exception. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@helix.apache.org For additional commands, e-mail: reviews-h...@helix.apache.org