alirezazamani commented on a change in pull request #741: Fix
ConcurrentModification exception in Workflow Garbage Collection
URL: https://github.com/apache/helix/pull/741#discussion_r377436725
##########
File path: helix-core/src/main/java/org/apache/helix/task/TaskUtil.java
##########
@@ -1043,23 +1043,33 @@ public static void purgeExpiredJobs(String workflow,
WorkflowConfig workflowConf
* @param dataProvider
* @param manager
*/
- public static void workflowGarbageCollection(WorkflowControllerDataProvider
dataProvider,
+ public static void workflowGarbageCollection(final
WorkflowControllerDataProvider dataProvider,
final HelixManager manager) {
// Garbage collections for conditions where workflow context exists but
config is missing.
- Map<String, ZNRecord> contexts = dataProvider.getContexts();
- HelixDataAccessor accessor = manager.getHelixDataAccessor();
- HelixPropertyStore<ZNRecord> propertyStore =
manager.getHelixPropertyStore();
+ // toBeDeletedWorkflows is a set that contains the name of the workflows
that their contexts
+ // should be deleted.
Set<String> toBeDeletedWorkflows = new HashSet<>();
- for (Map.Entry<String, ZNRecord> entry : contexts.entrySet()) {
- if (entry.getValue() != null
- && entry.getValue().getId().equals(TaskUtil.WORKFLOW_CONTEXT_KW)) {
- if (dataProvider.getWorkflowConfig(entry.getKey()) == null) {
- toBeDeletedWorkflows.add(entry.getKey());
+ try {
+ Set<String> existingWorkflowContexts = new
HashSet<>(dataProvider.getContexts().keySet());
+ for (String entry : existingWorkflowContexts) {
+ if (entry != null) {
+ WorkflowConfig cfg = dataProvider.getWorkflowConfig(entry);
+ WorkflowContext ctx = dataProvider.getWorkflowContext(entry);
+ if (ctx != null && ctx.getId().equals(TaskUtil.WORKFLOW_CONTEXT_KW)
&& cfg == null) {
+ toBeDeletedWorkflows.add(entry);
+ }
}
}
+ } catch (Exception e) {
+ LOG.warn(
+ "Exception occurred while creating a list of all existing contexts
with missing config!",
+ e);
}
Review comment:
I don't believe deep copy will help in this case (it would help in the
single threaded case where you want to remove elements from the map or list).
The code generate concurrent modification in this function: map.keySet(). We
cannot avoid this. How you are proposing to get the keys in the map in this
scenario? We need to get the keys no matter what (even for doing deep copy).
Even the implementation of deep copying involves to loop over the elements and
copy them one by one to the new map. A deep copy is merely done by iterating
through the elements (keys and values) and cloning those too. Right? In this
case if original map has been changed while copy operation is happening, we
might still get concurrent modification exception.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
