[ https://issues.apache.org/jira/browse/STORM-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Li updated STORM-2940: ---------------------------- Summary: Set the CAPACITY value of LoadAwareShuffleGrouping dynamically (was: Increase the CAPACITY value in LoadAwareShuffleGrouping) > Set the CAPACITY value of LoadAwareShuffleGrouping dynamically > -------------------------------------------------------------- > > Key: STORM-2940 > URL: https://issues.apache.org/jira/browse/STORM-2940 > Project: Apache Storm > Issue Type: Bug > Reporter: Ethan Li > Assignee: Ethan Li > Priority: Major > Labels: pull-request-available > Fix For: 2.0.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > We have seen exceptions from workers > {code:java} > java.lang.IllegalArgumentException: bound must be positive > {code} > The stack trace points to LoadAwareShuffleGrouping.java#L234: > {code:java} > //in case we didn't fill in enough > for (; currentIdx < CAPACITY; currentIdx++) { > prepareChoices[currentIdx] = prepareChoices[random.nextInt(currentIdx)]; > } > {code} > This is because in some situation, > {code:java} > int count = (int) ((indexAndWeights.weight / (double) weightSum) * CAPACITY); > {code} > the above code will be 0 during the whole for-loop. For example, when there > are more than 1000 (which is CAPACITY) downstream tasks, say 1001, and each > task has weight 1. Then for every target task, count =(int) ( (1 / 1001.0) * > 1000) = 0. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)