Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/1808#discussion_r96911529
--- Diff:
external/storm-kafka-client/src/main/java/org/apache/storm/kafka/spout/RoundRobinManualPartitioner.java
---
@@ -15,14 +15,26 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
+package org.apache.storm.kafka.spout;
-package org.apache.storm.kafka.spout.internal.partition;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
-import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.TopicPartition;
+import org.apache.storm.task.TopologyContext;
-import java.util.List;
+public class RoundRobinManualPartitioner implements ManualPartitioner {
-public interface KafkaPartitionReader {
- List<TopicPartition> readPartitions(KafkaConsumer<?, ?> consumer);
+ @Override
+ public List<TopicPartition> partition(List<TopicPartition>
allPartitions, TopologyContext context) {
+ int thisTaskIndex = context.getThisTaskIndex();
+ int totalTaskCount =
context.getComponentTasks(context.getThisComponentId()).size();
+ Set<TopicPartition> myPartitions = new
HashSet<>(allPartitions.size()/totalTaskCount+1);
+ for (int i = thisTaskIndex; i < allPartitions.size(); i +=
totalTaskCount) {
--- End diff --
No. `i < allPartitions.size()` guarantees that we will never call get on
allPartitions with an index that is out of bounds. The Set is just setting the
initial size to avoid more memory allocation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
