[
https://issues.apache.org/jira/browse/STORM-3399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936948#comment-16936948
]
Stig Rohde Døssing commented on STORM-3399:
-------------------------------------------
I'm not sure about your number 2. I think the ability to set priority is enough
for this issue. We want to make the spout keep track of emits per partition, so
we can make a decision at poll time whether to poll from a particular partition
or not. For example, that decision might look like "Last 10 minutes have had
90% emits from partition 1, and previous poll was a full batch. This means we
should pause partition 1 for this batch.", and then on the next poll, the
decision could be "Partition 1 is paused, and last 10 minutes have had 87%
emits from partition 1. Last poll was not a full batch, so we should unpause
partition 1".
> Add ability to set priority among partitions in KafkaSpout
> ----------------------------------------------------------
>
> Key: STORM-3399
> URL: https://issues.apache.org/jira/browse/STORM-3399
> Project: Apache Storm
> Issue Type: New Feature
> Components: storm-kafka-client
> Reporter: Stig Rohde Døssing
> Assignee: Rishabh Jain
> Priority: Minor
> Labels: newbie
>
> This idea was raised for the old storm-kafka spout at
> https://issues.apache.org/jira/browse/STORM-3160. As I'm closing that issue,
> I'm raising this one to implement that functionality in the new spout.
> I believe doing this should be possible using KafkaConsumer.pause and
> KafkaConsumer.resume.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
