[
https://issues.apache.org/jira/browse/KAFKA-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639796#comment-14639796
]
Jason Gustafson commented on KAFKA-2350:
----------------------------------------
I agree that even if we use pause/unpause, we have to solve some of the same
issues. For me, I just feel that mixing the semantics of pause with
subscription is not intuitive. Unsubscribe(partition) means I am no longer
interested in that partition. Pause(partition) means I am still interested, but
I would like to hold off messages for now. These are different concepts, and
I'm not convinced that they are even functionally equivalent. For example, I
expect that I should be able to still call seek(partition) and
position(partition) even if that partition has been paused. Would I still be
able to do that if I had unsubscribed from the partition? In the case of
automatic assignment, perhaps we would say that you can still seek on an
unsubscribed (i.e. paused) partition as long as it's still assigned. But can
you still call seek on an unsubscribed partition when you're not using
automatic assignment? To be consistent, we'd have to say yes, but that just
seems weird. Or if we say no, then we have no way to seek on a paused (umm..
unsubscribed?) position when not using automatic assignment.
> Add KafkaConsumer pause capability
> ----------------------------------
>
> Key: KAFKA-2350
> URL: https://issues.apache.org/jira/browse/KAFKA-2350
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
>
> There are some use cases in stream processing where it is helpful to be able
> to pause consumption of a topic. For example, when joining two topics, you
> may need to delay processing of one topic while you wait for the consumer of
> the other topic to catch up. The new consumer currently doesn't provide a
> nice way to do this. If you skip poll() or if you unsubscribe, then a
> rebalance will be triggered and your partitions will be reassigned.
> One way to achieve this would be to add two new methods to KafkaConsumer:
> {code}
> void pause(String... topics);
> void unpause(String... topics);
> {code}
> When a topic is paused, a call to KafkaConsumer.poll will not initiate any
> new fetches for that topic. After it is unpaused, fetches will begin again.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
