Ok. Let us know if bouncing works. Sounds like a bug thought, that the consumer group is not cleared. It would be helpful to get the broker logs (of the node that hosts the coordinator). Can you also provide a dump of the corresponding partitions of __consumer_offsets topic?
Thanks a lot. -Matthias On 2/16/18 12:35 PM, Russell Teabeault wrote: > Matthias, > > The session.timeout.ms is set to 10000. It has been in this "weird" state > now for going on 24 hours. > We are going to try to bounce the group coordinator. > > Thanks, > -russ > > On Fri, Feb 16, 2018 at 12:18 PM, Matthias J. Sax <[email protected]> > wrote: > >> Hi, >> >> this "weird" state can happen. Usually, a consumer sends a "leave group >> request" to inform the broker it's shutting down. However, this is a >> send-and-forget approach and if the message is lost, the broker might >> think that a stopped consumer is still alive. For this reason, the >> broker should eventually time out consumers and thus clean-up the >> consumer group. >> >> How long did you wait? >> >> Did you modify the `session.timeout.ms` config? >> >> As a last resort, bouncing the broker that hosts the consumer group >> coordinator should resolve the issue. >> >> -Matthias >> >> On 2/15/18 10:09 PM, Russell Teabeault wrote: >>> It appears that kafka and/or kafka streams has gotten itself into a weird >>> state. My KS app is not running. I try to reset it using the reset >> command >>> line tool and I get: >>> >>> ERROR: java.lang.IllegalStateException: Consumer group >>> 'dp-consumption-aggregation-processor-staging2' is still active. Make >> sure >>> to stop all running application instances before running the reset tool. >>> java.lang.IllegalStateException: Consumer group >>> 'dp-consumption-aggregation-processor-staging2' is still active. Make >> sure >>> to stop all running application instances before running the reset tool. >>> >>> But as I said the application is not running. So I ran >>> >>> ./kafka-consumer-groups.sh --bootstrap-server >>> smf1-abi-09-sr1.prod.twitter.com:9092 --group >>> dp-consumption-aggregation-processor-staging2 --describe >>> >>> and the first line is >>> >>> Warning: Consumer group 'dp-consumption-aggregation-processor-staging2' >> is >>> rebalancing. >>> >>> How do I get it out of this state and how did I get it into this state? >>> >>> Thanks! >>> >> >> > >
signature.asc
Description: OpenPGP digital signature
