Github user kaspersorensen commented on the issue:
https://github.com/apache/metamodel/pull/178
A bit of clarification to answer my own question/worry: It seems that the
`Consumer.unsubscribe(..)` call is the best thing I can reasonably do. This
seems to remove any Zookeeper entry for the consumer group. I think. At least
on my own system where I'm testing, I can't see any trace of my consumers in
Zookeeper. But the things that I don't think we can ever get rid of are the
entries/records in the consumer_offsets topic that is internal to Kafka.
I've been thinking that the best possibility would be to try and alleviate
the problem by reusing existing Consumer group IDs. Implemented by using some
pool of group IDs where leases for a group ID are granted when querying and
returned when the DataSet is closed. But then again, that would make concurrent
queries (or even worse - queries from multiple processes or nodes) very hard to
manage because we would never know for sure that another MetaModel client
wasn't _also_ firing a query with the same Consumer group ID. So my conclusion
so far is that it isn't really improvable, except maybe by giving some hooks
back to the user in case he really worries about storing too much consumer
group information.
---
