I guess adding a new component will increase the complexity of the system structure. And if the new component consists of one or a few nodes, it may becomes the bottleneck of the whole system, if it consists of many nodes, it will make the system even more complex.
Although every solution has its downsides, I think the current one is decent. 2015-05-27 17:10 GMT+08:00 Stevo Slavić <ssla...@gmail.com>: > It could be a separate server component, does not have to be > monolith/coupled with broker. > Such solution would have benefits - single API, pluggable implementations. > > On Wed, May 27, 2015 at 8:57 AM, Shady Xu <shad...@gmail.com> wrote: > > > Storing and managing offsets by broker will leave high pressure on the > > brokers which will affect the performance of the cluster. > > > > You can use the advanced consumer APIs, then you can get the offsets > either > > from zookeeper or the __consumer_offsets__ topic. On the other hand, if > you > > use the simple consumer APIs, you mean to manage offsets yourself, then > you > > should monitor them yourself, simple and plain, right? > > > > 2015-04-22 14:36 GMT+08:00 Stevo Slavić <ssla...@gmail.com>: > > > > > Hello Apache Kafka community, > > > > > > Please correct me if wrong, AFAIK currently (Kafka 0.8.2.x) offset > > > management responsibility is mainly client/consumer side > responsibility. > > > > > > Wouldn't it be better if it was broker side only responsibility? > > > > > > E.g. now if one wants to use custom offset management, any of the Kafka > > > monitoring tools cannot see the offsets - they would need to use same > > > custom client implementation which is practically not possible. > > > > > > Kind regards, > > > Stevo Slavic. > > > > > >
