Thanks all for fast feedback! It's great news if that aspect is improved as well in new HLC. I will test and get back with any related findings.
Kind regards, Stevo Slavic. On Mon, Jul 20, 2015 at 9:57 PM, Guozhang Wang <[email protected]> wrote: > Hi Stevo, > > I am still not very clear on your point yet, I guess I was trying to figure > out under which circumstances would users prefer to re-set the group id at > an existing consumer rather than creating a new instance. As Jason > mentioned, since the new consumer is single threaded it should usually be > cheap to construct. > > Guozhang > > On Mon, Jul 20, 2015 at 11:06 AM, Jason Gustafson <[email protected]> > wrote: > > > Hey Stevo, > > > > The new consumer doesn't have any threads of its own, so I think > > construction should be fairly cheap. > > > > -Jason > > > > On Sun, Jul 19, 2015 at 2:13 PM, Stevo Slavić <[email protected]> wrote: > > > > > Hello Guozhang, > > > > > > It would be enough if consumer group could, besides at construction > time, > > > be set once only after construction. Have to retest, but high level > > > consumer in 0.8.1 used to be very heavy weight object (lots of threads > > > started, and it would block and take time to construct it). It's > > > understandable, considering all of the high level features it has, and > > > since it's supposed to be long living object. What would improve with > > this > > > change is that construction penalty could be paid upfront, while topic > > > subscription and joining consumer group ensemble could be done on first > > > use, so that first use does not have to suffer from both init and > > > subscription penalties. > > > > > > It would be nice also if consumer group just as subscription could be > > > changed later even, so multiple times throughout lifetime of high level > > > consumer instance, to avoid constructing new consumer when instance > > purpose > > > changes. > > > > > > After looking more into the HLC API, thought maybe this is not needed, > > > since there is "public void subscribe(TopicPartition... partitions)" > > which > > > does not use consumer group management, but problem is that there is no > > > matching explicit commit where one could pass consumer group parameter > as > > > well, to label for which consumer group should offset(s) be committed. > > > > > > Seems like new HLC has split personality. Maybe (at least) two APIs > could > > > have been provided instead of one with such differing behaviors. > > > > > > Kind regards, > > > Stevo Slavic. > > > > > > On Sun, Jul 19, 2015 at 12:01 AM, Guozhang Wang <[email protected]> > > > wrote: > > > > > > > Hi Stevo, > > > > > > > > Hmm this is interesting, do you have any use cases in mind that need > > > > dynamic group changing? > > > > > > > > Guozhang > > > > > > > > On Fri, Jul 17, 2015 at 11:13 PM, Stevo Slavić <[email protected]> > > > wrote: > > > > > > > > > Hello Apache Kafka community, > > > > > > > > > > In new KafkaConsumer API on trunk, it seems it's only possible to > > > define > > > > > consumer group id at construction time of KafkaConsumer, through > > > property > > > > > with group.id key. > > > > > > > > > > Would it make sense and be possible to support setting/changing > > > consumer > > > > > group id after construction, but before it's actually used for the > > > first > > > > > time, similar to how subscription is supported through "public void > > > > > subscribe(String... topics)"? > > > > > > > > > > Maybe this can be done through additional method "public void > > > > > subscribe(String consumerGroupId, String... topics)" which would > > first > > > > set > > > > > provided consumer group id in coordinator and then call "public > void > > > > > subscribe(String... topics)". > > > > > > > > > > Kind regards, > > > > > Stevo Slavic. > > > > > > > > > > > > > > > > > > > > > -- > > > > -- Guozhang > > > > > > > > > > > > > -- > -- Guozhang >
