Hi All, Adding this setting does the trick it seems:
--offset.commit.interval.ms 5000 This defaults to 60,000. Not sure if this has any adverse affects by lowering it to 5 seconds. Cheers! On Thu, Jul 21, 2016 at 10:50 AM, cs user <acldstk...@gmail.com> wrote: > Hi All, > > I've recently enabled ssl for our cluster, and as we are using a mirror > maker I'm now starting our mirror maker processes with the --new.consumer > switch. I can see that now the mirror maker consumer group type has > switched from ZK to KF. > > However I've started to notice a delay when sending messages to the main > clusters and the messages arriving on the mirror maker cluster. Sometimes > this can take upwards of 30 seconds or more for the lag to catch up. > > It seems like there is a delay in the mirror maker process detecting that > new messages are waiting, as it doesn't seem to make a different how big > the lag is, 100 messages, 1000 or 50,000. Once it detects there is a lag it > downloads all the waiting messages fairly instantly. I'd like this delay to > be as minimal as possible though. > > It seems very similar to this: > https://github.com/Jroland/kafka-net/issues/44 > > Does anyone know if there is a similar option with the mirrormaker process? > > Thanks ! > > >
