I don't see the logs attached, but what does the GC look like in your applications? A lot of times this is caused (at least on the consumer side) by the Zookeeper session expiring due to excessive GC activity, which causes the consumers to go into a rebalance and change up their connections.
-Todd On Fri, Sep 25, 2015 at 1:25 PM, Gwen Shapira <g...@confluent.io> wrote: > How busy are the clients? > > The brokers occasionally close idle connections, this is normal and > typically not something to worry about. > However, this shouldn't happen to consumers that are actively reading data. > > I'm wondering if the "consumers not making any progress" could be due to a > different issue, and because they are idle, the connection closes (vs the > other way around). > > On Thu, Sep 24, 2015 at 2:32 PM, noah <iamn...@gmail.com> wrote: > > > We are having issues with producers and consumers frequently fully > > disconnecting (from both the brokers and ZK) and reconnecting without any > > apparent cause. On our production systems it can happen anywhere from > every > > 10-15 seconds to 15-20 minutes. On our less beefy test systems and > > developer laptops, it can happen almost constantly. > > > > We see no errors in the logs (sample attached), just a message for each > of > > our our consumers and producers disconnecting, then reconnecting. On the > > systems where it happens constantly, the consumers are not making any > > progress. > > > > The logs on the brokers are equally unhelpful, they show only frequent > > connects and reconnects, without any apparent cause. > > > > What could be causing this behavior? > > > > >
