Hi Nacho, yes, exactly.
On Mon, 31 Oct 2016 at 22:10 Nacho Solis <nso...@linkedin.com.invalid> wrote: > Hi Andrew. > > Is this what you're saying: > - sometimes you get stuck (like a blocking call) when you call some > function in the kafka client > - you can go around this case (by wrapping it) such that if the kafka > client call gets stuck your software doesn't get stuck > - you're wondering why the kafka client is not more resilient to the broker > being unavailable/down > > Is this correct? > > Nacho > > On Mon, Oct 31, 2016 at 1:54 PM, Andrey Dyachkov < > andrey.dyach...@gmail.com> > wrote: > > > Hi Becket, > > > > I asked the same question in the user mailing list and we had a small > > discussion there. > > > > We have default timeout for producer and consumer it is 30 secs. > > If Kafka is not available we will wait 30 secs on each request. > > Decreasing timeout won't help much because we have a lot requests > > but circuiting them will help us tremendous. We could implement such > > logic before actually calling Kafka client but there is problem what the > > trigger is to open the circuit. I thought about circuiting by unavailable > > broker > > but it is quite an expansive operation to figure out in the client's code > > on the flip side Kaka client already does all the job to make out the > > broker. > > > > For us very important to stay live even if Kafka is unavailable (it > > happens) that's why I think avoiding cascading issues will be very > useful. > > > > The circuit breaker here is a thing which can stop requests to Kafka for > > certain period of time until a specific topic/partition/broker is not > > available. > > > > On Mon, 31 Oct 2016 at 21:38 Becket Qin <becket....@gmail.com> wrote: > > > > Hi Andrey, > > > > I am not sure I understand the question "why doesn't Kafka client have > > the solution > > for this kind of problems?" Can you elaborate a little on what kind of > > problems are you referring to? And what do you mean by "circuit breaker" > > here? > > > > Jiangjie (Becket) Qin > > > > On Fri, Oct 28, 2016 at 12:07 PM, Andrey Dyachkov < > > andrey.dyach...@gmail.com > > > wrote: > > > > > Hi Kafka developers :) > > > > > > We use Kafka for around half a year as a main backend system for > storing > > > events from big variety of internal microservices. In general It works > > > quite well and we experience issues only in some rare 'corner cases' > > where > > > we either run against a bug or do not understand why Kafka behaves in > > > particular manner. The fact Kafka is the main system is used under our > > > software If it fails our system will fail too. That's why I am writing > to > > > ask about the solution which can help us tolerate different issues on > > Kafka > > > side. First of all could you tell me why doesn't Kafka client have the > > > solution for this kind of problems? Have you ever had thoughts about > it? > > > Thank you in advance! > > > -- > > > > > > With great enthusiasm, > > > Andrey > > > > > > > -- > > > > With great enthusiasm, > > Andrey > > > > > > -- > Nacho (Ignacio) Solis > Kafka > nso...@linkedin.com > -- With great enthusiasm, Andrey
