I’ve found it that if a timeout occurs on the client then its a toss up on if the communication stack will recover. So, if anything bad happens I tear the whole thing down and bring it back up, usually pointing at a different server in the cluster. If the call is idempotent I automatically retry the same API call on the new server. So most of the time people don’t even know. Most timeouts occur when we’re doing rolling restarts of the servers. If I run out of servers I just reset the list and keep trying until they come back.
> On Oct 30, 2015, at 10:10 AM, Tyler Treat <[email protected]> wrote: > > I guess my main question then is what the best practice is when dealing > with timeouts. It sounds like if a request times out, you should disconnect > the client and reconnect, which is an unfortunate amount of overhead for > what is potentially a transient error. > > Thanks, > Tyler > > On Fri, Oct 30, 2015 at 2:54 AM Jens Geyer <[email protected]> wrote: > >> The single most important thing that comes to my mind is this: What if the >> client is aware of this and *wants* both responses? >> >> If the framework automatically and implicitly drops older requests or >> refuses to deal with responses coming in a different order, how do you >> handle that? But if that logic is one layer above Thrift, it is much easier >> implement whatever model you need. >> >> There are situations for each model, IMHO. If we plan to do anything here, >> we should add comfort, but keep all possibilities. >> >> Have fun, >> JensG >> ________________________________ >> Von: Tyler Treat >> Gesendet: 30.10.2015 01:06 >> An: [email protected] >> Betreff: Dealing with out of sequence responses >> >> Hi, >> >> I'm curious about Thrift's design around out of sequence responses. >> Consider the sequence of events in this contrived example: >> >> Client calls ping() >> Server receives ping() and sleeps a few seconds >> Client ping() times out >> Client calls add() >> Server finishes sleeping and responds to ping() >> Client gets out of sequence response error (received ping() response when >> expecting add() response) >> >> It seems like what Thrift should do is drop responses with previous >> sequence ids? I.e. if ping() sequence id is 1 and it times out and I call >> add() which expects sequence id 2, I should drop the response with id 1. >> >> Am I thinking about this correctly? >> >> Thanks, >> Tyler >>
