A few things: what version of the Thrift Go lib are you using (commit SHA)? Also, what protocol and transports?
I've seen similar issues before with Go. One was due to the fact that the Go lib uses bufio.Readers and bufio.Writers in a few places which, if an error occurs when making calls on them, puts them in a bad state until they are reset (or a new transport/protocol is created). The other issue I've seen is due to the fact that the framed transport uses the same byte buffer for reads and writes. If, for any reason, reads and writes are happening concurrently, this will cause very subtle errors (e.g. incorrect frame size). On Mon, Apr 11, 2016 at 5:38 AM Murat Knecht <[email protected]> wrote: > Hello, > > we've a few Golang services connected with Thrift and are running into a > peculiar problem every week or so: Calls to one particular service would > fail with: > > Notenoughframesize0toread4bytes > > or > > outofsequenceresponse > Incorrectframesize(251658252) > > The weird thing is that most API endpoints of that service are working > fine, can be called and will respond properly. Only two methods are, > when they degrade, down for good. Until we restart the process. > > The one thing that makes these two API endpoints special (apart from the > errors): The service calls another Thrift-powered service to retrieve > data — which is running in the same process. So, we have two services > exposing Thrift endpoints on different ports, but running in the same > process (for good, old historic reasons), with one of those services > calling the other. > > Does that ring a bell with anyone familiar with the Thrift-Go > implementation? Is this obviously a bad idea? Is there some shared state > that would explain why sometimes the framebuffers leak memory / degrade? > This happens sporadically (usually Friday night, for the fun of it) so I > won't be able to provide a minimal code sample to reproduce this. > > Any thoughts / ideas? > > Thanks, > Murat > > >
