Can you explain the scenario further? Does the sidecar proxy both the sending and receiving socket (geode creates 2 sockets for each p2p member)? In normal cases, closing these sockets should clear up any unacknowledged messages, freeing up the thread.
Anthony > On Apr 20, 2021, at 7:31 AM, Mario Ivanac <mario.iva...@est.tech> wrote: > > Hi, > > after analysis, we assume that proxy at reception of packets, sends ACK on > TCP level, and after that moment proxy is restarted. > This is the reason, we dont see tcp retries. > > Simular problem to this (but not packet loss), can be reproduce on geode, > if on existing connection, after request is sent, tcp reset is received. In > that case, at reception of reset > connection will be closed, and thread will get stuck while waiting on reply. > I will add reproduction steps in ticket. > > ________________________________ > Šalje: Anthony Baker <bak...@vmware.com> > Poslano: 19. travnja 2021. 22:54 > Prima: dev@geode.apache.org <dev@geode.apache.org> > Predmet: Re: Geode retry/acknowledge improvement > > Do you have a tcpdump that demonstrates the packet loss? How long did you > wait for TCP to retry the failed packet delivery (sometimes this can be > tweaked with tcp_retries2). Does this manifest as a failed socket connection > in geode? That ought to trigger some error handling IIRC. > > Anthony > > >> On Apr 19, 2021, at 7:16 AM, Mario Ivanac <mario.iva...@est.tech> wrote: >> >> Hi all, >> >> we have deployed geode cluster in kubernetes environment, and Istio/SideCars >> are injected between cluster members. >> While running traffic, if any Istio/SideCar is restarted, thread will get >> stuck indefinitely, while waiting for reply on sent message. >> It seams that due to restarting of proxy, in some cases, messages are lost, >> and sending side is waiting indefinitely for reply. >> >> https://issues.apache.org/jira/browse/GEODE-9075 >> >> My question is, what is your estimation, how much effort/work is needed to >> implement message retry/acknowledge logic in geode, >> to solve this problem? >> >> BR, >> Mario >
