Hello, My setup has asynchronous grpc servers that implement 2 simple rpc calls and asynchronous clients making this calls. Clients create a channel and occasionally make rpc calls, in case of grpc failure they destroy and recreate the channel.
With the time servers stop accepting new connections (Connect failed returned), seemingly running out of available descriptors, and netstat shows most of the connections in CLOSE_WAIT state, here is the quote from netstat output for one of the servers (9484 is the grpc port, currently having over 1300 (and counting) CLOSE_WAIT connections and only 88 ESTABLISHED): tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.217:52670 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.225:61316 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.225:61770 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.225:61784 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.108:39800 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.108:40357 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.110:38224 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.111:61617 CLOSE_WAIT tcp6 0 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.119:53365 ESTABLISHED tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.120:43786 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.127:50726 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.129:56527 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.133:55294 CLOSE_WAIT tcp6 0 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.138:59345 ESTABLISHED tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.138:59450 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.141:46481 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.223:49494 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.227:49438 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.230:59050 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.232:50686 CLOSE_WAIT tcp6 0 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.232:51112 ESTABLISHED tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.232:51152 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.236:36119 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.169:44782 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.174:59332 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.174:59795 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.177:60827 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.177:61279 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.177:61303 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.178:43530 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.180:64340 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.184:59960 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.188:43219 CLOSE_WAIT tcp6 0 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.189:63695 ESTABLISHED tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.191:50126 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.196:63114 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.187:46165 CLOSE_WAIT tcp6 0 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.187:46591 ESTABLISHED tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.187:46614 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.199:35151 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.210:63536 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.219:54092 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.13:40982 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.13:41451 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.15:60025 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.16:49409 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.19:49737 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.23:37534 CLOSE_WAIT tcp6 190 0 YYY.YYY.YYY.YYY:9484 XXX.XXX.XXX.25:55782 CLOSE_WAIT Any idea for such behavior? Thank you in advance! -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/cfbf27c8-4a24-42f9-afac-65a14cf79a9c%40googlegroups.com.
