Ok I think i have a clue, looking at full log file for fd_create for that port, I see that the time between retries is monotonically increasing (doubling) even though I have the WaitForConnection deadline static. How can I disable this feature?
On Friday, January 18, 2019 at 9:44:13 AM UTC-5, [email protected] wrote: > > Yes start of file is about when the server connection goes up at the same > time client software is trying to connect using loop above, > so 12:54:24.361342246, fd_create in question is at 12:54:32.698523276, and > connection happens shortly after. Thanks for looking. > > > > On Friday, January 18, 2019 at 9:39:18 AM UTC-5, Robert Engels wrote: >> >> Can you identify the time this occurs in the file? It’s fairly large. >> >> On Jan 18, 2019, at 8:35 AM, [email protected] wrote: >> >> Yes see previous message: connect.txt >> >> On Friday, January 18, 2019 at 9:23:03 AM UTC-5, Robert Engels wrote: >>> >>> Is the debug file attached ? >>> >>> On Jan 18, 2019, at 8:18 AM, Brian Wagener <[email protected]> >>> wrote: >>> >>> I don't think it is either of those. I was able to capture grpc debug >>> and it shows that the actual fd_create to Port 50074 doesn't even get >>> called in this case until 8 seconds after trying to connect. Can someone >>> look at debug file and see what grpc is doing? >>> >>> Brian >>> >>> On Thu, Jan 17, 2019, 3:09 PM robert engels <[email protected]> >>> wrote: >>> >>>> If you are running a tight loop with lots of connection attempts there >>>> are a lot of reasons it can fail. usually resources (number of >>>> connections) >>>> - so while the OS is waiting to close the existing connections, future >>>> attempts will fail. >>>> >>>> On Jan 17, 2019, at 1:54 PM, [email protected] wrote: >>>> >>>> I don't really understand the question, but I have tested retry by >>>> just starting and stopping server. >>>> >>>> >>>> On Thursday, January 17, 2019 at 2:46:44 PM UTC-5, robert engels wrote: >>>>> >>>>> How are you testing the retry - pulling plug? iptables ? >>>>> >>>>> On Jan 17, 2019, at 1:39 PM, [email protected] wrote: >>>>> >>>>> >>>>> I have a gRPC streaming client, that has to handle server going up and >>>>> down, so I have a while loop, but sometimes it works fine, but other >>>>> times >>>>> it takes 15 seconds to connect even on the same machine. Is it something >>>>> wrong with my code, or how can I debug? As you can see below I have debug >>>>> to print out channel state, and is mostly GRPC_CHANNEL_CONNECTING or >>>>> GRPC_CHANNEL_TRANSIENT_FAILURE , but still can take 15 seconds to >>>>> connect. I haven't found a pattern. Can someone tell me how I get it to >>>>> connect faster and more reliably? Thanks. Note I am using a deadline, >>>>> so >>>>> that I can shut everything down at the end gracefully, and not have it >>>>> block forever. >>>>> >>>>> >>>>> >>>>> ... >>>>> >>>>> channel = grpc::CreateChannel(asServerAddress, channel_creds); >>>>> >>>>> while ((channel->GetState(true) != GRPC_CHANNEL_READY)) >>>>> { >>>>> time_point deadline = std::chrono::system_clock::now() + >>>>> std::chrono::milliseconds(1000); >>>>> >>>>> channel->WaitForConnected(deadline); >>>>> std::cout << "." << channel->GetState(false) << std::flush ; >>>>> } >>>>> std::cout << "Client Connected" << std::endl; >>>>> >>>>> .... >>>>> >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "grpc.io" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/grpc-io. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/grpc-io/06cb24fd-f91f-42d4-b495-9c701b2457ae%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/grpc-io/06cb24fd-f91f-42d4-b495-9c701b2457ae%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "grpc.io" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>>> >>>> To post to this group, send email to [email protected]. >>>> Visit this group at https://groups.google.com/group/grpc-io. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/grpc-io/6661bf81-8734-4c0f-a6a0-fc5b1adfce8e%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/grpc-io/6661bf81-8734-4c0f-a6a0-fc5b1adfce8e%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "grpc.io" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/grpc-io. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/grpc-io/CAMBUOonNs_AkpsYXeC5e0QMKoLjwgizuCMHo5g3LHAhJZ0SDsA%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/grpc-io/CAMBUOonNs_AkpsYXeC5e0QMKoLjwgizuCMHo5g3LHAhJZ0SDsA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >>> <connect.txt> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "grpc.io" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/grpc-io. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/grpc-io/c9ee5076-31d9-4d93-8471-6af3730d167b%40googlegroups.com >> >> <https://groups.google.com/d/msgid/grpc-io/c9ee5076-31d9-4d93-8471-6af3730d167b%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> >> -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/7212c0d9-9555-4d07-8266-f15e2d38b68d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
