I believe https://cs.corp.google.com/piper///depot/google3/third_party/grpc/include/grpc/impl/codegen/grpc_types.h?rcl=229586949&l=243 GRPC_ARG_MAX_RECONNECT_BACKOFF_MS <https://cs.corp.google.com/piper///depot/google3/third_party/grpc/include/grpc/impl/codegen/grpc_types.h?l=243&gs=kythe%253A%252F%252Fgoogle3%253Flang%253Dc%25252B%25252B%253Fpath%253Dthird_party%252Fgrpc%252Finclude%252Fgrpc%252Fimpl%252Fcodegen%252Fgrpc_types.h%2523GRPC_ARG_MAX_RECONNECT_BACKOFF_MS%252523m%25254010507&gsn=GRPC_ARG_MAX_RECONNECT_BACKOFF_MS&ct=xref_usages> is what you are looking for.
On Friday, January 18, 2019 at 8:36:11 AM UTC-8, [email protected] wrote: > > Ok I think i have a clue, looking at full log file for fd_create for that > port, I see that the time between retries is monotonically increasing > (doubling) even though I have the WaitForConnection deadline static. How > can I disable this feature? > > > > > On Friday, January 18, 2019 at 9:44:13 AM UTC-5, [email protected] > wrote: >> >> Yes start of file is about when the server connection goes up at the same >> time client software is trying to connect using loop above, >> so 12:54:24.361342246, fd_create in question is at 12:54:32.698523276, and >> connection happens shortly after. Thanks for looking. >> >> >> >> On Friday, January 18, 2019 at 9:39:18 AM UTC-5, Robert Engels wrote: >>> >>> Can you identify the time this occurs in the file? It’s fairly large. >>> >>> On Jan 18, 2019, at 8:35 AM, [email protected] wrote: >>> >>> Yes see previous message: connect.txt >>> >>> On Friday, January 18, 2019 at 9:23:03 AM UTC-5, Robert Engels wrote: >>>> >>>> Is the debug file attached ? >>>> >>>> On Jan 18, 2019, at 8:18 AM, Brian Wagener <[email protected]> >>>> wrote: >>>> >>>> I don't think it is either of those. I was able to capture grpc debug >>>> and it shows that the actual fd_create to Port 50074 doesn't even get >>>> called in this case until 8 seconds after trying to connect. Can someone >>>> look at debug file and see what grpc is doing? >>>> >>>> Brian >>>> >>>> On Thu, Jan 17, 2019, 3:09 PM robert engels <[email protected]> >>>> wrote: >>>> >>>>> If you are running a tight loop with lots of connection attempts there >>>>> are a lot of reasons it can fail. usually resources (number of >>>>> connections) >>>>> - so while the OS is waiting to close the existing connections, future >>>>> attempts will fail. >>>>> >>>>> On Jan 17, 2019, at 1:54 PM, [email protected] wrote: >>>>> >>>>> I don't really understand the question, but I have tested retry by >>>>> just starting and stopping server. >>>>> >>>>> >>>>> On Thursday, January 17, 2019 at 2:46:44 PM UTC-5, robert engels wrote: >>>>>> >>>>>> How are you testing the retry - pulling plug? iptables ? >>>>>> >>>>>> On Jan 17, 2019, at 1:39 PM, [email protected] wrote: >>>>>> >>>>>> >>>>>> I have a gRPC streaming client, that has to handle server going up >>>>>> and down, so I have a while loop, but sometimes it works fine, but other >>>>>> times it takes 15 seconds to connect even on the same machine. Is it >>>>>> something wrong with my code, or how can I debug? As you can see below I >>>>>> have debug to print out channel state, and is mostly >>>>>> GRPC_CHANNEL_CONNECTING or GRPC_CHANNEL_TRANSIENT_FAILURE , but >>>>>> still can take 15 seconds to connect. I haven't found a pattern. Can >>>>>> someone tell me how I get it to connect faster and more reliably? >>>>>> Thanks. >>>>>> Note I am using a deadline, so that I can shut everything down at the >>>>>> end >>>>>> gracefully, and not have it block forever. >>>>>> >>>>>> >>>>>> >>>>>> ... >>>>>> >>>>>> channel = grpc::CreateChannel(asServerAddress, channel_creds); >>>>>> >>>>>> while ((channel->GetState(true) != GRPC_CHANNEL_READY)) >>>>>> { >>>>>> time_point deadline = std::chrono::system_clock::now() + >>>>>> std::chrono::milliseconds(1000); >>>>>> >>>>>> channel->WaitForConnected(deadline); >>>>>> std::cout << "." << channel->GetState(false) << std::flush ; >>>>>> } >>>>>> std::cout << "Client Connected" << std::endl; >>>>>> >>>>>> .... >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "grpc.io" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at https://groups.google.com/group/grpc-io. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/grpc-io/06cb24fd-f91f-42d4-b495-9c701b2457ae%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/grpc-io/06cb24fd-f91f-42d4-b495-9c701b2457ae%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>>> >>>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "grpc.io" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>>> >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/grpc-io. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/grpc-io/6661bf81-8734-4c0f-a6a0-fc5b1adfce8e%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/grpc-io/6661bf81-8734-4c0f-a6a0-fc5b1adfce8e%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "grpc.io" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at https://groups.google.com/group/grpc-io. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/grpc-io/CAMBUOonNs_AkpsYXeC5e0QMKoLjwgizuCMHo5g3LHAhJZ0SDsA%40mail.gmail.com >>>> >>>> <https://groups.google.com/d/msgid/grpc-io/CAMBUOonNs_AkpsYXeC5e0QMKoLjwgizuCMHo5g3LHAhJZ0SDsA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>>> <connect.txt> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "grpc.io" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/grpc-io. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/grpc-io/c9ee5076-31d9-4d93-8471-6af3730d167b%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/grpc-io/c9ee5076-31d9-4d93-8471-6af3730d167b%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >>> -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/22750af6-d8be-4150-ab42-1e9b2ccca041%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
