[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/9186 ) Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. Patch Set 5: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 5 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Wed, 07 Feb 2018 03:24:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9186 ) Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC With the fix for KUDU-2228, the FLAGS_rpc_negotiation_timeout_ms was retired in KRPC. This patch introduces a flag to be able to configure that from the Impala side (FLAGS_rpc_negotiation_timeout_ms). It also introduces a flag to configure the negotiation thread count (FLAGS_rpc_negotiation_thread_count). Added a test to verify that setting FLAGS_rpc_negotiation_timeout_ms to 0 causes negotiation failures. We unfortunately can't write a test to check the same for FLAGS_rpc_negotiation_thread_count due to DCHECKS present in the code. Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Reviewed-on: http://gerrit.cloudera.org:8080/9186 Reviewed-by: Sailesh MukilTested-by: Impala Public Jenkins --- M be/src/rpc/rpc-mgr-test.cc M be/src/rpc/rpc-mgr.cc 2 files changed, 28 insertions(+), 0 deletions(-) Approvals: Sailesh Mukil: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 6 Gerrit-Owner: Sailesh Mukil Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/9186 ) Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. Patch Set 5: Code-Review+2 Rebase, carry +2. -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 5 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Tue, 06 Feb 2018 23:43:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/9186 ) Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. Patch Set 4: Code-Review+2 (2 comments) Thanks for the review. Holding off on GVO until IMPALA-6485 unblocks GVO. http://gerrit.cloudera.org:8080/#/c/9186/3/be/src/rpc/rpc-mgr-test.cc File be/src/rpc/rpc-mgr-test.cc: http://gerrit.cloudera.org:8080/#/c/9186/3/be/src/rpc/rpc-mgr-test.cc@235 PS3, Line 235: // establishment fails. > May help to clarify: Done http://gerrit.cloudera.org:8080/#/c/9186/3/be/src/rpc/rpc-mgr-test.cc@252 PS3, Line 252: secondary_rpc_mgr.Shutdown(); > nit: extra blank line. Done -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 4 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Tue, 06 Feb 2018 22:31:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Hello Michael Ho, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/9186 to look at the new patch set (#4). Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC With the fix for KUDU-2228, the FLAGS_rpc_negotiation_timeout_ms was retired in KRPC. This patch introduces a flag to be able to configure that from the Impala side (FLAGS_rpc_negotiation_timeout_ms). It also introduces a flag to configure the negotiation thread count (FLAGS_rpc_negotiation_thread_count). Added a test to verify that setting FLAGS_rpc_negotiation_timeout_ms to 0 causes negotiation failures. We unfortunately can't write a test to check the same for FLAGS_rpc_negotiation_thread_count due to DCHECKS present in the code. Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 --- M be/src/rpc/rpc-mgr-test.cc M be/src/rpc/rpc-mgr.cc 2 files changed, 28 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/9186/4 -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 4 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/9186 ) Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. Patch Set 3: (3 comments) http://gerrit.cloudera.org:8080/#/c/9186/2/be/src/rpc/rpc-mgr-test.cc File be/src/rpc/rpc-mgr-test.cc: http://gerrit.cloudera.org:8080/#/c/9186/2/be/src/rpc/rpc-mgr-test.cc@537 PS2, Line 537: > Should this use something like ScopedFlagSetter() in case the ASSERT below Yes, good idea. Done. http://gerrit.cloudera.org:8080/#/c/9186/2/be/src/rpc/rpc-mgr-test.cc@541 PS2, Line 541: > Is TLS always enabled ? Seem a bit misleading to have the "tls_" prefix her Sorry that was a mistake. I forgot to change the name of the variable. Done. http://gerrit.cloudera.org:8080/#/c/9186/2/be/src/rpc/rpc-mgr-test.cc@551 PS2, Line 551: > Just wondering if it'll be a problem if ASSERT() above gets triggered. Will If ASSERT() is triggered, we fail the BE test anyway right? So, even though the RpcMgr isn't shutdown, the process will kill it. -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 3 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Tue, 06 Feb 2018 19:14:02 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Hello Michael Ho, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/9186 to look at the new patch set (#3). Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC With the fix for KUDU-2228, the FLAGS_rpc_negotiation_timeout_ms was retired in KRPC. This patch introduces a flag to be able to configure that from the Impala side (FLAGS_rpc_negotiation_timeout_ms). It also introduces a flag to configure the negotiation thread count (FLAGS_rpc_negotiation_thread_count). Added a test to verify that setting FLAGS_rpc_negotiation_timeout_ms to 0 causes negotiation failures. We unfortunately can't write a test to check the same for FLAGS_rpc_negotiation_thread_count due to DCHECKS present in the code. Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 --- M be/src/rpc/rpc-mgr-test.cc M be/src/rpc/rpc-mgr.cc 2 files changed, 28 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/9186/3 -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 3 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/9186 ) Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. Patch Set 2: (4 comments) http://gerrit.cloudera.org:8080/#/c/9186/2/be/src/rpc/rpc-mgr-test.cc File be/src/rpc/rpc-mgr-test.cc: http://gerrit.cloudera.org:8080/#/c/9186/2/be/src/rpc/rpc-mgr-test.cc@537 PS2, Line 537: auto save_rpc_negotiation_timeout_ms = FLAGS_rpc_negotiation_timeout_ms; Should this use something like ScopedFlagSetter() in case the ASSERT below fires ? http://gerrit.cloudera.org:8080/#/c/9186/2/be/src/rpc/rpc-mgr-test.cc@541 PS2, Line 541: tls_krpc_address Is TLS always enabled ? Seem a bit misleading to have the "tls_" prefix here and below. http://gerrit.cloudera.org:8080/#/c/9186/2/be/src/rpc/rpc-mgr-test.cc@551 PS2, Line 551: secondary_rpc_mgr.Shutdown(); Just wondering if it'll be a problem if ASSERT() above gets triggered. Will we exit without calling Shutdown() here ? http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc File be/src/rpc/rpc-mgr.cc: http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc@84 PS1, Line 84: bld.set_rpc_negotiation_timeout_ms(FLAGS_rpc_negotiation_timeout_ms); > The min number of threads is 0 by default. What that means is that there wo Makes sense. -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 2 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Tue, 06 Feb 2018 07:42:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Sailesh Mukil has posted comments on this change. ( http://gerrit.cloudera.org:8080/9186 ) Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc File be/src/rpc/rpc-mgr.cc: http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc@70 PS1, Line 70: Number of threads to dedicate to process connection negotiations > Maximum number of threads dedicated to handling RPC connection negotiations Done http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc@84 PS1, Line 84: bld.set_rpc_negotiation_timeout_ms(FLAGS_rpc_negotiation_timeout_ms); > Does it make sense to also call bld.set_min_negotiation_threads(1) ? The min number of threads is 0 by default. What that means is that there won't be any "permanent" thread that is always waiting to do negotiation. They will be spawned as required (up to max_thread_count) and live for some small period after which it times out and kills itself. Given our workload, we would have bursts of negotiations at different points in time with potentially vast intervals in between. So maybe it's better to save the resources of that one thread. What do you think? http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc@84 PS1, Line 84: FLAGS_rpc_negotiation_timeout_ms > What happens if this flag is set to negative by accident ? Will the code cr There's a CHECK to make sure that this isn't negative: https://github.com/apache/impala/blob/master/be/src/kudu/util/threadpool.cc#L79 So it will cause a crash. http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc@85 PS1, Line 85: FLAGS_rpc_negotiation_thread_count > Will it be safer to do max(1, FLAGS_rpc_negotiation_thread_count) to guaran Yes that makes sense. Done. -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 1 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Michael Ho Gerrit-Reviewer: Sailesh Mukil Gerrit-Comment-Date: Fri, 02 Feb 2018 23:46:14 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Hello Michael Ho, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/9186 to look at the new patch set (#2). Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC With the fix for KUDU-2228, the FLAGS_rpc_negotiation_timeout_ms was retired in KRPC. This patch introduces a flag to be able to configure that from the Impala side (FLAGS_rpc_negotiation_timeout_ms). It also introduces a flag to configure the negotiation thread count (FLAGS_rpc_negotiation_thread_count). Added a test to verify that setting FLAGS_rpc_negotiation_timeout_ms to 0 causes negotiation failures. We unfortunately can't write a test to check the same for FLAGS_rpc_negotiation_thread_count due to DCHECKS present in the code. Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 --- M be/src/rpc/rpc-mgr-test.cc M be/src/rpc/rpc-mgr.cc 2 files changed, 31 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/9186/2 -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 2 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Michael Ho
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/9186 ) Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc File be/src/rpc/rpc-mgr.cc: http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc@70 PS1, Line 70: Number of threads to dedicate to process connection negotiations Maximum number of threads dedicated to handling RPC connection negotiations http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc@84 PS1, Line 84: bld.set_rpc_negotiation_timeout_ms(FLAGS_rpc_negotiation_timeout_ms); Does it make sense to also call bld.set_min_negotiation_threads(1) ? http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc@84 PS1, Line 84: FLAGS_rpc_negotiation_timeout_ms What happens if this flag is set to negative by accident ? Will the code crash or does it translate to a very large unsigned value ? http://gerrit.cloudera.org:8080/#/c/9186/1/be/src/rpc/rpc-mgr.cc@85 PS1, Line 85: FLAGS_rpc_negotiation_thread_count Will it be safer to do max(1, FLAGS_rpc_negotiation_thread_count) to guarantee there is at least on negotiation thread available ? -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 1 Gerrit-Owner: Sailesh MukilGerrit-Reviewer: Michael Ho Gerrit-Comment-Date: Fri, 02 Feb 2018 21:43:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-6456: Add flags to configure rpc negotiation timeout ms and negotiation thread count in KRPC
Sailesh Mukil has uploaded this change for review. ( http://gerrit.cloudera.org:8080/9186 Change subject: IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC .. IMPALA-6456: Add flags to configure rpc_negotiation_timeout_ms and negotiation thread count in KRPC With the fix for KUDU-2228, the FLAGS_rpc_negotiation_timeout_ms was retired in KRPC. This patch introduces a flag to be able to configure that from the Impala side (FLAGS_rpc_negotiation_timeout_ms). It also introduces a flag to configure the negotiation thread count (FLAGS_rpc_negotiation_thread_count). Added a test to verify that setting FLAGS_rpc_negotiation_timeout_ms to 0 causes negotiation failures. We unfortunately can't write a test to check the same for FLAGS_rpc_negotiation_thread_count due to DCHECKS present in the code. Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 --- M be/src/rpc/rpc-mgr-test.cc M be/src/rpc/rpc-mgr.cc 2 files changed, 31 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/9186/1 -- To view, visit http://gerrit.cloudera.org:8080/9186 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I108d700e7eac04b678e21a3a920aac81ba8eede5 Gerrit-Change-Number: 9186 Gerrit-PatchSet: 1 Gerrit-Owner: Sailesh Mukil