Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19500 )
Change subject: KUDU-3450 handling of oversized messages in subprocess server ...................................................................... KUDU-3450 handling of oversized messages in subprocess server This patch adds a new flag --subprocess_max_message_size_bytes to configure the maximum allowed size for the body of the response message received by the subprocess server, with default value of 8 MiB. Prior to this patch, the hard-coded limit was 1 MiB. I also updated the default values for the following flags, increasing them 4x times: * --subprocess_request_queue_size_bytes (4 --> 16 MiB) * --subprocess_response_queue_size_bytes (4 --> 16 MiB) With this patch, the behavior of the subprocess server has changed when encountering an error while reading a response message from its subprocess. Now, the server does not just bail upon a larger than expected message or in case of other error, but rather tries to read out and discard the message to clear the communication channel, so there is a chance to receive next messages from the subprocess. A new metric has been added for the subprocess server to report on the number of read and discarded messages because of various errors while reading and decoding the data from the communication channel: * server_dropped_messages In addition, this patch introduces a few test scenarios to cover the newly introduced functionality. I also sprayed the relevant code with syntactic sugar of structural binding since the code requires C++17-capable compiler anyways. A follow-up patch should take care of the corresponding client-side components that are used to run the Ranger client as a subprocess. Change-Id: I05b09e757f304b22e37438c2445ecc161ef412c9 Reviewed-on: http://gerrit.cloudera.org:8080/19500 Reviewed-by: Attila Bukor <[email protected]> Tested-by: Alexey Serbin <[email protected]> --- M src/kudu/ranger/ranger_client.cc M src/kudu/subprocess/echo_subprocess.cc M src/kudu/subprocess/server.cc M src/kudu/subprocess/server.h M src/kudu/subprocess/subprocess_protocol.cc M src/kudu/subprocess/subprocess_protocol.h M src/kudu/subprocess/subprocess_proxy-test.cc M src/kudu/subprocess/subprocess_server-test.cc 8 files changed, 311 insertions(+), 68 deletions(-) Approvals: Attila Bukor: Looks good to me, approved Alexey Serbin: Verified -- To view, visit http://gerrit.cloudera.org:8080/19500 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I05b09e757f304b22e37438c2445ecc161ef412c9 Gerrit-Change-Number: 19500 Gerrit-PatchSet: 4 Gerrit-Owner: Alexey Serbin <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Ashwani Raina <[email protected]> Gerrit-Reviewer: Attila Bukor <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120)
