[
https://issues.apache.org/jira/browse/KUDU-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Serbin resolved KUDU-3450.
---------------------------------
Fix Version/s: 1.17.0
Resolution: Fixed
> SubprocessProtocol has a hard-coded limit on message size, but RangerClient
> doesn't honor that while generating requests
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: KUDU-3450
> URL: https://issues.apache.org/jira/browse/KUDU-3450
> Project: Kudu
> Issue Type: Bug
> Components: subprocess
> Affects Versions: 1.12.0, 1.13.0, 1.14.0, 1.15.0, 1.16.0
> Reporter: Alexey Serbin
> Priority: Major
> Fix For: 1.17.0
>
>
> As one can see from the code in the
> [SubprocessProtocol|https://github.com/apache/kudu/blob/9684200713b5e1bf258437527127bd98acfa1e42/src/kudu/subprocess/subprocess_protocol.cc#L106-L110]
> and in [MessageIO utility Java
> class|https://github.com/apache/kudu/blob/9684200713b5e1bf258437527127bd98acfa1e42/java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/MessageIO.java#L69-L73],
> it's not possible to receive messages in PB format that are larger than a
> pre-defined limit. Also, there isn't a way to pass
> {{SubprocessRequestPB::request}} and {{SubprocessResponsePB::response}} data
> in multiple chunks in the SubprocessProtocol by design.
> The default setting for the maximum message size is 1MiByte in the
> implementation of [the relevant
> C++|https://github.com/apache/kudu/blob/9684200713b5e1bf258437527127bd98acfa1e42/src/kudu/subprocess/subprocess_protocol.cc#L46]
> and [Java subprocess
> components|https://github.com/apache/kudu/blob/9684200713b5e1bf258437527127bd98acfa1e42/java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/SubprocessConfiguration.java#L54].
> The implementation of the related C++ components has the setting effectively
> hard-coded and there isn't a way to change it. The implementation of the
> related Java components has this setting configurable via the {{maxMsgBytes}}
> configuration property.
> Anyways, nether the code in {{RangerClient}} nor the code in
> {{RangerProtocolHandler}} honors the limit:
> *
> [RangerClient::AuthorizeActionMultipleColumns()|https://github.com/apache/kudu/blob/9684200713b5e1bf258437527127bd98acfa1e42/src/kudu/ranger/ranger_client.cc#L435-L448]
> *
> [RangerClient::AuthorizeActionMultipleTables()|https://github.com/apache/kudu/blob/9684200713b5e1bf258437527127bd98acfa1e42/src/kudu/ranger/ranger_client.cc#L468-L492]
> *
> [ProtocolHandler.unpackAndExecuteRequest()|https://github.com/apache/kudu/blob/9684200713b5e1bf258437527127bd98acfa1e42/java/kudu-subprocess/src/main/java/org/apache/kudu/subprocess/ProtocolHandler.java#L47-L56]
> With that, it's possible to get into an impasse situation from the both
> sides. For example, if a large request has been generated by
> {{RangerClient::AuthorizeActionMultipleColumns()}} or
> {{RangerClient::AuthorizeActionMultipleTables()}}, the following stack trace
> might be observed in the output of the subprocess that runs the Ranger client
> plugin:
> {noformat}
> Exception in thread "main" java.util.concurrent.CompletionException:
> org.apache.kudu.subprocess.KuduSubprocessException: Unable to read the
> protobuf message
> at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
> at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1629)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.kudu.subprocess.KuduSubprocessException: Unable to read
> the protobuf message
> at org.apache.kudu.subprocess.MessageReader.run(MessageReader.java:74)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
> ... 3 more
> Caused by: java.io.IOException: message size (2763197) exceeds maximum
> message size (1048576)
> at org.apache.kudu.subprocess.MessageIO.readBytes(MessageIO.java:71)
> at org.apache.kudu.subprocess.MessageReader.run(MessageReader.java:68)
> ... 4 more
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
