[
https://issues.apache.org/jira/browse/KUDU-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Chennaka resolved KUDU-3489.
-------------------------------------
Fix Version/s: 1.17.0
Resolution: Fixed
> Subprocess communication with large messages
> --------------------------------------------
>
> Key: KUDU-3489
> URL: https://issues.apache.org/jira/browse/KUDU-3489
> Project: Kudu
> Issue Type: Bug
> Reporter: Abhishek Chennaka
> Priority: Major
> Fix For: 1.17.0
>
>
> With KUDU-3450 we started supporting messages of size up to 8MB by default to
> be transmitted between Kudu master and subprocess server. Given the the
> maximum pipe buffer size is 1MB in linux machines [1] we see flaky failures
> if the size of the message is more than 1MB between Kudu master and the
> subprocess server.
> [1] [https://man7.org/linux/man-pages/man7/pipe.7.html] (pipe-max-size)
> Example failure:
> {code:java}
> Exception in thread "main" java.util.concurrent.CompletionException:
> org.apache.kudu.subprocess.KuduSubprocessException: Unable to read the
> protobuf message
> at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
> at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1629)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.kudu.subprocess.KuduSubprocessException: Unable to read
> the protobuf message
> at org.apache.kudu.subprocess.MessageReader.run(MessageReader.java:77)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
> ... 3 more
> Caused by: java.io.IOException: unable to receive message, expected (2814368)
> bytes but read (1777660) bytes
> at org.apache.kudu.subprocess.MessageIO.doRead(MessageIO.java:101)
> at org.apache.kudu.subprocess.MessageIO.readBytes(MessageIO.java:81)
> at org.apache.kudu.subprocess.MessageReader.run(MessageReader.java:68)
> ... 4 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
