[
https://issues.apache.org/jira/browse/KUDU-3489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17747275#comment-17747275
]
ASF subversion and git services commented on KUDU-3489:
-------------------------------------------------------
Commit 93fa5bac55400261a53b3e51b4f77fd34bc26740 in kudu's branch
refs/heads/branch-1.17.x from Abhishek Chennaka
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=93fa5bac5 ]
[doc] Update release_notes.adoc
Add KUDU-3489 in the release_notes.adoc under fixed issues.
Change-Id: I26a793367c0c5253a732c090a160a89d72fc215a
Reviewed-on: http://gerrit.cloudera.org:8080/20260
Tested-by: Kudu Jenkins
Reviewed-by: Yingchun Lai <[email protected]>
> Subprocess communication with large messages
> --------------------------------------------
>
> Key: KUDU-3489
> URL: https://issues.apache.org/jira/browse/KUDU-3489
> Project: Kudu
> Issue Type: Bug
> Reporter: Abhishek Chennaka
> Priority: Major
> Fix For: 1.17.0
>
>
> With KUDU-3450 we started supporting messages of size up to 8MB by default to
> be transmitted between Kudu master and subprocess server. Given the the
> maximum pipe buffer size is 1MB in linux machines [1] we see flaky failures
> if the size of the message is more than 1MB between Kudu master and the
> subprocess server.
> [1] [https://man7.org/linux/man-pages/man7/pipe.7.html] (pipe-max-size)
> Example failure:
> {code:java}
> Exception in thread "main" java.util.concurrent.CompletionException:
> org.apache.kudu.subprocess.KuduSubprocessException: Unable to read the
> protobuf message
> at
> java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
> at
> java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1629)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.kudu.subprocess.KuduSubprocessException: Unable to read
> the protobuf message
> at org.apache.kudu.subprocess.MessageReader.run(MessageReader.java:77)
> at
> java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
> ... 3 more
> Caused by: java.io.IOException: unable to receive message, expected (2814368)
> bytes but read (1777660) bytes
> at org.apache.kudu.subprocess.MessageIO.doRead(MessageIO.java:101)
> at org.apache.kudu.subprocess.MessageIO.readBytes(MessageIO.java:81)
> at org.apache.kudu.subprocess.MessageReader.run(MessageReader.java:68)
> ... 4 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
