Vincent created SPARK-24968:
-------------------------------
Summary: Configurable Chunksize in ChunkedByteBufferOutputStream
Key: SPARK-24968
URL: https://issues.apache.org/jira/browse/SPARK-24968
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.3.0, 2.2.2, 2.4.0
Reporter: Vincent
Hello,
it seems that when creating a _ChunkedByteBufferOutputStream,_ the chunk size
is always configured to be 4MB. I suggest we make it configurable via spark
conf. This would allow to solve issues like SPARK-24917 (by increasing the
chunk size to a bigger value in the conf).
What do you think ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
