Vincent created SPARK-24968: ------------------------------- Summary: Configurable Chunksize in ChunkedByteBufferOutputStream Key: SPARK-24968 URL: https://issues.apache.org/jira/browse/SPARK-24968 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 2.3.0, 2.2.2, 2.4.0 Reporter: Vincent
Hello, it seems that when creating a _ChunkedByteBufferOutputStream,_ the chunk size is always configured to be 4MB. I suggest we make it configurable via spark conf. This would allow to solve issues like SPARK-24917 (by increasing the chunk size to a bigger value in the conf). What do you think ? -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org