[jira] [Commented] (SPARK-24968) Configurable Chunksize in ChunkedByteBufferOutputStream
[ https://issues.apache.org/jira/browse/SPARK-24968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563467#comment-16563467 ] Vincent commented on SPARK-24968: - indeed they are closely related I'll close this ticket > Configurable Chunksize in ChunkedByteBufferOutputStream > --- > > Key: SPARK-24968 > URL: https://issues.apache.org/jira/browse/SPARK-24968 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 2.2.2, 2.3.0, 2.4.0 >Reporter: Vincent >Priority: Minor > > Hello, > it seems that when creating a _ChunkedByteBufferOutputStream,_ the chunk size > is always configured to be 4MB. I suggest we make it configurable via spark > conf. This would allow to solve issues like SPARK-24917 (by increasing the > chunk size to a bigger value in the conf). > What do you think ? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24968) Configurable Chunksize in ChunkedByteBufferOutputStream
[ https://issues.apache.org/jira/browse/SPARK-24968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16563274#comment-16563274 ] Hyukjin Kwon commented on SPARK-24968: -- So is it a duplicate of SPARK-24917 roughly? I suggest to leave this resolved if that or this issue is a subset > Configurable Chunksize in ChunkedByteBufferOutputStream > --- > > Key: SPARK-24968 > URL: https://issues.apache.org/jira/browse/SPARK-24968 > Project: Spark > Issue Type: Improvement > Components: Spark Core >Affects Versions: 2.2.2, 2.3.0, 2.4.0 >Reporter: Vincent >Priority: Minor > > Hello, > it seems that when creating a _ChunkedByteBufferOutputStream,_ the chunk size > is always configured to be 4MB. I suggest we make it configurable via spark > conf. This would allow to solve issues like SPARK-24917 (by increasing the > chunk size to a bigger value in the conf). > What do you think ? -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
