Github user LinhongLiu commented on the issue:
https://github.com/apache/spark/pull/23040
Problem:
ChunkedByteBuffer has signature `ChunkedByteBuffer(var chunks:
Array[ByteBuffer])`. This means user is allowed to pass any kind of `chunks` to
it. Then we will face the problem described in
[JIRA](https://issues.apache.org/jira/browse/SPARK-26068). That's why I submit
this PR.
But on the other hand:
This is an internal class and Spark safely uses `ChunkedByteBuffer` in 2
ways:
1. Use `ChunkedByteBuffer(byteBuffer: ByteBuffer)` to pass only one buffer.
Even if it's an empty one, spark will handle this case correctly.
2. Use `ChunkedByteBufferOutputStream` to create `ChunkedByteBuffer` with
multiple `chunks`. In this case, empty ByteBuffer will never happen.
As a result, current spark code will never reach the problem as far as we
won't use `ChunkedByteBuffer(var chunks: Array[ByteBuffer])` directly.
So it's both OK either we fix this or not.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
