Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/20461#discussion_r165213755
--- Diff:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/OneForOneBlockFetcher.java
---
@@ -171,7 +171,9 @@ private void failRemainingBlocks(String[]
failedBlockIds, Throwable e) {
@Override
public void onData(String streamId, ByteBuffer buf) throws IOException
{
- channel.write(buf);
+ while (buf.hasRemaining()) {
+ channel.write(buf);
--- End diff --
I actually thought this is OK for a FileChannel (just based on this comment
in the original: https://github.com/apache/spark/pull/16989/files#r115409001)
but certainly this seems like a safe change. Just wondering if you
observed an issue, or this is just to be safe.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
