mccheah commented on a change in pull request #25341:
[SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice.
URL: https://github.com/apache/spark/pull/25341#discussion_r314905664
##########
File path:
core/src/main/java/org/apache/spark/shuffle/api/ShufflePartitionWriter.java
##########
@@ -85,14 +85,4 @@
default Optional<WritableByteChannelWrapper> openChannelWrapper() throws
IOException {
return Optional.empty();
}
-
- /**
- * Returns the number of bytes written either by this writer's output stream
opened by
- * {@link #openStream()} or the byte channel opened by {@link
#openChannelWrapper()}.
- * <p>
- * This can be different from the number of bytes given by the caller. For
example, the
- * stream might compress or encrypt the bytes before persisting the data to
the backing
- * data store.
- */
- long getNumBytesWritten();
Review comment:
Let's just keep the API for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
