squito commented on a change in pull request #23453: [SPARK-26089][CORE] Handle
corruption in large shuffle blocks
URL: https://github.com/apache/spark/pull/23453#discussion_r253979231
##########
File path: core/src/main/scala/org/apache/spark/util/Utils.scala
##########
@@ -335,6 +336,47 @@ private[spark] object Utils extends Logging {
}
}
+ /**
+ * Copy all data from an InputStream to an OutputStream upto maxSize and
+ * closes the input stream if closeStreams is true and all data is read.
+ */
+ def copyStreamUpTo(
+ in: InputStream,
+ maxSize: Long,
+ closeStreams: Boolean = false): (Boolean, InputStream) = {
Review comment:
I don't think you need to expose `closeStreams`, just have it always be
true. At least I can't think of a reason why it should default to `false`
(`copyStream` is used more places ...)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
