Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20119#discussion_r159370490
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ---
@@ -198,7 +196,7 @@ private[spark] class IndexShuffleBlockResolver(
// find out the consolidated file, then the offset within that from
our index
val indexFile = getIndexFile(blockId.shuffleId, blockId.mapId)
- val in = new DataInputStream(Files.newInputStream(indexFile.toPath))
+ val in = new DataInputStream(new FileInputStream(indexFile))
--- End diff --
@jerryshao this is another place. In addition, I'm not sure if there is any
compression codec using `skip` or not.
I also noticed `sun.nio.ch.ChannelInputStream` has extra `synchronized`s as
`Files.newInputStream` needs to be thread-safe. Not sure if it may cause
performance regression or not.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
