Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19184#discussion_r137981565 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeSorterSpillReader.java --- @@ -72,12 +77,27 @@ public UnsafeSorterSpillReader( bufferSizeBytes = DEFAULT_BUFFER_SIZE_BYTES; } + try (InputStream bs = new NioBufferedFileInputStream(file, (int) bufferSizeBytes); + DataInputStream dataIn = new DataInputStream(serializerManager.wrapStream(blockId, bs))) { + this.numRecords = dataIn.readInt(); + this.numRecordsRemaining = numRecords; + } + + this.buffSize = bufferSizeBytes; + this.file = file; + this.blockId = blockId; + this.serializerManager = serializerManager; + + logger.debug("bufSize: {}, file: {}, records: {}", buffSize, file, this.numRecords); --- End diff -- Is this log useful? If the number of spill readers is so many, I guess we don't want to see so many log info?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org