yaooqinn commented on pull request #33063: URL: https://github.com/apache/spark/pull/33063#issuecomment-868943286
@mridulm @dongjoon-hyun, I re-run the benchmark based on the final commit manually. The debug log below shows the performance regression is gone. ```log 21/06/26 04:04:01 INFO MapOutputTrackerWorker: Got the map output locations 21/06/26 04:04:01 DEBUG ShuffleBlockFetcherIterator: maxBytesInFlight: 50331648, targetRemoteRequestSize: 10066329, maxBlocksInFlightPerAddress: 2147483647 21/06/26 04:04:01 DEBUG ShuffleBlockFetcherIterator: Creating fetch request of 2606046 at BlockManagerId(2, 10.1.5.72, 37767, None) with 88 blocks 21/06/26 04:04:01 DEBUG ShuffleBlockFetcherIterator: Collected remote fetch requests for BlockManagerId(2, 10.1.5.72, 37767, None) in 82 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
