Ngone51 commented on code in PR #42296:
URL: https://github.com/apache/spark/pull/42296#discussion_r1321386699
##########
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala:
##########
@@ -1580,7 +1625,9 @@ object ShuffleBlockFetcherIterator {
* Result of a fetch request that should be deferred for some reasons, e.g.,
Netty OOM
Review Comment:
nit:
```suggestion
* Result of a fetch request that should be deferred for some reasons,
e.g., Netty OOM, decommission
```
##########
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala:
##########
@@ -1167,8 +1200,10 @@ final class ShuffleBlockFetcherIterator(
// immediately, defer the request until the next time it can be processed.
// Process any outstanding deferred fetch requests if possible.
+ // Skip when the address is the local BM Id (this may happen when shuffle
migration is enabled)
if (deferredFetchRequests.nonEmpty) {
- for ((remoteAddress, defReqQueue) <- deferredFetchRequests) {
+ for ((remoteAddress, defReqQueue) <- deferredFetchRequests
+ if remoteAddress != blockManager.blockManagerId) {
Review Comment:
nit 2 indents:
```suggestion
if remoteAddress != blockManager.blockManagerId) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
