hiboyang commented on pull request #31715: URL: https://github.com/apache/spark/pull/31715#issuecomment-791810374
> Just checked Uber RSS (as I know that is open source and once I read its source at a level) where the location is a dummy block manager and the topology info holds the RSS servers list: > > https://github.com/uber/RemoteShuffleService/blob/master/src/main/scala/org/apache/spark/shuffle/rss/RssUtils.scala#L56 > > But I know you know this already ;) Yes, I added there :) That was kind of a work around to make it work with open source Spark. Still waiting for another PR "[Adding metadata to MapStatus](https://github.com/apache/spark/pull/30004)) gets reviewed. That could provide a better solution for embedding RSS servers list inside MapStatus. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
