gengliangwang commented on issue #23310: [SPARK-26363][WebUI] Avoid duplicated KV store lookups for task table URL: https://github.com/apache/spark/pull/23310#issuecomment-449409161 @srowen The new changes use a hash map to reduce the duplicated KV store lookups inside one method call `taskList`. To further improve it, we can either: 1. keep the original change, and make the request to "/allexecutors" synchronous. 2.Use a global hash map for executor ID to executor log, which should be working for most cases. Not sure if the mapping can be changed. E.g. executor removed and added back with the old ID? I am not sure if this can happen in Spark. What do you think?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
