Hi, I think the comment [1] is only correct for "getStatistics" as it is called at driver side. It should be added in "getMapSizesByExecutorId" by mistake.
Jacek Laskowski wrote > Hi, > > I've been reviewing how MapOutputTracker works and can't understand > the comment [1]: > > // Synchronize on the returned array because, on the driver, it gets > mutated in place > > How is this possible since "the returned array" is a local value? I'm > stuck and would appreciate help. Thanks! > > (It also says "Called from executors" [2] so how could the driver be > involved?!) > > [1] > https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/MapOutputTracker.scala#L145 > > [2] > https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/MapOutputTracker.scala#L133 > > Pozdrawiam, > Jacek Laskowski > ---- > https://medium.com/@jaceklaskowski/ > Mastering Apache Spark 2.0 https://bit.ly/mastering-apache-spark > Follow me at https://twitter.com/jaceklaskowski > > --------------------------------------------------------------------- > To unsubscribe e-mail: > dev-unsubscribe@.apache ----- Liang-Chi Hsieh | @viirya Spark Technology Center http://www.spark.tc/ -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/MapOutputTracker-getMapSizesByExecutorId-and-mutation-on-the-driver-tp20342p20349.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
