HeartSaVioR commented on issue #25779: [SPARK-27468][core] Track correct storage level and mem/disk usage for RDDs. URL: https://github.com/apache/spark/pull/25779#issuecomment-536818810 > if the block is dropped via BlockManager.removeBlockInternal, which is triggered by unpersisting an RDD, then the block update sent to the master does not contain the previous size, neither for disk nor memory. Right, #25973 is for dealing with this, though the origin target of #25973 was broadcast. So if we would want to allow two different semantics of "size" on UpdateBlockInfo (either "addition" and "removal" depending on storage level), I guess #25973 is correct. If we feel awkward about current logic and only want to report "current" status of block, we may have to make non trivial change and #25973 will be no longer correct then.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
