Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21322
@JeetKunDoug The same issue we discuss above. I think if there's a
deserialized version of the variable, it can be not in `MemoryStore` but only
serialized bytes in disk store.
The reason is, we use `getLocalValues` to retrieve the deserialized object.
If it is stored in disk store by `putSingle`, we read it back and call
`maybeCacheDiskValuesInMemory` trying to cache it in `MemoryStore`. But it is
not guaranteed to be succeed. When it fails, you still only have serialized
value in disk store.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
