Hi, I would be grateful if someone knowledgeable can shed some light on this:
If a Cached Relation is removed via CachedManager, while concurrently a query plan has embedded it in as InMemoryRelation ( IMR). The thread removing from cache , clears the buffers etc. Which is not an issue afaik, for the query getting executed, it will re-build the buffer. Once the query is over, and say even when the query plan is GCed, would not it leave orphaned in memory data somewhere in BlockManager & also leave orphaned files ( assuming saved on disk). Is there some existing mechanism which cleans up the unreferenced data? Regards Asif
