viirya commented on pull request #30770:
URL: https://github.com/apache/spark/pull/30770#issuecomment-747173777
> Could you please elaborate for "stablization", via describing the
situation how things could be broken? It checks with driver that there's
another executor serving state before unloading, so I'm not sure what could be
broken.
For example, an inactive store could be unloaded after few seconds sometimes
and after maintenance interval sometimes? So in some cases a query can work,
just because a store is unloaded immediately to free memory. But in some cases,
it will OOM because the store is unloaded later?
I'm not sure if we are okay with inconsistent behavior like that, although
it describes a rare situation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
