[
https://issues.apache.org/jira/browse/SPARK-49660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Changgyoo Park resolved SPARK-49660.
------------------------------------
Resolution: Invalid
> Cached local relation lookup failure
> ------------------------------------
>
> Key: SPARK-49660
> URL: https://issues.apache.org/jira/browse/SPARK-49660
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: 4.0.0
> Reporter: Changgyoo Park
> Priority: Major
>
> ```
> private def transformCachedLocalRelation(rel: proto.CachedLocalRelation):
> LogicalPlan = {
> val blockManager = session.sparkContext.env.blockManager
> val blockId = CacheId(sessionHolder.session.sessionUUID, rel.getHash)
> val bytes = blockManager.getLocalBytes(blockId)
> bytes
> .map { blockData =>
> try {
> val localRelation = proto.Relation
> .newBuilder()
> .getLocalRelation
> .getParserForType
> .parseFrom(blockData.toInputStream())
> transformLocalRelation(localRelation)
> } finally {
> blockManager.releaseLock(blockId)
> }
> }
> .getOrElse {
> throw InvalidPlanInput(
> s"Not found any cached local relation with the hash:
> ${blockId.hash} in " +
> s"the session with sessionUUID ${blockId.sessionUUID}.")
> }
> }
> ```
> => `blockManager.releaseLock(blockId)` is wrong because locks are released
> inside `val bytes = blockManager.getLocalBytes(blockId)`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
