[
https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208016#comment-15208016
]
Ernest commented on SPARK-13796:
--------------------------------
I find this ERROR is still exist in executor's log. This error can be
reproduced by running _LiveJournalPageRank_ example.
By calling BlockManager.getOrElseUpdate, as mentioned in
https://github.com/apache/spark/blob/f2e855fba8eb73475cf312cdf880c1297d4323bb/core/src/main/scala/org/apache/spark/storage/BlockManager.scala#L660
, both of _doPut_ and _getLocalValues_ acquired a read lock, but only released
one at line 661.
The remaining lock should be release by _CompletionIterator_ located in
https://github.com/apache/spark/blob/f2e855fba8eb73475cf312cdf880c1297d4323bb/core/src/main/scala/org/apache/spark/storage/BlockManager.scala#L427
after successfully iterates through all the elements, however, this do not
always work.
At the last of task, _releaseAllLocksForTask_ is called, and blocks that
still have read lock are added to this method's result. And lead to lots of
this kind of error in executor.
> Lock release errors occur frequently in executor logs
> -----------------------------------------------------
>
> Key: SPARK-13796
> URL: https://issues.apache.org/jira/browse/SPARK-13796
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.0.0
> Reporter: Nishkam Ravi
> Priority: Minor
>
> Executor logs contain a lot of these error messages (irrespective of the
> workload):
> 16/03/08 17:53:07 ERROR executor.Executor: 1 block locks were not released by
> TID = 1119
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
