[
https://issues.apache.org/jira/browse/HADOOP-17728?focusedWorklogId=601507&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-601507
]
ASF GitHub Bot logged work on HADOOP-17728:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 25/May/21 03:04
Start Date: 25/May/21 03:04
Worklog Time Spent: 10m
Work Description: yikf commented on a change in pull request #3042:
URL: https://github.com/apache/hadoop/pull/3042#discussion_r638417055
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
##########
@@ -4004,12 +4004,14 @@ public void cleanUp() {
* Background action to act on references being removed.
*/
private static class StatisticsDataReferenceCleaner implements Runnable {
+ private static int REF_QUEUE_POLL_TIMEOUT = 100;
Review comment:
En, We should take this time into consideration. If it is too long, many
resources need to be recycled in a certain period of time, which may lead to
low recycling efficiency.
we can refer this: [referenceQueue
timeput](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/ContextCleaner.scala#L191)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 601507)
Time Spent: 2h (was: 1h 50m)
> Deadlock in FileSystem StatisticsDataReferenceCleaner cleanUp
> -------------------------------------------------------------
>
> Key: HADOOP-17728
> URL: https://issues.apache.org/jira/browse/HADOOP-17728
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 3.2.1
> Reporter: yikf
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Cleaner thread will be blocked if we remove reference from ReferenceQueue
> unless the `queue.enqueue` called.
> ----
> As shown below, We call ReferenceQueue.remove() now while cleanUp, Call
> chain as follow：
> *StatisticsDataReferenceCleaner#queue.remove() ->
> ReferenceQueue.remove(0) -> lock.wait(0)*
> But, lock.notifyAll is called when queue.enqueue only, so Cleaner thread
> will be blocked.
>
> ThreadDump:
> {code:java}
> "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f7afc088800
> nid=0x2119 in Object.wait() [0x00007f7b00230000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:502)
> at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
> - locked <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
