[
https://issues.apache.org/jira/browse/HADOOP-17728?focusedWorklogId=600920&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-600920
]
ASF GitHub Bot logged work on HADOOP-17728:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 23/May/21 11:49
Start Date: 23/May/21 11:49
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#3042:
URL: https://github.com/apache/hadoop/pull/3042#discussion_r637533874
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
##########
@@ -4004,12 +4004,14 @@ public void cleanUp() {
* Background action to act on references being removed.
*/
private static class StatisticsDataReferenceCleaner implements Runnable {
+ private static int REF_QUEUE_POLL_TIMEOUT = 100;
Review comment:
that's going to be waking every 100 milliseconds, demanding cpu time etc
etc. If there has to be a timeout, it needs to be something less disruptive,
like 100 seconds.
What would happen if that was the case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 600920)
Time Spent: 1.5h (was: 1h 20m)
> Deadlock in FileSystem StatisticsDataReferenceCleaner cleanUp
> -------------------------------------------------------------
>
> Key: HADOOP-17728
> URL: https://issues.apache.org/jira/browse/HADOOP-17728
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 3.2.1
> Reporter: yikf
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Cleaner thread will be blocked if we remove reference from ReferenceQueue
> unless the `queue.enqueue` called.
> ----
> As shown below, We call ReferenceQueue.remove() now while cleanUp, Call
> chain as follow：
> *StatisticsDataReferenceCleaner#queue.remove() ->
> ReferenceQueue.remove(0) -> lock.wait(0)*
> But, lock.notifyAll is called when queue.enqueue only, so Cleaner thread
> will be blocked.
>
> ThreadDump:
> {code:java}
> "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f7afc088800
> nid=0x2119 in Object.wait() [0x00007f7b00230000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:502)
> at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
> - locked <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
