[
https://issues.apache.org/jira/browse/HADOOP-17728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17362577#comment-17362577
]
Sangjin Lee commented on HADOOP-17728:
--------------------------------------
This caught my attention (sorry I'm not very active in the Hadoop codebase
lately).
I'm not quite sure what the problem statement here is. Is there a real problem
that can be demonstrated with a reproducible test case? The reference queue
gets enqueued not by user's explicit code but by the JVM via weak references in
this case. The GC will enqueue the reference that's being garbage collected
into the reference queue. That's why there is no code in the Hadoop codebase
that enqueues objects explicitly to this queue.
The cleaner thread is essentially a daemon thread that needs to run for the
duration of the runtime to handle this task. If there is no work to be done (no
relevant threads to garbage collect), that it will sit idle on the queue which
is fine. If the program needs to exit and there is an interrupt, the cleaner
thread *does* respond to the interrupt and does an orderly exit (see the while
loop condition). So I'm still wondering what real-world problems we're
observing.
It might be helpful to jog your memory on HADOOP-12107 and HADOOP-12958 for
past analyses that went into this.
> Fix issue of the StatisticsDataReferenceCleaner cleanUp
> -------------------------------------------------------
>
> Key: HADOOP-17728
> URL: https://issues.apache.org/jira/browse/HADOOP-17728
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 3.2.1
> Reporter: yikf
> Assignee: yikf
> Priority: Minor
> Labels: pull-request-available, reverted
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Cleaner thread will be blocked if we remove reference from ReferenceQueue
> unless the `queue.enqueue` called.
> ----
> As shown below, We call ReferenceQueue.remove() now while cleanUp, Call
> chain as follow：
> *StatisticsDataReferenceCleaner#queue.remove() ->
> ReferenceQueue.remove(0) -> lock.wait(0)*
> But, lock.notifyAll is called when queue.enqueue only, so Cleaner thread
> will be blocked.
>
> ThreadDump:
> {code:java}
> "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f7afc088800
> nid=0x2119 in Object.wait() [0x00007f7b00230000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:502)
> at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
> - locked <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
