[
https://issues.apache.org/jira/browse/HDFS-16033?focusedWorklogId=600432&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-600432
]
ASF GitHub Bot logged work on HDFS-16033:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 21/May/21 15:23
Start Date: 21/May/21 15:23
Worklog Time Spent: 10m
Work Description: yikf commented on a change in pull request #3042:
URL: https://github.com/apache/hadoop/pull/3042#discussion_r637005610
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
##########
@@ -4004,12 +4004,13 @@ public void cleanUp() {
* Background action to act on references being removed.
*/
private static class StatisticsDataReferenceCleaner implements Runnable {
+ private int REF_QUEUE_POLL_TIMEOUT = 100;
@Override
public void run() {
while (!Thread.interrupted()) {
try {
StatisticsDataReference ref =
- (StatisticsDataReference)STATS_DATA_REF_QUEUE.remove();
Review comment:
For reviewer, quote JDK ReferenceQueue#:
```
public Reference<? extends T> remove(long timeout)
throws IllegalArgumentException, InterruptedException
{
if (timeout < 0) {
throw new IllegalArgumentException("Negative timeout value");
}
synchronized (lock) {
Reference<? extends T> r = reallyPoll();
if (r != null) return r;
long start = (timeout == 0) ? 0 : System.nanoTime();
for (;;) {
lock.wait(timeout);
r = reallyPoll();
if (r != null) return r;
if (timeout != 0) {
long end = System.nanoTime();
timeout -= (end - start) / 1000_000;
if (timeout <= 0) return null;
start = end;
}
}
}
}
/**
* Removes the next reference object in this queue, blocking until one
* becomes available.
*
* @return A reference object, blocking until one becomes available
* @throws InterruptedException If the wait is interrupted
*/
public Reference<? extends T> remove() throws InterruptedException {
return remove(0);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 600432)
Time Spent: 1h (was: 50m)
> Fix issue of the StatisticsDataReferenceCleaner cleanUp
> -------------------------------------------------------
>
> Key: HDFS-16033
> URL: https://issues.apache.org/jira/browse/HDFS-16033
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 3.2.1
> Reporter: yikf
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Cleaner thread will be blocked if we remove reference from ReferenceQueue
> unless the `queue.enqueue` called.
> ----
> As shown below, We call ReferenceQueue.remove() now while cleanUp, Call
> chain as follow：
> *StatisticsDataReferenceCleaner#queue.remove() ->
> ReferenceQueue.remove(0) -> lock.wait(0)*
> But, lock.notifyAll is called when queue.enqueue only, so Cleaner thread
> will be blocked.
>
> ThreadDump:
> {code:java}
> "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f7afc088800
> nid=0x2119 in Object.wait() [0x00007f7b00230000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:502)
> at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
> - locked <0x00000000c00c2f58> (a java.lang.ref.Reference$Lock)
> at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
