[
https://issues.apache.org/jira/browse/HDFS-797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783957#action_12783957
]
Todd Lipcon commented on HDFS-797:
----------------------------------
Strange. It reproducibly takes 12-15 minutes on my laptop (Linux, Sun JDK
1.6.0_16) with a clean checkout. On one of our build boxes (Linux, JDK
1.6.0_14) it takes around 2 minutes like it's supposed to. "Poor man's
profiling" of jstacking every few seconds shows most of the traces like this:
"main" prio=10 tid=0x0000000040d83800 nid=0x147b in Object.wait()
[0x00007f738eeea000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00007f7381623b30> (a
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer)
at java.lang.Thread.join(Thread.java:1143)
- locked <0x00007f7381623b30> (a
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer)
at java.lang.Thread.join(Thread.java:1196)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeThreads(DFSClient.java:3639)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3683)
- locked <0x00007f7381622e58> (a
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:61)
at
org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:86)
at org.apache.hadoop.fs.FsShell.touchz(FsShell.java:818)
Can anyone else reproduce?
> TestHDFSCLI much slower after HDFS-265 merge
> --------------------------------------------
>
> Key: HDFS-797
> URL: https://issues.apache.org/jira/browse/HDFS-797
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 0.21.0, 0.22.0
> Reporter: Todd Lipcon
>
> After the merge of HDFS-265 into trunk, TestHDFSCLI takes much longer to run
> (>10min, got tired of waiting). Changing dataQueue.wait(1000) to
> dataQueue.wait(10) on line 2708 speeds up the test back to its original
> performance. This isn't a fix, but displays the issue - somehow the thread
> isn't getting properly interrupted. In general this probably represents a
> performance regression for the exit of all HDFS writers (most noticeable in
> CLI tools)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
