[
https://issues.apache.org/jira/browse/HADOOP-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12696415#action_12696415
]
Hadoop QA commented on HADOOP-3810:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12404288/globalLock1.patch
against trunk revision 762509.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
-1 Eclipse classpath. The patch causes the Eclipse classpath to differ from
the contents of the lib directories.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/157/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/157/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/157/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/157/console
This message is automatically generated.
> NameNode seems unstable on a cluster with little space left
> -----------------------------------------------------------
>
> Key: HADOOP-3810
> URL: https://issues.apache.org/jira/browse/HADOOP-3810
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.17.1
> Reporter: Raghu Angadi
> Assignee: Hairong Kuang
> Fix For: 0.21.0
>
> Attachments: globalLock.patch, globalLock1.patch, simon-namenode.PNG
>
>
> NameNode seems not very responsive and unstable when the cluster has very
> little space left. The clients timeout. The main problem is that it is not
> clear to the user what is going on. Once I have more details about a NameNode
> that was in this state, I will fill in here.
> If there is not enough space left on a cluster, it is ok for clients to
> receive something like "DiskOutOfSpace" exception.
> Right now it looks like NameNode tries too hard find a node with any space
> left and ends up being slow to respond to clients. If the CPU taken by
> chooseTarger() is the main cause, there are two possible fixes :
> # chooseTarget() iterates and takes quite a bit of CPU for allocating
> datanodes. Usually this not much of a problem. It takes even more cpu when it
> needs to search multiple racks for a datanode. We could probably reduce some
> CPU for these searches. The benefit should be measurable.
> # Once NameNode can not find any datanode that has space on a rack, it could
> mark the rack as "full" and skip searching the rack for next one minute or
> so. This flag gets cleared after a minute or if any new node is added to the
> rack.
> #* Of course, this might not be optimal w.r.t disk space usage.. but only for
> a short duration. Once a cluster is mostly full, the user does expect errors.
> #* On the flip side, this fix does not require extremely CPU optimized
> version of chooseTarget().
> #* I think it is reasonable for NameNode to throw DiskOutOfSpace exception,
> even though it could have found space if it searched much more extensively.
> ---
> edit : minor changes
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
