[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480493#comment-13480493 ] Hudson commented on HBASE-6949: --- Integrated in HBase-TRUNK #3464 (See [https://builds.apache.org/job/HBase-TRUNK/3464/]) HBASE-6949 Automatically delete empty directories in CleanerChore (Revision 1400286) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestCleanerChore.java Automatically delete empty directories in CleanerChore -- Key: HBASE-6949 URL: https://issues.apache.org/jira/browse/HBASE-6949 Project: HBase Issue Type: Bug Affects Versions: 0.94.3, 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch Currently the CleanerChore asks cleaner delegates if both directories and files should be deleted. However, this leads to somewhat odd behavior in some delegates - you don't actually care if the directory hierarchy is preserved, the files; this means you always will delete directories and then implement the logic you actually want for preserving files. Instead we can handle this logic one layer higher in the CleanerChore and let the delegates just worry about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480543#comment-13480543 ] Hudson commented on HBASE-6949: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #228 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/228/]) HBASE-6949 Automatically delete empty directories in CleanerChore (Revision 1400286) Result = FAILURE stack : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestCleanerChore.java Automatically delete empty directories in CleanerChore -- Key: HBASE-6949 URL: https://issues.apache.org/jira/browse/HBASE-6949 Project: HBase Issue Type: Bug Affects Versions: 0.94.3, 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch Currently the CleanerChore asks cleaner delegates if both directories and files should be deleted. However, this leads to somewhat odd behavior in some delegates - you don't actually care if the directory hierarchy is preserved, the files; this means you always will delete directories and then implement the logic you actually want for preserving files. Instead we can handle this logic one layer higher in the CleanerChore and let the delegates just worry about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480288#comment-13480288 ] Jesse Yates commented on HBASE-6949: To follow up on stack's comment, we don't actually have any directory heirarchy for the logs right now - all logs regardless of source (e.g. .logs/[servername]/[hlog]) just get moved to the .oldlogs directory. And since we just added the HFileCleaner there aren't any existing implementations that can be broken by automatically removing empty directories. Automatically delete empty directories in CleanerChore -- Key: HBASE-6949 URL: https://issues.apache.org/jira/browse/HBASE-6949 Project: HBase Issue Type: Bug Affects Versions: 0.94.3, 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.94.3, 0.96.0 Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch Currently the CleanerChore asks cleaner delegates if both directories and files should be deleted. However, this leads to somewhat odd behavior in some delegates - you don't actually care if the directory hierarchy is preserved, the files; this means you always will delete directories and then implement the logic you actually want for preserving files. Instead we can handle this logic one layer higher in the CleanerChore and let the delegates just worry about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480392#comment-13480392 ] Jesse Yates commented on HBASE-6949: Thanks [~stack]! I was going to roll this into the the backport of HBASE-5547 to 0.94 - I'll bug Lars when that's ready. Automatically delete empty directories in CleanerChore -- Key: HBASE-6949 URL: https://issues.apache.org/jira/browse/HBASE-6949 Project: HBase Issue Type: Bug Affects Versions: 0.94.3, 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.96.0 Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch Currently the CleanerChore asks cleaner delegates if both directories and files should be deleted. However, this leads to somewhat odd behavior in some delegates - you don't actually care if the directory hierarchy is preserved, the files; this means you always will delete directories and then implement the logic you actually want for preserving files. Instead we can handle this logic one layer higher in the CleanerChore and let the delegates just worry about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477207#comment-13477207 ] Jesse Yates commented on HBASE-6949: [~stack], [~lhofhansl] what do you guys think? Good to go? Automatically delete empty directories in CleanerChore -- Key: HBASE-6949 URL: https://issues.apache.org/jira/browse/HBASE-6949 Project: HBase Issue Type: Bug Affects Versions: 0.94.3, 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.94.3, 0.96.0 Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch Currently the CleanerChore asks cleaner delegates if both directories and files should be deleted. However, this leads to somewhat odd behavior in some delegates - you don't actually care if the directory hierarchy is preserved, the files; this means you always will delete directories and then implement the logic you actually want for preserving files. Instead we can handle this logic one layer higher in the CleanerChore and let the delegates just worry about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13475424#comment-13475424 ] Jesse Yates commented on HBASE-6949: bq. I think I understand. Delegates just do files and its the cleaner chore that does the dir remove? Is that it? Yes, that is a much better explanation than the one I gave. Do you want an RB of the patch? Automatically delete empty directories in CleanerChore -- Key: HBASE-6949 URL: https://issues.apache.org/jira/browse/HBASE-6949 Project: HBase Issue Type: Bug Affects Versions: 0.94.3, 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.94.3, 0.96.0 Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch Currently the CleanerChore asks cleaner delegates if both directories and files should be deleted. However, this leads to somewhat odd behavior in some delegates - you don't actually care if the directory hierarchy is preserved, the files; this means you always will delete directories and then implement the logic you actually want for preserving files. Instead we can handle this logic one layer higher in the CleanerChore and let the delegates just worry about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[
https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473906#comment-13473906
]
stack commented on HBASE-6949:
--
I like this addition Jesse:
{code}
+// if all the children have been deleted, then we should try to delete
this directory. However,
+// don't do so recursively so we don't delete files that have been added
since we checked.
{code}
The tests look good.
I'm not clear on this ...However, this leads to somewhat odd behavior in some
delegates - you don't actually care if the directory hierarchy is preserved,
the files; this means you always will delete directories and then implement the
logic you actually want for preserving files... I think I understand.
Delegates just do files and its the cleaner chore that does the dir remove? Is
that it?
Automatically delete empty directories in CleanerChore
--
Key: HBASE-6949
URL: https://issues.apache.org/jira/browse/HBASE-6949
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.3, 0.96.0
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.94.3, 0.96.0
Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch
Currently the CleanerChore asks cleaner delegates if both directories and
files should be deleted. However, this leads to somewhat odd behavior in some
delegates - you don't actually care if the directory hierarchy is preserved,
the files; this means you always will delete directories and then implement
the logic you actually want for preserving files. Instead we can handle this
logic one layer higher in the CleanerChore and let the delegates just worry
about preserving files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469637#comment-13469637 ] Jesse Yates commented on HBASE-6949: Attached is the 0.96 version. The 0.94.3 version is part of the backport of HBASE-5547 and will be rolled into that backport, just marking it for some history there. Automatically delete empty directories in CleanerChore -- Key: HBASE-6949 URL: https://issues.apache.org/jira/browse/HBASE-6949 Project: HBase Issue Type: Bug Affects Versions: 0.94.3, 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.94.3, 0.96.0 Attachments: hbase-6949-v0.patch Currently the CleanerChore asks cleaner delegates if both directories and files should be deleted. However, this leads to somewhat odd behavior in some delegates - you don't actually care if the directory hierarchy is preserved, the files; this means you always will delete directories and then implement the logic you actually want for preserving files. Instead we can handle this logic one layer higher in the CleanerChore and let the delegates just worry about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[
https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469718#comment-13469718
]
Hadoop QA commented on HBASE-6949:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12547809/hbase-6949-v0.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3004//console
This message is automatically generated.
Automatically delete empty directories in CleanerChore
--
Key: HBASE-6949
URL: https://issues.apache.org/jira/browse/HBASE-6949
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.3, 0.96.0
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.94.3, 0.96.0
Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch
Currently the CleanerChore asks cleaner delegates if both directories and
files should be deleted. However, this leads to somewhat odd behavior in some
delegates - you don't actually care if the directory hierarchy is preserved,
the files; this means you always will delete directories and then implement
the logic you actually want for preserving files. Instead we can handle this
logic one layer higher in the CleanerChore and let the delegates just worry
about preserving files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[
https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469740#comment-13469740
]
Hadoop QA commented on HBASE-6949:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12547816/hbase-6949-v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
81 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestSplitLogManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/3005//console
This message is automatically generated.
Automatically delete empty directories in CleanerChore
--
Key: HBASE-6949
URL: https://issues.apache.org/jira/browse/HBASE-6949
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.3, 0.96.0
Reporter: Jesse Yates
Assignee: Jesse Yates
Fix For: 0.94.3, 0.96.0
Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch
Currently the CleanerChore asks cleaner delegates if both directories and
files should be deleted. However, this leads to somewhat odd behavior in some
delegates - you don't actually care if the directory hierarchy is preserved,
the files; this means you always will delete directories and then implement
the logic you actually want for preserving files. Instead we can handle this
logic one layer higher in the CleanerChore and let the delegates just worry
about preserving files.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6949) Automatically delete empty directories in CleanerChore
[ https://issues.apache.org/jira/browse/HBASE-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469823#comment-13469823 ] Lars Hofhansl commented on HBASE-6949: -- I like the simplified logic you describe. Need to look at the patch... Automatically delete empty directories in CleanerChore -- Key: HBASE-6949 URL: https://issues.apache.org/jira/browse/HBASE-6949 Project: HBase Issue Type: Bug Affects Versions: 0.94.3, 0.96.0 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.94.3, 0.96.0 Attachments: hbase-6949-v0.patch, hbase-6949-v1.patch Currently the CleanerChore asks cleaner delegates if both directories and files should be deleted. However, this leads to somewhat odd behavior in some delegates - you don't actually care if the directory hierarchy is preserved, the files; this means you always will delete directories and then implement the logic you actually want for preserving files. Instead we can handle this logic one layer higher in the CleanerChore and let the delegates just worry about preserving files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
