[jira] [Updated] (HBASE-8101) Cleanup: findbugs and javadoc warning fixes as well as making it illegal passing null row to Put/Delete, etc.
[ https://issues.apache.org/jira/browse/HBASE-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8101: - Status: Patch Available (was: Open) > Cleanup: findbugs and javadoc warning fixes as well as making it illegal > passing null row to Put/Delete, etc. > - > > Key: HBASE-8101 > URL: https://issues.apache.org/jira/browse/HBASE-8101 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack > Fix For: 0.95.0 > > Attachments: 8101.txt > > > Part of hbase-7900 broken out so that patch gets smaller. This is a patch > with cleanup mostly findbugs fixes (general ones) as well as adding check for > null row being passed to Put, Get, etc. This patch helps rpc along. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[
https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602090#comment-13602090
]
Hadoop QA commented on HBASE-8099:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573684/8099-example.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4812//console
This message is automatically generated.
> ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues
> if it failed to execute.
> -
>
> Key: HBASE-8099
> URL: https://issues.apache.org/jira/browse/HBASE-8099
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Himanshu Vashishtha
>Priority: Blocker
> Fix For: 0.95.0, 0.98.0, 0.94.6
>
> Attachments: 8099-example.txt, HBase-8099-94.patch,
> HBase-8099-94-v2.patch, HBase-8099-94-v3.patch, HBase-8099-trunk-2.patch,
> HBase-8099-trunk.patch, HBase-8099-trunk-v3.patch
>
>
> We just ran into an interesting scenario. We restarted a cluster that was
> setup as a replication source.
> The stop went cleanly.
> Upon restart *all* regionservers aborted within a few seconds with variations
> of these errors:
> http://pastebin.com/3iQVuBqS
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[
https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602089#comment-13602089
]
Hadoop QA commented on HBASE-8099:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573672/HBase-8099-trunk-v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4811//console
This message is automatically generated.
> ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues
> if it failed to execute.
> -
>
> Key: HBASE-8099
> URL: https://issues.apache.org/jira/browse/HBASE-8099
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Himanshu Vashishtha
>Priority: Blocker
> Fix For: 0.95.0, 0.98.0, 0.94.6
>
> Attachments: 8099-example.txt, HBase-8099-94.patch,
> HBase-8099-94-v2.patch, HBase-8099-94-v3.patch, HBase-8099-trunk-2.patch,
> HBase-8099-trunk.patch, HBase-8099-trunk-v3.patch
>
>
> We just ran into an interesting scenario. We restarted a cluster that was
> setup as a replication source.
> The stop went cleanly.
> Upon restart *all* regionservers aborted within a few seconds with variations
> of these errors:
> http://pastebin.com/3iQVuBqS
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8099: - Attachment: 8099-example.txt Like this. > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.95.0, 0.98.0, 0.94.6 > > Attachments: 8099-example.txt, HBase-8099-94.patch, > HBase-8099-94-v2.patch, HBase-8099-94-v3.patch, HBase-8099-trunk-2.patch, > HBase-8099-trunk.patch, HBase-8099-trunk-v3.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8099: - Fix Version/s: 0.98.0 0.95.0 > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.95.0, 0.98.0, 0.94.6 > > Attachments: HBase-8099-94.patch, HBase-8099-94-v2.patch, > HBase-8099-94-v3.patch, HBase-8099-trunk-2.patch, HBase-8099-trunk.patch, > HBase-8099-trunk-v3.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8099: - Fix Version/s: (was: 0.94.7) 0.94.6 > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.94.6 > > Attachments: HBase-8099-94.patch, HBase-8099-94-v2.patch, > HBase-8099-94-v3.patch, HBase-8099-trunk-2.patch, HBase-8099-trunk.patch, > HBase-8099-trunk-v3.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602085#comment-13602085 ] Lars Hofhansl commented on HBASE-8099: -- With that change you can also leave the initialization of queues where it was and remove the null check in the run() method, which is nicer I think... I.e. what leaves copyQueuesFromRSUsingMulti is a list of queues or an empty list (just as it is case for copyQueuesFromRS) > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.94.7 > > Attachments: HBase-8099-94.patch, HBase-8099-94-v2.patch, > HBase-8099-94-v3.patch, HBase-8099-trunk-2.patch, HBase-8099-trunk.patch, > HBase-8099-trunk-v3.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8103) Fix pom so 0.94 can generate site reports
[ https://issues.apache.org/jira/browse/HBASE-8103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-8103. -- Resolution: Fixed Fix Version/s: 0.94.6 Assignee: stack Committed to 0.94 branch. > Fix pom so 0.94 can generate site reports > - > > Key: HBASE-8103 > URL: https://issues.apache.org/jira/browse/HBASE-8103 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.94.6 > > Attachments: 8103.txt > > > Site and info plugins are too old. The site plugin is in the wrong place. > Can't generate reports w/o update and move of location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8103) Fix pom so 0.94 can generate site reports
stack created HBASE-8103: Summary: Fix pom so 0.94 can generate site reports Key: HBASE-8103 URL: https://issues.apache.org/jira/browse/HBASE-8103 Project: HBase Issue Type: Bug Reporter: stack Attachments: 8103.txt Site and info plugins are too old. The site plugin is in the wrong place. Can't generate reports w/o update and move of location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8103) Fix pom so 0.94 can generate site reports
[ https://issues.apache.org/jira/browse/HBASE-8103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8103: - Attachment: 8103.txt Small change to update site and info plugins and moved site out of reporting to be actual plugin. > Fix pom so 0.94 can generate site reports > - > > Key: HBASE-8103 > URL: https://issues.apache.org/jira/browse/HBASE-8103 > Project: HBase > Issue Type: Bug >Reporter: stack > Attachments: 8103.txt > > > Site and info plugins are too old. The site plugin is in the wrong place. > Can't generate reports w/o update and move of location. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6991) Escape "\" in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()
[
https://issues.apache.org/jira/browse/HBASE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602062#comment-13602062
]
Hudson commented on HBASE-6991:
---
Integrated in HBase-0.94 #898 (See
[https://builds.apache.org/job/HBase-0.94/898/])
HBASE-8085 Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
(Tianying Chang) (Revision 1456306)
Result = FAILURE
enis :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockCompatibility.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestBytes.java
> Escape "\" in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()
> --
>
> Key: HBASE-6991
> URL: https://issues.apache.org/jira/browse/HBASE-6991
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.96.0
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
> Fix For: 0.95.0
>
> Attachments: HBASE-6991_trunk.patch
>
>
> Since "\" is used to escape non-printable character but not treated as
> special character in conversion, it could lead to unexpected conversion.
> For example, please consider the following code snippet.
> {code}
> public void testConversion() {
> byte[] original = {
> '\\', 'x', 'A', 'D'
> };
> String stringFromBytes = Bytes.toStringBinary(original);
> byte[] converted = Bytes.toBytesBinary(stringFromBytes);
> System.out.println("Original: " + Arrays.toString(original));
> System.out.println("Converted: " + Arrays.toString(converted));
> System.out.println("Reversible?: " + (Bytes.compareTo(original, converted)
> == 0));
> }
> Output:
> ---
> Original: [92, 120, 65, 68]
> Converted: [-83]
> Reversible?: false
> {code}
> The "\" character needs to be treated as special and must be encoded as a
> non-printable character ("\x5C") to avoid any kind of ambiguity during
> conversion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8040) Race condition in AM after HBASE-7521 (only 0.94)
[ https://issues.apache.org/jira/browse/HBASE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602061#comment-13602061 ] Hudson commented on HBASE-8040: --- Integrated in HBase-0.94 #898 (See [https://builds.apache.org/job/HBase-0.94/898/]) HBASE-8040 - Race condition in AM after HBASE-7521 (only 0.94) (Ram) (Revision 1456310) Result = FAILURE ramkrishna : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java > Race condition in AM after HBASE-7521 (only 0.94) > - > > Key: HBASE-8040 > URL: https://issues.apache.org/jira/browse/HBASE-8040 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.94.6 > > Attachments: HBASE-8040_1.patch, HBASE-8040.patch > > > This is a problem that introduced when we tried to solve HBASE-7521. > https://issues.apache.org/jira/browse/HBASE-7521?focusedCommentId=13576083&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13576083 > See the above comment and exactly the same has happened. Will come up with a > solution for the same. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7521) fix HBASE-6060 (regions stuck in opening state) in 0.94
[ https://issues.apache.org/jira/browse/HBASE-7521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602060#comment-13602060 ] Hudson commented on HBASE-7521: --- Integrated in HBase-0.94 #898 (See [https://builds.apache.org/job/HBase-0.94/898/]) HBASE-8040 - Race condition in AM after HBASE-7521 (only 0.94) (Ram) (Revision 1456310) Result = FAILURE ramkrishna : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java > fix HBASE-6060 (regions stuck in opening state) in 0.94 > --- > > Key: HBASE-7521 > URL: https://issues.apache.org/jira/browse/HBASE-7521 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.6 > > Attachments: 7521-original-patch-ported-v4.patch, > HBASE-7521-original-patch-ported-v0.patch, > HBASE-7521-original-patch-ported-v1.patch, > HBASE-7521-original-patch-ported-v2.patch, > HBASE-7521-original-patch-ported-v3.patch, HBASE-7521-v0.patch, > HBASE-7521-v1.patch, HBASE-7521-v5.patch > > > Discussion in HBASE-6060 implies that the fix there does not work on 0.94. > Still, we may want to fix the issue in 0.94 (via some different fix) because > the regions stuck in opening for ridiculous amounts of time is not a good > thing to have. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602059#comment-13602059
]
Hudson commented on HBASE-8085:
---
Integrated in HBase-0.94 #898 (See
[https://builds.apache.org/job/HBase-0.94/898/])
HBASE-8085 Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
(Tianying Chang) (Revision 1456306)
Result = FAILURE
enis :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/Bytes.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockCompatibility.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestBytes.java
> Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
>
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.6
>
> Attachments: HBASE-8085.patch, HBASE-8085-v2.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602058#comment-13602058 ] Hudson commented on HBASE-8088: --- Integrated in HBase-0.94 #898 (See [https://builds.apache.org/job/HBase-0.94/898/]) HBASE-8088 Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site; SVN ADD NEW FILES (Revision 1456317) Result = FAILURE stack : Files : * /hbase/branches/0.94/src/docbkx/case_studies.xml * /hbase/branches/0.94/src/docbkx/community.xml * /hbase/branches/0.94/src/docbkx/security.xml * /hbase/branches/0.94/src/docbkx/zookeeper.xml * /hbase/branches/0.94/src/site/resources/doap_Hbase.rdf * /hbase/branches/0.94/src/site/xdoc/resources.xml > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.6 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7905) Add passing of optional cell blocks over rpc
[ https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7905: - Attachment: 7900v12-depends-on-8101.txt Smaller patch. Depends on hbase-8101 going in first. > Add passing of optional cell blocks over rpc > > > Key: HBASE-7905 > URL: https://issues.apache.org/jira/browse/HBASE-7905 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 0.95.0 > > Attachments: 7900v12-depends-on-8101.txt, 7905.txt, 7905v3.txt, > 7905v4.txt, 7905v6.txt, 7905v8.txt, 7905v9.txt > > > Make it so we can pass Cells/data w/o having to bury it all in protobuf to > get it over the wire. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8102) Replication NodeFailoverWorker should check other rs znodes before proceeding
Himanshu Vashishtha created HBASE-8102: -- Summary: Replication NodeFailoverWorker should check other rs znodes before proceeding Key: HBASE-8102 URL: https://issues.apache.org/jira/browse/HBASE-8102 Project: HBase Issue Type: Improvement Components: Replication Affects Versions: 0.94.5 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Fix For: 0.95.0, 0.94.7 NodeFailoverWorker takes the dead rs znode and starts processing it (moving the log znodes under it, etc). Even a regionserver restart will trigger this znodes movement. This cause some other regionserver to read the log files remotely, even though, the original regionserver is up. Ideally, it should check available regionserver znodes after it comes out it sleep in its run method in order to decide whether it should really process the znode or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602049#comment-13602049 ] Himanshu Vashishtha commented on HBASE-8099: Hmm, I added the random member to the Failover worker. I just noticed that you mentioned to add that to the RSM class rather. I don't have any strong opinion for this. Let me know and I'll change it. > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.94.7 > > Attachments: HBase-8099-94.patch, HBase-8099-94-v2.patch, > HBase-8099-94-v3.patch, HBase-8099-trunk-2.patch, HBase-8099-trunk.patch, > HBase-8099-trunk-v3.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8099: --- Attachment: HBase-8099-trunk-v3.patch HBase-8099-94-v3.patch Adding jitter in the NodeFailoverworker > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.94.7 > > Attachments: HBase-8099-94.patch, HBase-8099-94-v2.patch, > HBase-8099-94-v3.patch, HBase-8099-trunk-2.patch, HBase-8099-trunk.patch, > HBase-8099-trunk-v3.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8101) Cleanup: findbugs and javadoc warning fixes as well as making it illegal passing null row to Put/Delete, etc.
[ https://issues.apache.org/jira/browse/HBASE-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602029#comment-13602029 ] stack commented on HBASE-8101: -- rb is here https://reviews.apache.org/r/9919/ > Cleanup: findbugs and javadoc warning fixes as well as making it illegal > passing null row to Put/Delete, etc. > - > > Key: HBASE-8101 > URL: https://issues.apache.org/jira/browse/HBASE-8101 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack > Fix For: 0.95.0 > > Attachments: 8101.txt > > > Part of hbase-7900 broken out so that patch gets smaller. This is a patch > with cleanup mostly findbugs fixes (general ones) as well as adding check for > null row being passed to Put, Get, etc. This patch helps rpc along. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7295) Contention in HBaseClient.getConnection
[ https://issues.apache.org/jira/browse/HBASE-7295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602025#comment-13602025 ] Varun Sharma commented on HBASE-7295: - I would like to revive this issue. We have been running with this patch for over a month now. Just to recap on the patch, we had contention issues at one of our thrift proxy(s). The proxy does the following: python client(s) <-> Thrift Proxy <-> HBase The Proxy upto thousands of calls per second. However there are certain calls which result in multi operations - these operations generate 30 calls for every single call (which are upto 200-300 per second) - so basically this requires locking the pool of connections, thousands of times per second - once for each fanout call. I have not rerun the tests yet against the most current 0.94 release but I would like to pull it back into 0.94 (since we have tested it in production for a while now). Varun > Contention in HBaseClient.getConnection > --- > > Key: HBASE-7295 > URL: https://issues.apache.org/jira/browse/HBASE-7295 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.3 >Reporter: Varun Sharma >Assignee: Varun Sharma > Fix For: 0.95.0 > > Attachments: 7295-0.94.txt, 7295-0.94-v2.txt, 7295-0.94-v3.txt, > 7295-0.94-v4.txt, 7295-0.94-v5.txt, 7295-trunk.txt, 7295-trunk.txt, > 7295-trunk-v2.txt, 7295-trunk-v3.txt, 7295-trunk-v3.txt > > > HBaseClient.getConnection() synchronizes on the connections object. We found > severe contention on a thrift gateway which was fanning out roughly 3000+ > calls per second to hbase region servers. The thrift gateway had 2000+ > threads for handling incoming connections. Threads were blocked on the > syncrhonized block - we set ipc.pool.size to 200. Since we are using > RoundRobin/ThreadLocal pool only - its not necessary to synchronize on > connections - it might lead to cases where we might go slightly over the > ipc.max.pool.size() but the additional connections would timeout after > maxIdleTime - underlying PoolMap connections object is thread safe. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8101) Cleanup: findbugs and javadoc warning fixes as well as making it illegal passing null row to Put/Delete, etc.
[ https://issues.apache.org/jira/browse/HBASE-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8101: - Attachment: 8101.txt Cleanup, findbugs fixes, and make it so illegal pass null row to Put/Delete, etc. CatalogTracker -- just cleanup of whitespace and unused data members. Action -- findbugs fix Append -- Add check for null row and all passing of row, offset,length so easier working w/ Cells. Delete -- ditto. Throw new WrongRowException if not matching row. Increment -- ditto plus findbugs and allow adding a Cell/KeyValue. Get -- findbugs Put -- ditto HBaseAdmin -- javadoc warning fixes. RwoMutations -- findbugs WrongRowException -- new exception if Cell row does not match that of the Put, Delete, etc. host NullComparator -- findbugs AuthMethod -- make valueOf public.. need to look at it down in ipc CellComparator -- findbugs KeyValue -- findbugs CellCodec/BaseDecoder/BaseEncoder -- make accessible from other packages ByteBufferOutputStream -- move to hbase-common and into io package. DiffKeyDeltaEncoder -- findbugs Bytes -- add copy method. TestPutDeleteEtcCellIteration -- tests to Iterate Cells. > Cleanup: findbugs and javadoc warning fixes as well as making it illegal > passing null row to Put/Delete, etc. > - > > Key: HBASE-8101 > URL: https://issues.apache.org/jira/browse/HBASE-8101 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack > Fix For: 0.95.0 > > Attachments: 8101.txt > > > Part of hbase-7900 broken out so that patch gets smaller. This is a patch > with cleanup mostly findbugs fixes (general ones) as well as adding check for > null row being passed to Put, Get, etc. This patch helps rpc along. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8101) Cleanup: findbugs and javadoc warning fixes as well as making it illegal passing null row to Put/Delete, etc.
stack created HBASE-8101: Summary: Cleanup: findbugs and javadoc warning fixes as well as making it illegal passing null row to Put/Delete, etc. Key: HBASE-8101 URL: https://issues.apache.org/jira/browse/HBASE-8101 Project: HBase Issue Type: Sub-task Reporter: stack Part of hbase-7900 broken out so that patch gets smaller. This is a patch with cleanup mostly findbugs fixes (general ones) as well as adding check for null row being passed to Put, Get, etc. This patch helps rpc along. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7905) Add passing of optional cell blocks over rpc
[ https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7905: - Assignee: stack Status: Open (was: Patch Available) Let me break up this patch. I'm having trouble getting it in. Will piecemeal it. > Add passing of optional cell blocks over rpc > > > Key: HBASE-7905 > URL: https://issues.apache.org/jira/browse/HBASE-7905 > Project: HBase > Issue Type: Sub-task > Components: IPC/RPC >Reporter: stack >Assignee: stack > Fix For: 0.95.0 > > Attachments: 7905.txt, 7905v3.txt, 7905v4.txt, 7905v6.txt, > 7905v8.txt, 7905v9.txt > > > Make it so we can pass Cells/data w/o having to bury it all in protobuf to > get it over the wire. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[
https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602012#comment-13602012
]
Lars Hofhansl commented on HBASE-8099:
--
That works. Personally I'd probably just return queues in the first case and do
a clear() for the second like this:
{code}
- if (peerIdsToProcess == null) return null; // node already processed
+ if (peerIdsToProcess == null) return queues; // node already processed
...
LOG.warn("Got exception in copyQueuesFromRSUsingMulti: ", e);
+ queues.clear();
{code}
Maybe while we're add it, we could add a random jitter to the failover.
Add a Random member to ReplicationSourceManager and than do this in
NodeFailoverWorker:
{code}
-Thread.sleep(sleepBeforeFailover);
+Thread.sleep(sleepBeforeFailover +
(long)(random.nextFloat()*sleepBeforeFailover));
{code}
> ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues
> if it failed to execute.
> -
>
> Key: HBASE-8099
> URL: https://issues.apache.org/jira/browse/HBASE-8099
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Himanshu Vashishtha
>Priority: Blocker
> Fix For: 0.94.7
>
> Attachments: HBase-8099-94.patch, HBase-8099-94-v2.patch,
> HBase-8099-trunk-2.patch, HBase-8099-trunk.patch
>
>
> We just ran into an interesting scenario. We restarted a cluster that was
> setup as a replication source.
> The stop went cleanly.
> Upon restart *all* regionservers aborted within a few seconds with variations
> of these errors:
> http://pastebin.com/3iQVuBqS
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8094) TestTableInputFormatScan doesn't assert anything
[
https://issues.apache.org/jira/browse/HBASE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602010#comment-13602010
]
Hudson commented on HBASE-8094:
---
Integrated in hbase-0.95 #72 (See
[https://builds.apache.org/job/hbase-0.95/72/])
HBASE-8094 TestTableInputFormatScan doesn't assert anything (Nick Dimiduk)
(Revision 1456286)
Result = FAILURE
enis :
Files :
*
/hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> TestTableInputFormatScan doesn't assert anything
>
>
> Key: HBASE-8094
> URL: https://issues.apache.org/jira/browse/HBASE-8094
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.0, 0.98.0
>
> Attachments: 0001-HBASE-8094-fix-broken-TestTableInputFormatScan.patch
>
>
> This test makes assertions from within the Reducer. These tests are not
> respected because the control harness asserts only that jobs complete, not
> that they succeed.
> Verify this is true by adding a failure to the reducer:
> {noformat}
> $ git diff
> diff --git
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
>
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> index bab9633..322cb4f 100644
> ---
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> +++
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> @@ -160,6 +160,7 @@ public class TestTableInputFormatScan {
>if (lastRow != null && lastRow.length() > 0) {
> assertEquals(lastRow, last);
>}
> + assertTrue(false);
> }
>
>}
> {noformat}
> The test still passes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7827) Improve the speed of Hbase Thirft Batch mutation for deletes
[ https://issues.apache.org/jira/browse/HBASE-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-7827: - Fix Version/s: (was: 0.94.7) 0.94.6 > Improve the speed of Hbase Thirft Batch mutation for deletes > > > Key: HBASE-7827 > URL: https://issues.apache.org/jira/browse/HBASE-7827 > Project: HBase > Issue Type: Improvement > Components: Thrift >Affects Versions: 0.94.0 >Reporter: Shivendra Pratap Singh >Assignee: Shivendra Pratap Singh >Priority: Minor > Labels: Hbase, Thrift > Fix For: 0.95.0, 0.98.0, 0.94.6 > > Attachments: 7827-trunk.txt, hbase_7827.patch > > > A batch mutate operation does both puts and deletes. Batch mutate for put > uses table.put(puts) however batch mutate for delete loops over all deletes > and calls table.delete for every single cell. This causes delete performance > to degrade. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8040) Race condition in AM after HBASE-7521 (only 0.94)
[ https://issues.apache.org/jira/browse/HBASE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8040: - Fix Version/s: (was: 0.94.7) 0.94.6 > Race condition in AM after HBASE-7521 (only 0.94) > - > > Key: HBASE-8040 > URL: https://issues.apache.org/jira/browse/HBASE-8040 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.94.6 > > Attachments: HBASE-8040_1.patch, HBASE-8040.patch > > > This is a problem that introduced when we tried to solve HBASE-7521. > https://issues.apache.org/jira/browse/HBASE-7521?focusedCommentId=13576083&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13576083 > See the above comment and exactly the same has happened. Will come up with a > solution for the same. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8088: - Fix Version/s: (was: 0.94.7) 0.94.6 > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.6 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8031) Adopt goraci as an Integration test
[
https://issues.apache.org/jira/browse/HBASE-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8031:
-
Fix Version/s: (was: 0.94.7)
0.94.6
> Adopt goraci as an Integration test
> ---
>
> Key: HBASE-8031
> URL: https://issues.apache.org/jira/browse/HBASE-8031
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.0, 0.98.0, 0.94.6
>
> Attachments: hbase-8031_v1-0.94.patch, hbase-8031_v1.patch
>
>
> As you might know, I am a big fan of the goraci test that Keith Turner has
> developed, which in turn is inspired by the Accumulo test called Continuous
> Ingest.
> As much as I hate to say it, having to rely on gora and and external github
> library makes using this lib cumbersome. And lately we had to use this for
> testing against secure clusters and with Hadoop2, which gora does not support
> for now.
> So, I am proposing we add this test as an IT in the HBase code base so that
> all HBase devs can benefit from it.
> The original source code can be found here:
> * https://github.com/keith-turner/goraci
> * https://github.com/enis/goraci/
> From the javadoc:
> {code}
> Apache Accumulo [0] has a simple test suite that verifies that data is not
> * lost at scale. This test suite is called continuous ingest. This test runs
> * many ingest clients that continually create linked lists containing 25
> * million nodes. At some point the clients are stopped and a map reduce job
> is
> * run to ensure no linked list has a hole. A hole indicates data was lost.··
> *
> * The nodes in the linked list are random. This causes each linked list to
> * spread across the table. Therefore if one part of a table loses data, then
> it
> * will be detected by references in another part of the table.
> *
> Below is rough sketch of how data is written. For specific details look at
> * the Generator code.
> *
> * 1 Write out 1 million nodes· 2 Flush the client· 3 Write out 1 million that
> * reference previous million· 4 If this is the 25th set of 1 million nodes,
> * then update 1st set of million to point to last· 5 goto 1
> *
> * The key is that nodes only reference flushed nodes. Therefore a node should
> * never reference a missing node, even if the ingest client is killed at any
> * point in time.
> *
> * Some ASCII art time:
> * [ . . . ] represents one batch of random longs of length WIDTH
> *
> *_
> * | __ |
> * | | ||
> * __+_+_ ||
> * v v v |||
> * first = [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * prev= [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * current = [ . . . . . . . . . . . ] |||
> * |||
> * ... |||
> * |||
> * last= [ . . . . . . . . . . . ] |||
> * | | | | | | | | | | |-|||
> * | |||
> * |___|
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8085:
-
Fix Version/s: (was: 0.94.7)
0.94.6
> Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
>
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.6
>
> Attachments: HBASE-8085.patch, HBASE-8085-v2.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8055) Null check missing in StoreFile.Reader.getMaxTimestamp()
[
https://issues.apache.org/jira/browse/HBASE-8055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8055:
-
Fix Version/s: (was: 0.94.7)
0.94.6
> Null check missing in StoreFile.Reader.getMaxTimestamp()
>
>
> Key: HBASE-8055
> URL: https://issues.apache.org/jira/browse/HBASE-8055
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 0.95.0, 0.98.0, 0.94.6
>
> Attachments: 8055-0.94.txt
>
>
> We just ran into a scenario where we got the following NPE:
> {code}
> 13/03/08 11:52:13 INFO regionserver.Store: Successfully loaded store file
> file:/tmp/hfile-import-00Dxx001lmJ-09CxxJm/COLFAM/file09CxxJm
> into store COLFAM (new location:
> file:/tmp/localhbase/data/SFDC.ENTITY_HISTORY_ARCHIVE/aeacee43aaf1748c6e60b9cc12bcac3d/COLFAM/120d683414e44478984b50ddd79b6826)
> 13/03/08 11:52:13 ERROR regionserver.HRegionServer: Failed openScanner
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.getMaxTimestamp(StoreFile.java:1702)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.requestSeek(StoreFileScanner.java:301)
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:127)
> at org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2070)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3383)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1628)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1620)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1596)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2342)
> at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1400)
> 13/03/08 11:52:14 ERROR regionserver.HRegionServer: Failed openScanner
> {code}
> It's not clear, yet, how we got into this situation (we are generating HFiles
> via HFileOutputFormat and bulk load those). It seems that can only happen
> when the HFile itself is corrupted.
> Looking at the code, though, I see this is the only place where we access
> StoreFile.reader.timeRangeTracker without a null check. So it appears we are
> expecting scenarios in which it can be null.
> A simple fix would be:
> {code}
> public long getMaxTimestamp() {
> return timeRangeTracker == null ? Long.MAX_VALUE :
> timeRangeTracker.maximumTimestamp;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8061) Missing test from TestFlushSnapshotFromClient in 0.94
[ https://issues.apache.org/jira/browse/HBASE-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8061: - Fix Version/s: (was: 0.94.7) 0.94.6 > Missing test from TestFlushSnapshotFromClient in 0.94 > - > > Key: HBASE-8061 > URL: https://issues.apache.org/jira/browse/HBASE-8061 > Project: HBase > Issue Type: Bug > Components: snapshots >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: 0.94.6 > > Attachments: HBASE-8061-v0.patch, HBASE-8061-v1.patch, > HBASE-8061-v2.patch, long-lines-trunk.patch > > > There's a missing test from TestFlushSnapshotFromClient (0.94) and a missing > delete snapshot at the end of another test that cases jdk7 to fail, since the > tests are executed in a different order. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8069) TestHLog is dependent on the execution order
[
https://issues.apache.org/jira/browse/HBASE-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8069:
-
Fix Version/s: (was: 0.94.7)
0.94.6
> TestHLog is dependent on the execution order
>
>
> Key: HBASE-8069
> URL: https://issues.apache.org/jira/browse/HBASE-8069
> Project: HBase
> Issue Type: Bug
> Components: test, wal
>Affects Versions: 0.95.0, 0.94.5
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 0.95.0, 0.94.6
>
> Attachments: HBASE-8069-v0.patch, HBASE-8069-v0.patch
>
>
> Changing the execution order of the tests, TestHLog show up as failing
> {code}alphabetical{code}
> testAppendClose() changes the DFS cluster of the test (set globally) so the
> tests executed after has the new DFS. Trying to start a new mini cluster for
> every test @Before seems to solve the problem.
> {code}
> testSplit(org.apache.hadoop.hbase.regionserver.wal.TestHLog): 3 exceptions
> [org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException):
> No lease on
> /user/th30z/hbase/TestHLog/21985ea121a1d65fa82d90d521be7313/recovered.edits/001.temp
> File is not open for writing. Holder DFSClient_NONMAPREDUCE_1044150136_583
> does not have any open files.
> {code}
> Looking at testSplit() the failure seems to be in logSplitter.splitLog(),
> OutputSink.finishWritingAndClose() is not able to close the files and rethrow
> the exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[
https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602008#comment-13602008
]
Hadoop QA commented on HBASE-8099:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573661/HBase-8099-trunk-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.backup.TestHFileArchiving
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4810//console
This message is automatically generated.
> ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues
> if it failed to execute.
> -
>
> Key: HBASE-8099
> URL: https://issues.apache.org/jira/browse/HBASE-8099
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Himanshu Vashishtha
>Priority: Blocker
> Fix For: 0.94.7
>
> Attachments: HBase-8099-94.patch, HBase-8099-94-v2.patch,
> HBase-8099-trunk-2.patch, HBase-8099-trunk.patch
>
>
> We just ran into an interesting scenario. We restarted a cluster that was
> setup as a replication source.
> The stop went cleanly.
> Upon restart *all* regionservers aborted within a few seconds with variations
> of these errors:
> http://pastebin.com/3iQVuBqS
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[
https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602007#comment-13602007
]
Lars Hofhansl commented on HBASE-8088:
--
He the dreaded missing {{svn add}} :)
I would have done that, but wasn't sure whether there was a reason that those
were not copied.
> Versioning site: part one, put stake in the ground for 0.94 by copying
> current versions of book and site
>
>
> Key: HBASE-8088
> URL: https://issues.apache.org/jira/browse/HBASE-8088
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
> Fix For: 0.94.7
>
> Attachments: addendum.txt
>
>
> Doug Meil suggests its time we started versioning doc and site. First step
> is copying current version over to 0.94 so we have something to work against
> (site and book have only been kept up on trunk up to this). Let me do this
> much for now. Can figure how to do the rest later.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602006#comment-13602006 ] stack commented on HBASE-8088: -- Sorry. Forgot to svn add the new files. Fixing now. > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602004#comment-13602004 ] stack commented on HBASE-8088: -- Sorry [~lhofhansl] Was out and about today. I've actually been working on getting 0.94 version of site up on hbase.org... The site target seems to work for me. Let me look at the 0.94 build fails > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8066) Provide Admin.isTableAvailable() for a given table along with splitkeys
[ https://issues.apache.org/jira/browse/HBASE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8066: -- Attachment: HBASE-8066_addendum.patch > Provide Admin.isTableAvailable() for a given table along with splitkeys > --- > > Key: HBASE-8066 > URL: https://issues.apache.org/jira/browse/HBASE-8066 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.95.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8066_1.patch, HBASE-8066_addendum.patch, > HBASE-8066.patch > > > As part of HBASE-5583 if the master reboots during creation of table there is > a chance that the table gets created with partial split keys. This api helps > to check if the table was created with the required number of splitkeys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8066) Provide Admin.isTableAvailable() for a given table along with splitkeys
[ https://issues.apache.org/jira/browse/HBASE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8066: -- Attachment: (was: HBASE-8066_1.patch) > Provide Admin.isTableAvailable() for a given table along with splitkeys > --- > > Key: HBASE-8066 > URL: https://issues.apache.org/jira/browse/HBASE-8066 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.95.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8066_1.patch, HBASE-8066_addendum.patch, > HBASE-8066.patch > > > As part of HBASE-5583 if the master reboots during creation of table there is > a chance that the table gets created with partial split keys. This api helps > to check if the table was created with the required number of splitkeys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8066) Provide Admin.isTableAvailable() for a given table along with splitkeys
[ https://issues.apache.org/jira/browse/HBASE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8066: -- Attachment: HBASE-8066_1.patch Committed the following addendum. Thanks Ted for the follow up. > Provide Admin.isTableAvailable() for a given table along with splitkeys > --- > > Key: HBASE-8066 > URL: https://issues.apache.org/jira/browse/HBASE-8066 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.95.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8066_1.patch, HBASE-8066_addendum.patch, > HBASE-8066.patch > > > As part of HBASE-5583 if the master reboots during creation of table there is > a chance that the table gets created with partial split keys. This api helps > to check if the table was created with the required number of splitkeys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7992) provide pre/post region offline hooks for HMaster.offlineRegion().
[ https://issues.apache.org/jira/browse/HBASE-7992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13602002#comment-13602002 ] Anoop Sam John commented on HBASE-7992: --- Got your point Andrew. Yes We can go with Option 1 for the scope of this issue. Thanks > provide pre/post region offline hooks for HMaster.offlineRegion(). > --- > > Key: HBASE-7992 > URL: https://issues.apache.org/jira/browse/HBASE-7992 > Project: HBase > Issue Type: Bug > Components: Coprocessors >Affects Versions: 0.95.0 >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.0 > > Attachments: HBASE-7992_trunk.patch > > > presently no hooks to provide access control to offline region in master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8094) TestTableInputFormatScan doesn't assert anything
[
https://issues.apache.org/jira/browse/HBASE-8094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601999#comment-13601999
]
Hudson commented on HBASE-8094:
---
Integrated in HBase-TRUNK #3957 (See
[https://builds.apache.org/job/HBase-TRUNK/3957/])
HBASE-8094 TestTableInputFormatScan doesn't assert anything (Nick Dimiduk)
(Revision 1456285)
Result = FAILURE
enis :
Files :
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> TestTableInputFormatScan doesn't assert anything
>
>
> Key: HBASE-8094
> URL: https://issues.apache.org/jira/browse/HBASE-8094
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, test
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.0, 0.98.0
>
> Attachments: 0001-HBASE-8094-fix-broken-TestTableInputFormatScan.patch
>
>
> This test makes assertions from within the Reducer. These tests are not
> respected because the control harness asserts only that jobs complete, not
> that they succeed.
> Verify this is true by adding a failure to the reducer:
> {noformat}
> $ git diff
> diff --git
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
>
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> index bab9633..322cb4f 100644
> ---
> a/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> +++
> b/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java
> @@ -160,6 +160,7 @@ public class TestTableInputFormatScan {
>if (lastRow != null && lastRow.length() > 0) {
> assertEquals(lastRow, last);
>}
> + assertTrue(false);
> }
>
>}
> {noformat}
> The test still passes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8040) Race condition in AM after HBASE-7521 (only 0.94)
[ https://issues.apache.org/jira/browse/HBASE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan resolved HBASE-8040. --- Resolution: Fixed Hadoop Flags: Reviewed Committed to 0.94. Thanks for the review Ted, Lars and Sergey. > Race condition in AM after HBASE-7521 (only 0.94) > - > > Key: HBASE-8040 > URL: https://issues.apache.org/jira/browse/HBASE-8040 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 0.94.7 > > Attachments: HBASE-8040_1.patch, HBASE-8040.patch > > > This is a problem that introduced when we tried to solve HBASE-7521. > https://issues.apache.org/jira/browse/HBASE-7521?focusedCommentId=13576083&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13576083 > See the above comment and exactly the same has happened. Will come up with a > solution for the same. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601995#comment-13601995
]
Hadoop QA commented on HBASE-8097:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573646/hbase-8097_1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4808//console
This message is automatically generated.
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
> Attachments: 8097.txt, hbase-8097_1.patch
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
[
https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601992#comment-13601992
]
Hadoop QA commented on HBASE-7938:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573650/0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 11 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4807//console
This message is automatically generated.
> Add integration test for ImportTsv/LoadIncrementalHFiles workflow
> -
>
> Key: HBASE-7938
> URL: https://issues.apache.org/jira/browse/HBASE-7938
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.0, 0.98.0
>
> Attachments:
> 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch,
> 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch
>
>
> We have existing unit tests for smoke-testing the packaged MR jobs, however
> they do not create a runtime environment that is true to running on a real MR
> cluster. This is particularly true in regard to classpaths (HBASE-7934) but
> also other static state (HBASE-4802). An integration test that can be pointed
> to run on a pseudo-distributed Hadoop deployed on localhost would find these
> kinds of problems.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
[ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601991#comment-13601991 ] Enis Soztutar commented on HBASE-5487: -- Yes, please write is up. Although AM related things were not initially in the main focus. > Generic framework for Master-coordinated tasks > -- > > Key: HBASE-5487 > URL: https://issues.apache.org/jira/browse/HBASE-5487 > Project: HBase > Issue Type: New Feature > Components: master, regionserver, Zookeeper >Affects Versions: 0.94.0 >Reporter: Mubarak Seyed > > Need a framework to execute master-coordinated tasks in a fault-tolerant > manner. > Master-coordinated tasks such as online-scheme change and delete-range > (deleting region(s) based on start/end key) can make use of this framework. > The advantages of framework are > 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for > master-coordinated tasks > 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK > 3. Easy to plugin new master-coordinated tasks without adding code to core > components -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8099: --- Status: Patch Available (was: Open) > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.94.7 > > Attachments: HBase-8099-94.patch, HBase-8099-94-v2.patch, > HBase-8099-trunk-2.patch, HBase-8099-trunk.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8099: --- Attachment: HBase-8099-trunk-2.patch HBase-8099-94-v2.patch Taking in Ted's comments. > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.94.7 > > Attachments: HBase-8099-94.patch, HBase-8099-94-v2.patch, > HBase-8099-trunk-2.patch, HBase-8099-trunk.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8100) Wiki Thrift Documentation For Scan Methods
Jesse Anderson created HBASE-8100: - Summary: Wiki Thrift Documentation For Scan Methods Key: HBASE-8100 URL: https://issues.apache.org/jira/browse/HBASE-8100 Project: HBase Issue Type: Bug Components: documentation Reporter: Jesse Anderson The documentation on the Wiki for the Scan methods in Thrift are wrong. The wiki page is http://wiki.apache.org/hadoop/Hbase/ThriftApi. The openScanner, getScannerResult and closeScanner methods aren't methods in the Thrift interface. They should be scannerOpen, scannerGet, and scannerClose. Some of the method paramaters might need to be changed too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7122) Proper warning message when opening a log file with no entries (idle cluster)
[
https://issues.apache.org/jira/browse/HBASE-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601985#comment-13601985
]
Himanshu Vashishtha commented on HBASE-7122:
trunk patch is still good to apply.
> Proper warning message when opening a log file with no entries (idle cluster)
> -
>
> Key: HBASE-7122
> URL: https://issues.apache.org/jira/browse/HBASE-7122
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.2
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-7122-94.patch, HBase-7122.patch,
> HBASE-7122.v2.patch
>
>
> In case the cluster is idle and the log has rolled (offset to 0),
> replicationSource tries to open the log and gets an EOF exception. This gets
> printed after every 10 sec until an entry is inserted in it.
> {code}
> 2012-11-07 15:47:40,924 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:openReader(487)) - Opening log for replication
> c0315.hal.cloudera.com%2C40020%2C1352324202860.1352327804874 at 0
> 2012-11-07 15:47:40,926 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(543)) - 1 Got:
> java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at java.io.DataInputStream.readFully(DataInputStream.java:152)
> at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1486)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1475)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1470)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:55)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:175)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:716)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:491)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:290)
> 2012-11-07 15:47:40,927 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(547)) - Waited too long for this file,
> considering dumping
> 2012-11-07 15:47:40,927 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:sleepForRetries(562)) - Unable to open a reader,
> sleeping 1000 times 10
> {code}
> We should reduce the log spewing in this case (or some informative message,
> based on the offset).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6991) Escape "\" in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()
[
https://issues.apache.org/jira/browse/HBASE-6991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-6991:
-
Release Note:
This patch changes Bytes.toStringBinary() and Bytes.toBytesBinary() to escape
the character "\". Unprintable characters are escaped using "\\x%02X" format,
but "\" was not escaped properly leading to irreversible ser/de using
toStringBinary() -> toBytesBinary().
> Escape "\" in Bytes.toStringBinary() and its counterpart Bytes.toBytesBinary()
> --
>
> Key: HBASE-6991
> URL: https://issues.apache.org/jira/browse/HBASE-6991
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.96.0
>Reporter: Aditya Kishore
>Assignee: Aditya Kishore
> Fix For: 0.95.0
>
> Attachments: HBASE-6991_trunk.patch
>
>
> Since "\" is used to escape non-printable character but not treated as
> special character in conversion, it could lead to unexpected conversion.
> For example, please consider the following code snippet.
> {code}
> public void testConversion() {
> byte[] original = {
> '\\', 'x', 'A', 'D'
> };
> String stringFromBytes = Bytes.toStringBinary(original);
> byte[] converted = Bytes.toBytesBinary(stringFromBytes);
> System.out.println("Original: " + Arrays.toString(original));
> System.out.println("Converted: " + Arrays.toString(converted));
> System.out.println("Reversible?: " + (Bytes.compareTo(original, converted)
> == 0));
> }
> Output:
> ---
> Original: [92, 120, 65, 68]
> Converted: [-83]
> Reversible?: false
> {code}
> The "\" character needs to be treated as special and must be encoded as a
> non-printable character ("\x5C") to avoid any kind of ambiguity during
> conversion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-8085:
-
Resolution: Fixed
Release Note: This patch changes Bytes.toStringBinary() and
Bytes.toBytesBinary() to escape the character "\". Unprintable characters are
escaped using "\\x%02X" format, but "\" was not escaped properly leading to
irreversible ser/de using toStringBinary() -> toBytesBinary().
Hadoop Flags: Incompatible change,Reviewed
Status: Resolved (was: Patch Available)
Committed this. Thanks Tianying.
> Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
>
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch, HBASE-8085-v2.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7122) Proper warning message when opening a log file with no entries (idle cluster)
[
https://issues.apache.org/jira/browse/HBASE-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601980#comment-13601980
]
Hadoop QA commented on HBASE-7122:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573657/HBase-7122-94.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4809//console
This message is automatically generated.
> Proper warning message when opening a log file with no entries (idle cluster)
> -
>
> Key: HBASE-7122
> URL: https://issues.apache.org/jira/browse/HBASE-7122
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.2
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-7122-94.patch, HBase-7122.patch,
> HBASE-7122.v2.patch
>
>
> In case the cluster is idle and the log has rolled (offset to 0),
> replicationSource tries to open the log and gets an EOF exception. This gets
> printed after every 10 sec until an entry is inserted in it.
> {code}
> 2012-11-07 15:47:40,924 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:openReader(487)) - Opening log for replication
> c0315.hal.cloudera.com%2C40020%2C1352324202860.1352327804874 at 0
> 2012-11-07 15:47:40,926 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(543)) - 1 Got:
> java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at java.io.DataInputStream.readFully(DataInputStream.java:152)
> at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1486)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1475)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1470)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:55)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:175)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:716)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:491)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:290)
> 2012-11-07 15:47:40,927 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(547)) - Waited too long for this file,
> considering dumping
> 2012-11-07 15:47:40,927 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:sleepForRetries(562)) - Unable to open a reader,
> sleeping 1000 times 10
> {code}
> We should reduce the log spewing in this case (or some informative message,
> based on the offset).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601978#comment-13601978
]
Jeffrey Zhong commented on HBASE-8097:
--
[~ted_yu] You raised a good point. Since we just reput the dead server in
queue, should we updated the timestamp at all. The timestamp updated in
DeadServer.add seems bogus to me.
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
> Attachments: 8097.txt, hbase-8097_1.patch
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7122) Proper warning message when opening a log file with no entries (idle cluster)
[
https://issues.apache.org/jira/browse/HBASE-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Vashishtha updated HBASE-7122:
---
Issue Type: Bug (was: Sub-task)
Parent: (was: HBASE-6804)
> Proper warning message when opening a log file with no entries (idle cluster)
> -
>
> Key: HBASE-7122
> URL: https://issues.apache.org/jira/browse/HBASE-7122
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.2
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-7122-94.patch, HBase-7122.patch,
> HBASE-7122.v2.patch
>
>
> In case the cluster is idle and the log has rolled (offset to 0),
> replicationSource tries to open the log and gets an EOF exception. This gets
> printed after every 10 sec until an entry is inserted in it.
> {code}
> 2012-11-07 15:47:40,924 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:openReader(487)) - Opening log for replication
> c0315.hal.cloudera.com%2C40020%2C1352324202860.1352327804874 at 0
> 2012-11-07 15:47:40,926 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(543)) - 1 Got:
> java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at java.io.DataInputStream.readFully(DataInputStream.java:152)
> at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1486)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1475)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1470)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:55)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:175)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:716)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:491)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:290)
> 2012-11-07 15:47:40,927 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(547)) - Waited too long for this file,
> considering dumping
> 2012-11-07 15:47:40,927 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:sleepForRetries(562)) - Unable to open a reader,
> sleeping 1000 times 10
> {code}
> We should reduce the log spewing in this case (or some informative message,
> based on the offset).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-8085:
-
Summary: Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
(was: Backport the fix for Bytes.toStringBinary() into 94)
> Backport the fix for Bytes.toStringBinary() into 94 (HBASE-6991)
>
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch, HBASE-8085-v2.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[
https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601975#comment-13601975
]
Ted Yu commented on HBASE-8099:
---
Patch looks good.
{code}
- if (newQueues.size() == 0) {
+ if (newQueues == null || newQueues.size() == 0) {
{code}
nit: you can replace newQueues.size() == 0 with newQueues.isEmpty().
> ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues
> if it failed to execute.
> -
>
> Key: HBASE-8099
> URL: https://issues.apache.org/jira/browse/HBASE-8099
> Project: HBase
> Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Himanshu Vashishtha
>Priority: Blocker
> Fix For: 0.94.7
>
> Attachments: HBase-8099-94.patch, HBase-8099-trunk.patch
>
>
> We just ran into an interesting scenario. We restarted a cluster that was
> setup as a replication source.
> The stop went cleanly.
> Upon restart *all* regionservers aborted within a few seconds with variations
> of these errors:
> http://pastebin.com/3iQVuBqS
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7122) Proper warning message when opening a log file with no entries (idle cluster)
[
https://issues.apache.org/jira/browse/HBASE-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Himanshu Vashishtha updated HBASE-7122:
---
Attachment: HBase-7122-94.patch
0.94 patch.
> Proper warning message when opening a log file with no entries (idle cluster)
> -
>
> Key: HBASE-7122
> URL: https://issues.apache.org/jira/browse/HBASE-7122
> Project: HBase
> Issue Type: Sub-task
> Components: Replication
>Affects Versions: 0.94.2
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-7122-94.patch, HBase-7122.patch,
> HBASE-7122.v2.patch
>
>
> In case the cluster is idle and the log has rolled (offset to 0),
> replicationSource tries to open the log and gets an EOF exception. This gets
> printed after every 10 sec until an entry is inserted in it.
> {code}
> 2012-11-07 15:47:40,924 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:openReader(487)) - Opening log for replication
> c0315.hal.cloudera.com%2C40020%2C1352324202860.1352327804874 at 0
> 2012-11-07 15:47:40,926 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(543)) - 1 Got:
> java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at java.io.DataInputStream.readFully(DataInputStream.java:152)
> at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1486)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1475)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1470)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:55)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:175)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:716)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:491)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:290)
> 2012-11-07 15:47:40,927 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(547)) - Waited too long for this file,
> considering dumping
> 2012-11-07 15:47:40,927 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:sleepForRetries(562)) - Unable to open a reader,
> sleeping 1000 times 10
> {code}
> We should reduce the log spewing in this case (or some informative message,
> based on the offset).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601970#comment-13601970
]
Ted Yu commented on HBASE-8097:
---
I am not sure whether the add() call should be dropped.
Take a look at this method in DeadServer:
{code}
public synchronized List> copyDeadServersSince(long
ts){
List> res = new ArrayList>(size());
for (Map.Entry entry:deadServers.entrySet()){
if (entry.getValue() >= ts){
{code}
Without calling add(), we don't have up-to-date timestamp.
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
> Attachments: 8097.txt, hbase-8097_1.patch
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7122) Proper warning message when opening a log file with no entries (idle cluster)
[
https://issues.apache.org/jira/browse/HBASE-7122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601967#comment-13601967
]
Himanshu Vashishtha commented on HBASE-7122:
will upload a revised patch for 0.94 soon.
> Proper warning message when opening a log file with no entries (idle cluster)
> -
>
> Key: HBASE-7122
> URL: https://issues.apache.org/jira/browse/HBASE-7122
> Project: HBase
> Issue Type: Sub-task
> Components: Replication
>Affects Versions: 0.94.2
>Reporter: Himanshu Vashishtha
>Assignee: Himanshu Vashishtha
> Fix For: 0.95.0
>
> Attachments: HBase-7122.patch, HBASE-7122.v2.patch
>
>
> In case the cluster is idle and the log has rolled (offset to 0),
> replicationSource tries to open the log and gets an EOF exception. This gets
> printed after every 10 sec until an entry is inserted in it.
> {code}
> 2012-11-07 15:47:40,924 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:openReader(487)) - Opening log for replication
> c0315.hal.cloudera.com%2C40020%2C1352324202860.1352327804874 at 0
> 2012-11-07 15:47:40,926 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(543)) - 1 Got:
> java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at java.io.DataInputStream.readFully(DataInputStream.java:152)
> at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1508)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1486)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1475)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1470)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:55)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:175)
> at
> org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:716)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:491)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:290)
> 2012-11-07 15:47:40,927 WARN regionserver.ReplicationSource
> (ReplicationSource.java:openReader(547)) - Waited too long for this file,
> considering dumping
> 2012-11-07 15:47:40,927 DEBUG regionserver.ReplicationSource
> (ReplicationSource.java:sleepForRetries(562)) - Unable to open a reader,
> sleeping 1000 times 10
> {code}
> We should reduce the log spewing in this case (or some informative message,
> based on the offset).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8099: --- Attachment: HBase-8099-94.patch patch for 0.94 > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.94.7 > > Attachments: HBase-8099-94.patch, HBase-8099-trunk.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chunhui shen updated HBASE-8097:
Status: Patch Available (was: Open)
Seems good, run HadoopQA first
{code}
/**
* Shutdown handler for the server hosting -ROOT-,
* .META., or both.
*/
{code}
Also could remove ROOT from the annotation
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
> Attachments: 8097.txt, hbase-8097_1.patch
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8099: --- Attachment: HBase-8099-trunk.patch patch for trunk. > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.94.7 > > Attachments: HBase-8099-trunk.patch > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601956#comment-13601956 ] Lars Hofhansl commented on HBASE-8088: -- Or we can revert and try again. > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
[ https://issues.apache.org/jira/browse/HBASE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601955#comment-13601955 ] Lars Hofhansl commented on HBASE-8092: -- I'd have to digest this a bit more. > bulk assignment in 0.94 doesn't handle ZK errors very well > -- > > Key: HBASE-8092 > URL: https://issues.apache.org/jira/browse/HBASE-8092 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.7 > > Attachments: HBASE-8092-v0.patch > > > It can abort the master if node already exists, and as far as I see, if > exists check fails it will get stuck forever (the latter I haven't seen > though). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601954#comment-13601954
]
Lars Hofhansl commented on HBASE-8085:
--
I'd agree. +1
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch, HBASE-8085-v2.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
[ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7938: Attachment: 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch New patch makes use of the subdirectory features of the test util's test directory method. Running via mvn and IntegrationTestsDriver locally, will report back. [~enis] care to give this patch a go as well? > Add integration test for ImportTsv/LoadIncrementalHFiles workflow > - > > Key: HBASE-7938 > URL: https://issues.apache.org/jira/browse/HBASE-7938 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0, 0.98.0 > > Attachments: > 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch, > 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch > > > We have existing unit tests for smoke-testing the packaged MR jobs, however > they do not create a runtime environment that is true to running on a real MR > cluster. This is particularly true in regard to classpaths (HBASE-7934) but > also other static state (HBASE-4802). An integration test that can be pointed > to run on a pseudo-distributed Hadoop deployed on localhost would find these > kinds of problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
[ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7938: Hadoop Flags: Reviewed Status: Patch Available (was: Open) > Add integration test for ImportTsv/LoadIncrementalHFiles workflow > - > > Key: HBASE-7938 > URL: https://issues.apache.org/jira/browse/HBASE-7938 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0, 0.98.0 > > Attachments: > 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch, > 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch > > > We have existing unit tests for smoke-testing the packaged MR jobs, however > they do not create a runtime environment that is true to running on a real MR > cluster. This is particularly true in regard to classpaths (HBASE-7934) but > also other static state (HBASE-4802). An integration test that can be pointed > to run on a pseudo-distributed Hadoop deployed on localhost would find these > kinds of problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
[ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-7938: Status: Open (was: Patch Available) > Add integration test for ImportTsv/LoadIncrementalHFiles workflow > - > > Key: HBASE-7938 > URL: https://issues.apache.org/jira/browse/HBASE-7938 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0, 0.98.0 > > Attachments: > 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch > > > We have existing unit tests for smoke-testing the packaged MR jobs, however > they do not create a runtime environment that is true to running on a real MR > cluster. This is particularly true in regard to classpaths (HBASE-7934) but > also other static state (HBASE-4802). An integration test that can be pointed > to run on a pseudo-distributed Hadoop deployed on localhost would find these > kinds of problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5487) Generic framework for Master-coordinated tasks
[ https://issues.apache.org/jira/browse/HBASE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601950#comment-13601950 ] Sergey Shelukhin commented on HBASE-5487: - Just checking; is this issue still relevant? I am particularly interested in all-encompassing solution for region management that would allow you to read AM and not go insane (I was just reading/debugging 0.94 AM for some time, that's why I remembered). I have a sketch of an idea, should I write it up? > Generic framework for Master-coordinated tasks > -- > > Key: HBASE-5487 > URL: https://issues.apache.org/jira/browse/HBASE-5487 > Project: HBase > Issue Type: New Feature > Components: master, regionserver, Zookeeper >Affects Versions: 0.94.0 >Reporter: Mubarak Seyed > > Need a framework to execute master-coordinated tasks in a fault-tolerant > manner. > Master-coordinated tasks such as online-scheme change and delete-range > (deleting region(s) based on start/end key) can make use of this framework. > The advantages of framework are > 1. Eliminate repeated code in Master, ZooKeeper tracker and Region-server for > master-coordinated tasks > 2. Ability to abstract the common functions across Master -> ZK and RS -> ZK > 3. Easy to plugin new master-coordinated tasks without adding code to core > components -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8031) Adopt goraci as an Integration test
[
https://issues.apache.org/jira/browse/HBASE-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601948#comment-13601948
]
Hudson commented on HBASE-8031:
---
Integrated in HBase-0.94 #897 (See
[https://builds.apache.org/job/HBase-0.94/897/])
HBASE-8031 Adopt goraci as an Integration test (Revision 1456284)
Result = FAILURE
enis :
Files :
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java
> Adopt goraci as an Integration test
> ---
>
> Key: HBASE-8031
> URL: https://issues.apache.org/jira/browse/HBASE-8031
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.0, 0.98.0, 0.94.7
>
> Attachments: hbase-8031_v1-0.94.patch, hbase-8031_v1.patch
>
>
> As you might know, I am a big fan of the goraci test that Keith Turner has
> developed, which in turn is inspired by the Accumulo test called Continuous
> Ingest.
> As much as I hate to say it, having to rely on gora and and external github
> library makes using this lib cumbersome. And lately we had to use this for
> testing against secure clusters and with Hadoop2, which gora does not support
> for now.
> So, I am proposing we add this test as an IT in the HBase code base so that
> all HBase devs can benefit from it.
> The original source code can be found here:
> * https://github.com/keith-turner/goraci
> * https://github.com/enis/goraci/
> From the javadoc:
> {code}
> Apache Accumulo [0] has a simple test suite that verifies that data is not
> * lost at scale. This test suite is called continuous ingest. This test runs
> * many ingest clients that continually create linked lists containing 25
> * million nodes. At some point the clients are stopped and a map reduce job
> is
> * run to ensure no linked list has a hole. A hole indicates data was lost.··
> *
> * The nodes in the linked list are random. This causes each linked list to
> * spread across the table. Therefore if one part of a table loses data, then
> it
> * will be detected by references in another part of the table.
> *
> Below is rough sketch of how data is written. For specific details look at
> * the Generator code.
> *
> * 1 Write out 1 million nodes· 2 Flush the client· 3 Write out 1 million that
> * reference previous million· 4 If this is the 25th set of 1 million nodes,
> * then update 1st set of million to point to last· 5 goto 1
> *
> * The key is that nodes only reference flushed nodes. Therefore a node should
> * never reference a missing node, even if the ingest client is killed at any
> * point in time.
> *
> * Some ASCII art time:
> * [ . . . ] represents one batch of random longs of length WIDTH
> *
> *_
> * | __ |
> * | | ||
> * __+_+_ ||
> * v v v |||
> * first = [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * prev= [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * current = [ . . . . . . . . . . . ] |||
> * |||
> * ... |||
> * |||
> * last= [ . . . . . . . . . . . ] |||
> * | | | | | | | | | | |-|||
> * | |||
> * |___|
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7938) Add integration test for ImportTsv/LoadIncrementalHFiles workflow
[ https://issues.apache.org/jira/browse/HBASE-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601945#comment-13601945 ] Nick Dimiduk commented on HBASE-7938: - bq. No need to do fancy stuff. We can just call the tests one-by-one within run(). Are you sure? That sounds brittle? IntegrationTestsDriver invokes JUnit manually; I'd prefer to leave this up to the driver. bq. This looks like the IntegrationTestingUtility didn't clean up the test directory after a run. I assumed way too much. HBaseCluster#restoreClusterStatus() does nothing, thus IntegrationTestingUtility#restoreCluster() does nothing. Does this mean paths returned by HBaseTestingUtility#getDataTestDirOnTestFS() are not cleaned after a test? Sad panda. > Add integration test for ImportTsv/LoadIncrementalHFiles workflow > - > > Key: HBASE-7938 > URL: https://issues.apache.org/jira/browse/HBASE-7938 > Project: HBase > Issue Type: Sub-task > Components: mapreduce >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.0, 0.98.0 > > Attachments: > 0001-HBASE-7938-Add-integration-test-for-ImportTsv-LoadIn.patch > > > We have existing unit tests for smoke-testing the packaged MR jobs, however > they do not create a runtime environment that is true to running on a real MR > cluster. This is particularly true in regard to classpaths (HBASE-7934) but > also other static state (HBASE-4802). An integration test that can be pointed > to run on a pseudo-distributed Hadoop deployed on localhost would find these > kinds of problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7403) Online Merge
[ https://issues.apache.org/jira/browse/HBASE-7403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601944#comment-13601944 ] chunhui shen commented on HBASE-7403: - [~enis] Could you take a review ? > Online Merge > > > Key: HBASE-7403 > URL: https://issues.apache.org/jira/browse/HBASE-7403 > Project: HBase > Issue Type: New Feature >Affects Versions: 0.95.0 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.95.0, 0.98.0 > > Attachments: 7403-trunkv5.patch, 7403-trunkv6.patch, 7403v5.diff, > 7403-v5.txt, 7403v5.txt, hbase-7403-94v1.patch, hbase-7403-trunkv10.patch, > hbase-7403-trunkv11.patch, hbase-7403-trunkv12.patch, > hbase-7403-trunkv13.patch, hbase-7403-trunkv14.patch, > hbase-7403-trunkv15.patch, hbase-7403-trunkv16.patch, > hbase-7403-trunkv19.patch, hbase-7403-trunkv1.patch, > hbase-7403-trunkv20.patch, hbase-7403-trunkv22.patch, > hbase-7403-trunkv5.patch, hbase-7403-trunkv6.patch, hbase-7403-trunkv7.patch, > hbase-7403-trunkv8.patch, hbase-7403-trunkv9.patch, merge region.pdf > > > Support executing region merge transaction on Regionserver, similar with > split transaction > Process of merging two regions: > a.client send RPC(dispacth merging regions) to master > b.master move the regions together (on the same regionserver) > c.master send RPC(merge regions) to regionserver > d.Regionserver execute the regions merge transaction in the thread pool > e.the above b,c,d run asynchronously > Process of region merge transaction: > a.Construct a new region merge transaction. > b.prepare for the merge transaction, the transaction will be canceled if it > is unavailable, > e.g. two regions don't belong to same table; two regions are not adjacent in > a non-compulsory merge; region is closed or has reference > c.execute the transaction as the following: > /** > * Set region as in transition, set it into MERGING state. > */ > SET_MERGING_IN_ZK, > /** > * We created the temporary merge data directory. > */ > CREATED_MERGE_DIR, > /** > * Closed the merging region A. > */ > CLOSED_REGION_A, > /** > * The merging region A has been taken out of the server's online regions > list. > */ > OFFLINED_REGION_A, > /** > * Closed the merging region B. > */ > CLOSED_REGION_B, > /** > * The merging region B has been taken out of the server's online regions > list. > */ > OFFLINED_REGION_B, > /** > * Started in on creation of the merged region. > */ > STARTED_MERGED_REGION_CREATION, > /** > * Point of no return. If we got here, then transaction is not recoverable > * other than by crashing out the regionserver. > */ > PONR > d.roll back if step c throws exception > Usage: > HBaseAdmin#mergeRegions > See more details from the patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601943#comment-13601943 ] Himanshu Vashishtha commented on HBASE-8099: yes, looking at it. > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.7 > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha reassigned HBASE-8099: -- Assignee: Himanshu Vashishtha > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Assignee: Himanshu Vashishtha >Priority: Blocker > Fix For: 0.94.7 > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7905) Add passing of optional cell blocks over rpc
[
https://issues.apache.org/jira/browse/HBASE-7905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601940#comment-13601940
]
Hadoop QA commented on HBASE-7905:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573631/7905v9.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 48 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 5
warning messages.
{color:red}-1 javac{color}. The applied patch generated 5 javac compiler
warnings (more than the trunk's current 4 warnings).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift.TestThriftServer
org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary
org.apache.hadoop.hbase.client.TestHCM
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
org.apache.hadoop.hbase.util.TestHBaseFsck
org.apache.hadoop.hbase.replication.TestReplicationDisableInactivePeer
org.apache.hadoop.hbase.replication.TestReplicationQueueFailover
org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed
org.apache.hadoop.hbase.rest.TestScannersWithFilters
org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.TestAcidGuarantees
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
org.apache.hadoop.hbase.backup.TestHFileArchiving
org.apache.hadoop.hbase.security.access.TestAccessControlFilter
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
org.apache.hadoop.hbase.TestDrainingServer
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
org.apache.hadoop.hbase.client.TestScannerTimeout
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.client.TestFromClientSide.testUnmanagedHConnectionReconnect(TestFromClientSide.java:3986)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4806//console
This message is automatically generated.
> Add passing of optional cell blocks over rpc
>
>
> Key: HBASE-7905
> URL: https://issues.apache.org/jira/browse/HBASE-7905
[jira] [Commented] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
[ https://issues.apache.org/jira/browse/HBASE-8099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601938#comment-13601938 ] Lars Hofhansl commented on HBASE-8099: -- [~v.himanshu] Wanna have a look? > ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues > if it failed to execute. > - > > Key: HBASE-8099 > URL: https://issues.apache.org/jira/browse/HBASE-8099 > Project: HBase > Issue Type: Bug >Reporter: Lars Hofhansl >Priority: Blocker > Fix For: 0.94.7 > > > We just ran into an interesting scenario. We restarted a cluster that was > setup as a replication source. > The stop went cleanly. > Upon restart *all* regionservers aborted within a few seconds with variations > of these errors: > http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-8099) ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute.
Lars Hofhansl created HBASE-8099: Summary: ReplicationZookeeper.copyQueuesFromRSUsingMulti should not return any queues if it failed to execute. Key: HBASE-8099 URL: https://issues.apache.org/jira/browse/HBASE-8099 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Priority: Blocker Fix For: 0.94.7 We just ran into an interesting scenario. We restarted a cluster that was setup as a replication source. The stop went cleanly. Upon restart *all* regionservers aborted within a few seconds with variations of these errors: http://pastebin.com/3iQVuBqS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Zhong updated HBASE-8097:
-
Attachment: hbase-8097_1.patch
1. Remove this.deadServers.add(serverName);
2. Clean MetaServerShutdownHandler as we don't have _ROOT_ any more
Thanks,
-Jeffrey
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
> Attachments: 8097.txt, hbase-8097_1.patch
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601933#comment-13601933
]
Jeffrey Zhong commented on HBASE-8097:
--
That will work but we don't need call this.deadServers.add(serverName); at all.
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
> Attachments: 8097.txt, hbase-8097_1.patch
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
[ https://issues.apache.org/jira/browse/HBASE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601934#comment-13601934 ] Sergey Shelukhin commented on HBASE-8092: - On the other hand, these nodes will be there either way, if master restarts like now, so shouldn't matter. > bulk assignment in 0.94 doesn't handle ZK errors very well > -- > > Key: HBASE-8092 > URL: https://issues.apache.org/jira/browse/HBASE-8092 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.7 > > Attachments: HBASE-8092-v0.patch > > > It can abort the master if node already exists, and as far as I see, if > exists check fails it will get stuck forever (the latter I haven't seen > though). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
[ https://issues.apache.org/jira/browse/HBASE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8092: Status: Patch Available (was: Open) > bulk assignment in 0.94 doesn't handle ZK errors very well > -- > > Key: HBASE-8092 > URL: https://issues.apache.org/jira/browse/HBASE-8092 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.7 > > Attachments: HBASE-8092-v0.patch > > > It can abort the master if node already exists, and as far as I see, if > exists check fails it will get stuck forever (the latter I haven't seen > though). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8097:
--
Attachment: 8097.txt
How about this patch ?
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
> Attachments: 8097.txt
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8088) Versioning site: part one, put stake in the ground for 0.94 by copying current versions of book and site
[ https://issues.apache.org/jira/browse/HBASE-8088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601927#comment-13601927 ] Lars Hofhansl commented on HBASE-8088: -- You gonna fix it [~stack]? :) 0.94 build is hosed right now. > Versioning site: part one, put stake in the ground for 0.94 by copying > current versions of book and site > > > Key: HBASE-8088 > URL: https://issues.apache.org/jira/browse/HBASE-8088 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack > Fix For: 0.94.7 > > Attachments: addendum.txt > > > Doug Meil suggests its time we started versioning doc and site. First step > is copying current version over to 0.94 so we have something to work against > (site and book have only been kept up on trunk up to this). Let me do this > much for now. Can figure how to do the rest later. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-8074) Consolidate map-side features across mapreduce tools into a single place
[
https://issues.apache.org/jira/browse/HBASE-8074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reassigned HBASE-8074:
---
Assignee: Nick Dimiduk
> Consolidate map-side features across mapreduce tools into a single place
>
>
> Key: HBASE-8074
> URL: https://issues.apache.org/jira/browse/HBASE-8074
> Project: HBase
> Issue Type: Sub-task
> Components: mapreduce, Usability
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>
> The mapreduce tools support a similar but divergent set of features for
> mapping over KeyValue data:
> * {{Export}} supports specifying a version-range window, application of a
> rowkey regex or prefix filter, and a "raw mode" that includes delete markers.
> * {{Import}} can apply an arbitrary filter and can also apply a "transform",
> renaming column families in the emitted KeyValues.
> * {{CopyTable}} allows specifying a version-range window, limiting to a
> fixed number of versions, a "raw mode", and column family transformation.
> * {{WALPlayer}} supports reading a time-range.
> * {{ImportTsv}} could incorporate a number of these features, especially the
> filter and transform capabilities, allowing a user to avoid implementing a
> custom mapper where the existing parser is sufficient, but for a slight
> massage of the data.
> The proposal is to create a single implementation for these features with a
> single configuration interface. Ideally, such an implementation would be
> exposed via the common utility classes as well (ie, IdentityTableMapper).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8001) Avoid unnecessary lazy seek
[ https://issues.apache.org/jira/browse/HBASE-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601923#comment-13601923 ] Raymond Liu commented on HBASE-8001: @Ted: That's due to HBASE-8012 is not landed, now since 8012 landed, this one should pass the test > Avoid unnecessary lazy seek > --- > > Key: HBASE-8001 > URL: https://issues.apache.org/jira/browse/HBASE-8001 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Raymond Liu >Assignee: Raymond Liu > Fix For: 0.98.0 > > Attachments: HBASE-8001_onescanner.patch, > HBASE-8001_onescanner_v2.patch > > > Lazy seek helps to reduce the real seek needed for multi hfile, when the kv > from newer hfile is enough to satisfy the query. > While in many case, it just push the real seek later, and do not reduce the > number of real seek. e.g. there are only one hfile, or storefilescanner is > closed and only one left, or the scan need to go through all the versions, or > there are only one version of row and a sequence scan is performed. In these > case, lazy seek just bring extra overhead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8080) refactor default compactor to make its parts easier to reuse
[
https://issues.apache.org/jira/browse/HBASE-8080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601918#comment-13601918
]
Hadoop QA commented on HBASE-8080:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573622/HBASE-8080-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testFixByTable(TestHBaseFsck.java:1170)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4805//console
This message is automatically generated.
> refactor default compactor to make its parts easier to reuse
>
>
> Key: HBASE-8080
> URL: https://issues.apache.org/jira/browse/HBASE-8080
> Project: HBase
> Issue Type: Improvement
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HBASE-8080-v0.patch, HBASE-8080-v1.patch,
> HBASE-8080-v2.patch
>
>
> Refactor default compactor to make its parts easier to reuse. To make
> eventual HBASE-7967 patch smaller.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8036) ProtobufUtil.multi behavior is inconsistent in case of errors
[ https://issues.apache.org/jira/browse/HBASE-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601911#comment-13601911 ] Hudson commented on HBASE-8036: --- Integrated in hbase-0.95-on-hadoop2 #25 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/25/]) HBASE-8036 ProtobufUtil.multi behavior is inconsistent in case of errors (Revision 1456064) Result = FAILURE enis : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/protobuf/ProtobufUtil.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestMultiParallel.java > ProtobufUtil.multi behavior is inconsistent in case of errors > - > > Key: HBASE-8036 > URL: https://issues.apache.org/jira/browse/HBASE-8036 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.0 >Reporter: Sergey Shelukhin >Assignee: Enis Soztutar >Priority: Blocker > Fix For: 0.95.0, 0.98.0 > > Attachments: hbase-8036_v1.patch, hbase-8036_v2.patch, > hbase-8036_v3.patch, hbase-8036_v4.patch > > > ProtobufUtil splits operations by regions and performs multiple client.multi > calls. In case if there are certain errors inside RS, HRegionServer adds the > corresponding exceptions to MultiResponse, PU continues the multi request for > other regions, and returns partial failure. > In case of other errors (for example, region not served exception), the > entire multi operation stops executing, and previous successes and partial > results are disregarded. > ProtobufUtil should probably catch ServiceException separately for each > client.multi call, make it a partial-failure exception for all actions for > this region, and also continue the batch, to make the behavior consistent. > Alternatively, if we want to avoid continuing the batch in case of some > server-wide errors/connection problems/etc., server should do that for > region-specific errors (add exception to results for each action). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8031) Adopt goraci as an Integration test
[
https://issues.apache.org/jira/browse/HBASE-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601910#comment-13601910
]
Hudson commented on HBASE-8031:
---
Integrated in hbase-0.95-on-hadoop2 #25 (See
[https://builds.apache.org/job/hbase-0.95-on-hadoop2/25/])
HBASE-8031 Adopt goraci as an Integration test (Revision 1456079)
Result = FAILURE
enis :
Files :
*
/hbase/branches/0.95/hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java
> Adopt goraci as an Integration test
> ---
>
> Key: HBASE-8031
> URL: https://issues.apache.org/jira/browse/HBASE-8031
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.95.0, 0.98.0, 0.94.7
>
> Attachments: hbase-8031_v1-0.94.patch, hbase-8031_v1.patch
>
>
> As you might know, I am a big fan of the goraci test that Keith Turner has
> developed, which in turn is inspired by the Accumulo test called Continuous
> Ingest.
> As much as I hate to say it, having to rely on gora and and external github
> library makes using this lib cumbersome. And lately we had to use this for
> testing against secure clusters and with Hadoop2, which gora does not support
> for now.
> So, I am proposing we add this test as an IT in the HBase code base so that
> all HBase devs can benefit from it.
> The original source code can be found here:
> * https://github.com/keith-turner/goraci
> * https://github.com/enis/goraci/
> From the javadoc:
> {code}
> Apache Accumulo [0] has a simple test suite that verifies that data is not
> * lost at scale. This test suite is called continuous ingest. This test runs
> * many ingest clients that continually create linked lists containing 25
> * million nodes. At some point the clients are stopped and a map reduce job
> is
> * run to ensure no linked list has a hole. A hole indicates data was lost.··
> *
> * The nodes in the linked list are random. This causes each linked list to
> * spread across the table. Therefore if one part of a table loses data, then
> it
> * will be detected by references in another part of the table.
> *
> Below is rough sketch of how data is written. For specific details look at
> * the Generator code.
> *
> * 1 Write out 1 million nodes· 2 Flush the client· 3 Write out 1 million that
> * reference previous million· 4 If this is the 25th set of 1 million nodes,
> * then update 1st set of million to point to last· 5 goto 1
> *
> * The key is that nodes only reference flushed nodes. Therefore a node should
> * never reference a missing node, even if the ingest client is killed at any
> * point in time.
> *
> * Some ASCII art time:
> * [ . . . ] represents one batch of random longs of length WIDTH
> *
> *_
> * | __ |
> * | | ||
> * __+_+_ ||
> * v v v |||
> * first = [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * prev= [ . . . . . . . . . . . ] |||
> * ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ |||
> * | | | | | | | | | | | |||
> * current = [ . . . . . . . . . . . ] |||
> * |||
> * ... |||
> * |||
> * last= [ . . . . . . . . . . . ] |||
> * | | | | | | | | | | |-|||
> * | |||
> * |___|
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8012) Reseek should position to the beginning of file for the first time it is invoked with a KV smaller than the first KV in file
[ https://issues.apache.org/jira/browse/HBASE-8012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601907#comment-13601907 ] Hudson commented on HBASE-8012: --- Integrated in hbase-0.95-on-hadoop2 #25 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/25/]) HBASE-8012 - Reseek should position to the beginning of file for the first time it is invoked with a KV smaller than the first KV in file (Raymond Liu) (Revision 1455990) Result = FAILURE ramkrishna : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java > Reseek should position to the beginning of file for the first time it is > invoked with a KV smaller than the first KV in file > > > Key: HBASE-8012 > URL: https://issues.apache.org/jira/browse/HBASE-8012 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.94.5 >Reporter: Raymond Liu >Assignee: Raymond Liu >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8012.patch, HBASE-8012.patch, HBASE-8012_v2.patch > > > The storeFileScanner's seekAtOrAfter method will position at the beginning of > the file when the passed KV is smaller than first KV in file. While for > reseekAtOrAfter, I think it should also do the same thing when it is the > first time it been seeked. originally, this is workaround by adding a > isReseekable property in StoreFileScanner, and is checked upon each > enforceSeek(), if it is not seeked before, it will go with seek approaching > instead of reseek approaching. While why not make reseekAtOrAfter working > correctly for the first time it been reseek (also never been seek before), > since the file is never seeked before, so position it at the beginning of the > file don't break the idea of "reseek", say never rewind. > It will save the effort for HBASE-8001, with this fixed, it won't need to > check isReseekable any more. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8056) allow StoreScanner to drop deletes from some part of the compaction range
[ https://issues.apache.org/jira/browse/HBASE-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601908#comment-13601908 ] Hudson commented on HBASE-8056: --- Integrated in hbase-0.95-on-hadoop2 #25 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/25/]) HBASE-8056 allow StoreScanner to drop deletes from some part of the compaction range (Revision 1456108) Result = FAILURE sershe : Files : * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java > allow StoreScanner to drop deletes from some part of the compaction range > - > > Key: HBASE-8056 > URL: https://issues.apache.org/jira/browse/HBASE-8056 > Project: HBase > Issue Type: Task > Components: Compaction, Scanners >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HBASE-8056-v0.patch, HBASE-8056-v1.patch, > HBASE-8056-v1.patch > > > Allow StoreScanner to drop deletes from some part of the compaction range. > Needed for stripe compactor, and maybe level compactor (although at present I > am not sure how level compactor will drop deletes at all). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8066) Provide Admin.isTableAvailable() for a given table along with splitkeys
[ https://issues.apache.org/jira/browse/HBASE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601909#comment-13601909 ] Hudson commented on HBASE-8066: --- Integrated in hbase-0.95-on-hadoop2 #25 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/25/]) HBASE-8066 - Provide Admin.isTableAvailable() for a given table along with splitkeys (Ram) (Revision 1455962) Result = FAILURE ramkrishna : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnection.java * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java * /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java > Provide Admin.isTableAvailable() for a given table along with splitkeys > --- > > Key: HBASE-8066 > URL: https://issues.apache.org/jira/browse/HBASE-8066 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 0.95.0 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Minor > Fix For: 0.95.0, 0.98.0 > > Attachments: HBASE-8066_1.patch, HBASE-8066.patch > > > As part of HBASE-5583 if the master reboots during creation of table there is > a chance that the table gets created with partial split keys. This api helps > to check if the table was created with the required number of splitkeys. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7680) implement compaction policy for stripe compactions
[
https://issues.apache.org/jira/browse/HBASE-7680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601891#comment-13601891
]
Hadoop QA commented on HBASE-7680:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573617/HBASE-7680-v2-with-7679-and-8034.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4804//console
This message is automatically generated.
> implement compaction policy for stripe compactions
> --
>
> Key: HBASE-7680
> URL: https://issues.apache.org/jira/browse/HBASE-7680
> Project: HBase
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HBASE-7680-v0.patch,
> HBASE-7680-v0-with-7679-and-7935.patch, HBASE-7680-v1.patch,
> HBASE-7680-v1-with-7679.patch, HBASE-7680-v2.patch,
> HBASE-7680-v2-with-7679-and-8034.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
[ https://issues.apache.org/jira/browse/HBASE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8092: Status: Open (was: Patch Available) > bulk assignment in 0.94 doesn't handle ZK errors very well > -- > > Key: HBASE-8092 > URL: https://issues.apache.org/jira/browse/HBASE-8092 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.7 > > Attachments: HBASE-8092-v0.patch > > > It can abort the master if node already exists, and as far as I see, if > exists check fails it will get stuck forever (the latter I haven't seen > though). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8092) bulk assignment in 0.94 doesn't handle ZK errors very well
[ https://issues.apache.org/jira/browse/HBASE-8092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601889#comment-13601889 ] Sergey Shelukhin commented on HBASE-8092: - Hmm, actually, this may result in stale nodes when one of the creates fails. Maybe sometimes it's good for master to die. Let me double check... > bulk assignment in 0.94 doesn't handle ZK errors very well > -- > > Key: HBASE-8092 > URL: https://issues.apache.org/jira/browse/HBASE-8092 > Project: HBase > Issue Type: Bug >Affects Versions: 0.94.6 >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 0.94.7 > > Attachments: HBASE-8092-v0.patch > > > It can abort the master if node already exists, and as far as I see, if > exists check fails it will get stuck forever (the latter I haven't seen > though). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8085) Backport the fix for Bytes.toStringBinary() into 94
[
https://issues.apache.org/jira/browse/HBASE-8085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601880#comment-13601880
]
Enis Soztutar commented on HBASE-8085:
--
This is technically backwards incompatible, but we should fix this anyway. +1
from me, but will seek [~lhofhansl]'s input.
> Backport the fix for Bytes.toStringBinary() into 94
> ---
>
> Key: HBASE-8085
> URL: https://issues.apache.org/jira/browse/HBASE-8085
> Project: HBase
> Issue Type: Bug
> Components: util
>Affects Versions: 0.94.5
>Reporter: Tianying Chang
>Assignee: Tianying Chang
> Fix For: 0.94.7
>
> Attachments: HBASE-8085.patch, HBASE-8085-v2.patch
>
>
> there is a bug in Bytes.toStringBinary(), which will return the same string
> for 1) byte[] a = {'\\', 'x', 'D', 'A'} 2) "\xDA".
> It seems this bug has already been fixed in trunk with HBASE 6991. We should
> backport it to 94.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601870#comment-13601870
]
Ted Yu edited comment on HBASE-8097 at 3/14/13 12:17 AM:
-
super.process() would be skipped in the above case.
In ServerShutdownHandler.process(), the code is structured this way:
{code}
try {
try {
if (this.shouldSplitHlog) {
LOG.info("Splitting logs for " + serverName);
this.services.getMasterFileSystem().splitLog(serverName);
} else {
LOG.info("Skipping log splitting for " + serverName);
}
} catch (IOException ioe) {
//typecast to SSH so that we make sure that it is the SSH instance that
//gets submitted as opposed to MSSH or some other derived instance of
SSH
this.services.getExecutorService().submit((ServerShutdownHandler)this);
this.deadServers.add(serverName);
throw new IOException("failed log splitting for " +
...
} finally {
this.deadServers.finish(serverName);
}
{code}
Can we use similar structure in MetaServerShutdownHandler ?
was (Author: yuzhih...@gmail.com):
super.process() would be skipped in the above case.
In ServerShutdownHandler.process(), the code is structured this way:
{code}
try {
try {
if (this.shouldSplitHlog) {
LOG.info("Splitting logs for " + serverName);
this.services.getMasterFileSystem().splitLog(serverName);
} else {
LOG.info("Skipping log splitting for " + serverName);
}
} catch (IOException ioe) {
//typecast to SSH so that we make sure that it is the SSH instance that
//gets submitted as opposed to MSSH or some other derived instance of
SSH
this.services.getExecutorService().submit((ServerShutdownHandler)this);
this.deadServers.add(serverName);
throw new IOException("failed log splitting for " +
...
} finally {
this.deadServers.finish(serverName);
}
{code}
Can you use similar structure in MetaServerShutdownHandler ?
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601875#comment-13601875
]
Jeffrey Zhong commented on HBASE-8097:
--
I thought about that in that case I could call
this.deadServers.finish(serverName); twice.
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8097) MetaServerShutdownHandler may potentially keep bumping up DeadServer.numProcessing
[
https://issues.apache.org/jira/browse/HBASE-8097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13601870#comment-13601870
]
Ted Yu commented on HBASE-8097:
---
super.process() would be skipped in the above case.
In ServerShutdownHandler.process(), the code is structured this way:
{code}
try {
try {
if (this.shouldSplitHlog) {
LOG.info("Splitting logs for " + serverName);
this.services.getMasterFileSystem().splitLog(serverName);
} else {
LOG.info("Skipping log splitting for " + serverName);
}
} catch (IOException ioe) {
//typecast to SSH so that we make sure that it is the SSH instance that
//gets submitted as opposed to MSSH or some other derived instance of
SSH
this.services.getExecutorService().submit((ServerShutdownHandler)this);
this.deadServers.add(serverName);
throw new IOException("failed log splitting for " +
...
} finally {
this.deadServers.finish(serverName);
}
{code}
Can you use similar structure in MetaServerShutdownHandler ?
> MetaServerShutdownHandler may potentially keep bumping up
> DeadServer.numProcessing
> --
>
> Key: HBASE-8097
> URL: https://issues.apache.org/jira/browse/HBASE-8097
> Project: HBase
> Issue Type: Bug
>Reporter: Jeffrey Zhong
>Assignee: Jeffrey Zhong
> Fix For: 0.96.0
>
>
> {code}
> } catch (IOException ioe) {
> this.services.getExecutorService().submit(this);
> this.deadServers.add(serverName);
> throw new IOException("failed log splitting for " +
> serverName + ", will retry", ioe);
> }
> {code}
> this.deadServers.add(serverName); will keep incrementing
> DeadServer.numProcessing
> We can't get rid of numProcessing by just checking deadServers.size() because
> deadServers is also used to report some historically failed RSs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
