[jira] [Updated] (HBASE-9359) Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter
[ https://issues.apache.org/jira/browse/HBASE-9359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9359: -- Attachment: hbase-9359-9334.v6.patch v6 fixes javadoc warnings. Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter -- Key: HBASE-9359 URL: https://issues.apache.org/jira/browse/HBASE-9359 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 0.95.2 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.0 Attachments: hbase-9334-9359.v4.patch, hbase-9359-9334.v5.patch, hbase-9359-9334.v6.patch, hbase-9359.patch, hbase-9359.v2.patch, hbase-9359.v3.patch, hbase-9359.v5.patch This path is the second half of eliminating KeyValue from the client interfaces. This percolated through quite a bit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9382) replicateWALEntry doesn't use the replication handlers
[ https://issues.apache.org/jira/browse/HBASE-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9382: - Attachment: 9382.txt All priority was broke. Here is simple fix and test. This patch is for 0.95. replicateWALEntry doesn't use the replication handlers -- Key: HBASE-9382 URL: https://issues.apache.org/jira/browse/HBASE-9382 Project: HBase Issue Type: Bug Affects Versions: 0.95.2 Reporter: Jean-Daniel Cryans Assignee: stack Fix For: 0.98.0, 0.96.0 Attachments: 9382.txt By default we assign 3 handlers for replication, but as far as I can tell the replication traffic uses the normal handlers in 0.96 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9359) Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter
[ https://issues.apache.org/jira/browse/HBASE-9359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9359: -- Attachment: (was: hbase-9359-9334.v6.patch) Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter -- Key: HBASE-9359 URL: https://issues.apache.org/jira/browse/HBASE-9359 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 0.95.2 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.0 Attachments: hbase-9334-9359.v4.patch, hbase-9359-9334.v5.patch, hbase-9359-9334.v6.patch, hbase-9359.patch, hbase-9359.v2.patch, hbase-9359.v3.patch, hbase-9359.v5.patch, hbase-9359.v6.patch This path is the second half of eliminating KeyValue from the client interfaces. This percolated through quite a bit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9359) Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter
[ https://issues.apache.org/jira/browse/HBASE-9359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9359: -- Attachment: hbase-9359-9334.v6.patch hbase-9359.v6.patch NOTE - the javadoc problems and test failure problem were in the HBASE-9334 portion of the patch. Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter -- Key: HBASE-9359 URL: https://issues.apache.org/jira/browse/HBASE-9359 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 0.95.2 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.0 Attachments: hbase-9334-9359.v4.patch, hbase-9359-9334.v5.patch, hbase-9359-9334.v6.patch, hbase-9359.patch, hbase-9359.v2.patch, hbase-9359.v3.patch, hbase-9359.v5.patch, hbase-9359.v6.patch This path is the second half of eliminating KeyValue from the client interfaces. This percolated through quite a bit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-9334) Convert KeyValue to Cell in hbase-client module - Filters
[ https://issues.apache.org/jira/browse/HBASE-9334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754415#comment-13754415 ] Jonathan Hsieh edited comment on HBASE-9334 at 8/30/13 6:08 AM: v6. This includes javadoc fixes and shim correcntess fix. was (Author: jmhsieh): This includes javadoc fixes and shim correcntess fix. Convert KeyValue to Cell in hbase-client module - Filters - Key: HBASE-9334 URL: https://issues.apache.org/jira/browse/HBASE-9334 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 0.95.2 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9334.patch, hbase-9334.v2.patch, hbase-9334.v3.patch, hbase-9334.v4.patch, hbase-9334.v6.patch The goal is is to remove KeyValue from the publicly exposed API and require clients to use the cleaner mroe encapsulated Cell API instead. For filters, this affects #filterKeyValue, #transform, #filterrow, and #getNextKeyHint. Since Cell is a base interface for KeyValue, changing these means that 0.94 apps may need a recompile but probably no modifications. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9334) Convert KeyValue to Cell in hbase-client module - Filters
[ https://issues.apache.org/jira/browse/HBASE-9334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-9334: -- Attachment: hbase-9334.v6.patch This includes javadoc fixes and shim correcntess fix. Convert KeyValue to Cell in hbase-client module - Filters - Key: HBASE-9334 URL: https://issues.apache.org/jira/browse/HBASE-9334 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 0.95.2 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: hbase-9334.patch, hbase-9334.v2.patch, hbase-9334.v3.patch, hbase-9334.v4.patch, hbase-9334.v6.patch The goal is is to remove KeyValue from the publicly exposed API and require clients to use the cleaner mroe encapsulated Cell API instead. For filters, this affects #filterKeyValue, #transform, #filterrow, and #getNextKeyHint. Since Cell is a base interface for KeyValue, changing these means that 0.94 apps may need a recompile but probably no modifications. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9382) replicateWALEntry doesn't use the replication handlers
[ https://issues.apache.org/jira/browse/HBASE-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9382: - Attachment: 9382.trunk.txt Trunk version. replicateWALEntry doesn't use the replication handlers -- Key: HBASE-9382 URL: https://issues.apache.org/jira/browse/HBASE-9382 Project: HBase Issue Type: Bug Affects Versions: 0.95.2 Reporter: Jean-Daniel Cryans Assignee: stack Fix For: 0.98.0, 0.96.0 Attachments: 9382.trunk.txt, 9382.txt By default we assign 3 handlers for replication, but as far as I can tell the replication traffic uses the normal handlers in 0.96 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9382) replicateWALEntry doesn't use the replication handlers
[ https://issues.apache.org/jira/browse/HBASE-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9382: - Release Note: Fix regression introduced by pb styling of method names. TODO: have client say priority of method and remove all this QosFunction reflection stuff; its brittle and messy. Status: Patch Available (was: Open) replicateWALEntry doesn't use the replication handlers -- Key: HBASE-9382 URL: https://issues.apache.org/jira/browse/HBASE-9382 Project: HBase Issue Type: Bug Affects Versions: 0.95.2 Reporter: Jean-Daniel Cryans Assignee: stack Fix For: 0.98.0, 0.96.0 Attachments: 9382.trunk.txt, 9382.txt By default we assign 3 handlers for replication, but as far as I can tell the replication traffic uses the normal handlers in 0.96 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7954) Fix the retrying logic of memstore flushes to avoid extra sleep
[ https://issues.apache.org/jira/browse/HBASE-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754418#comment-13754418 ] stack commented on HBASE-7954: -- [~himan...@cloudera.com] Patch is for trunk and 0.94? Fix the retrying logic of memstore flushes to avoid extra sleep --- Key: HBASE-7954 URL: https://issues.apache.org/jira/browse/HBASE-7954 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.5, 0.95.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Priority: Minor Attachments: HBase-7954-v0.patch Matteo pointed out: We can avoid the redundant sleep in the retrying logic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
[ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754419#comment-13754419 ] haosdent commented on HBASE-5954: - hi, [~lhofhansl], whether your disks have raid or not? I have tested the hsync of hdfs again and again. I found it will spent nearly 50ms while hflush just spent 2ms on non-raid disks. Allow proper fsync support for HBase Key: HBASE-5954 URL: https://issues.apache.org/jira/browse/HBASE-5954 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.98.0 Attachments: 5954-trunk-hdfs-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt, 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt, hbase-hdfs-744.txt At least get recommendation into 0.96 doc and some numbers running w/ this hdfs feature enabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9385) Log HBase Master command line arguments on startup
[ https://issues.apache.org/jira/browse/HBASE-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754424#comment-13754424 ] Hudson commented on HBASE-9385: --- SUCCESS: Integrated in hbase-0.95 #508 (See [https://builds.apache.org/job/hbase-0.95/508/]) HBASE-9385 Log HBase Master command line arguments on startup (stack: rev 1518886) * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java Log HBase Master command line arguments on startup -- Key: HBASE-9385 URL: https://issues.apache.org/jira/browse/HBASE-9385 Project: HBase Issue Type: Improvement Components: master Reporter: Aditya Kishore Assignee: Aditya Kishore Priority: Minor Labels: troubleshooting Fix For: 0.98.0, 0.96.0 Attachments: HBASE-9385.patch Region server already log JVM information (name, version, vendor, command line arguments, etc). Would be useful to have Master do it too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7954) Fix the retrying logic of memstore flushes to avoid extra sleep
[
https://issues.apache.org/jira/browse/HBASE-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754429#comment-13754429
]
Hadoop QA commented on HBASE-7954:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600727/HBase-7954-v0.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6977//console
This message is automatically generated.
Fix the retrying logic of memstore flushes to avoid extra sleep
---
Key: HBASE-7954
URL: https://issues.apache.org/jira/browse/HBASE-7954
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.5, 0.95.0
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Priority: Minor
Attachments: HBase-7954-v0.patch
Matteo pointed out:
We can avoid the redundant sleep in the retrying logic.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9389) Favorednodes command line not verifying assignments correctly
Devaraj Das created HBASE-9389: -- Summary: Favorednodes command line not verifying assignments correctly Key: HBASE-9389 URL: https://issues.apache.org/jira/browse/HBASE-9389 Project: HBase Issue Type: Bug Reporter: Devaraj Das In manual testing, encountered an issue to do with the command line tool (HBASE-9116) not verifying the assignments correctly. Although the regions are assigned to the favored nodes, the verification/print showed otherwise. The command to reproduce the problem (after having created the tables, and loading some data): bin/hbase org.apache.hadoop.hbase.master.RegionPlacementMaintainer -tables cluster_test1,cluster_test2 -hbase_root /apps/hbase/data -v -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9389) Favorednodes command line not verifying assignments correctly
[ https://issues.apache.org/jira/browse/HBASE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-9389: --- Attachment: verification-fix.1.txt Patch with unit tests. Tested manually as well. Favorednodes command line not verifying assignments correctly - Key: HBASE-9389 URL: https://issues.apache.org/jira/browse/HBASE-9389 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: verification-fix.1.txt In manual testing, encountered an issue to do with the command line tool (HBASE-9116) not verifying the assignments correctly. Although the regions are assigned to the favored nodes, the verification/print showed otherwise. The command to reproduce the problem (after having created the tables, and loading some data): bin/hbase org.apache.hadoop.hbase.master.RegionPlacementMaintainer -tables cluster_test1,cluster_test2 -hbase_root /apps/hbase/data -v -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9389) Favorednodes command line not verifying assignments correctly
[ https://issues.apache.org/jira/browse/HBASE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-9389: --- Affects Version/s: 0.96.0 Fix Version/s: 0.96.0 Assignee: Devaraj Das Favorednodes command line not verifying assignments correctly - Key: HBASE-9389 URL: https://issues.apache.org/jira/browse/HBASE-9389 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 In manual testing, encountered an issue to do with the command line tool (HBASE-9116) not verifying the assignments correctly. Although the regions are assigned to the favored nodes, the verification/print showed otherwise. The command to reproduce the problem (after having created the tables, and loading some data): bin/hbase org.apache.hadoop.hbase.master.RegionPlacementMaintainer -tables cluster_test1,cluster_test2 -hbase_root /apps/hbase/data -v -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9389) Favorednodes command line not verifying assignments correctly
[ https://issues.apache.org/jira/browse/HBASE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-9389: --- Priority: Critical (was: Major) Favorednodes command line not verifying assignments correctly - Key: HBASE-9389 URL: https://issues.apache.org/jira/browse/HBASE-9389 Project: HBase Issue Type: Bug Reporter: Devaraj Das Priority: Critical In manual testing, encountered an issue to do with the command line tool (HBASE-9116) not verifying the assignments correctly. Although the regions are assigned to the favored nodes, the verification/print showed otherwise. The command to reproduce the problem (after having created the tables, and loading some data): bin/hbase org.apache.hadoop.hbase.master.RegionPlacementMaintainer -tables cluster_test1,cluster_test2 -hbase_root /apps/hbase/data -v -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9389) Favorednodes command line not verifying assignments correctly
[ https://issues.apache.org/jira/browse/HBASE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-9389: --- Status: Patch Available (was: Open) Favorednodes command line not verifying assignments correctly - Key: HBASE-9389 URL: https://issues.apache.org/jira/browse/HBASE-9389 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: verification-fix.1.txt In manual testing, encountered an issue to do with the command line tool (HBASE-9116) not verifying the assignments correctly. Although the regions are assigned to the favored nodes, the verification/print showed otherwise. The command to reproduce the problem (after having created the tables, and loading some data): bin/hbase org.apache.hadoop.hbase.master.RegionPlacementMaintainer -tables cluster_test1,cluster_test2 -hbase_root /apps/hbase/data -v -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9249) Add cp hook before setting PONR in split
[
https://issues.apache.org/jira/browse/HBASE-9249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754438#comment-13754438
]
Hadoop QA commented on HBASE-9249:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600730/HBASE-9249_v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6978//console
This message is automatically generated.
Add cp hook before setting PONR in split
Key: HBASE-9249
URL: https://issues.apache.org/jira/browse/HBASE-9249
Project: HBase
Issue Type: Sub-task
Reporter: rajeshbabu
Assignee: rajeshbabu
Attachments: HBASE-9249.patch, HBASE-9249_v2.patch,
HBASE-9249_v3.patch
This hook helps to perform split on user region and corresponding index
region such that both will be split or none.
With this hook split for user and index region as follows
user region
===
1) Create splitting znode for user region split
2) Close parent user region
3) split user region storefiles
4) instantiate child regions of user region
Through the new hook we can call index region transitions as below
index region
===
5) Create splitting znode for index region split
6) Close parent index region
7) Split storefiles of index region
8) instantiate child regions of the index region
If any failures in 5,6,7,8 rollback the steps and return null, on null return
throw exception to rollback for 1,2,3,4
9) set PONR
10) do batch put of offline and split entries for user and index regions
index region
11) open daughers of index regions and transition znode to split. This step
we will do through preSplitAfterPONR hook. Opening index regions before
opening user regions helps to avoid put failures if there is colocation
mismatch(this can happen if user regions opening completed but index regions
opening in progress)
user region
===
12) open daughers of user regions and transition znode to split.
Even if region server crashes also at the end both user and index regions
will be split or none
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7525) A canary monitoring program specifically for regionserver
[
https://issues.apache.org/jira/browse/HBASE-7525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754458#comment-13754458
]
takeshi.miao commented on HBASE-7525:
-
Dear [~stack]
Here is the answer for your questions
{quote}
./hbase-0.95.3-SNAPSHOT/bin/hbase --config /home/stack/conf_hbase
org.apache.hadoop.hbase.tool.Canary
... it goes off and does something; default looks to go and get from all
regions.
{quote}
Yes, it's default behavior is just align with the old one, does the all regions
monitoring
bq. You add 2013-08-29 09:32:16,463 DEBUG [main] tool.Canary: runCount=2. What
does it mean ?
It is the internal DEBUG msg, for counting how many loop of this monitor
instance did; It can help user to observe the monitor instance's behavior
whether as expected
Following are the questions you asked about _'-regionserver'_ option
{quote}
{code}
Usage: bin/hbase org.apache.hadoop.hbase.tool.Canary [opts] [table/regionserver
1 [table/regionserver 2...]]
...
{code}
{quote}
{quote}
Would it be clearer if the -regionserver option took arguments as in
-regionserver=rs1,rs2,rs3 etc.?
How to interpret this then:
Usage: bin/hbase org.apache.hadoop.hbase.tool.Canary -regionserver=rs1 table1
Would above only get regions from table1 on rs1? If no regions from table1 then
it would print out there are none?
{quote}
The option _'-regionserver'_ (regionserver mode) is exclusive with the default
mode (region mode), which means user can only choose to use default mode or
regionserver mode either
bq. I do not know how to read 'table/regionserver 1'. What is the '1'?
So it seems the usage output confuses the user, I would like to change it to
following, how do you think ?
{code}
Usage: bin/hbase org.apache.hadoop.hbase.tool.Canary [opts] [table|regionserver
[table|regionserver ...]]
...
{code}
{quote}
Or if you pass a table1 when you have a -regionserver option specified, you
could just fail with Cannot pass a tablename when using the -regionserver
option – that'd probably be simplest.
{quote}
Yes, this is a good suggestion, but currently I would not check this if the
passed arguments are whether tableNames in HBase, due to I need to new a
HBaseAdim instance to get the table list firstly, then compare them with the
passed argument.
How do you think that I modify the usage output more precisely for
-regionserver option ? such as...
{code}
...
-regionserver replace the table argument to regionserver,
which means to enable regionserver mode, instead of region mode (default)
...
{code}
Either way is ok for me.
I will upload the new patches after we confirm which way to go, and tks for
your questions and suggestions :)
A canary monitoring program specifically for regionserver
-
Key: HBASE-7525
URL: https://issues.apache.org/jira/browse/HBASE-7525
Project: HBase
Issue Type: New Feature
Components: monitoring
Affects Versions: 0.94.0
Reporter: takeshi.miao
Priority: Critical
Fix For: 0.98.0
Attachments: HBASE-7525-0.95-v0.patch, HBASE-7525-0.95-v1.patch,
HBASE-7525-0.95-v3.patch, HBASE-7525-0.95-v4.patch, HBASE-7525-0.95-v6.patch,
HBASE-7525-trunk-v2.patch, HBASE-7525-v0.patch, RegionServerCanary.java
*Motivation*
This ticket is to provide a canary monitoring tool specifically for
HRegionserver, details as follows
1. This tool is required by operation team due to they thought that the
canary for each region of a HBase is too many for them, so I implemented this
coarse-granular one based on the original o.a.h.h.tool.Canary for them
2. And this tool is implemented by multi-threading, which means the each Get
request sent by a thread. the reason I use this way is due to we suffered the
region server hung issue by now the root cause is still not clear. so this
tool can help operation team to detect hung region server if any.
*example*
1. the tool docs
./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary -help
Usage: [opts] [regionServerName 1 [regionServrName 2...]]
regionServerName - FQDN serverName, can use linux command:hostname -f to
check your serverName
where [-opts] are:
-help Show this help and exit.
-eUse regionServerName as regular expression
which means the regionServerName is regular expression pattern
-f B stop whole program if first error occurs, default is true
-t N timeout for a check, default is 60 (milisecs)
-daemonContinuous check at defined intervals.
-interval N Interval between checks (sec)
2. Will send a request to each regionserver in a HBase cluster
./bin/hbase org.apache.hadoop.hbase.tool.RegionServerCanary
3. Will send a request to a regionserver by given name
./bin/hbase
[jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
[ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754467#comment-13754467 ] haosdent commented on HBASE-5954: - [~lhofhansl] My test result: Without WAL/HFile sync: ~13s With WAL/HFile sync: ~120s Anything wrong? Allow proper fsync support for HBase Key: HBASE-5954 URL: https://issues.apache.org/jira/browse/HBASE-5954 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.98.0 Attachments: 5954-trunk-hdfs-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt, 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt, hbase-hdfs-744.txt At least get recommendation into 0.96 doc and some numbers running w/ this hdfs feature enabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9359) Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter
[ https://issues.apache.org/jira/browse/HBASE-9359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754469#comment-13754469 ] Jonathan Hsieh commented on HBASE-9359: --- v6 applies cleanly to 0.95/0.96. Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter -- Key: HBASE-9359 URL: https://issues.apache.org/jira/browse/HBASE-9359 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 0.95.2 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.0 Attachments: hbase-9334-9359.v4.patch, hbase-9359-9334.v5.patch, hbase-9359-9334.v6.patch, hbase-9359.patch, hbase-9359.v2.patch, hbase-9359.v3.patch, hbase-9359.v5.patch, hbase-9359.v6.patch This path is the second half of eliminating KeyValue from the client interfaces. This percolated through quite a bit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9359) Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter
[
https://issues.apache.org/jira/browse/HBASE-9359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754470#comment-13754470
]
Hadoop QA commented on HBASE-9359:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600739/hbase-9359-9334.v6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 231
new or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6979//console
This message is automatically generated.
Convert KeyValue to Cell in hbase-client module - Result/Put/Delete,
ColumnInterpreter
--
Key: HBASE-9359
URL: https://issues.apache.org/jira/browse/HBASE-9359
Project: HBase
Issue Type: Sub-task
Components: Client
Affects Versions: 0.95.2
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.98.0, 0.96.0
Attachments: hbase-9334-9359.v4.patch, hbase-9359-9334.v5.patch,
hbase-9359-9334.v6.patch, hbase-9359.patch, hbase-9359.v2.patch,
hbase-9359.v3.patch, hbase-9359.v5.patch, hbase-9359.v6.patch
This path is the second half of eliminating KeyValue from the client
interfaces. This percolated through quite a bit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
[ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754475#comment-13754475 ] haosdent commented on HBASE-5954: - Only when we open write barrier and mount disk with data=ordered, we could make sure that the data have been flush to physics disk after we call fsync system call. Allow proper fsync support for HBase Key: HBASE-5954 URL: https://issues.apache.org/jira/browse/HBASE-5954 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.98.0 Attachments: 5954-trunk-hdfs-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt, 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt, hbase-hdfs-744.txt At least get recommendation into 0.96 doc and some numbers running w/ this hdfs feature enabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
[ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754495#comment-13754495 ] Liang Xie commented on HBASE-5954: -- [~haosd...@gmail.com], IMHO, fsync + write barrier combination should has guarantee the data be written to disk (with issuing a disk cache flush instruction). is it relatived with data=ordered mount option? thanks Allow proper fsync support for HBase Key: HBASE-5954 URL: https://issues.apache.org/jira/browse/HBASE-5954 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.98.0 Attachments: 5954-trunk-hdfs-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt, 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt, hbase-hdfs-744.txt At least get recommendation into 0.96 doc and some numbers running w/ this hdfs feature enabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9359) Convert KeyValue to Cell in hbase-client module - Result/Put/Delete, ColumnInterpreter
[
https://issues.apache.org/jira/browse/HBASE-9359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754519#comment-13754519
]
Hadoop QA commented on HBASE-9359:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600739/hbase-9359-9334.v6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 231
new or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to cause Findbugs
(version 1.3.9) to fail.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6982//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6982//console
This message is automatically generated.
Convert KeyValue to Cell in hbase-client module - Result/Put/Delete,
ColumnInterpreter
--
Key: HBASE-9359
URL: https://issues.apache.org/jira/browse/HBASE-9359
Project: HBase
Issue Type: Sub-task
Components: Client
Affects Versions: 0.95.2
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Fix For: 0.98.0, 0.96.0
Attachments: hbase-9334-9359.v4.patch, hbase-9359-9334.v5.patch,
hbase-9359-9334.v6.patch, hbase-9359.patch, hbase-9359.v2.patch,
hbase-9359.v3.patch, hbase-9359.v5.patch, hbase-9359.v6.patch
This path is the second half of eliminating KeyValue from the client
interfaces. This percolated through quite a bit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9389) Favorednodes command line not verifying assignments correctly
[
https://issues.apache.org/jira/browse/HBASE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754533#comment-13754533
]
Nicolas Liochon commented on HBASE-9389:
for verifyRegionPlacement:
bq. + * @param report
It should be @ return
But while this method now returns a report it's never used?
+if (cfStatus.getPath().getName().startsWith(.) ||
+
HConstants.HBASE_NON_USER_TABLE_DIRS.contains(cfStatus.getPath().getName())) {
We were skipping the technical directories, and .META., and now explicitly
the system tables, but are we right to skip the system tables, .META. included?
I could imagine that we want to use the favored node configuration for them as
well.
Favorednodes command line not verifying assignments correctly
-
Key: HBASE-9389
URL: https://issues.apache.org/jira/browse/HBASE-9389
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Devaraj Das
Assignee: Devaraj Das
Priority: Critical
Fix For: 0.96.0
Attachments: verification-fix.1.txt
In manual testing, encountered an issue to do with the command line tool
(HBASE-9116) not verifying the assignments correctly. Although the regions
are assigned to the favored nodes, the verification/print showed otherwise.
The command to reproduce the problem (after having created the tables, and
loading some data):
bin/hbase org.apache.hadoop.hbase.master.RegionPlacementMaintainer -tables
cluster_test1,cluster_test2 -hbase_root /apps/hbase/data -v
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
Nicolas Liochon created HBASE-9390: -- Summary: coprocessors observers are not called during a recovery with the new log replay algorithm Key: HBASE-9390 URL: https://issues.apache.org/jira/browse/HBASE-9390 Project: HBase Issue Type: Bug Components: Coprocessors, MTTR Affects Versions: 0.95.2 Reporter: Nicolas Liochon See the patch to reproduce the issue: If we activate log replay we don't have the events on WAL restore. Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-9390: --- Attachment: copro.patch coprocessors observers are not called during a recovery with the new log replay algorithm - Key: HBASE-9390 URL: https://issues.apache.org/jira/browse/HBASE-9390 Project: HBase Issue Type: Bug Components: Coprocessors, MTTR Affects Versions: 0.95.2 Reporter: Nicolas Liochon Attachments: copro.patch See the patch to reproduce the issue: If we activate log replay we don't have the events on WAL restore. Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9314) Dropping a table always prints a TableInfoMissingException in the master log
[
https://issues.apache.org/jira/browse/HBASE-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-9314:
--
Attachment: 9314-0.94.patch
9314.patch
Testing the attached patches now.
Dropping a table always prints a TableInfoMissingException in the master log
Key: HBASE-9314
URL: https://issues.apache.org/jira/browse/HBASE-9314
Project: HBase
Issue Type: Improvement
Affects Versions: 0.95.2, 0.94.10
Reporter: Jean-Daniel Cryans
Assignee: Andrew Purtell
Priority: Minor
Fix For: 0.98.0, 0.94.12, 0.96.0
Attachments: 9314-0.94.patch, 9314.patch
Everytime I drop a table I get the same stack trace in the master's log:
{noformat}
2013-08-22 23:11:31,939 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Table 't' archived!
2013-08-22 23:11:31,939 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Removing 't'
descriptor.
2013-08-22 23:11:31,940 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Marking 't' as
deleted.
2013-08-22 23:11:31,944 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.zookeeper.lock.ZKInterProcessLockBase: Released
/hbase/table-lock/t/write-master:602
2013-08-22 23:11:32,024 DEBUG [RpcServer.handler=0,port=6]
org.apache.hadoop.hbase.util.FSTableDescriptors: Exception during
readTableDecriptor. Current table name = t
org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file
under hdfs://jdec2hbase0403-1.vpc.cloudera.com:9000/hbase/data/default/t
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorAndModtime(FSTableDescriptors.java:503)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorAndModtime(FSTableDescriptors.java:496)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:170)
at
org.apache.hadoop.hbase.master.HMaster.getTableDescriptors(HMaster.java:2629)
at
org.apache.hadoop.hbase.protobuf.generated.MasterMonitorProtos$MasterMonitorService$2.callBlockingMethod(MasterMonitorProtos.java:4634)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2156)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1861)
2013-08-22 23:11:32,024 WARN [RpcServer.handler=0,port=6]
org.apache.hadoop.hbase.util.FSTableDescriptors: The following folder is in
HBase's root directory and doesn't contain a table descriptor, do consider
deleting it: t
{noformat}
But the operation completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9391) Compilation problem in AccessController with JDK 6
Andrew Purtell created HBASE-9391:
-
Summary: Compilation problem in AccessController with JDK 6
Key: HBASE-9391
URL: https://issues.apache.org/jira/browse/HBASE-9391
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.2, 0.98.0
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Seeing this with a fresh checkout of trunk and 0.95, only with JDK 6:
{noformat}
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[541,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1072,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1383,64]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1473,63]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to
java.util.Mapbyte[],java.util.Collectionbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Collectionbyte[]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9391) Compilation problem in AccessController with JDK 6
[
https://issues.apache.org/jira/browse/HBASE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved HBASE-9391.
---
Resolution: Cannot Reproduce
Tried to reproduce this on another box and couldn't. Switched JDKs on the
troublesome box, compiled ok, switched back, compiled ok.
Compilation problem in AccessController with JDK 6
--
Key: HBASE-9391
URL: https://issues.apache.org/jira/browse/HBASE-9391
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Seeing this with a fresh checkout of trunk and 0.95, only with JDK 6:
{noformat}
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[541,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1072,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1383,64]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1473,63]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to
java.util.Mapbyte[],java.util.Collectionbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Collectionbyte[]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9249) Add cp hook before setting PONR in split
[ https://issues.apache.org/jira/browse/HBASE-9249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-9249: -- Attachment: HBASE-9249_v4.patch Fixing javadoc and findbug warnings. Add cp hook before setting PONR in split Key: HBASE-9249 URL: https://issues.apache.org/jira/browse/HBASE-9249 Project: HBase Issue Type: Sub-task Reporter: rajeshbabu Assignee: rajeshbabu Attachments: HBASE-9249.patch, HBASE-9249_v2.patch, HBASE-9249_v3.patch, HBASE-9249_v4.patch This hook helps to perform split on user region and corresponding index region such that both will be split or none. With this hook split for user and index region as follows user region === 1) Create splitting znode for user region split 2) Close parent user region 3) split user region storefiles 4) instantiate child regions of user region Through the new hook we can call index region transitions as below index region === 5) Create splitting znode for index region split 6) Close parent index region 7) Split storefiles of index region 8) instantiate child regions of the index region If any failures in 5,6,7,8 rollback the steps and return null, on null return throw exception to rollback for 1,2,3,4 9) set PONR 10) do batch put of offline and split entries for user and index regions index region 11) open daughers of index regions and transition znode to split. This step we will do through preSplitAfterPONR hook. Opening index regions before opening user regions helps to avoid put failures if there is colocation mismatch(this can happen if user regions opening completed but index regions opening in progress) user region === 12) open daughers of user regions and transition znode to split. Even if region server crashes also at the end both user and index regions will be split or none -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9392) Add RestartBackupMastersAction for ChaosMonkey
chendihao created HBASE-9392: Summary: Add RestartBackupMastersAction for ChaosMonkey Key: HBASE-9392 URL: https://issues.apache.org/jira/browse/HBASE-9392 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.3 Reporter: chendihao Priority: Minor Attachments: RestartBackupMastersAction.patch Just implement RestartBackupMastersAction for more failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9392) Add RestartBackupMastersAction for ChaosMonkey
[ https://issues.apache.org/jira/browse/HBASE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chendihao updated HBASE-9392: - Tags: it ChaosMonkey (was: ch) Add RestartBackupMastersAction for ChaosMonkey -- Key: HBASE-9392 URL: https://issues.apache.org/jira/browse/HBASE-9392 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.3 Reporter: chendihao Priority: Minor Attachments: RestartBackupMastersAction.patch Just implement RestartBackupMastersAction for more failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9392) Add RestartBackupMastersAction for ChaosMonkey
[ https://issues.apache.org/jira/browse/HBASE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chendihao updated HBASE-9392: - Attachment: RestartBackupMastersAction.patch patch for trunk Add RestartBackupMastersAction for ChaosMonkey -- Key: HBASE-9392 URL: https://issues.apache.org/jira/browse/HBASE-9392 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.3 Reporter: chendihao Priority: Minor Attachments: RestartBackupMastersAction.patch Just implement RestartBackupMastersAction for more failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9392) Add RestartBackupMastersAction for ChaosMonkey
[ https://issues.apache.org/jira/browse/HBASE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chendihao updated HBASE-9392: - Status: Patch Available (was: Open) Add RestartBackupMastersAction for ChaosMonkey -- Key: HBASE-9392 URL: https://issues.apache.org/jira/browse/HBASE-9392 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.3 Reporter: chendihao Priority: Minor Attachments: RestartBackupMastersAction.patch Just implement RestartBackupMastersAction for more failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-9391) Compilation problem in AccessController with JDK 6
[
https://issues.apache.org/jira/browse/HBASE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell reopened HBASE-9391:
---
Back after a clean and a new shell. Reopening to figure what is going on.
Compilation problem in AccessController with JDK 6
--
Key: HBASE-9391
URL: https://issues.apache.org/jira/browse/HBASE-9391
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Seeing this with a fresh checkout of trunk and 0.95, only with JDK 6:
{noformat}
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[541,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1072,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1383,64]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1473,63]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to
java.util.Mapbyte[],java.util.Collectionbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Collectionbyte[]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9391) Compilation problem in AccessController with JDK 6
[
https://issues.apache.org/jira/browse/HBASE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-9391:
--
Attachment: 9391.patch
Attached a patch that creates a TreeMap using its constructor.
Compilation problem in AccessController with JDK 6
--
Key: HBASE-9391
URL: https://issues.apache.org/jira/browse/HBASE-9391
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 9391.patch
Seeing this with a fresh checkout of trunk and 0.95, only with JDK 6:
{noformat}
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[541,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1072,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1383,64]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1473,63]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to
java.util.Mapbyte[],java.util.Collectionbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Collectionbyte[]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[ https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754617#comment-13754617 ] Jean-Marc Spaggiari commented on HBASE-9346: Only changes to HBaseFsck are intentionnals. Everything else is junk. Working on that. HBCK should provide an option to check if regions boundaries are the same in META and in stores. Key: HBASE-9346 URL: https://issues.apache.org/jira/browse/HBASE-9346 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.95.2, 0.94.11 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch, HBASE-9346-v2-trunk.patch If META don't have the same region boundaries as the stores files, writes and read might go to the wrong place. We need to provide a way to check that withing HBCK. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9249) Add cp hook before setting PONR in split
[
https://issues.apache.org/jira/browse/HBASE-9249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754619#comment-13754619
]
Hadoop QA commented on HBASE-9249:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600759/HBASE-9249_v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6983//console
This message is automatically generated.
Add cp hook before setting PONR in split
Key: HBASE-9249
URL: https://issues.apache.org/jira/browse/HBASE-9249
Project: HBase
Issue Type: Sub-task
Reporter: rajeshbabu
Assignee: rajeshbabu
Attachments: HBASE-9249.patch, HBASE-9249_v2.patch,
HBASE-9249_v3.patch, HBASE-9249_v4.patch
This hook helps to perform split on user region and corresponding index
region such that both will be split or none.
With this hook split for user and index region as follows
user region
===
1) Create splitting znode for user region split
2) Close parent user region
3) split user region storefiles
4) instantiate child regions of user region
Through the new hook we can call index region transitions as below
index region
===
5) Create splitting znode for index region split
6) Close parent index region
7) Split storefiles of index region
8) instantiate child regions of the index region
If any failures in 5,6,7,8 rollback the steps and return null, on null return
throw exception to rollback for 1,2,3,4
9) set PONR
10) do batch put of offline and split entries for user and index regions
index region
11) open daughers of index regions and transition znode to split. This step
we will do through preSplitAfterPONR hook. Opening index regions before
opening user regions helps to avoid put failures if there is colocation
mismatch(this can happen if user regions opening completed but index regions
opening in progress)
user region
===
12) open daughers of user regions and transition znode to split.
Even if region server crashes also at the end both user and index regions
will be split or none
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9261) Add cp hooks after {start|close}RegionOperation in batchMutate
[
https://issues.apache.org/jira/browse/HBASE-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
rajeshbabu updated HBASE-9261:
--
Attachment: HBASE-9261_v2.patch
@Ted,
Thanks for review.
In current patch Passing Operation enum to postStartRegionOperation.
Add cp hooks after {start|close}RegionOperation in batchMutate
--
Key: HBASE-9261
URL: https://issues.apache.org/jira/browse/HBASE-9261
Project: HBase
Issue Type: Sub-task
Reporter: rajeshbabu
Assignee: rajeshbabu
Attachments: HBASE-9261.patch, HBASE-9261_v2.patch
These hooks helps for checking Resources(blocking memstore size) and
necessary locking on index region while performing batch of mutations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[ https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9346: --- Attachment: HBASE-9346-v3-trunk.patch HBCK should provide an option to check if regions boundaries are the same in META and in stores. Key: HBASE-9346 URL: https://issues.apache.org/jira/browse/HBASE-9346 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.95.2, 0.94.11 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch, HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch If META don't have the same region boundaries as the stores files, writes and read might go to the wrong place. We need to provide a way to check that withing HBCK. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[ https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9346: --- Status: Patch Available (was: Open) Strange. Eclipse keep generating those extra files even if they are not selected in the UI... Anyway, cleaned version attached. Thanks. HBCK should provide an option to check if regions boundaries are the same in META and in stores. Key: HBASE-9346 URL: https://issues.apache.org/jira/browse/HBASE-9346 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.94.11, 0.95.2 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch, HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch If META don't have the same region boundaries as the stores files, writes and read might go to the wrong place. We need to provide a way to check that withing HBCK. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[ https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9346: --- Status: Open (was: Patch Available) HBCK should provide an option to check if regions boundaries are the same in META and in stores. Key: HBASE-9346 URL: https://issues.apache.org/jira/browse/HBASE-9346 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.94.11, 0.95.2 Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch, HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch If META don't have the same region boundaries as the stores files, writes and read might go to the wrong place. We need to provide a way to check that withing HBCK. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9392) Add RestartBackupMastersAction for ChaosMonkey
[
https://issues.apache.org/jira/browse/HBASE-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754629#comment-13754629
]
Hadoop QA commented on HBASE-9392:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600762/RestartBackupMastersAction.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6984//console
This message is automatically generated.
Add RestartBackupMastersAction for ChaosMonkey
--
Key: HBASE-9392
URL: https://issues.apache.org/jira/browse/HBASE-9392
Project: HBase
Issue Type: Improvement
Components: test
Affects Versions: 0.94.3
Reporter: chendihao
Priority: Minor
Attachments: RestartBackupMastersAction.patch
Just implement RestartBackupMastersAction for more failures.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8859) truncate_preserve should get table split keys as it is instead of converting them to string type and then again to bytes
[
https://issues.apache.org/jira/browse/HBASE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754639#comment-13754639
]
rajeshbabu commented on HBASE-8859:
---
Ted,
All split keys related APIs present only in HTable not in HTableInterface,
thats why not defined getSplitKeys() there. If we add it in HTI we need to
implement it in multiple classes. If its ok, we can add it.
truncate_preserve should get table split keys as it is instead of converting
them to string type and then again to bytes
Key: HBASE-8859
URL: https://issues.apache.org/jira/browse/HBASE-8859
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 0.95.1
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.96.0
Attachments: HBASE-8859-Test_to_reproduce.patch,
HBASE-8859_trunk_2.patch, HBASE-8859_trunk.patch
If we take int,long or double bytes as split keys then we are not creating
table with same split keys because converting them to strings directly and to
bytes is giving different split keys, sometimes getting IllegalArgument
exception because of same split keys(converted). Instead we can get split
keys directly from HTable and pass them while creating table.
{code}
h_table = org.apache.hadoop.hbase.client.HTable.new(conf, table_name)
splits = h_table.getRegionLocations().keys().map{|i| i.getStartKey}
:byte
splits = org.apache.hadoop.hbase.util.Bytes.toByteArrays(splits)
{code}
{code}
Truncating 'emp3' table (it may take a while):
- Disabling table...
- Dropping table...
- Creating table with region boundaries...
ERROR: java.lang.IllegalArgumentException: All split keys must be unique,
found duplicate: B\x11S\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD\x00\x00,
B\x11S\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD\x00\x00
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9385) Log HBase Master command line arguments on startup
[ https://issues.apache.org/jira/browse/HBASE-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754644#comment-13754644 ] Hudson commented on HBASE-9385: --- FAILURE: Integrated in hbase-0.95-on-hadoop2 #279 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/279/]) HBASE-9385 Log HBase Master command line arguments on startup (stack: rev 1518886) * /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java Log HBase Master command line arguments on startup -- Key: HBASE-9385 URL: https://issues.apache.org/jira/browse/HBASE-9385 Project: HBase Issue Type: Improvement Components: master Reporter: Aditya Kishore Assignee: Aditya Kishore Priority: Minor Labels: troubleshooting Fix For: 0.98.0, 0.96.0 Attachments: HBASE-9385.patch Region server already log JVM information (name, version, vendor, command line arguments, etc). Would be useful to have Master do it too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
[ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754646#comment-13754646 ] haosdent commented on HBASE-5954: - [~xieliang007]If mount disk with data=writeback, the dirty data may be in disk cache after fsync system call return. Until the data more than a ratio in disk cache or timer is trigger, them will flush to physics storage. We could improve the performance of hsync by disable journal and write cache. But after disable write cache, the whole write performance is worse than before. Fsync is a very heavy system call, I think it is unfeasible to call fsync after every write operation. Just post my test result about fsync roughly below: 1.ext4,noatime,barrier=1,data=ordered, enable disk write cache, enable journal, append 4k to a file fdatasync 25ms fsync 25ms 2.ext4,noatime,barrier=0,data=writeback, disable disk write cache, enable journal, append 4k to a file fdatasync 33ms fsync 33ms 3.ext4,noatime,barrier=0,data=writeback, disable disk write cache, disable journal, append 4k to a file fdatasync 8ms fsync 8ms Allow proper fsync support for HBase Key: HBASE-5954 URL: https://issues.apache.org/jira/browse/HBASE-5954 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.98.0 Attachments: 5954-trunk-hdfs-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt, 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt, hbase-hdfs-744.txt At least get recommendation into 0.96 doc and some numbers running w/ this hdfs feature enabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
[ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754654#comment-13754654 ] haosdent commented on HBASE-5954: - [~xieliang007]Because there is only have a journal file on every disk in extN, the system will commit all files metadata transactions when we open write barrier. After flush all files metadata transactions to the journal file in physics disk, the system will flush data(both metadata and block) of the special file to disk. So the fsync would spent more time when we have a lot of IO in a disk. My weibo is http://weibo.com/haosdent , welcome for more discussion. :-) Allow proper fsync support for HBase Key: HBASE-5954 URL: https://issues.apache.org/jira/browse/HBASE-5954 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.98.0 Attachments: 5954-trunk-hdfs-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt, 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt, hbase-hdfs-744.txt At least get recommendation into 0.96 doc and some numbers running w/ this hdfs feature enabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9261) Add cp hooks after {start|close}RegionOperation in batchMutate
[
https://issues.apache.org/jira/browse/HBASE-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754662#comment-13754662
]
Hadoop QA commented on HBASE-9261:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600771/HBASE-9261_v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6985//console
This message is automatically generated.
Add cp hooks after {start|close}RegionOperation in batchMutate
--
Key: HBASE-9261
URL: https://issues.apache.org/jira/browse/HBASE-9261
Project: HBase
Issue Type: Sub-task
Reporter: rajeshbabu
Assignee: rajeshbabu
Attachments: HBASE-9261.patch, HBASE-9261_v2.patch
These hooks helps for checking Resources(blocking memstore size) and
necessary locking on index region while performing batch of mutations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8859) truncate_preserve should get table split keys as it is instead of converting them to string type and then again to bytes
[
https://issues.apache.org/jira/browse/HBASE-8859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754682#comment-13754682
]
Ted Yu commented on HBASE-8859:
---
We don't need to add it in HTableInterface.
truncate_preserve should get table split keys as it is instead of converting
them to string type and then again to bytes
Key: HBASE-8859
URL: https://issues.apache.org/jira/browse/HBASE-8859
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 0.95.1
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.98.0, 0.96.0
Attachments: HBASE-8859-Test_to_reproduce.patch,
HBASE-8859_trunk_2.patch, HBASE-8859_trunk.patch
If we take int,long or double bytes as split keys then we are not creating
table with same split keys because converting them to strings directly and to
bytes is giving different split keys, sometimes getting IllegalArgument
exception because of same split keys(converted). Instead we can get split
keys directly from HTable and pass them while creating table.
{code}
h_table = org.apache.hadoop.hbase.client.HTable.new(conf, table_name)
splits = h_table.getRegionLocations().keys().map{|i| i.getStartKey}
:byte
splits = org.apache.hadoop.hbase.util.Bytes.toByteArrays(splits)
{code}
{code}
Truncating 'emp3' table (it may take a while):
- Disabling table...
- Dropping table...
- Creating table with region boundaries...
ERROR: java.lang.IllegalArgumentException: All split keys must be unique,
found duplicate: B\x11S\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD\x00\x00,
B\x11S\xEF\xBF\xBD\xEF\xBF\xBD\xEF\xBF\xBD\x00\x00
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9346) HBCK should provide an option to check if regions boundaries are the same in META and in stores.
[
https://issues.apache.org/jira/browse/HBASE-9346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754684#comment-13754684
]
Hadoop QA commented on HBASE-9346:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600774/HBASE-9346-v3-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6986//console
This message is automatically generated.
HBCK should provide an option to check if regions boundaries are the same in
META and in stores.
Key: HBASE-9346
URL: https://issues.apache.org/jira/browse/HBASE-9346
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.95.2, 0.94.11
Reporter: Jean-Marc Spaggiari
Assignee: Jean-Marc Spaggiari
Attachments: HBASE-9346-v0-0.94.patch, HBASE-9346-v1-trunk.patch,
HBASE-9346-v2-trunk.patch, HBASE-9346-v3-trunk.patch
If META don't have the same region boundaries as the stores files, writes and
read might go to the wrong place. We need to provide a way to check that
withing HBCK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9385) Log HBase Master command line arguments on startup
[ https://issues.apache.org/jira/browse/HBASE-9385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754687#comment-13754687 ] Hudson commented on HBASE-9385: --- FAILURE: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #703 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/703/]) HBASE-9385 Log HBase Master command line arguments on startup (stack: rev 1518887) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMasterCommandLine.java Log HBase Master command line arguments on startup -- Key: HBASE-9385 URL: https://issues.apache.org/jira/browse/HBASE-9385 Project: HBase Issue Type: Improvement Components: master Reporter: Aditya Kishore Assignee: Aditya Kishore Priority: Minor Labels: troubleshooting Fix For: 0.98.0, 0.96.0 Attachments: HBASE-9385.patch Region server already log JVM information (name, version, vendor, command line arguments, etc). Would be useful to have Master do it too. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9391) Compilation problem in AccessController with JDK 6
[
https://issues.apache.org/jira/browse/HBASE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-9391:
--
Priority: Critical (was: Major)
Seeing this on another newly made dev host also, raising to critical.
Compilation problem in AccessController with JDK 6
--
Key: HBASE-9391
URL: https://issues.apache.org/jira/browse/HBASE-9391
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Critical
Attachments: 9391.patch
Seeing this with a fresh checkout of trunk and 0.95, only with JDK 6:
{noformat}
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[541,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1072,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1383,64]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1473,63]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to
java.util.Mapbyte[],java.util.Collectionbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Collectionbyte[]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7954) Fix the retrying logic of memstore flushes to avoid extra sleep
[ https://issues.apache.org/jira/browse/HBASE-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-7954: --- Attachment: HBase-7954-94.patch 0.94 patch. Fix the retrying logic of memstore flushes to avoid extra sleep --- Key: HBASE-7954 URL: https://issues.apache.org/jira/browse/HBASE-7954 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.5, 0.95.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Priority: Minor Attachments: HBase-7954-94.patch, HBase-7954-v0.patch Matteo pointed out: We can avoid the redundant sleep in the retrying logic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7954) Fix the retrying logic of memstore flushes to avoid extra sleep
[
https://issues.apache.org/jira/browse/HBASE-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754738#comment-13754738
]
Hadoop QA commented on HBASE-7954:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600791/HBase-7954-94.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6987//console
This message is automatically generated.
Fix the retrying logic of memstore flushes to avoid extra sleep
---
Key: HBASE-7954
URL: https://issues.apache.org/jira/browse/HBASE-7954
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.5, 0.95.0
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
Priority: Minor
Attachments: HBase-7954-94.patch, HBase-7954-v0.patch
Matteo pointed out:
We can avoid the redundant sleep in the retrying logic.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9391) Compilation problem in AccessController with JDK 6
[
https://issues.apache.org/jira/browse/HBASE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754775#comment-13754775
]
stack commented on HBASE-9391:
--
Patch looks innocuous. +1
Compilation problem in AccessController with JDK 6
--
Key: HBASE-9391
URL: https://issues.apache.org/jira/browse/HBASE-9391
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 9391.patch
Seeing this with a fresh checkout of trunk and 0.95, only with JDK 6:
{noformat}
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[541,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1072,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1383,64]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1473,63]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to
java.util.Mapbyte[],java.util.Collectionbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Collectionbyte[]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9391) Compilation problem in AccessController with JDK 6
[
https://issues.apache.org/jira/browse/HBASE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754769#comment-13754769
]
Andrew Purtell commented on HBASE-9391:
---
Thanks for the pointer. Yes, confirmed - the build systems complaining are new
CentOS 6 and Debian 7 installs where I haven't changed the JDK from the
default.
Compilation problem in AccessController with JDK 6
--
Key: HBASE-9391
URL: https://issues.apache.org/jira/browse/HBASE-9391
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Critical
Attachments: 9391.patch
Seeing this with a fresh checkout of trunk and 0.95, only with JDK 6:
{noformat}
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[541,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1072,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1383,64]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1473,63]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to
java.util.Mapbyte[],java.util.Collectionbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Collectionbyte[]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9391) Compilation problem in AccessController with JDK 6
[
https://issues.apache.org/jira/browse/HBASE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-9391:
--
Priority: Major (was: Critical)
Not critical though if only OpenJDK.
Compilation problem in AccessController with JDK 6
--
Key: HBASE-9391
URL: https://issues.apache.org/jira/browse/HBASE-9391
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Attachments: 9391.patch
Seeing this with a fresh checkout of trunk and 0.95, only with JDK 6:
{noformat}
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[541,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1072,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1383,64]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1473,63]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to
java.util.Mapbyte[],java.util.Collectionbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Collectionbyte[]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9314) Dropping a table always prints a TableInfoMissingException in the master log
[
https://issues.apache.org/jira/browse/HBASE-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754777#comment-13754777
]
stack commented on HBASE-9314:
--
+1 if the return of null does not cause new issue
Dropping a table always prints a TableInfoMissingException in the master log
Key: HBASE-9314
URL: https://issues.apache.org/jira/browse/HBASE-9314
Project: HBase
Issue Type: Improvement
Affects Versions: 0.95.2, 0.94.10
Reporter: Jean-Daniel Cryans
Assignee: Andrew Purtell
Priority: Minor
Fix For: 0.98.0, 0.94.12, 0.96.0
Attachments: 9314-0.94.patch, 9314.patch
Everytime I drop a table I get the same stack trace in the master's log:
{noformat}
2013-08-22 23:11:31,939 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Table 't' archived!
2013-08-22 23:11:31,939 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Removing 't'
descriptor.
2013-08-22 23:11:31,940 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Marking 't' as
deleted.
2013-08-22 23:11:31,944 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.zookeeper.lock.ZKInterProcessLockBase: Released
/hbase/table-lock/t/write-master:602
2013-08-22 23:11:32,024 DEBUG [RpcServer.handler=0,port=6]
org.apache.hadoop.hbase.util.FSTableDescriptors: Exception during
readTableDecriptor. Current table name = t
org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file
under hdfs://jdec2hbase0403-1.vpc.cloudera.com:9000/hbase/data/default/t
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorAndModtime(FSTableDescriptors.java:503)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorAndModtime(FSTableDescriptors.java:496)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:170)
at
org.apache.hadoop.hbase.master.HMaster.getTableDescriptors(HMaster.java:2629)
at
org.apache.hadoop.hbase.protobuf.generated.MasterMonitorProtos$MasterMonitorService$2.callBlockingMethod(MasterMonitorProtos.java:4634)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2156)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1861)
2013-08-22 23:11:32,024 WARN [RpcServer.handler=0,port=6]
org.apache.hadoop.hbase.util.FSTableDescriptors: The following folder is in
HBase's root directory and doesn't contain a table descriptor, do consider
deleting it: t
{noformat}
But the operation completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754780#comment-13754780 ] stack commented on HBASE-9390: -- lgtm coprocessors observers are not called during a recovery with the new log replay algorithm - Key: HBASE-9390 URL: https://issues.apache.org/jira/browse/HBASE-9390 Project: HBase Issue Type: Bug Components: Coprocessors, MTTR Affects Versions: 0.95.2 Reporter: Nicolas Liochon Attachments: copro.patch See the patch to reproduce the issue: If we activate log replay we don't have the events on WAL restore. Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9389) Favorednodes command line not verifying assignments correctly
[ https://issues.apache.org/jira/browse/HBASE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-9389: --- Attachment: 9389-1.txt bq. But while this method now returns a report it's never used? It's used in the test. bq. I could imagine that we want to use the favored node configuration for them as well. The favored node is skipped for meta and that is true everywhere in the favored node code (it can be handled but it's not currently). The check in the block of code is to skip files/directories that don't have store files in them. For example, .regioninfo, oldWALs, etc. within the region directory. It's not skipping the system tables really. Favorednodes command line not verifying assignments correctly - Key: HBASE-9389 URL: https://issues.apache.org/jira/browse/HBASE-9389 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 9389-1.txt, verification-fix.1.txt In manual testing, encountered an issue to do with the command line tool (HBASE-9116) not verifying the assignments correctly. Although the regions are assigned to the favored nodes, the verification/print showed otherwise. The command to reproduce the problem (after having created the tables, and loading some data): bin/hbase org.apache.hadoop.hbase.master.RegionPlacementMaintainer -tables cluster_test1,cluster_test2 -hbase_root /apps/hbase/data -v -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9314) Dropping a table always prints a TableInfoMissingException in the master log
[
https://issues.apache.org/jira/browse/HBASE-9314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754782#comment-13754782
]
Andrew Purtell commented on HBASE-9314:
---
I looked at all users and they check for and handle nulls. Running unit tests
locally to see if any new complaints - not finished yet.
Dropping a table always prints a TableInfoMissingException in the master log
Key: HBASE-9314
URL: https://issues.apache.org/jira/browse/HBASE-9314
Project: HBase
Issue Type: Improvement
Affects Versions: 0.95.2, 0.94.10
Reporter: Jean-Daniel Cryans
Assignee: Andrew Purtell
Priority: Minor
Fix For: 0.98.0, 0.94.12, 0.96.0
Attachments: 9314-0.94.patch, 9314.patch
Everytime I drop a table I get the same stack trace in the master's log:
{noformat}
2013-08-22 23:11:31,939 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Table 't' archived!
2013-08-22 23:11:31,939 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Removing 't'
descriptor.
2013-08-22 23:11:31,940 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Marking 't' as
deleted.
2013-08-22 23:11:31,944 DEBUG
[MASTER_TABLE_OPERATIONS-jdec2hbase0403-1:6-0]
org.apache.hadoop.hbase.zookeeper.lock.ZKInterProcessLockBase: Released
/hbase/table-lock/t/write-master:602
2013-08-22 23:11:32,024 DEBUG [RpcServer.handler=0,port=6]
org.apache.hadoop.hbase.util.FSTableDescriptors: Exception during
readTableDecriptor. Current table name = t
org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file
under hdfs://jdec2hbase0403-1.vpc.cloudera.com:9000/hbase/data/default/t
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorAndModtime(FSTableDescriptors.java:503)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorAndModtime(FSTableDescriptors.java:496)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:170)
at
org.apache.hadoop.hbase.master.HMaster.getTableDescriptors(HMaster.java:2629)
at
org.apache.hadoop.hbase.protobuf.generated.MasterMonitorProtos$MasterMonitorService$2.callBlockingMethod(MasterMonitorProtos.java:4634)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2156)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1861)
2013-08-22 23:11:32,024 WARN [RpcServer.handler=0,port=6]
org.apache.hadoop.hbase.util.FSTableDescriptors: The following folder is in
HBase's root directory and doesn't contain a table descriptor, do consider
deleting it: t
{noformat}
But the operation completes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9389) Favorednodes command line not verifying assignments correctly
[ https://issues.apache.org/jira/browse/HBASE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754786#comment-13754786 ] Nicolas Liochon commented on HBASE-9389: bq. The favored node is skipped for meta and that is true everywhere in the favored node code (it can be handled but it's not currently). Oh. I missed that previously. Good to know. +1 Favorednodes command line not verifying assignments correctly - Key: HBASE-9389 URL: https://issues.apache.org/jira/browse/HBASE-9389 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Devaraj Das Assignee: Devaraj Das Priority: Critical Fix For: 0.96.0 Attachments: 9389-1.txt, verification-fix.1.txt In manual testing, encountered an issue to do with the command line tool (HBASE-9116) not verifying the assignments correctly. Although the regions are assigned to the favored nodes, the verification/print showed otherwise. The command to reproduce the problem (after having created the tables, and loading some data): bin/hbase org.apache.hadoop.hbase.master.RegionPlacementMaintainer -tables cluster_test1,cluster_test2 -hbase_root /apps/hbase/data -v -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9261) Add cp hooks after {start|close}RegionOperation in batchMutate
[
https://issues.apache.org/jira/browse/HBASE-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754823#comment-13754823
]
Ted Yu commented on HBASE-9261:
---
lgtm
Add cp hooks after {start|close}RegionOperation in batchMutate
--
Key: HBASE-9261
URL: https://issues.apache.org/jira/browse/HBASE-9261
Project: HBase
Issue Type: Sub-task
Reporter: rajeshbabu
Assignee: rajeshbabu
Attachments: HBASE-9261.patch, HBASE-9261_v2.patch
These hooks helps for checking Resources(blocking memstore size) and
necessary locking on index region while performing batch of mutations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754837#comment-13754837
]
Jimmy Xiang commented on HBASE-9387:
Most likely your ZK/network was in a bad state at that moment. RS setData was
retried while master already got it and deleted it, so RS setData retry failed.
The RS ZKUtil.setData retry took so long, this must be some thread scheduling
issue too. This issue should apply to all scenario setData retry is used. The
root cause is that setData succeeds, the other party gets it and does something
with the data, the setData retry fails since the data is updated by the other
party, so the setData caller is fooled/screwed.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 9387-v1.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754825#comment-13754825 ] Nicolas Liochon commented on HBASE-9390: bq. Where is the above method called ? Dynamically, that's how SimpleRegionObserver works today. But it's just a scenario to reproduce the problem, there is no fix. coprocessors observers are not called during a recovery with the new log replay algorithm - Key: HBASE-9390 URL: https://issues.apache.org/jira/browse/HBASE-9390 Project: HBase Issue Type: Bug Components: Coprocessors, MTTR Affects Versions: 0.95.2 Reporter: Nicolas Liochon Attachments: copro.patch See the patch to reproduce the issue: If we activate log replay we don't have the events on WAL restore. Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9388) [replication] ZK Dump prints the raw PBUF for the HLog positions
[ https://issues.apache.org/jira/browse/HBASE-9388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754831#comment-13754831 ] Jean-Daniel Cryans commented on HBASE-9388: --- Oh yeah I missed that, I thought this was just used for HLog stuff. [replication] ZK Dump prints the raw PBUF for the HLog positions Key: HBASE-9388 URL: https://issues.apache.org/jira/browse/HBASE-9388 Project: HBase Issue Type: Bug Affects Versions: 0.95.2 Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans Fix For: 0.98.0, 0.96.0 Attachments: HBASE-9388.patch Looking at the ZK dump in the master's web ui, I can see that we're not trying to parse the positions for the HLogs so it looks like PBU��� -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7954) Fix the retrying logic of memstore flushes to avoid extra sleep
[ https://issues.apache.org/jira/browse/HBASE-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7954: - Resolution: Fixed Fix Version/s: 0.94.12 0.98.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and to 0.94. Thanks for patch Himanshu. Fix the retrying logic of memstore flushes to avoid extra sleep --- Key: HBASE-7954 URL: https://issues.apache.org/jira/browse/HBASE-7954 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.5, 0.95.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Priority: Minor Fix For: 0.98.0, 0.94.12 Attachments: HBase-7954-94.patch, HBase-7954-v0.patch Matteo pointed out: We can avoid the redundant sleep in the retrying logic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9373) [replication] data loss because replication doesn't expect partial reads
[
https://issues.apache.org/jira/browse/HBASE-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Daniel Cryans updated HBASE-9373:
--
Attachment: 9373-v3.txt
Patch v3 wraps the method with a try catch that handles EOF, which is now
thrown inside if something goes wrong while parsing. Now we do the seek+return
false only in one place. I also dropped down the log level to trace.
I tested it twice at this point and didn't lose data.
[replication] data loss because replication doesn't expect partial reads
Key: HBASE-9373
URL: https://issues.apache.org/jira/browse/HBASE-9373
Project: HBase
Issue Type: Improvement
Affects Versions: 0.95.2
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Blocker
Fix For: 0.98.0, 0.96.0
Attachments: 9373.txt, 9373-v2.txt, 9373-v3.txt
When I see this in the logs it often means we got a partial read and then we
have the wrong offset when reading the rest of the file
{noformat}
2013-08-28 23:16:07,182 ERROR
[ReplicationExecutor-0.replicationSource,1-jdec2hbase0403-5,60020,1377730319617]
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader: Invalid PB while
reading WAL, probably an unexpected EOF, ignoring
com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
invalid wire type.
at
com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:99)
at
com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:498)
at
com.google.protobuf.GeneratedMessage.parseUnknownField(GeneratedMessage.java:193)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey.init(WALProtos.java:686)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey.init(WALProtos.java:644)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey$1.parsePartialFrom(WALProtos.java:771)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey$1.parsePartialFrom(WALProtos.java:766)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey$Builder.mergeFrom(WALProtos.java:1444)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey$Builder.mergeFrom(WALProtos.java:1218)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:220)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:912)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:290)
at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:926)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:296)
at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:918)
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:197)
at
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:98)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.readNextAndSetPosition(ReplicationHLogReaderManager.java:89)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:390)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:298)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754896#comment-13754896
]
Jeffrey Zhong commented on HBASE-9387:
--
If we abort RS when state is in un-expected state during transition, we could
abort RS too easily. That's why I suggest to only abort for the issue we know.
In this issue, znode is gone and our region assignment statement machine can't
continue. I think we should just target this case when znode is gone
unexpectedly.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 9387-v1.txt, 9387-v3.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9386) [WINDOWS] Small improvements to .cmd scripts
[ https://issues.apache.org/jira/browse/HBASE-9386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754900#comment-13754900 ] Nicolas Liochon commented on HBASE-9386: I'm not a windows script expert, but the patch is ok as far as I can tell (I've check that the variable names were ok, this kind of things). [WINDOWS] Small improvements to .cmd scripts Key: HBASE-9386 URL: https://issues.apache.org/jira/browse/HBASE-9386 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.98.0, 0.96.0 Attachments: hbase-9386_v1.patch A collection of small improvements to the .cmd scripts are needed: - Have hbase.cmd honor --config option - Fix log4j default appenders for standalone or service mode - classpath fixes - Honor JRuby options -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7954) Fix the retrying logic of memstore flushes to avoid extra sleep
[ https://issues.apache.org/jira/browse/HBASE-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754893#comment-13754893 ] Hudson commented on HBASE-7954: --- SUCCESS: Integrated in HBase-0.94-security #276 (See [https://builds.apache.org/job/HBase-0.94-security/276/]) HBASE-7954 Fix the retrying logic of memstore flushes to avoid extra sleep (stack: rev 1519011) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Fix the retrying logic of memstore flushes to avoid extra sleep --- Key: HBASE-7954 URL: https://issues.apache.org/jira/browse/HBASE-7954 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.5, 0.95.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Priority: Minor Fix For: 0.98.0, 0.94.12 Attachments: HBase-7954-94.patch, HBase-7954-v0.patch Matteo pointed out: We can avoid the redundant sleep in the retrying logic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5954) Allow proper fsync support for HBase
[ https://issues.apache.org/jira/browse/HBASE-5954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754869#comment-13754869 ] Lars Hofhansl commented on HBASE-5954: -- @haosdent, we can't break the laws of physics. :) If you sync *every single edit* you'll see terrible performance, how can we expect otherwise? HBase (even without fsync) wants things in batches, in PE HTable is doing it's default batching (2m batches), so that's where the cost is amortized. Enabling sync behind writes should improve this too (since we're writing immutable data), since by the time we issue the sync some data will already be sync'ed. Lastly, fsync is fsync (or rather fdatasync and friends since we're sync'ing files and not filesystems)... Once executed, previously cached data is on disk no matter what the filesystem chooses to cache during normal operations; only barriers are needed for correctness (AFAIK). Allow proper fsync support for HBase Key: HBASE-5954 URL: https://issues.apache.org/jira/browse/HBASE-5954 Project: HBase Issue Type: Improvement Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.98.0 Attachments: 5954-trunk-hdfs-trunk.txt, 5954-trunk-hdfs-trunk-v2.txt, 5954-trunk-hdfs-trunk-v3.txt, 5954-trunk-hdfs-trunk-v4.txt, 5954-trunk-hdfs-trunk-v5.txt, 5954-trunk-hdfs-trunk-v6.txt, hbase-hdfs-744.txt At least get recommendation into 0.96 doc and some numbers running w/ this hdfs feature enabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9386) [WINDOWS] Small improvements to .cmd scripts
[ https://issues.apache.org/jira/browse/HBASE-9386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754904#comment-13754904 ] stack commented on HBASE-9386: -- What [~liochon] said [WINDOWS] Small improvements to .cmd scripts Key: HBASE-9386 URL: https://issues.apache.org/jira/browse/HBASE-9386 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.98.0, 0.96.0 Attachments: hbase-9386_v1.patch A collection of small improvements to the .cmd scripts are needed: - Have hbase.cmd honor --config option - Fix log4j default appenders for standalone or service mode - classpath fixes - Honor JRuby options -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754871#comment-13754871
]
Jimmy Xiang commented on HBASE-9387:
Agree. In this case, can we check again if the node still exists? If it is
still there, that means we failed to transition the node, we need to close the
region. If it is not there, rs can abort, just to be safe?
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 9387-v1.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9389) Favorednodes command line not verifying assignments correctly
[
https://issues.apache.org/jira/browse/HBASE-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754870#comment-13754870
]
Hadoop QA commented on HBASE-9389:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12600797/9389-1.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6989//console
This message is automatically generated.
Favorednodes command line not verifying assignments correctly
-
Key: HBASE-9389
URL: https://issues.apache.org/jira/browse/HBASE-9389
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Devaraj Das
Assignee: Devaraj Das
Priority: Critical
Fix For: 0.96.0
Attachments: 9389-1.txt, verification-fix.1.txt
In manual testing, encountered an issue to do with the command line tool
(HBASE-9116) not verifying the assignments correctly. Although the regions
are assigned to the favored nodes, the verification/print showed otherwise.
The command to reproduce the problem (after having created the tables, and
loading some data):
bin/hbase org.apache.hadoop.hbase.master.RegionPlacementMaintainer -tables
cluster_test1,cluster_test2 -hbase_root /apps/hbase/data -v
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9387:
--
Attachment: 9387-v3.txt
Thanks for the comments.
Patch v3 also handles failure scenario in
tryTransitionFromOfflineToFailedOpen().
Overnight I looped TestFullLogReconstruction 100 times on the same machine
where this issue was first produced, with patch v1.
They all passed.
Follow-on JIRA can be filed to make region transition handling better.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 9387-v1.txt, 9387-v3.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754895#comment-13754895
]
stack commented on HBASE-9387:
--
-1 on making new returns other than -1, at least not w/o review and design;
ain't this all complex enough?
On the patch, should be log.error, not log.warn and not needed anyways since
doesn't abort log?
This is an awful workaround but I have no better idea currently.
Regards other places needing review in open region handler, I think they are
fine... it is just this edge where control of the region is being handed off
that is the issue.
Are there other edges around close say that have similar issues?
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 9387-v1.txt, 9387-v3.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9394) [replication] size accounting is completely off in the source
[
https://issues.apache.org/jira/browse/HBASE-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Daniel Cryans updated HBASE-9394:
--
Attachment: HBASE-9394.patch
I messed up in HBASE-5778, I fixed it in 0.94 but not in trunk.
I'm also adding my new size debugging to the existing log, and putting more
stuff to trace.
[replication] size accounting is completely off in the source
-
Key: HBASE-9394
URL: https://issues.apache.org/jira/browse/HBASE-9394
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.2
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.98.0, 0.96.0
Attachments: HBASE-9394.patch
I was under the impression that I was sending way more data than I was
expecting, so adding some more tracing I can see how much data I read VS how
I much I think I'm sending:
{noformat}
Seeking at position 163771687
Replicating 2 entries of total size 2790
Seeking at position 166723643
{noformat}
That's about 2MB of data, not 2KB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754906#comment-13754906
]
stack commented on HBASE-9387:
--
oh, needs a test too.
[~jeffreyz] agree; agree that we should pick out the explicit scenarios where
we can lose region accountability. Since this is the only one we know of
currently, perhaps just do patch for this case for now... and try to figure
other holes in state machine outside of this issue.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 9387-v1.txt, 9387-v3.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9384) [WINDOWS] Using file://{hbase.tmp.dir}/hbase for hbase.rootdir causes illegal argument exception on windows
[
https://issues.apache.org/jira/browse/HBASE-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-9384:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed this. Thanks Stack for review.
[WINDOWS] Using file://{hbase.tmp.dir}/hbase for hbase.rootdir causes illegal
argument exception on windows
---
Key: HBASE-9384
URL: https://issues.apache.org/jira/browse/HBASE-9384
Project: HBase
Issue Type: Bug
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.98.0, 0.96.0
Attachments: hbase-9384_v1.patch
We define hbase.rootdir as follows in hbase-default.xml:
{code}
property
namehbase.tmp.dir/name
value${java.io.tmpdir}/hbase-${user.name}/value
/property
property
namehbase.rootdir/name
valuefile://${hbase.tmp.dir}/hbase/value
/property
{code}
This causes an
java.lang.IllegalArgumentException: Wrong FS:
file://C:\Users\Administrator\AppData\Local\Temp\2\/hbase-Administrator/hbase,
expected: file:///
on windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754877#comment-13754877
]
Ted Yu commented on HBASE-9387:
---
Should a special return value be used in the following case of
ZKAssign.transitionNode() ?
{code}
} catch (KeeperException.NoNodeException nne) {
{code}
That way, tryTransitionFromOpeningToFailedOpen() would be able to check the
return value and act accordingly.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 9387-v1.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9387:
-
Priority: Critical (was: Major)
Suggested workaround would be to review all transitions and on edges such as
this one, going from OPENING to OPENED, if it fails, do a radical abort (not
just for meta region).
Then in another issue stepback and revisit our system for managing region
manipulation. It is way to complex consuming way too many hours of eng. time
and there are holes.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 9387-v1.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9394) [replication] size accounting is completely off in the source
[
https://issues.apache.org/jira/browse/HBASE-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jean-Daniel Cryans updated HBASE-9394:
--
Status: Patch Available (was: Open)
[replication] size accounting is completely off in the source
-
Key: HBASE-9394
URL: https://issues.apache.org/jira/browse/HBASE-9394
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.2
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.98.0, 0.96.0
Attachments: HBASE-9394.patch
I was under the impression that I was sending way more data than I was
expecting, so adding some more tracing I can see how much data I read VS how
I much I think I'm sending:
{noformat}
Seeking at position 163771687
Replicating 2 entries of total size 2790
Seeking at position 166723643
{noformat}
That's about 2MB of data, not 2KB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9387:
--
Attachment: 9387-v4.txt
Patch v4 removes the LOG.warn() statements.
Let me see if I can write a test.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 9387-v1.txt, 9387-v3.txt, 9387-v4.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754885#comment-13754885
]
stack commented on HBASE-9387:
--
[~jxiang] regards the xtra step of confirming the znode in still there, that
would be tough given the master is going to remove it. I'd imagine we'd see
many cases of the region getting closed by the RS because it could not do this
last (new) step. I say that on the edges of transitions, we just abort for now.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 9387-v1.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9393) Hbase dose not closing a closed socket resulting in many CLOSE_WAIT
[
https://issues.apache.org/jira/browse/HBASE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754912#comment-13754912
]
Enis Soztutar commented on HBASE-9393:
--
We have also observed a similar situation which rendered some of the
regionservers unusable because master was not able to open more sockets to the
regionserver.
{code}
$ for i in `cat allnodes`; do echo $i; ssh $i netstat -to | grep CLOSE_WAIT ;
done
horn04
tcp 22 0 horn04.gq1.ygridcore:49853 horn04.gq1.ygridcore:50010
CLOSE_WAIT off (0.00/0/0)
tcp1 0 horn04.gq1.ygridcore:49812 horn04.gq1.ygridcore:50010
CLOSE_WAIT off (0.00/0/0)
horn05
tcp 76 0 horn05.gq1.ygridcore:40253 horn05.gq1.ygridcore:50010
CLOSE_WAIT off (0.00/0/0)
tcp1 0 horn05.gq1.ygridcore:39667 horn05.gq1.ygridcore:50010
CLOSE_WAIT off (0.00/0/0)
tcp 166 0 horn05.gq1.ygridcore:39919 horn05.gq1.ygridcore:50010
CLOSE_WAIT off (0.00/0/0)
tcp 97 0 horn05.gq1.ygridcore:40631 horn05.gq1.ygridcore:50010
CLOSE_WAIT off (0.00/0/0)
tcp5 0 horn05.gq1.ygridcore:40227 horn05.gq1.ygridcore:50010
CLOSE_WAIT off (0.00/0/0)
tcp 32 0 horn05.gq1.ygridcore:39707 horn05.gq1.ygridcore:50010
CLOSE_WAIT off (0.00/0/0)
{code}
I was not able to nail down the root cause at that time though.
Hbase dose not closing a closed socket resulting in many CLOSE_WAIT
Key: HBASE-9393
URL: https://issues.apache.org/jira/browse/HBASE-9393
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.2
Environment: Centos 6.4 - 7 regionservers/datanodes, 8 TB per node,
7279 regions
Reporter: Avi Zrachya
HBase dose not close a dead connection with the datanode.
This resulting in over 60K CLOSE_WAIT and at some point HBase can not connect
to the datanode because too many mapped sockets from one host to another on
the same port.
The example below is with low CLOSE_WAIT count because we had to restart
hbase to solve the porblem, later in time it will incease to 60-100K sockets
on CLOSE_WAIT
[root@hd2-region3 ~]# netstat -nap |grep CLOSE_WAIT |grep 21592 |wc -l
13156
[root@hd2-region3 ~]# ps -ef |grep 21592
root 17255 17219 0 12:26 pts/000:00:00 grep 21592
hbase21592 1 17 Aug29 ?03:29:06
/usr/java/jdk1.6.0_26/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx8000m
-ea -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
-Dhbase.log.dir=/var/log/hbase
-Dhbase.log.file=hbase-hbase-regionserver-hd2-region3.swnet.corp.log ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754888#comment-13754888
]
Jimmy Xiang commented on HBASE-9387:
Agree what stack said. Since it is rare, I +1 Ted's patch, just abort. We
need to check other place in OpenRegionHander too to do the same.
As to Ted's suggestion to return special return value, it's a good idea.
However, it could cause other issues since it impacts other places where
depends on the -1 thing. I think we should not do it now when we are close to
a release.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 9387-v1.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9394) [replication] size accounting is completely off in the source
Jean-Daniel Cryans created HBASE-9394:
-
Summary: [replication] size accounting is completely off in the
source
Key: HBASE-9394
URL: https://issues.apache.org/jira/browse/HBASE-9394
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.2
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.98.0, 0.96.0
I was under the impression that I was sending way more data than I was
expecting, so adding some more tracing I can see how much data I read VS how I
much I think I'm sending:
{noformat}
Seeking at position 163771687
Replicating 2 entries of total size 2790
Seeking at position 166723643
{noformat}
That's about 2MB of data, not 2KB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7954) Fix the retrying logic of memstore flushes to avoid extra sleep
[ https://issues.apache.org/jira/browse/HBASE-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754796#comment-13754796 ] Himanshu Vashishtha commented on HBASE-7954: The first patch was for trunk, second one is for 0.94. Fix the retrying logic of memstore flushes to avoid extra sleep --- Key: HBASE-7954 URL: https://issues.apache.org/jira/browse/HBASE-7954 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.5, 0.95.0 Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Priority: Minor Attachments: HBase-7954-94.patch, HBase-7954-v0.patch Matteo pointed out: We can avoid the redundant sleep in the retrying logic. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9373) [replication] data loss because replication doesn't expect partial reads
[
https://issues.apache.org/jira/browse/HBASE-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754914#comment-13754914
]
stack commented on HBASE-9373:
--
nit: there is stuff like the below that does not have to be inside the try (not
important)
if (trailerPresent originalPosition == this.walEditsStopOffset) return false;
Remove this on commit:
+// See if available is any good to us. Record before we start
reading. My guess is that it
+// does not change once reader has been opened but check see.
...especially as my 'guess' above turns out to be wrong (its embarrassing to
have it persist in code!)
I think I preferred the old repetitive way of doing things rather than this
messaging via exceptions that you have here (throwing EOFEs only to catch them
locally)
Good to include available and size in this exception message throw new
EOFException(Available stream not enough for edit);
Could we already have an EOFE when you do this throw new EOFException(Partial
PB while reading WAL, +
+ probably an unexpected EOF, ignoring); ? If so, are you
losing info when you throw this on? Add original exception as 'cause'?
Above is minor stuff. +1 on commit since it works for you.
[replication] data loss because replication doesn't expect partial reads
Key: HBASE-9373
URL: https://issues.apache.org/jira/browse/HBASE-9373
Project: HBase
Issue Type: Improvement
Affects Versions: 0.95.2
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Blocker
Fix For: 0.98.0, 0.96.0
Attachments: 9373.txt, 9373-v2.txt, 9373-v3.txt
When I see this in the logs it often means we got a partial read and then we
have the wrong offset when reading the rest of the file
{noformat}
2013-08-28 23:16:07,182 ERROR
[ReplicationExecutor-0.replicationSource,1-jdec2hbase0403-5,60020,1377730319617]
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader: Invalid PB while
reading WAL, probably an unexpected EOF, ignoring
com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had
invalid wire type.
at
com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:99)
at
com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:498)
at
com.google.protobuf.GeneratedMessage.parseUnknownField(GeneratedMessage.java:193)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey.init(WALProtos.java:686)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey.init(WALProtos.java:644)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey$1.parsePartialFrom(WALProtos.java:771)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey$1.parsePartialFrom(WALProtos.java:766)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey$Builder.mergeFrom(WALProtos.java:1444)
at
org.apache.hadoop.hbase.protobuf.generated.WALProtos$WALKey$Builder.mergeFrom(WALProtos.java:1218)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:220)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:912)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:290)
at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:926)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:296)
at
com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:918)
at
org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.readNext(ProtobufLogReader.java:197)
at
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.next(ReaderBase.java:98)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.readNextAndSetPosition(ReplicationHLogReaderManager.java:89)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.readAllEntriesToReplicateOrNextFile(ReplicationSource.java:390)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:298)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9394) [replication] size accounting is completely off in the source
[
https://issues.apache.org/jira/browse/HBASE-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754922#comment-13754922
]
stack commented on HBASE-9394:
--
+1
[replication] size accounting is completely off in the source
-
Key: HBASE-9394
URL: https://issues.apache.org/jira/browse/HBASE-9394
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.2
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.98.0, 0.96.0
Attachments: HBASE-9394.patch
I was under the impression that I was sending way more data than I was
expecting, so adding some more tracing I can see how much data I read VS how
I much I think I'm sending:
{noformat}
Seeking at position 163771687
Replicating 2 entries of total size 2790
Seeking at position 166723643
{noformat}
That's about 2MB of data, not 2KB.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754846#comment-13754846
]
Jeffrey Zhong commented on HBASE-9387:
--
This seems a classic retry issue because when a request fails, the request
could be successfully processed at server side while the server has issue to
send response back or the client(caller) has issue to receive the response.
Then a client retries because it think the previous request didn't go through.
Then we have the above scenario.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 9387-v1.txt, hbase-9387.patch,
org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9393) Hbase dose not closing a closed socket resulting in many CLOSE_WAIT
[ https://issues.apache.org/jira/browse/HBASE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754919#comment-13754919 ] stack commented on HBASE-9393: -- Lots of random reads? Hbase dose not closing a closed socket resulting in many CLOSE_WAIT Key: HBASE-9393 URL: https://issues.apache.org/jira/browse/HBASE-9393 Project: HBase Issue Type: Bug Affects Versions: 0.94.2 Environment: Centos 6.4 - 7 regionservers/datanodes, 8 TB per node, 7279 regions Reporter: Avi Zrachya HBase dose not close a dead connection with the datanode. This resulting in over 60K CLOSE_WAIT and at some point HBase can not connect to the datanode because too many mapped sockets from one host to another on the same port. The example below is with low CLOSE_WAIT count because we had to restart hbase to solve the porblem, later in time it will incease to 60-100K sockets on CLOSE_WAIT [root@hd2-region3 ~]# netstat -nap |grep CLOSE_WAIT |grep 21592 |wc -l 13156 [root@hd2-region3 ~]# ps -ef |grep 21592 root 17255 17219 0 12:26 pts/000:00:00 grep 21592 hbase21592 1 17 Aug29 ?03:29:06 /usr/java/jdk1.6.0_26/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx8000m -ea -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -Dhbase.log.dir=/var/log/hbase -Dhbase.log.file=hbase-hbase-regionserver-hd2-region3.swnet.corp.log ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-9390) coprocessors observers are not called during a recovery with the new log replay algorithm
[ https://issues.apache.org/jira/browse/HBASE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong reassigned HBASE-9390: Assignee: Jeffrey Zhong coprocessors observers are not called during a recovery with the new log replay algorithm - Key: HBASE-9390 URL: https://issues.apache.org/jira/browse/HBASE-9390 Project: HBase Issue Type: Bug Components: Coprocessors, MTTR Affects Versions: 0.95.2 Reporter: Nicolas Liochon Assignee: Jeffrey Zhong Attachments: copro.patch See the patch to reproduce the issue: If we activate log replay we don't have the events on WAL restore. Pinging [~jeffreyz], we discussed this offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9382) replicateWALEntry doesn't use the replication handlers
[ https://issues.apache.org/jira/browse/HBASE-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754861#comment-13754861 ] Jean-Daniel Cryans commented on HBASE-9382: --- It all checks out, +1 replicateWALEntry doesn't use the replication handlers -- Key: HBASE-9382 URL: https://issues.apache.org/jira/browse/HBASE-9382 Project: HBase Issue Type: Bug Affects Versions: 0.95.2 Reporter: Jean-Daniel Cryans Assignee: stack Fix For: 0.98.0, 0.96.0 Attachments: 9382.trunk.txt, 9382.txt By default we assign 3 handlers for replication, but as far as I can tell the replication traffic uses the normal handlers in 0.96 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9387) Region could get lost during assignment
[
https://issues.apache.org/jira/browse/HBASE-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9387:
--
Attachment: 9387-v5.txt
Patch v5 checks whether znode exists.
If znode doesn't exist, abort region server.
Otherwise log warning.
Region could get lost during assignment
---
Key: HBASE-9387
URL: https://issues.apache.org/jira/browse/HBASE-9387
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.95.2
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 9387-v1.txt, 9387-v3.txt, 9387-v4.txt, 9387-v5.txt,
hbase-9387.patch, org.apache.hadoop.hbase.TestFullLogReconstruction-output.txt
I observed test timeout running against hadoop 2.1.0 with distributed log
replay turned on.
Looks like region state for 1588230740 became inconsistent between master and
the surviving region server:
{code}
2013-08-29 22:15:34,180 INFO [AM.ZK.Worker-pool2-t4]
master.RegionStates(299): Onlined 1588230740 on
kiyo.gq1.ygridcore.net,57016,1377814510039
...
2013-08-29 22:15:34,587 DEBUG [Thread-221]
client.HConnectionManager$HConnectionImplementation(1269): locateRegionInMeta
parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=kiyo.gq1.ygridcore.net,57016,1377814510039, seqNum=0}, attempt=2 of
35 failed; retrying after sleep of 302 because:
org.apache.hadoop.hbase.exceptions.RegionOpeningException: Region is being
opened: 1588230740
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2574)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:3949)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2733)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26965)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2063)
at
org.apache.hadoop.hbase.ipc.RpcServer$CallRunner.run(RpcServer.java:1800)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:165)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:41)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9386) [WINDOWS] Small improvements to .cmd scripts
[ https://issues.apache.org/jira/browse/HBASE-9386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-9386. -- Resolution: Fixed Hadoop Flags: Reviewed Committed this. Thanks for the review guys. [WINDOWS] Small improvements to .cmd scripts Key: HBASE-9386 URL: https://issues.apache.org/jira/browse/HBASE-9386 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.98.0, 0.96.0 Attachments: hbase-9386_v1.patch A collection of small improvements to the .cmd scripts are needed: - Have hbase.cmd honor --config option - Fix log4j default appenders for standalone or service mode - classpath fixes - Honor JRuby options -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9391) Compilation problem in AccessController with JDK 6
[
https://issues.apache.org/jira/browse/HBASE-9391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13754732#comment-13754732
]
Nicolas Liochon commented on HBASE-9391:
It's not this (open jdk stuff)?:
http://code.google.com/p/guava-libraries/issues/detail?id=635
+1 anyway, the patch is ok.
Compilation problem in AccessController with JDK 6
--
Key: HBASE-9391
URL: https://issues.apache.org/jira/browse/HBASE-9391
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.0, 0.95.2
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Critical
Attachments: 9391.patch
Seeing this with a fresh checkout of trunk and 0.95, only with JDK 6:
{noformat}
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[541,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1072,56]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1383,64]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to java.util.Mapbyte[],java.util.Setbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Setbyte[]
[ERROR]
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java:[1473,63]
incompatible types; no instance(s) of type variable(s) K,V exist so that
java.util.TreeMapK,V conforms to
java.util.Mapbyte[],java.util.Collectionbyte[]
[ERROR] found : K,Vjava.util.TreeMapK,V
[ERROR] required: java.util.Mapbyte[],java.util.Collectionbyte[]
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
