[jira] [Updated] (HBASE-9035) Incorrect example for using a scan stopRow in HBase book
[ https://issues.apache.org/jira/browse/HBASE-9035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabriel Reid updated HBASE-9035: Attachment: HBASE-9035.patch Patch with working example, as well as a pointer to InclusiveStopFilter for easier scanning over a specific range. Incorrect example for using a scan stopRow in HBase book Key: HBASE-9035 URL: https://issues.apache.org/jira/browse/HBASE-9035 Project: HBase Issue Type: Bug Components: documentation Reporter: Gabriel Reid Attachments: HBASE-9035.patch The example of how to use a stop row in a scan in the Section 5.7.3 of the HBase book [1] is incorrect. It demonstrates using a start and stop row to only retrieve records with a given prefix, creating the stop row by appending a null byte to the start row. This creates a scan that does not include any of the target rows, because the the stop row is less than the target rows via lexicographical sorting. [1] http://hbase.apache.org/book/data_model_operations.html#scan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9035) Incorrect example for using a scan stopRow in HBase book
Gabriel Reid created HBASE-9035: --- Summary: Incorrect example for using a scan stopRow in HBase book Key: HBASE-9035 URL: https://issues.apache.org/jira/browse/HBASE-9035 Project: HBase Issue Type: Bug Components: documentation Reporter: Gabriel Reid Attachments: HBASE-9035.patch The example of how to use a stop row in a scan in the Section 5.7.3 of the HBase book [1] is incorrect. It demonstrates using a start and stop row to only retrieve records with a given prefix, creating the stop row by appending a null byte to the start row. This creates a scan that does not include any of the target rows, because the the stop row is less than the target rows via lexicographical sorting. [1] http://hbase.apache.org/book/data_model_operations.html#scan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9035) Incorrect example for using a scan stopRow in HBase book
[ https://issues.apache.org/jira/browse/HBASE-9035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabriel Reid updated HBASE-9035: Status: Patch Available (was: Open) Incorrect example for using a scan stopRow in HBase book Key: HBASE-9035 URL: https://issues.apache.org/jira/browse/HBASE-9035 Project: HBase Issue Type: Bug Components: documentation Reporter: Gabriel Reid Attachments: HBASE-9035.patch The example of how to use a stop row in a scan in the Section 5.7.3 of the HBase book [1] is incorrect. It demonstrates using a start and stop row to only retrieve records with a given prefix, creating the stop row by appending a null byte to the start row. This creates a scan that does not include any of the target rows, because the the stop row is less than the target rows via lexicographical sorting. [1] http://hbase.apache.org/book/data_model_operations.html#scan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9033) Add tracing to ReplicationSource and enable it in tests
[
https://issues.apache.org/jira/browse/HBASE-9033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718219#comment-13718219
]
Hadoop QA commented on HBASE-9033:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593813/HBASE-9033.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6446//console
This message is automatically generated.
Add tracing to ReplicationSource and enable it in tests
---
Key: HBASE-9033
URL: https://issues.apache.org/jira/browse/HBASE-9033
Project: HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Fix For: 0.98.0, 0.95.2
Attachments: HBASE-9033.patch
We used to have a lot of logging in ReplicationSource but most of it went
away, but debugging the big replication tests is still pretty hard. I think
we should add that logging back but at the TRACE level, and enable it only
for the unit tests.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9031) ImmutableBytesWritable.toString() should downcast the bytes before converting to hex string
[
https://issues.apache.org/jira/browse/HBASE-9031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718222#comment-13718222
]
Hadoop QA commented on HBASE-9031:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593815/HBASE-9031.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestDrainingServer
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hdfs.TestModTime.testModTimePersistsAfterRestart(TestModTime.java:209)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6447//console
This message is automatically generated.
ImmutableBytesWritable.toString() should downcast the bytes before converting
to hex string
---
Key: HBASE-9031
URL: https://issues.apache.org/jira/browse/HBASE-9031
Project: HBase
Issue Type: Bug
Components: io
Affects Versions: 0.95.1, 0.94.9
Reporter: Aditya Kishore
Assignee: Aditya Kishore
Priority: Minor
Fix For: 0.95.2
Attachments: HBASE-9031.patch, HBASE-9031.patch
The attached patch addresses few issues.
# We need only (3*this.length) capacity in ByteBuffer and not
(3*this.bytes.length).
# Do not calculate (offset + length) at every iteration.
# No test is required at every iteration to add space (' ') before every byte
other than the first one. Uses {{sb.substring(1)}} instead.
# Finally and most importantly (the original issue of this report), downcast
the promoted int (the parameter to {{Integer.toHexString()}}) to byte range.
Without #4, the byte array \{54,125,64, -1, -45\} is transformed to 36 7d 40
ffd3 instead of 36 7d 40 ff d3.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9035) Incorrect example for using a scan stopRow in HBase book
[ https://issues.apache.org/jira/browse/HBASE-9035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718250#comment-13718250 ] Jean-Marc Spaggiari commented on HBASE-9035: I agree. However, I will not say that it's generally, but I will more than that it's an easy and simple way to achieve the same thing, since some people will prefer to reduce the number of objects created and so avoid to use this filter but add a #FF at the end of the stop row instead. Incorrect example for using a scan stopRow in HBase book Key: HBASE-9035 URL: https://issues.apache.org/jira/browse/HBASE-9035 Project: HBase Issue Type: Bug Components: documentation Reporter: Gabriel Reid Attachments: HBASE-9035.patch The example of how to use a stop row in a scan in the Section 5.7.3 of the HBase book [1] is incorrect. It demonstrates using a start and stop row to only retrieve records with a given prefix, creating the stop row by appending a null byte to the start row. This creates a scan that does not include any of the target rows, because the the stop row is less than the target rows via lexicographical sorting. [1] http://hbase.apache.org/book/data_model_operations.html#scan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput
[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718252#comment-13718252 ] Jean-Marc Spaggiari commented on HBASE-8755: Hi [~fenghh], I was in a process to give it a try, is there a new version for 0.94 coming? Or the one attached in the JIRA is already the good one? A new write thread model for HLog to improve the overall HBase write throughput --- Key: HBASE-8755 URL: https://issues.apache.org/jira/browse/HBASE-8755 Project: HBase Issue Type: Improvement Components: wal Reporter: Feng Honghua Attachments: HBASE-8755-0.94-V0.patch, HBASE-8755-0.94-V1.patch, HBASE-8755-trunk-V0.patch, HBASE-8755-trunk-V1.patch In current write model, each write handler thread (executing put()) will individually go through a full 'append (hlog local buffer) = HLog writer append (write to hdfs) = HLog writer sync (sync hdfs)' cycle for each write, which incurs heavy race condition on updateLock and flushLock. The only optimization where checking if current syncTillHere txid in expectation for other thread help write/sync its own txid to hdfs and omitting the write/sync actually help much less than expectation. Three of my colleagues(Ye Hangjun / Wu Zesheng / Zhang Peng) at Xiaomi proposed a new write thread model for writing hdfs sequence file and the prototype implementation shows a 4X improvement for throughput (from 17000 to 7+). I apply this new write thread model in HLog and the performance test in our test cluster shows about 3X throughput improvement (from 12150 to 31520 for 1 RS, from 22000 to 7 for 5 RS), the 1 RS write throughput (1K row-size) even beats the one of BigTable (Precolator published in 2011 says Bigtable's write throughput then is 31002). I can provide the detailed performance test results if anyone is interested. The change for new write thread model is as below: 1 All put handler threads append the edits to HLog's local pending buffer; (it notifies AsyncWriter thread that there is new edits in local buffer) 2 All put handler threads wait in HLog.syncer() function for underlying threads to finish the sync that contains its txid; 3 An single AsyncWriter thread is responsible for retrieve all the buffered edits in HLog's local pending buffer and write to the hdfs (hlog.writer.append); (it notifies AsyncFlusher thread that there is new writes to hdfs that needs a sync) 4 An single AsyncFlusher thread is responsible for issuing a sync to hdfs to persist the writes by AsyncWriter; (it notifies the AsyncNotifier thread that sync watermark increases) 5 An single AsyncNotifier thread is responsible for notifying all pending put handler threads which are waiting in the HLog.syncer() function 6 No LogSyncer thread any more (since there is always AsyncWriter/AsyncFlusher threads do the same job it does) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9035) Incorrect example for using a scan stopRow in HBase book
[ https://issues.apache.org/jira/browse/HBASE-9035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718262#comment-13718262 ] Gabriel Reid commented on HBASE-9035: - If I'm not mistaken, adding a #FF still isn't sufficient to do what's given as the example in the book. The example specifies that it will return all rows whose row key begins with row. Setting the stop row to row\xFF will still exclude all rows that begin with row\xFF. Incorrect example for using a scan stopRow in HBase book Key: HBASE-9035 URL: https://issues.apache.org/jira/browse/HBASE-9035 Project: HBase Issue Type: Bug Components: documentation Reporter: Gabriel Reid Attachments: HBASE-9035.patch The example of how to use a stop row in a scan in the Section 5.7.3 of the HBase book [1] is incorrect. It demonstrates using a start and stop row to only retrieve records with a given prefix, creating the stop row by appending a null byte to the start row. This creates a scan that does not include any of the target rows, because the the stop row is less than the target rows via lexicographical sorting. [1] http://hbase.apache.org/book/data_model_operations.html#scan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8947) Thrift 2 : Replace bool writeToWAL with TDurability durability
[ https://issues.apache.org/jira/browse/HBASE-8947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718264#comment-13718264 ] Lars George commented on HBASE-8947: Hi [~madani], I am not talking about the client adopting the new features, they need to do that eventually. But for a rolling upgrade, you can have an older client talking to a newer server, or vice versa. See [this|http://www.quora.com/Are-different-versions-of-Thrift-compatible-with-each-other-on-the-wire-RPC-level] for reference. Similar to [ProtoBufs|https://developers.google.com/protocol-buffers/docs/proto] (see Assigning Tags) the IDs are what is used during serialization and deserialization to match values to fields. If you change this, then you mix up assignment. As for ordering, if you look at the Thrift example, there is a test including a Backwards test, which assigns the IDs out of order. I had assumed this is arbitrary (i.e. the ordering) as long as you match IDs to fields and not change them across versions. And you should be able to have gaps, since you can that way retire older fields - the deserialization will simply ignore them. I just wish Thrift had a proper documentation. :( Thrift 2 : Replace bool writeToWAL with TDurability durability --- Key: HBASE-8947 URL: https://issues.apache.org/jira/browse/HBASE-8947 Project: HBase Issue Type: Sub-task Components: Thrift Reporter: Hamed Madani Assignee: Hamed Madani Attachments: HBASE-8947.patch, HBASE-8947-v2.patch, HBASE-8947-v3-0.94.patch, HBASE-8947-v3.patch Introduce new enum *TDurability* to expose more options for *Write To Wal.* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8755) A new write thread model for HLog to improve the overall HBase write throughput
[ https://issues.apache.org/jira/browse/HBASE-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718270#comment-13718270 ] Feng Honghua commented on HBASE-8755: - [~jmspaggi] HBASE-8755-0.94-V1.patch is good. Let me know if any problem. Thanks Jean-Marc A new write thread model for HLog to improve the overall HBase write throughput --- Key: HBASE-8755 URL: https://issues.apache.org/jira/browse/HBASE-8755 Project: HBase Issue Type: Improvement Components: wal Reporter: Feng Honghua Attachments: HBASE-8755-0.94-V0.patch, HBASE-8755-0.94-V1.patch, HBASE-8755-trunk-V0.patch, HBASE-8755-trunk-V1.patch In current write model, each write handler thread (executing put()) will individually go through a full 'append (hlog local buffer) = HLog writer append (write to hdfs) = HLog writer sync (sync hdfs)' cycle for each write, which incurs heavy race condition on updateLock and flushLock. The only optimization where checking if current syncTillHere txid in expectation for other thread help write/sync its own txid to hdfs and omitting the write/sync actually help much less than expectation. Three of my colleagues(Ye Hangjun / Wu Zesheng / Zhang Peng) at Xiaomi proposed a new write thread model for writing hdfs sequence file and the prototype implementation shows a 4X improvement for throughput (from 17000 to 7+). I apply this new write thread model in HLog and the performance test in our test cluster shows about 3X throughput improvement (from 12150 to 31520 for 1 RS, from 22000 to 7 for 5 RS), the 1 RS write throughput (1K row-size) even beats the one of BigTable (Precolator published in 2011 says Bigtable's write throughput then is 31002). I can provide the detailed performance test results if anyone is interested. The change for new write thread model is as below: 1 All put handler threads append the edits to HLog's local pending buffer; (it notifies AsyncWriter thread that there is new edits in local buffer) 2 All put handler threads wait in HLog.syncer() function for underlying threads to finish the sync that contains its txid; 3 An single AsyncWriter thread is responsible for retrieve all the buffered edits in HLog's local pending buffer and write to the hdfs (hlog.writer.append); (it notifies AsyncFlusher thread that there is new writes to hdfs that needs a sync) 4 An single AsyncFlusher thread is responsible for issuing a sync to hdfs to persist the writes by AsyncWriter; (it notifies the AsyncNotifier thread that sync watermark increases) 5 An single AsyncNotifier thread is responsible for notifying all pending put handler threads which are waiting in the HLog.syncer() function 6 No LogSyncer thread any more (since there is always AsyncWriter/AsyncFlusher threads do the same job it does) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8764) Some MasterMonitorCallable should retry
[
https://issues.apache.org/jira/browse/HBASE-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718275#comment-13718275
]
Hadoop QA commented on HBASE-8764:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593799/8764v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 9 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
14 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.wal.TestHLog
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6448//console
This message is automatically generated.
Some MasterMonitorCallable should retry
---
Key: HBASE-8764
URL: https://issues.apache.org/jira/browse/HBASE-8764
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.95.1
Reporter: Elliott Clark
Assignee: stack
Fix For: 0.95.2
Attachments: 8764.txt, 8764v2.txt
Calls in the admin that only get status should re-try.
got a call stack like:
{code}
org.apache.hadoop.hbase.exceptions.PleaseHoldException:
org.apache.hadoop.hbase.exceptions.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2266)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1610)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1646)
at
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:20930)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2122)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1829)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:230)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2705)
at
[jira] [Commented] (HBASE-9035) Incorrect example for using a scan stopRow in HBase book
[
https://issues.apache.org/jira/browse/HBASE-9035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718277#comment-13718277
]
Hadoop QA commented on HBASE-9035:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593906/HBASE-9035.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6449//console
This message is automatically generated.
Incorrect example for using a scan stopRow in HBase book
Key: HBASE-9035
URL: https://issues.apache.org/jira/browse/HBASE-9035
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Gabriel Reid
Attachments: HBASE-9035.patch
The example of how to use a stop row in a scan in the Section 5.7.3 of the
HBase book [1] is incorrect. It demonstrates using a start and stop row to
only retrieve records with a given prefix, creating the stop row by appending
a null byte to the start row.
This creates a scan that does not include any of the target rows, because the
the stop row is less than the target rows via lexicographical sorting.
[1] http://hbase.apache.org/book/data_model_operations.html#scan
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9035) Incorrect example for using a scan stopRow in HBase book
[ https://issues.apache.org/jira/browse/HBASE-9035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718278#comment-13718278 ] Jean-Marc Spaggiari commented on HBASE-9035: You're totally right. This was with the assumption that it was only printable characters only on the key. I should have specified that. Incorrect example for using a scan stopRow in HBase book Key: HBASE-9035 URL: https://issues.apache.org/jira/browse/HBASE-9035 Project: HBase Issue Type: Bug Components: documentation Reporter: Gabriel Reid Attachments: HBASE-9035.patch The example of how to use a stop row in a scan in the Section 5.7.3 of the HBase book [1] is incorrect. It demonstrates using a start and stop row to only retrieve records with a given prefix, creating the stop row by appending a null byte to the start row. This creates a scan that does not include any of the target rows, because the the stop row is less than the target rows via lexicographical sorting. [1] http://hbase.apache.org/book/data_model_operations.html#scan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8947) Thrift 2 : Replace bool writeToWAL with TDurability durability
[
https://issues.apache.org/jira/browse/HBASE-8947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718287#comment-13718287
]
Lars George commented on HBASE-8947:
For reference from
[online|http://svn.apache.org/repos/asf/thrift/attic/trunk/doc/thrift.tex]
material:
{quote}
Structs
...
Field identifiers will be automatically assigned if omitted, though they are
strongly encouraged for versioning reasons discussed later.
...
Versioning
...
Thrift is robust in the face of versioning and data definition changes. This is
critical to enable staged rollouts of changes to deployed services.
...
Field Identifiers
...
To avoid conflicts between manually and automatically assigned identifiers,
fields with identifiers omitted are assigned identifiers decrementing from -1,
and the language only supports the manual assignment of positive identifiers.
...
If a field identifier is not recognized, the generated code can use the type
specifier to skip the unknown field without any error.
{quote}
So there is no notion of ordering, but states the fact that there may be gaps
because of older to newer definitions.
Thrift 2 : Replace bool writeToWAL with TDurability durability
---
Key: HBASE-8947
URL: https://issues.apache.org/jira/browse/HBASE-8947
Project: HBase
Issue Type: Sub-task
Components: Thrift
Reporter: Hamed Madani
Assignee: Hamed Madani
Attachments: HBASE-8947.patch, HBASE-8947-v2.patch,
HBASE-8947-v3-0.94.patch, HBASE-8947-v3.patch
Introduce new enum *TDurability* to expose more options for *Write To Wal.*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8015) Support for Namespaces
[ https://issues.apache.org/jira/browse/HBASE-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718323#comment-13718323 ] Francis Liu commented on HBASE-8015: I've pushed some changes into github addressing the following: - Change delimiter to ':' - Use a filesystem delimiter ',' instead of ':' where a fqtn is referenced as part of a file/dir name - changed filesystem table path to: root/data/ns/table qualifier - updated namespace and table name valid chars. namespace now only allows [A-Za-z0-9_] - rename FullyQualifiedTableName to TableName since all table names are fully qualified not unless explicitly stated Some issues: - found out that ':' is not a valid character in HDFS (scratches head). so we'll really need an fs delimiter if we stick with this delimiter. - snapshot names character names - are all valid tables names valid snapshot names as well? Presently snapshot creation will complain if it has a ':' in it - small and medium tests pass, didn't have time to run large yet What do you guys think? I'll continue working on the other comments. Check out the changes here: https://github.com/francisliu/hbase_namespace/tree/core_8408_rebase_1 Support for Namespaces -- Key: HBASE-8015 URL: https://issues.apache.org/jira/browse/HBASE-8015 Project: HBase Issue Type: New Feature Reporter: Francis Liu Assignee: Francis Liu Attachments: HBASE-8015_draft_94.patch, Namespace Design.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8015) Support for Namespaces
[
https://issues.apache.org/jira/browse/HBASE-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718330#comment-13718330
]
Francis Liu commented on HBASE-8015:
{quote}
A possible follow up might be to hardcode .META. naming for backwards
compatibility for 0.96 release? (scan '.META.' does not work right now)
{quote}
Is meta public?
Support for Namespaces
--
Key: HBASE-8015
URL: https://issues.apache.org/jira/browse/HBASE-8015
Project: HBase
Issue Type: New Feature
Reporter: Francis Liu
Assignee: Francis Liu
Attachments: HBASE-8015_draft_94.patch, Namespace Design.pdf
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6143: -- Attachment: 6143-v2.txt How about patch v2 ? Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 6143-v1.txt, 6143-v2.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6143: -- Attachment: (was: 6143-v2.txt) Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 6143-v1.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718436#comment-13718436 ] Ted Yu commented on HBASE-6143: --- Note on patch v2: EnableTableHandler.retainAssignment was improperly named. I renamed it this.skipTableStateCheck to be consistent with the name of parameter passed to ctor. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 6143-v1.txt, 6143-v2.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6143: -- Attachment: 6143-v2.txt Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 6143-v1.txt, 6143-v2.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-6143: -- Assignee: Ted Yu (was: Elliott Clark) Status: Patch Available (was: Open) Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Attachments: 6143-v1.txt, 6143-v2.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8974) graceful-restart.sh restarts all active RS's with each iteration instead of one at a time
[
https://issues.apache.org/jira/browse/HBASE-8974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718443#comment-13718443
]
Jean-Marc Spaggiari commented on HBASE-8974:
Hi [~ndimiduk],
I'm not able to find graceful-restart.sh. Is that into another file?
graceful-restart.sh restarts all active RS's with each iteration instead of
one at a time
-
Key: HBASE-8974
URL: https://issues.apache.org/jira/browse/HBASE-8974
Project: HBase
Issue Type: Bug
Components: scripts
Reporter: Nick Dimiduk
I'm exercising the patch over on HBASE-8803 and I've noticed something in the
logs: it looks like {{graceful-restart.sh}} is restarting all the region
servers multiple times instead of just the current entry in the loop
iteration.
The logic looks like this:
{noformat}
for each rs in active region server list:
unload $rs // move all regions to other RS's
restart all Region Servers // !?! bug?
reload $rs // pile 'em back on
{noformat}
Shouldn't that step 2 be only {{restart $rs}}?
This is what I see in the logs. My cluster has 9 active RegionServers. Notice
the bit in the middle where all 9 are stopped and started again after
unloading the target RS.
{noformat}
$ time /usr/lib/hbase/bin/rolling-restart.sh --rs-only --graceful
--maxthreads 30
Gracefully restarting: hor18n39.gq1.ygridcore.net
Disabling balancer!
...
Unloading hor18n39.gq1.ygridcore.net region(s)
...
Valid region move targets:
hor18n37.gq1.ygridcore.net,60020,1374094975268
hor17n37.gq1.ygridcore.net,60020,1374094975264
hor18n35.gq1.ygridcore.net,60020,1374094975327
hor17n39.gq1.ygridcore.net,60020,1374094975281
hor18n36.gq1.ygridcore.net,60020,1374094975254
hor17n36.gq1.ygridcore.net,60020,1374094975277
hor17n34.gq1.ygridcore.net,60020,1374094975291
hor18n38.gq1.ygridcore.net,60020,1374094975259
13/07/17 21:44:38 INFO region_mover: Moving 330 region(s) from
hor18n39.gq1.ygridcore.net,60020,1374094975326 during this cycle
13/07/17 21:44:38 INFO region_mover: Moving region
b59050cf97aabcef838e3c50e93e6d13 (1 of 330) to
server=hor18n37.gq1.ygridcore.net,60020,1374094975268
...
13/07/17 21:54:20 INFO region_mover: Moving region
d00026d7cc396bb3e6ea91106cc6ab55 (329 of 330) to
server=hor18n37.gq1.ygridcore.net,60020,1374094975268
13/07/17 21:54:20 INFO region_mover: Moving region
a722179b33e6ece8c9cee3fba3056acd (330 of 330) to
server=hor17n37.gq1.ygridcore.net,60020,1374094975264
13/07/17 21:54:21 INFO region_mover: Wrote list of moved regions to
/tmp/hor18n39.gq1.ygridcore.net
Unloaded hor18n39.gq1.ygridcore.net region(s)
hor18n35.gq1.ygridcore.net: stopping regionserver.
hor17n39.gq1.ygridcore.net: stopping regionserver.
hor18n36.gq1.ygridcore.net: stopping regionserver.
hor17n37.gq1.ygridcore.net: stopping regionserver.
hor17n34.gq1.ygridcore.net: stopping regionserver.
hor18n38.gq1.ygridcore.net: stopping regionserver.
hor18n37.gq1.ygridcore.net: stopping regionserver.
hor17n36.gq1.ygridcore.net: stopping regionserver.
hor18n39.gq1.ygridcore.net: stopping regionserver.
hor18n36.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n36.gq1.ygridcore.net.out
hor17n36.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor17n36.gq1.ygridcore.net.out
hor17n37.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor17n37.gq1.ygridcore.net.out
hor18n37.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n37.gq1.ygridcore.net.out
hor18n38.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n38.gq1.ygridcore.net.out
hor17n34.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor17n34.gq1.ygridcore.net.out
hor18n35.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n35.gq1.ygridcore.net.out
hor18n39.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n39.gq1.ygridcore.net.out
hor17n39.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor17n39.gq1.ygridcore.net.out
Reloading hor18n39.gq1.ygridcore.net region(s)
...
13/07/17 21:54:27 INFO region_mover: Moving 330 regions to
hor18n39.gq1.ygridcore.net,60020,1374098064602
13/07/17 21:56:47 INFO region_mover: Moving region
7d0a02f452c334a12026b45346a87d36 (1 of 330) to
[jira] [Commented] (HBASE-8973) Some improvements in the RowCounter MR job
[ https://issues.apache.org/jira/browse/HBASE-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718452#comment-13718452 ] Jean-Marc Spaggiari commented on HBASE-8973: Hi [~devaraj], are you still doing any work on this? Or can it be closed? Some improvements in the RowCounter MR job -- Key: HBASE-8973 URL: https://issues.apache.org/jira/browse/HBASE-8973 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Attachments: 8973-1.txt While trying to test some things, ended up modifying the rowcounter job a little. Proposing a patch with these changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9034) hbase-daemon.sh swallows start up scripts.
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-9034: - Fix Version/s: 0.95.2 0.98.0 Affects Version/s: 0.98.0 0.95.1 Status: Patch Available (was: Open) hbase-daemon.sh swallows start up scripts. -- Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Affects Versions: 0.95.1, 0.98.0 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9034) hbase-daemon.sh swallows start up scripts.
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-9034: - Component/s: scripts hbase-daemon.sh swallows start up scripts. -- Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9036) Few small code cleanup
Jean-Marc Spaggiari created HBASE-9036: -- Summary: Few small code cleanup Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9036: --- Status: Patch Available (was: Open) Modifications applied, trigger Hadoop QA. Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9036: --- Attachment: HBASE-9036-v0-trunk.patch Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718520#comment-13718520
]
Hadoop QA commented on HBASE-6143:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593947/6143-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestIOFencing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//console
This message is automatically generated.
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Attachments: 6143-v1.txt, 6143-v2.txt
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718521#comment-13718521 ] Ted Yu commented on HBASE-9036: --- Nice findings. For ClassFinder.java, can you add code to close jarFile at the end of findClassesFromJar() ? Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9036: --- Attachment: HBASE-9036-v1-trunk.patch Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9036: --- Status: Open (was: Patch Available) Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9036: --- Status: Patch Available (was: Open) I have added the jarFile close in a finally statement because some exceptions might be throw in the while (true) loop. Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9034) hbase-daemon.sh swallows start up scripts.
[
https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718548#comment-13718548
]
Hadoop QA commented on HBASE-9034:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593955/HBASE-9034-0.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6451//console
This message is automatically generated.
hbase-daemon.sh swallows start up scripts.
--
Key: HBASE-9034
URL: https://issues.apache.org/jira/browse/HBASE-9034
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 0.98.0, 0.95.1
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 0.98.0, 0.95.2
Attachments: HBASE-9034-0.patch
Fix it so that start up errors go into .out.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9036) Few small code cleanup
[
https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718590#comment-13718590
]
Hadoop QA commented on HBASE-9036:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593966/HBASE-9036-v0-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6452//console
This message is automatically generated.
Few small code cleanup
--
Key: HBASE-9036
URL: https://issues.apache.org/jira/browse/HBASE-9036
Project: HBase
Issue Type: Bug
Reporter: Jean-Marc Spaggiari
Assignee: Jean-Marc Spaggiari
Priority: Minor
Fix For: 0.98.0
Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch
Few code cleanup from HBase trunk.
1) TestOperation use String.format with 6 %s but give 7 parameters.
Resolution: Trivial
2) ClassFinder can throw a NPE.
If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an
exception and we want to proceed on exceptions, jarFile will be null, and
just few lines after we will do a jarFile.getNextJarEntry() where NPE is not
catch and will fail and throw an NPE. So I thinkg we can't proceed on
exceptions for this first try since it will fail just the after with an NPE
and we will loose the information about the real cause of the exception.
Therefor, we should always throw ioEx is the InputStream creation fails.
3)AccessController declare cfs but never use it.
4) FavoredNodeAssignmentHelper invokes toString on an array.
Just changed that to Bytes.toString() to print the server name.
5) ModifyTableHandler invokes toString on the tableName array.
Just changed that to Bytes.toString() to print the table name.
6) HFileWriterV2 invokes toString on the keys arrays.
Just changed that to Bytes.toStringBinary() to print the keys. And change
some toString() calls to toStringBinary()
7) ServerAndLoad want to be serializable, but ServerName is not.
Made ServerName serializable since it's only Strings, numbers and bytes.
8) StorageClusterStatusModel want to be serializable, but its nested class
Node is not.
Made Node serializable since it's only numbers and bytes.
9) In HRegion outResults can't be null since it's already used for
outResults.isEmpty() few lines above.
Just remove the test.
10) In RegionScannerHolder region
[jira] [Commented] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718605#comment-13718605 ] Sergey Shelukhin commented on HBASE-9036: - The patch adds tabs, otherwise +1. One could conceivably work around the file open error, I guess we can do it later if needed Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718614#comment-13718614 ] Ted Yu commented on HBASE-9036: --- @Jean-Marc: If you upload your patch to review board, the tabs would be visible. Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9034) hbase-daemon.sh swallows start up scripts.
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718613#comment-13718613 ] stack commented on HBASE-9034: -- +1 Excellent. hbase-daemon.sh swallows start up scripts. -- Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9036: --- Status: Open (was: Patch Available) Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718618#comment-13718618 ] Jean-Marc Spaggiari commented on HBASE-9036: I will fix the tab issue. I display all characters in Eclipse, so I should have catch that. [~sershe] I'm not sure to get your comment about the file open error. Can you please specify? Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9033) Add tracing to ReplicationSource and enable it in tests
[ https://issues.apache.org/jira/browse/HBASE-9033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9033: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk and 0.95. Thanks [~jdcryans] Add tracing to ReplicationSource and enable it in tests --- Key: HBASE-9033 URL: https://issues.apache.org/jira/browse/HBASE-9033 Project: HBase Issue Type: Improvement Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9033.patch We used to have a lot of logging in ReplicationSource but most of it went away, but debugging the big replication tests is still pretty hard. I think we should add that logging back but at the TRACE level, and enable it only for the unit tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9036) Few small code cleanup
[
https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718619#comment-13718619
]
Hadoop QA commented on HBASE-9036:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593973/HBASE-9036-v1-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6453//console
This message is automatically generated.
Few small code cleanup
--
Key: HBASE-9036
URL: https://issues.apache.org/jira/browse/HBASE-9036
Project: HBase
Issue Type: Bug
Reporter: Jean-Marc Spaggiari
Assignee: Jean-Marc Spaggiari
Priority: Minor
Fix For: 0.98.0
Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch
Few code cleanup from HBase trunk.
1) TestOperation use String.format with 6 %s but give 7 parameters.
Resolution: Trivial
2) ClassFinder can throw a NPE.
If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an
exception and we want to proceed on exceptions, jarFile will be null, and
just few lines after we will do a jarFile.getNextJarEntry() where NPE is not
catch and will fail and throw an NPE. So I thinkg we can't proceed on
exceptions for this first try since it will fail just the after with an NPE
and we will loose the information about the real cause of the exception.
Therefor, we should always throw ioEx is the InputStream creation fails.
3)AccessController declare cfs but never use it.
4) FavoredNodeAssignmentHelper invokes toString on an array.
Just changed that to Bytes.toString() to print the server name.
5) ModifyTableHandler invokes toString on the tableName array.
Just changed that to Bytes.toString() to print the table name.
6) HFileWriterV2 invokes toString on the keys arrays.
Just changed that to Bytes.toStringBinary() to print the keys. And change
some toString() calls to toStringBinary()
7) ServerAndLoad want to be serializable, but ServerName is not.
Made ServerName serializable since it's only Strings, numbers and bytes.
8) StorageClusterStatusModel want to be serializable, but its nested class
Node is not.
Made Node serializable since it's only numbers and bytes.
9) In HRegion outResults can't be null since it's already used for
outResults.isEmpty() few lines above.
Just remove the test.
10) In RegionScannerHolder region
[jira] [Commented] (HBASE-9032) Result.getBytes() returns null if backed by KeyValue array
[ https://issues.apache.org/jira/browse/HBASE-9032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718622#comment-13718622 ] stack commented on HBASE-9032: -- +1 for 0.94 Result.getBytes() returns null if backed by KeyValue array -- Key: HBASE-9032 URL: https://issues.apache.org/jira/browse/HBASE-9032 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.94.9 Reporter: Aditya Kishore Assignee: Aditya Kishore Fix For: 0.94.11 Attachments: HBASE-9032.patch, HBASE-9032.patch This applies only to 0.94 (and earlier) branch. If the Result object was constructed using either of Result(KeyValue[]) or Result(ListKeyValue), calling Result.getBytes() returns null instead of the serialized ImmutableBytesWritable object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9036: --- Status: Patch Available (was: Open) New version without tabs. I did a big cleanup also in StorageClusterStatusModel which was already full of tabs all over the place. Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch, HBASE-9036-v2-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Marc Spaggiari updated HBASE-9036: --- Attachment: HBASE-9036-v2-trunk.patch Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch, HBASE-9036-v2-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8974) bin/rolling-restart.sh restarts all active RS's with each iteration instead of one at a time
[
https://issues.apache.org/jira/browse/HBASE-8974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-8974:
Description:
I'm exercising the patch over on HBASE-8803 and I've noticed something in the
logs: it looks like {{rolling-restart.sh}} is restarting all the region servers
multiple times instead of just the current entry in the loop iteration.
The logic looks like this:
{noformat}
for each rs in active region server list:
unload $rs // move all regions to other RS's
restart all Region Servers // !?! bug?
reload $rs // pile 'em back on
{noformat}
Shouldn't that step 2 be only {{restart $rs}}?
This is what I see in the logs. My cluster has 9 active RegionServers. Notice
the bit in the middle where all 9 are stopped and started again after unloading
the target RS.
{noformat}
$ time /usr/lib/hbase/bin/rolling-restart.sh --rs-only --graceful --maxthreads
30
Gracefully restarting: hor18n39.gq1.ygridcore.net
Disabling balancer!
...
Unloading hor18n39.gq1.ygridcore.net region(s)
...
Valid region move targets:
hor18n37.gq1.ygridcore.net,60020,1374094975268
hor17n37.gq1.ygridcore.net,60020,1374094975264
hor18n35.gq1.ygridcore.net,60020,1374094975327
hor17n39.gq1.ygridcore.net,60020,1374094975281
hor18n36.gq1.ygridcore.net,60020,1374094975254
hor17n36.gq1.ygridcore.net,60020,1374094975277
hor17n34.gq1.ygridcore.net,60020,1374094975291
hor18n38.gq1.ygridcore.net,60020,1374094975259
13/07/17 21:44:38 INFO region_mover: Moving 330 region(s) from
hor18n39.gq1.ygridcore.net,60020,1374094975326 during this cycle
13/07/17 21:44:38 INFO region_mover: Moving region
b59050cf97aabcef838e3c50e93e6d13 (1 of 330) to
server=hor18n37.gq1.ygridcore.net,60020,1374094975268
...
13/07/17 21:54:20 INFO region_mover: Moving region
d00026d7cc396bb3e6ea91106cc6ab55 (329 of 330) to
server=hor18n37.gq1.ygridcore.net,60020,1374094975268
13/07/17 21:54:20 INFO region_mover: Moving region
a722179b33e6ece8c9cee3fba3056acd (330 of 330) to
server=hor17n37.gq1.ygridcore.net,60020,1374094975264
13/07/17 21:54:21 INFO region_mover: Wrote list of moved regions to
/tmp/hor18n39.gq1.ygridcore.net
Unloaded hor18n39.gq1.ygridcore.net region(s)
hor18n35.gq1.ygridcore.net: stopping regionserver.
hor17n39.gq1.ygridcore.net: stopping regionserver.
hor18n36.gq1.ygridcore.net: stopping regionserver.
hor17n37.gq1.ygridcore.net: stopping regionserver.
hor17n34.gq1.ygridcore.net: stopping regionserver.
hor18n38.gq1.ygridcore.net: stopping regionserver.
hor18n37.gq1.ygridcore.net: stopping regionserver.
hor17n36.gq1.ygridcore.net: stopping regionserver.
hor18n39.gq1.ygridcore.net: stopping regionserver.
hor18n36.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n36.gq1.ygridcore.net.out
hor17n36.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor17n36.gq1.ygridcore.net.out
hor17n37.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor17n37.gq1.ygridcore.net.out
hor18n37.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n37.gq1.ygridcore.net.out
hor18n38.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n38.gq1.ygridcore.net.out
hor17n34.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor17n34.gq1.ygridcore.net.out
hor18n35.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n35.gq1.ygridcore.net.out
hor18n39.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor18n39.gq1.ygridcore.net.out
hor17n39.gq1.ygridcore.net: starting regionserver, logging to
/grid/0/var/log/hbase/hbase-hbase-regionserver-hor17n39.gq1.ygridcore.net.out
Reloading hor18n39.gq1.ygridcore.net region(s)
...
13/07/17 21:54:27 INFO region_mover: Moving 330 regions to
hor18n39.gq1.ygridcore.net,60020,1374098064602
13/07/17 21:56:47 INFO region_mover: Moving region
7d0a02f452c334a12026b45346a87d36 (1 of 330) to
server=hor18n39.gq1.ygridcore.net,60020,1374098064602 in thread 0
13/07/17 21:56:54 INFO region_mover: Moving region
af5448c90e78a8f0d935efb0b380502e (2 of 330) to
server=hor18n39.gq1.ygridcore.net,60020,1374098064602 in thread 1
...
{noformat}
was:
I'm exercising the patch over on HBASE-8803 and I've noticed something in the
logs: it looks like {{graceful-restart.sh}} is restarting all the region
servers multiple times instead of just the current entry in the loop iteration.
The logic looks like this:
{noformat}
for each rs in active region server list:
unload $rs // move all regions to other RS's
restart all Region Servers // !?!
[jira] [Updated] (HBASE-9013) Backport HBASE-8994 (Adding log to chaos monkey actions to show what're performed) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-9013: -- Issue Type: Improvement (was: Bug) Backport HBASE-8994 (Adding log to chaos monkey actions to show what're performed) to 0.94 -- Key: HBASE-9013 URL: https://issues.apache.org/jira/browse/HBASE-9013 Project: HBase Issue Type: Improvement Affects Versions: 0.94.11 Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Attachments: 9013.patch Keep the debugability of the 0.94 chaos monkey in line with 0.95/trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9036) Few small code cleanup
[
https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718708#comment-13718708
]
Hadoop QA commented on HBASE-9036:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593984/HBASE-9036-v2-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6454//console
This message is automatically generated.
Few small code cleanup
--
Key: HBASE-9036
URL: https://issues.apache.org/jira/browse/HBASE-9036
Project: HBase
Issue Type: Bug
Reporter: Jean-Marc Spaggiari
Assignee: Jean-Marc Spaggiari
Priority: Minor
Fix For: 0.98.0
Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch,
HBASE-9036-v2-trunk.patch
Few code cleanup from HBase trunk.
1) TestOperation use String.format with 6 %s but give 7 parameters.
Resolution: Trivial
2) ClassFinder can throw a NPE.
If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an
exception and we want to proceed on exceptions, jarFile will be null, and
just few lines after we will do a jarFile.getNextJarEntry() where NPE is not
catch and will fail and throw an NPE. So I thinkg we can't proceed on
exceptions for this first try since it will fail just the after with an NPE
and we will loose the information about the real cause of the exception.
Therefor, we should always throw ioEx is the InputStream creation fails.
3)AccessController declare cfs but never use it.
4) FavoredNodeAssignmentHelper invokes toString on an array.
Just changed that to Bytes.toString() to print the server name.
5) ModifyTableHandler invokes toString on the tableName array.
Just changed that to Bytes.toString() to print the table name.
6) HFileWriterV2 invokes toString on the keys arrays.
Just changed that to Bytes.toStringBinary() to print the keys. And change
some toString() calls to toStringBinary()
7) ServerAndLoad want to be serializable, but ServerName is not.
Made ServerName serializable since it's only Strings, numbers and bytes.
8) StorageClusterStatusModel want to be serializable, but its nested class
Node is not.
Made Node serializable since it's only numbers and bytes.
9) In HRegion outResults can't be null since it's already used for
outResults.isEmpty() few lines above.
Just remove the test.
10)
[jira] [Commented] (HBASE-8015) Support for Namespaces
[
https://issues.apache.org/jira/browse/HBASE-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718718#comment-13718718
]
Jesse Yates commented on HBASE-8015:
{quote}
snapshot names character names - are all valid tables names valid snapshot
names as well? Presently snapshot creation will complain if it has a ':' in it
{quote}
It used to follow table name semantics, but looks like it just tries on the FS
and fails if its not a valid character.
Support for Namespaces
--
Key: HBASE-8015
URL: https://issues.apache.org/jira/browse/HBASE-8015
Project: HBase
Issue Type: New Feature
Reporter: Francis Liu
Assignee: Francis Liu
Attachments: HBASE-8015_draft_94.patch, Namespace Design.pdf
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718735#comment-13718735 ] Sergey Shelukhin commented on HBASE-9036: - +1 Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch, HBASE-9036-v2-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718747#comment-13718747 ] Ted Yu commented on HBASE-9036: --- Integrated to trunk. Thanks for the patch, Jean-Marc. Thanks for the review, Sergey. Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch, HBASE-9036-v2-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8935) IntegrationTestBigLinkedList fails under load on 0.94 due to some scan issues - add logging
[
https://issues.apache.org/jira/browse/HBASE-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8935:
-
Fix Version/s: (was: 0.94.10)
0.94.11
IntegrationTestBigLinkedList fails under load on 0.94 due to some scan issues
- add logging
---
Key: HBASE-8935
URL: https://issues.apache.org/jira/browse/HBASE-8935
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Fix For: 0.95.2, 0.94.11
Attachments: HBASE-8935-v0-94-logging.patch, HBASE-8935-v0.patch
We observe occasional failures (4-5k undefined and unreferenced nodes in the
list, running) when the cluster is stressed while running
IntegrationTestBigLinkedList on 0.94. Unfortunately debug logging is disabled.
If Verify is rerun after the fact, the data is there, so it's not data loss.
All the missing keys come from very narrow range from the very beginning of
one region; most of this region is not affected.
In the case I'm looking at, region range is
{code}\xD0\x0C`=%\xA2\xD3\xA8\x0A\xF8\x917\x05\x94\x1D ..
\xD4\x0Bx\xAF\x88\x0C\xF47\x9A\x9F{\xCE\x0E\x8A{code}
and problematic renage
{code}\xD0\x0C`QLD\xED2\xD5c\x8D\xDB5\x01\xD2H] ...
\xD0\x0D\x89)\x11\x0E8\xC5\x08o\xD7\xE3$\xB3\xAAu{code}
There are no splits and compactions on the region during MR job; there are no
compactions after that could have affected the data, although there is one
flush that happened long after, and region was moved and reopened (before I
ran verify manually that showed that data is in HBase).
One thing that happened was that the scanners used by all the map tasks had
lease expirations during the MR job that had missing data, some of them twice.
First scanner expiration for each task I looked at was exactly 1 minute after
Processing split log message, when the scanner is opened.
Only the tasks where the scanners have expired twice (2 of them) logged any
errors; presumably one expiration in each task happened after the reading was
finished, because everything was slow and scanner wasn't closed in time - no
errors or exceptions are logged in the tasks where scanner only expired once.
The job I ran afterwards had no scanner expirations so it does close them
correctly in normal case...
The task that was reading the problematic range had one scanner expiration
and didn't log any errors.
It's a little bit special (or it may be a total red herring) in that in other
tasks, scanner expired while the task was splitting partial outputs (which
may mean end of input reading?), whereas in the task that lost data spilling
started long (~40s) after the scanner expired.
However there's one another such task (spilling started 25s after scanner
expired) and it didn't log any errors and didn't lose data.
At this point I am not sure about the root cause but I suspect it might be
related to scanner expiration handling, given the patterns.
One thing for sure is that there isn't enough logging... so I will start with
adding that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7826) Improve Hbase Thrift v1 to return results in sorted order
[ https://issues.apache.org/jira/browse/HBASE-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivendra Pratap Singh updated HBASE-7826: -- Attachment: hbase_7826_sortcolumnFlag.5.patch Patch with unit test and generated with thrift-0.9.0 Improve Hbase Thrift v1 to return results in sorted order - Key: HBASE-7826 URL: https://issues.apache.org/jira/browse/HBASE-7826 Project: HBase Issue Type: New Feature Components: Thrift Affects Versions: 0.94.0 Reporter: Shivendra Pratap Singh Assignee: Shivendra Pratap Singh Priority: Minor Labels: Hbase, Thrift Attachments: hbase_7826.patch, hbase_7826.patch, hbase_7826_sortcolumnFlag.1.patch, hbase_7826_sortcolumnFlag.2.patch, hbase_7826_sortcolumnFlag.3.patch, hbase_7826_sortcolumnFlag.4.patch, hbase_7826_sortcolumnFlag.5.patch, hbase_7826_sortcolumnFlag.patch, hbase_7826_trunk.patch Hbase natively stores columns sorted based on the column qualifier. A scan is guaranteed to return sorted columns. The Java API works fine but the Thrift API is broken. Hbase uses TreeMap that ensures that sort order is maintained. However Hbase thrift specification uses a simple Map to store the data. A map, since it is unordered doesn't result in columns being returned in a sort order that is consistent with their storage in Hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8764) Some MasterMonitorCallable should retry
[
https://issues.apache.org/jira/browse/HBASE-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-8764:
-
Attachment: 8764v3.txt
Add master retying. Uses same mechanism as regionserver retrying. Add test
too of the exception seen here proving we now retry.
Main change is in HBaseAdmin where we get rid of executable methods and all do
executeCallable instead. Here is main change:
{code}
9 - private V V executeCallable(CallableV function) throws IOException {
10 + private V V executeCallable(MasterCallableV callable) throws
IOException {
11 +RpcRetryingCallerV caller = new RpcRetryingCallerV();
12 try {
13 - return function.call();
14 -} catch (RemoteException re) {
15 - throw re.unwrapRemoteException();
16 -} catch (IOException e) {
17 - throw e;
18 -} catch (ServiceException se) {
19 - throw ProtobufUtil.getRemoteException(se);
20 -} catch (Exception e) {
21 - // This should not happen...
22 - throw new IOException(Unexpected exception when calling master, e);
23 + return caller.callWithRetries(callable, getConfiguration());
24 +} finally {
25 + callable.close();
26 }
27}
{code}
The exceptions are handled inside in the callWithRetries.
Some MasterMonitorCallable should retry
---
Key: HBASE-8764
URL: https://issues.apache.org/jira/browse/HBASE-8764
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.95.1
Reporter: Elliott Clark
Assignee: stack
Fix For: 0.95.2
Attachments: 8764.txt, 8764v2.txt, 8764v3.txt
Calls in the admin that only get status should re-try.
got a call stack like:
{code}
org.apache.hadoop.hbase.exceptions.PleaseHoldException:
org.apache.hadoop.hbase.exceptions.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2266)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1610)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1646)
at
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:20930)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2122)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1829)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:230)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2705)
at
org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2674)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:524)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:417)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:349)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.createSchema(IntegrationTestBigLinkedList.java:437)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.runGenerator(IntegrationTestBigLinkedList.java:471)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.run(IntegrationTestBigLinkedList.java:505)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runGenerator(IntegrationTestBigLinkedList.java:698)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:748)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedListWithChaosMonkey.testContinuousIngest(IntegrationTestBigLinkedListWithChaosMonkey.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
[jira] [Updated] (HBASE-8778) Region assigments scan table directory making them slow for huge tables
[
https://issues.apache.org/jira/browse/HBASE-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dave Latham updated HBASE-8778:
---
Attachment: HBASE-8778.patch
Attached HBASE-8778.patch which is a patch for trunk. Up at reviewboard at:
https://reviews.apache.org/r/12920/
This patch is similar to HBASE-8778-0.94.5-v2 but because it is for trunk only
it does not keep two copies (one in tabledir another in .tabledesc subdir) - it
only keeps table info files in the known subdir. As a result it also does not
use any locking.
The patch still includes some cleanup and refactoring of FSTableDescriptors to
consistently enforce the fsreadonly flag and use more instance methods than
static methods in order to do so.
It also changes snapshots to likewise store their table descriptors in the
.tabledesc subdirectory so they continue to share code and look just like table
directories (option #1 from the previous comment.)
It also adds a migration step to HMaster.finishInitialization which migrates
existing snapshot directories, user tables, and system tables to store
descriptors in the .tabledesc subdirectory.
Going to run it by Hadoop QA and welcome review and comment.
Region assigments scan table directory making them slow for huge tables
---
Key: HBASE-8778
URL: https://issues.apache.org/jira/browse/HBASE-8778
Project: HBase
Issue Type: Improvement
Reporter: Dave Latham
Assignee: Dave Latham
Fix For: 0.98.0, 0.95.2, 0.94.11
Attachments: 8778-dirmodtime.txt, HBASE-8778-0.94.5.patch,
HBASE-8778-0.94.5-v2.patch, HBASE-8778.patch
On a table with 130k regions it takes about 3 seconds for a region server to
open a region once it has been assigned.
Watching the threads for a region server running 0.94.5 that is opening many
such regions shows the thread opening the reigon in code like this:
{noformat}
PRI IPC Server handler 4 on 60020 daemon prio=10 tid=0x2aaac07e9000
nid=0x6566 runnable [0x4c46d000]
java.lang.Thread.State: RUNNABLE
at java.lang.String.indexOf(String.java:1521)
at java.net.URI$Parser.scan(URI.java:2912)
at java.net.URI$Parser.parse(URI.java:3004)
at java.net.URI.init(URI.java:736)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
at org.apache.hadoop.fs.Path.init(Path.java:126)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:215)
at
org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:252)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:311)
at
org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:842)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:867)
at org.apache.hadoop.hbase.util.FSUtils.listStatus(FSUtils.java:1168)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:269)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:255)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoModtime(FSTableDescriptors.java:368)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:155)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:126)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2834)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2807)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
{noformat}
To open the region, the region server first loads the latest
HTableDescriptor. Since HBASE-4553 HTableDescriptor's are stored in the file
system at /hbase/tableDir/.tableinfo.sequenceNum. The file with the
largest sequenceNum is the current descriptor. This is done so that the
current descirptor is updated atomically. However, since the filename is not
known in advance FSTableDescriptors it has to do a FileSystem.listStatus
operation which has to list all files in the directory to find it. The
directory also contains all the region directories, so in our
[jira] [Updated] (HBASE-9034) hbase-daemon.sh swallows start up errors
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-9034: - Summary: hbase-daemon.sh swallows start up errors (was: hbase-daemon.sh swallows start up scripts.) hbase-daemon.sh swallows start up errors Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9034) hbase-daemon.sh swallows start up errors
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718842#comment-13718842 ] Jean-Daniel Cryans commented on HBASE-9034: --- Didn't that use to go to console? Right now if something goes wrong you still don't get any feedback. hbase-daemon.sh swallows start up errors Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9034) hbase-daemon.sh swallows start up errors
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-9034: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) hbase-daemon.sh swallows start up errors Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9034) hbase-daemon.sh swallows start up errors
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718844#comment-13718844 ] Elliott Clark commented on HBASE-9034: -- It goes to the .out. Because it's being nohup'd there's no option to put it on the stdout. hbase-daemon.sh swallows start up errors Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9034) hbase-daemon.sh swallows start up errors
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718850#comment-13718850 ] Jean-Daniel Cryans commented on HBASE-9034: --- I understand, and I think it's very unfortunate... hbase-daemon.sh swallows start up errors Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7826) Improve Hbase Thrift v1 to return results in sorted order
[
https://issues.apache.org/jira/browse/HBASE-7826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718866#comment-13718866
]
Hadoop QA commented on HBASE-7826:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12594017/hbase_7826_sortcolumnFlag.5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.thrift.TestThriftServer
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6455//console
This message is automatically generated.
Improve Hbase Thrift v1 to return results in sorted order
-
Key: HBASE-7826
URL: https://issues.apache.org/jira/browse/HBASE-7826
Project: HBase
Issue Type: New Feature
Components: Thrift
Affects Versions: 0.94.0
Reporter: Shivendra Pratap Singh
Assignee: Shivendra Pratap Singh
Priority: Minor
Labels: Hbase, Thrift
Attachments: hbase_7826.patch, hbase_7826.patch,
hbase_7826_sortcolumnFlag.1.patch, hbase_7826_sortcolumnFlag.2.patch,
hbase_7826_sortcolumnFlag.3.patch, hbase_7826_sortcolumnFlag.4.patch,
hbase_7826_sortcolumnFlag.5.patch, hbase_7826_sortcolumnFlag.patch,
hbase_7826_trunk.patch
Hbase natively stores columns sorted based on the column qualifier. A scan is
guaranteed to return sorted columns. The Java API works fine but the Thrift
API is broken. Hbase uses TreeMap that ensures that sort order is maintained.
However Hbase thrift specification uses a simple Map to store the data. A
map, since it is unordered doesn't result in columns being returned in a sort
order that is consistent with their storage in Hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9034) hbase-daemon.sh swallows start up errors
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718877#comment-13718877 ] Elliott Clark commented on HBASE-9034: -- I don't really think that asking people to look in a log for an error is that huge a deal vs shaving 30 seconds off of mttr. hbase-daemon.sh swallows start up errors Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9034) hbase-daemon.sh swallows start up errors
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718897#comment-13718897 ] Jean-Daniel Cryans commented on HBASE-9034: --- bq. I don't really think that asking people to look in a log for an error is that huge a deal vs shaving 30 seconds off of mttr. I can haz both? Should those two things even be related? Or could we at least show this line like we did before 0.95? bq. starting master, logging to /home/jdcryans/hbase-0.94.9/bin/../logs/hbase-jdcryans-master-server.out I'm not blaming your patch, it's actually much better than hacking up the scripts to see what went wrong. hbase-daemon.sh swallows start up errors Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8984) Test online snapshots with online merge
[ https://issues.apache.org/jira/browse/HBASE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718921#comment-13718921 ] stack commented on HBASE-8984: -- [~mbertozzi] Mind taking a look at this https://builds.apache.org/view/H-L/view/HBase/job/hbase-0.95/352/testReport/junit/org.apache.hadoop.hbase.snapshot/TestFlushSnapshotFromClient/testFlushTableSnapshot/ ? It is fail in TestFlushSnapshotFromClient.testFlushTableSnapshot which is amended in this patch. The reported load is 400 at time of test run so could just be that but might be something here. Thanks boss. Test online snapshots with online merge --- Key: HBASE-8984 URL: https://issues.apache.org/jira/browse/HBASE-8984 Project: HBase Issue Type: Test Components: snapshots Affects Versions: 0.95.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.98.0, 0.95.2 Attachments: HBASE-8984.patch Add unit tests to verify that after an online merge: * taking a snapshot still works * snapshots and cloned tables are still in a good state. Also on the same line, fix the unit test to have more than one region, and test multi RS/multi regions online snapshot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9034) hbase-daemon.sh swallows start up errors
[ https://issues.apache.org/jira/browse/HBASE-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-9034: - Attachment: HBASE-9034-ADD.patch :-) hbase-daemon.sh swallows start up errors Key: HBASE-9034 URL: https://issues.apache.org/jira/browse/HBASE-9034 Project: HBase Issue Type: Bug Components: scripts Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 0.98.0, 0.95.2 Attachments: HBASE-9034-0.patch, HBASE-9034-ADD.patch Fix it so that start up errors go into .out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8778) Region assigments scan table directory making them slow for huge tables
[
https://issues.apache.org/jira/browse/HBASE-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718935#comment-13718935
]
Hadoop QA commented on HBASE-8778:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12594033/HBASE-8778.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 30 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the trunk's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.handler.TestTableDeleteFamilyHandler
org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient
org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6456//console
This message is automatically generated.
Region assigments scan table directory making them slow for huge tables
---
Key: HBASE-8778
URL: https://issues.apache.org/jira/browse/HBASE-8778
Project: HBase
Issue Type: Improvement
Reporter: Dave Latham
Assignee: Dave Latham
Fix For: 0.98.0, 0.95.2, 0.94.11
Attachments: 8778-dirmodtime.txt, HBASE-8778-0.94.5.patch,
HBASE-8778-0.94.5-v2.patch, HBASE-8778.patch
On a table with 130k regions it takes about 3 seconds for a region server to
open a region once it has been assigned.
Watching the threads for a region server running 0.94.5 that is opening many
such regions shows the thread opening the reigon in code like this:
{noformat}
PRI IPC Server handler 4 on 60020 daemon prio=10 tid=0x2aaac07e9000
nid=0x6566 runnable [0x4c46d000]
java.lang.Thread.State: RUNNABLE
at java.lang.String.indexOf(String.java:1521)
at java.net.URI$Parser.scan(URI.java:2912)
at java.net.URI$Parser.parse(URI.java:3004)
at java.net.URI.init(URI.java:736)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
at org.apache.hadoop.fs.Path.init(Path.java:126)
at
[jira] [Commented] (HBASE-8764) Some MasterMonitorCallable should retry
[
https://issues.apache.org/jira/browse/HBASE-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718955#comment-13718955
]
Elliott Clark commented on HBASE-8764:
--
+1 (if it passes tests). There's a lot of code movement, but it all looks
pretty sane.
Is there anyway to make some of these callable's paramterized ? Rather than
having MasterAdminCallableVoid have MasterCallableAdminProtocol, Void
Some MasterMonitorCallable should retry
---
Key: HBASE-8764
URL: https://issues.apache.org/jira/browse/HBASE-8764
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.95.1
Reporter: Elliott Clark
Assignee: stack
Fix For: 0.95.2
Attachments: 8764.txt, 8764v2.txt, 8764v3.txt
Calls in the admin that only get status should re-try.
got a call stack like:
{code}
org.apache.hadoop.hbase.exceptions.PleaseHoldException:
org.apache.hadoop.hbase.exceptions.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2266)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1610)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1646)
at
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:20930)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2122)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1829)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:230)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2705)
at
org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2674)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:524)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:417)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:349)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.createSchema(IntegrationTestBigLinkedList.java:437)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.runGenerator(IntegrationTestBigLinkedList.java:471)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.run(IntegrationTestBigLinkedList.java:505)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runGenerator(IntegrationTestBigLinkedList.java:698)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:748)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedListWithChaosMonkey.testContinuousIngest(IntegrationTestBigLinkedListWithChaosMonkey.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at
[jira] [Commented] (HBASE-8764) Some MasterMonitorCallable should retry
[
https://issues.apache.org/jira/browse/HBASE-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718968#comment-13718968
]
Hadoop QA commented on HBASE-8764:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12594030/8764v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 14 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
14 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.security.access.TestAccessController
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove
org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster
org.apache.hadoop.hbase.master.handler.TestTableDeleteFamilyHandler
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient
org.apache.hadoop.hbase.master.TestTableLockManager
org.apache.hadoop.hbase.regionserver.wal.TestHLog
org.apache.hadoop.hbase.master.TestRestartCluster
org.apache.hadoop.hbase.client.TestSnapshotFromClient
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6457//console
This message is automatically generated.
Some MasterMonitorCallable should retry
---
Key: HBASE-8764
URL: https://issues.apache.org/jira/browse/HBASE-8764
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.95.1
Reporter: Elliott Clark
Assignee: stack
Fix For: 0.95.2
Attachments: 8764.txt, 8764v2.txt, 8764v3.txt
Calls in the admin that only get status should re-try.
got a call stack like:
{code}
org.apache.hadoop.hbase.exceptions.PleaseHoldException:
org.apache.hadoop.hbase.exceptions.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2266)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1610)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1646)
at
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:20930)
at
[jira] [Commented] (HBASE-8764) Some MasterMonitorCallable should retry
[
https://issues.apache.org/jira/browse/HBASE-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718983#comment-13718983
]
Elliott Clark commented on HBASE-8764:
--
So that's a no on tests passing :-)
Some MasterMonitorCallable should retry
---
Key: HBASE-8764
URL: https://issues.apache.org/jira/browse/HBASE-8764
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.95.1
Reporter: Elliott Clark
Assignee: stack
Fix For: 0.95.2
Attachments: 8764.txt, 8764v2.txt, 8764v3.txt
Calls in the admin that only get status should re-try.
got a call stack like:
{code}
org.apache.hadoop.hbase.exceptions.PleaseHoldException:
org.apache.hadoop.hbase.exceptions.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2266)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1610)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1646)
at
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:20930)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2122)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1829)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:230)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2705)
at
org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2674)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:524)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:417)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:349)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.createSchema(IntegrationTestBigLinkedList.java:437)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.runGenerator(IntegrationTestBigLinkedList.java:471)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.run(IntegrationTestBigLinkedList.java:505)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runGenerator(IntegrationTestBigLinkedList.java:698)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:748)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedListWithChaosMonkey.testContinuousIngest(IntegrationTestBigLinkedListWithChaosMonkey.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at
[jira] [Commented] (HBASE-8764) Some MasterMonitorCallable should retry
[
https://issues.apache.org/jira/browse/HBASE-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718985#comment-13718985
]
stack commented on HBASE-8764:
--
Thanks [~eclark] To what end your parameterizing? There are two master
interfaces only and both inherit MasterCallable. You'd have a single one
parameterized?
The unit test failures, it seems, are expecting particular exceptions up out of
master ops -- ops that are not retried and if they fail, are wrapped in a
RetriesExhaustedException... let me fix them (or my patch) as appropriate.
Some MasterMonitorCallable should retry
---
Key: HBASE-8764
URL: https://issues.apache.org/jira/browse/HBASE-8764
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.95.1
Reporter: Elliott Clark
Assignee: stack
Fix For: 0.95.2
Attachments: 8764.txt, 8764v2.txt, 8764v3.txt
Calls in the admin that only get status should re-try.
got a call stack like:
{code}
org.apache.hadoop.hbase.exceptions.PleaseHoldException:
org.apache.hadoop.hbase.exceptions.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2266)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1610)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1646)
at
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:20930)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2122)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1829)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:230)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2705)
at
org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2674)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:524)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:417)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:349)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.createSchema(IntegrationTestBigLinkedList.java:437)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.runGenerator(IntegrationTestBigLinkedList.java:471)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.run(IntegrationTestBigLinkedList.java:505)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runGenerator(IntegrationTestBigLinkedList.java:698)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:748)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedListWithChaosMonkey.testContinuousIngest(IntegrationTestBigLinkedListWithChaosMonkey.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at
[jira] [Commented] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[ https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718992#comment-13718992 ] stack commented on HBASE-9022: -- This tests still fails. http://54.241.6.143/job/HBase-0.95-Hadoop-2/687/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/420/testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/418/testReport/junit/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ http://54.241.6.143/job/HBase-TRUNK-Hadoop-2/org.apache.hbase$hbase-server/418/testReport/junit/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ ... and its compressed version here: https://builds.apache.org/view/H-L/view/HBase/job/hbase-0.95-on-hadoop2/191/testReport/junit/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplitCompressed/testSplitWillFailIfWritingToRegionFails/ Usually the test completes in a couple of seconds. When it goes 300seconds, we are not closing one of our writers. It is sticking around. It is not a daemon thread I suppose because we want to make sure our edits all go in. So, it is stuck somewhere. I can't see it at momemnt. Looking TestHLogSplit.testIOEOnOutputThread fails - Key: HBASE-9022 URL: https://issues.apache.org/jira/browse/HBASE-9022 Project: HBase Issue Type: Bug Reporter: stack Attachments: 9022.txt, 9022.txt, 9022.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6428//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9037) LruBlockCache is too verbose at Debug level
Lars Hofhansl created HBASE-9037:
Summary: LruBlockCache is too verbose at Debug level
Key: HBASE-9037
URL: https://issues.apache.org/jira/browse/HBASE-9037
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Fix For: 0.94.10
From Stack on the mailing list:
{quote}
I see this every 4ms or so when reading.
21 2013-07-24 10:55:24,594 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
completed; freed=24.78 MB, total=185.13 MB, single=0 KB, multi=207.88 MB,
memory=0.64 KB
20 2013-07-24 10:55:28,975 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
started; Attempting to free 24.75 MB of total=209.91 MB
Loads of it.
{quote}
Should either limit the log frequency of this, or change to trace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8627) HBCK can not fix meta not assigned issue
[ https://issues.apache.org/jira/browse/HBASE-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719017#comment-13719017 ] Jimmy Xiang commented on HBASE-8627: I think the patch is ok. As to the recordMetaRegion issue Rajesh mentioned, I think it is ok to error out. We can run hbck again. For the second issue Rajesh mentioned, about checkMetaRegion, the current patch is fine, right? It handles both meta is not assigned, and it is double assigned. Anything did I miss? HBCK can not fix meta not assigned issue Key: HBASE-8627 URL: https://issues.apache.org/jira/browse/HBASE-8627 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.95.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Attachments: HBASE-8627_Trunk.patch, HBASE-8627_Trunk-V2.patch When meta table region is not assigned to any RS, HBCK run will get exception. I can see code added in checkMetaRegion() to solve this issue but it wont work. It still refers to ROOT region! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[
https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719022#comment-13719022
]
stack commented on HBASE-9022:
--
So here is a local run where testIOEOnOutputThread completes a couple of
seconds:
{code}
2013-07-24 15:32:10,936 INFO [main] wal.HLogSplitter(334): Finishing writing
output logs and closing down.
2013-07-24 15:32:10,936 INFO [main] wal.HLogSplitter$OutputSink(925): Waiting
for split writer threads to finish
2013-07-24 15:32:10,962 INFO [IPC Server handler 8 on 65470]
namenode.FSNamesystem(169): ugi=stack ip=/127.0.0.1 cmd=mkdirs
src=/hbase/t1/bbb/recovered.edits dst=null
perm=stack:supergroup:rwxr-xr-x
2013-07-24 15:32:10,963 INFO [IPC Server handler 6 on 65470]
namenode.FSNamesystem(169): ugi=stack ip=/127.0.0.1 cmd=mkdirs
src=/hbase/t1/ccc/recovered.edits dst=null
perm=stack:supergroup:rwxr-xr-x
2013-07-24 15:32:11,078 DEBUG [WriterThread-1]
wal.HLogSplitter$LogRecoveredEditsOutputSink(1200): Creating writer
path=/hbase/t1/bbb/recovered.edits/001.temp region=bbb
2013-07-24 15:32:11,078 DEBUG [WriterThread-2]
wal.HLogSplitter$LogRecoveredEditsOutputSink(1200): Creating writer
path=/hbase/t1/ccc/recovered.edits/002.temp region=ccc
2013-07-24 15:32:11,082 FATAL [WriterThread-2]
wal.HLogSplitter$LogRecoveredEditsOutputSink(1234): Got while writing log
entry to log
java.io.IOException: Injected
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogRecoveredEditsOutputSink.append(HLogSplitter.java:1225)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:829)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:821)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:791)
2013-07-24 15:32:11,082 FATAL [WriterThread-1]
wal.HLogSplitter$LogRecoveredEditsOutputSink(1234): Got while writing log
entry to log
java.io.IOException: Injected
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogRecoveredEditsOutputSink.append(HLogSplitter.java:1225)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:829)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:821)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:791)
2013-07-24 15:32:11,082 ERROR [WriterThread-1]
wal.HLogSplitter$WriterThread(793): Error in log splitting write thread
java.io.IOException: Injected
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogRecoveredEditsOutputSink.append(HLogSplitter.java:1225)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:829)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:821)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:791)
2013-07-24 15:32:11,082 ERROR [WriterThread-2]
wal.HLogSplitter$WriterThread(793): Error in log splitting write thread
java.io.IOException: Injected
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$LogRecoveredEditsOutputSink.append(HLogSplitter.java:1225)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.writeBuffer(HLogSplitter.java:829)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.doRun(HLogSplitter.java:821)
at
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter$WriterThread.run(HLogSplitter.java:791)
2013-07-24 15:32:11,087 INFO [split-log-closeStream-2]
wal.HLogSplitter$LogRecoveredEditsOutputSink$2(1042): Closed path
/hbase/t1/ccc/recovered.edits/002.temp (wrote 0 edits in 0ms)
2013-07-24 15:32:11,087 INFO [split-log-closeStream-1]
wal.HLogSplitter$LogRecoveredEditsOutputSink$2(1042): Closed path
/hbase/t1/bbb/recovered.edits/001.temp (wrote 0 edits in 0ms)
{code}
It is not easy to read but notice how we do the 'Creating writer' lines one
after the other.
Here is an example fail.
{code}
2013-07-24 20:52:06,520 INFO [Thread-297] wal.HLogSplitter(334): Finishing
writing output logs and closing down.
2013-07-24 20:52:06,520 INFO [Thread-297] wal.HLogSplitter$OutputSink(925):
Waiting for split writer threads to finish
2013-07-24 20:52:06,529 DEBUG [WriterThread-2]
wal.HLogSplitter$WriterThread(799): Writer thread
Thread[WriterThread-2,5,main]: starting
2013-07-24 20:52:06,542 INFO [IPC Server handler 9 on 50391]
namenode.FSNamesystem$DefaultAuditLogger(5567): allowed=true ugi=ec2-user
(auth:SIMPLE) ip=/127.0.0.1 cmd=getfileinfo src=/hbase/t1/bbb
dst=nullperm=null
[jira] [Commented] (HBASE-9026) RestartRsHoldingRoot action in org.apache.hadoop.hbase.util.ChaosMonkey restarting the server holding .META. instead of -ROOT-
[
https://issues.apache.org/jira/browse/HBASE-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719023#comment-13719023
]
Ted Yu commented on HBASE-9026:
---
Integrated to 0.94
Thanks for the patch, Screenivasulu.
Thanks for the reviews.
RestartRsHoldingRoot action in org.apache.hadoop.hbase.util.ChaosMonkey
restarting the server holding .META. instead of -ROOT-
--
Key: HBASE-9026
URL: https://issues.apache.org/jira/browse/HBASE-9026
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.8
Reporter: Y. SREENIVASULU REDDY
Priority: Minor
Fix For: 0.94.11
Attachments: ChaosMonkey.java.patch
In ChaosMonkey instead of restarting Root holded regionServer it is
restarting META holded regionServer.
{code}
public static class RestartRsHoldingRoot extends RestartRandomRs {
public RestartRsHoldingRoot(long sleepTime) {
super(sleepTime);
}
@Override
void perform() throws Exception {
LOG.info(Performing action: Restart region server holding ROOT);
ServerName server = cluster.getServerHoldingMeta();
if (server == null) {
LOG.warn(No server is holding -ROOT- right now.);
return;
}
restartRs(server, sleepTime);
}
}
{code}
{noformat}
13/07/23 17:03:54 INFO util.ChaosMonkey: Performing action: Restart region
server holding ROOT
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@52b57e9a;
serverName=ocean06,60020,1374569995361
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Removed .META.,,1.1028785192 for tableName=.META. from cache because of
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for .META.,,1.1028785192 is ocean06:60020
13/07/23 17:03:54 INFO util.ChaosMonkey: Killing region
server:ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.HBaseCluster: Aborting RS:
ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.ClusterManager: Executing remote command: ps ux
| grep regionserver | grep hbase | grep -v grep | tr -s ' ' | cut -d ' ' -f2
| xargs kill -s SIGKILL , hostname:ocean06
13/07/23 17:03:54 INFO util.Shell: Executing full command [/usr/bin/ssh
ocean06 ps ux | grep regionserver | grep hbase | grep -v grep | tr -s ' ' |
cut -d ' ' -f2 | xargs kill -s SIGKILL]
13/07/23 17:03:54 INFO hbase.ClusterManager: Executed remote command, exit
code:0 , output:
13/07/23 17:03:54 INFO hbase.HBaseCluster: Waiting service:regionserver to
stop: ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.ClusterManager: Executing remote command: ps ux
| grep regionserver | grep hbase | grep -v grep | tr -s ' ' | cut -d ' ' -f2
, hostname:ocean06
13/07/23 17:03:54 INFO util.Shell: Executing full command [/usr/bin/ssh
ocean06 ps ux | grep regionserver | grep hbase | grep -v grep | tr -s ' ' |
cut -d ' ' -f2]
13/07/23 17:03:55 INFO hbase.ClusterManager: Executed remote command, exit
code:0 , output:
13/07/23 17:03:55 INFO util.ChaosMonkey: Killed region
server:ocean06,60020,1374569995361. Reported num of rs:2
{noformat}
This is only in 0.94.X
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8764) Some MasterMonitorCallable should retry
[
https://issues.apache.org/jira/browse/HBASE-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719030#comment-13719030
]
Elliott Clark commented on HBASE-8764:
--
bq.Thanks Elliott Clark To what end your parameterizing?
That was my first thought, but those classes are so small that it's probably
not a good thing in the end. Disregard.
Some MasterMonitorCallable should retry
---
Key: HBASE-8764
URL: https://issues.apache.org/jira/browse/HBASE-8764
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.95.1
Reporter: Elliott Clark
Assignee: stack
Fix For: 0.95.2
Attachments: 8764.txt, 8764v2.txt, 8764v3.txt
Calls in the admin that only get status should re-try.
got a call stack like:
{code}
org.apache.hadoop.hbase.exceptions.PleaseHoldException:
org.apache.hadoop.hbase.exceptions.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2266)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1610)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1646)
at
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:20930)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2122)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1829)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:230)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2705)
at
org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2674)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:524)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:417)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:349)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.createSchema(IntegrationTestBigLinkedList.java:437)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.runGenerator(IntegrationTestBigLinkedList.java:471)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.run(IntegrationTestBigLinkedList.java:505)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runGenerator(IntegrationTestBigLinkedList.java:698)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:748)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedListWithChaosMonkey.testContinuousIngest(IntegrationTestBigLinkedListWithChaosMonkey.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
[jira] [Comment Edited] (HBASE-8764) Some MasterMonitorCallable should retry
[
https://issues.apache.org/jira/browse/HBASE-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719030#comment-13719030
]
Elliott Clark edited comment on HBASE-8764 at 7/24/13 11:44 PM:
bq.Thanks Elliott Clark To what end your parameterizing? There are two master
interfaces only and both inherit MasterCallable. You'd have a single one
parameterized?
That was my first thought, but those classes are so small that it's probably
not a good thing in the end. Disregard.
was (Author: eclark):
bq.Thanks Elliott Clark To what end your parameterizing?
That was my first thought, but those classes are so small that it's probably
not a good thing in the end. Disregard.
Some MasterMonitorCallable should retry
---
Key: HBASE-8764
URL: https://issues.apache.org/jira/browse/HBASE-8764
Project: HBase
Issue Type: Bug
Components: IPC/RPC
Affects Versions: 0.95.1
Reporter: Elliott Clark
Assignee: stack
Fix For: 0.95.2
Attachments: 8764.txt, 8764v2.txt, 8764v3.txt
Calls in the admin that only get status should re-try.
got a call stack like:
{code}
org.apache.hadoop.hbase.exceptions.PleaseHoldException:
org.apache.hadoop.hbase.exceptions.PleaseHoldException: Master is initializing
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2266)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1610)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1646)
at
org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:20930)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2122)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1829)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:230)
at
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2705)
at
org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2674)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:524)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:417)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:349)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.createSchema(IntegrationTestBigLinkedList.java:437)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.runGenerator(IntegrationTestBigLinkedList.java:471)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.run(IntegrationTestBigLinkedList.java:505)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runGenerator(IntegrationTestBigLinkedList.java:698)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:748)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.test.IntegrationTestBigLinkedListWithChaosMonkey.testContinuousIngest(IntegrationTestBigLinkedListWithChaosMonkey.java:80)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
[jira] [Updated] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[ https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9022: - Attachment: 9022.addthreaddumping.txt Will thread dump every minute if stuck. TestHLogSplit.testIOEOnOutputThread fails - Key: HBASE-9022 URL: https://issues.apache.org/jira/browse/HBASE-9022 Project: HBase Issue Type: Bug Reporter: stack Attachments: 9022.addthreaddumping.txt, 9022.txt, 9022.txt, 9022.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6428//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8973) Some improvements in the RowCounter MR job
[ https://issues.apache.org/jira/browse/HBASE-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das resolved HBASE-8973. Resolution: Not A Problem Some improvements in the RowCounter MR job -- Key: HBASE-8973 URL: https://issues.apache.org/jira/browse/HBASE-8973 Project: HBase Issue Type: Bug Reporter: Devaraj Das Assignee: Devaraj Das Attachments: 8973-1.txt While trying to test some things, ended up modifying the rowcounter job a little. Proposing a patch with these changes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[ https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719049#comment-13719049 ] stack commented on HBASE-9022: -- Adds thread dumping and wraps an IOE in another so we can where the 'fatal' is being thrown from. I think the fix is waiting on writer threads to all complete before throwing the fatal exception (they have all been signalled) but let me get some evidence first. This looks to be a bug in the log splitter that the unit test is exposing sometimes when scheduling is a little jittery rather than your usual nice and smooth on laptop. TestHLogSplit.testIOEOnOutputThread fails - Key: HBASE-9022 URL: https://issues.apache.org/jira/browse/HBASE-9022 Project: HBase Issue Type: Bug Reporter: stack Attachments: 9022.addthreaddumping.txt, 9022.txt, 9022.txt, 9022.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6428//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[ https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719052#comment-13719052 ] stack commented on HBASE-9022: -- I committed the thread dumping patch to 0.95 and trunk. Lets see how tonights builds do. TestHLogSplit.testIOEOnOutputThread fails - Key: HBASE-9022 URL: https://issues.apache.org/jira/browse/HBASE-9022 Project: HBase Issue Type: Bug Reporter: stack Attachments: 9022.addthreaddumping.txt, 9022.txt, 9022.txt, 9022.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6428//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[
https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719091#comment-13719091
]
Hadoop QA commented on HBASE-9022:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12594062/9022.addthreaddumping.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6458//console
This message is automatically generated.
TestHLogSplit.testIOEOnOutputThread fails
-
Key: HBASE-9022
URL: https://issues.apache.org/jira/browse/HBASE-9022
Project: HBase
Issue Type: Bug
Reporter: stack
Attachments: 9022.addthreaddumping.txt, 9022.txt, 9022.txt, 9022.txt
https://builds.apache.org/job/PreCommit-HBASE-Build/6428//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9038) Compaction WALEdit gives NPEs with Replication enabled
Jean-Daniel Cryans created HBASE-9038:
-
Summary: Compaction WALEdit gives NPEs with Replication enabled
Key: HBASE-9038
URL: https://issues.apache.org/jira/browse/HBASE-9038
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.1
Reporter: Jean-Daniel Cryans
Assignee: Jean-Daniel Cryans
Priority: Blocker
Fix For: 0.98.0, 0.95.2
If you enable replication, and get a compaction requested, you'll see this in
the logs:
{noformat}
2013-07-24 15:16:38,831 ERROR
[regionserver60020-smallCompactions-1374704194254]
regionserver.CompactSplitThread: Compaction failed Request =
regionName=TestTable,057204,1374704192994.6bb7c58d1e6cc99fbfe04592e44fbc35.,
storeName=info, fileCount=4, fileSize=335.1m (115.4m, 115.4m, 69.0m, 35.2m),
priority=6, time=1374704194257521000
java.lang.NullPointerException
at
org.apache.hadoop.hbase.replication.regionserver.Replication.visitLogEntryBeforeWrite(Replication.java:215)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.doWrite(FSHLog.java:1204)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:890)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:840)
at
org.apache.hadoop.hbase.regionserver.wal.HLogUtil.writeCompactionMarker(HLogUtil.java:262)
at
org.apache.hadoop.hbase.regionserver.HStore.writeCompactionWalRecord(HStore.java:1026)
at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:973)
at
org.apache.hadoop.hbase.regionserver.HRegion.compact(HRegion.java:1278)
at
org.apache.hadoop.hbase.regionserver.CompactSplitThread$CompactionRunner.run(CompactSplitThread.java:465)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
{noformat}
It's a simple case of filtering METAFAMILY like this:
bq. if (kv.matchingFamily(WALEdit.METAFAMILY)) continue;
and add a unit test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9037) LruBlockCache is too verbose at Debug level
[
https://issues.apache.org/jira/browse/HBASE-9037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719113#comment-13719113
]
Jean-Daniel Cryans commented on HBASE-9037:
---
I've always found them useful and they only tend to annoy devs testing loads of
reads on small heaps. On live systems you don't see it as often, and if you do
then you got a pretty bad churn problem.
Also on trunk those factors are different:
{code}
static final float DEFAULT_MIN_FACTOR = 0.95f;
static final float DEFAULT_ACCEPTABLE_FACTOR = 0.99f;
{code}
LruBlockCache is too verbose at Debug level
---
Key: HBASE-9037
URL: https://issues.apache.org/jira/browse/HBASE-9037
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Fix For: 0.94.10
From Stack on the mailing list:
{quote}
I see this every 4ms or so when reading.
21 2013-07-24 10:55:24,594 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
completed; freed=24.78 MB, total=185.13 MB, single=0 KB, multi=207.88 MB,
memory=0.64 KB
20 2013-07-24 10:55:28,975 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
started; Attempting to free 24.75 MB of total=209.91 MB
Loads of it.
{quote}
Should either limit the log frequency of this, or change to trace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8015) Support for Namespaces
[ https://issues.apache.org/jira/browse/HBASE-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719135#comment-13719135 ] Enis Soztutar commented on HBASE-8015: -- I think above scheme is acceptable and the best we can do right now. ns:table also fits into cf:column naming as well. bq. rename FullyQualifiedTableName to TableName makes sense. Support for Namespaces -- Key: HBASE-8015 URL: https://issues.apache.org/jira/browse/HBASE-8015 Project: HBase Issue Type: New Feature Reporter: Francis Liu Assignee: Francis Liu Attachments: HBASE-8015_draft_94.patch, Namespace Design.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9039) During recovery, perform assignment and distributed log replay in parallel
Devaraj Das created HBASE-9039: -- Summary: During recovery, perform assignment and distributed log replay in parallel Key: HBASE-9039 URL: https://issues.apache.org/jira/browse/HBASE-9039 Project: HBase Issue Type: Improvement Reporter: Devaraj Das Assignee: Devaraj Das Priority: Minor In the ServerShutDownHandler, the log replay starts only after all the regions have been assigned. It might make sense to do the log replay as and when assignment happens for the regions rather than wait for the batch of assignments to complete. As part of region opening, the store files are read, and if a certain store file block is on a slow datanode, it slows down the assignment process. Maybe this can be amortized by having the log replay in parallel. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-9026) RestartRsHoldingRoot action in org.apache.hadoop.hbase.util.ChaosMonkey restarting the server holding .META. instead of -ROOT-
[
https://issues.apache.org/jira/browse/HBASE-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HBASE-9026:
-
Assignee: Y. SREENIVASULU REDDY
RestartRsHoldingRoot action in org.apache.hadoop.hbase.util.ChaosMonkey
restarting the server holding .META. instead of -ROOT-
--
Key: HBASE-9026
URL: https://issues.apache.org/jira/browse/HBASE-9026
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.8
Reporter: Y. SREENIVASULU REDDY
Assignee: Y. SREENIVASULU REDDY
Priority: Minor
Fix For: 0.94.11
Attachments: ChaosMonkey.java.patch
In ChaosMonkey instead of restarting Root holded regionServer it is
restarting META holded regionServer.
{code}
public static class RestartRsHoldingRoot extends RestartRandomRs {
public RestartRsHoldingRoot(long sleepTime) {
super(sleepTime);
}
@Override
void perform() throws Exception {
LOG.info(Performing action: Restart region server holding ROOT);
ServerName server = cluster.getServerHoldingMeta();
if (server == null) {
LOG.warn(No server is holding -ROOT- right now.);
return;
}
restartRs(server, sleepTime);
}
}
{code}
{noformat}
13/07/23 17:03:54 INFO util.ChaosMonkey: Performing action: Restart region
server holding ROOT
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@52b57e9a;
serverName=ocean06,60020,1374569995361
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Removed .META.,,1.1028785192 for tableName=.META. from cache because of
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for .META.,,1.1028785192 is ocean06:60020
13/07/23 17:03:54 INFO util.ChaosMonkey: Killing region
server:ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.HBaseCluster: Aborting RS:
ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.ClusterManager: Executing remote command: ps ux
| grep regionserver | grep hbase | grep -v grep | tr -s ' ' | cut -d ' ' -f2
| xargs kill -s SIGKILL , hostname:ocean06
13/07/23 17:03:54 INFO util.Shell: Executing full command [/usr/bin/ssh
ocean06 ps ux | grep regionserver | grep hbase | grep -v grep | tr -s ' ' |
cut -d ' ' -f2 | xargs kill -s SIGKILL]
13/07/23 17:03:54 INFO hbase.ClusterManager: Executed remote command, exit
code:0 , output:
13/07/23 17:03:54 INFO hbase.HBaseCluster: Waiting service:regionserver to
stop: ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.ClusterManager: Executing remote command: ps ux
| grep regionserver | grep hbase | grep -v grep | tr -s ' ' | cut -d ' ' -f2
, hostname:ocean06
13/07/23 17:03:54 INFO util.Shell: Executing full command [/usr/bin/ssh
ocean06 ps ux | grep regionserver | grep hbase | grep -v grep | tr -s ' ' |
cut -d ' ' -f2]
13/07/23 17:03:55 INFO hbase.ClusterManager: Executed remote command, exit
code:0 , output:
13/07/23 17:03:55 INFO util.ChaosMonkey: Killed region
server:ocean06,60020,1374569995361. Reported num of rs:2
{noformat}
This is only in 0.94.X
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9036) Few small code cleanup
[ https://issues.apache.org/jira/browse/HBASE-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-9036: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Few small code cleanup -- Key: HBASE-9036 URL: https://issues.apache.org/jira/browse/HBASE-9036 Project: HBase Issue Type: Bug Reporter: Jean-Marc Spaggiari Assignee: Jean-Marc Spaggiari Priority: Minor Fix For: 0.98.0 Attachments: HBASE-9036-v0-trunk.patch, HBASE-9036-v1-trunk.patch, HBASE-9036-v2-trunk.patch Few code cleanup from HBase trunk. 1) TestOperation use String.format with 6 %s but give 7 parameters. Resolution: Trivial 2) ClassFinder can throw a NPE. If jarFile = new JarInputStream(new FileInputStream(jarFileName)); throw an exception and we want to proceed on exceptions, jarFile will be null, and just few lines after we will do a jarFile.getNextJarEntry() where NPE is not catch and will fail and throw an NPE. So I thinkg we can't proceed on exceptions for this first try since it will fail just the after with an NPE and we will loose the information about the real cause of the exception. Therefor, we should always throw ioEx is the InputStream creation fails. 3)AccessController declare cfs but never use it. 4) FavoredNodeAssignmentHelper invokes toString on an array. Just changed that to Bytes.toString() to print the server name. 5) ModifyTableHandler invokes toString on the tableName array. Just changed that to Bytes.toString() to print the table name. 6) HFileWriterV2 invokes toString on the keys arrays. Just changed that to Bytes.toStringBinary() to print the keys. And change some toString() calls to toStringBinary() 7) ServerAndLoad want to be serializable, but ServerName is not. Made ServerName serializable since it's only Strings, numbers and bytes. 8) StorageClusterStatusModel want to be serializable, but its nested class Node is not. Made Node serializable since it's only numbers and bytes. 9) In HRegion outResults can't be null since it's already used for outResults.isEmpty() few lines above. Just remove the test. 10) In RegionScannerHolder region can't be null since it's already used for region.startRegionOperation (and others) few lines above. Just remove the test. 11) CellCounter thisRowFamilyName can't be null since toStringBinary will return the string null for a null value. Just remove the test. 12) CellCounter again, thisRowQualifierName can't be null since it's strings concatenations. Just remove the test. 13) HBaseFsck setDisplayFullReport should be static since writing to a static field. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9026) RestartRsHoldingRoot action in org.apache.hadoop.hbase.util.ChaosMonkey restarting the server holding .META. instead of -ROOT-
[
https://issues.apache.org/jira/browse/HBASE-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-9026:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
RestartRsHoldingRoot action in org.apache.hadoop.hbase.util.ChaosMonkey
restarting the server holding .META. instead of -ROOT-
--
Key: HBASE-9026
URL: https://issues.apache.org/jira/browse/HBASE-9026
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.8
Reporter: Y. SREENIVASULU REDDY
Priority: Minor
Fix For: 0.94.11
Attachments: ChaosMonkey.java.patch
In ChaosMonkey instead of restarting Root holded regionServer it is
restarting META holded regionServer.
{code}
public static class RestartRsHoldingRoot extends RestartRandomRs {
public RestartRsHoldingRoot(long sleepTime) {
super(sleepTime);
}
@Override
void perform() throws Exception {
LOG.info(Performing action: Restart region server holding ROOT);
ServerName server = cluster.getServerHoldingMeta();
if (server == null) {
LOG.warn(No server is holding -ROOT- right now.);
return;
}
restartRs(server, sleepTime);
}
}
{code}
{noformat}
13/07/23 17:03:54 INFO util.ChaosMonkey: Performing action: Restart region
server holding ROOT
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@52b57e9a;
serverName=ocean06,60020,1374569995361
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Removed .META.,,1.1028785192 for tableName=.META. from cache because of
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for .META.,,1.1028785192 is ocean06:60020
13/07/23 17:03:54 INFO util.ChaosMonkey: Killing region
server:ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.HBaseCluster: Aborting RS:
ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.ClusterManager: Executing remote command: ps ux
| grep regionserver | grep hbase | grep -v grep | tr -s ' ' | cut -d ' ' -f2
| xargs kill -s SIGKILL , hostname:ocean06
13/07/23 17:03:54 INFO util.Shell: Executing full command [/usr/bin/ssh
ocean06 ps ux | grep regionserver | grep hbase | grep -v grep | tr -s ' ' |
cut -d ' ' -f2 | xargs kill -s SIGKILL]
13/07/23 17:03:54 INFO hbase.ClusterManager: Executed remote command, exit
code:0 , output:
13/07/23 17:03:54 INFO hbase.HBaseCluster: Waiting service:regionserver to
stop: ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.ClusterManager: Executing remote command: ps ux
| grep regionserver | grep hbase | grep -v grep | tr -s ' ' | cut -d ' ' -f2
, hostname:ocean06
13/07/23 17:03:54 INFO util.Shell: Executing full command [/usr/bin/ssh
ocean06 ps ux | grep regionserver | grep hbase | grep -v grep | tr -s ' ' |
cut -d ' ' -f2]
13/07/23 17:03:55 INFO hbase.ClusterManager: Executed remote command, exit
code:0 , output:
13/07/23 17:03:55 INFO util.ChaosMonkey: Killed region
server:ocean06,60020,1374569995361. Reported num of rs:2
{noformat}
This is only in 0.94.X
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719201#comment-13719201
]
Lars Hofhansl commented on HBASE-6143:
--
v2 *could* work. The code path for assignment after enabling is now quite
different, though.
Might still be better to call {{AssignmentManager.assign(MapHRegionInfo,
ServerName regions)}} {{BulkEnabler.populatePool}}, I think we can build the
map ahead of time.
This is the same call used by AssignmentManager.joinCluster(), so we'd use the
same code.
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Attachments: 6143-v1.txt, 6143-v2.txt
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719209#comment-13719209 ] stack commented on HBASE-6143: -- Unit test? Is bulk assigner safe? ([~jxiang]?) In the past, bulk assigner was known to not be safe and if we failed, because we were using it on startup only, then we'd just exit so user would retry. Jimmy may have fixed this. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Attachments: 6143-v1.txt, 6143-v2.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9026) RestartRsHoldingRoot action in org.apache.hadoop.hbase.util.ChaosMonkey restarting the server holding .META. instead of -ROOT-
[
https://issues.apache.org/jira/browse/HBASE-9026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719217#comment-13719217
]
Y. SREENIVASULU REDDY commented on HBASE-9026:
--
thanks Ted.
RestartRsHoldingRoot action in org.apache.hadoop.hbase.util.ChaosMonkey
restarting the server holding .META. instead of -ROOT-
--
Key: HBASE-9026
URL: https://issues.apache.org/jira/browse/HBASE-9026
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.8
Reporter: Y. SREENIVASULU REDDY
Assignee: Y. SREENIVASULU REDDY
Priority: Minor
Fix For: 0.94.11
Attachments: ChaosMonkey.java.patch
In ChaosMonkey instead of restarting Root holded regionServer it is
restarting META holded regionServer.
{code}
public static class RestartRsHoldingRoot extends RestartRandomRs {
public RestartRsHoldingRoot(long sleepTime) {
super(sleepTime);
}
@Override
void perform() throws Exception {
LOG.info(Performing action: Restart region server holding ROOT);
ServerName server = cluster.getServerHoldingMeta();
if (server == null) {
LOG.warn(No server is holding -ROOT- right now.);
return;
}
restartRs(server, sleepTime);
}
}
{code}
{noformat}
13/07/23 17:03:54 INFO util.ChaosMonkey: Performing action: Restart region
server holding ROOT
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@52b57e9a;
serverName=ocean06,60020,1374569995361
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Removed .META.,,1.1028785192 for tableName=.META. from cache because of
13/07/23 17:03:54 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for .META.,,1.1028785192 is ocean06:60020
13/07/23 17:03:54 INFO util.ChaosMonkey: Killing region
server:ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.HBaseCluster: Aborting RS:
ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.ClusterManager: Executing remote command: ps ux
| grep regionserver | grep hbase | grep -v grep | tr -s ' ' | cut -d ' ' -f2
| xargs kill -s SIGKILL , hostname:ocean06
13/07/23 17:03:54 INFO util.Shell: Executing full command [/usr/bin/ssh
ocean06 ps ux | grep regionserver | grep hbase | grep -v grep | tr -s ' ' |
cut -d ' ' -f2 | xargs kill -s SIGKILL]
13/07/23 17:03:54 INFO hbase.ClusterManager: Executed remote command, exit
code:0 , output:
13/07/23 17:03:54 INFO hbase.HBaseCluster: Waiting service:regionserver to
stop: ocean06,60020,1374569995361
13/07/23 17:03:54 INFO hbase.ClusterManager: Executing remote command: ps ux
| grep regionserver | grep hbase | grep -v grep | tr -s ' ' | cut -d ' ' -f2
, hostname:ocean06
13/07/23 17:03:54 INFO util.Shell: Executing full command [/usr/bin/ssh
ocean06 ps ux | grep regionserver | grep hbase | grep -v grep | tr -s ' ' |
cut -d ' ' -f2]
13/07/23 17:03:55 INFO hbase.ClusterManager: Executed remote command, exit
code:0 , output:
13/07/23 17:03:55 INFO util.ChaosMonkey: Killed region
server:ocean06,60020,1374569995361. Reported num of rs:2
{noformat}
This is only in 0.94.X
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719226#comment-13719226 ] Lars Hofhansl commented on HBASE-6143: -- If we use AssignmentManager.assign(MapHRegionInfo, ServerName regions) we're using the same code as used in startup, hopefully that is safe. We absolutely would need a test for this. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Attachments: 6143-v1.txt, 6143-v2.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9037) LruBlockCache is too verbose at Debug level
[
https://issues.apache.org/jira/browse/HBASE-9037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719227#comment-13719227
]
Lars Hofhansl commented on HBASE-9037:
--
So in trunk we'd see *more* of these messages as the difference between min and
max is smaller.
I think I should just mark this as invalid.
LruBlockCache is too verbose at Debug level
---
Key: HBASE-9037
URL: https://issues.apache.org/jira/browse/HBASE-9037
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Fix For: 0.94.10
From Stack on the mailing list:
{quote}
I see this every 4ms or so when reading.
21 2013-07-24 10:55:24,594 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
completed; freed=24.78 MB, total=185.13 MB, single=0 KB, multi=207.88 MB,
memory=0.64 KB
20 2013-07-24 10:55:28,975 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
started; Attempting to free 24.75 MB of total=209.91 MB
Loads of it.
{quote}
Should either limit the log frequency of this, or change to trace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[ https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9022: - Attachment: 9022.addthreaddumping-part2.txt Addendum for debugging. My first patch was a disaster. Broke builds as all we did was thread dump. TestHLogSplit.testIOEOnOutputThread fails - Key: HBASE-9022 URL: https://issues.apache.org/jira/browse/HBASE-9022 Project: HBase Issue Type: Bug Reporter: stack Attachments: 9022.addthreaddumping-part2.txt, 9022.addthreaddumping.txt, 9022.txt, 9022.txt, 9022.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6428//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[ https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719231#comment-13719231 ] stack commented on HBASE-9022: -- Applied debug addendum to 0.95 and trunk. TestHLogSplit.testIOEOnOutputThread fails - Key: HBASE-9022 URL: https://issues.apache.org/jira/browse/HBASE-9022 Project: HBase Issue Type: Bug Reporter: stack Attachments: 9022.addthreaddumping-part2.txt, 9022.addthreaddumping.txt, 9022.txt, 9022.txt, 9022.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6428//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9022) TestHLogSplit.testIOEOnOutputThread fails
[
https://issues.apache.org/jira/browse/HBASE-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719233#comment-13719233
]
Hadoop QA commented on HBASE-9022:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12594098/9022.addthreaddumping-part2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6459//console
This message is automatically generated.
TestHLogSplit.testIOEOnOutputThread fails
-
Key: HBASE-9022
URL: https://issues.apache.org/jira/browse/HBASE-9022
Project: HBase
Issue Type: Bug
Reporter: stack
Attachments: 9022.addthreaddumping-part2.txt,
9022.addthreaddumping.txt, 9022.txt, 9022.txt, 9022.txt
https://builds.apache.org/job/PreCommit-HBASE-Build/6428//testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLogSplit/testIOEOnOutputThread/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9037) LruBlockCache is too verbose at Debug level
[
https://issues.apache.org/jira/browse/HBASE-9037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719232#comment-13719232
]
stack commented on HBASE-9037:
--
[~lhofhansl] ok.
LruBlockCache is too verbose at Debug level
---
Key: HBASE-9037
URL: https://issues.apache.org/jira/browse/HBASE-9037
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Fix For: 0.94.10
From Stack on the mailing list:
{quote}
I see this every 4ms or so when reading.
21 2013-07-24 10:55:24,594 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
completed; freed=24.78 MB, total=185.13 MB, single=0 KB, multi=207.88 MB,
memory=0.64 KB
20 2013-07-24 10:55:28,975 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction
started; Attempting to free 24.75 MB of total=209.91 MB
Loads of it.
{quote}
Should either limit the log frequency of this, or change to trace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9025) Potential Thread safety issue in MetaScanner
[
https://issues.apache.org/jira/browse/HBASE-9025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-9025:
-
Summary: Potential Thread safety issue in MetaScanner (was: Thread safety
issue in MetaScanner)
Potential Thread safety issue in MetaScanner
Key: HBASE-9025
URL: https://issues.apache.org/jira/browse/HBASE-9025
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.9
Reporter: Lars Hofhansl
I just saw this in a test run in 0.94:
{code}
Stacktrace
java.lang.NullPointerException
at java.util.TreeMap.getEntry(TreeMap.java:324)
at java.util.TreeMap.get(TreeMap.java:255)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(TestHBaseFsck.java:1346)
...
{code}
The TreeMap in question here is actually returned from
{{HTable.getRegionLocations()}}, which in turns calls
{{MetaScanner.allTableRegions(getConfiguration(), getTableName(), false);}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-6143:
--
Attachment: 6143-v3.txt
Turns out that there was a test in TestAdmin involving enabling table.
With patch v2, the modified test passes.
Without patch v2, I got:
{code}
Failed tests:
testEnableTableRetainAssignment(org.apache.hadoop.hbase.client.TestAdmin):
expected:192.168.0.13,50614,1374729419850 but
was:192.168.0.13,50613,1374729419822
{code}
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
