[jira] [Updated] (HBASE-5051) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call
[ https://issues.apache.org/jira/browse/HBASE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5051: --- Status: Open (was: Patch Available) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call -- Key: HBASE-5051 URL: https://issues.apache.org/jira/browse/HBASE-5051 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5051.patch, 5051.v2.patch As it's a new instance, it should be closed. As the function name seems to imply that it's an instance managed by HBaseTestingUtility, most of the users don't close it = leak -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5051) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call
[ https://issues.apache.org/jira/browse/HBASE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5051: --- Attachment: 5051.v2.patch HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call -- Key: HBASE-5051 URL: https://issues.apache.org/jira/browse/HBASE-5051 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5051.patch, 5051.v2.patch, 5051.v2.patch As it's a new instance, it should be closed. As the function name seems to imply that it's an instance managed by HBaseTestingUtility, most of the users don't close it = leak -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5051) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call
[ https://issues.apache.org/jira/browse/HBASE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5051: --- Status: Patch Available (was: Open) retrying with the same patch HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call -- Key: HBASE-5051 URL: https://issues.apache.org/jira/browse/HBASE-5051 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5051.patch, 5051.v2.patch, 5051.v2.patch As it's a new instance, it should be closed. As the function name seems to imply that it's an instance managed by HBaseTestingUtility, most of the users don't close it = leak -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5051) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call
[ https://issues.apache.org/jira/browse/HBASE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171506#comment-13171506 ] Hadoop QA commented on HBASE-5051: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507771/5051.v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 39 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -152 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 76 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestInstantSchemaChange org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/530//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/530//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/530//console This message is automatically generated. HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call -- Key: HBASE-5051 URL: https://issues.apache.org/jira/browse/HBASE-5051 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5051.patch, 5051.v2.patch, 5051.v2.patch As it's a new instance, it should be closed. As the function name seems to imply that it's an instance managed by HBaseTestingUtility, most of the users don't close it = leak -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5051) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call
[ https://issues.apache.org/jira/browse/HBASE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5051: --- Attachment: 5051.v2.patch HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call -- Key: HBASE-5051 URL: https://issues.apache.org/jira/browse/HBASE-5051 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5051.patch, 5051.v2.patch, 5051.v2.patch, 5051.v2.patch As it's a new instance, it should be closed. As the function name seems to imply that it's an instance managed by HBaseTestingUtility, most of the users don't close it = leak -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5051) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call
[ https://issues.apache.org/jira/browse/HBASE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5051: --- Status: Open (was: Patch Available) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call -- Key: HBASE-5051 URL: https://issues.apache.org/jira/browse/HBASE-5051 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5051.patch, 5051.v2.patch, 5051.v2.patch, 5051.v2.patch As it's a new instance, it should be closed. As the function name seems to imply that it's an instance managed by HBaseTestingUtility, most of the users don't close it = leak -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5051) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call
[ https://issues.apache.org/jira/browse/HBASE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171522#comment-13171522 ] nkeywal commented on HBASE-5051: mapred.TestTableMapReduce.testMultiRegionTable: NumberFormatException: For input string: 18446743988037478956 mapreduce.TestHFileOutputFormat.testMRIncrementalLoad: NumberFormatException: For input string: 18446743988037169444 mapreduce.TestHFileOutputFormat.testMRIncrementalLoadWithSplit NumberFormatException: For input string: 18446743988037182259 mapreduce.TestHFileOutputFormat.testExcludeMinorCompaction: NumberFormatException: For input string: 18446743988037186552 TestReplication.queueFailover: ?? TestInstantSchemaChange.testInstantSchemaJanitor: Too many open files TestMasterReplication: hung TestMasterReplication works locally. The patch should not have impacted it. It hungs on trunk as well (see: https://builds.apache.org/job/HBase-TRUNK/2553/testReport/org.apache.hadoop.hbase.replication/ : there is no entry for TestMasterReplication) TestReplication works locally. This test is usually flaky. mapred.TestTableMapReduce worked in the previous prebuild. mapreduce.TestHFileOutputFormat worked in the previous prebuild. Let's retry. HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call -- Key: HBASE-5051 URL: https://issues.apache.org/jira/browse/HBASE-5051 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5051.patch, 5051.v2.patch, 5051.v2.patch, 5051.v2.patch As it's a new instance, it should be closed. As the function name seems to imply that it's an instance managed by HBaseTestingUtility, most of the users don't close it = leak -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5051) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call
[ https://issues.apache.org/jira/browse/HBASE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal updated HBASE-5051: --- Status: Patch Available (was: Open) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call -- Key: HBASE-5051 URL: https://issues.apache.org/jira/browse/HBASE-5051 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5051.patch, 5051.v2.patch, 5051.v2.patch, 5051.v2.patch As it's a new instance, it should be closed. As the function name seems to imply that it's an instance managed by HBaseTestingUtility, most of the users don't close it = leak -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5051) HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call
[ https://issues.apache.org/jira/browse/HBASE-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171562#comment-13171562 ] Hadoop QA commented on HBASE-5051: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507776/5051.v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 39 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -152 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 76 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestInstantSchemaChange org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/531//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/531//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/531//console This message is automatically generated. HBaseTestingUtility#getHBaseAdmin() creates a new HBaseAdmin instance at each call -- Key: HBASE-5051 URL: https://issues.apache.org/jira/browse/HBASE-5051 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor Attachments: 5051.patch, 5051.v2.patch, 5051.v2.patch, 5051.v2.patch As it's a new instance, it should be closed. As the function name seems to imply that it's an instance managed by HBaseTestingUtility, most of the users don't close it = leak -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171648#comment-13171648 ] Phabricator commented on HBASE-4218: Kannan has commented on the revision [jira] [HBASE-4218] Delta encoding for keys in HFile. some more comments... INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncoder.java:65 javadoc fix for the new param includesMemstoreTS is needed on a few of these methods. src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncoder.java:126 little confused with the doc. Could you clarify what happens in the inexact match case: where are we left pointing to for the seekBefore = true and seekBefore=false cases. src/main/java/org/apache/hadoop/hbase/io/deltaencoder/PrefixKeyDeltaEncoder.java:34 here and a bunch of other places... 128 bit encoding should read 7 bit encoding src/main/java/org/apache/hadoop/hbase/io/deltaencoder/BufferedDeltaEncoder.java:475 It seems like we are missing a: keyBuffer = newKeyBuffer; step here after the arrayCopy step. src/main/java/org/apache/hadoop/hbase/io/deltaencoder/BufferedDeltaEncoder.java:470 I think the logic here has an unintentional bug. newKeyBufferLength = keyLength * 2; should be: newKeyBufferLength = keyBuffer.length * 2; Otherwise, the check on the subsequent line will always be FALSE. REVISION DETAIL https://reviews.facebook.net/D447 Delta Encoding of KeyValues (aka prefix compression) - Key: HBASE-4218 URL: https://issues.apache.org/jira/browse/HBASE-4218 Project: HBase Issue Type: Improvement Components: io Affects Versions: 0.94.0 Reporter: Jacek Migdal Assignee: Mikhail Bautin Labels: compression Attachments: 0001-Delta-encoding-fixed-encoded-scanners.patch, D447.1.patch, D447.2.patch, D447.3.patch, D447.4.patch, D447.5.patch, D447.6.patch, D447.7.patch, D447.8.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff A compression for keys. Keys are sorted in HFile and they are usually very similar. Because of that, it is possible to design better compression than general purpose algorithms, It is an additional step designed to be used in memory. It aims to save memory in cache as well as speeding seeks within HFileBlocks. It should improve performance a lot, if key lengths are larger than value lengths. For example, it makes a lot of sense to use it when value is a counter. Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) shows that I could achieve decent level of compression: key compression ratio: 92% total compression ratio: 85% LZO on the same data: 85% LZO after delta encoding: 91% While having much better performance (20-80% faster decompression ratio than LZO). Moreover, it should allow far more efficient seeking which should improve performance a bit. It seems that a simple compression algorithms are good enough. Most of the savings are due to prefix compression, int128 encoding, timestamp diffs and bitfields to avoid duplication. That way, comparisons of compressed data can be much faster than a byte comparator (thanks to prefix compression and bitfields). In order to implement it in HBase two important changes in design will be needed: -solidify interface to HFileBlock / HFileReader Scanner to provide seeking and iterating; access to uncompressed buffer in HFileBlock will have bad performance -extend comparators to support comparison assuming that N first bytes are equal (or some fields are equal) Link to a discussion about something similar: http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windowssubj=Re+prefix+compression -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5040) Secure HBase builds fail
[
https://issues.apache.org/jira/browse/HBASE-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171651#comment-13171651
]
Andrew Purtell commented on HBASE-5040:
---
@Stack: The current patch on this issue works here. Build completes, I can set
up a working all-secure cluster with Hadoop 1.0.0rc2, HBase 0.92.0rc and ZK
3.4.0. But it may not address a similar issue in the 0.22 build that Ted
mentioned. (I haven't checked that.)
Secure HBase builds fail
Key: HBASE-5040
URL: https://issues.apache.org/jira/browse/HBASE-5040
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Zhihong Yu
Fix For: 0.92.0
Attachments: 5040.txt
I saw the following in HBase-0.92-security build #39:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
[ERROR] - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
{code}
The above was probably introduced by HBASE-5006
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171662#comment-13171662 ] Phabricator commented on HBASE-4218: Kannan has commented on the revision [jira] [HBASE-4218] Delta encoding for keys in HFile. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/io/deltaencoder/BufferedDeltaEncoder.java:635 Since we are only copying the non-common-suffix part in this case, shouldn't the offset arguments in both current previous be current.lastCommonPrefix (instead of 0s)? src/main/java/org/apache/hadoop/hbase/io/deltaencoder/PrefixKeyDeltaEncoder.java:147 perhaps we add an assertion that the commonLength == 0 for the first key in the block? REVISION DETAIL https://reviews.facebook.net/D447 Delta Encoding of KeyValues (aka prefix compression) - Key: HBASE-4218 URL: https://issues.apache.org/jira/browse/HBASE-4218 Project: HBase Issue Type: Improvement Components: io Affects Versions: 0.94.0 Reporter: Jacek Migdal Assignee: Mikhail Bautin Labels: compression Attachments: 0001-Delta-encoding-fixed-encoded-scanners.patch, D447.1.patch, D447.2.patch, D447.3.patch, D447.4.patch, D447.5.patch, D447.6.patch, D447.7.patch, D447.8.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff A compression for keys. Keys are sorted in HFile and they are usually very similar. Because of that, it is possible to design better compression than general purpose algorithms, It is an additional step designed to be used in memory. It aims to save memory in cache as well as speeding seeks within HFileBlocks. It should improve performance a lot, if key lengths are larger than value lengths. For example, it makes a lot of sense to use it when value is a counter. Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) shows that I could achieve decent level of compression: key compression ratio: 92% total compression ratio: 85% LZO on the same data: 85% LZO after delta encoding: 91% While having much better performance (20-80% faster decompression ratio than LZO). Moreover, it should allow far more efficient seeking which should improve performance a bit. It seems that a simple compression algorithms are good enough. Most of the savings are due to prefix compression, int128 encoding, timestamp diffs and bitfields to avoid duplication. That way, comparisons of compressed data can be much faster than a byte comparator (thanks to prefix compression and bitfields). In order to implement it in HBase two important changes in design will be needed: -solidify interface to HFileBlock / HFileReader Scanner to provide seeking and iterating; access to uncompressed buffer in HFileBlock will have bad performance -extend comparators to support comparison assuming that N first bytes are equal (or some fields are equal) Link to a discussion about something similar: http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windowssubj=Re+prefix+compression -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5040) Secure HBase builds fail
[
https://issues.apache.org/jira/browse/HBASE-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171705#comment-13171705
]
stack commented on HBASE-5040:
--
@Andrew. OK. Let me put up a new 0.92.0RC but I'm going to do it with
1.0.0rc3. Will commit this patch but change it to point at 1.0.0rc3 instead of
1.0.0rc2.
Secure HBase builds fail
Key: HBASE-5040
URL: https://issues.apache.org/jira/browse/HBASE-5040
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Zhihong Yu
Fix For: 0.92.0
Attachments: 5040.txt
I saw the following in HBase-0.92-security build #39:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
[ERROR] - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
{code}
The above was probably introduced by HBASE-5006
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5040) Secure HBase builds fail
[
https://issues.apache.org/jira/browse/HBASE-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5040:
-
Attachment: 5040-v2.txt
Upgrade secure and insecure to default to 1.0.0rc3
Secure HBase builds fail
Key: HBASE-5040
URL: https://issues.apache.org/jira/browse/HBASE-5040
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Zhihong Yu
Fix For: 0.92.0
Attachments: 5040-v2.txt, 5040.txt
I saw the following in HBase-0.92-security build #39:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
[ERROR] - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
{code}
The above was probably introduced by HBASE-5006
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5040) Secure HBase builds fail
[
https://issues.apache.org/jira/browse/HBASE-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-5040:
-
Resolution: Fixed
Assignee: stack
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed branch and trunk. Checked that both build. Will see how things do
when this commits on trunk, branches, and security up on jenkins.
Secure HBase builds fail
Key: HBASE-5040
URL: https://issues.apache.org/jira/browse/HBASE-5040
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Zhihong Yu
Assignee: stack
Fix For: 0.92.0
Attachments: 5040-v2.txt, 5040.txt
I saw the following in HBase-0.92-security build #39:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
[ERROR] - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
{code}
The above was probably introduced by HBASE-5006
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4605) Constraints
[
https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171708#comment-13171708
]
stack commented on HBASE-4605:
--
Ok. Thanks lads. Good there is not extra dependency client-side (its a pain
MR'ing)
I'll do a bit of review.
Constraints
---
Key: HBASE-4605
URL: https://issues.apache.org/jira/browse/HBASE-4605
Project: HBase
Issue Type: Improvement
Components: client, coprocessors
Affects Versions: 0.94.0
Reporter: Jesse Yates
Assignee: Jesse Yates
Attachments: 4605-v6.txt, 4605.v7, constraint_as_cp.txt,
java_Constraint_v2.patch, java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch,
java_HBASE-4605_v3.patch, java_HBASE-4605_v5.patch, java_HBASE-4605_v7.patch
From Jesse's comment on dev:
{quote}
What I would like to propose is a simple interface that people can use to
implement a 'constraint' (matching the classic database definition). This
would help ease of adoption by helping HBase more easily check that box, help
minimize code duplication across organizations, and lead to easier adoption.
Essentially, people would implement a 'Constraint' interface for checking
keys before they are put into a table. Puts that are valid get written to the
table, but if not people can will throw an exception that gets propagated
back to the client explaining why the put was invalid.
Constraints would be set on a per-table basis and the user would be expected
to ensure the jars containing the constraint are present on the machines
serving that table.
Yes, people could roll their own mechanism for doing this via coprocessors
each time, but this would make it easier to do so, so you only have to
implement a very minimal interface and not worry about the specifics.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4934) Display Master server and Regionserver start time on respective info servers.
[ https://issues.apache.org/jira/browse/HBASE-4934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4934: - Resolution: Fixed Fix Version/s: 0.92.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to branch and trunk. Display Master server and Regionserver start time on respective info servers. - Key: HBASE-4934 URL: https://issues.apache.org/jira/browse/HBASE-4934 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Minor Fix For: 0.92.0 Attachments: hbase-4934.patch, hmaster.png, hregion.png With operations like rolling restart or master failovers, it is difficult to tell if a server is the old instance or the new restarted instance. Adding a start date stamp on the info web pages would be helpful for determining this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5061) HFileLocalityChecker
HFileLocalityChecker Key: HBASE-5061 URL: https://issues.apache.org/jira/browse/HBASE-5061 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor org.apache.hadoop.hbase.HFileLocalityChecker [options] A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage. Where options are: |-f file|Analyze a store file| |-r region|Analyze all store files for the region| |-t table|Analyze all store files for regions of the table served by the local regionserver| |-h host|Consider host local, defaults to the local host| |-v|Verbose operation| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5058) Allow HBaseAmin to use an existing connection
[ https://issues.apache.org/jira/browse/HBASE-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5058: - Status: Open (was: Patch Available) Allow HBaseAmin to use an existing connection - Key: HBASE-5058 URL: https://issues.apache.org/jira/browse/HBASE-5058 Project: HBase Issue Type: Sub-task Components: client Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5058.txt What HBASE-4805 does for HTables, this should do for HBaseAdmin. Along with this the shared error handling and retrying between HBaseAdmin and HConnectionManager can also be improved. I'll attach a first pass patch soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5061) HFileLocalityChecker
[ https://issues.apache.org/jira/browse/HBASE-5061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171715#comment-13171715 ] Lars Hofhansl commented on HBASE-5061: -- Cool idea. +1 HFileLocalityChecker Key: HBASE-5061 URL: https://issues.apache.org/jira/browse/HBASE-5061 Project: HBase Issue Type: New Feature Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor org.apache.hadoop.hbase.HFileLocalityChecker [options] A tool to report the number of local and nonlocal HFile blocks, and the ratio of as a percentage. Where options are: |-f file|Analyze a store file| |-r region|Analyze all store files for the region| |-t table|Analyze all store files for regions of the table served by the local regionserver| |-h host|Consider host local, defaults to the local host| |-v|Verbose operation| -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5062) [rest] Missing logon if security is enabled
[rest] Missing logon if security is enabled --- Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Somehow the attached change is missing from the security integration. The corresponding change to the Thrift server is in place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Component/s: thrift security rest Description: Somehow the attached changes are missing from the security integration. (was: Somehow the attached change is missing from the security integration. The corresponding change to the Thrift server is in place.) Summary: Missing logons if security is enabled (was: [rest] Missing logon if security is enabled) Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Attachment: HBASE-5062.patch Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Status: Patch Available (was: Open) Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5058) Allow HBaseAmin to use an existing connection
[ https://issues.apache.org/jira/browse/HBASE-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-5058. -- Resolution: Invalid Marking this as invalid. HBaseAdmin is not used nearly as often as HTable, and a bit churn through HConnction is not so bad. There are layers of retrying (which is bad) and RPC runtime exceptions are passed to up to HBaseAdmin (also bad). But none of those are horrible. The entire client needs to be revisited, this is not the jira to do that. Allow HBaseAmin to use an existing connection - Key: HBASE-5058 URL: https://issues.apache.org/jira/browse/HBASE-5058 Project: HBase Issue Type: Sub-task Components: client Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5058.txt What HBASE-4805 does for HTables, this should do for HBaseAdmin. Along with this the shared error handling and retrying between HBaseAdmin and HConnectionManager can also be improved. I'll attach a first pass patch soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171719#comment-13171719 ] Hadoop QA commented on HBASE-5062: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507816/HBASE-5062.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/532//console This message is automatically generated. Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-5058) Allow HBaseAmin to use an existing connection
[ https://issues.apache.org/jira/browse/HBASE-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl reopened HBASE-5058: -- Actually, never mind. I have a simple patch now, that bandaids the problem. Allow HBaseAmin to use an existing connection - Key: HBASE-5058 URL: https://issues.apache.org/jira/browse/HBASE-5058 Project: HBase Issue Type: Sub-task Components: client Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5058.txt What HBASE-4805 does for HTables, this should do for HBaseAdmin. Along with this the shared error handling and retrying between HBaseAdmin and HConnectionManager can also be improved. I'll attach a first pass patch soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5058) Allow HBaseAmin to use an existing connection
[ https://issues.apache.org/jira/browse/HBASE-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5058: - Attachment: 5058-v2.txt Simpler patch. Just avoid throwing RPC related runtime exceptions up to the client along with a simple new constructor for HBaseAdmin. Allow HBaseAmin to use an existing connection - Key: HBASE-5058 URL: https://issues.apache.org/jira/browse/HBASE-5058 Project: HBase Issue Type: Sub-task Components: client Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5058-v2.txt, 5058.txt What HBASE-4805 does for HTables, this should do for HBaseAdmin. Along with this the shared error handling and retrying between HBaseAdmin and HConnectionManager can also be improved. I'll attach a first pass patch soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Attachment: HBASE-5062-v2.patch v2 patch: Since REST and Thrift are not authenticating for HDFS access, only HBase secure RPC, additionally check if _HBase_ authentication is set to kerberos. Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062-v2.patch, HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171730#comment-13171730 ] Hadoop QA commented on HBASE-5062: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507819/HBASE-5062-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/533//console This message is automatically generated. Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062-v2.patch, HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171733#comment-13171733 ] Andrew Purtell commented on HBASE-5062: --- Deleted invalid HadoopQA results. Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062-v2.patch, HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507816/HBASE-5062.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/532//console This message is automatically generated.) Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062-v2.patch, HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5062) Missing logons if security is enabled
[ https://issues.apache.org/jira/browse/HBASE-5062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-5062: -- Comment: was deleted (was: -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507819/HBASE-5062-v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/533//console This message is automatically generated.) Missing logons if security is enabled - Key: HBASE-5062 URL: https://issues.apache.org/jira/browse/HBASE-5062 Project: HBase Issue Type: Bug Components: rest, security, thrift Affects Versions: 0.92.0 Reporter: Andrew Purtell Assignee: Andrew Purtell Attachments: HBASE-5062-v2.patch, HBASE-5062.patch Somehow the attached changes are missing from the security integration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5060) HBase client is blocked forever
[ https://issues.apache.org/jira/browse/HBASE-5060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] gaojinchao updated HBASE-5060: -- Attachment: HBASE-5060_Branch90trial.patch 1.This issue is diffcult to fix. I made a trial version. 2.I have checked all using ZooKeeperNodeTracker, It seems fine. HBase client is blocked forever --- Key: HBASE-5060 URL: https://issues.apache.org/jira/browse/HBASE-5060 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.90.4 Reporter: gaojinchao Assignee: gaojinchao Priority: Critical Fix For: 0.90.6 Attachments: HBASE-5060_Branch90trial.patch Since the client had a temporary network failure, After it recovered. I found my client thread was blocked. Looks below stack and logs, It said that we use a invalid CatalogTracker in function tableExists. Block stack: WriteHbaseThread33 prio=10 tid=0x7f76bc27a800 nid=0x2540 in Object.wait() [0x7f76af4f3000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:331) - locked 0x7f7a67817c98 (a java.util.concurrent.atomic.AtomicBoolean) at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:366) at org.apache.hadoop.hbase.catalog.MetaReader.tableExists(MetaReader.java:427) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:164) at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source) at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source) - locked 0x7f7a4c5dc578 (a com.huawei.hdi.hbase.HbaseReOper) at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source) at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source) In ZooKeeperNodeTracker, We don't throw the KeeperException to high level. So in CatalogTracker level, We think ZooKeeperNodeTracker start success and continue to process . [WriteHbaseThread33]2011-12-16 17:07:33,153[WARN ] | hconnection-0x334129cf6890051-0x334129cf6890051-0x334129cf6890051 Unable to get data of znode /hbase/root-region-server | org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:557) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server at org.apache.zookeeper.KeeperException.create(KeeperException.java:90) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:931) at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:549) at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:73) at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:136) at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:111) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:162) at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source) at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source) at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source) at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source) [WriteHbaseThread33]2011-12-16 17:07:33,361[ERROR] | hconnection-0x334129cf6890051-0x334129cf6890051-0x334129cf6890051 Received unexpected KeeperException, re-throwing exception | org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.keeperException(ZooKeeperWatcher.java:385) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server at org.apache.zookeeper.KeeperException.create(KeeperException.java:90) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:931) at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:549) at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:73) at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:136) at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:111) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:162) at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source) at
[jira] [Created] (HBASE-5063) RegionServers fail to report to backup HMaster after primary goes down.
RegionServers fail to report to backup HMaster after primary goes down. --- Key: HBASE-5063 URL: https://issues.apache.org/jira/browse/HBASE-5063 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Priority: Critical # Setup cluster with two HMasters # Observe that HM1 is up and that all RS's are in the RegionServer list on web page. # Kill (not even -9) the active HMaster # Wait for ZK to time out (default 3 minutes). # Observe that HM2 is now active. Tables may show up but RegionServers never report on web page. Existing connections are fine. New connections cannot find regionservers. Note: * If we replace a new HM1 in the same place and kill HM2, the cluster functions normally again after recovery. This sees to indicate that regionservers are stuck trying to talk to the old HM1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5040) Secure HBase builds fail
[
https://issues.apache.org/jira/browse/HBASE-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171761#comment-13171761
]
Hudson commented on HBASE-5040:
---
Integrated in HBase-0.92 #194 (See
[https://builds.apache.org/job/HBase-0.92/194/])
HBASE-5040 Secure HBase builds fail
HBASE-5040 Secure HBase builds fail
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
stack :
Files :
* /hbase/branches/0.92/pom.xml
Secure HBase builds fail
Key: HBASE-5040
URL: https://issues.apache.org/jira/browse/HBASE-5040
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Zhihong Yu
Assignee: stack
Fix For: 0.92.0
Attachments: 5040-v2.txt, 5040.txt
I saw the following in HBase-0.92-security build #39:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
[ERROR] - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
{code}
The above was probably introduced by HBASE-5006
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4934) Display Master server and Regionserver start time on respective info servers.
[ https://issues.apache.org/jira/browse/HBASE-4934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171762#comment-13171762 ] Hudson commented on HBASE-4934: --- Integrated in HBase-0.92 #194 (See [https://builds.apache.org/job/HBase-0.92/194/]) HBASE-4934 Display Master server and Regionserver start time on respective info servers stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/jamon/org/apache/hbase/tmpl/master/MasterStatusTmpl.jamon * /hbase/branches/0.92/src/main/jamon/org/apache/hbase/tmpl/regionserver/RSStatusTmpl.jamon * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Display Master server and Regionserver start time on respective info servers. - Key: HBASE-4934 URL: https://issues.apache.org/jira/browse/HBASE-4934 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Minor Fix For: 0.92.0 Attachments: hbase-4934.patch, hmaster.png, hregion.png With operations like rolling restart or master failovers, it is difficult to tell if a server is the old instance or the new restarted instance. Adding a start date stamp on the info web pages would be helpful for determining this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5063) RegionServers fail to report to backup HMaster after primary goes down.
[ https://issues.apache.org/jira/browse/HBASE-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-5063: -- Attachment: HBASE-5063.patch No test case yet but this seems to fix the problem. RegionServers fail to report to backup HMaster after primary goes down. --- Key: HBASE-5063 URL: https://issues.apache.org/jira/browse/HBASE-5063 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Priority: Critical Attachments: HBASE-5063.patch # Setup cluster with two HMasters # Observe that HM1 is up and that all RS's are in the RegionServer list on web page. # Kill (not even -9) the active HMaster # Wait for ZK to time out (default 3 minutes). # Observe that HM2 is now active. Tables may show up but RegionServers never report on web page. Existing connections are fine. New connections cannot find regionservers. Note: * If we replace a new HM1 in the same place and kill HM2, the cluster functions normally again after recovery. This sees to indicate that regionservers are stuck trying to talk to the old HM1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5063) RegionServers fail to report to backup HMaster after primary goes down.
[ https://issues.apache.org/jira/browse/HBASE-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh reassigned HBASE-5063: - Assignee: Jonathan Hsieh RegionServers fail to report to backup HMaster after primary goes down. --- Key: HBASE-5063 URL: https://issues.apache.org/jira/browse/HBASE-5063 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Critical Attachments: HBASE-5063.patch # Setup cluster with two HMasters # Observe that HM1 is up and that all RS's are in the RegionServer list on web page. # Kill (not even -9) the active HMaster # Wait for ZK to time out (default 3 minutes). # Observe that HM2 is now active. Tables may show up but RegionServers never report on web page. Existing connections are fine. New connections cannot find regionservers. Note: * If we replace a new HM1 in the same place and kill HM2, the cluster functions normally again after recovery. This sees to indicate that regionservers are stuck trying to talk to the old HM1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5063) RegionServers fail to report to backup HMaster after primary goes down.
[
https://issues.apache.org/jira/browse/HBASE-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171765#comment-13171765
]
Jonathan Hsieh commented on HBASE-5063:
---
Here's the exception -- unfortunately it doesn't say which master it is unable
to connect to.
{code}
11/12/17 18:50:24 WARN regionserver.HRegionServer: Unable to connect to master.
Retrying. Error was:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupConnection(HBaseClient.java:328)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:362)
at
org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1024)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:876)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
at $Proxy8.getProtocolVersion(Unknown Source)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1616)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:787)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:674)
at java.lang.Thread.run(Thread.java:619)
{code}
RegionServers fail to report to backup HMaster after primary goes down.
---
Key: HBASE-5063
URL: https://issues.apache.org/jira/browse/HBASE-5063
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Critical
Attachments: HBASE-5063.patch
# Setup cluster with two HMasters
# Observe that HM1 is up and that all RS's are in the RegionServer list on
web page.
# Kill (not even -9) the active HMaster
# Wait for ZK to time out (default 3 minutes).
# Observe that HM2 is now active. Tables may show up but RegionServers never
report on web page. Existing connections are fine. New connections cannot
find regionservers.
Note:
* If we replace a new HM1 in the same place and kill HM2, the cluster
functions normally again after recovery. This sees to indicate that
regionservers are stuck trying to talk to the old HM1.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5060) HBase client is blocked forever
[ https://issues.apache.org/jira/browse/HBASE-5060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171767#comment-13171767 ] Zhihong Yu commented on HBASE-5060: --- @Jinchao: Can you attach a patch for TRUNK ? HBase client is blocked forever --- Key: HBASE-5060 URL: https://issues.apache.org/jira/browse/HBASE-5060 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.90.4 Reporter: gaojinchao Assignee: gaojinchao Priority: Critical Fix For: 0.90.6 Attachments: HBASE-5060_Branch90trial.patch Since the client had a temporary network failure, After it recovered. I found my client thread was blocked. Looks below stack and logs, It said that we use a invalid CatalogTracker in function tableExists. Block stack: WriteHbaseThread33 prio=10 tid=0x7f76bc27a800 nid=0x2540 in Object.wait() [0x7f76af4f3000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:331) - locked 0x7f7a67817c98 (a java.util.concurrent.atomic.AtomicBoolean) at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:366) at org.apache.hadoop.hbase.catalog.MetaReader.tableExists(MetaReader.java:427) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:164) at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source) at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source) - locked 0x7f7a4c5dc578 (a com.huawei.hdi.hbase.HbaseReOper) at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source) at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source) In ZooKeeperNodeTracker, We don't throw the KeeperException to high level. So in CatalogTracker level, We think ZooKeeperNodeTracker start success and continue to process . [WriteHbaseThread33]2011-12-16 17:07:33,153[WARN ] | hconnection-0x334129cf6890051-0x334129cf6890051-0x334129cf6890051 Unable to get data of znode /hbase/root-region-server | org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:557) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server at org.apache.zookeeper.KeeperException.create(KeeperException.java:90) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:931) at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:549) at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:73) at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:136) at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:111) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:162) at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source) at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source) at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source) at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source) [WriteHbaseThread33]2011-12-16 17:07:33,361[ERROR] | hconnection-0x334129cf6890051-0x334129cf6890051-0x334129cf6890051 Received unexpected KeeperException, re-throwing exception | org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.keeperException(ZooKeeperWatcher.java:385) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server at org.apache.zookeeper.KeeperException.create(KeeperException.java:90) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:931) at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:549) at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:73) at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:136) at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:111) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:162) at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source) at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source) at
[jira] [Updated] (HBASE-5063) RegionServers fail to report to backup HMaster after primary goes down.
[ https://issues.apache.org/jira/browse/HBASE-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Yu updated HBASE-5063: -- Status: Patch Available (was: Open) RegionServers fail to report to backup HMaster after primary goes down. --- Key: HBASE-5063 URL: https://issues.apache.org/jira/browse/HBASE-5063 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Critical Attachments: HBASE-5063.patch # Setup cluster with two HMasters # Observe that HM1 is up and that all RS's are in the RegionServer list on web page. # Kill (not even -9) the active HMaster # Wait for ZK to time out (default 3 minutes). # Observe that HM2 is now active. Tables may show up but RegionServers never report on web page. Existing connections are fine. New connections cannot find regionservers. Note: * If we replace a new HM1 in the same place and kill HM2, the cluster functions normally again after recovery. This sees to indicate that regionservers are stuck trying to talk to the old HM1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5058) Allow HBaseAmin to use an existing connection
[ https://issues.apache.org/jira/browse/HBASE-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5058: - Status: Patch Available (was: Reopened) Allow HBaseAmin to use an existing connection - Key: HBASE-5058 URL: https://issues.apache.org/jira/browse/HBASE-5058 Project: HBase Issue Type: Sub-task Components: client Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5058-v2.txt, 5058.txt What HBASE-4805 does for HTables, this should do for HBaseAdmin. Along with this the shared error handling and retrying between HBaseAdmin and HConnectionManager can also be improved. I'll attach a first pass patch soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4934) Display Master server and Regionserver start time on respective info servers.
[ https://issues.apache.org/jira/browse/HBASE-4934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171772#comment-13171772 ] Hudson commented on HBASE-4934: --- Integrated in HBase-TRUNK #2554 (See [https://builds.apache.org/job/HBase-TRUNK/2554/]) HBASE-4934 Display Master server and Regionserver start time on respective info servers stack : Files : * /hbase/trunk/src/main/jamon/org/apache/hbase/tmpl/master/MasterStatusTmpl.jamon * /hbase/trunk/src/main/jamon/org/apache/hbase/tmpl/regionserver/RSStatusTmpl.jamon * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Display Master server and Regionserver start time on respective info servers. - Key: HBASE-4934 URL: https://issues.apache.org/jira/browse/HBASE-4934 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Minor Fix For: 0.92.0 Attachments: hbase-4934.patch, hmaster.png, hregion.png With operations like rolling restart or master failovers, it is difficult to tell if a server is the old instance or the new restarted instance. Adding a start date stamp on the info web pages would be helpful for determining this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5040) Secure HBase builds fail
[
https://issues.apache.org/jira/browse/HBASE-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171771#comment-13171771
]
Hudson commented on HBASE-5040:
---
Integrated in HBase-TRUNK #2554 (See
[https://builds.apache.org/job/HBase-TRUNK/2554/])
HBASE-5040 Secure HBase builds fail
stack :
Files :
* /hbase/trunk/pom.xml
Secure HBase builds fail
Key: HBASE-5040
URL: https://issues.apache.org/jira/browse/HBASE-5040
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Zhihong Yu
Assignee: stack
Fix For: 0.92.0
Attachments: 5040-v2.txt, 5040.txt
I saw the following in HBase-0.92-security build #39:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
[ERROR] - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
{code}
The above was probably introduced by HBASE-5006
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5060) HBase client is blocked forever
[ https://issues.apache.org/jira/browse/HBASE-5060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171778#comment-13171778 ] ramkrishna.s.vasudevan commented on HBASE-5060: --- +1 on patch... good work gao.. HBase client is blocked forever --- Key: HBASE-5060 URL: https://issues.apache.org/jira/browse/HBASE-5060 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.90.4 Reporter: gaojinchao Assignee: gaojinchao Priority: Critical Fix For: 0.90.6 Attachments: HBASE-5060_Branch90trial.patch Since the client had a temporary network failure, After it recovered. I found my client thread was blocked. Looks below stack and logs, It said that we use a invalid CatalogTracker in function tableExists. Block stack: WriteHbaseThread33 prio=10 tid=0x7f76bc27a800 nid=0x2540 in Object.wait() [0x7f76af4f3000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:331) - locked 0x7f7a67817c98 (a java.util.concurrent.atomic.AtomicBoolean) at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:366) at org.apache.hadoop.hbase.catalog.MetaReader.tableExists(MetaReader.java:427) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:164) at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source) at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source) - locked 0x7f7a4c5dc578 (a com.huawei.hdi.hbase.HbaseReOper) at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source) at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source) In ZooKeeperNodeTracker, We don't throw the KeeperException to high level. So in CatalogTracker level, We think ZooKeeperNodeTracker start success and continue to process . [WriteHbaseThread33]2011-12-16 17:07:33,153[WARN ] | hconnection-0x334129cf6890051-0x334129cf6890051-0x334129cf6890051 Unable to get data of znode /hbase/root-region-server | org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:557) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server at org.apache.zookeeper.KeeperException.create(KeeperException.java:90) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:931) at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:549) at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:73) at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:136) at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:111) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:162) at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source) at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source) at com.huawei.hdi.hbase.HbaseFileOperate.writeToHbase(Unknown Source) at com.huawei.hdi.hbase.WriteHbaseThread.run(Unknown Source) [WriteHbaseThread33]2011-12-16 17:07:33,361[ERROR] | hconnection-0x334129cf6890051-0x334129cf6890051-0x334129cf6890051 Received unexpected KeeperException, re-throwing exception | org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.keeperException(ZooKeeperWatcher.java:385) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/root-region-server at org.apache.zookeeper.KeeperException.create(KeeperException.java:90) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:931) at org.apache.hadoop.hbase.zookeeper.ZKUtil.getDataAndWatch(ZKUtil.java:549) at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:73) at org.apache.hadoop.hbase.catalog.CatalogTracker.start(CatalogTracker.java:136) at org.apache.hadoop.hbase.client.HBaseAdmin.getCatalogTracker(HBaseAdmin.java:111) at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:162) at com.huawei.hdi.hbase.HbaseFileOperate.checkHtableState(Unknown Source) at com.huawei.hdi.hbase.HbaseReOper.reCreateHtable(Unknown Source) at
[jira] [Commented] (HBASE-5063) RegionServers fail to report to backup HMaster after primary goes down.
[ https://issues.apache.org/jira/browse/HBASE-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171779#comment-13171779 ] Hadoop QA commented on HBASE-5063: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507826/HBASE-5063.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -152 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 76 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestInstantSchemaChange org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/534//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/534//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/534//console This message is automatically generated. RegionServers fail to report to backup HMaster after primary goes down. --- Key: HBASE-5063 URL: https://issues.apache.org/jira/browse/HBASE-5063 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Critical Attachments: HBASE-5063.patch # Setup cluster with two HMasters # Observe that HM1 is up and that all RS's are in the RegionServer list on web page. # Kill (not even -9) the active HMaster # Wait for ZK to time out (default 3 minutes). # Observe that HM2 is now active. Tables may show up but RegionServers never report on web page. Existing connections are fine. New connections cannot find regionservers. Note: * If we replace a new HM1 in the same place and kill HM2, the cluster functions normally again after recovery. This sees to indicate that regionservers are stuck trying to talk to the old HM1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5058) Allow HBaseAmin to use an existing connection
[ https://issues.apache.org/jira/browse/HBASE-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171783#comment-13171783 ] Hadoop QA commented on HBASE-5058: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12507817/5058-v2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -152 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 76 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestLogRolling org.apache.hadoop.hbase.client.TestInstantSchemaChange Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/535//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/535//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/535//console This message is automatically generated. Allow HBaseAmin to use an existing connection - Key: HBASE-5058 URL: https://issues.apache.org/jira/browse/HBASE-5058 Project: HBase Issue Type: Sub-task Components: client Affects Versions: 0.94.0 Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: 5058-v2.txt, 5058.txt What HBASE-4805 does for HTables, this should do for HBaseAdmin. Along with this the shared error handling and retrying between HBaseAdmin and HConnectionManager can also be improved. I'll attach a first pass patch soon. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5040) Secure HBase builds fail
[
https://issues.apache.org/jira/browse/HBASE-5040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171790#comment-13171790
]
Hudson commented on HBASE-5040:
---
Integrated in HBase-TRUNK-security #36 (See
[https://builds.apache.org/job/HBase-TRUNK-security/36/])
HBASE-5040 Secure HBase builds fail
stack :
Files :
* /hbase/trunk/pom.xml
Secure HBase builds fail
Key: HBASE-5040
URL: https://issues.apache.org/jira/browse/HBASE-5040
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Zhihong Yu
Assignee: stack
Fix For: 0.92.0
Attachments: 5040-v2.txt, 5040.txt
I saw the following in HBase-0.92-security build #39:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
[ERROR] - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
https://builds.apache.org/job/HBase-0.92-security/ws/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java:[590,4]
method does not override or implement a method from a supertype
{code}
The above was probably introduced by HBASE-5006
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4934) Display Master server and Regionserver start time on respective info servers.
[ https://issues.apache.org/jira/browse/HBASE-4934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171791#comment-13171791 ] Hudson commented on HBASE-4934: --- Integrated in HBase-TRUNK-security #36 (See [https://builds.apache.org/job/HBase-TRUNK-security/36/]) HBASE-4934 Display Master server and Regionserver start time on respective info servers stack : Files : * /hbase/trunk/src/main/jamon/org/apache/hbase/tmpl/master/MasterStatusTmpl.jamon * /hbase/trunk/src/main/jamon/org/apache/hbase/tmpl/regionserver/RSStatusTmpl.jamon * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java Display Master server and Regionserver start time on respective info servers. - Key: HBASE-4934 URL: https://issues.apache.org/jira/browse/HBASE-4934 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Minor Fix For: 0.92.0 Attachments: hbase-4934.patch, hmaster.png, hregion.png With operations like rolling restart or master failovers, it is difficult to tell if a server is the old instance or the new restarted instance. Adding a start date stamp on the info web pages would be helpful for determining this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5064) use surefire tests parallelization
use surefire tests parallelization -- Key: HBASE-5064 URL: https://issues.apache.org/jira/browse/HBASE-5064 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Reporter: nkeywal Assignee: nkeywal Priority: Minor To be tried multiple times on hadoop-qa before committing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
