[jira] [Commented] (HBASE-3967) Support deletes in HFileOutputFormat based bulk import mechanism
[ https://issues.apache.org/jira/browse/HBASE-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258599#comment-13258599 ] Lars Hofhansl commented on HBASE-3967: -- HFileOutputFormat just stores KeyValues so it already handles Deletes. The question is: How do you get Deletes output from a Mapper? One solution (the one I took in HBASE-5440) is to have Mapper emit KeyValues and then use KeyValueSortReducer in the reduce phase. Another approach is this jira, which is what Facebook has internally (as far as I know). Yet another approach could be use Mutation (which is new in 0.92), and write a new SortReducer. I think the point of this jira is to get the Facebook approach into 0.94/trunk in order to make upgrading more palatable for Facebook. Kannan, correct me if I am wrong. Support deletes in HFileOutputFormat based bulk import mechanism Key: HBASE-3967 URL: https://issues.apache.org/jira/browse/HBASE-3967 Project: HBase Issue Type: Sub-task Reporter: Kannan Muthukkaruppan Priority: Critical Fix For: 0.96.0 Attachments: diff.patch During bulk imports, it'll be useful to be able to do delete mutations (either to delete data that already exists in HBase or was inserted earlier during this run of the import). For example, we have a use case, where we are processing a log of data which may have both inserts and deletes in the mix and we want to upload that into HBase using the bulk import mechanism. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
Lars Hofhansl created HBASE-5848: Summary: Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3792) TableInputFormat leaks ZK connections
[ https://issues.apache.org/jira/browse/HBASE-3792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258739#comment-13258739 ] Lars Hofhansl commented on HBASE-3792: -- @Patrik: In 0.92+ you can create a standalone HConnection (HConnectionManager.createConnection(...)) and then create HTables using that HConnection. See HBASE-4805. TableInputFormat leaks ZK connections - Key: HBASE-3792 URL: https://issues.apache.org/jira/browse/HBASE-3792 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.90.1 Environment: Java 1.6.0_24, Mac OS X 10.6.7 Reporter: Bryan Keller Attachments: patch0.90.4, tableinput.patch The TableInputFormat creates an HTable using a new Configuration object, and it never cleans it up. When running a Mapper, the TableInputFormat is instantiated and the ZK connection is created. While this connection is not explicitly cleaned up, the Mapper process eventually exits and thus the connection is closed. Ideally the TableRecordReader would close the connection in its close() method rather than relying on the process to die for connection cleanup. This is fairly easy to implement by overriding TableRecordReader, and also overriding TableInputFormat to specify the new record reader. The leak occurs when the JobClient is initializing and needs to retrieves the splits. To get the splits, it instantiates a TableInputFormat. Doing so creates a ZK connection that is never cleaned up. Unlike the mapper, however, my job client process does not die. Thus the ZK connections accumulate. I was able to fix the problem by writing my own TableInputFormat that does not initialize the HTable in the getConf() method and does not have an HTable member variable. Rather, it has a variable for the table name. The HTable is instantiated where needed and then cleaned up. For example, in the getSplits() method, I create the HTable, then close the connection once the splits are retrieved. I also create the HTable when creating the record reader, and I have a record reader that closes the connection when done. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258740#comment-13258740
]
Lars Hofhansl commented on HBASE-5848:
--
Specifically
{code}
admin.createTable(htd, byte[][] {HConstants.EMPTY_START_ROW})
{code}
Will cause the HMaster to abort.
Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to
abort
Key: HBASE-5848
URL: https://issues.apache.org/jira/browse/HBASE-5848
Project: HBase
Issue Type: Bug
Components: client
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
A coworker of mine just had this scenario. It does not make sense the
EMPTY_START_ROW as splitKey (since the region with the empty start key is
implicit), but it should not cause the HMaster to abort.
The abort happens because it tries to bulk assign the same region twice and
then runs into race conditions with ZK.
The same would (presumably) happen when two identical split keys are passed,
but the client blocks that. The simplest solution here is to also block
passed null or EMPTY_START_ROW as split key by the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13258784#comment-13258784
]
Lars Hofhansl commented on HBASE-5848:
--
@Ram: Are you sure?
This one liner caused the HMaster to abort in every cluster I tried:
{code}
new HBaseAdmin(HBaseConfiguration.create()).createTable(new
HTableDescriptor(x), new byte[][] {HConstants.EMPTY_START_ROW});
{code}
Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to
abort
Key: HBASE-5848
URL: https://issues.apache.org/jira/browse/HBASE-5848
Project: HBase
Issue Type: Bug
Components: client
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
A coworker of mine just had this scenario. It does not make sense the
EMPTY_START_ROW as splitKey (since the region with the empty start key is
implicit), but it should not cause the HMaster to abort.
The abort happens because it tries to bulk assign the same region twice and
then runs into race conditions with ZK.
The same would (presumably) happen when two identical split keys are passed,
but the client blocks that. The simplest solution here is to also block
passed null or EMPTY_START_ROW as split key by the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5850) Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized
[ https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5850: - Fix Version/s: (was: 0.94.0) HBASE-5454 is in 0.94 already. Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized --- Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2 Attachments: backport-5454(createTable)-to-94.patch, backport-5454(createTable)-to-94_surefire-report.html, backport-5454(createTable)-to-trunk.patch, backport-5454(createTable)-to-trunk_surefire-report.html, backport-5454-to-90.patch, backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5850) Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized
[ https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5850: - Fix Version/s: 0.94.1 I see. Thanks xufeng. I assume this does not sink the RC, so I am targetting this for 0.94.1. Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized --- Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2, 0.94.1 Attachments: backport-5454(createTable)-to-94.patch, backport-5454(createTable)-to-94_surefire-report.html, backport-5454(createTable)-to-trunk.patch, backport-5454(createTable)-to-trunk_surefire-report.html, backport-5454-to-90.patch, backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13259392#comment-13259392 ] Lars Hofhansl commented on HBASE-5848: -- Interestingly, I cannot reproduce this in a test. MiniCluster seems to be different enough to not have this problem. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-1936) ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services
[ https://issues.apache.org/jira/browse/HBASE-1936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13259395#comment-13259395 ] Lars Hofhansl commented on HBASE-1936: -- Thank Jieshan. Yes, I think you should be able to reuse most of the CP classloading code. In the end you only need a classloader which can load a jar from HDFS. Filters (like coprocessors) then would be packed into a jar (together with any supporting classes needed). ClassLoader that loads from hdfs; useful adding filters to classpath without having to restart services --- Key: HBASE-1936 URL: https://issues.apache.org/jira/browse/HBASE-1936 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Jieshan Bean Labels: noob Attachments: HBASE-1936-trunk(forReview).patch, cp_from_hdfs.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260016#comment-13260016
]
Lars Hofhansl commented on HBASE-5848:
--
Thanks for the patch Ram.
The problem also occurs with
{code}
createTable(new HTableDescriptor(x), new byte[][]
{HConstants.EMPTY_START_ROW, new byte[] {1,2,3}});
{code}
I do not think the patch would address that.
Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to
abort
Key: HBASE-5848
URL: https://issues.apache.org/jira/browse/HBASE-5848
Project: HBase
Issue Type: Bug
Components: client
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Attachments: HBASE-5848.patch
A coworker of mine just had this scenario. It does not make sense the
EMPTY_START_ROW as splitKey (since the region with the empty start key is
implicit), but it should not cause the HMaster to abort.
The abort happens because it tries to bulk assign the same region twice and
then runs into race conditions with ZK.
The same would (presumably) happen when two identical split keys are passed,
but the client blocks that. The simplest solution here is to also block
passed null or EMPTY_START_ROW as split key by the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5850) Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized
[ https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5850: - Fix Version/s: (was: 0.94.1) 0.94.0 Pulling back to 0.94.0, as RC2 was sunk. Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized --- Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2, 0.94.0 Attachments: 5850-trunk.txt, No_patch_90_surefire-report.html, backport-5454(createTable)-to-94.patch, backport-5454(createTable)-to-94_surefire-report.html, backport-5454(createTable)-to-trunk.patch, backport-5454(createTable)-to-trunk_surefire-report.html, backport-5454-to-90-surefire-report.html, backport-5454-to-90.patch, backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260040#comment-13260040 ] Lars Hofhansl commented on HBASE-5848: -- The client already checks for duplicate keys in splitKeys, we could just check for the empty key as well (no sense passing the empty key to createTable anyway), and throw an error as well. Obviously that does not guard against rogue clients and it would be extremely simple to DOS an HBase cluster. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Attachments: HBASE-5848.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260044#comment-13260044
]
Lars Hofhansl commented on HBASE-5861:
--
Just realized it was me who added this test as part of HBASE-5604... :(
I should fix it.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260069#comment-13260069
]
Lars Hofhansl commented on HBASE-5861:
--
I have a fix. There's util method in MapReduceTestUtil to create a dummy
TaskAttemptContext.
Will upload a patch momentarily.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5861:
-
Attachment: 5861.txt
Works fine for Hadoop 1.0.* and should work for 0.23.*.
For various other reasons I cannot currently build against hadoop 0.23, if
somebody could quick verify, that'd be great.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5833) 0.92 build has been failing pretty consistently on TestMasterFailover....
[ https://issues.apache.org/jira/browse/HBASE-5833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5833: - Fix Version/s: (was: 0.94.1) 0.94.0 0.92 build has been failing pretty consistently on TestMasterFailover - Key: HBASE-5833 URL: https://issues.apache.org/jira/browse/HBASE-5833 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.90.7, 0.92.2, 0.94.0 Attachments: 5833-trunk.txt, 5833-v2.092.txt, 5833.txt, 5833v3092.txt, 5833v4090.txt, 5833v4092.txt, 5833v4trunk.txt, 5833v5094.txt, 5833v5trunk.txt, closehregions.txt Trunk seems fine but 0.92 fails on this test pretty regularly. Running it local it seems to hang for me. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5787) Table owner can't disable/delete his/her own table
[
https://issues.apache.org/jira/browse/HBASE-5787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5787:
-
Fix Version/s: (was: 0.94.1)
0.94.0
Table owner can't disable/delete his/her own table
--
Key: HBASE-5787
URL: https://issues.apache.org/jira/browse/HBASE-5787
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.92.1, 0.94.0, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Priority: Minor
Labels: acl, security
Fix For: 0.92.2, 0.94.0, 0.96.0
Attachments: HBASE-5787-tests-wrong-names.patch, HBASE-5787-v0.patch,
HBASE-5787-v1.patch
An user with CREATE privileges can create a table, but can not disable it,
because disable operation require ADMIN privileges. Also if a table is
already disabled, anyone can remove it.
{code}
public void preDeleteTable(ObserverContextMasterCoprocessorEnvironment c,
byte[] tableName) throws IOException {
requirePermission(Permission.Action.CREATE);
}
public void preDisableTable(ObserverContextMasterCoprocessorEnvironment c,
byte[] tableName) throws IOException {
/* TODO: Allow for users with global CREATE permission and the table owner
*/
requirePermission(Permission.Action.ADMIN);
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5737) Minor Improvements related to balancer.
[ https://issues.apache.org/jira/browse/HBASE-5737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5737: - Fix Version/s: (was: 0.94.1) 0.94.0 Minor Improvements related to balancer. --- Key: HBASE-5737 URL: https://issues.apache.org/jira/browse/HBASE-5737 Project: HBase Issue Type: Bug Components: master Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: HBASE-5737.patch, HBASE-5737_1.patch, HBASE-5737_2.patch, HBASE-5737_3.patch Currently in Am.getAssignmentByTable() we use a result map which is currenly a hashmap. It could be better if we have a treeMap. Even in MetaReader.fullScan we have the treeMap only so that we have the naming order maintained. I felt this change could be very useful in cases where we are extending the DefaultLoadBalancer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5635) If getTaskList() returns null, splitlogWorker would go down and it won't serve any requests
[
https://issues.apache.org/jira/browse/HBASE-5635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5635:
-
Fix Version/s: (was: 0.94.1)
0.94.0
If getTaskList() returns null, splitlogWorker would go down and it won't
serve any requests
---
Key: HBASE-5635
URL: https://issues.apache.org/jira/browse/HBASE-5635
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 0.92.1
Reporter: Kristam Subba Swathi
Assignee: Chinna Rao Lalam
Fix For: 0.94.0, 0.96.0
Attachments: HBASE-5635.1.patch, HBASE-5635.2.patch,
HBASE-5635._trunk.patch, HBASE-5635.patch, HBASE-5635_0.94.patch
During the hlog split operation if all the zookeepers are down ,then the
paths will be returned as null and the splitworker thread wil be exited
Now this regionserver wil not be able to acquire any other tasks since the
splitworker thread is exited
Please find the attached code for more details
{code}
private ListString getTaskList() {
for (int i = 0; i zkretries; i++) {
try {
return (ZKUtil.listChildrenAndWatchForNewChildren(this.watcher,
this.watcher.splitLogZNode));
} catch (KeeperException e) {
LOG.warn(Could not get children of znode +
this.watcher.splitLogZNode, e);
try {
Thread.sleep(1000);
} catch (InterruptedException e1) {
LOG.warn(Interrupted while trying to get task list ..., e1);
Thread.currentThread().interrupt();
return null;
}
}
}
{code}
in the org.apache.hadoop.hbase.regionserver.SplitLogWorker
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260074#comment-13260074
]
Lars Hofhansl commented on HBASE-5861:
--
Ah nope, there's the JobContext too.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5857) RIT map in RS not getting cleared while region opening
[
https://issues.apache.org/jira/browse/HBASE-5857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5857:
-
Fix Version/s: (was: 0.94.1)
0.94.0
RIT map in RS not getting cleared while region opening
--
Key: HBASE-5857
URL: https://issues.apache.org/jira/browse/HBASE-5857
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Chinna Rao Lalam
Assignee: Chinna Rao Lalam
Fix For: 0.94.0, 0.96.0
Attachments: HBASE-5857_0.92.1.patch, HBASE-5857_0.92.patch,
HBASE-5857_0.94.1.patch, HBASE-5857_94.patch, HBASE-5857_trunk.1.patch,
HBASE-5857_trunk.1.patch, HBASE-5857_trunk.2.patch, HBASE-5857_trunk.patch
While opening the region in RS after adding the region to
regionsInTransitionInRS if tableDescriptors.get() throws exception the region
wont be cleared from regionsInTransitionInRS. So next time if it tries to
open the region in the same RS it will throw the
RegionAlreadyInTransitionException.
if swap the below statement this issue wont come.
{code}
this.regionsInTransitionInRS.putIfAbsent(region.getEncodedNameAsBytes(),true);
HTableDescriptor htd = this.tableDescriptors.get(region.getTableName());
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260077#comment-13260077
]
Lars Hofhansl commented on HBASE-5861:
--
@Jon: Tried that. For some reason I can 100's of compilation errors complaining
about missing hadoop packages.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260102#comment-13260102
]
Lars Hofhansl edited comment on HBASE-5861 at 4/24/12 12:07 AM:
Did you forget to include HadoopShim?
was (Author: lhofhansl):
Did you forgot to include HadoopShim?
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260077#comment-13260077
]
Lars Hofhansl edited comment on HBASE-5861 at 4/24/12 12:07 AM:
@Jon: Tried that. For some reason I get 100's of compilation errors complaining
about missing hadoop packages.
was (Author: lhofhansl):
@Jon: Tried that. For some reason I can 100's of compilation errors
complaining about missing hadoop packages.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260102#comment-13260102
]
Lars Hofhansl commented on HBASE-5861:
--
Did you forgot to include HadoopShim?
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5850) Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized
[ https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260103#comment-13260103 ] Lars Hofhansl commented on HBASE-5850: -- The patches look good to me. Anybody opposed if I committed them to all branches? Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized --- Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2, 0.94.0 Attachments: 5850-trunk.txt, No_patch_90_surefire-report.html, backport-5454(createTable)-to-94.patch, backport-5454(createTable)-to-94_surefire-report.html, backport-5454(createTable)-to-trunk.patch, backport-5454(createTable)-to-trunk_surefire-report.html, backport-5454-to-90-surefire-report.html, backport-5454-to-90.patch, backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5850) Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized
[ https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260119#comment-13260119 ] Lars Hofhansl commented on HBASE-5850: -- Or phrased differently: +1 on patches. Anybody else +1? Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized --- Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2, 0.94.0 Attachments: 5850-trunk.txt, No_patch_90_surefire-report.html, backport-5454(createTable)-to-94.patch, backport-5454(createTable)-to-94_surefire-report.html, backport-5454(createTable)-to-trunk.patch, backport-5454(createTable)-to-trunk_surefire-report.html, backport-5454-to-90-surefire-report.html, backport-5454-to-90.patch, backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260117#comment-13260117
]
Lars Hofhansl commented on HBASE-5861:
--
For TaskAttemptContext we could use Hadoop's MapReduceTestUtil class, right?
I.e. HadoopShim could use
MapReduceTestUtil.createDummyMapTaskAttemptContext(...)
For JobContext, why not use JobContextImpl in hadoop2?
Rest looks good.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260126#comment-13260126
]
Lars Hofhansl commented on HBASE-5861:
--
Problem above was due to Maven 2... G.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5861:
-
Attachment: 5782-v3.txt
How about this one. (slight variation of Jon's patch).
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5782-v3.txt, 5861.txt, hbase-5861-jon.patch,
hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260175#comment-13260175
]
Lars Hofhansl commented on HBASE-5861:
--
Whoops... Removed.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5861:
-
Attachment: (was: 5782-v3.txt)
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260188#comment-13260188
]
Lars Hofhansl commented on HBASE-5861:
--
My patch didn't work anyway, since MapReduceTestUtil is a test class in Hadoop.
This brings to another point: Some of the HBase tests need this lower layer for
UnitTests, but the production part of HBase should not operate at this layer.
So whatever fix we have for this, should only involve test code. If folks agree
with this then HadoopShim should be a test class.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260195#comment-13260195
]
Lars Hofhansl commented on HBASE-5861:
--
I'll have a sketch of what I mean a bit later.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus
[ https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5853: - Fix Version/s: 0.94.0 java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus -- Key: HBASE-5853 URL: https://issues.apache.org/jira/browse/HBASE-5853 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.1 Environment: hadoop-0.23.1 hbase-0.92.1 Reporter: jiafeng.zhang Fix For: 0.92.1, 0.94.0 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error reading responses on connection Thread[IPC Client (1260987126) connection to server121/172.16.40.121:9000 from smp,5,main] java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223) at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756) Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151) at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368) ... 4 more 2012-04-23 12:51:07,797 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server server124,60020,1335152900476: Replay of HLog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Failed on local exception: java.io.IOException: Error reading responses; Host Details : local host is: server124/172.16.40.124; destination host is: server121:9000; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724) at org.apache.hadoop.ipc.Client.call(Client.java:1094) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193) at $Proxy10.getFileInfo(Unknown Source) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65) at $Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725) at org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548) at org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595) at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506) at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89) at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254) ... 6 more Caused by: java.io.IOException: Error reading responses at org.apache.hadoop.ipc.Client$Connection.run(Client.java:763) Caused by: java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus at
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260217#comment-13260217
]
Lars Hofhansl commented on HBASE-5861:
--
Seems that way. And that is not just a test issue. So going from that
HadoopShim needs to be part of production HBase.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus
[ https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260220#comment-13260220 ] Lars Hofhansl commented on HBASE-5853: -- Did you recompile you version of 0.92.1 against Hadoop 0.23? java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus -- Key: HBASE-5853 URL: https://issues.apache.org/jira/browse/HBASE-5853 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.1 Environment: hadoop-0.23.1 hbase-0.92.1 Reporter: jiafeng.zhang Fix For: 0.92.1, 0.94.0 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error reading responses on connection Thread[IPC Client (1260987126) connection to server121/172.16.40.121:9000 from smp,5,main] java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223) at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756) Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151) at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368) ... 4 more 2012-04-23 12:51:07,797 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server server124,60020,1335152900476: Replay of HLog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Failed on local exception: java.io.IOException: Error reading responses; Host Details : local host is: server124/172.16.40.124; destination host is: server121:9000; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724) at org.apache.hadoop.ipc.Client.call(Client.java:1094) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193) at $Proxy10.getFileInfo(Unknown Source) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65) at $Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725) at org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548) at org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595) at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506) at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89) at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254) ... 6 more Caused by: java.io.IOException: Error reading responses at org.apache.hadoop.ipc.Client$Connection.run(Client.java:763) Caused by: java.lang.RuntimeException: readObject can't find class
[jira] [Issue Comment Edited] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus
[ https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260220#comment-13260220 ] Lars Hofhansl edited comment on HBASE-5853 at 4/24/12 4:57 AM: --- Did you recompile your version of 0.92.1 against Hadoop 0.23? was (Author: lhofhansl): Did you recompile you version of 0.92.1 against Hadoop 0.23? java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus -- Key: HBASE-5853 URL: https://issues.apache.org/jira/browse/HBASE-5853 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.1 Environment: hadoop-0.23.1 hbase-0.92.1 Reporter: jiafeng.zhang Fix For: 0.92.1, 0.94.0 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error reading responses on connection Thread[IPC Client (1260987126) connection to server121/172.16.40.121:9000 from smp,5,main] java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223) at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756) Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151) at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368) ... 4 more 2012-04-23 12:51:07,797 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server server124,60020,1335152900476: Replay of HLog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Failed on local exception: java.io.IOException: Error reading responses; Host Details : local host is: server124/172.16.40.124; destination host is: server121:9000; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724) at org.apache.hadoop.ipc.Client.call(Client.java:1094) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193) at $Proxy10.getFileInfo(Unknown Source) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65) at $Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725) at org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548) at org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595) at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506) at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89) at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254) ... 6 more Caused by: java.io.IOException: Error reading
[jira] [Updated] (HBASE-5850) Refuse operations from Admin before master is initialized - fix for all branches.
[ https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5850: - Summary: Refuse operations from Admin before master is initialized - fix for all branches. (was: Backport HBASE-5454 to 90 and 92 Refuse operations from Admin before master is initialized) Changed title Refuse operations from Admin before master is initialized - fix for all branches. - Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2, 0.94.0 Attachments: 5850-trunk.txt, No_patch_90_surefire-report.html, backport-5454(createTable)-to-94.patch, backport-5454(createTable)-to-94_surefire-report.html, backport-5454(createTable)-to-trunk.patch, backport-5454(createTable)-to-trunk_surefire-report.html, backport-5454-to-90-surefire-report.html, backport-5454-to-90.patch, backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5850) Refuse operations from Admin before master is initialized - fix for all branches.
[ https://issues.apache.org/jira/browse/HBASE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5850: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to 0.90, 0.92, 0.94, and 0.96 (just noticed that I did forget the jira number in the commit messages, sorry) Refuse operations from Admin before master is initialized - fix for all branches. - Key: HBASE-5850 URL: https://issues.apache.org/jira/browse/HBASE-5850 Project: HBase Issue Type: Bug Reporter: xufeng Assignee: xufeng Fix For: 0.90.7, 0.92.2, 0.94.0 Attachments: 5850-trunk.txt, No_patch_90_surefire-report.html, backport-5454(createTable)-to-94.patch, backport-5454(createTable)-to-94_surefire-report.html, backport-5454(createTable)-to-trunk.patch, backport-5454(createTable)-to-trunk_surefire-report.html, backport-5454-to-90-surefire-report.html, backport-5454-to-90.patch, backport-5454-to-92.patch, backport-5454-to-92_surefire-report.html This issue is needed in 0.90 0.92 also. And update the hbase-5454 patch that add the checkInitialized() into HMaster#createTable(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260235#comment-13260235 ] Lars Hofhansl commented on HBASE-5848: -- Should also check for null (which is interpreted as empty key at the server). +1 otherwise. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Minor Fix For: 0.94.0 Attachments: HBASE-5848.patch, HBASE-5848.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Fix Version/s: 0.94.0 Assignee: ramkrishna.s.vasudevan (was: Lars Hofhansl) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0 Attachments: HBASE-5848.patch, HBASE-5848.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260254#comment-13260254
]
Lars Hofhansl commented on HBASE-5848:
--
@Ram: What I meant was doing this:
{code}
createTable(new HTableDescriptor(x), new byte[][] {null});
{code}
splitKeys is not null, but one of the contained keys is.
Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to
abort
Key: HBASE-5848
URL: https://issues.apache.org/jira/browse/HBASE-5848
Project: HBase
Issue Type: Bug
Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.94.0
Attachments: HBASE-5848.patch, HBASE-5848.patch
A coworker of mine just had this scenario. It does not make sense the
EMPTY_START_ROW as splitKey (since the region with the empty start key is
implicit), but it should not cause the HMaster to abort.
The abort happens because it tries to bulk assign the same region twice and
then runs into race conditions with ZK.
The same would (presumably) happen when two identical split keys are passed,
but the client blocks that. The simplest solution here is to also block
passed null or EMPTY_START_ROW as split key by the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260269#comment-13260269 ] Lars Hofhansl commented on HBASE-5848: -- @Ram: That is probably overkill, means another loop over the splitKeys before they are sorted. As long as it fails on the client (even with an NPE) and does not cause the master to abort it's fine I think. If you like to add the extra check now, that fine too. +1 in either case. Going to bed now, feel free to commit :) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0 Attachments: HBASE-5848.patch, HBASE-5848.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260271#comment-13260271
]
Lars Hofhansl commented on HBASE-5861:
--
I am generally a bit unclear at what point we require a recompile of HBase
against a specific version of Hadoop and when we make it just work via
reflection.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260725#comment-13260725
]
Lars Hofhansl commented on HBASE-5861:
--
The comment in InputSampler also says that one should use the actual Hadoop
code in 0.23+ (not the backported code), so I would guess that this would be OK
as long as there is no compile time error.
If the only problem is TestHLogRecordReader, I should just fix up the test and
be done with this.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[
https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261002#comment-13261002
]
Lars Hofhansl commented on HBASE-5848:
--
As I said above, I can kill the HMaster by issuing.
{code}
createTable(new HTableDescriptor(x), new byte[][]
{HConstants.EMPTY_START_ROW, new byte[] {1,2,3}});
{code}
So it happens also when HConstants.EMPTY_START_ROW is passed along with other
split keys.
Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to
abort
Key: HBASE-5848
URL: https://issues.apache.org/jira/browse/HBASE-5848
Project: HBase
Issue Type: Bug
Components: client
Reporter: Lars Hofhansl
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.94.0, 0.96.0
Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt,
HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch,
HBASE-5848_addendum.patch
A coworker of mine just had this scenario. It does not make sense the
EMPTY_START_ROW as splitKey (since the region with the empty start key is
implicit), but it should not cause the HMaster to abort.
The abort happens because it tries to bulk assign the same region twice and
then runs into race conditions with ZK.
The same would (presumably) happen when two identical split keys are passed,
but the client blocks that. The simplest solution here is to also block
passed null or EMPTY_START_ROW as split key by the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261027#comment-13261027
]
Lars Hofhansl commented on HBASE-5861:
--
+1 on v3 for 0.94. JobTrackerRunner.getJobTracker is a separate issue for trunk.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch,
hbase-5861-v3.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261093#comment-13261093 ] Lars Hofhansl commented on HBASE-5848: -- The problem is that HBaseTestUtil.KEYS is used to two different things in the Tests: in HBaseAdmin.createTable* and in HBaseTestUtility.createMultiRegionsInMeta. Ideally createMultiRegionsInMeta would be similar to HMaster.getHRegionInfos, but it is not. At this point it is too risky to change createMultiRegionsInMeta and createMultiRegions, so we should have to versions of HBaseTestUtil.KEYS - one that has the EMPTY_ROW for the test methods, and one that doesn't for the call to HMaster.createTable. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261099#comment-13261099
]
Lars Hofhansl commented on HBASE-5861:
--
Yep. So let's get this issue into 0.94 and trunk.
+1 from me for v3 in 0.94 and trunk.
Then we file a separate issue for trunk to deal with
JobTrackerRunner.getJobTracker.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861.txt, hbase-5861-jon.patch, hbase-5861-v2.patch,
hbase-5861-v3.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261108#comment-13261108 ] Lars Hofhansl commented on HBASE-5848: -- @Ted: The minicluster (or timing in tests) is sufficiently different so that we cannot write a test that crashes the master (see my and Ram's comment above). Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Attachment: 5848-addendum-v5.txt New addendum for HadoopQA. Introduces different KEYS and TEST_KEYS in HBaseTestingUtility. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261174#comment-13261174 ] Lars Hofhansl commented on HBASE-5848: -- Yes, that's also what I said above :) There're more DOS attacks against HBase (like passing a doctored RPC with a size of 4GB, etc), though. +1 on fixing the root cause too, of course. Lemme do a combined patch quickly... Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Attachment: 5848-addendum-v6.txt How about this one. Addresses the root cause and uses the right set of split keys when used in TestRegionRebalancing. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Status: Open (was: Patch Available) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261179#comment-13261179
]
Lars Hofhansl commented on HBASE-5861:
--
Awesome. Thanks Jon and Ted.
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861-v4.patch, 5861.txt, hbase-5861-jon.patch,
hbase-5861-v2.patch, hbase-5861-v3.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5871) Usability regression, we don't parse compression algos anymore
[
https://issues.apache.org/jira/browse/HBASE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261180#comment-13261180
]
Lars Hofhansl commented on HBASE-5871:
--
Use LZO? I think you always had to match the case, no?
Usability regression, we don't parse compression algos anymore
--
Key: HBASE-5871
URL: https://issues.apache.org/jira/browse/HBASE-5871
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.92.2, 0.94.0, 0.96.0
It seems that string with 0.92.0 we can't create tables in the shell by
specifying lzo anymore. I remember we used to do better parsing than that,
but right now if you follow the wiki doing this:
bq. create 'mytable', {NAME='colfam:', COMPRESSION='lzo'}
You'll get:
bq. ERROR: java.lang.IllegalArgumentException: No enum const class
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.lzo
Bad for usability.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus
[ https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261181#comment-13261181 ] Lars Hofhansl commented on HBASE-5853: -- I agree that looks like a setup problem, rather than an HBase problem. java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus -- Key: HBASE-5853 URL: https://issues.apache.org/jira/browse/HBASE-5853 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.1 Environment: hadoop-0.23.1 hbase-0.92.1 Reporter: jiafeng.zhang Fix For: 0.92.1, 0.94.0 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error reading responses on connection Thread[IPC Client (1260987126) connection to server121/172.16.40.121:9000 from smp,5,main] java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223) at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756) Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151) at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368) ... 4 more 2012-04-23 12:51:07,797 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server server124,60020,1335152900476: Replay of HLog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Failed on local exception: java.io.IOException: Error reading responses; Host Details : local host is: server124/172.16.40.124; destination host is: server121:9000; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724) at org.apache.hadoop.ipc.Client.call(Client.java:1094) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193) at $Proxy10.getFileInfo(Unknown Source) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65) at $Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725) at org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548) at org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595) at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506) at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89) at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254) ... 6 more Caused by: java.io.IOException: Error reading responses at org.apache.hadoop.ipc.Client$Connection.run(Client.java:763) Caused by: java.lang.RuntimeException: readObject can't find class
[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261183#comment-13261183
]
Lars Hofhansl commented on HBASE-5864:
--
Dhruba should have a look this too.
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261187#comment-13261187 ] Lars Hofhansl commented on HBASE-5848: -- :) Yes, was thinking that too. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5861) Hadoop 23 compile broken due to tests introduced in HBASE-5064
[
https://issues.apache.org/jira/browse/HBASE-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261191#comment-13261191
]
Lars Hofhansl commented on HBASE-5861:
--
As Ted points out MapreduceTestingShim.java is missing :), build is broken
(0.94 and trunk)
Hadoop 23 compile broken due to tests introduced in HBASE-5064
---
Key: HBASE-5861
URL: https://issues.apache.org/jira/browse/HBASE-5861
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Blocker
Fix For: 0.94.0, 0.96.0
Attachments: 5861-v4.patch, 5861.txt, hbase-5861-jon.patch,
hbase-5861-v2.patch, hbase-5861-v3.patch
When attempting to compile HBase 0.94rc1 against hadoop 23, I got this set of
compilation error messages:
{code}
jon@swoop:~/proj/hbase-0.94$ mvn clean test -Dhadoop.profile=23 -DskipTests
...
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 18.926s
[INFO] Finished at: Mon Apr 23 10:38:47 PDT 2012
[INFO] Final Memory: 55M/555M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[147,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[153,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[194,46]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[206,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[213,29]
org.apache.hadoop.mapreduce.JobContext is abstract; cannot be instantiated
[ERROR]
[ERROR]
/home/jon/proj/hbase-0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHLogRecordReader.java:[226,29]
org.apache.hadoop.mapreduce.TaskAttemptContext is abstract; cannot be
instantiated
[ERROR] - [Help 1]
{code}
Upon further investigation this issue is due to code introduced in HBASE-5064
and is also present in trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261194#comment-13261194 ] Lars Hofhansl commented on HBASE-5848: -- Will wait for HadoopQA, and commit if all is good. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Status: Patch Available (was: Open) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261278#comment-13261278 ] Lars Hofhansl commented on HBASE-5848: -- @Ram: No worries. I looked at the patch too. This is part of doing development. :) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Attachment: 5848-addendum-v7.txt Arrghhh... Off by one. Now fixed in TestRegionBalancing Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5870) Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method is not found
[
https://issues.apache.org/jira/browse/HBASE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5870:
-
Status: Patch Available (was: Open)
Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method
is not found
-
Key: HBASE-5870
URL: https://issues.apache.org/jira/browse/HBASE-5870
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Priority: Blocker
Fix For: 0.96.0
Attachments: 5870.txt
After HBASE-5861 on 0.94 we are left with this issue on trunk.
{code}
$ mvn clean test -PlocalTests -DskipTests -Dhadoop.profile=23
...
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
/home/jon/proj/hbase-svn/hbase/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java:[1333,35]
cannot find symbol
[ERROR] symbol : method getJobTracker()
[ERROR] location: class
org.apache.hadoop.mapred.MiniMRCluster.JobTrackerRunner
[ERROR] - [Help 1]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5870) Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method is not found
[
https://issues.apache.org/jira/browse/HBASE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5870:
-
Attachment: 5870.txt
Ted's proposed patch extracted from parent.
Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method
is not found
-
Key: HBASE-5870
URL: https://issues.apache.org/jira/browse/HBASE-5870
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Priority: Blocker
Fix For: 0.96.0
Attachments: 5870.txt
After HBASE-5861 on 0.94 we are left with this issue on trunk.
{code}
$ mvn clean test -PlocalTests -DskipTests -Dhadoop.profile=23
...
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
/home/jon/proj/hbase-svn/hbase/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java:[1333,35]
cannot find symbol
[ERROR] symbol : method getJobTracker()
[ERROR] location: class
org.apache.hadoop.mapred.MiniMRCluster.JobTrackerRunner
[ERROR] - [Help 1]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4393) Implement a canary monitoring program
[ https://issues.apache.org/jira/browse/HBASE-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261286#comment-13261286 ] Lars Hofhansl commented on HBASE-4393: -- Thanks Stack... I do like this in 0.94 (- the wrong package of course :) ) Implement a canary monitoring program - Key: HBASE-4393 URL: https://issues.apache.org/jira/browse/HBASE-4393 Project: HBase Issue Type: New Feature Components: monitoring Affects Versions: 0.92.0 Reporter: Todd Lipcon Assignee: Matteo Bertozzi Fix For: 0.94.0, 0.96.0 Attachments: Canary-v0.java, HBASE-4393-v0.patch, HBaseCanary.java This JIRA is to implement a standalone program that can be used to do canary monitoring of a running HBase cluster. This program would gather a list of the regions in the cluster, then iterate over them doing lightweight operations (eg short scans) to provide metrics about latency as well as alert on availability issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Status: Open (was: Patch Available) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Attachment: 5848-addendum-v7.txt Trying again for HadoopQA Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Status: Patch Available (was: Open) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5848: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Addendum committed to 0.94 and 0.96. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5871) Usability regression, we don't parse compression algos anymore
[
https://issues.apache.org/jira/browse/HBASE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5871:
-
Attachment: 5871-trunk.txt
5871-0.94.txt
5871-0.92.txt
Indeed in 0.92+ somebody added a superfluous call to setCompression, which
causes this problem.
Simple patches for 0.92, 0.94, and 0.96, removing that. Works fine now.
Usability regression, we don't parse compression algos anymore
--
Key: HBASE-5871
URL: https://issues.apache.org/jira/browse/HBASE-5871
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Jean-Daniel Cryans
Priority: Critical
Fix For: 0.92.2, 0.94.0, 0.96.0
Attachments: 5871-0.92.txt, 5871-0.94.txt, 5871-trunk.txt
It seems that string with 0.92.0 we can't create tables in the shell by
specifying lzo anymore. I remember we used to do better parsing than that,
but right now if you follow the wiki doing this:
bq. create 'mytable', {NAME='colfam:', COMPRESSION='lzo'}
You'll get:
bq. ERROR: java.lang.IllegalArgumentException: No enum const class
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.lzo
Bad for usability.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5871) Usability regression, we don't parse compression algos anymore
[
https://issues.apache.org/jira/browse/HBASE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl reassigned HBASE-5871:
Assignee: Lars Hofhansl
Usability regression, we don't parse compression algos anymore
--
Key: HBASE-5871
URL: https://issues.apache.org/jira/browse/HBASE-5871
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Jean-Daniel Cryans
Assignee: Lars Hofhansl
Priority: Critical
Fix For: 0.92.2, 0.94.0, 0.96.0
Attachments: 5871-0.92.txt, 5871-0.94.txt, 5871-trunk.txt
It seems that string with 0.92.0 we can't create tables in the shell by
specifying lzo anymore. I remember we used to do better parsing than that,
but right now if you follow the wiki doing this:
bq. create 'mytable', {NAME='colfam:', COMPRESSION='lzo'}
You'll get:
bq. ERROR: java.lang.IllegalArgumentException: No enum const class
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.lzo
Bad for usability.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus
[ https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261310#comment-13261310 ] Lars Hofhansl commented on HBASE-5853: -- Had anybody else seen this? Seem any HBase flushCache on top of Hadoop 0.23 should cause this. java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus -- Key: HBASE-5853 URL: https://issues.apache.org/jira/browse/HBASE-5853 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.1 Environment: hadoop-0.23.1 hbase-0.92.1 Reporter: jiafeng.zhang Fix For: 0.92.1, 0.94.0 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error reading responses on connection Thread[IPC Client (1260987126) connection to server121/172.16.40.121:9000 from smp,5,main] java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223) at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756) Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151) at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368) ... 4 more 2012-04-23 12:51:07,797 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server server124,60020,1335152900476: Replay of HLog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Failed on local exception: java.io.IOException: Error reading responses; Host Details : local host is: server124/172.16.40.124; destination host is: server121:9000; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724) at org.apache.hadoop.ipc.Client.call(Client.java:1094) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193) at $Proxy10.getFileInfo(Unknown Source) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65) at $Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725) at org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548) at org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595) at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506) at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89) at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254) ... 6 more Caused by: java.io.IOException: Error reading responses at org.apache.hadoop.ipc.Client$Connection.run(Client.java:763) Caused by: java.lang.RuntimeException: readObject
[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261315#comment-13261315
]
Lars Hofhansl commented on HBASE-5864:
--
I also don't quite follow the patch or problem.
How is it that HBase during normal operation (scanning, etc) can read HFiles
correctly?
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Critical
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5849) On first cluster startup, RS aborts if root znode is not available
[ https://issues.apache.org/jira/browse/HBASE-5849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261316#comment-13261316 ] Lars Hofhansl commented on HBASE-5849: -- TestRegionRebalancing is unrelated (see HBASE-5848). TestReplication passes for me locally with v4 applied. On first cluster startup, RS aborts if root znode is not available -- Key: HBASE-5849 URL: https://issues.apache.org/jira/browse/HBASE-5849 Project: HBase Issue Type: Bug Components: master, regionserver, zookeeper Affects Versions: 0.92.2, 0.96.0, 0.94.1 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.92.2, 0.94.0 Attachments: 5849v3.txt, HBASE-5849_v1.patch, HBASE-5849_v2.patch, HBASE-5849_v4-0.92.patch, HBASE-5849_v4.patch, HBASE-5849_v4.patch, HBASE-5849_v4.patch When launching a fresh new cluster, the master has to be started first, which might create race conditions for starting master and rs at the same time. Master startup code is smt like this: - establish zk connection - create root znodes in zk (/hbase) - create ephemeral node for master /hbase/master, Region server start up code is smt like this: - establish zk connection - check whether the root znode (/hbase) is there. If not, shutdown. - wait for the master to create znodes /hbase/master So, the problem is on the very first launch of the cluster, RS aborts to start since /hbase znode might not have been created yet (only the master creates it if needed). Since /hbase/ is not deleted on cluster shutdown, on subsequent cluster starts, it does not matter which order the servers are started. So this affects only first launchs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5871) Usability regression, we don't parse compression algos anymore
[
https://issues.apache.org/jira/browse/HBASE-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl resolved HBASE-5871.
--
Resolution: Fixed
Committed to 0.92, 0.94, 0.96.
Usability regression, we don't parse compression algos anymore
--
Key: HBASE-5871
URL: https://issues.apache.org/jira/browse/HBASE-5871
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: Jean-Daniel Cryans
Assignee: Lars Hofhansl
Priority: Critical
Fix For: 0.92.2, 0.94.0, 0.96.0
Attachments: 5871-0.92.txt, 5871-0.94.txt, 5871-trunk.txt
It seems that string with 0.92.0 we can't create tables in the shell by
specifying lzo anymore. I remember we used to do better parsing than that,
but right now if you follow the wiki doing this:
bq. create 'mytable', {NAME='colfam:', COMPRESSION='lzo'}
You'll get:
bq. ERROR: java.lang.IllegalArgumentException: No enum const class
org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.lzo
Bad for usability.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5864:
-
Priority: Blocker (was: Critical)
I see. This is a blocker then.
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5870) Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method is not found
[
https://issues.apache.org/jira/browse/HBASE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261717#comment-13261717
]
Lars Hofhansl commented on HBASE-5870:
--
Lemme clean up this mess.
Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method
is not found
-
Key: HBASE-5870
URL: https://issues.apache.org/jira/browse/HBASE-5870
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Priority: Blocker
Fix For: 0.96.0
Attachments: 5870.txt
After HBASE-5861 on 0.94 we are left with this issue on trunk.
{code}
$ mvn clean test -PlocalTests -DskipTests -Dhadoop.profile=23
...
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
/home/jon/proj/hbase-svn/hbase/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java:[1333,35]
cannot find symbol
[ERROR] symbol : method getJobTracker()
[ERROR] location: class
org.apache.hadoop.mapred.MiniMRCluster.JobTrackerRunner
[ERROR] - [Help 1]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5870) Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method is not found
[
https://issues.apache.org/jira/browse/HBASE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261719#comment-13261719
]
Lars Hofhansl commented on HBASE-5870:
--
Reverted.
Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method
is not found
-
Key: HBASE-5870
URL: https://issues.apache.org/jira/browse/HBASE-5870
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Priority: Blocker
Fix For: 0.96.0
Attachments: 5870.txt
After HBASE-5861 on 0.94 we are left with this issue on trunk.
{code}
$ mvn clean test -PlocalTests -DskipTests -Dhadoop.profile=23
...
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
/home/jon/proj/hbase-svn/hbase/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java:[1333,35]
cannot find symbol
[ERROR] symbol : method getJobTracker()
[ERROR] location: class
org.apache.hadoop.mapred.MiniMRCluster.JobTrackerRunner
[ERROR] - [Help 1]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261721#comment-13261721 ] Lars Hofhansl commented on HBASE-5848: -- Yep, I committed the wrong the patch. Reverted over in HBASE-5870, and committed the right patch. Sorry about that. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5870) Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method is not found
[
https://issues.apache.org/jira/browse/HBASE-5870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261722#comment-13261722
]
Lars Hofhansl commented on HBASE-5870:
--
Test run looked good, though. So I'm +1 on commit.
Hadoop 23 compilation broken because JobTrackerRunner#getJobTracker() method
is not found
-
Key: HBASE-5870
URL: https://issues.apache.org/jira/browse/HBASE-5870
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0
Reporter: Jonathan Hsieh
Priority: Blocker
Fix For: 0.96.0
Attachments: 5870.txt
After HBASE-5861 on 0.94 we are left with this issue on trunk.
{code}
$ mvn clean test -PlocalTests -DskipTests -Dhadoop.profile=23
...
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure
[ERROR]
/home/jon/proj/hbase-svn/hbase/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java:[1333,35]
cannot find symbol
[ERROR] symbol : method getJobTracker()
[ERROR] location: class
org.apache.hadoop.mapred.MiniMRCluster.JobTrackerRunner
[ERROR] - [Help 1]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5848) Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort
[ https://issues.apache.org/jira/browse/HBASE-5848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261724#comment-13261724 ] Lars Hofhansl commented on HBASE-5848: -- Also double checked 0.94. All's good now. Create table with EMPTY_START_ROW passed as splitKey causes the HMaster to abort Key: HBASE-5848 URL: https://issues.apache.org/jira/browse/HBASE-5848 Project: HBase Issue Type: Bug Components: client Reporter: Lars Hofhansl Assignee: ramkrishna.s.vasudevan Priority: Minor Fix For: 0.94.0, 0.96.0 Attachments: 5848-addendum-v2.txt, 5848-addendum-v3.txt, 5848-addendum-v4.txt, 5848-addendum-v5.txt, 5848-addendum-v6.txt, 5848-addendum-v7.txt, 5848-addendum-v7.txt, HBASE-5848.patch, HBASE-5848.patch, HBASE-5848_0.94.patch, HBASE-5848_addendum.patch A coworker of mine just had this scenario. It does not make sense the EMPTY_START_ROW as splitKey (since the region with the empty start key is implicit), but it should not cause the HMaster to abort. The abort happens because it tries to bulk assign the same region twice and then runs into race conditions with ZK. The same would (presumably) happen when two identical split keys are passed, but the client blocks that. The simplest solution here is to also block passed null or EMPTY_START_ROW as split key by the client. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5873:
-
Status: Open (was: Patch Available)
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5873:
-
Attachment: 5873-trunk.txt
Trunk patch for HadoopQA
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5873:
-
Status: Patch Available (was: Open)
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261755#comment-13261755
]
Lars Hofhansl commented on HBASE-5873:
--
Oops, rajesh, we crossed comments.
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus
[ https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261758#comment-13261758 ] Lars Hofhansl commented on HBASE-5853: -- @jiafeng: Can you tell us what exactly you did when this happened? java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus -- Key: HBASE-5853 URL: https://issues.apache.org/jira/browse/HBASE-5853 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.1 Environment: hadoop-0.23.1 hbase-0.92.1 Reporter: jiafeng.zhang Fix For: 0.92.1, 0.94.0 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error reading responses on connection Thread[IPC Client (1260987126) connection to server121/172.16.40.121:9000 from smp,5,main] java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223) at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756) Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151) at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368) ... 4 more 2012-04-23 12:51:07,797 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server server124,60020,1335152900476: Replay of HLog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Failed on local exception: java.io.IOException: Error reading responses; Host Details : local host is: server124/172.16.40.124; destination host is: server121:9000; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724) at org.apache.hadoop.ipc.Client.call(Client.java:1094) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193) at $Proxy10.getFileInfo(Unknown Source) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65) at $Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725) at org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548) at org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595) at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506) at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89) at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254) ... 6 more Caused by: java.io.IOException: Error reading responses at org.apache.hadoop.ipc.Client$Connection.run(Client.java:763) Caused by: java.lang.RuntimeException: readObject can't find class
[jira] [Commented] (HBASE-5875) Process RIT and Master restart may remove an online server considering it as a dead server
[
https://issues.apache.org/jira/browse/HBASE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261759#comment-13261759
]
Lars Hofhansl commented on HBASE-5875:
--
Can we move this to 0.94.1?
Process RIT and Master restart may remove an online server considering it as
a dead server
--
Key: HBASE-5875
URL: https://issues.apache.org/jira/browse/HBASE-5875
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.94.0
If on master restart it finds the ROOT/META to be in RIT state, master tries
to assign the ROOT region through ProcessRIT.
Master will trigger the assignment and next will try to verify the Root
Region Location.
Root region location verification is done seeing if the RS has the region in
its online list.
If the master triggered assignment has not yet been completed in RS then the
verify root region location will fail.
Because it failed
{code}
splitLogAndExpireIfOnline(currentRootServer);
{code}
we do split log and also remove the server from online server list. Ideally
here there is nothing to do in splitlog as no region server was restarted.
So master, though the server is online, master just invalidates the region
server.
In a special case, if i have only one RS then my cluster will become non
operative.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5611:
-
Fix Version/s: (was: 0.94.1)
0.94.0
Yep... Looks bad.
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: HBASE-5611-trunk.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261908#comment-13261908
]
Lars Hofhansl commented on HBASE-5873:
--
Looks good, tests pass. +1 from me.
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5611) Replayed edits from regions that failed to open during recovery aren't removed from the global MemStore size
[
https://issues.apache.org/jira/browse/HBASE-5611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5611:
-
Fix Version/s: (was: 0.94.0)
0.94.1
Actually... It seems we had this problem since 0.90.
I'll pull it in if it gets done in time, otherwise it'll be in the next point
release.
Replayed edits from regions that failed to open during recovery aren't
removed from the global MemStore size
Key: HBASE-5611
URL: https://issues.apache.org/jira/browse/HBASE-5611
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: Jean-Daniel Cryans
Assignee: Jieshan Bean
Priority: Critical
Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.1
Attachments: HBASE-5611-trunk.patch
This bug is rather easy to get if the {{TimeoutMonitor}} is on, else I think
it's still possible to hit it if a region fails to open for more obscure
reasons like HDFS errors.
Consider a region that just went through distributed splitting and that's now
being opened by a new RS. The first thing it does is to read the recovery
files and put the edits in the {{MemStores}}. If this process takes a long
time, the master will move that region away. At that point the edits are
still accounted for in the global {{MemStore}} size but they are dropped when
the {{HRegion}} gets cleaned up. It's completely invisible until the
{{MemStoreFlusher}} needs to force flush a region and that none of them have
edits:
{noformat}
2012-03-21 00:33:39,303 DEBUG
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Flush thread woke up
because memory above low water=5.9g
2012-03-21 00:33:39,303 ERROR
org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Cache flusher failed
for entry null
java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:129)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:199)
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223)
at java.lang.Thread.run(Thread.java:662)
{noformat}
The {{null}} here is a region. In my case I had so many edits in the
{{MemStore}} during recovery that I'm over the low barrier although in fact
I'm at 0. It happened yesterday and it still printing this out.
To fix this we need to be able to decrease the global {{MemStore}} size when
the region can't open.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5853) java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus
[ https://issues.apache.org/jira/browse/HBASE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-5853: - Fix Version/s: (was: 0.94.0) 0.94.1 Moving this out until somebody confirms that it is serious problem. java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus -- Key: HBASE-5853 URL: https://issues.apache.org/jira/browse/HBASE-5853 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.1 Environment: hadoop-0.23.1 hbase-0.92.1 Reporter: jiafeng.zhang Fix For: 0.92.1, 0.94.1 2012-04-23 12:51:07,474 WARN org.apache.hadoop.ipc.Client: Unexpected error reading responses on connection Thread[IPC Client (1260987126) connection to server121/172.16.40.121:9000 from smp,5,main] java.lang.RuntimeException: readObject can't find class org.apache.hadoop.hdfs.protocol.HdfsFileStatus at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:372) at org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:223) at org.apache.hadoop.io.ObjectWritable.readFields(ObjectWritable.java:75) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:832) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:756) Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.hdfs.protocol.HdfsFileStatus not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1151) at org.apache.hadoop.io.ObjectWritable.loadClass(ObjectWritable.java:368) ... 4 more 2012-04-23 12:51:07,797 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server server124,60020,1335152900476: Replay of HLog required. Forcing server shutdown org.apache.hadoop.hbase.DroppedSnapshotException: region: hbase_cdr,e0072b2b-5e19-431f-bb69-a6427765eac4,1334902272934.8365a7cbf90dd558f297d70224113c8a. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1278) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1162) at org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1104) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:400) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushOneForGlobalPressure(MemStoreFlusher.java:202) at org.apache.hadoop.hbase.regionserver.MemStoreFlusher.run(MemStoreFlusher.java:223) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: Failed on local exception: java.io.IOException: Error reading responses; Host Details : local host is: server124/172.16.40.124; destination host is: server121:9000; at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:724) at org.apache.hadoop.ipc.Client.call(Client.java:1094) at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:193) at $Proxy10.getFileInfo(Unknown Source) at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:100) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:65) at $Proxy10.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1172) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:725) at org.apache.hadoop.hbase.regionserver.StoreFile.computeHDFSBlockDistribution(StoreFile.java:449) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:473) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:548) at org.apache.hadoop.hbase.regionserver.Store.internalFlushCache(Store.java:595) at org.apache.hadoop.hbase.regionserver.Store.flushCache(Store.java:506) at org.apache.hadoop.hbase.regionserver.Store.access$100(Store.java:89) at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.flushCache(Store.java:1905) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1254) ... 6 more Caused by: java.io.IOException: Error reading responses at org.apache.hadoop.ipc.Client$Connection.run(Client.java:763) Caused by: java.lang.RuntimeException: readObject can't find class
[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261917#comment-13261917
]
Lars Hofhansl commented on HBASE-5864:
--
TestRegionRebalancing is unrelated (HBASE-5848)
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_3.patch, HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261940#comment-13261940
]
Lars Hofhansl commented on HBASE-5873:
--
This change does violate encapsulation a bit.
I double checked where in the code we create instances of AssignmentManager.
Besides the HMaster it is only from tests (and they all pass it's good).
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262061#comment-13262061
]
Lars Hofhansl commented on HBASE-5873:
--
Committed to 0.94 and trunk.
Creating 0.90 and 0.92 patches now.
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5873) TimeOut Monitor thread should be started after atleast one region server registers.
[
https://issues.apache.org/jira/browse/HBASE-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-5873:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
And committed to 0.90 and 0.92
TimeOut Monitor thread should be started after atleast one region server
registers.
---
Key: HBASE-5873
URL: https://issues.apache.org/jira/browse/HBASE-5873
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.6
Reporter: ramkrishna.s.vasudevan
Assignee: rajeshbabu
Priority: Minor
Fix For: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Attachments: 5873-trunk.txt, HBASE-5873.patch
Currently timeout monitor thread is started even before the region server has
registered with the master.
In timeout monitor we depend on the region server to be online
{code}
boolean allRSsOffline = this.serverManager.getOnlineServersList().
isEmpty();
{code}
Now when the master starts up it sees there are no online servers and hence
sets
allRSsOffline to true.
{code}
setAllRegionServersOffline(allRSsOffline);
{code}
So this.allRegionServersOffline is also true.
By this time an RS has come up,
Now timeout comes up again (after 10secs) in the next cycle he sees
allRSsOffline as false.
Hence
{code}
else if (this.allRegionServersOffline !allRSsOffline) {
// if some RSs just came back online, we can start the
// the assignment right away
actOnTimeOut(regionState);
{code}
This condition makes him to take action based on timeout.
Because of this even if one Region assignment of ROOT is going on, this piece
of code triggers another assignment and thus we get RegionAlreadyinTransition
Exception. Later we need to wait for 30 mins for assigning ROOT itself.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262363#comment-13262363
]
Lars Hofhansl commented on HBASE-5864:
--
This is the only issue in the way of the next RC attempt for 0.94.0.
I don't feel I can +1 this until a lot of study of the implication.
Was hoping Dhruba would be able to grok it. :)
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_3.patch, HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5864) Error while reading from hfile in 0.94
[
https://issues.apache.org/jira/browse/HBASE-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262391#comment-13262391
]
Lars Hofhansl commented on HBASE-5864:
--
bq. The meat of the change is in readMultiLevelIndexRoot() in which, instead of
using in.available() it uses the in.available() - sizeofchecksum
So it seems we could have a smaller change that just does that (plus the tests)?
I agree that this is a great catch!
Error while reading from hfile in 0.94
--
Key: HBASE-5864
URL: https://issues.apache.org/jira/browse/HBASE-5864
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Priority: Blocker
Fix For: 0.94.0
Attachments: HBASE-5864_1.patch, HBASE-5864_2.patch,
HBASE-5864_3.patch, HBASE-5864_test.patch
Got the following stacktrace during region split.
{noformat}
2012-04-24 16:05:42,168 WARN org.apache.hadoop.hbase.regionserver.Store:
Failed getting store size for value
java.io.IOException: Requested block is out of range: 2906737606134037404,
lastDataBlockOffset: 84764558
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:278)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.midkey(HFileBlockIndex.java:285)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.midkey(HFileReaderV2.java:402)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.midkey(StoreFile.java:1638)
at
org.apache.hadoop.hbase.regionserver.Store.getSplitPoint(Store.java:1943)
at
org.apache.hadoop.hbase.regionserver.RegionSplitPolicy.getSplitPoint(RegionSplitPolicy.java:77)
at
org.apache.hadoop.hbase.regionserver.HRegion.checkSplit(HRegion.java:4921)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.splitRegion(HRegionServer.java:2901)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
