[jira] [Work started] (HBASE-6119) Region server logs its own address at the end of getMaster()
[
https://issues.apache.org/jira/browse/HBASE-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-6119 started by Zhihong Ted Yu.
Region server logs its own address at the end of getMaster()
Key: HBASE-6119
URL: https://issues.apache.org/jira/browse/HBASE-6119
Project: HBase
Issue Type: Bug
Reporter: Zhihong Ted Yu
Assignee: Zhihong Ted Yu
Priority: Minor
Fix For: 0.96.0
Attachments: 6119-trunk.txt
I saw the following in region server log where a.ebay.com is region server
itself:
{code}
2012-05-28 08:56:35,315 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
a.ebay.com/10.115.13.20:60020
{code}
We should be logging the address of master
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6119) Region server logs its own address at the end of getMaster()
[
https://issues.apache.org/jira/browse/HBASE-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu reassigned HBASE-6119:
-
Assignee: Zhihong Ted Yu (was: Ted Yu)
Region server logs its own address at the end of getMaster()
Key: HBASE-6119
URL: https://issues.apache.org/jira/browse/HBASE-6119
Project: HBase
Issue Type: Bug
Reporter: Zhihong Ted Yu
Assignee: Zhihong Ted Yu
Priority: Minor
Fix For: 0.96.0
Attachments: 6119-trunk.txt
I saw the following in region server log where a.ebay.com is region server
itself:
{code}
2012-05-28 08:56:35,315 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
a.ebay.com/10.115.13.20:60020
{code}
We should be logging the address of master
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6119) Region server logs its own address at the end of getMaster()
[
https://issues.apache.org/jira/browse/HBASE-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-6119:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
Region server logs its own address at the end of getMaster()
Key: HBASE-6119
URL: https://issues.apache.org/jira/browse/HBASE-6119
Project: HBase
Issue Type: Bug
Reporter: Zhihong Ted Yu
Assignee: Zhihong Ted Yu
Priority: Minor
Fix For: 0.96.0
Attachments: 6119-trunk.txt
I saw the following in region server log where a.ebay.com is region server
itself:
{code}
2012-05-28 08:56:35,315 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
a.ebay.com/10.115.13.20:60020
{code}
We should be logging the address of master
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6119) Region server logs its own address at the end of getMaster()
[
https://issues.apache.org/jira/browse/HBASE-6119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-6119:
--
Hadoop Flags: Reviewed
Status: Patch Available (was: In Progress)
Region server logs its own address at the end of getMaster()
Key: HBASE-6119
URL: https://issues.apache.org/jira/browse/HBASE-6119
Project: HBase
Issue Type: Bug
Reporter: Zhihong Ted Yu
Assignee: Zhihong Ted Yu
Priority: Minor
Fix For: 0.96.0
Attachments: 6119-trunk.txt
I saw the following in region server log where a.ebay.com is region server
itself:
{code}
2012-05-28 08:56:35,315 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Connected to master at
a.ebay.com/10.115.13.20:60020
{code}
We should be logging the address of master
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6618) Implement FuzzyRowFilter with ranges support
[ https://issues.apache.org/jira/browse/HBASE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440268#comment-13440268 ] Zhihong Ted Yu commented on HBASE-6618: --- Enhancing existing class is fine. Implement FuzzyRowFilter with ranges support Key: HBASE-6618 URL: https://issues.apache.org/jira/browse/HBASE-6618 Project: HBase Issue Type: New Feature Components: filters Reporter: Alex Baranau Priority: Minor Attachments: HBASE-6618-algo-desc-bits.png, HBASE-6618-algo.patch Apart from current ability to specify fuzzy row filter e.g. for userId_actionId format as _0004 (where 0004 - actionId) it would be great to also have ability to specify the fuzzy range , e.g. _0004, ..., _0099. See initial discussion here: http://search-hadoop.com/m/WVLJdX0Z65 Note: currently it is possible to provide multiple fuzzy row rules to existing FuzzyRowFilter, but in case when the range is big (contains thousands of values) it is not efficient. Filter should perform efficient fast-forwarding during the scan (this is what distinguishes it from regex row filter). While such functionality may seem like a proper fit for custom filter (i.e. not including into standard filter set) it looks like the filter may be very re-useable. We may judge based on the implementation that will hopefully be added. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440289#comment-13440289
]
Zhihong Ted Yu commented on HBASE-3271:
---
{code}
Running org.apache.hadoop.hbase.mapreduce.TestImportExport
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.517 sec
{code}
Failed test is not related to this patch.
Patch integrated to trunk.
Thanks for the patch, Liang.
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0
Attachments: HBASE-3271.patch, HBASE-3271-v2.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440295#comment-13440295
]
Zhihong Ted Yu commented on HBASE-5329:
---
I ran TestSplitLogManager#testOrphanTaskAcquisition manually and it passed.
Integrated to trunk.
Thanks for the patch, Ian.
addRowLock() may allocate duplicate lock id, causing the client to be blocked
-
Key: HBASE-5329
URL: https://issues.apache.org/jira/browse/HBASE-5329
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.3
Environment: Red Hat Enterprise Linux Server release 5.4
Reporter: liaoxiangui
Assignee: Ian Varley
Priority: Minor
Fix For: 0.96.0
Attachments: 5329-v2.patch, HBASE-5329.patch
{code}
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
{code}
In addRowLock(),rand may generate duplicated lock id, it may cause
regionserver throw exception(Leases$LeaseStillHeldException).The client will
be blocked until old rowlock is released.
{code}
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2155) Add the option to bind to a specific IP address to the Nonblocking Thrift servers
[ https://issues.apache.org/jira/browse/HBASE-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440298#comment-13440298 ] Zhihong Ted Yu commented on HBASE-2155: --- Test failure is not related to thrift. Integrated to trunk. Thanks for the patch, Liang. Add the option to bind to a specific IP address to the Nonblocking Thrift servers - Key: HBASE-2155 URL: https://issues.apache.org/jira/browse/HBASE-2155 Project: HBase Issue Type: Improvement Components: thrift Reporter: Lars Francke Assignee: liang xie Priority: Minor Fix For: 0.96.0 Attachments: HBASE-2155.patch This is not possible in Thrift 0.2.0 so we'll have to wait until the next version is released (which includes THRIFT-684). After that is released this is an easy and quick fix. For a few more details see HBASE-1373 and HBASE-65. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6563) s.isMajorCompaction() throws npe will cause current major Compaction checking abort
[
https://issues.apache.org/jira/browse/HBASE-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440299#comment-13440299
]
Zhihong Ted Yu commented on HBASE-6563:
---
{code}
+ } catch (Exception e) {
+// Ignore the npe, see HBASE-6563
{code}
I would expect the exception caught to match comment.
s.isMajorCompaction() throws npe will cause current major Compaction checking
abort
---
Key: HBASE-6563
URL: https://issues.apache.org/jira/browse/HBASE-6563
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Zhou wenjian
Assignee: Zhou wenjian
Fix For: 0.94.1
Attachments: HBASE-6563-trunk.patch, HBASE-6563-trunk-v2.patch,
HBASE-6563-trunk-v3.patch
2012-05-05 00:49:43,265 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker:
Caught exception
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:938)
at
org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:917)
at
org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3250)
at
org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1222)
at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[
https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440303#comment-13440303
]
Zhihong Ted Yu commented on HBASE-6581:
---
160 test failures.
Here is partial list:
{code}
testDelayedDeleteOnFailure(org.apache.hadoop.hbase.master.TestDistributedLogSplitting):
test timed out after 25000 milliseconds
org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion: Shutting down
testMasterShutdown(org.apache.hadoop.hbase.master.TestMasterShutdown):
Shutting down
testMasterFailoverWithMockedRIT(org.apache.hadoop.hbase.master.TestMasterFailover):
test timed out after 18 milliseconds
testMasterFailoverWithMockedRITOnDeadRS(org.apache.hadoop.hbase.master.TestMasterFailover):
test timed out after 18 milliseconds
testShouldCheckMasterFailOverWhenMETAIsInOpenedState(org.apache.hadoop.hbase.master.TestMasterFailover):
test timed out after 18 milliseconds
testSimpleMasterFailover(org.apache.hadoop.hbase.master.TestMasterFailover):
Shutting down
testRestartClusterAfterKill(org.apache.hadoop.hbase.master.TestRestartCluster):
Shutting down
testClusterRestart(org.apache.hadoop.hbase.master.TestRestartCluster):
Shutting down
org.apache.hadoop.hbase.master.TestMaster: Shutting down
testOpenedRegionHandlerOnMasterRestart(org.apache.hadoop.hbase.master.TestOpenedRegionHandler):
Shutting down
org.apache.hadoop.hbase.master.TestMasterTransitions: Shutting down
org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster: Shutting down
org.apache.hadoop.hbase.rest.TestTableResource: Shutting down
org.apache.hadoop.hbase.rest.client.TestRemoteAdmin: Shutting down
org.apache.hadoop.hbase.rest.client.TestRemoteTable: Shutting down
org.apache.hadoop.hbase.rest.client.TestRemoteTable
org.apache.hadoop.hbase.rest.TestScannerResource: Shutting down
org.apache.hadoop.hbase.rest.TestVersionResource: Shutting down
org.apache.hadoop.hbase.rest.TestGzipFilter: Shutting down
org.apache.hadoop.hbase.rest.TestScannersWithFilters: Shutting down
org.apache.hadoop.hbase.rest.TestSchemaResource: Shutting down
org.apache.hadoop.hbase.rest.TestRowResource: Shutting down
org.apache.hadoop.hbase.rest.TestStatusResource: Shutting down
org.apache.hadoop.hbase.rest.TestMultiRowResource: Shutting down
testMultiClusters(org.apache.hadoop.hbase.TestHBaseTestingUtility): test
timed out after 18 milliseconds
testMiniCluster(org.apache.hadoop.hbase.TestHBaseTestingUtility): Shutting
down
testMultipleStartStop(org.apache.hadoop.hbase.TestHBaseTestingUtility):
Shutting down
testMiniDFSCluster(org.apache.hadoop.hbase.TestHBaseTestingUtility): Port in
use: localhost:0
testSetupClusterTestBuildDir(org.apache.hadoop.hbase.TestHBaseTestingUtility):
Port in use: localhost:0
Tests run: 1210, Failures: 0, Errors: 160, Skipped: 2
{code}
Build with hadoop.profile=3.0
-
Key: HBASE-6581
URL: https://issues.apache.org/jira/browse/HBASE-6581
Project: HBase
Issue Type: Bug
Reporter: Eric Charles
Attachments: HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581.diff
Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to
change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT
instead of 3.0.0-SNAPSHOT in hbase-common).
I can provide a patch that would move most of hadoop dependencies in their
respective profiles and will define the correct hadoop deps in the 3.0
profile.
Please tell me if that's ok to go this way.
Thx, Eric
[1]
$ mvn clean install -Dhadoop.profile=3.0
[INFO] Scanning for projects...
[ERROR] The build could not read 3 projects - [Help 1]
[ERROR]
[ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT
(/d/hbase.svn/hbase-server/pom.xml) has 3 errors
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21
[ERROR]
[ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT
(/d/hbase.svn/hbase-common/pom.xml) has 3 errors
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21
[ERROR]
[ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT
(/d/hbase.svn/hbase-it/pom.xml) has 3 errors
[ERROR]
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440421#comment-13440421
]
Zhihong Ted Yu commented on HBASE-6524:
---
Thanks Todd for the explanation. I understand the support of open source from
Cloudera.
http://cloudera.org is redirected to cloudera.com
As for Google code, it is under com.google:
{code}
import com.google.common.base.Function;
import com.google.protobuf.ByteString;
{code}
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu reassigned HBASE-6524: - Assignee: Jonathan Leavitt Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Assignee: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[
https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440486#comment-13440486
]
Zhihong Ted Yu commented on HBASE-6564:
---
TestHRegion failed. See
https://builds.apache.org/job/PreCommit-HBASE-Build/2669/console
{code}
+ public void deleteFamily(HRegionInfo region, byte[] familyName)
{code}
Please rename the above method deleteFamilyFromFS().
{code}
+ if (fs.delete(delDir, true) == false) {
+ throw new IOException(Could not delete family
{code}
Indent the throw statement to the right by two spaces.
{code}
+ + Bytes.toString(familyName) + from FileSystem for region
+ + region.getTableNameAsString());
{code}
I think getTableNameAsString() shouldn't be used because you want to tell user
the name of the region.
For TestTableDeleteFamilyHandler.java:
{code}
+ * Copyright 2012 The Apache Software Foundation
{code}
Year is not needed in license header.
HDFS space is not reclaimed when a column family is deleted
---
Key: HBASE-6564
URL: https://issues.apache.org/jira/browse/HBASE-6564
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.94.1
Reporter: J Mohamed Zahoor
Assignee: J Mohamed Zahoor
Priority: Minor
Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch,
HBASE-6564-v3.patch, HBASE-6564-v4.patch
When a column family of a table is deleted, the HDFS space of the column
family does not seem to be reclaimed even after a major compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440687#comment-13440687
]
Zhihong Ted Yu commented on HBASE-6524:
---
I received the following from Jonathan:
{code}
with htrace:
Failed tests: testWALPlayer(org.apache.hadoop.hbase.mapreduce.TestWALPlayer):
expected:0 but was:1
testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)
testMROnTableWithTimestamp(org.apache.hadoop.hbase.mapreduce.TestImportTsv)
testMROnTableWithCustomMapper(org.apache.hadoop.hbase.mapreduce.TestImportTsv)
testRowCounterExclusiveColumn(org.apache.hadoop.hbase.mapreduce.TestRowCounter)
testRowCounterHiddenColumn(org.apache.hadoop.hbase.mapreduce.TestRowCounter)
Tests in error:
testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
testMROnTable(org.apache.hadoop.hbase.mapreduce.TestImportTsv)
testBulkOutputWithoutAnExistingTable(org.apache.hadoop.hbase.mapreduce.TestImportTsv)
testRowCounterNoColumn(org.apache.hadoop.hbase.mapreduce.TestRowCounter)
testMultithreadedTableMapper(org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper)
testSimpleCase(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testMetaExport(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testWithDeletes(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testScanOBBToOPP(org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan)
testGetRowVersions(org.apache.hadoop.hbase.TestMultiVersions): Shutting down
testScanMultipleVersions(org.apache.hadoop.hbase.TestMultiVersions):
org.apache.hadoop.hbase.MasterNotRunningException: Can create a proxy to
master, but it is not running
testMultiRegionTable(org.apache.hadoop.hbase.mapred.TestTableMapReduce): Job
failed!
testUpgrade(org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding):
Shutting down
{code}
As you can see, the count was much higher than 3
Please refer to
https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-2.0.0/143/testReport/
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Assignee: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-6524: -- Attachment: 6524-v2.txt Patch v2 changes htrace version to 1.49 Also adds license headers to new files. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Assignee: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, 6524-v2.txt, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440738#comment-13440738
]
Zhihong Ted Yu commented on HBASE-6524:
---
Here is the OS:
Linux s-2 2.6.38-11-generic #48-Ubuntu SMP Fri Jul 29 19:02:55 UTC 2011 x86_64
x86_64 x86_64 GNU/Linux
I got the following failures with htrace patch:
{code}
testGetRowVersions(org.apache.hadoop.hbase.TestMultiVersions) Time elapsed:
201.963 sec ERROR!
java.io.IOException: Shutting down
at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:229)
at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:92)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:688)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:661)
at
org.apache.hadoop.hbase.TestMultiVersions.testGetRowVersions(TestMultiVersions.java:143)
...
Caused by: java.lang.RuntimeException: Master not initialized after 200 seconds
at
org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:208)
at
org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:424)
at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:222)
testScanMultipleVersions(org.apache.hadoop.hbase.TestMultiVersions) Time
elapsed: 70.623 sec ERROR!
org.apache.hadoop.hbase.MasterNotRunningException:
org.apache.hadoop.hbase.MasterNotRunningException: Can create a proxy to
master, but it is not running
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.createMasterWithRetries(HConnectionManager.java:782)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterProtocol(HConnectionManager.java:1618)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterAdmin(HConnectionManager.java:1649)
at org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2049)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:510)
at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:397)
at
org.apache.hadoop.hbase.TestMultiVersions.testScanMultipleVersions(TestMultiVersions.java:199)
{code}
The test passed without patch.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Assignee: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, 6524-v2.txt, createTableTrace.png,
hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440835#comment-13440835
]
Zhihong Ted Yu commented on HBASE-6524:
---
Looks like TestMultiVersions is broken in trunk, as evidenced by trunk build
#3260 and the above QA report.
Here is list of failed tests with htrace patch against hadoop 2.0:
{code}
Failed tests:
testMultiSlaveReplication(org.apache.hadoop.hbase.replication.TestMultiSlaveReplication):
Waited too much time for put replication
Tests in error:
testGetRowVersions(org.apache.hadoop.hbase.TestMultiVersions): Shutting down
testScanMultipleVersions(org.apache.hadoop.hbase.TestMultiVersions):
org.apache.hadoop.hbase.MasterNotRunningException: Can create a proxy to
master, but it is not running
testDisablingTableRegionsAssignmentDuringCleanClusterStartup(org.apache.hadoop.hbase.master.TestAssignmentManager):
Problem binding to sea-lab-0/10.249.196.101:6 : Address already in use
testStopDuringStart(org.apache.hadoop.hbase.master.TestMasterNoCluster):
Problem binding to sea-lab-0/10.249.196.101:6 : Address already in use
testFailover(org.apache.hadoop.hbase.master.TestMasterNoCluster): Problem
binding to sea-lab-0/10.249.196.101:6 : Address already in use
testCatalogDeploys(org.apache.hadoop.hbase.master.TestMasterNoCluster):
Problem binding to sea-lab-0/10.249.196.101:6 : Address already in use
testDisableInactivePeer(org.apache.hadoop.hbase.replication.TestReplication):
Shutting down
testSimpleCase(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testMetaExport(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testWithDeletes(org.apache.hadoop.hbase.mapreduce.TestImportExport)
{code}
bq. Unless the calls into htrace are stalling somehow?
That was my guess as well.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Assignee: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, 6524-v2.txt, createTableTrace.png,
hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5169) Group of Region Server, a subtask of issue 4120
[ https://issues.apache.org/jira/browse/HBASE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440840#comment-13440840 ] Zhihong Ted Yu commented on HBASE-5169: --- Please take a look at the following thread: http://search-hadoop.com/m/pSRIf1BGMIN1/Code+review+request+for+hbase-4120+table+prioritysubj=Re+Code+review+request+for+hbase+4120+table+priority Group of Region Server, a subtask of issue 4120 - Key: HBASE-5169 URL: https://issues.apache.org/jira/browse/HBASE-5169 Project: HBase Issue Type: Sub-task Components: master Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.96.0 Attachments: GroupOfRegionServer_v1.patch, GroupOfRegionServer_v2.patch This is a subtask of issue 4120,this patch provides the region server group feature of HBase. With this patch, region servers can be divided into groups,one table could belong to one or more groups while the region server can only belong to one group. Work load in defferent groups will not affect each other. This patch provides table level and group level load balance,the default load balance and region assignments will consider the group configuration and assign regions to their corresponding groups. More information, please check out the documents of issue 4120. There is a web tool of this patch providing operations of group managements like add/delete group, move in/out servers,change table's group attribute ,balance groups, balance tables. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6647) [performance regression] appendNoSync/HBASE-4528 doesn't take deferred log flush into account
[ https://issues.apache.org/jira/browse/HBASE-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440841#comment-13440841 ] Zhihong Ted Yu commented on HBASE-6647: --- Patch looks good. [performance regression] appendNoSync/HBASE-4528 doesn't take deferred log flush into account - Key: HBASE-6647 URL: https://issues.apache.org/jira/browse/HBASE-6647 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Jean-Daniel Cryans Fix For: 0.96.0, 0.94.2 Attachments: HBASE-6647-0.94.patch Since we upgraded to 0.94.1 from 0.92 I saw that our ICVs are about twice as slow as they were. jstack'ing I saw that most of the time we are waiting on sync()... but those tables have deferred log flush turned on so they shouldn't even be calling it. HTD.isDeferredLogFlush is currently only called in the append() methods which are pretty much not in use anymore. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440874#comment-13440874
]
Zhihong Ted Yu commented on HBASE-6524:
---
I ran through tests against hadoop 2.0 on another box.
Here is list of failed tests:
{code}
Failed tests:
queueFailover(org.apache.hadoop.hbase.replication.TestReplication): Waited too
much time for queueFailover replication. Waited 33733ms.
Tests in error:
loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel): test
timed out after 12 milliseconds
loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel): test
timed out after 12 milliseconds
loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test
timed out after 12 milliseconds
testSimpleCase(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testMetaExport(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testWithDeletes(org.apache.hadoop.hbase.mapreduce.TestImportExport)
{code}
The list is shorter than the one where patch was applied.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Assignee: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, 6524-v2.txt, createTableTrace.png,
hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440874#comment-13440874
]
Zhihong Ted Yu edited comment on HBASE-6524 at 8/24/12 1:06 PM:
I ran through tests using freshly checked out workspace against hadoop 2.0 on
another box.
Here is list of failed tests:
{code}
Failed tests:
queueFailover(org.apache.hadoop.hbase.replication.TestReplication): Waited too
much time for queueFailover replication. Waited 33733ms.
Tests in error:
loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel): test
timed out after 12 milliseconds
loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel): test
timed out after 12 milliseconds
loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test
timed out after 12 milliseconds
testSimpleCase(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testMetaExport(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testWithDeletes(org.apache.hadoop.hbase.mapreduce.TestImportExport)
{code}
The list is shorter than the one where patch was applied.
was (Author: zhi...@ebaysf.com):
I ran through tests against hadoop 2.0 on another box.
Here is list of failed tests:
{code}
Failed tests:
queueFailover(org.apache.hadoop.hbase.replication.TestReplication): Waited too
much time for queueFailover replication. Waited 33733ms.
Tests in error:
loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel): test
timed out after 12 milliseconds
loadTest[1](org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel): test
timed out after 12 milliseconds
loadTest[0](org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential): test
timed out after 12 milliseconds
testSimpleCase(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testMetaExport(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testWithDeletes(org.apache.hadoop.hbase.mapreduce.TestImportExport)
{code}
The list is shorter than the one where patch was applied.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Assignee: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, 6524-v2.txt, createTableTrace.png,
hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-3271:
--
Status: Open (was: Patch Available)
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0
Attachments: HBASE-3271.patch, HBASE-3271-v2.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-3271:
--
Attachment: 3271.94
Patch for 0.94 branch.
TestImportExport passes.
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0
Attachments: 3271.94, HBASE-3271.patch, HBASE-3271-v2.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-3271:
--
Fix Version/s: 0.94.2
Release Note: mapreduce.Export is able to export .META. table
Integrated to 0.94 branch as well.
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0, 0.94.2
Attachments: 3271.94, HBASE-3271.patch, HBASE-3271-v2.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6650) Import .META. table exported from 0.94
Zhihong Ted Yu created HBASE-6650: - Summary: Import .META. table exported from 0.94 Key: HBASE-6650 URL: https://issues.apache.org/jira/browse/HBASE-6650 Project: HBase Issue Type: Sub-task Reporter: Zhihong Ted Yu HBASE-6052 converts .META. and ROOT table content to protobuf. This JIRA allows .META. table exported from 0.94 (see HBASE-3271) to be imported into live cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6586) Quarantine Corrupted HFiles
[
https://issues.apache.org/jira/browse/HBASE-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440901#comment-13440901
]
Zhihong Ted Yu commented on HBASE-6586:
---
@Jon:
As long as Hadoop QA shows green build (barring flaky tests) and your test
doesn't turn up surprises, I am fine with integration.
Quarantine Corrupted HFiles
---
Key: HBASE-6586
URL: https://issues.apache.org/jira/browse/HBASE-6586
Project: HBase
Issue Type: Improvement
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: 0001-hbase-6568-hbck-quarantine-v6.patch,
hbase-6586-92-v3.patch, hbase-6586-92-v8.patch, hbase-6586-94-v3.patch,
hbase-6586-94-v8.patch, hbase-6586.patch, hbase-6586-trunk-v3.patch,
hbase-6586-trunk-v4.patch, hbase-6586-trunk-v5.patch,
hbase-6586-trunk-v6.patch, hbase-6586-trunk-v7.patch,
hbase-6586-trunk-v8.patch
We've encountered a few upgrades from 0.90 hbases + 20.2/1.x hdfs to 0.92
hbases + hdfs 2.x that get stuck. I haven't been able to duplicate the
problem in my dev environment but we suspect this may be related to
HDFS-3731. On the HBase side, it seems reasonable to quarantine what are
most likely truncated hfiles, so that can could later be recovered.
Here's an example of the exception we've encountered:
{code}
2012-07-18 05:55:01,152 ERROR handler.OpenRegionHandler
(OpenRegionHandler.java:openRegion(346)) - Failed open of
region=user_mappings,080112102AA76EF98197605D341B9E6C5824D2BC|1001,1317824890618.eaed0e7abc6d27d28ff0e5a9b49c4c
0d.
java.io.IOException: java.lang.IllegalArgumentException: Invalid HFile
version: 842220600 (expected to be between 1 and 2)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:306)
at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:371)
at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:387)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1026)
at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:485)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:566)
at org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:286)
at org.apache.hadoop.hbase.regionserver.Store.init(Store.java:223)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:2534)
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:454)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3282)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3230)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:331)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:107)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalArgumentException: Invalid HFile version:
842220600 (expected to be between 1 and 2)
at org.apache.hadoop.hbase.io.hfile.HFile.checkFormatVersion(HFile.java:515)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:303)
... 17 more
{code}
Specifically -- the FixedFileTrailer are incorrect, and seemingly missing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[ https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439628#comment-13439628 ] Zhihong Ted Yu commented on HBASE-6581: --- Meanwhile, I think we should maintain no more than 3 test failures for hadoop-2.0 profile. Build with hadoop.profile=3.0 - Key: HBASE-6581 URL: https://issues.apache.org/jira/browse/HBASE-6581 Project: HBase Issue Type: Bug Reporter: Eric Charles Attachments: HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581.diff Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT instead of 3.0.0-SNAPSHOT in hbase-common). I can provide a patch that would move most of hadoop dependencies in their respective profiles and will define the correct hadoop deps in the 3.0 profile. Please tell me if that's ok to go this way. Thx, Eric [1] $ mvn clean install -Dhadoop.profile=3.0 [INFO] Scanning for projects... [ERROR] The build could not read 3 projects - [Help 1] [ERROR] [ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT (/d/hbase.svn/hbase-server/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT (/d/hbase.svn/hbase-common/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT (/d/hbase.svn/hbase-it/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 220, column 18 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 224, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 228, column 21 [ERROR] -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2155) Add the option to bind to a specific IP address to the Nonblocking Thrift servers
[
https://issues.apache.org/jira/browse/HBASE-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439632#comment-13439632
]
Zhihong Ted Yu commented on HBASE-2155:
---
What about this server type :
{code}
THREADED_SELECTOR(
threadedselector, true, TThreadedSelectorServer.class, false);
{code}
Does the change apply to the above type ?
Add the option to bind to a specific IP address to the Nonblocking Thrift
servers
-
Key: HBASE-2155
URL: https://issues.apache.org/jira/browse/HBASE-2155
Project: HBase
Issue Type: Improvement
Components: thrift
Reporter: Lars Francke
Assignee: Lars Francke
Priority: Minor
Attachments: HBASE-2155.patch
This is not possible in Thrift 0.2.0 so we'll have to wait until the next
version is released (which includes THRIFT-684). After that is released this
is an easy and quick fix. For a few more details see HBASE-1373 and HBASE-65.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439633#comment-13439633 ] Zhihong Ted Yu commented on HBASE-6524: --- There were 90 more failed tests introduced for hadoop 2.0 profile. I suggest we temporarily rollback the patch and give Jonathan enough time to iron out the 90+ test failures. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6634) REST API ScannerModel's protobuf converter code duplicates the setBatch call
[ https://issues.apache.org/jira/browse/HBASE-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439640#comment-13439640 ] Zhihong Ted Yu commented on HBASE-6634: --- +1 on patch. REST API ScannerModel's protobuf converter code duplicates the setBatch call Key: HBASE-6634 URL: https://issues.apache.org/jira/browse/HBASE-6634 Project: HBase Issue Type: Bug Components: rest Affects Versions: 0.94.0 Reporter: Harsh J Assignee: Harsh J Priority: Trivial Attachments: HBASE-6634.patch There's a dupe call to setBatch when a scanner model object is created for protobuf outputs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6364) Powering down the server host holding the .META. table causes HBase Client to take excessively long to recover and connect to reassigned .META. table
[ https://issues.apache.org/jira/browse/HBASE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439655#comment-13439655 ] Zhihong Ted Yu commented on HBASE-6364: --- Addendum looks good to me. Powering down the server host holding the .META. table causes HBase Client to take excessively long to recover and connect to reassigned .META. table - Key: HBASE-6364 URL: https://issues.apache.org/jira/browse/HBASE-6364 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.90.6, 0.92.1, 0.94.0 Reporter: Suraj Varma Assignee: nkeywal Labels: client Fix For: 0.96.0, 0.94.2 Attachments: 6364.94.v2.nolargetest.patch, 6364.94.v2.nolargetest.security-addendum.patch, 6364-host-serving-META.v1.patch, 6364.v11.nolargetest.patch, 6364.v1.patch, 6364.v1.patch, 6364.v2.patch, 6364.v3.patch, 6364.v3.patch, 6364.v5.patch, 6364.v5.withtests.patch, 6364.v6.patch, 6364.v6.withtests.patch, 6364.v7.withtests.patch, 6364.v8.withtests.patch, 6364.v9.patch, stacktrace.txt When a server host with a Region Server holding the .META. table is powered down on a live cluster, while the HBase cluster itself detects and reassigns the .META. table, connected HBase Client's take an excessively long time to detect this and re-discover the reassigned .META. Workaround: Decrease the ipc.socket.timeout on HBase Client side to a low value (default is 20s leading to 35 minute recovery time; we were able to get acceptable results with 100ms getting a 3 minute recovery) This was found during some hardware failure testing scenarios. Test Case: 1) Apply load via client app on HBase cluster for several minutes 2) Power down the region server holding the .META. server (i.e. power off ... and keep it off) 3) Measure how long it takes for cluster to reassign META table and for client threads to re-lookup and re-orient to the lesser cluster (minus the RS and DN on that host). Observation: 1) Client threads spike up to maxThreads size ... and take over 35 mins to recover (i.e. for the thread count to go back to normal) - no client calls are serviced - they just back up on a synchronized method (see #2 below) 2) All the client app threads queue up behind the oahh.ipc.HBaseClient#setupIOStreams method http://tinyurl.com/7js53dj After taking several thread dumps we found that the thread within this synchronized method was blocked on NetUtils.connect(this.socket, remoteId.getAddress(), getSocketTimeout(conf)); The client thread that gets the synchronized lock would try to connect to the dead RS (till socket times out after 20s), retries, and then the next thread gets in and so forth in a serial manner. Workaround: --- Default ipc.socket.timeout is set to 20s. We dropped this to a low number (1000 ms, 100 ms, etc) on the client side hbase-site.xml. With this setting, the client threads recovered in a couple of minutes by failing fast and re-discovering the .META. table on a reassigned RS. Assumption: This ipc.socket.timeout is only ever used during the initial HConnection setup via the NetUtils.connect and should only ever be used when connectivity to a region server is lost and needs to be re-established. i.e it does not affect the normal RPC actiivity as this is just the connect timeout. During RS GC periods, any _new_ clients trying to connect will fail and will require .META. table re-lookups. This above timeout workaround is only for the HBase client side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-3271:
--
Fix Version/s: 0.96.0
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
nit: the following line is too long:
{code}
+ InputSplit split = new TableSplit(table.getTableName(),
HConstants.EMPTY_BYTE_ARRAY, HConstants.EMPTY_BYTE_ARRAY,
{code}
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0
Attachments: HBASE-3271.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439684#comment-13439684
]
Zhihong Ted Yu edited comment on HBASE-3271 at 8/23/12 4:07 AM:
nit: the following two lines are too long:
{code}
+GenericOptionsParser opts = new GenericOptionsParser(new
Configuration(cluster.getConfiguration()), args);
...
+ InputSplit split = new TableSplit(table.getTableName(),
HConstants.EMPTY_BYTE_ARRAY, HConstants.EMPTY_BYTE_ARRAY,
{code}
was (Author: zhi...@ebaysf.com):
nit: the following line is too long:
{code}
+ InputSplit split = new TableSplit(table.getTableName(),
HConstants.EMPTY_BYTE_ARRAY, HConstants.EMPTY_BYTE_ARRAY,
{code}
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0
Attachments: HBASE-3271.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6610) HFileLink: Hardlink alternative for snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439709#comment-13439709 ] Zhihong Ted Yu commented on HBASE-6610: --- TestStore#testDeleteExpiredStoreFiles failure seems to be related to the patch. HFileLink: Hardlink alternative for snapshot restore Key: HBASE-6610 URL: https://issues.apache.org/jira/browse/HBASE-6610 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.96.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Labels: snapshot Fix For: 0.96.0 Attachments: HBASE-6610-v1.patch To avoid copying data during restore snapshot we need to introduce an HFile Link that allows to reference a file that can be in the original path (/hbase/table/region/cf/hfile) or, if the file is archived, in the archive directory (/hbase/.archive/table/region/cf/hfile). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439802#comment-13439802
]
Zhihong Ted Yu commented on HBASE-6524:
---
Using snapshot build of htrace 1.49, I was able to run TestClusterBootOrder
with hadoop 2.0 profile.
But the test failed with:
{code}
testBootRegionServerFirst(org.apache.hadoop.hbase.TestClusterBootOrder) Time
elapsed: 0.263 sec ERROR!
java.lang.IncompatibleClassChangeError: Implementing class
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at
org.apache.hadoop.hbase.mapreduce.MapreduceTestingShim.clinit(MapreduceTestingShim.java:45)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createDirsAndSetProperties(HBaseTestingUtility.java:475)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:426)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:392)
at
org.apache.hadoop.hbase.TestClusterBootOrder.setUp(TestClusterBootOrder.java:47)
{code}
The test passed with hadoop 1.0 profile.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439814#comment-13439814 ] Zhihong Ted Yu commented on HBASE-6524: --- The above error might be related to local .m2 repo. I will wait till maven repo is populated. In the mean time, please run the whole test suite with hadoop 2.0 profile. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439867#comment-13439867
]
Zhihong Ted Yu commented on HBASE-5329:
---
Thanks for activating this JIRA.
{code}
+ do {
+ lockName = String.valueOf(nextLong());
+ } while (rowlocks.putIfAbsent(lockName, r) != null);
{code}
Indentation is off for the above block.
{code}
+return Long.parseLong(lockName);
{code}
I prefer the old way of keeping long lockId so that the above parsing can be
omitted.
addRowLock() may allocate duplicate lock id, causing the client to be blocked
-
Key: HBASE-5329
URL: https://issues.apache.org/jira/browse/HBASE-5329
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.3
Environment: Red Hat Enterprise Linux Server release 5.4
Reporter: liaoxiangui
Assignee: Zhihong Ted Yu
Priority: Minor
Attachments: HBASE-5329.patch
{code}
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
{code}
In addRowLock(),rand may generate duplicated lock id, it may cause
regionserver throw exception(Leases$LeaseStillHeldException).The client will
be blocked until old rowlock is released.
{code}
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439882#comment-13439882
]
Zhihong Ted Yu commented on HBASE-6524:
---
With the following change:
{code}
Index: hbase-server/pom.xml
===
--- hbase-server/pom.xml(revision 1376206)
+++ hbase-server/pom.xml(working copy)
@@ -454,7 +454,7 @@
dependency
groupIdorg.cloudera.htrace/groupId
artifactIdhtrace/artifactId
- version1.48/version
+ version1.49/version
/dependency
/dependencies
profiles
{code}
I got:
{code}
[ERROR] Failed to execute goal on project hbase-server: Could not resolve
dependencies for project org.apache.hbase:hbase-server:jar:0.95-SNAPSHOT:
Failure to find org.cloudera.htrace:htrace:jar:1.49 in
http://repository-netty.forge.cloudbees.com/snapshot/ was cached in the local
repository, resolution will not be reattempted until the update interval of
cloudbees netty has elapsed or updates are forced - [Help 1]
{code}
Please list the failed tests when hadoop 2.0 profile is used.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439887#comment-13439887 ] Zhihong Ted Yu commented on HBASE-6524: --- Using another Linux box I was able to run tests against hadoop 2.0 Will see what tests fail. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6586) Quarantine Corrupted HFiles
[
https://issues.apache.org/jira/browse/HBASE-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439889#comment-13439889
]
Zhihong Ted Yu commented on HBASE-6586:
---
@Benoit:
What do you think of the proposed exception hierarchy in the patch ?
Thanks
Quarantine Corrupted HFiles
---
Key: HBASE-6586
URL: https://issues.apache.org/jira/browse/HBASE-6586
Project: HBase
Issue Type: Improvement
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: 0001-hbase-6568-hbck-quarantine-v6.patch,
hbase-6586-92-v3.patch, hbase-6586-94-v3.patch, hbase-6586.patch,
hbase-6586-trunk-v3.patch, hbase-6586-trunk-v4.patch
We've encountered a few upgrades from 0.90 hbases + 20.2/1.x hdfs to 0.92
hbases + hdfs 2.x that get stuck. I haven't been able to duplicate the
problem in my dev environment but we suspect this may be related to
HDFS-3731. On the HBase side, it seems reasonable to quarantine what are
most likely truncated hfiles, so that can could later be recovered.
Here's an example of the exception we've encountered:
{code}
2012-07-18 05:55:01,152 ERROR handler.OpenRegionHandler
(OpenRegionHandler.java:openRegion(346)) - Failed open of
region=user_mappings,080112102AA76EF98197605D341B9E6C5824D2BC|1001,1317824890618.eaed0e7abc6d27d28ff0e5a9b49c4c
0d.
java.io.IOException: java.lang.IllegalArgumentException: Invalid HFile
version: 842220600 (expected to be between 1 and 2)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:306)
at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:371)
at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:387)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1026)
at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:485)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:566)
at org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:286)
at org.apache.hadoop.hbase.regionserver.Store.init(Store.java:223)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:2534)
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:454)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3282)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3230)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:331)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:107)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalArgumentException: Invalid HFile version:
842220600 (expected to be between 1 and 2)
at org.apache.hadoop.hbase.io.hfile.HFile.checkFormatVersion(HFile.java:515)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:303)
... 17 more
{code}
Specifically -- the FixedFileTrailer are incorrect, and seemingly missing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-6524: -- Attachment: 6524.addendum Addendum uses 1.49 of htrace Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-6524: -- Hadoop Flags: (was: Reviewed) Status: Patch Available (was: Reopened) Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439954#comment-13439954
]
Zhihong Ted Yu commented on HBASE-6524:
---
I got a few test failures running test suite:
{code}
Running org.apache.hadoop.hbase.mapreduce.TestImportExport
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 51.918 sec
FAILURE!
--
Running org.apache.hadoop.hbase.client.TestHCM
Tests run: 7, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 72.583 sec
FAILURE!
--
Running org.apache.hadoop.hbase.client.TestAdmin
Tests run: 40, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 129.109 sec
FAILURE!
--
Running org.apache.hadoop.hbase.TestFullLogReconstruction
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 44.385 sec
FAILURE!
--
Running org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient
Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 178.483 sec
FAILURE!
{code}
I ran the following:
mvn -Dhadoop.profile=2.0 -P localTests test -Dtest=TestHCM#testRegionCaching
and got:
{code}
testRegionCaching(org.apache.hadoop.hbase.client.TestHCM) Time elapsed: 60.047
sec ERROR!
java.lang.Exception: test timed out after 6 milliseconds
at
java.util.concurrent.ConcurrentHashMap$Segment.get(ConcurrentHashMap.java:344)
at
java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:769)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getOnlineRegion(HRegionServer.java:2596)
at
org.apache.hadoop.hbase.client.TestHCM.testRegionCaching(TestHCM.java:244)
{code}
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439956#comment-13439956
]
Zhihong Ted Yu commented on HBASE-6524:
---
Two other failed tests:
{code}
testCreateTableRPCTimeOut(org.apache.hadoop.hbase.client.TestAdmin) Time
elapsed: 15.187 sec ERROR!
org.apache.hadoop.hbase.TableNotEnabledException: Retries exhausted while still
waiting for table: testCreateTableRPCTimeOut to be enabled
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:460)
at
org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:373)
at
org.apache.hadoop.hbase.client.TestAdmin.testCreateTableRPCTimeOut(TestAdmin.java:1125)
{code}
{code}
testReconstruction(org.apache.hadoop.hbase.TestFullLogReconstruction) Time
elapsed: 26.141 sec ERROR!
java.lang.NullPointerException
at
org.apache.hadoop.hbase.HBaseTestingUtility.createMultiRegions(HBaseTestingUtility.java:1211)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createMultiRegions(HBaseTestingUtility.java:1142)
at
org.apache.hadoop.hbase.HBaseTestingUtility.createMultiRegions(HBaseTestingUtility.java:1116)
at
org.apache.hadoop.hbase.TestFullLogReconstruction.testReconstruction(TestFullLogReconstruction.java:99)
{code}
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439989#comment-13439989
]
Zhihong Ted Yu commented on HBASE-6524:
---
Here is the list of failed tests:
{code}
Failed tests:
testMultipleTables(org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient):
Archived HFiles
(hdfs://localhost:58500/user/zhihyu/hbase/.archive/otherTable/8f2bf3dfdf9ce281bec682f7160e4098/fam)
should have gotten deleted, but didn't, remaining files:...
Tests in error:
testDisableInactivePeer(org.apache.hadoop.hbase.replication.TestReplication):
Shutting down
testCreateTableRPCTimeOut(org.apache.hadoop.hbase.client.TestAdmin): Retries
exhausted while still waiting for table: testCreateTableRPCTimeOut to be enabled
testGetRowVersions(org.apache.hadoop.hbase.TestMultiVersions): Shutting down
testScanMultipleVersions(org.apache.hadoop.hbase.TestMultiVersions):
org.apache.hadoop.hbase.MasterNotRunningException: Can create a proxy to
master, but it is not running
{code}
I also saw two hanging tests:
{code}
main prio=10 tid=0x40544800 nid=0x1e09 in Object.wait()
[0x7ffc85ca6000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on 0xa8807010 (a
org.apache.hadoop.hbase.util.JVMClusterUtil$MasterThread)
at java.lang.Thread.join(Thread.java:1194)
- locked 0xa8807010 (a
org.apache.hadoop.hbase.util.JVMClusterUtil$MasterThread)
at
org.apache.hadoop.hbase.util.Threads.threadDumpingIsAlive(Threads.java:116)
at
org.apache.hadoop.hbase.LocalHBaseCluster.join(LocalHBaseCluster.java:405)
at
org.apache.hadoop.hbase.MiniHBaseCluster.join(MiniHBaseCluster.java:408)
at
org.apache.hadoop.hbase.HBaseTestingUtility.shutdownMiniHBaseCluster(HBaseTestingUtility.java:599)
at
org.apache.hadoop.hbase.HBaseTestingUtility.shutdownMiniCluster(HBaseTestingUtility.java:573)
at
org.apache.hadoop.hbase.rest.TestVersionResource.tearDownAfterClass(TestVersionResource.java:69)
{code}
{code}
main prio=10 tid=0x40d5b800 nid=0x64df waiting on condition
[0x7f50b379b000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hbase.zookeeper.ZKUtil.blockUntilAvailable(ZKUtil.java:1225)
at
org.apache.hadoop.hbase.zookeeper.RootRegionTracker.blockUntilAvailable(RootRegionTracker.java:176)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:940)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1056)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:955)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1056)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:959)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:923)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:834)
at
org.apache.hadoop.hbase.client.ServerCallable.connect(ServerCallable.java:88)
at
org.apache.hadoop.hbase.client.ScannerCallable.connect(ScannerCallable.java:95)
at
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:168)
at
org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:212)
at
org.apache.hadoop.hbase.client.ClientScanner.init(ClientScanner.java:127)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:649)
at
org.apache.hadoop.hbase.replication.TestReplication.setUp(TestReplication.java:183)
{code}
In order to clear all the above, I think more time is needed.
Rolling back the patch would allow HBASE-6581 to proceed.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more
[jira] [Commented] (HBASE-6052) Convert .META. and -ROOT- content to pb
[ https://issues.apache.org/jira/browse/HBASE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439990#comment-13439990 ] Zhihong Ted Yu commented on HBASE-6052: --- @Enis: Thanks for the update. Can you attach TestMetaMigrationConvertToPB.tgz to this JIRA, if it has changed ? Convert .META. and -ROOT- content to pb --- Key: HBASE-6052 URL: https://issues.apache.org/jira/browse/HBASE-6052 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: Enis Soztutar Priority: Blocker Fix For: 0.96.0 Attachments: 6052-v5.txt, HBASE-6052_v1.patch, HBASE-6052_v2.patch, HBASE-6052_v3.patch, HBASE-6052_v4.patch, HBASE-6052_v4.patch, HBASE-6052_v7.patch, TestMetaMigrationConvertToPB.tgz -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439997#comment-13439997 ] Zhihong Ted Yu commented on HBASE-6524: --- I actually trimmed a few tests which passed when run individually from my list posted @ 23/Aug/12 00:57 I agree the tests in list might just be flaky. HBASE-6627 only covers one of the TestMultiVersions tests. I think we should add a QA build for hadoop 2.0 so that we don't have to use trunk build to discover these failed tests. Thanks for your persistence, Jonathan. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6627) TestMultiVersions.testGetRowVersions is flaky
[ https://issues.apache.org/jira/browse/HBASE-6627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439998#comment-13439998 ] Zhihong Ted Yu commented on HBASE-6627: --- @N: Do you mind submitting for another QA run since the previous attempt didn't give us results ? TestMultiVersions.testGetRowVersions is flaky - Key: HBASE-6627 URL: https://issues.apache.org/jira/browse/HBASE-6627 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Environment: hadoop-qa mainly, seems to happen tests in parallel; difficult to reproduce on a single test. Reporter: nkeywal Assignee: nkeywal Attachments: 6627.v1.patch org.apache.hadoop.hbase.TestMultiVersions.testGetRowVersions Shutting down Stacktrace java.io.IOException: Shutting down at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:229) at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:92) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:688) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:661) at org.apache.hadoop.hbase.TestMultiVersions.testGetRowVersions(TestMultiVersions.java:143) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440017#comment-13440017 ] Zhihong Ted Yu commented on HBASE-6524: --- Will revert the patch late tonight if I don't hear objection. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-2155) Add the option to bind to a specific IP address to the Nonblocking Thrift servers
[ https://issues.apache.org/jira/browse/HBASE-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu reassigned HBASE-2155: - Assignee: liang xie (was: Lars Francke) Add the option to bind to a specific IP address to the Nonblocking Thrift servers - Key: HBASE-2155 URL: https://issues.apache.org/jira/browse/HBASE-2155 Project: HBase Issue Type: Improvement Components: thrift Reporter: Lars Francke Assignee: liang xie Priority: Minor Attachments: HBASE-2155.patch This is not possible in Thrift 0.2.0 so we'll have to wait until the next version is released (which includes THRIFT-684). After that is released this is an easy and quick fix. For a few more details see HBASE-1373 and HBASE-65. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2155) Add the option to bind to a specific IP address to the Nonblocking Thrift servers
[ https://issues.apache.org/jira/browse/HBASE-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-2155: -- Fix Version/s: 0.96.0 Hadoop Flags: Reviewed Status: Patch Available (was: Open) Latest patch looks good. Add the option to bind to a specific IP address to the Nonblocking Thrift servers - Key: HBASE-2155 URL: https://issues.apache.org/jira/browse/HBASE-2155 Project: HBase Issue Type: Improvement Components: thrift Reporter: Lars Francke Assignee: liang xie Priority: Minor Fix For: 0.96.0 Attachments: HBASE-2155.patch This is not possible in Thrift 0.2.0 so we'll have to wait until the next version is released (which includes THRIFT-684). After that is released this is an easy and quick fix. For a few more details see HBASE-1373 and HBASE-65. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-5329:
--
Attachment: 5329-v2.patch
Patch v2 does some cosmetics changes based on Ian's patch.
addRowLock() may allocate duplicate lock id, causing the client to be blocked
-
Key: HBASE-5329
URL: https://issues.apache.org/jira/browse/HBASE-5329
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.3
Environment: Red Hat Enterprise Linux Server release 5.4
Reporter: liaoxiangui
Assignee: Zhihong Ted Yu
Priority: Minor
Fix For: 0.96.0
Attachments: 5329-v2.patch, HBASE-5329.patch
{code}
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
{code}
In addRowLock(),rand may generate duplicated lock id, it may cause
regionserver throw exception(Leases$LeaseStillHeldException).The client will
be blocked until old rowlock is released.
{code}
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-5329:
--
Fix Version/s: 0.96.0
Status: Patch Available (was: Open)
addRowLock() may allocate duplicate lock id, causing the client to be blocked
-
Key: HBASE-5329
URL: https://issues.apache.org/jira/browse/HBASE-5329
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.3
Environment: Red Hat Enterprise Linux Server release 5.4
Reporter: liaoxiangui
Assignee: Zhihong Ted Yu
Priority: Minor
Fix For: 0.96.0
Attachments: 5329-v2.patch, HBASE-5329.patch
{code}
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
{code}
In addRowLock(),rand may generate duplicated lock id, it may cause
regionserver throw exception(Leases$LeaseStillHeldException).The client will
be blocked until old rowlock is released.
{code}
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu reassigned HBASE-5329:
-
Assignee: Ian Varley (was: Zhihong Ted Yu)
addRowLock() may allocate duplicate lock id, causing the client to be blocked
-
Key: HBASE-5329
URL: https://issues.apache.org/jira/browse/HBASE-5329
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.3
Environment: Red Hat Enterprise Linux Server release 5.4
Reporter: liaoxiangui
Assignee: Ian Varley
Priority: Minor
Fix For: 0.96.0
Attachments: 5329-v2.patch, HBASE-5329.patch
{code}
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
{code}
In addRowLock(),rand may generate duplicated lock id, it may cause
regionserver throw exception(Leases$LeaseStillHeldException).The client will
be blocked until old rowlock is released.
{code}
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6639) Class.newInstance() can throw any checked exceptions and must be encapsulated with catching Exception
[ https://issues.apache.org/jira/browse/HBASE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440029#comment-13440029 ] Zhihong Ted Yu commented on HBASE-6639: --- @Hiroshi: Can you clarify your observation ? See http://docs.oracle.com/javase/6/docs/api/java/lang/reflect/Constructor.html#newInstance%28java.lang.Object...%29 Class.newInstance() can throw any checked exceptions and must be encapsulated with catching Exception - Key: HBASE-6639 URL: https://issues.apache.org/jira/browse/HBASE-6639 Project: HBase Issue Type: Bug Affects Versions: 0.94.1 Reporter: Hiroshi Ikeda Priority: Minor There are some logics to call Class.newInstance() without catching Exception, for example, in the method CoprocessorHost.loadInstance(). Class.newInstance() is declared to throw InstantiationException and IllegalAccessException but indeed the method can throw any checked exceptions without declaration. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[
https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440030#comment-13440030
]
Zhihong Ted Yu commented on HBASE-6581:
---
Here is the command I used:
nohup mvn clean -Dhadoop.profile=3.0 -P localTests test suite.txt
TestConstraint failed with:
{code}
org.apache.hadoop.hbase.constraint.TestConstraint Time elapsed: 0 sec
ERROR!
java.io.IOException: Shutting down
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:229)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:92)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:688)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:661)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:656)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:598)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:567)
at
org.apache.hadoop.hbase.constraint.TestConstraint.setUpBeforeClass(TestConstraint.java:60)
...
Caused by: java.lang.RuntimeException: Master not initialized after 200 seconds
at
org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:208)
at
org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:424)
at
org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:222)
{code}
Build with hadoop.profile=3.0
-
Key: HBASE-6581
URL: https://issues.apache.org/jira/browse/HBASE-6581
Project: HBase
Issue Type: Bug
Reporter: Eric Charles
Attachments: HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581.diff
Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to
change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT
instead of 3.0.0-SNAPSHOT in hbase-common).
I can provide a patch that would move most of hadoop dependencies in their
respective profiles and will define the correct hadoop deps in the 3.0
profile.
Please tell me if that's ok to go this way.
Thx, Eric
[1]
$ mvn clean install -Dhadoop.profile=3.0
[INFO] Scanning for projects...
[ERROR] The build could not read 3 projects - [Help 1]
[ERROR]
[ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT
(/d/hbase.svn/hbase-server/pom.xml) has 3 errors
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21
[ERROR]
[ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT
(/d/hbase.svn/hbase-common/pom.xml) has 3 errors
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21
[ERROR]
[ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT
(/d/hbase.svn/hbase-it/pom.xml) has 3 errors
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-common:jar is missing. @ line 220, column 18
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-annotations:jar is missing. @ line 224, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 228, column 21
[ERROR]
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6618) Implement FuzzyRowFilter with ranges support
[ https://issues.apache.org/jira/browse/HBASE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440039#comment-13440039 ] Zhihong Ted Yu commented on HBASE-6618: --- Thanks for the update, Alex. I get your idea, though a few arrows seem to be missing (e.g. CCF is ?) in the diagram for toInc. Implement FuzzyRowFilter with ranges support Key: HBASE-6618 URL: https://issues.apache.org/jira/browse/HBASE-6618 Project: HBase Issue Type: New Feature Components: filters Reporter: Alex Baranau Priority: Minor Attachments: HBASE-6618-algo-desc-bits.png, HBASE-6618-algo.patch Apart from current ability to specify fuzzy row filter e.g. for userId_actionId format as _0004 (where 0004 - actionId) it would be great to also have ability to specify the fuzzy range , e.g. _0004, ..., _0099. See initial discussion here: http://search-hadoop.com/m/WVLJdX0Z65 Note: currently it is possible to provide multiple fuzzy row rules to existing FuzzyRowFilter, but in case when the range is big (contains thousands of values) it is not efficient. Filter should perform efficient fast-forwarding during the scan (this is what distinguishes it from regex row filter). While such functionality may seem like a proper fit for custom filter (i.e. not including into standard filter set) it looks like the filter may be very re-useable. We may judge based on the implementation that will hopefully be added. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440051#comment-13440051
]
Zhihong Ted Yu commented on HBASE-3271:
---
Please address my comment @ 22/Aug/12 18:03 and attach new patch.
Thanks
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0
Attachments: HBASE-3271.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6639) Class.newInstance() can throw any checked exceptions and must be encapsulated with catching Exception
[
https://issues.apache.org/jira/browse/HBASE-6639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440056#comment-13440056
]
Zhihong Ted Yu commented on HBASE-6639:
---
Looks like there is more than one place where Exception is not caught:
{code}
o = implClass.newInstance();
./hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java
HLog.Reader reader = logReaderClass.newInstance();
HLog.Writer writer = (HLog.Writer) logWriterClass.newInstance();
return keyClass.newInstance();
./hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
key = keyClass.newInstance();
./hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/SequenceFileLogReader.java
{code}
Class.newInstance() can throw any checked exceptions and must be encapsulated
with catching Exception
-
Key: HBASE-6639
URL: https://issues.apache.org/jira/browse/HBASE-6639
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.1
Reporter: Hiroshi Ikeda
Priority: Minor
There are some logics to call Class.newInstance() without catching Exception,
for example, in the method CoprocessorHost.loadInstance().
Class.newInstance() is declared to throw InstantiationException and
IllegalAccessException but indeed the method can throw any checked exceptions
without declaration.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440059#comment-13440059
]
Zhihong Ted Yu commented on HBASE-6524:
---
Reverted patch from trunk.
TestHTraceHooks.java missed license header.
{code}
+import org.cloudera.htrace.impl.NullSpan;
{code}
I think the intent of putting htrace into maven repo is for wider adoption.
I wonder if the above namespace (involving cloudera which should not be an org)
would serve that purpose well.
For HBaseLocalFileSpanReceiver.java, year is not needed in license header:
{code}
+ * Copyright 2010 The Apache Software Foundation
{code}
Let's conduct thorough testing against hadoop 2.0 profile
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6631) TestHMasterRPCException in 0.92 failed twice on socket timeout
[
https://issues.apache.org/jira/browse/HBASE-6631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439141#comment-13439141
]
Zhihong Ted Yu commented on HBASE-6631:
---
0.94 fails to compile, it seems:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase: Compilation failure: Compilation
failure:
[ERROR]
/Users/zhihyu/94hbase/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java:[52,73]
')' expected
[ERROR]
[ERROR]
/Users/zhihyu/94hbase/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java:[52,76]
not a statement
[ERROR]
[ERROR]
/Users/zhihyu/94hbase/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java:[52,78]
';' expected
[ERROR] - [Help 1]
{code}
TestHMasterRPCException in 0.92 failed twice on socket timeout
--
Key: HBASE-6631
URL: https://issues.apache.org/jira/browse/HBASE-6631
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.92.2, 0.94.2
Attachments: 6631-trunk.txt, 6631-trunk.txt, 6631.txt
#502 and #498 0.92 builds have TestHMasterRPCException failing because of
socket timeout when servernotrunning is expected. Socket timeout is 100ms
only.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439168#comment-13439168
]
Zhihong Ted Yu commented on HBASE-6524:
---
When I remove the patch from this JIRA, TestClusterBootOrder passed.
With patch, TestClusterBootOrder fails.
{code}
testBootRegionServerFirst(org.apache.hadoop.hbase.TestClusterBootOrder) Time
elapsed: 0.002 sec ERROR!
java.lang.NullPointerException
at
org.apache.hadoop.hbase.TestClusterBootOrder.tearDown(TestClusterBootOrder.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
{code}
Here is the command I used:
mvn clean -Dhadoop.profile=2.0 test
-Dtest=TestClusterBootOrder#testBootRegionServerFirst
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu reopened HBASE-6524: --- Hadoop 2.0 build is broken. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6622) TestUpgradeFromHFileV1ToEncoding#testUpgrade fails in trunk
[
https://issues.apache.org/jira/browse/HBASE-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439283#comment-13439283
]
Zhihong Ted Yu commented on HBASE-6622:
---
One possibility was that in Handler.run() method, started flag was false when
the first Call was received.
{code}
CurCall.set(call);
try {
if (!started)
throw new ServerNotRunningYetException(Server is not running
yet);
{code}
This could be due to HBaseServer.openServer() being called after
HBaseServer.startThreads() was called.
One solution is to give HBaseServer.startThreads() more time before checking
started flag in openServer().
TestUpgradeFromHFileV1ToEncoding#testUpgrade fails in trunk
---
Key: HBASE-6622
URL: https://issues.apache.org/jira/browse/HBASE-6622
Project: HBase
Issue Type: Bug
Reporter: Zhihong Ted Yu
TestUpgradeFromHFileV1ToEncoding started to fail since build #3242
Build #3246 was more recent one where it failed.
{code}
2012-08-21 00:49:06,536 INFO
[SplitLogWorker-vesta.apache.org,40294,1345510146310]
regionserver.SplitLogWorker(135): SplitLogWorker
vesta.apache.org,40294,1345510146310 starting
2012-08-21 00:49:06,537 INFO
[RegionServer:0;vesta.apache.org,40294,1345510146310]
regionserver.HRegionServer(2431): Registered RegionServer MXBean
2012-08-21 00:49:06,620 WARN [Master:0;vesta.apache.org,60969,1345510146282]
master.AssignmentManager(1606): Failed assignment of -ROOT-,,0.70236052 to
vesta.apache.org,40294,1345510146310, trying to assign elsewhere instead;
retry=0
org.apache.hadoop.hbase.ipc.ServerNotRunningYetException:
org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not
running yet
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:187)
at $Proxy15.openRegion(Unknown Source)
at
org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:500)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1587)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1256)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1226)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1221)
at
org.apache.hadoop.hbase.master.AssignmentManager.assignRoot(AssignmentManager.java:2103)
at
org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:785)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:665)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:439)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not
running yet
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1766)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1187)
at
org.apache.hadoop.hbase.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:178)
... 11 more
2012-08-21 00:49:06,621 INFO [Master:0;vesta.apache.org,60969,1345510146282]
master.RegionStates(250): Region {NAME = '-ROOT-,,0', STARTKEY = '', ENDKEY
= '', ENCODED = 70236052,} transitioned from {-ROOT-,,0.70236052
state=PENDING_OPEN, ts=1345510146520,
server=vesta.apache.org,40294,1345510146310} to {-ROOT-,,0.70236052
state=OFFLINE, ts=1345510146621, server=null}
2012-08-21 00:49:06,621 WARN [Master:0;vesta.apache.org,60969,1345510146282]
master.AssignmentManager(1772): Can't move the region 70236052, there is no
destination server available.
2012-08-21 00:49:06,621 WARN [Master:0;vesta.apache.org,60969,1345510146282]
master.AssignmentManager(1618): Unable to find a viable location to assign
region -ROOT-,,0.70236052
2012-08-21 00:50:06,406 DEBUG
[Master:0;vesta.apache.org,60969,1345510146282.archivedHFileCleaner]
cleaner.CleanerChore(145): Checking directory:
hdfs://localhost:56237/user/hudson/hbase/.archive/UpgradeTable
{code}
Looks like ROOT region couldn't be assigned.
--
This message is
[jira] [Commented] (HBASE-6588) enable table throws npe and leaves trash in zk in competition with delete table
[
https://issues.apache.org/jira/browse/HBASE-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437972#comment-13437972
]
Zhihong Ted Yu commented on HBASE-6588:
---
For patch v5, if you look at
https://builds.apache.org/job/PreCommit-HBASE-Build/2623/console, you can see
that compilation failed for hadoop 2.0 profile.
I got the following error compiling against hadoop 2.0, locally:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile (default-compile)
on project hbase-server: Compilation failure: Compilation failure:
[ERROR]
/home/zhihyu/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java:[722,50]
cannot find symbol
[ERROR] symbol : variable COMPACTION_KV_MAX
[ERROR] location: class org.apache.hadoop.hbase.HConstants
[ERROR]
[ERROR]
/home/zhihyu/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:[3578,36]
cannot find symbol
[ERROR] symbol : method isInternal()
[ERROR] location: class org.apache.hadoop.hbase.KeyValue
[ERROR]
[ERROR]
/home/zhihyu/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java:[113,53]
cannot find symbol
[ERROR] symbol : variable COMPACTION_KV_MAX
[ERROR] location: class org.apache.hadoop.hbase.HConstants
{code}
enable table throws npe and leaves trash in zk in competition with delete
table
---
Key: HBASE-6588
URL: https://issues.apache.org/jira/browse/HBASE-6588
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Zhou wenjian
Assignee: Zhou wenjian
Fix For: 0.94.2
Attachments: HBASE-6588-trunk.patch, HBASE-6588-trunk-v2.patch,
HBASE-6588-trunk-v3.patch, HBASE-6588-trunk-v4.patch,
HBASE-6588-trunk-v5.patch
2012-08-15 19:23:36,178 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'test,,'
2012-08-15 19:23:36,178 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'test,,'
2012-08-15 19:24:09,180 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key ''
2012-08-15 19:24:09,180 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at ''
2012-08-15 19:24:09,183 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Finished with scanning at {NAME = '.META.,,1', STARTKEY = '', ENDKEY = '',
ENCODED = 1028785192,}
2012-08-15 19:24:09,183 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor:
Scanned 2 catalog row(s) and gc'd 0 unreferenced parent region(s)
2012-08-15 19:25:12,260 DEBUG
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Deleting region
test,,1345029764571.d1e24b251ca6286c840a9a5f571b7db1. from META and FS
2012-08-15 19:25:12,263 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Deleted region test,,1345029764571.d1e24b251ca6286c840a9a5f571b7db1. from META
2012-08-15 19:25:12,265 INFO
org.apache.hadoop.hbase.master.handler.EnableTableHandler: Attemping to
enable the table test
2012-08-15 19:25:12,265 WARN org.apache.hadoop.hbase.zookeeper.ZKTable:
Moving table test state to enabling but was not first in disabled state: null
2012-08-15 19:25:12,267 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'test,,'
2012-08-15 19:25:12,267 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'test,,'
2012-08-15 19:25:12,270 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Finished with scanning at {NAME = '.META.,,1', STARTKEY = '', ENDKEY = '',
ENCODED = 1028785192,}
2012-08-15 19:25:12,270 ERROR org.apache.hadoop.hbase.executor.EventHandler:
Caught throwable while processing event C_M_ENABLE_TABLE
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.handleEnableTable(EnableTableHandler.java:116)
at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.process(EnableTableHandler.java:97)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
table is disabled now, then we enable and delete the table at the same time.
Since the thread num of MASTER_TABLE_OPERATIONS is 1 by default. The two
operations are serial in master.Before deletetable deletes all the regions in
meta, CreateTableHandler ships the check of tableExists,then it will block
until deletetable finishs, then
[jira] [Comment Edited] (HBASE-6588) enable table throws npe and leaves trash in zk in competition with delete table
[
https://issues.apache.org/jira/browse/HBASE-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437972#comment-13437972
]
Zhihong Ted Yu edited comment on HBASE-6588 at 8/21/12 3:17 AM:
For patch v5, if you look at
https://builds.apache.org/job/PreCommit-HBASE-Build/2623/console, you can see
that compilation failed for hadoop 2.0 profile.
I got the following error compiling against hadoop 2.0, locally:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile
(default-testCompile) on project hbase-server: Compilation failure: Compilation
failure:
[ERROR]
/home/zhihyu/trunk-hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java:[849,16]
cannot find symbol
[ERROR] symbol : class RegionState
[ERROR] location: class org.apache.hadoop.hbase.client.TestAdmin
[ERROR]
[ERROR]
/home/zhihyu/trunk-hbase/hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java:[854,29]
cannot find symbol
[ERROR] symbol : class RegionState
[ERROR] location: class org.apache.hadoop.hbase.client.TestAdmin
{code}
was (Author: zhi...@ebaysf.com):
For patch v5, if you look at
https://builds.apache.org/job/PreCommit-HBASE-Build/2623/console, you can see
that compilation failed for hadoop 2.0 profile.
I got the following error compiling against hadoop 2.0, locally:
{code}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile (default-compile)
on project hbase-server: Compilation failure: Compilation failure:
[ERROR]
/home/zhihyu/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java:[722,50]
cannot find symbol
[ERROR] symbol : variable COMPACTION_KV_MAX
[ERROR] location: class org.apache.hadoop.hbase.HConstants
[ERROR]
[ERROR]
/home/zhihyu/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java:[3578,36]
cannot find symbol
[ERROR] symbol : method isInternal()
[ERROR] location: class org.apache.hadoop.hbase.KeyValue
[ERROR]
[ERROR]
/home/zhihyu/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/Compactor.java:[113,53]
cannot find symbol
[ERROR] symbol : variable COMPACTION_KV_MAX
[ERROR] location: class org.apache.hadoop.hbase.HConstants
{code}
enable table throws npe and leaves trash in zk in competition with delete
table
---
Key: HBASE-6588
URL: https://issues.apache.org/jira/browse/HBASE-6588
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Zhou wenjian
Assignee: Zhou wenjian
Fix For: 0.94.2
Attachments: HBASE-6588-trunk.patch, HBASE-6588-trunk-v2.patch,
HBASE-6588-trunk-v3.patch, HBASE-6588-trunk-v4.patch,
HBASE-6588-trunk-v5.patch
2012-08-15 19:23:36,178 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'test,,'
2012-08-15 19:23:36,178 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'test,,'
2012-08-15 19:24:09,180 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key ''
2012-08-15 19:24:09,180 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at ''
2012-08-15 19:24:09,183 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Finished with scanning at {NAME = '.META.,,1', STARTKEY = '', ENDKEY = '',
ENCODED = 1028785192,}
2012-08-15 19:24:09,183 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor:
Scanned 2 catalog row(s) and gc'd 0 unreferenced parent region(s)
2012-08-15 19:25:12,260 DEBUG
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Deleting region
test,,1345029764571.d1e24b251ca6286c840a9a5f571b7db1. from META and FS
2012-08-15 19:25:12,263 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Deleted region test,,1345029764571.d1e24b251ca6286c840a9a5f571b7db1. from META
2012-08-15 19:25:12,265 INFO
org.apache.hadoop.hbase.master.handler.EnableTableHandler: Attemping to
enable the table test
2012-08-15 19:25:12,265 WARN org.apache.hadoop.hbase.zookeeper.ZKTable:
Moving table test state to enabling but was not first in disabled state: null
2012-08-15 19:25:12,267 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'test,,'
2012-08-15 19:25:12,267 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'test,,'
2012-08-15 19:25:12,270 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Finished with scanning at {NAME = '.META.,,1', STARTKEY = '', ENDKEY = '',
ENCODED = 1028785192,}
2012-08-15 19:25:12,270 ERROR
[jira] [Created] (HBASE-6617) ReplicationSourceManager should be able to track multiple WAL paths
Zhihong Ted Yu created HBASE-6617: - Summary: ReplicationSourceManager should be able to track multiple WAL paths Key: HBASE-6617 URL: https://issues.apache.org/jira/browse/HBASE-6617 Project: HBase Issue Type: Sub-task Reporter: Zhihong Ted Yu Currently ReplicationSourceManager uses logRolled() to receive notification about new HLog and remembers it in latestPath. When region server has multiple WAL support, we need to keep track of multiple Path's in ReplicationSourceManager -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-5937) Refactor HLog into an interface.
[ https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu reassigned HBASE-5937: - Assignee: Flavio Junqueira (was: Li Pi) Refactor HLog into an interface. Key: HBASE-5937 URL: https://issues.apache.org/jira/browse/HBASE-5937 Project: HBase Issue Type: Sub-task Reporter: Li Pi Assignee: Flavio Junqueira Priority: Minor What the summary says. Create HLog interface. Make current implementation use it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
[
https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13438053#comment-13438053
]
Zhihong Ted Yu commented on HBASE-6608:
---
Integrated to 0.92 and 0.94 as well.
Thanks for the patch, Enis.
Fix for HBASE-6160, META entries from daughters can be deleted before parent
entries, shouldn't compare HRegionInfo's
-
Key: HBASE-6608
URL: https://issues.apache.org/jira/browse/HBASE-6608
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.1, 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.92.2, 0.96.0, 0.94.2
Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch,
hbase-6608_v1.patch
Our nightlies discovered that the patch for HBASE-6160 did not actually fix
the issue of META entries from daughters can be deleted before parent
entries. Instead of reopening the HBASE-6160, it is cleaner to track it
here.
The original issue is:
{quote}
HBASE-5986 fixed and issue, where the client sees the META entry for the
parent, but not the children. However, after the fix, we have seen the
following issue in tests:
Region A is split to - B, C
Region B is split to - D, E
After some time, META entry for B is deleted since it is not needed anymore,
but META entry for Region A stays in META (C still refers it). In this case,
the client throws RegionOfflineException for B.
{quote}
The problem with the fix seems to be that we keep and compare HRegionInfo's
in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are
not equal.
{code}
HashSetHRegionInfo parentNotCleaned = new HashSetHRegionInfo(); //regions
whose parents are still around
for (Map.EntryHRegionInfo, Result e : splitParents.entrySet()) {
if (!parentNotCleaned.contains(e.getKey()) cleanParent(e.getKey(),
e.getValue())) {
cleaned++;
} else {
...
{code}
In the above case, Meta row for region A will contain a serialized version of
B that is not offline. However Meta row for region B will contain a
serialized version of B that is offline (MetaEditor.offlineParentInMeta()
does that). So the deserialized version we put to HashSet and the
deserialized version we query contains() from HashSet are different in the
offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5169) Group of Region Server, a subtask of issue 4120
[ https://issues.apache.org/jira/browse/HBASE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13438261#comment-13438261 ] Zhihong Ted Yu commented on HBASE-5169: --- This task has been idle for 5 months. @Vandana: What do you think of the current design ? It would be nice if you can continue with this feature. Group of Region Server, a subtask of issue 4120 - Key: HBASE-5169 URL: https://issues.apache.org/jira/browse/HBASE-5169 Project: HBase Issue Type: Sub-task Components: master Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.96.0 Attachments: GroupOfRegionServer_v1.patch, GroupOfRegionServer_v2.patch This is a subtask of issue 4120,this patch provides the region server group feature of HBase. With this patch, region servers can be divided into groups,one table could belong to one or more groups while the region server can only belong to one group. Work load in defferent groups will not affect each other. This patch provides table level and group level load balance,the default load balance and region assignments will consider the group configuration and assign regions to their corresponding groups. More information, please check out the documents of issue 4120. There is a web tool of this patch providing operations of group managements like add/delete group, move in/out servers,change table's group attribute ,balance groups, balance tables. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6621) Reduce calls to Bytes.toInt
[
https://issues.apache.org/jira/browse/HBASE-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13438415#comment-13438415
]
Zhihong Ted Yu commented on HBASE-6621:
---
{code}
+ * for length codelength/code, and a know codekeyLength/code.
{code}
'know' - 'known'
{code}
+ * Use with caution.
{code}
Can you tell us what caution should be taken ?
Reduce calls to Bytes.toInt
---
Key: HBASE-6621
URL: https://issues.apache.org/jira/browse/HBASE-6621
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.96.0, 0.94.2
Attachments: 6621-0.96.txt, 6621-0.96-v2.txt, 6621-0.96-v3.txt
Bytes.toInt shows up quite often in a profiler run.
It turns out that one source is HFileReaderV2$ScannerV2.getKeyValue().
Notice that we call the KeyValue(byte[], int) constructor, which forces the
constructor to determine its size by reading some of the header information
and calculate the size. In this case, however, we already know the size (from
the call to readKeyValueLen), so we could just use that.
In the extreme case of 1's of columns this noticeably reduces CPU.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6622) TestUpgradeFromHFileV1ToEncoding#testUpgrade fails in trunk
Zhihong Ted Yu created HBASE-6622:
-
Summary: TestUpgradeFromHFileV1ToEncoding#testUpgrade fails in
trunk
Key: HBASE-6622
URL: https://issues.apache.org/jira/browse/HBASE-6622
Project: HBase
Issue Type: Bug
Reporter: Zhihong Ted Yu
TestUpgradeFromHFileV1ToEncoding started to fail since build #3242
Build #3246 was more recent one where it failed.
{code}
2012-08-21 00:49:06,536 INFO
[SplitLogWorker-vesta.apache.org,40294,1345510146310]
regionserver.SplitLogWorker(135): SplitLogWorker
vesta.apache.org,40294,1345510146310 starting
2012-08-21 00:49:06,537 INFO
[RegionServer:0;vesta.apache.org,40294,1345510146310]
regionserver.HRegionServer(2431): Registered RegionServer MXBean
2012-08-21 00:49:06,620 WARN [Master:0;vesta.apache.org,60969,1345510146282]
master.AssignmentManager(1606): Failed assignment of -ROOT-,,0.70236052 to
vesta.apache.org,40294,1345510146310, trying to assign elsewhere instead;
retry=0
org.apache.hadoop.hbase.ipc.ServerNotRunningYetException:
org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running
yet
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
at
org.apache.hadoop.hbase.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:187)
at $Proxy15.openRegion(Unknown Source)
at
org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:500)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1587)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1256)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1226)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1221)
at
org.apache.hadoop.hbase.master.AssignmentManager.assignRoot(AssignmentManager.java:2103)
at
org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:785)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:665)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:439)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running
yet
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1766)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1187)
at
org.apache.hadoop.hbase.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:178)
... 11 more
2012-08-21 00:49:06,621 INFO [Master:0;vesta.apache.org,60969,1345510146282]
master.RegionStates(250): Region {NAME = '-ROOT-,,0', STARTKEY = '', ENDKEY
= '', ENCODED = 70236052,} transitioned from {-ROOT-,,0.70236052
state=PENDING_OPEN, ts=1345510146520,
server=vesta.apache.org,40294,1345510146310} to {-ROOT-,,0.70236052
state=OFFLINE, ts=1345510146621, server=null}
2012-08-21 00:49:06,621 WARN [Master:0;vesta.apache.org,60969,1345510146282]
master.AssignmentManager(1772): Can't move the region 70236052, there is no
destination server available.
2012-08-21 00:49:06,621 WARN [Master:0;vesta.apache.org,60969,1345510146282]
master.AssignmentManager(1618): Unable to find a viable location to assign
region -ROOT-,,0.70236052
2012-08-21 00:50:06,406 DEBUG
[Master:0;vesta.apache.org,60969,1345510146282.archivedHFileCleaner]
cleaner.CleanerChore(145): Checking directory:
hdfs://localhost:56237/user/hudson/hbase/.archive/UpgradeTable
{code}
Looks like ROOT region couldn't be assigned.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[
https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437515#comment-13437515
]
Zhihong Ted Yu commented on HBASE-6564:
---
@J:
If you look at patch v2 closely, you can see that it is composed of two patch
files (line 310).
I suggest renaming TestTableFamilyHandlers.java -
TestTableDeleteFamilyHandler.java because only TableDeleteFamilyHandler is
tested.
There're a lot of two (successive) empty lines in this new file.
Please remove redundant empty line.
{code}
* Copyright 2009 The Apache Software Foundation
{code}
The above line should be removed from license header.
{code}
// 4 - Check if all the 3 column families exists in FS
{code}
'exists' - 'exist'
{code}
public org.apache.hadoop.hbase.ResourceCheckerJUnitRule cu = new
org.apache.hadoop.hbase.ResourceCheckerJUnitRule();
{code}
The above line is longer than 100 characters. Please wrap.
Please regenerate patch and attach to this JIRA.
Thanks
HDFS space is not reclaimed when a column family is deleted
---
Key: HBASE-6564
URL: https://issues.apache.org/jira/browse/HBASE-6564
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.94.1
Reporter: J Mohamed Zahoor
Priority: Minor
Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch
When a column family of a table is deleted, the HDFS space of the column
family does not seem to be reclaimed even after a major compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[ https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437517#comment-13437517 ] Zhihong Ted Yu commented on HBASE-6564: --- For MasterFileSystem.deleteFamily(), if fs.delete() returns false, I think we should raise IOException. HDFS space is not reclaimed when a column family is deleted --- Key: HBASE-6564 URL: https://issues.apache.org/jira/browse/HBASE-6564 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.1 Reporter: J Mohamed Zahoor Priority: Minor Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch When a column family of a table is deleted, the HDFS space of the column family does not seem to be reclaimed even after a major compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[ https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu reassigned HBASE-6564: - Assignee: J Mohamed Zahoor HDFS space is not reclaimed when a column family is deleted --- Key: HBASE-6564 URL: https://issues.apache.org/jira/browse/HBASE-6564 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.1 Reporter: J Mohamed Zahoor Assignee: J Mohamed Zahoor Priority: Minor Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch When a column family of a table is deleted, the HDFS space of the column family does not seem to be reclaimed even after a major compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6584) Test flappers due to port 60000 already in use.
[ https://issues.apache.org/jira/browse/HBASE-6584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-6584: -- Status: Patch Available (was: Open) Test flappers due to port 6 already in use. --- Key: HBASE-6584 URL: https://issues.apache.org/jira/browse/HBASE-6584 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.94.0 Reporter: s...@hotmail.com Priority: Critical Attachments: HBASE-6584_trunk.patch, HBASE-6584_trunk.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6473) deleted table is not deleted completely, some region may be still online
[
https://issues.apache.org/jira/browse/HBASE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu reassigned HBASE-6473:
-
Assignee: Zhou wenjian
deleted table is not deleted completely, some region may be still online
Key: HBASE-6473
URL: https://issues.apache.org/jira/browse/HBASE-6473
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: zhou wenjian
Assignee: Zhou wenjian
Fix For: 0.96.0, 0.94.2
Attachments: HBASE-6473-trunk.patch, HBASE-6473-trunk-v2.patch,
HBASE-6473-trunk-v3.patch
consider such Scenario:
we have a table called T1, which has 1 regions: A
1. move A from rs1 to rs2,and A is now closed
2. disable T1,
3. delete T1.
when we disable T1, disable handler will just set the zk to disabled and A
will still be assigned. when Ais opened, A in transition will be clean out.
At that time, Deletetable found it is safe to delete all regions and table in
meta and fs , it will also delete the zk node of T1.
{code}
while (System.currentTimeMillis() done) {
AssignmentManager.RegionState rs = am.isRegionInTransition(region);
if (rs == null) break;
Threads.sleep(waitingTimeForEvents);
LOG.debug(Waiting on region to clear regions in transition; + rs);
}
if (am.isRegionInTransition(region) != null) {
throw new IOException(Waited hbase.master.wait.on.region ( +
waitTime + ms) for region to leave region +
region.getRegionNameAsString() + in transitions);
}
{code}
however A is still being unassigned, when it finished closed the A,it finds
that the disabled state in zk is deleted, and then A will be assigned again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6588) enable table throws npe and leaves trash in zk in competition with delete table
[
https://issues.apache.org/jira/browse/HBASE-6588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu reassigned HBASE-6588:
-
Assignee: Zhou wenjian
enable table throws npe and leaves trash in zk in competition with delete
table
---
Key: HBASE-6588
URL: https://issues.apache.org/jira/browse/HBASE-6588
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Zhou wenjian
Assignee: Zhou wenjian
Fix For: 0.94.2
Attachments: HBASE-6588-trunk.patch, HBASE-6588-trunk-v2.patch,
HBASE-6588-trunk-v3.patch, HBASE-6588-trunk-v4.patch,
HBASE-6588-trunk-v5.patch
2012-08-15 19:23:36,178 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'test,,'
2012-08-15 19:23:36,178 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'test,,'
2012-08-15 19:24:09,180 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key ''
2012-08-15 19:24:09,180 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at ''
2012-08-15 19:24:09,183 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Finished with scanning at {NAME = '.META.,,1', STARTKEY = '', ENDKEY = '',
ENCODED = 1028785192,}
2012-08-15 19:24:09,183 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor:
Scanned 2 catalog row(s) and gc'd 0 unreferenced parent region(s)
2012-08-15 19:25:12,260 DEBUG
org.apache.hadoop.hbase.master.handler.DeleteTableHandler: Deleting region
test,,1345029764571.d1e24b251ca6286c840a9a5f571b7db1. from META and FS
2012-08-15 19:25:12,263 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Deleted region test,,1345029764571.d1e24b251ca6286c840a9a5f571b7db1. from META
2012-08-15 19:25:12,265 INFO
org.apache.hadoop.hbase.master.handler.EnableTableHandler: Attemping to
enable the table test
2012-08-15 19:25:12,265 WARN org.apache.hadoop.hbase.zookeeper.ZKTable:
Moving table test state to enabling but was not first in disabled state: null
2012-08-15 19:25:12,267 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Creating scanner over .META. starting at key 'test,,'
2012-08-15 19:25:12,267 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Advancing internal scanner to startKey at 'test,,'
2012-08-15 19:25:12,270 DEBUG org.apache.hadoop.hbase.client.ClientScanner:
Finished with scanning at {NAME = '.META.,,1', STARTKEY = '', ENDKEY = '',
ENCODED = 1028785192,}
2012-08-15 19:25:12,270 ERROR org.apache.hadoop.hbase.executor.EventHandler:
Caught throwable while processing event C_M_ENABLE_TABLE
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.handleEnableTable(EnableTableHandler.java:116)
at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.process(EnableTableHandler.java:97)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
table is disabled now, then we enable and delete the table at the same time.
Since the thread num of MASTER_TABLE_OPERATIONS is 1 by default. The two
operations are serial in master.Before deletetable deletes all the regions in
meta, CreateTableHandler ships the check of tableExists,then it will block
until deletetable finishs, then CreateTableHandler will set zk enabling, and
find no data in meta:
regionsInMeta = MetaReader.getTableRegions(this.ct, tableName, true);
int countOfRegionsInTable = regionsInMeta.size();
npe will be throwed here. And we could not create the same table anymore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[
https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437358#comment-13437358
]
Zhihong Ted Yu commented on HBASE-6564:
---
{code}
+fs.delete(delDir, true);
{code}
Please check the return value from delete() call.
TestTableFamilyHandlers.java misses license header.
{code}
+ // TETS - Disable and delete the column family
{code}
'TETS' seems to be misspelled above.
Please add the following to the end of the new test:
{code}
@org.junit.Rule
public org.apache.hadoop.hbase.ResourceCheckerJUnitRule cu =
new org.apache.hadoop.hbase.ResourceCheckerJUnitRule();
{code}
Please try the patch on a cluster.
Thanks
HDFS space is not reclaimed when a column family is deleted
---
Key: HBASE-6564
URL: https://issues.apache.org/jira/browse/HBASE-6564
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.94.1
Reporter: J Mohamed Zahoor
Priority: Minor
Attachments: HBASE-6564-trunk.patch
When a column family of a table is deleted, the HDFS space of the column
family does not seem to be reclaimed even after a major compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6340) HBase RPC does not allow protocol extension with common interfaces.
[ https://issues.apache.org/jira/browse/HBASE-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-6340: -- Status: Open (was: Patch Available) HBase RPC does not allow protocol extension with common interfaces. --- Key: HBASE-6340 URL: https://issues.apache.org/jira/browse/HBASE-6340 Project: HBase Issue Type: Bug Components: coprocessors, regionserver Affects Versions: 0.92.0 Reporter: Konstantin Shvachko Assignee: Konstantin Shvachko Attachments: RPCInvocation.patch HBase RPC fails if MyProtocol extends an interface, which is not a VersionedProtocol even if MyProtocol also directly extends VersionedProtocol. The reason is that rpc Invocation uses Method.getDeclaringClass(), which returns the interface class rather than the class of MyProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6340) HBase RPC does not allow protocol extension with common interfaces.
[ https://issues.apache.org/jira/browse/HBASE-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-6340: -- Attachment: 6340-RPCInvocation.patch Patch for 0.94 TestCoprocessorEndpoint and TestProtocolExtension pass. Running test suite. HBase RPC does not allow protocol extension with common interfaces. --- Key: HBASE-6340 URL: https://issues.apache.org/jira/browse/HBASE-6340 Project: HBase Issue Type: Bug Components: coprocessors, regionserver Affects Versions: 0.92.0 Reporter: Konstantin Shvachko Assignee: Konstantin Shvachko Attachments: 6340-RPCInvocation.patch, RPCInvocation.patch HBase RPC fails if MyProtocol extends an interface, which is not a VersionedProtocol even if MyProtocol also directly extends VersionedProtocol. The reason is that rpc Invocation uses Method.getDeclaringClass(), which returns the interface class rather than the class of MyProtocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
[
https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437435#comment-13437435
]
Zhihong Ted Yu commented on HBASE-6608:
---
Integrated to trunk.
@Enis, please attach patch(es) for 0.92 / 0.94
Thanks for the review, Stack.
Fix for HBASE-6160, META entries from daughters can be deleted before parent
entries, shouldn't compare HRegionInfo's
-
Key: HBASE-6608
URL: https://issues.apache.org/jira/browse/HBASE-6608
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.1, 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.92.2, 0.96.0, 0.94.2
Attachments: 6608-v2.patch, hbase-6608_v1.patch
Our nightlies discovered that the patch for HBASE-6160 did not actually fix
the issue of META entries from daughters can be deleted before parent
entries. Instead of reopening the HBASE-6160, it is cleaner to track it
here.
The original issue is:
{quote}
HBASE-5986 fixed and issue, where the client sees the META entry for the
parent, but not the children. However, after the fix, we have seen the
following issue in tests:
Region A is split to - B, C
Region B is split to - D, E
After some time, META entry for B is deleted since it is not needed anymore,
but META entry for Region A stays in META (C still refers it). In this case,
the client throws RegionOfflineException for B.
{quote}
The problem with the fix seems to be that we keep and compare HRegionInfo's
in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are
not equal.
{code}
HashSetHRegionInfo parentNotCleaned = new HashSetHRegionInfo(); //regions
whose parents are still around
for (Map.EntryHRegionInfo, Result e : splitParents.entrySet()) {
if (!parentNotCleaned.contains(e.getKey()) cleanParent(e.getKey(),
e.getValue())) {
cleaned++;
} else {
...
{code}
In the above case, Meta row for region A will contain a serialized version of
B that is not offline. However Meta row for region B will contain a
serialized version of B that is offline (MetaEditor.offlineParentInMeta()
does that). So the deserialized version we put to HashSet and the
deserialized version we query contains() from HashSet are different in the
offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6165) Replication can overrun .META. scans on cluster re-start
[
https://issues.apache.org/jira/browse/HBASE-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437452#comment-13437452
]
Zhihong Ted Yu commented on HBASE-6165:
---
Patch v3 looks clean.
nit:
{code}
+if(handlers != null) {
+ for(Handler h : handlers) {
{code}
Space should be added immediately before '('
Replication can overrun .META. scans on cluster re-start
Key: HBASE-6165
URL: https://issues.apache.org/jira/browse/HBASE-6165
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Assignee: Himanshu Vashishtha
Fix For: 0.96.0, 0.94.2
Attachments: HBase-6165-v1.patch, HBase-6165-v2.patch,
HBase-6165-v3.patch
When restarting a large set of regions on a reasonably small cluster the
replication from another cluster tied up every xceiver meaning nothing could
be onlined.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6590) [0.89-fb] Assign sequence number to bulk loaded data
[ https://issues.apache.org/jira/browse/HBASE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13436949#comment-13436949 ] Zhihong Ted Yu commented on HBASE-6590: --- @Amit: Please create another JIRA and rebase your patch for HBase trunk. Thanks a lot. [0.89-fb] Assign sequence number to bulk loaded data Key: HBASE-6590 URL: https://issues.apache.org/jira/browse/HBASE-6590 Project: HBase Issue Type: Bug Reporter: Amitanand Aiyer Assignee: Amitanand Aiyer Priority: Minor Fix For: 0.89-fb Currently bulk loaded files are not assigned a sequence number. Thus, they can only be used to import historical data, dating to the past. There are cases where we want to bulk load current data; but the bulk load mechanism does not support this, as the bulk loaded files are always sorted behind the non-bulkloaded hfiles. Assigning Sequence Id to bulk loaded files should solve this issue. StoreFiles within a store are sorted based on the sequenceId. SequenceId is a monotonically increasing number that accompanies every edit written to the WAL. For entries that update the same cell, we would like the latter edit to win. This comparision is accomplished using memstoreTS, at the KV level; and sequenceId at the StoreFile level (to order scanners in the KeyValueHeap). BulkLoaded files are generated outside of HBase/RegionServer, so they do not have a sequenceId written in the file. This causes HBase to lose track of the point in time, when the BulkLoaded file was imported to HBase. Resulting in a behavior, that **only** supports viewing bulkLoaded files as files back-filling data from the begining of time. By assigning a sequence number to the file, we can allow the bulk loaded file to fit in where we want. Either at the current time or the begining of time. The latter is the default, to maintain backward compatibility. Design approach: Store files keep track of the sequence Id in the trailer. Since we do not wish to edit/rewrite the bulk loaded file upon import, we will encode the assigned sequenceId into the fileName. The filename RegEx is updated for this regard. If the sequenceId is encoded in the filename, the sequenceId will be used as the sequenceId for the file. If none is found, the sequenceId will be considered 0 (as per the default, backward-compatible behavior). To enable clients to request pre-existing behavior, the command line utility allows for 2 ways to import BulkLoaded Files: to assign or not assign a sequence Number. - If a sequence Number is assigned, the imporeted file will be imported with the current sequence Id. - if the sequence Number is not assigned, it will be as if it was backfilling old data, from the begining of time. Compaction behavior: - With the current compaction algorithm, bulk loaded files -- that backfill data, to the begining of time -- can cause a compaction storm, converting every minor compaction to a major compaction. To address this, these files are excluded from minor compaction, based on a config param. (enabled for the messages use case). - Since, bulk loaded files that are not back-filling data do not cause this issue, they will not be ignored during minor compactions based on the config parameter. This is also required to ensure that there are no holes in the set of files selected for compaction -- this is necessary to preserve the order of KV's comparision before and after compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) HBASE-6160 fix does not work
[
https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-6608:
--
Fix Version/s: 0.94.2
0.96.0
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
Nice finding, Enis.
HBASE-6160 fix does not work
-
Key: HBASE-6608
URL: https://issues.apache.org/jira/browse/HBASE-6608
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.2, 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.96.0, 0.94.2
Attachments: hbase-6608_v1.patch
Our nightlies discovered that the patch for HBASE-6160 did not actually fix
the issue of META entries from daughters can be deleted before parent
entries. Instead of reopening the HBASE-6160, it is cleaner to track it
here.
The original issue is:
{quote}
HBASE-5986 fixed and issue, where the client sees the META entry for the
parent, but not the children. However, after the fix, we have seen the
following issue in tests:
Region A is split to - B, C
Region B is split to - D, E
After some time, META entry for B is deleted since it is not needed anymore,
but META entry for Region A stays in META (C still refers it). In this case,
the client throws RegionOfflineException for B.
{quote}
The problem with the fix seems to be that we keep and compare HRegionInfo's
in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are
not equal.
{code}
HashSetHRegionInfo parentNotCleaned = new HashSetHRegionInfo(); //regions
whose parents are still around
for (Map.EntryHRegionInfo, Result e : splitParents.entrySet()) {
if (!parentNotCleaned.contains(e.getKey()) cleanParent(e.getKey(),
e.getValue())) {
cleaned++;
} else {
...
{code}
In the above case, Meta row for region A will contain a serialized version of
B that is not offline. However Meta row for region B will contain a
serialized version of B that is offline (MetaEditor.offlineParentInMeta()
does that). So the deserialized version we put to HashSet and the
deserialized version we query contains() from HashSet are different in the
offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) HBASE-6160 fix does not work
[
https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-6608:
--
Attachment: 6608-v2.patch
Looks like Enis based his patch off a stale workspace.
Here is updated patch where TestCatalogJanitor passes.
HBASE-6160 fix does not work
-
Key: HBASE-6608
URL: https://issues.apache.org/jira/browse/HBASE-6608
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.2, 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.92.2, 0.96.0, 0.94.2
Attachments: 6608-v2.patch, hbase-6608_v1.patch
Our nightlies discovered that the patch for HBASE-6160 did not actually fix
the issue of META entries from daughters can be deleted before parent
entries. Instead of reopening the HBASE-6160, it is cleaner to track it
here.
The original issue is:
{quote}
HBASE-5986 fixed and issue, where the client sees the META entry for the
parent, but not the children. However, after the fix, we have seen the
following issue in tests:
Region A is split to - B, C
Region B is split to - D, E
After some time, META entry for B is deleted since it is not needed anymore,
but META entry for Region A stays in META (C still refers it). In this case,
the client throws RegionOfflineException for B.
{quote}
The problem with the fix seems to be that we keep and compare HRegionInfo's
in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are
not equal.
{code}
HashSetHRegionInfo parentNotCleaned = new HashSetHRegionInfo(); //regions
whose parents are still around
for (Map.EntryHRegionInfo, Result e : splitParents.entrySet()) {
if (!parentNotCleaned.contains(e.getKey()) cleanParent(e.getKey(),
e.getValue())) {
cleaned++;
} else {
...
{code}
In the above case, Meta row for region A will contain a serialized version of
B that is not offline. However Meta row for region B will contain a
serialized version of B that is offline (MetaEditor.offlineParentInMeta()
does that). So the deserialized version we put to HashSet and the
deserialized version we query contains() from HashSet are different in the
offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6414) Remove the WritableRpcEngine associated Invocation classes
[ https://issues.apache.org/jira/browse/HBASE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu updated HBASE-6414: -- Attachment: 6414-6.patch.txt Patch from review board. Remove the WritableRpcEngine associated Invocation classes Key: HBASE-6414 URL: https://issues.apache.org/jira/browse/HBASE-6414 Project: HBase Issue Type: Improvement Affects Versions: 0.96.0 Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.96.0 Attachments: 6414-1.patch.txt, 6414-3.patch.txt, 6414-4.patch.txt, 6414-4.patch.txt, 6414-5.patch.txt, 6414-5.patch.txt, 6414-5.patch.txt, 6414-6.patch.txt, 6414-6.txt, 6414-initial.patch.txt, 6414-initial.patch.txt Remove the WritableRpcEngine Invocation classes once HBASE-5705 gets committed and all the protocols are rebased to use PB. Raising this jira in advance.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
[
https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-6608:
--
Affects Version/s: (was: 0.92.2)
0.92.1
Summary: Fix for HBASE-6160, META entries from daughters can be
deleted before parent entries, shouldn't compare HRegionInfo's (was:
HBASE-6160 fix does not work )
Fix for HBASE-6160, META entries from daughters can be deleted before parent
entries, shouldn't compare HRegionInfo's
-
Key: HBASE-6608
URL: https://issues.apache.org/jira/browse/HBASE-6608
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.1, 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.92.2, 0.96.0, 0.94.2
Attachments: 6608-v2.patch, hbase-6608_v1.patch
Our nightlies discovered that the patch for HBASE-6160 did not actually fix
the issue of META entries from daughters can be deleted before parent
entries. Instead of reopening the HBASE-6160, it is cleaner to track it
here.
The original issue is:
{quote}
HBASE-5986 fixed and issue, where the client sees the META entry for the
parent, but not the children. However, after the fix, we have seen the
following issue in tests:
Region A is split to - B, C
Region B is split to - D, E
After some time, META entry for B is deleted since it is not needed anymore,
but META entry for Region A stays in META (C still refers it). In this case,
the client throws RegionOfflineException for B.
{quote}
The problem with the fix seems to be that we keep and compare HRegionInfo's
in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are
not equal.
{code}
HashSetHRegionInfo parentNotCleaned = new HashSetHRegionInfo(); //regions
whose parents are still around
for (Map.EntryHRegionInfo, Result e : splitParents.entrySet()) {
if (!parentNotCleaned.contains(e.getKey()) cleanParent(e.getKey(),
e.getValue())) {
cleaned++;
} else {
...
{code}
In the above case, Meta row for region A will contain a serialized version of
B that is not offline. However Meta row for region B will contain a
serialized version of B that is offline (MetaEditor.offlineParentInMeta()
does that). So the deserialized version we put to HashSet and the
deserialized version we query contains() from HashSet are different in the
offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
[
https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13437241#comment-13437241
]
Zhihong Ted Yu commented on HBASE-6608:
---
Test failures are not related to the patch.
Will integrate Monday if there is no objection.
Fix for HBASE-6160, META entries from daughters can be deleted before parent
entries, shouldn't compare HRegionInfo's
-
Key: HBASE-6608
URL: https://issues.apache.org/jira/browse/HBASE-6608
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.1, 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.92.2, 0.96.0, 0.94.2
Attachments: 6608-v2.patch, hbase-6608_v1.patch
Our nightlies discovered that the patch for HBASE-6160 did not actually fix
the issue of META entries from daughters can be deleted before parent
entries. Instead of reopening the HBASE-6160, it is cleaner to track it
here.
The original issue is:
{quote}
HBASE-5986 fixed and issue, where the client sees the META entry for the
parent, but not the children. However, after the fix, we have seen the
following issue in tests:
Region A is split to - B, C
Region B is split to - D, E
After some time, META entry for B is deleted since it is not needed anymore,
but META entry for Region A stays in META (C still refers it). In this case,
the client throws RegionOfflineException for B.
{quote}
The problem with the fix seems to be that we keep and compare HRegionInfo's
in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are
not equal.
{code}
HashSetHRegionInfo parentNotCleaned = new HashSetHRegionInfo(); //regions
whose parents are still around
for (Map.EntryHRegionInfo, Result e : splitParents.entrySet()) {
if (!parentNotCleaned.contains(e.getKey()) cleanParent(e.getKey(),
e.getValue())) {
cleaned++;
} else {
...
{code}
In the above case, Meta row for region A will contain a serialized version of
B that is not offline. However Meta row for region B will contain a
serialized version of B that is offline (MetaEditor.offlineParentInMeta()
does that). So the deserialized version we put to HashSet and the
deserialized version we query contains() from HashSet are different in the
offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
[
https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihong Ted Yu updated HBASE-6608:
--
Status: Open (was: Patch Available)
Fix for HBASE-6160, META entries from daughters can be deleted before parent
entries, shouldn't compare HRegionInfo's
-
Key: HBASE-6608
URL: https://issues.apache.org/jira/browse/HBASE-6608
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.1, 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.92.2, 0.96.0, 0.94.2
Attachments: 6608-v2.patch, hbase-6608_v1.patch
Our nightlies discovered that the patch for HBASE-6160 did not actually fix
the issue of META entries from daughters can be deleted before parent
entries. Instead of reopening the HBASE-6160, it is cleaner to track it
here.
The original issue is:
{quote}
HBASE-5986 fixed and issue, where the client sees the META entry for the
parent, but not the children. However, after the fix, we have seen the
following issue in tests:
Region A is split to - B, C
Region B is split to - D, E
After some time, META entry for B is deleted since it is not needed anymore,
but META entry for Region A stays in META (C still refers it). In this case,
the client throws RegionOfflineException for B.
{quote}
The problem with the fix seems to be that we keep and compare HRegionInfo's
in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are
not equal.
{code}
HashSetHRegionInfo parentNotCleaned = new HashSetHRegionInfo(); //regions
whose parents are still around
for (Map.EntryHRegionInfo, Result e : splitParents.entrySet()) {
if (!parentNotCleaned.contains(e.getKey()) cleanParent(e.getKey(),
e.getValue())) {
cleaned++;
} else {
...
{code}
In the above case, Meta row for region A will contain a serialized version of
B that is not offline. However Meta row for region B will contain a
serialized version of B that is offline (MetaEditor.offlineParentInMeta()
does that). So the deserialized version we put to HashSet and the
deserialized version we query contains() from HashSet are different in the
offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HBASE-6587) Region would be assigned twice in the case of all RS offline
[
https://issues.apache.org/jira/browse/HBASE-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435805#comment-13435805
]
Zhihong Ted Yu edited comment on HBASE-6587 at 8/17/12 12:37 AM:
-
@ram
{code}
2012-08-14 20:42:54,367 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Unable to determine a plan to assign .META.,,1.1028785192 state=OFFLINE, ts=1
344948174367, server=null
{code}
After the above log, TimeoutMonitor set allRegionServersOffline true
{code}2012-08-14 20:44:31,640 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan
was found (or we are ignoring an existing plan) for writete
st,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be1353b4fa25f4222. so
generated a random one;
hri=writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be13
53b4fa25f4222., src=, dest=dw92.kgb.sqa.cm4,60020,1344948267642; 1 (online=1,
available=1) available {code}
At the 2012-08-14 20:44:31, one server is onlined now, and region
277b9b6df6de2b9be1353b4fa25f4222 is sucessfully assigned.
However, at that time TimeoutMonitor, in th chore(), it would act on time out
because the if block
{code}
if (this.allRegionServersOffline !allRSsOffline) return true;
{code}
So we see the following log
{code}
2012-08-14 20:44:32,518 INFO org.apache.hadoop.hbase.master.AssignmentManager:
Regions in transition timed out:
writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df
6de2b9be1353b4fa25f4222. state=OPENING, ts=1344948272279,
server=dw92.kgb.sqa.cm4,60020,1344948267642
2012-08-14 20:44:32,518 INFO org.apache.hadoop.hbase.master.AssignmentManager:
Region has been OPENING for too long, reassigning
region=writetest,VHXYHJN0BL48HMR4DI1L,
1344925649429.277b9b6df6de2b9be1353b4fa25f4222.
{code}
The region is assigned at the time 2012-08-14 20:44:31, but is timed out by
TimeoutMonitor at the time 2012-08-14 20:44:32.
It cause the collision by two assign thread,
And the result is that the region is onlined after 30mins.
was (Author: zjushch):
@ram
{code}
2012-08-14 20:42:54,367 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Unable to determine a plan to assign .META.,,1.1028785192 state=OFFLINE, ts=1
344948174367, server=null
{code}
After the above log, TimeoutMonitor set allRegionServersOffline true
{code}2012-08-14 20:44:31,640 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan
was found (or we are ignoring an existing plan) for writete
st,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be1353b4fa25f4222. so
generated a random one;
hri=writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be13
53b4fa25f4222., src=, dest=dw92.kgb.sqa.cm4,60020,1344948267642; 1 (online=1,
available=1) available {code}
At the 2012-08-14 20:44:31, one server is onlined now, and region
277b9b6df6de2b9be1353b4fa25f4222 is sucessfully assigned.
However, at that time TimeoutMonitor, in th chore(), it would act on time out
because the if block {
code}if (this.allRegionServersOffline !allRSsOffline){code} return true;
So we see the following log
{code}2012-08-14 20:44:32,518 INFO
org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed
out: writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df
6de2b9be1353b4fa25f4222. state=OPENING, ts=1344948272279,
server=dw92.kgb.sqa.cm4,60020,1344948267642
2012-08-14 20:44:32,518 INFO org.apache.hadoop.hbase.master.AssignmentManager:
Region has been OPENING for too long, reassigning
region=writetest,VHXYHJN0BL48HMR4DI1L,
1344925649429.277b9b6df6de2b9be1353b4fa25f4222.
{code}
The region is assigned at the time 2012-08-14 20:44:31, but is timed out by
TimeoutMonitor at the time 2012-08-14 20:44:32.
It cause the collision by two assign thread,
And the result is that the region is onlined after 30mins.
Region would be assigned twice in the case of all RS offline
Key: HBASE-6587
URL: https://issues.apache.org/jira/browse/HBASE-6587
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.1
Reporter: chunhui shen
Assignee: chunhui shen
Fix For: 0.96.0
Attachments: 6587.patch, HBASE-6587.patch
In the TimeoutMonitor, we would act on time out for the regions if
(this.allRegionServersOffline !noRSAvailable)
The code is as the following:
{code}
if (regionState.getStamp() + timeout = now ||
(this.allRegionServersOffline !noRSAvailable)) {
//decide on action upon timeout or, if some RSs just came back
online, we can start the
// the assignment
actOnTimeOut(regionState);
}
{code}
But we found it exists a bug that it would act on time out for the region
which was assigned just now , and
[jira] [Commented] (HBASE-6587) Region would be assigned twice in the case of all RS offline
[
https://issues.apache.org/jira/browse/HBASE-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435957#comment-13435957
]
Zhihong Ted Yu commented on HBASE-6587:
---
Integrated to trunk.
Thanks for the patch, Chunhui.
Thanks for the review, Ram.
Region would be assigned twice in the case of all RS offline
Key: HBASE-6587
URL: https://issues.apache.org/jira/browse/HBASE-6587
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.1
Reporter: chunhui shen
Assignee: chunhui shen
Fix For: 0.96.0
Attachments: 6587.patch, HBASE-6587.patch
In the TimeoutMonitor, we would act on time out for the regions if
(this.allRegionServersOffline !noRSAvailable)
The code is as the following:
{code}
if (regionState.getStamp() + timeout = now ||
(this.allRegionServersOffline !noRSAvailable)) {
//decide on action upon timeout or, if some RSs just came back
online, we can start the
// the assignment
actOnTimeOut(regionState);
}
{code}
But we found it exists a bug that it would act on time out for the region
which was assigned just now , and cause assigning the region twice.
Master log for the region 277b9b6df6de2b9be1353b4fa25f4222:
{code}
2012-08-14 20:42:54,367 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Unable to determine a plan
to assign .META.,,1.1028785192 state=OFFLINE, ts=1
344948174367, server=null
2012-08-14 20:44:31,640 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan
was found (or we are ignoring an existing plan) for writete
st,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be1353b4fa25f4222. so
generated a random one;
hri=writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be13
53b4fa25f4222., src=, dest=dw92.kgb.sqa.cm4,60020,1344948267642; 1 (online=1,
available=1) available servers
2012-08-14 20:44:31,640 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
master:6-0x438f53bbf9b0acd Creating (or updating) unassigned node for
277b9b6df6de2b9be13
53b4fa25f4222 with OFFLINE state
2012-08-14 20:44:31,643 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Assigning region
writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df6de2b9be1353b4fa
25f4222. to dw92.kgb.sqa.cm4,60020,1344948267642
2012-08-14 20:44:32,291 DEBUG
org.apache.hadoop.hbase.master.AssignmentManager: Handling
transition=RS_ZK_REGION_OPENING, server=dw92.kgb.sqa.cm4,60020,1344948267642,
region=277b9b6df6de2b9be1353b4fa25f4222
// 异常的超时
2012-08-14 20:44:32,518 INFO
org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed
out: writetest,VHXYHJN0BL48HMR4DI1L,1344925649429.277b9b6df
6de2b9be1353b4fa25f4222. state=OPENING, ts=1344948272279,
server=dw92.kgb.sqa.cm4,60020,1344948267642
2012-08-14 20:44:32,518 INFO
org.apache.hadoop.hbase.master.AssignmentManager: Region has been OPENING for
too long, reassigning region=writetest,VHXYHJN0BL48HMR4DI1L,
1344925649429.277b9b6df6de2b9be1353b4fa25f4222.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6576) HBaseAdmin.createTable should wait until the table is enabled
[
https://issues.apache.org/jira/browse/HBASE-6576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435965#comment-13435965
]
Zhihong Ted Yu commented on HBASE-6576:
---
Integrated to 0.92 and 0.94 as well.
Thanks for the reminder, Lars.
HBaseAdmin.createTable should wait until the table is enabled
-
Key: HBASE-6576
URL: https://issues.apache.org/jira/browse/HBASE-6576
Project: HBase
Issue Type: Bug
Components: client, test
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Fix For: 0.92.2, 0.96.0, 0.94.2
Attachments: HBASE-6576-92.patch, HBASE-6576-94.patch,
HBASE-6576-trunk.patch
The function:
{code}
public void createTable(final HTableDescriptor desc, byte [][] splitKeys)
{code}
in HBaseAdmin is synchronous and returns once all the regions of the table
are online, but does not wait for the table to be enabled, which is the last
step of table creation (see CreateTableHandler).
This is confusing and leads to racy code because users do not realize that
this is the case. For example, I saw the following test failure in 0.92 when
I ran:
mvn test
-Dtest=org.apache.hadoop.hbase.client.TestAdmin#testEnableDisableAddColumnDeleteColumn
{code}
Error Message
org.apache.hadoop.hbase.TableNotEnabledException: testMasterAdmin at
org.apache.hadoop.hbase.master.handler.DisableTableHandler.init(DisableTableHandler.java:75)
at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1154) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
Stacktrace
org.apache.hadoop.hbase.TableNotEnabledException:
org.apache.hadoop.hbase.TableNotEnabledException: testMasterAdmin
at
org.apache.hadoop.hbase.master.handler.DisableTableHandler.init(DisableTableHandler.java:75)
at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1154)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
{code}
The issue is that code will create and table and immediately disable it in
order to do some testing, for example, to test an operation that only works
when the table is disabled. If the table has not been enabled yet, they will
get back a TableNotEnabledException.
The specific test above was fixed in HBASE-5206, but other examples exist in
the code, for example the following:
{code}
hbase org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat newtable asdf14
{code}
The code in question is:
{code}
byte[] tname = args[1].getBytes();
HTable table = util.createTable(tname, FAMILIES);
HBaseAdmin admin = new HBaseAdmin(conf);
admin.disableTable(tname);
{code}
It would be better if createTable just waited until the table was enabled, or
threw a TableNotEnabledException if it exhausted the configured number of
retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4364) Filters applied to columns not in the selected column list are ignored
[ https://issues.apache.org/jira/browse/HBASE-4364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13436032#comment-13436032 ] Zhihong Ted Yu commented on HBASE-4364: --- @Alex: Take a look at SingleColumnValueExcludeFilter Filters applied to columns not in the selected column list are ignored -- Key: HBASE-4364 URL: https://issues.apache.org/jira/browse/HBASE-4364 Project: HBase Issue Type: Bug Components: filters Affects Versions: 0.90.4, 0.92.0, 0.94.0 Reporter: Todd Lipcon Priority: Critical Attachments: HBASE-4364-failing-test-with-simplest-custom-filter.patch, hbase-4364_trunk.patch, hbase-4364_trunk-v2.patch For a scan, if you select some set of columns using addColumns(), and then apply a SingleColumnValueFilter that restricts the results based on some other columns which aren't selected, then those filter conditions are ignored. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6565) Coprocessor exec result Map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13436090#comment-13436090 ] Zhihong Ted Yu commented on HBASE-6565: --- @Andy: Fix version for 0.92 should be 0.92.3, I think. Coprocessor exec result Map is not thread safe -- Key: HBASE-6565 URL: https://issues.apache.org/jira/browse/HBASE-6565 Project: HBase Issue Type: Bug Components: client, coprocessors Affects Versions: 0.92.2, 0.94.0, 0.96.0 Environment: hadoop1.0.2,hbase0.94,jdk1.6 Reporter: Yuan Kang Labels: coprocessors, patch Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: Coprocessor-result-thread unsafe-bug-fix.patch Original Estimate: 168h Remaining Estimate: 168h I develop a coprocessor program ,but found some different results in repeated tests.for example,normally,the result's size is 10.but sometimes it appears 9. I read the HTable.java code,found a TreeMap(thread-unsafe) be used in multithreading environment.It cause the bug happened -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6565) Coprocessor exec result Map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu reassigned HBASE-6565: - Assignee: Yuan Kang Nice catch, Yuan. Assigning issue to you. Coprocessor exec result Map is not thread safe -- Key: HBASE-6565 URL: https://issues.apache.org/jira/browse/HBASE-6565 Project: HBase Issue Type: Bug Components: client, coprocessors Affects Versions: 0.92.2, 0.94.0, 0.96.0 Environment: hadoop1.0.2,hbase0.94,jdk1.6 Reporter: Yuan Kang Assignee: Yuan Kang Labels: coprocessors, patch Fix For: 0.96.0, 0.92.3, 0.94.2 Attachments: Coprocessor-result-thread unsafe-bug-fix.patch Original Estimate: 168h Remaining Estimate: 168h I develop a coprocessor program ,but found some different results in repeated tests.for example,normally,the result's size is 10.but sometimes it appears 9. I read the HTable.java code,found a TreeMap(thread-unsafe) be used in multithreading environment.It cause the bug happened -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6596) Revert HBASE-5022; it undoes HBC.create
[
https://issues.apache.org/jira/browse/HBASE-6596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13436232#comment-13436232
]
Zhihong Ted Yu commented on HBASE-6596:
---
Looks like TestClassLoading failure in trunk is reproducible.
In test log, I saw:
{code}
2012-08-16 11:59:55,134 WARN
[RS_OPEN_REGION-lm-sjn-00713032.corp.ebay.com,61729,1345143569962-2]
regionserver.RegionCoprocessorHost(201):
java.lang.UnsupportedOperationException: Immutable Configuration
at
org.apache.hadoop.hbase.regionserver.CompoundConfiguration.iterator(CompoundConfiguration.java:408)
at
org.apache.hadoop.hbase.HBaseConfiguration.merge(HBaseConfiguration.java:135)
at
org.apache.hadoop.hbase.HBaseConfiguration.create(HBaseConfiguration.java:124)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:183)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.lt;initgt;(RegionCoprocessorHost.java:152)
at org.apache.hadoop.hbase.regionserver.HRegion.lt;initgt;(HRegion.java:461)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:3715)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3896)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:334)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:110)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:195)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
{code}
Revert HBASE-5022; it undoes HBC.create
---
Key: HBASE-6596
URL: https://issues.apache.org/jira/browse/HBASE-6596
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Fix For: 0.94.2
Attachments: 6596_094.txt, 6596_trunk.txt
See this thread:
http://search-hadoop.com/m/WAXXV1OQ1QY/What+is+HBaseConfiguration.create%2528Configuration%2529+good+for%253Fsubj=What+is+HBaseConfiguration+create+Configuration+good+for+
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6602) Region Server Dynamic Metrics can cause high cpu usage.
[ https://issues.apache.org/jira/browse/HBASE-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13436423#comment-13436423 ] Zhihong Ted Yu commented on HBASE-6602: --- Patch looks good. Maybe renaming needsUpdateMessage as metricsNeedUpdate. Region Server Dynamic Metrics can cause high cpu usage. --- Key: HBASE-6602 URL: https://issues.apache.org/jira/browse/HBASE-6602 Project: HBase Issue Type: Bug Reporter: Elliott Clark Attachments: HBASE-6602-0.94-0.patch When regions are getting added and removed lots of cpu time can be used by jmx. This is caused by sending jmx messages for every new metric that is added or removed. Seeing jstacks like this: RMI TCP Connection(3)-10.4.19.33 daemon prio=10 tid=0x7f9d64b1d000 nid=0x353 runnable [0x7f9d598d6000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.put(HashMap.java:374) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.createMBeanInfo(MetricsDynamicMBeanBase.java:103) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.updateMbeanInfoIfMetricsListChanged(MetricsDynamicMBeanBase.java:75) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.getAttribute(MetricsDynamicMBeanBase.java:133) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666) at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1404) -- at java.util.TimerThread.run(Timer.java:462) Timer thread for monitoring hbase daemon prio=10 tid=0x7f9d648fe000 nid=0x2b5 runnable [0x7f9d624c7000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.put(HashMap.java:374) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.createMBeanInfo(MetricsDynamicMBeanBase.java:103) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.updateMbeanInfoIfMetricsListChanged(MetricsDynamicMBeanBase.java:75) at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.regionserver.metrics.RegionServerDynamicMetrics.setNumericMetric(RegionServerDynamicMetrics.java:105) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6593) TestAdmin times out sometimes
[ https://issues.apache.org/jira/browse/HBASE-6593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13436458#comment-13436458 ] Zhihong Ted Yu commented on HBASE-6593: --- TestAdmin#splitTest is fixed. Should subject of the JIRA be modified to reflect the actual fix ? TestAdmin times out sometimes - Key: HBASE-6593 URL: https://issues.apache.org/jira/browse/HBASE-6593 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.96.0, 0.94.2 Attachments: trunk-6593.patch In TestAdmin#splitTest, individual put is used to prepare the test data. We can group them together so as to avoid possible timeout. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6602) Region Server Dynamic Metrics can cause high cpu usage.
[ https://issues.apache.org/jira/browse/HBASE-6602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13436464#comment-13436464 ] Zhihong Ted Yu commented on HBASE-6602: --- I am fine with the current naming. Region Server Dynamic Metrics can cause high cpu usage. --- Key: HBASE-6602 URL: https://issues.apache.org/jira/browse/HBASE-6602 Project: HBase Issue Type: Bug Reporter: Elliott Clark Priority: Critical Fix For: 0.94.2 Attachments: HBASE-6602-0.94-0.patch When regions are getting added and removed lots of cpu time can be used by jmx. This is caused by sending jmx messages for every new metric that is added or removed. Seeing jstacks like this: RMI TCP Connection(3)-10.4.19.33 daemon prio=10 tid=0x7f9d64b1d000 nid=0x353 runnable [0x7f9d598d6000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.put(HashMap.java:374) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.createMBeanInfo(MetricsDynamicMBeanBase.java:103) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.updateMbeanInfoIfMetricsListChanged(MetricsDynamicMBeanBase.java:75) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.getAttribute(MetricsDynamicMBeanBase.java:133) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666) at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1404) -- at java.util.TimerThread.run(Timer.java:462) Timer thread for monitoring hbase daemon prio=10 tid=0x7f9d648fe000 nid=0x2b5 runnable [0x7f9d624c7000] java.lang.Thread.State: RUNNABLE at java.util.HashMap.put(HashMap.java:374) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.createMBeanInfo(MetricsDynamicMBeanBase.java:103) at org.apache.hadoop.metrics.util.MetricsDynamicMBeanBase.updateMbeanInfoIfMetricsListChanged(MetricsDynamicMBeanBase.java:75) at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.hbase.regionserver.metrics.RegionServerDynamicMetrics.setNumericMetric(RegionServerDynamicMetrics.java:105) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
