[jira] [Created] (HBASE-5835) [hbck] Catch and handle NotServingRegionException when close region attempt fails
[hbck] Catch and handle NotServingRegionException when close region attempt
fails
-
Key: HBASE-5835
URL: https://issues.apache.org/jira/browse/HBASE-5835
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Currently, if hbck attempts to close a region and catches a
NotServerRegionException, hbck may hang outputting a stack trace. Since the
goal is to close the region at a particular server, and since it is not serving
the region, the region is closed, and we should just warn and eat this
exception.
{code}
Exception in thread main org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hbase.NotServingRegionException: Received close for
regionid but we are not serving it
at
org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2162)
at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy5.closeRegion(Unknown Source)
at
org.apache.hadoop.hbase.util.HBaseFsckRepair.closeRegionSilentlyAndWait(HBaseFsckRepair.java:165)
at org.apache.hadoop.hbase.util.HBaseFsck.closeRegion(HBaseFsck.java:1185)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkRegionConsistency(HBaseFsck.java:1302)
at
org.apache.hadoop.hbase.util.HBaseFsck.checkAndFixConsistency(HBaseFsck.java:1065)
at
org.apache.hadoop.hbase.util.HBaseFsck.onlineConsistencyRepair(HBaseFsck.java:351)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:370)
at org.apache.hadoop.hbase.util.HBaseFsck.main(HBaseFsck.java:3001)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5837) hbase shell deleteall to .META. allows insertion of malformed rowkey.
hbase shell deleteall to .META. allows insertion of malformed rowkey.
-
Key: HBASE-5837
URL: https://issues.apache.org/jira/browse/HBASE-5837
Project: HBase
Issue Type: Bug
Components: master, shell
Affects Versions: 0.90.6
Reporter: Jonathan Hsieh
When using the hbase shell to manipulate meta entries, one is allowed to
'delete' malformed rows (entries with less than 2 ascii 44 ',' chars). When
this happens HBase servers may go down and the cluster will not be restartable
without manual intervention.
The delete results in a durable malformed rowkey in .META.'s memstore, .META.'s
HLog, and eventually .META.'s HFiles. Subsequent scans to meta (such as when a
HMaster starts) fail in the scanner because the comparator fails. In the case
of an HMaster startup, it causes an abort that kills the HMaster process.
{code}
12/04/18 22:07:34 FATAL master.HMaster: Unhandled exception. Starting shutdown.
org.apache.hadoop.ipc.RemoteException: java.io.IOException:
java.lang.IllegalArgumentException: No 44 in
blah,1334744821162.81f2df35c332dd2d3bb966fb5b419568., length=47, offset=54
at
org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:990)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.convertThrowableToIOE(HRegionServer.java:979)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1894)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1834)
at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
Caused by: java.lang.IllegalArgumentException: No 44 in
blah,1334744821162.81f2df35c332dd2d3bb966fb5b419568., length=47, offset=54
at
org.apache.hadoop.hbase.KeyValue.getRequiredDelimiterInReverse(KeyValue.java:1300)
at
org.apache.hadoop.hbase.KeyValue$MetaKeyComparator.compareRows(KeyValue.java:1846)
at
org.apache.hadoop.hbase.regionserver.ScanQueryMatcher.match(ScanQueryMatcher.java:130)
at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:257)
at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:114)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.nextInternal(HRegion.java:2435)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:2391)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScanner.next(HRegion.java:2408)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1870)
... 6 more
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:771)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy9.next(Unknown Source)
at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:264)
at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:237)
at
org.apache.hadoop.hbase.catalog.MetaReader.fullScanOfResults(MetaReader.java:220)
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:1580)
at
org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:221)
at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:422)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:295)
12/04/18 22:07:34 INFO master.HMaster: Aborting
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5820) hbck should check fs permissions.
hbck should check fs permissions. - Key: HBASE-5820 URL: https://issues.apache.org/jira/browse/HBASE-5820 Project: HBase Issue Type: New Feature Components: hbck Reporter: Jonathan Hsieh In some cases, hbck needs to be run as a user that has write perms to the file system. If it writes data to hbase's directories, it may write new files/dirs that the hbase processes's user does not have permissions to. We should alert the user of this situation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5801) [hbck] Hbck should handle case where some regions have different HTD settings in .regioninfo files (0.90 specific)
[hbck] Hbck should handle case where some regions have different HTD settings in .regioninfo files (0.90 specific) --- Key: HBASE-5801 URL: https://issues.apache.org/jira/browse/HBASE-5801 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7 Reporter: Jonathan Hsieh Assignee: Jimmy Xiang Recently, we encountered a case where some regions in a table have different HTableDescriptor settings serialized into HDFS their HRegionInfo .regioninfo file. hbck expects all HTDs within a table to be the same and currently bails out in this situation. We need to either point out a proper set of actions for the user to execute or automatically convert the region to a common HTD (likely the most common on, or possibly the first one.) Not sure if this requires reformatting data but may require closing and restarting a region. This issue is hbase 0.90.x specific -- 0.92+ keep all table info in a single .tableinfo file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5793) TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747
TestHBaseFsck#TestNoHdfsTable test hangs after HBASE-5747 - Key: HBASE-5793 URL: https://issues.apache.org/jira/browse/HBASE-5793 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh After the HBSAE-5747 modification, this one particular case hangs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5786) Implement histogram metrics for flush and compaction latencies and sizes.
Implement histogram metrics for flush and compaction latencies and sizes. - Key: HBASE-5786 URL: https://issues.apache.org/jira/browse/HBASE-5786 Project: HBase Issue Type: New Feature Components: metrics, regionserver Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Shaneal Manek Average time for region operations doesn't really tell a useful story when that help diagnose anomalous conditions. It would be extremely useful to add histogramming metrics similar to HBASE-5533 for region operations like flush, compaction and splitting. The probably should be forward biased at a much coarser granularity however (maybe decay every day?) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5712) Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
Parallelize load of .regioninfo files in diagnostic/repair portion of hbck. --- Key: HBASE-5712 URL: https://issues.apache.org/jira/browse/HBASE-5712 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh On heavily loaded hdfs's some dfs nodes may not respond quickly and backs off for 60s before attempting to read data from another datanode. Portions of the information gathered from hdfs (.regioninfo files) are loaded serially. With HBase with clusters with 100's, or 1000's, or 1's regions encountering these 60s delay blocks progress and can be very painful. There is already some parallelization of portions of the hdfs information load operations and the goal here is move the reading of .regioninfos into the parallelized sections.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5714) Add write permissions check before any hbck run that modifies hdfs.
Add write permissions check before any hbck run that modifies hdfs. --- Key: HBASE-5714 URL: https://issues.apache.org/jira/browse/HBASE-5714 Project: HBase Issue Type: Sub-task Components: hbck Affects Versions: 0.90.6, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh We encoutered a situation where hbase was run by a different user and was unable to write/modify/merge regions due to hdfs perms. Unfortunately, this happened after several minutes of read-only operations. hbck should fail early by having a write perm check and providing actionable advice to the hbase admin. Maybe something like: Current user yy does not have write perms to hbase home. Please run hbck as hdfs user xxx -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5697) Audit HBase for usage of deprecated hadoop 0.20.x property names.
Audit HBase for usage of deprecated hadoop 0.20.x property names. - Key: HBASE-5697 URL: https://issues.apache.org/jira/browse/HBASE-5697 Project: HBase Issue Type: Task Reporter: Jonathan Hsieh Many xml config properties in Hadoop have changed in 0.23. We should audit hbase to insulate it from hadoop property name changes. Here is a list of the hadoop property name changes: http://hadoop.apache.org/common/docs/r0.23.1/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5685) [findbugs] Exclude Protobuf warnings from wire compat patches.
[findbugs] Exclude Protobuf warnings from wire compat patches. -- Key: HBASE-5685 URL: https://issues.apache.org/jira/browse/HBASE-5685 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5642) [findbugs] Exclude Thrift and Protobuf warnings
[findbugs] Exclude Thrift and Protobuf warnings --- Key: HBASE-5642 URL: https://issues.apache.org/jira/browse/HBASE-5642 Project: HBase Issue Type: Sub-task Components: build Affects Versions: 0.96.0 Reporter: Jonathan Hsieh Exclude thrift and protobuf warnings since these are machine generated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5643) [findbugs] Fix compareTo/equals/hashcode warnings
[findbugs] Fix compareTo/equals/hashcode warnings - Key: HBASE-5643 URL: https://issues.apache.org/jira/browse/HBASE-5643 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Fix code to eliminate [Eq,ES,HE] categories. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5644) [findbugs] Fix null pointer warnings.
[findbugs] Fix null pointer warnings. - Key: HBASE-5644 URL: https://issues.apache.org/jira/browse/HBASE-5644 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Fix the NP category -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5645) [findbugs] Fix correctness warnings
[findbugs] Fix correctness warnings --- Key: HBASE-5645 URL: https://issues.apache.org/jira/browse/HBASE-5645 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Fix the warnings in the correctness section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5646) [findbugs] Investigate experimental warnings
[findbugs] Investigate experimental warnings Key: HBASE-5646 URL: https://issues.apache.org/jira/browse/HBASE-5646 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html analyze and fix/exclude warnings in the experimental section. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5647) [findbugs] Address Exposed internal representation warnings
[findbugs] Address Exposed internal representation warnings - Key: HBASE-5647 URL: https://issues.apache.org/jira/browse/HBASE-5647 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html This class of warning may need to be fixed or excluded. Fix or justify + exclude. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5649) [findbugs] Fix security warning
[findbugs] Fix security warning --- Key: HBASE-5649 URL: https://issues.apache.org/jira/browse/HBASE-5649 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_SECURITY Fix possible XSS Vuln. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5648) [findbugs] Fix final/protected/constant declarations.
[findbugs] Fix final/protected/constant declarations. - Key: HBASE-5648 URL: https://issues.apache.org/jira/browse/HBASE-5648 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Fix warnings from class MS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5651) [findbugs] Address wiat/notify synchronization/inconsistency in sync
[findbugs] Address wiat/notify synchronization/inconsistency in sync Key: HBASE-5651 URL: https://issues.apache.org/jira/browse/HBASE-5651 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS fix classes IS,LI,MWM, NN, SWL, UG, UW -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5650) [findbugs] Address extra synchronization on CLSM, Atomic*
[findbugs] Address extra synchronization on CLSM, Atomic* - Key: HBASE-5650 URL: https://issues.apache.org/jira/browse/HBASE-5650 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_MT_CORRECTNESS Fix/exclude class JLM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5652) [find
[find - Key: HBASE-5652 URL: https://issues.apache.org/jira/browse/HBASE-5652 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5653) [findbugs] fix perf warnings
[findbugs] fix perf warnings Key: HBASE-5653 URL: https://issues.apache.org/jira/browse/HBASE-5653 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_PERFORMANCE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5654) [findbugs] Address dodgy bugs
[findbugs] Address dodgy bugs - Key: HBASE-5654 URL: https://issues.apache.org/jira/browse/HBASE-5654 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh See https://builds.apache.org/job/PreCommit-HBASE-Build/1313//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html#Warnings_STYLE This may be broken down further. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5634) document how to use uberhbck
document how to use uberhbck Key: HBASE-5634 URL: https://issues.apache.org/jira/browse/HBASE-5634 Project: HBase Issue Type: Improvement Components: documentation, hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh The updated hbck from HBASE-5128 introduces many new repair options and, as a side effect, offers many new opportunities to durably shoot oneself in the foot. Docs need to be written and added to the ref guide to explain its usage and ramifications and discuss repair strategies. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5628) Improve performance of uberhbck
Improve performance of uberhbck --- Key: HBASE-5628 URL: https://issues.apache.org/jira/browse/HBASE-5628 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh During reviews of HBASE-5128 there are several opportunities investigate for improving the performance of the tool. - Change regionInfoMap and tablesInfo from TreeMap to HashMap. - Change some full region set reloads to be incremental to require fewer passes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5629) Unify OfflineMetaRebuild with Hbck
Unify OfflineMetaRebuild with Hbck -- Key: HBASE-5629 URL: https://issues.apache.org/jira/browse/HBASE-5629 Project: HBase Issue Type: Improvement Affects Versions: 0.90.7, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Currently hbck and OfflineMetaRepair share a lot of code but OfflineMetaRepair is currently behind in functionality. It seems that we could merge hbck and OfflineMetaRepair adding something like a -hdfsOnly or -offline flag to hbck. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5630) hbck should disable the balancer using synchronousBalanceSwitch.
hbck should disable the balancer using synchronousBalanceSwitch. Key: HBASE-5630 URL: https://issues.apache.org/jira/browse/HBASE-5630 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Priority: Minor hbck disable the balancer using admin.balanceSwith(bool) when it would be preferable to use the newer synchronusBalanceSwitch method found in 0.94 and trunk branches. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5631) hbck should handle case where .tableinfo file is missing.
hbck should handle case where .tableinfo file is missing. - Key: HBASE-5631 URL: https://issues.apache.org/jira/browse/HBASE-5631 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh 0.92+ branches have a .tableinfo file which could be missing from hdfs. hbck should be able to detect and repair this properly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5603) rolling-restart.sh script hangs when attempting to detect expiration of /hbase/master znode.
rolling-restart.sh script hangs when attempting to detect expiration of
/hbase/master znode.
Key: HBASE-5603
URL: https://issues.apache.org/jira/browse/HBASE-5603
Project: HBase
Issue Type: Bug
Components: zookeeper
Affects Versions: 0.92.0, 0.94.0, 0.96.0
Reporter: Jonathan Hsieh
Due to bugfix ZOOKEEPER-1059 (ZK 3.4.0+), the rolling-restart.sh script will
hang when attempting to make sure the /hbase/master znode is deleted.
Here's the code
{code}
# make sure the master znode has been deleted before continuing
zparent=`$bin/hbase org.apache.hadoop.hbase.util.HBaseConfTool
zookeeper.znode.parent`
if [ $zparent == null ]; then zparent=/hbase; fi
zmaster=`$bin/hbase org.apache.hadoop.hbase.util.HBaseConfTool
zookeeper.znode.master`
if [ $zmaster == null ]; then zmaster=master; fi
zmaster=$zparent/$zmaster
echo -n Waiting for Master ZNode ${zmaster} to expire
while bin/hbase zkcli stat $zmaster /dev/null 21; do
echo -n .
sleep 1
done
echo #force a newline
{code}
Prior to ZOOKEEPER-1059, stat on a null znode would NPE and cause zkcli to exit
with retcode 1. Afterwards, the null is caught, zkcli will exit with 0 in the
case where the znode is present and in the case where it does not exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5588) Deprecate/remove AssignmentManager#clearRegionFromTransition
Deprecate/remove AssignmentManager#clearRegionFromTransition Key: HBASE-5588 URL: https://issues.apache.org/jira/browse/HBASE-5588 Project: HBase Issue Type: Sub-task Affects Versions: 0.92.0, 0.90.5, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh This method is essentially a dupe of Assignment#regionOffline. As suggested in early review of HBASE-5128 - deprecate up to 0.94 and remove from 0.96/trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5589) Add of the offline call to the Master Interface
Add of the offline call to the Master Interface --- Key: HBASE-5589 URL: https://issues.apache.org/jira/browse/HBASE-5589 Project: HBase Issue Type: Sub-task Affects Versions: 0.92.0, 0.90.6, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Hbck from HBASE-5128 requires an offline method on the master to properly cleanup state during certain assignment repair operations. This will this method will be added to recent and older versions of HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5590) Add more server mode state to jmx output.
Add more server mode state to jmx output. - Key: HBASE-5590 URL: https://issues.apache.org/jira/browse/HBASE-5590 Project: HBase Issue Type: Improvement Components: metrics Reporter: Jonathan Hsieh Related to HBASE-5325, and HBASE-5533 there is more state information that would be good to expose in a machine readable fashion. Some suggestions for state information include: * the balancer is on or off. * if a master is active or a backup. * If a we are in hlog recovery mode * tasks distributed from distributed log splitting More suggestions are welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5540) Update apache jenkins to include 0.94 and builds against Hadoop 0.23
Update apache jenkins to include 0.94 and builds against Hadoop 0.23 Key: HBASE-5540 URL: https://issues.apache.org/jira/browse/HBASE-5540 Project: HBase Issue Type: Task Components: build, test Reporter: Jonathan Hsieh Currently there is no hbase 0.94 apache jenkins build and the trunk on hadoop 0.23 builds are disabled. Ideally we should add the former and ren-enable the latter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5363) Add rat check to run automatically on mvn build.
Add rat check to run automatically on mvn build. Key: HBASE-5363 URL: https://issues.apache.org/jira/browse/HBASE-5363 Project: HBase Issue Type: Improvement Components: build Affects Versions: 0.92.0, 0.90.5 Reporter: Jonathan Hsieh Some of the recent hbase release failed rat checks (mvn rat:check). We should add checks likely in the mvn package phase so that this becomes a non-issue in the future. Here's an example from Whirr: https://github.com/apache/whirr/blob/trunk/pom.xml line 388 for an example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5364) Fix source files missing licenses
Fix source files missing licenses - Key: HBASE-5364 URL: https://issues.apache.org/jira/browse/HBASE-5364 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.90.5, 0.94.0 Reporter: Jonathan Hsieh Priority: Blocker running 'mvn rat:check' shows that a few files have snuck in that do not have proper apache licenses. Ideally we should fix these before we cut another release/release candidate. This is a blocker for 0.94, and probably should be for the other branches as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.90.3, 0.94.0
Reporter: Jonathan Hsieh
Priority: Minor
I was testing the region_mover.rb script on a loaded hbase and noticed that it
can hang (thus hanging graceful shutdown) if a region that it is attempting to
move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5360) [uberhbck] Add options for how to handle offline split parents.
[uberhbck] Add options for how to handle offline split parents. Key: HBASE-5360 URL: https://issues.apache.org/jira/browse/HBASE-5360 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.94.0, 0.90.7, 0.92.1 Reporter: Jonathan Hsieh In a recent case, we attempted to repair a cluster that suffered from HBASE-4238 that had about 6-7 generations of leftover split data. The hbck repair options in an development version of HBASE-5128 treat HDFS as ground truth but didn't check SPLIT and OFFLINE flags only found in meta. The net effect was that it essentially attempted to merge many regions back into its eldest geneneration's parent's range. More safe guards to prevent mega-merges are being added on HBASE-5128. This issue would automate the handling of the mega-merge avoiding cases such as lingering grandparents. The strategy here would be to add more checks against .META., and perform part of the catalog janitor's responsibilities for lingering grandparents. This would potentially include options to sideline regions, deleting grandparent regions, min size for sidelining, and mechanisms for cleaning .META.. Note: There already exists an mechanism to reload these regions -- the bulk loaded mechanisms in LoadIncrementalHFiles can be used to re-add grandparents (automatically splitting them if necessary) to HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5340) HFile/LoadIncrementalHFiles should specify file name when it fails to load a file.
HFile/LoadIncrementalHFiles should specify file name when it fails to load a
file.
--
Key: HBASE-5340
URL: https://issues.apache.org/jira/browse/HBASE-5340
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.90.5
Reporter: Jonathan Hsieh
I was attempting to do a bulk load and got this error message. Unfortunately
it didn't tell me which file had the problem.
{code}
Exception in thread main java.io.IOException: Trailer 'header' is wrong; does
the trailer size match content?
at
org.apache.hadoop.hbase.io.hfile.HFile$FixedFileTrailer.deserialize(HFile.java:1527)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.readTrailer(HFile.java:885)
at
org.apache.hadoop.hbase.io.hfile.HFile$Reader.loadFileInfo(HFile.java:819)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.tryLoad(LoadIncrementalHFiles.java:204)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:173)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:452)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:457)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5282) Possible file handle leak with truncate HLog file.
Possible file handle leak with truncate HLog file.
--
Key: HBASE-5282
URL: https://issues.apache.org/jira/browse/HBASE-5282
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.90.5, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
When debugging hbck, found that the code responsible for this exception can
leak open file handles.
{code}
12/01/15 05:58:11 INFO regionserver.HRegion: Replaying edits from hdfs://haus01.
sf.cloudera.com:56020/hbase-jon/test5/98a1e7255731aae44b3836641840113e/recovered
.edits/3211315; minSequenceid=3214658
12/01/15 05:58:11 ERROR handler.OpenRegionHandler: Failed open of region=test5,8
\x90\x00\x00\x00\x00\x00\x00/05_0,1326597390073.98a1e7255731aae44b3836641840
113e.
java.io.EOFException
at java.io.DataInputStream.readByte(DataInputStream.java:250)
at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:299)
at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:320)
at org.apache.hadoop.io.Text.readString(Text.java:400)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1486)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1437)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1424)
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1419)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.init(SequenceFileLogReader.java:57)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:158)
at
org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:572)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEdits(HRegion.java:1940)
at
org.apache.hadoop.hbase.regionserver.HRegion.replayRecoveredEditsIfAny(HRegion.java:1896)
at
org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:366)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2661)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2647)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:312)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:99)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:158)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5288) Security source code dirs missing from 0.92.0 release tarballs.
Security source code dirs missing from 0.92.0 release tarballs. --- Key: HBASE-5288 URL: https://issues.apache.org/jira/browse/HBASE-5288 Project: HBase Issue Type: Bug Components: build Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Hsieh Priority: Blocker Fix For: 0.94.0, 0.92.1 The release tarballs have a compiled version of the hbase jars and the security tarball seems to have the compiled security bits. However, the source code and resources for security implementation are missing from the release tarballs in both distributions. They should be included in both. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5280) Remove AssignmentManager#clearRegionFromTransition and replace with assignmentManager#regionOffline
Remove AssignmentManager#clearRegionFromTransition and replace with assignmentManager#regionOffline --- Key: HBASE-5280 URL: https://issues.apache.org/jira/browse/HBASE-5280 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.92.0, 0.90.5, 0.94.0 Reporter: Jonathan Hsieh These two methods are essentially the same and both present in the code base. It was suggested in the review for HBASE-5128 to remove #clearRegionFromTransition in favor of #regionOffline (HBASE-5128 deprecates this method, but it is internal to the HMaster, so should be safely removable from 0.92 and 0.94). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5128) [uber hbck] Enable hbck to automatically repair table integrity problems as well as region consistency problems while online.
[uber hbck] Enable hbck to automatically repair table integrity problems as
well as region consistency problems while online.
-
Key: HBASE-5128
URL: https://issues.apache.org/jira/browse/HBASE-5128
Project: HBase
Issue Type: New Feature
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
The current (0.90.5, 0.92.0rc2) versions of hbck detect most of the invariant
violations (orphans is new). However with '-fix' it can only automatically
handle deployment problems with region consistency cases. This updated version
should be able to handle all cases. When complete will likely deprecate the
OfflineMetaRepair tool and subsume several META hole related problems.
{code}
/**
* HBaseFsck (hbck) is a tool for checking and repairing region consistency and
* table integrity.
*
* Region consistency checks verify that META, region deployment on
* region servers and the state of data in HDFS (.regioninfo files) all are in
* accordance.
*
* Table integrity checks verify that that all possible row keys can resolve to
* exactly one region of a table. This means there are no individual degenerate
* or backwards regions; no holes between regions; and that there no overlapping
* regions.
*
* The general repair strategy works in these steps.
* 1) Repair Table Integrity on HDFS. (merge or fabricate regions)
* 2) Repair Region Consistency with META and assignments
*
* For table integrity repairs, the tables their region directories are scanned
* for .regioninfo files. Each table's integrity is then verified. If there
* are any orphan regions (regions with no .regioninfo files), or holes, new
* regions are fabricated. Backwards regions are sidelined as well as empty
* degenerate (endkey==startkey) regions. If there are any overlapping regions,
* a new region is created and all data is merged into the new region.
*
* Table integrity repairs deal solely with HDFS and can be done offline -- the
* hbase region servers or master do not need to be running. These phase can be
* use to completely reconstruct the META table in an offline fashion.
*
* Region consistency requires three conditions -- 1) valid .regioninfo file
* present in an hdfs region dir, 2) valid row with .regioninfo data in META,
* and 3) a region is deployed only at the regionserver that is was assigned to.
*
* Region consistency requires hbck to contact the HBase master and region
* servers, so the connect() must first be called successfully. Much of the
* region consistency information is transient and less risky to repair.
*/
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5103) Fix a places where master znode is improperly deserialized.
Fix a places where master znode is improperly deserialized.
---
Key: HBASE-5103
URL: https://issues.apache.org/jira/browse/HBASE-5103
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
In ActiveMasterManager#blockUntilBecomingActiveMaster the master znode is
created as a versioned serialized version of ServerName
{code}
if (ZKUtil.createEphemeralNodeAndWatch(this.watcher,
this.watcher.masterAddressZNode, sn.getVersionedBytes())) {
{code}
There are a few user visible places where it is used but not deserialized
properly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5101) Add a max number of regions per regionserver limit
Add a max number of regions per regionserver limit -- Key: HBASE-5101 URL: https://issues.apache.org/jira/browse/HBASE-5101 Project: HBase Issue Type: Improvement Components: regionserver Affects Versions: 0.92.0 Reporter: Jonathan Hsieh In a testing environment, a cluster got to a state with more than 1500 regions per region server, and essentially because stuck and unavailable. We could add a limit to the number of regions that a region server can serve to prevent this from happening. This looks like it could be implemented in the core or as a coprocessor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5083) Backup HMaster should have http infoport open with link to the active master
Backup HMaster should have http infoport open with link to the active master Key: HBASE-5083 URL: https://issues.apache.org/jira/browse/HBASE-5083 Project: HBase Issue Type: Improvement Components: master Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Without ssh'ing and jps/ps'ing, it is difficult to see if a backup hmaster is up. It seems like it would be good for a backup hmaster to have a basic web page up on the info port so that users could see that it is up. Also it should probably either provide a link to the active master. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5076) HBase shell hangs when creating some 'illegal' tables.
HBase shell hangs when creating some 'illegal' tables.
--
Key: HBASE-5076
URL: https://issues.apache.org/jira/browse/HBASE-5076
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.92.0
Reporter: Jonathan Hsieh
Priority: Minor
In hbase shell. These commands hang:
{code}
create 'hbase.version','foo'
create 'splitlog','foo'
{code}
Interestingly
{code}
create 'hbase.id','foo'
create existingtablename, 'foo'
create '.META.','foo'
create '-ROOT-','foo'
{code}
are properly rejected.
We should probably either rename to make the files illegal table names
(hbase.version to .hbase.version and splitlog to .splitlog) or we could add
more special cases.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5063) RegionServers fail to report to backup HMaster after primary goes down.
RegionServers fail to report to backup HMaster after primary goes down. --- Key: HBASE-5063 URL: https://issues.apache.org/jira/browse/HBASE-5063 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Priority: Critical # Setup cluster with two HMasters # Observe that HM1 is up and that all RS's are in the RegionServer list on web page. # Kill (not even -9) the active HMaster # Wait for ZK to time out (default 3 minutes). # Observe that HM2 is now active. Tables may show up but RegionServers never report on web page. Existing connections are fine. New connections cannot find regionservers. Note: * If we replace a new HM1 in the same place and kill HM2, the cluster functions normally again after recovery. This sees to indicate that regionservers are stuck trying to talk to the old HM1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-5034) Make distributed log splitting the default once we gain confidence in it.
Make distributed log splitting the default once we gain confidence in it. - Key: HBASE-5034 URL: https://issues.apache.org/jira/browse/HBASE-5034 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Jonathan Hsieh As a suggestion: To reduce the number of paths necessary for testing, we should make distributed log splitting the default setting for recovery once we gain confidence with it. After a release where it is the default (0.94 hopefully?), the release after could remove the original non-distributed version. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4972) Investigate and port patches on 0.90 branch that are not on 0.92/trunk branch.
Investigate and port patches on 0.90 branch that are not on 0.92/trunk branch. -- Key: HBASE-4972 URL: https://issues.apache.org/jira/browse/HBASE-4972 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Critical Fix For: 0.92.0 There are several issues that have been committed in the 0.90 branch but were not in trunk/0.92 branch. These regressions should be forward ported. HBASE-3320 ! HBASE-3380 ! - HBASE-4610 is a jira to backports this, but it is not done. HBASE-3410 ! HBASE-3501 ! HBASE-3714 ! HBASE-3729 !! Maked in 0.92 but not committed there, committed in 0.90 branch. HBASE-3848 ! HBASE-3892 ! * Comments say trunk does not need. HBASE-3906 ! HBASE-3989 ! HBASE-4109 ! HBASE-4160 !! Marked resolved 0.90.5, but no corresponding commit in either 0.90 or 0.92 HBASE-4423 ! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4947) CopyTable warns about being unable to find table from wrong instance of HBase
CopyTable warns about being unable to find table from wrong instance of HBase - Key: HBASE-4947 URL: https://issues.apache.org/jira/browse/HBASE-4947 Project: HBase Issue Type: Bug Affects Versions: 0.90.4, 0.92.0 Reporter: Jonathan Hsieh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4934) Display Master server and Regionserver start time on respective info servers.
Display Master server and Regionserver start time on respective info servers. - Key: HBASE-4934 URL: https://issues.apache.org/jira/browse/HBASE-4934 Project: HBase Issue Type: Improvement Reporter: Jonathan Hsieh Priority: Minor With operations like rolling restart or master failovers, it is difficult to tell if a server is the old instance or the new restarted instance. Adding a start date stamp on the info web pages would be helpful for determining this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4931) CopyTable instructions are of date in book and usage in source.
CopyTable instructions are of date in book and usage in source.
---
Key: HBASE-4931
URL: https://issues.apache.org/jira/browse/HBASE-4931
Project: HBase
Issue Type: Bug
Components: util
Affects Versions: 0.90.4, 0.92.0
Reporter: Jonathan Hsieh
The book and the usage instructions refer to ReplicationRegionInterface and
ReplicationRegionServer which are no longer present in the 0.90+ versions.
{code}
$ bin/hbase org.apache.hadoop.hbase.mapreduce.CopyTable
--rs.class=org.apache.hadoop.hbase.ipc.ReplicationRegionInterface
--rs.impl=org.apache.hadoop.hbase.regionserver.replication.ReplicationRegionServer
--starttime=1265875194289 --endtime=1265878794289
--peer.adr=server1,server2,server3:2181:/hbase TestTable
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4890) fix possible NPE in HConnectionManager
fix possible NPE in HConnectionManager
--
Key: HBASE-4890
URL: https://issues.apache.org/jira/browse/HBASE-4890
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0
Reporter: Jonathan Hsieh
I was running YCSB against a 0.92 branch and encountered this error message:
{code}
11/11/29 08:47:16 WARN client.HConnectionManager$HConnectionImplementation:
Failed all from
region=usertable,user3917479014967760871,1322555655231.f78d161e5724495a9723bcd972f97f41.,
hostname=c0316.hal.cloudera.com, port=57020
java.util.concurrent.ExecutionException: java.lang.RuntimeException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1501)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1353)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:898)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:775)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:750)
at com.yahoo.ycsb.db.HBaseClient.update(Unknown Source)
at com.yahoo.ycsb.DBWrapper.update(Unknown Source)
at com.yahoo.ycsb.workloads.CoreWorkload.doTransactionUpdate(Unknown
Source)
at com.yahoo.ycsb.workloads.CoreWorkload.doTransaction(Unknown Source)
at com.yahoo.ycsb.ClientThread.run(Unknown Source)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1315)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1327)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1325)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:158)
at $Proxy4.multi(Unknown Source)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1330)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1328)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithoutRetries(HConnectionManager.java:1309)
... 7 more
{code}
It looks like the NPE is caused by server being null in the MultiRespone call()
method.
{code}
public MultiResponse call() throws IOException {
return getRegionServerWithoutRetries(
new ServerCallableMultiResponse(connection, tableName, null) {
public MultiResponse call() throws IOException {
return server.multi(multi);
}
@Override
public void connect(boolean reload) throws IOException {
server =
connection.getHRegionConnection(loc.getHostname(),
loc.getPort());
}
}
);
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4866) Fix possible NPE in AssignmentManager#regionOnline
Fix possible NPE in AssignmentManager#regionOnline
--
Key: HBASE-4866
URL: https://issues.apache.org/jira/browse/HBASE-4866
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.90.4
Reporter: Jonathan Hsieh
NPE encountered in users's HMaster logs:
{code}
11/11/22 23:45:37 FATAL master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.AssignmentManager.regionOnline(AssignmentManager.java:731)
at
org.apache.hadoop.hbase.master.AssignmentManager.processFailover(AssignmentManager.java:215)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:422)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:295)
{code}
From user list:
http://mail-archives.apache.org/mod_mbox/hbase-user/20.mbox/%3C4ECC9AFC.6030307%40qualtrics.com%3E
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4852) Tests that use RegionServer.openRegion such as TestHBaseFsck#testHBaseFsck should call openRegion synchronously
Tests that use RegionServer.openRegion such as TestHBaseFsck#testHBaseFsck should call openRegion synchronously --- Key: HBASE-4852 URL: https://issues.apache.org/jira/browse/HBASE-4852 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Hsieh Certain test cases like HBaseFsck#testHBaseFsck make calls to assign region servers and then read meta. The tests or hbck should be modified to make the RegionServer.openRegion call act synchronously. The main issue isn't what is returned, but when. Specifically in HBaseFsck#testHBaseFsck, the first 'hbck -fix', the master makes a call to the regionserver to issue an asynchronous request to open the region (which adds data to meta). The regionserver returns right away. The next hbck call will cause the master query meta again which is used to check consistency. A race is exposed -- sometimes the new meta entries are fixed before the second hbck call is done (failing the test), sometimes it is not (not failing). The hack in HBASE-4842 introduces a slight delay which usually allows the open request to finish and the meta entry to be updated before the subsequent 'hbck' call. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4842) [hbck] Fix intermittent failures on TestHBaseFsck.testHBaseFsck
[hbck] Fix intermittent failures on TestHBaseFsck.testHBaseFsck --- Key: HBASE-4842 URL: https://issues.apache.org/jira/browse/HBASE-4842 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh Its seems that on the 0.92 branch in particular, TestHBaseFsck.testHBaseFsck is intermittently failing. In the test, a region's assignment is changed in META but not in ZK. After the equivalent of 'hbck -fix', a subsequent check that should be clean comes up with a new ZK assignment but with META still being inconsistent with ZK. The RS in ZK sometimes this points to the same RS, but sometimes it moves to another ZK. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4804) Minor Dyslexia in CHANGES.txt
Minor Dyslexia in CHANGES.txt - Key: HBASE-4804 URL: https://issues.apache.org/jira/browse/HBASE-4804 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh I going through the 0.92 CHANGES and found are a few entries in CHANGES.txt where jira numbers don't match up descriptions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4806) Fix logging message in HbaseObjectWritable
Fix logging message in HbaseObjectWritable -- Key: HBASE-4806 URL: https://issues.apache.org/jira/browse/HBASE-4806 Project: HBase Issue Type: Bug Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Trivial This is a trivial fix to HBASE-3316 to fix an error message. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4740) [bulk load] the HBASE-4552 API can't tell if errors on region server is recoverable or unrecoverable error.
[bulk load] the HBASE-4552 API can't tell if errors on region server is recoverable or unrecoverable error. Key: HBASE-4740 URL: https://issues.apache.org/jira/browse/HBASE-4740 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Priority: Critical Running TestHFileOutputFormat more frequently seems to show that it has become flaky. It is difficult to tell if this is because of a unrecoverable failure or a recoverable failure. To make this visiable from test and for users, we need to make a change to bulkload call's interface on HRegionServer. The change should make successful rpcs return true, recoverable failures return false, and unrecoverable failure throw an IOException. This an RPC change so would be really good to get this api right before the final 0.92 goes out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4734) [bulk load] Warn if bulk load director contained no files.
[bulk load] Warn if bulk load director contained no files. -- Key: HBASE-4734 URL: https://issues.apache.org/jira/browse/HBASE-4734 Project: HBase Issue Type: Improvement Components: mapreduce Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Jonathan Hsieh Bulk load exits if no files are found in the specified directory. This can happen if a directory has been bulk loaded already (bulk load renames/moves files). It would be good to provide some sort of warning when this happens. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4718) Backport HBASE-4552 to 0.90 branch.
Backport HBASE-4552 to 0.90 branch. --- Key: HBASE-4718 URL: https://issues.apache.org/jira/browse/HBASE-4718 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Jonathan Hsieh In discussion of HBASE-4552 / HBASE-4677 there has been some discussion about whether and how to backport HBASE-4552 to the 0.90 branch. This is a potentially compatibility breaking so several approaches hav ebeen suggested. 1) provide patch but do not integrate 2) integrate patch that extends and deprecates old api without removing old api. It has been argued that clients are supposed to use LoadIncrementalHFiles api and not at the internal HRegionServer RPC api. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4677) Remove old single bulkLoadHFile method
Remove old single bulkLoadHFile method -- Key: HBASE-4677 URL: https://issues.apache.org/jira/browse/HBASE-4677 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh In review for HBASE-4649, there is some debate as whether to remove, deprecate, or leave the HRegionServer.bulkLoadHFile method. https://reviews.apache.org/r/2545/ . This jira will take care of that for the 0.92 and trunk releases, and allow the same patch to remain for an optional 0.90.x patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4652) Mechanism for atomic recovery from partial failure when atomically bulk-loading multi-CF rows.
Mechanism for atomic recovery from partial failure when atomically bulk-loading multi-CF rows. -- Key: HBASE-4652 URL: https://issues.apache.org/jira/browse/HBASE-4652 Project: HBase Issue Type: Bug Reporter: Jonathan Hsieh The suggestions from HBASE-4552 and initial implementations could violate row atomicity if there are multiple CFs. A mechanism to atomically rollback or commit only upon completion (mvcc at the store level) is needed if a partial failures encountered when attempting to bulk load multiple families. For example, let's say I want to bulk load a region with cfs A, B, C. I issue a call to an RS region to atomically bulkload the HFiles. The RS loads A and B successfully but fails on C (hdfs failure, or rs goes down, etc). We could rollback A and B - if we don't we would have A and B loaded but not C and have an atomicity violation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4649) Add atomic bulk load function to region server
Add atomic bulk load function to region server -- Key: HBASE-4649 URL: https://issues.apache.org/jira/browse/HBASE-4649 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh Add a method that atomically bulk load multiple hfiles. Row atomicity guarantees for multi-column family rows require this functionality. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4622) Remove trivial 0.90 deprecated code from 0.92 and trunk.
Remove trivial 0.90 deprecated code from 0.92 and trunk. Key: HBASE-4622 URL: https://issues.apache.org/jira/browse/HBASE-4622 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4623) Remove @deprecated Scan methods in 0.90 from TRUNK and 0.92
Remove @deprecated Scan methods in 0.90 from TRUNK and 0.92 --- Key: HBASE-4623 URL: https://issues.apache.org/jira/browse/HBASE-4623 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4624) Remove and convert @deprecated RemoteExceptionHandler.decodeRemoteException calls
Remove and convert @deprecated RemoteExceptionHandler.decodeRemoteException calls - Key: HBASE-4624 URL: https://issues.apache.org/jira/browse/HBASE-4624 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4625) Convert @deprecated HBaseTestCase tests in 0.90 into JUnit4 style tests in 0.92 and TRUNK
Convert @deprecated HBaseTestCase tests in 0.90 into JUnit4 style tests in 0.92 and TRUNK - Key: HBASE-4625 URL: https://issues.apache.org/jira/browse/HBASE-4625 Project: HBase Issue Type: Sub-task Reporter: Jonathan Hsieh This will class has 47 references so separating out into a separate subtask. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4632) Offline meta rebuild should check if hbase is online before acting.
Offline meta rebuild should check if hbase is online before acting. --- Key: HBASE-4632 URL: https://issues.apache.org/jira/browse/HBASE-4632 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0, 0.90.5 Reporter: Jonathan Hsieh Filing this issue because this patch seems to work on the 0.90 branch tests but seems to timeout on the 0.92/trunk branches. Depends on HBASE-4337 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4570) Scan ACID problem with concurrent puts.
Scan ACID problem with concurrent puts.
---
Key: HBASE-4570
URL: https://issues.apache.org/jira/browse/HBASE-4570
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.90.3, 0.90.1
Reporter: Jonathan Hsieh
When scanning a table sometimes rows that have multiple column families get
split into two rows if there are concurrent writes. In this particular case we
are overwriting the contents of a Get directly back onto itself as a Put.
For example, this is a two cf row (with f1, f2, .. f9 cfs). It is
actually returned as two rows (#55 and #56). Interestingly if the two were
merged we would have a single proper row.
Row row024461 had time stamps: [55:
keyvalues={row024461/f0:data/1318200440867/Put/vlen=1000,
row024461/f0:qual/1318200440867/Put/vlen=10,
row024461/f1:data/1318200440867/Put/vlen=1000,
row024461/f1:qual/1318200440867/Put/vlen=10,
row024461/f2:data/1318200440867/Put/vlen=1000,
row024461/f2:qual/1318200440867/Put/vlen=10,
row024461/f3:data/1318200440867/Put/vlen=1000,
row024461/f3:qual/1318200440867/Put/vlen=10,
row024461/f4:data/1318200440867/Put/vlen=1000,
row024461/f4:qual/1318200440867/Put/vlen=10},
56: keyvalues={row024461/f5:data/1318200440867/Put/vlen=1000,
row024461/f5:qual/1318200440867/Put/vlen=10,
row024461/f6:data/1318200440867/Put/vlen=1000,
row024461/f6:qual/1318200440867/Put/vlen=10,
row024461/f7:data/1318200440867/Put/vlen=1000,
row024461/f7:qual/1318200440867/Put/vlen=10,
row024461/f8:data/1318200440867/Put/vlen=1000,
row024461/f8:qual/1318200440867/Put/vlen=10,
row024461/f9:data/1318200440867/Put/vlen=1000,
row024461/f9:qual/1318200440867/Put/vlen=10}]
I've only tested this on 0.90.1+patches and 0.90.3+patches, but it is
consistent and duplicatable.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4515) User.getCurrent() can cause NPE.
User.getCurrent() can cause NPE.
Key: HBASE-4515
URL: https://issues.apache.org/jira/browse/HBASE-4515
Project: HBase
Issue Type: Bug
Affects Versions: 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
When testing with miniclusters that shutdown and are restarted, sometimes a
call to User.getCurrent().getName() NPEs when attempting to restart hbase.
Oddly this happens consistently on particular branches and not on others. I
don't know or understand why this happens but it has something to do with the
getCurrentUGI call in o.a.h.h.security.User.HadoopUser sometimes returning
null and sometimes returning data.
{code}
private HadoopUser() {
try {
ugi = (UserGroupInformation) callStatic(getCurrentUGI);
if (ugi == null) {
LOG.warn(Although successfully retrieved UserGroupInformation
+ it was null!);
}
} catch (RuntimeException re) {
{code}
This patch essentially is a workaround -- it propagates the null so that
clients can check and avoid the NPE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4506) [hbck] Allow HBaseFsck to be instantiated without connecting
[hbck] Allow HBaseFsck to be instantiated without connecting Key: HBASE-4506 URL: https://issues.apache.org/jira/browse/HBASE-4506 Project: HBase Issue Type: Improvement Components: hbck Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh This is a semantics preserving patch that allows for offline meta rebuild (HBASE-4377) to reuse code in the existing hbck code when hbase is down. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4509) [hbck] Improve region map output
[hbck] Improve region map output Key: HBASE-4509 URL: https://issues.apache.org/jira/browse/HBASE-4509 Project: HBase Issue Type: Improvement Affects Versions: 0.92.0, 0.94.0, 0.90.5 Reporter: Jonathan Hsieh HBASE-4375 added a region coverage visualization to hbck in details mode. When users have binary row keys the output is difficult to parse (awk/sed) or pull into programs (numeric, excel) capable of handling tsv formatted data. This patch * improves output by using Bytes.toStringBinary (which escapes binary) instead of Bytes.toString when printing keys, * suggests some repair actions, and * collects problem group that groups regions that are overlapping. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
