[
https://issues.apache.org/jira/browse/HADOOP-1421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12503122
]
Jim Kellerman commented on HADOOP-1421:
---------------------------------------
Modifications for this issue:
Configuration files:
conf/hbase-default.xml
- change hbase.regiondir to hbase.rootdir
Code base:
HAbstractScanner
- Change ColumnMatcher.matches so that when matching a family, it only
compares the family name instead of using startsWith
HClient
- used KeyedData instead of LabelledData
- getRow now returns SortedMap<Text, byte[]> instead of array of LabelledData
- added public API for renewLease
HConstants
- change parameter name hbase.regiondir to hbase.rootdir
- change HREGION_DIR to HBASE_DIR
- change DEFAULT_HREGION_DIR to DEFAULT_HBASE_DIR
HLog
- add method splitLog
- remove method consolidateOldLog
- add method closeAndDelete
HMaster
- change HREGION_DIR to HBASE_DIR
- change DEFAULT_HREGION_DIR to DEFAULT_HBASE_DIR
- change processing when region server reports that it is shutting down
- new method to capture common code: cancelLease
- PendingServerShutdown now splits dead region server's log
- move some methods around so that they are in the section pertaining
to a particular interface
- use KeyedData instead of LabelledData
- added pendingRegions for regions that have been assigned, but for
which the region server has not finished processing yet.
- root and meta scanners retry instead of exiting on first error
- added time outs for indefinate waits
- change waitForMetaScan to waitForMetaScanOrClose
- bootstrap code calls HLog.closeAndDelete so that the log directory
is not present when a region server loads the region
- separate master shutdown from region server reports shutdown
- instead of using find to determine if a region belongs to a table,
compare the table name to the name of the table in the
HTableDescriptor in the HRegionInfo
- remove waitForRootRegion in favor of waitForRootRegionOrClose
- LeaseListener is now an interface instead of an abstract class
- when a lease expires or when a region server reports that it is
shutting down and it was serving the root region, change
rootRegionLocation to null so that threads who need to talk to the
root region will wait until it becomes available again.
- if a meta region server goes down, set allMetaRegionsScanned to
false so that threads that need to scan the meta regions will wait
until the region comes back on line
HMerge
- change HREGION_DIR to HBASE_DIR
- change DEFAULT_HREGION_DIR to DEFAULT_HBASE_DIR
- use HLog closeAndDelete
- new signature for HRegion constructor
- remove useless else
HMsg
- remove MSG_REGION_MERGE, MSG_REGION_SPLIT
- add toString
HRegion
- use new HRegion constructor signature
HRegiondirReader
- use new HRegion constructor signature
HRegionInterface
- getRow and next return array of KeyedData instead of LabelledData
- drop unused parameter from next
HRegionServer
- change HREGION_DIR to HBASE_DIR
- change DEFAULT_HREGION_DIR to DEFAULT_HBASE_DIR
- throw new RegionServerRunningException if log directory exists
- clean up shutdown, report to master what regions were being served
- use HLog closeAndDelete
- remove processing for MSG_REGION_MERGE
- remove redundant MSG_CALL_SERVER_STARTUP and MSG_REGIONSERVER_STOP processing
- use new HRegion constructor signature
- LeaseListener is now an interface
- Drop unused parameter from next
- add new method abort for testing
- onlineRegions is now synchronized
- instead of using find to determine if a region belongs to a table,
compare the table name to the name of the table in the
HTableDescriptor in the HRegionInfo
- start threads and set their names in run instead of constructor
- tell master what regions were being served on a clean shutdown
- worker thread retries if an operation fails
- add timeout to previously indefinate waits
- don't try to serve a region if it is already being served (should
the master request that we open it twice)
- getRow and next return array of KeyedData instead of LabelledData
HStore
- fix logic error in determinining if an edit in an HLog applies to
this HStore
- remove useless else
HStoreKey
- add new constructor which copies another key
- add new set method which copies all the fields from another key
KeyedData
- new class, replaces LabelledData. Instead of a column name, value
pair, KeyedData is an HStoreKey, value pair
LabelledData
- class removed
LeaseListener
- change from abstract class to interface
- remove leaseRenewed and leaseCancelled APIs as they were never used
Leases
- createLease now throws AssertionError instead of IOException for
"this can't possibly happen" error
RegionServerRunningException
- added. Thrown when the log directory for a region server exists
Test cases:
AbstractMergeTestBase
- move createNewHRegion from HRegion to here
- use HRegion.addRegionToMETA instead of HRegion.addRegionToMeta
- use HLog.closeAndDelete
HBaseClusterTestCase
- add constructor that lets you specify the number of region servers
MiniHBaseCluster
- change HREGION_DIR to HBASE_DIR
- change DEFAULT_HREGION_DIR to DEFAULT_HBASE_DIR
- stopRegionServer and abortRegionServer APIs
TestCleanRegionServerExit
- new test: test failover when a region server is shut down cleanly
TestGet
- use new signature for HRegion constructor
- use HLog closeAndDelete
TestHLog
- fix order of arguments to assert
TestMemcache
- use HLog closeAndDelete
TestHRegion
- HRegion now handles old log file transparently
- use new signature for HRegion constructor
- remove extraneous throws clause
TestRegionServerAbort
- new test. Tests failover when a region server does not shut down cleanly
TestScanner
- use new signature for HRegion constructor
- use HLog closeAndDelete
TestScanner2
- add copyright statement
- use new signature for HRegion constructor
- fix order of arguments to assert
- use KeyedData instead of LabelledData
> When a region server dies, its log must be distributed to all the new servers
> that have been assigned its regions
> -----------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1421
> URL: https://issues.apache.org/jira/browse/HADOOP-1421
> Project: Hadoop
> Issue Type: New Feature
> Components: contrib/hbase
> Affects Versions: 0.14.0
> Reporter: Jim Kellerman
> Assignee: Jim Kellerman
> Fix For: 0.14.0
>
> Attachments: patch.txt
>
>
> When a region server dies, the master reallocates all the regions it was
> serving to other region servers. In order for the regions to be recovered
> properly, the log from the old region server must be split up into chunks
> that contain only the commits for a single region.
> Ideally, if a region server is assigned more than one of the old servers
> regions, the log for that server would contain all the commits for all the
> regions it has been assigned.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
