Update: I've completed the merge for branch hbase-10070 to master now. As a short reminder the committed changes include phase 1 work for the new feature called "read high availability using timeline consistent reads with region replicas". Some more context can be found in [1] and vote thread here [2].
The merge contains 44 commits in total as follows. I've uploaded the committed patches to relevant jiras, and set the fix versions for the jiras as well (using git format-patch and the excellent jira-cli client). Applying: HBASE-10347 HRegionInfo changes for adding replicaId and MetaEditor/MetaReader changes for region replicas Applying: HBASE-10354 Add an API for defining consistency per request Applying: HBASE-10348. HTableDescriptor changes for region replicas Applying: HBASE-10517 NPE in MetaCache.clearCache() Applying: HBASE-10354. Addendum commit. 10354.add.patch committed Applying: HBASE-10356 Failover RPC's for multi-get Applying: HBASE-10350. Master/AM/RegionStates changes to create and assign region replicas (ddas) Applying: HBASE-10359 Master/RS WebUI changes for region replicas Applying: HBASE-10361. Enable/AlterTable support for region replicas (ddas) Applying: HBASE-10352 Region and RegionServer changes for opening region replicas, and refreshing store files Applying: HBASE-10362. HBCK changes for supporting region replicas Applying: HBASE-10351 LoadBalancer changes for supporting region replicas Applying: HBASE-10630 NullPointerException in ConnectionManager$HConnectionImplementation.locateRegionInMeta() due to missing region info Applying: HBASE-10633. StoreFileRefresherChore throws ConcurrentModificationException sometimes Applying: HBASE-10672. Table snapshot should handle tables whose REGION_REPLICATION is greater than one Applying: HBASE-10620. LoadBalancer.needsBalance() should check for co-located region replicas as well Applying: HBASE-10355 Failover RPC's from client using region replicas Applying: HBASE-10704 BaseLoadBalancer#roundRobinAssignment() may add same region to assignment plan multiple times Applying: HBASE-10572 Create an IntegrationTest for region replicas Applying: HBASE-10729. Enable table doesn't balance out replicas evenly if the replicas were unassigned earlier Applying: HBASE-10620 LoadBalancer.needsBalance() should check for co-located region replicas as well (Addendum patch to fix return value from balance() call) Applying: HBASE-10734. Fix RegionStates.getRegionAssignments to not add duplicate regions Applying: HBASE-10616. Integration test for multi-get calls Applying: HBASE-10743. Replica map update is problematic in RegionStates Applying: HBASE-10616. Addendum that fixes the case of multiget_batchsize == 1 Applying: HBASE-10778 Unique keys accounting in MultiThreadedReader is incorrect Applying: HBASE-10858 TestRegionRebalancing is failing Applying: HBASE-10817 Add some tests on a real cluster for replica: multi master, replication Applying: HBASE-10701 Cache invalidation improvements from client side Applying: HBASE-10859 Use HFileLink in opening region files from secondaries Applying: HBASE-10791 Add integration test to demonstrate performance improvement Applying: HBASE-10634 Multiget doesn't fully work Applying: HBASE-10794 multi-get should handle replica location missing from cache Applying: HBASE-10785 Metas own location should be cached Applying: HBASE-10957 HMaster can abort with NPE in #rebuildUserRegions (Nicolas Liochon) Applying: HBASE-10661 TestStochasticLoadBalancer.testRegionReplicationOnMidClusterWithRacks() is flaky Applying: HBASE-10513 Provide user documentation for region replicas Applying: HBASE-10810 LoadTestTool should share the connection and connection pool Applying: HBASE-10357 Failover RPC's for scans (Devaraj Das) Applying: HBASE-11214. Fixes for scans on a replicated table Applying: HBASE-10818. Add integration test for bulkload with replicas (Nick Dimiduk and Devaraj Das) Applying: HBASE-10818. Addendum to reduce the number of times sleep happens in the coprocessor. Also, puts back code to set consistency that got deleted by accident in the last commit. Applying: HBASE-11332 Fix for metas location cache from HBASE-10785 Applying: Increase -XX:MaxPermSize for unit tests Unit tests should be working now, but in case it fails on the apache infra, I'll take a closer look at the jenkins build at https://builds.apache.org/view/All/job/HBase-TRUNK/. We've been running the integration tests as well but ideally we want to spend some more time on perf regression analysis and long tail latency improvements. We can iron out the edges in time. Nicolas has already been doing work around this. Let me know if you see any issues related to this merge. [1] https://mail-archives.apache.org/mod_mbox/hbase-dev/201405.mbox/browser [2] https://mail-archives.apache.org/mod_mbox/hbase-dev/201406.mbox/%3CCAMUu0w8uvb29%2BQYpZ8VwSO4qEpKRyKGNPzyj1kw7tLRhLPoZkA%40mail.gmail.com%3E On Fri, Jun 27, 2014 at 7:03 PM, Ted Yu <[email protected]> wrote: > Good job. > > > On Fri, Jun 27, 2014 at 5:45 PM, Enis Söztutar <[email protected]> wrote: > > > Pushed the rebased branch. You can go crazy on the master now. > > > > Enis > > > > > > On Fri, Jun 27, 2014 at 4:37 PM, Enis Söztutar <[email protected]> wrote: > > > > > Hi, > > > > > > I am merging the hbase-10070 branch to master. No checkins please. > > > > > > We were able to get unit tests working, and IT's working. There may be > a > > > couple of edge cases after rebase, but we can resolve them in master. > > > > > > I'll send a detailed update once done. > > > > > > Enis > > > > > >
