You're welcome. Fix is attached to HBASE-11443
Cheers On Mon, Jun 30, 2014 at 3:13 PM, Enis Söztutar <[email protected]> wrote: > I'm taking a look at the TestIOFencing. I can repro locally. Thanks Ted for > reporting. > > Enis > > > On Fri, Jun 27, 2014 at 8:16 PM, Ted Yu <[email protected]> wrote: > > > Enis: > > I ran test suite on Linux and found that TestIOFencing hung. > > > > jstack is here: > > http://people.apache.org/~tedyu/TestIOFencing.stack > > > > Unfortunately there was no test output. > > > > Will continue digging in the coming days. > > > > Cheers > > > > > > On Fri, Jun 27, 2014 at 7:32 PM, Enis Söztutar <[email protected]> wrote: > > > > > Update: > > > > > > I've completed the merge for branch hbase-10070 to master now. As a > short > > > reminder the committed changes include phase 1 work for the new feature > > > called "read high availability using timeline consistent reads with > > region > > > replicas". Some more context can be found in [1] and vote thread here > > [2]. > > > > > > The merge contains 44 commits in total as follows. I've uploaded the > > > committed patches to relevant jiras, and set the fix versions for the > > jiras > > > as well (using git format-patch and the excellent jira-cli client). > > > > > > Applying: HBASE-10347 HRegionInfo changes for adding replicaId and > > > MetaEditor/MetaReader changes for region replicas > > > Applying: HBASE-10354 Add an API for defining consistency per request > > > Applying: HBASE-10348. HTableDescriptor changes for region replicas > > > Applying: HBASE-10517 NPE in MetaCache.clearCache() > > > Applying: HBASE-10354. Addendum commit. 10354.add.patch committed > > > Applying: HBASE-10356 Failover RPC's for multi-get > > > Applying: HBASE-10350. Master/AM/RegionStates changes to create and > > assign > > > region replicas (ddas) > > > Applying: HBASE-10359 Master/RS WebUI changes for region replicas > > > Applying: HBASE-10361. Enable/AlterTable support for region replicas > > (ddas) > > > Applying: HBASE-10352 Region and RegionServer changes for opening > region > > > replicas, and refreshing store files > > > Applying: HBASE-10362. HBCK changes for supporting region replicas > > > Applying: HBASE-10351 LoadBalancer changes for supporting region > replicas > > > Applying: HBASE-10630 NullPointerException in > > > ConnectionManager$HConnectionImplementation.locateRegionInMeta() due to > > > missing region info > > > Applying: HBASE-10633. StoreFileRefresherChore throws > > > ConcurrentModificationException sometimes > > > Applying: HBASE-10672. Table snapshot should handle tables whose > > > REGION_REPLICATION is greater than one > > > Applying: HBASE-10620. LoadBalancer.needsBalance() should check for > > > co-located region replicas as well > > > Applying: HBASE-10355 Failover RPC's from client using region replicas > > > Applying: HBASE-10704 BaseLoadBalancer#roundRobinAssignment() may add > > same > > > region to assignment plan multiple times > > > Applying: HBASE-10572 Create an IntegrationTest for region replicas > > > Applying: HBASE-10729. Enable table doesn't balance out replicas evenly > > if > > > the replicas were unassigned earlier > > > Applying: HBASE-10620 LoadBalancer.needsBalance() should check for > > > co-located region replicas as well (Addendum patch to fix return value > > from > > > balance() call) > > > Applying: HBASE-10734. Fix RegionStates.getRegionAssignments to not add > > > duplicate regions > > > Applying: HBASE-10616. Integration test for multi-get calls > > > Applying: HBASE-10743. Replica map update is problematic in > RegionStates > > > Applying: HBASE-10616. Addendum that fixes the case of > multiget_batchsize > > > == 1 > > > Applying: HBASE-10778 Unique keys accounting in MultiThreadedReader is > > > incorrect > > > Applying: HBASE-10858 TestRegionRebalancing is failing > > > Applying: HBASE-10817 Add some tests on a real cluster for replica: > multi > > > master, replication > > > Applying: HBASE-10701 Cache invalidation improvements from client side > > > Applying: HBASE-10859 Use HFileLink in opening region files from > > > secondaries > > > Applying: HBASE-10791 Add integration test to demonstrate performance > > > improvement > > > Applying: HBASE-10634 Multiget doesn't fully work > > > Applying: HBASE-10794 multi-get should handle replica location missing > > from > > > cache > > > Applying: HBASE-10785 Metas own location should be cached > > > Applying: HBASE-10957 HMaster can abort with NPE in #rebuildUserRegions > > > (Nicolas Liochon) > > > Applying: HBASE-10661 > > > TestStochasticLoadBalancer.testRegionReplicationOnMidClusterWithRacks() > > is > > > flaky > > > Applying: HBASE-10513 Provide user documentation for region replicas > > > Applying: HBASE-10810 LoadTestTool should share the connection and > > > connection pool > > > Applying: HBASE-10357 Failover RPC's for scans (Devaraj Das) > > > Applying: HBASE-11214. Fixes for scans on a replicated table > > > Applying: HBASE-10818. Add integration test for bulkload with replicas > > > (Nick Dimiduk and Devaraj Das) > > > Applying: HBASE-10818. Addendum to reduce the number of times sleep > > happens > > > in the coprocessor. Also, puts back code to set consistency that got > > > deleted by accident in the last commit. > > > Applying: HBASE-11332 Fix for metas location cache from HBASE-10785 > > > Applying: Increase -XX:MaxPermSize for unit tests > > > > > > > > > Unit tests should be working now, but in case it fails on the apache > > infra, > > > I'll take a closer look at the jenkins build at > > > https://builds.apache.org/view/All/job/HBase-TRUNK/. We've been > running > > > the > > > integration tests as well but ideally we want to spend some more time > on > > > perf regression analysis and long tail latency improvements. We can > iron > > > out the edges in time. Nicolas has already been doing work around this. > > > > > > Let me know if you see any issues related to this merge. > > > > > > > > > [1] > > > > https://mail-archives.apache.org/mod_mbox/hbase-dev/201405.mbox/browser > > > [2] > > > > > > > > > https://mail-archives.apache.org/mod_mbox/hbase-dev/201406.mbox/%3CCAMUu0w8uvb29%2BQYpZ8VwSO4qEpKRyKGNPzyj1kw7tLRhLPoZkA%40mail.gmail.com%3E > > > > > > > > > > > > > > > On Fri, Jun 27, 2014 at 7:03 PM, Ted Yu <[email protected]> wrote: > > > > > > > Good job. > > > > > > > > > > > > On Fri, Jun 27, 2014 at 5:45 PM, Enis Söztutar <[email protected]> > > wrote: > > > > > > > > > Pushed the rebased branch. You can go crazy on the master now. > > > > > > > > > > Enis > > > > > > > > > > > > > > > On Fri, Jun 27, 2014 at 4:37 PM, Enis Söztutar <[email protected]> > > > wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > I am merging the hbase-10070 branch to master. No checkins > please. > > > > > > > > > > > > We were able to get unit tests working, and IT's working. There > may > > > be > > > > a > > > > > > couple of edge cases after rebase, but we can resolve them in > > master. > > > > > > > > > > > > I'll send a detailed update once done. > > > > > > > > > > > > Enis > > > > > > > > > > > > > > > > > > > > >
