I'm taking a look at the TestIOFencing. I can repro locally. Thanks Ted for reporting.
Enis On Fri, Jun 27, 2014 at 8:16 PM, Ted Yu <[email protected]> wrote: > Enis: > I ran test suite on Linux and found that TestIOFencing hung. > > jstack is here: > http://people.apache.org/~tedyu/TestIOFencing.stack > > Unfortunately there was no test output. > > Will continue digging in the coming days. > > Cheers > > > On Fri, Jun 27, 2014 at 7:32 PM, Enis Söztutar <[email protected]> wrote: > > > Update: > > > > I've completed the merge for branch hbase-10070 to master now. As a short > > reminder the committed changes include phase 1 work for the new feature > > called "read high availability using timeline consistent reads with > region > > replicas". Some more context can be found in [1] and vote thread here > [2]. > > > > The merge contains 44 commits in total as follows. I've uploaded the > > committed patches to relevant jiras, and set the fix versions for the > jiras > > as well (using git format-patch and the excellent jira-cli client). > > > > Applying: HBASE-10347 HRegionInfo changes for adding replicaId and > > MetaEditor/MetaReader changes for region replicas > > Applying: HBASE-10354 Add an API for defining consistency per request > > Applying: HBASE-10348. HTableDescriptor changes for region replicas > > Applying: HBASE-10517 NPE in MetaCache.clearCache() > > Applying: HBASE-10354. Addendum commit. 10354.add.patch committed > > Applying: HBASE-10356 Failover RPC's for multi-get > > Applying: HBASE-10350. Master/AM/RegionStates changes to create and > assign > > region replicas (ddas) > > Applying: HBASE-10359 Master/RS WebUI changes for region replicas > > Applying: HBASE-10361. Enable/AlterTable support for region replicas > (ddas) > > Applying: HBASE-10352 Region and RegionServer changes for opening region > > replicas, and refreshing store files > > Applying: HBASE-10362. HBCK changes for supporting region replicas > > Applying: HBASE-10351 LoadBalancer changes for supporting region replicas > > Applying: HBASE-10630 NullPointerException in > > ConnectionManager$HConnectionImplementation.locateRegionInMeta() due to > > missing region info > > Applying: HBASE-10633. StoreFileRefresherChore throws > > ConcurrentModificationException sometimes > > Applying: HBASE-10672. Table snapshot should handle tables whose > > REGION_REPLICATION is greater than one > > Applying: HBASE-10620. LoadBalancer.needsBalance() should check for > > co-located region replicas as well > > Applying: HBASE-10355 Failover RPC's from client using region replicas > > Applying: HBASE-10704 BaseLoadBalancer#roundRobinAssignment() may add > same > > region to assignment plan multiple times > > Applying: HBASE-10572 Create an IntegrationTest for region replicas > > Applying: HBASE-10729. Enable table doesn't balance out replicas evenly > if > > the replicas were unassigned earlier > > Applying: HBASE-10620 LoadBalancer.needsBalance() should check for > > co-located region replicas as well (Addendum patch to fix return value > from > > balance() call) > > Applying: HBASE-10734. Fix RegionStates.getRegionAssignments to not add > > duplicate regions > > Applying: HBASE-10616. Integration test for multi-get calls > > Applying: HBASE-10743. Replica map update is problematic in RegionStates > > Applying: HBASE-10616. Addendum that fixes the case of multiget_batchsize > > == 1 > > Applying: HBASE-10778 Unique keys accounting in MultiThreadedReader is > > incorrect > > Applying: HBASE-10858 TestRegionRebalancing is failing > > Applying: HBASE-10817 Add some tests on a real cluster for replica: multi > > master, replication > > Applying: HBASE-10701 Cache invalidation improvements from client side > > Applying: HBASE-10859 Use HFileLink in opening region files from > > secondaries > > Applying: HBASE-10791 Add integration test to demonstrate performance > > improvement > > Applying: HBASE-10634 Multiget doesn't fully work > > Applying: HBASE-10794 multi-get should handle replica location missing > from > > cache > > Applying: HBASE-10785 Metas own location should be cached > > Applying: HBASE-10957 HMaster can abort with NPE in #rebuildUserRegions > > (Nicolas Liochon) > > Applying: HBASE-10661 > > TestStochasticLoadBalancer.testRegionReplicationOnMidClusterWithRacks() > is > > flaky > > Applying: HBASE-10513 Provide user documentation for region replicas > > Applying: HBASE-10810 LoadTestTool should share the connection and > > connection pool > > Applying: HBASE-10357 Failover RPC's for scans (Devaraj Das) > > Applying: HBASE-11214. Fixes for scans on a replicated table > > Applying: HBASE-10818. Add integration test for bulkload with replicas > > (Nick Dimiduk and Devaraj Das) > > Applying: HBASE-10818. Addendum to reduce the number of times sleep > happens > > in the coprocessor. Also, puts back code to set consistency that got > > deleted by accident in the last commit. > > Applying: HBASE-11332 Fix for metas location cache from HBASE-10785 > > Applying: Increase -XX:MaxPermSize for unit tests > > > > > > Unit tests should be working now, but in case it fails on the apache > infra, > > I'll take a closer look at the jenkins build at > > https://builds.apache.org/view/All/job/HBase-TRUNK/. We've been running > > the > > integration tests as well but ideally we want to spend some more time on > > perf regression analysis and long tail latency improvements. We can iron > > out the edges in time. Nicolas has already been doing work around this. > > > > Let me know if you see any issues related to this merge. > > > > > > [1] > > https://mail-archives.apache.org/mod_mbox/hbase-dev/201405.mbox/browser > > [2] > > > > > https://mail-archives.apache.org/mod_mbox/hbase-dev/201406.mbox/%3CCAMUu0w8uvb29%2BQYpZ8VwSO4qEpKRyKGNPzyj1kw7tLRhLPoZkA%40mail.gmail.com%3E > > > > > > > > > > On Fri, Jun 27, 2014 at 7:03 PM, Ted Yu <[email protected]> wrote: > > > > > Good job. > > > > > > > > > On Fri, Jun 27, 2014 at 5:45 PM, Enis Söztutar <[email protected]> > wrote: > > > > > > > Pushed the rebased branch. You can go crazy on the master now. > > > > > > > > Enis > > > > > > > > > > > > On Fri, Jun 27, 2014 at 4:37 PM, Enis Söztutar <[email protected]> > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I am merging the hbase-10070 branch to master. No checkins please. > > > > > > > > > > We were able to get unit tests working, and IT's working. There may > > be > > > a > > > > > couple of edge cases after rebase, but we can resolve them in > master. > > > > > > > > > > I'll send a detailed update once done. > > > > > > > > > > Enis > > > > > > > > > > > > > > >
