[ https://issues.apache.org/jira/browse/HBASE-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16104582#comment-16104582 ]
Zheng Hu commented on HBASE-16466: ---------------------------------- [~sukuna...@gmail.com], I have one question for your test provided. Did you run the MR job on the same HDFS cluster for Source/Peer HBase Cluster & Yarn Cluster ? Seems like that when source hbase cluster / peer hbase cluster / yarn cluster locate in three different HDFS cluster , it has one problem. when restoring the snapshot into tmpdir , we need to create region by following code (HRegion#createHRegion) {code} public static HRegion createHRegion(final HRegionInfo info, final Path rootDir, final Configuration conf, final TableDescriptor hTableDescriptor, final WAL wal, final boolean initialize) throws IOException { LOG.info("creating HRegion " + info.getTable().getNameAsString() + " HTD == " + hTableDescriptor + " RootDir = " + rootDir + " Table name == " + info.getTable().getNameAsString()); FileSystem fs = FileSystem.get(conf); <------------------- Here our code use fs.defaultFs configuration to create region. Path tableDir = FSUtils.getTableDir(rootDir, info.getTable()); HRegionFileSystem.createRegionOnFileSystem(conf, fs, tableDir, info); HRegion region = HRegion.newHRegion(tableDir, wal, fs, conf, info, hTableDescriptor, null); if (initialize) region.initialize(null); return region; } {code} When source cluster & peer cluster locate in two difference file systems , then their fs.defaultFs should be difference, so at least one cluster will fail when restore snapshot into tmpdir . after I added the following fix, it works fine for me. {code} - FileSystem fs = FileSystem.get(conf); + FileSystem fs = rootDir.getFileSystem(conf); {code} Looking forward to your reply, Thanks. > HBase snapshots support in VerifyReplication tool to reduce load on live > HBase cluster with large tables > -------------------------------------------------------------------------------------------------------- > > Key: HBASE-16466 > URL: https://issues.apache.org/jira/browse/HBASE-16466 > Project: HBase > Issue Type: Improvement > Components: hbase > Affects Versions: 0.98.21 > Reporter: Sukumar Maddineni > Assignee: Maddineni Sukumar > Fix For: 2.0.0 > > Attachments: HBASE-16466.branch-1.3.001.patch, HBASE-16466.v1.patch, > HBASE-16466.v2.patch, HBASE-16466.v3.patch, HBASE-16466.v4.patch, > HBASE-16466.v5.patch > > > As of now VerifyReplicatin tool is running using normal HBase scanners. If > you want to run VerifyReplication multiple times on a production live > cluster with large tables then it creates extra load on HBase layer. So if we > implement snapshot based support then both in source and target we can read > data from snapshots which reduces load on HBase -- This message was sent by Atlassian JIRA (v6.4.14#64029)