[
https://issues.apache.org/jira/browse/HBASE-16466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16104582#comment-16104582
]
Zheng Hu commented on HBASE-16466:
----------------------------------
[[email protected]], I have one question for your test provided. Did you
run the MR job on the same HDFS cluster for Source/Peer HBase Cluster & Yarn
Cluster ?
Seems like that when source hbase cluster / peer hbase cluster / yarn cluster
locate in three different HDFS cluster , it has one problem.
when restoring the snapshot into tmpdir , we need to create region by
following code (HRegion#createHRegion)
{code}
public static HRegion createHRegion(final HRegionInfo info, final Path
rootDir,
final Configuration conf, final TableDescriptor hTableDescriptor,
final WAL wal, final boolean initialize)
throws IOException {
LOG.info("creating HRegion " + info.getTable().getNameAsString()
+ " HTD == " + hTableDescriptor + " RootDir = " + rootDir +
" Table name == " + info.getTable().getNameAsString());
FileSystem fs = FileSystem.get(conf);
<------------------- Here our code use fs.defaultFs configuration to create
region.
Path tableDir = FSUtils.getTableDir(rootDir, info.getTable());
HRegionFileSystem.createRegionOnFileSystem(conf, fs, tableDir, info);
HRegion region = HRegion.newHRegion(tableDir, wal, fs, conf, info,
hTableDescriptor, null);
if (initialize) region.initialize(null);
return region;
}
{code}
When source cluster & peer cluster locate in two difference file systems , then
their fs.defaultFs should be difference, so at least one cluster will fail
when restore snapshot into tmpdir . after I added the following fix, it works
fine for me.
{code}
- FileSystem fs = FileSystem.get(conf);
+ FileSystem fs = rootDir.getFileSystem(conf);
{code}
Looking forward to your reply, Thanks.
> HBase snapshots support in VerifyReplication tool to reduce load on live
> HBase cluster with large tables
> --------------------------------------------------------------------------------------------------------
>
> Key: HBASE-16466
> URL: https://issues.apache.org/jira/browse/HBASE-16466
> Project: HBase
> Issue Type: Improvement
> Components: hbase
> Affects Versions: 0.98.21
> Reporter: Sukumar Maddineni
> Assignee: Maddineni Sukumar
> Fix For: 2.0.0
>
> Attachments: HBASE-16466.branch-1.3.001.patch, HBASE-16466.v1.patch,
> HBASE-16466.v2.patch, HBASE-16466.v3.patch, HBASE-16466.v4.patch,
> HBASE-16466.v5.patch
>
>
> As of now VerifyReplicatin tool is running using normal HBase scanners. If
> you want to run VerifyReplication multiple times on a production live
> cluster with large tables then it creates extra load on HBase layer. So if we
> implement snapshot based support then both in source and target we can read
> data from snapshots which reduces load on HBase
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
