joshelser commented on a change in pull request #740: HBASE-23197
'IllegalArgumentException: Wrong FS' on edits replay when…
URL: https://github.com/apache/hbase/pull/740#discussion_r346413129
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/backup/HFileArchiver.java
##########
@@ -314,9 +316,21 @@ public static void archiveStoreFiles(Configuration conf,
FileSystem fs, RegionIn
// build the archive path
if (regionInfo == null || family == null) throw new IOException(
"Need to have a region and a family to archive from.");
-
- Path storeArchiveDir = HFileArchiveUtil.getStoreArchivePath(conf,
regionInfo, tableDir, family);
-
+ //NOTE: This extra check is needed for the exceptional scenario where we
archive wal edits
+ // replayed, when hbase.region.archive.recovered.edits is on. Since WALs
may be configured to
+ // use different FS than root dir, we need to make sure to pick the proper
FS when deciding
+ // on the archiving path.
+ //1) If no wal dir setting, wals and root dir are on same FS, so good to
go with the root dir;
+ //2) When we have wal dir set it will only be on different FS if
"scheme://authority" is
+ // defined on wal path. In this case, if this is a proper store file
archiving call, the passed
+ // FS scheme will be different from the wal dir one, and you should pick
root dir as base.
+ String workingDir = conf.get(CommonFSUtils.HBASE_WAL_DIR);
+ if(workingDir == null || !workingDir.startsWith(fs.getScheme())){
+ workingDir = conf.get(HConstants.HBASE_DIR);
+ }
+ Path rootDir = new Path(workingDir);
Review comment:
What about if `hbase.wal.dir` is null, `hbase.rootdir` is 's3a://...', but
we have a FileSystem with `hdfs://` scheme? Should we fail-fast?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
