ramkrish86 commented on a change in pull request #2419:
URL: https://github.com/apache/hbase/pull/2419#discussion_r507471545
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/fs/HFileSystem.java
##########
@@ -82,11 +82,9 @@ public HFileSystem(Configuration conf, boolean
useHBaseChecksum)
// Create the default filesystem with checksum verification switched on.
// By default, any operation to this FilterFileSystem occurs on
// the underlying filesystem that has checksums switched on.
- this.fs = FileSystem.get(conf);
+ this.fs = FileSystem.get(getDefaultUri(conf), conf);
Review comment:
The background of this PR is that - in a normal case where we have
root.dir and wal.dir set on different FS, the WAL FS is getting initialized 2
times. Those places are
`public static Path getWALRootDir(final Configuration c) throws IOException {
Path p = new Path(c.get(HBASE_WAL_DIR, c.get(HConstants.HBASE_DIR)));
if (!isValidWALRootDir(p, c)) {
return getRootDir(c);
}
FileSystem fs = p.getFileSystem(c);
return p.makeQualified(fs.getUri(), fs.getWorkingDirectory());
}`
So before even we create the HFileSystem from HRS#initializeFileSystem we
tend to get the WAL's path but there we do p.getFileSystem() to get the URI and
the path. That is where the filesytem gets inited without even we intentionally
doing it.
Now before this patch - We had
`FileSystem.get(conf);` &
`fs.initialize()`
So the file system that was already created and inited was once again
getting initialized(). So there were two instances of the FS.
So the first thing to be done is remove the `fs.initialize()` and only do
FileSystem.get() but the other flavour of
FileSystem.get(URI, conf) has a clear javadoc saying that if the FS is
already created we will just return the Fs or we will initialize and then
return the FS. Hence the API was changed.
But the other change to pass URI is to avoid the unintentional init of the
FS from a Util method. What we need is a URI which we set it with fs.defaultFS.
So for doing that we just extract the URI from the path and set it to the
fs.defaultFS. So the subsequent code in FS.get(URI, conf) can really do the
actual init that we are interested in.
@Apache9 - Hope the above clarifies. Let me know if you need further
details.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
