anoopsjohn commented on a change in pull request #3596:
URL: https://github.com/apache/hbase/pull/3596#discussion_r696273459
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java
##########
@@ -163,22 +168,57 @@ private StoreFileInfo(final Configuration conf, final
FileSystem fs, final FileS
}
LOG.trace("{} is a {} reference to {}", p, reference.getFileRegion(),
referencePath);
} else if (isHFile(p)) {
- // HFile
- if (fileStatus != null) {
- this.createdTimestamp = fileStatus.getModificationTime();
- this.size = fileStatus.getLen();
- } else {
- FileStatus fStatus = fs.getFileStatus(initialPath);
- this.createdTimestamp = fStatus.getModificationTime();
- this.size = fStatus.getLen();
- }
+ // If a fileStatus, use it. It is pointed at the hfile, not at a link or
reference.
+ this.cachedFileStatus = fileStatus != null? fileStatus:
fs.getFileStatus(p);
Review comment:
Yep.. This is really useful stuff specially when storage is remote cloud
storage. Getting the FileStatus is not that cheap. And that too 2 or 3 times
in the path of open an input stream over an HFile. This is more clear when we
have many scans where switch from pread to stream read happens. There we will
have to open the IS over HFile again. Some times the data that we are looking
for, might be already there in Cache. The switch will happen and so open new
FIS.. There these calls becoming the overhead. I did some profiling last weeks
and it was evident .. BTW my version is bit old 2.1.x where even some recent
optimization in the path of HFile IS open was missing. This is really useful
stuff even though a small patch... :-) Good on you Stack.. I will also
closely see and how max we can avoid the calls for getFileStatus. BTW on the
Jira I put another idea also. That involves more work and useful.. The
required API from hadoop common is available in some later vers
ions only. So we might have to play with it a bit.. Once this patch is in,
lets discuss that also.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]