Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/9370 )
Change subject: IMPALA-6530: Track time spent opening HDFS file handles ...................................................................... Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/9370/3/be/src/runtime/io/disk-io-mgr.cc File be/src/runtime/io/disk-io-mgr.cc: http://gerrit.cloudera.org:8080/#/c/9370/3/be/src/runtime/io/disk-io-mgr.cc@1237 PS3, Line 1237: ImpaladMetrics::IO_MGR_CACHED_FILE_HANDLES_REOPENED->Increment(1L); > is this the accumulated count of reopens? Yes, this tracks reopens due to an error case. The number of cache misses (IO_MGR_CACHED_FILE_HANDLES_MISS_COUNT) would be equivalent to the number of opens. Open time is tracked at the HDFS scan node level (similar to read time). We also have cache hits and misses at the scan node level, so for most cases, you can figure out the average time to open a file handle by dividing by the misses. These don't incorporate reopens (which should be rare), so this is a bit off if there are reopens. We could track reopens at the scan node level to make this exact. -- To view, visit http://gerrit.cloudera.org:8080/9370 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia560af2d9b12f158e8811900a7b9d98f8e760858 Gerrit-Change-Number: 9370 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Thu, 22 Feb 2018 00:22:27 +0000 Gerrit-HasComments: Yes
