Marcel Kornacker has posted comments on this change. Change subject: IMPALA-4623: Enable file handle cache ......................................................................
Patch Set 10: (4 comments) http://gerrit.cloudera.org:8080/#/c/6478/8/be/src/runtime/disk-io-mgr-handle-cache.h File be/src/runtime/disk-io-mgr-handle-cache.h: Line 88: /// is set to true. Otherwise, the partition will try to construct a file handle and > Added a TODO for this. I plan on dealing with this when doing eviction by t true, but you're describing a scenario with concurrent queries and updates. we don't make any guarantees in that case. it would be fine for step 6 to notice that the mtime changed when it tries to reopen that file, and return an error at that point. http://gerrit.cloudera.org:8080/#/c/6478/10/be/src/runtime/disk-io-mgr-handle-cache.inline.h File be/src/runtime/disk-io-mgr-handle-cache.inline.h: Line 130: // return code, and we close the file handle and remove it from the cache. thanks for the explanation http://gerrit.cloudera.org:8080/#/c/6478/8/be/src/runtime/disk-io-mgr-scan-range.cc File be/src/runtime/disk-io-mgr-scan-range.cc: Line 417: } else { > This is the number of bytes read by either hdfsPread or hdfsRead. In case o so it's just num_bytes_read then. when i see 'last' or 'prev' or something like that in a loop i usually expect this value gets set in iteration n and used in n+1. http://gerrit.cloudera.org:8080/#/c/6478/10/be/src/runtime/disk-io-mgr-scan-range.cc File be/src/runtime/disk-io-mgr-scan-range.cc: Line 352: int success = hdfsGetHedgedReadMetrics(fs_, &hedged_metrics); you're getting metrics for the whole fs_, but every scan range does that on its own, and then updates a global counter? how is that not double-counting? -- To view, visit http://gerrit.cloudera.org:8080/6478 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ibe5ff60971dd653c3b6a0e13928cfa9fc59d078d Gerrit-PatchSet: 10 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
