Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16532 )
Change subject: IMPALA-10202: Enable file handle cache for ABFS files ...................................................................... Patch Set 1: > This looks good to me. > > If you haven't already, there are a couple sanity checks I would > do: > 1. Cache an ABFS file handle and then remove the file and see how > it behaves. For an object store, I don't expect any weirdness. If > the file is gone, I expect the read calls to eventually fail, then > subsequent open calls for that file would fail. > 2. Take the cache up to its default max_cached_file_handles limit > (20,000) and make sure things are ok. It sounds like you did > something like this already. 1: I did that, seems to work as expected. The query eventually fails with a FileNotFoundException. After I run invalidate metadata, the query works again. 2: I created a table with 500 files and ran a full table scan. Seems to work as expected. I checked the amount of untracked memory before + after I ran the query and it looked okay. -- To view, visit http://gerrit.cloudera.org:8080/16532 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I64f12f832980f4e0207af78368402dd09e370fc3 Gerrit-Change-Number: 16532 Gerrit-PatchSet: 1 Gerrit-Owner: Sahil Takiar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Sahil Takiar <[email protected]> Gerrit-Comment-Date: Fri, 02 Oct 2020 00:12:17 +0000 Gerrit-HasComments: No
