Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/9371 )
Change subject: IMPALA-6549: Enable file handle cache by default ...................................................................... Patch Set 1: (1 comment) > (1 comment) > > Do you have a list of tests you plan to run? I ran core tests with no issues. This will get incorporated into the next run of performance tests. Separately, I plan to do a warmup test where I run with a table with a large number of files with a cold cache. Since remote files don't participate in the cache, I don't forsee any issues with S3 or other platforms. http://gerrit.cloudera.org:8080/#/c/9371/1/be/src/runtime/io/disk-io-mgr.cc File be/src/runtime/io/disk-io-mgr.cc: http://gerrit.cloudera.org:8080/#/c/9371/1/be/src/runtime/io/disk-io-mgr.cc@109 PS1, Line 109: 21600 > how was that chosen? This is a good point. The concern for aging out file handles was about disk space getting freed up. This is controlled by the fs.trash.interval. We wanted the eviction time to be less than the typical fs.trash.interval so that the disk space usage would not be impacted by the cache. This is reverting back to what we originally merged in IMPALA-5352. I checked some Cloudera deployments, and they default to 1 day for fs.trash.interval. HDP has a default of 6 hours. I think there is a range of appropriate defaults. -- To view, visit http://gerrit.cloudera.org:8080/9371 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6935825a1c4c7b2da0bb877f732027be1a57a8b7 Gerrit-Change-Number: 9371 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Comment-Date: Thu, 22 Feb 2018 00:05:09 +0000 Gerrit-HasComments: Yes
