Kunal Khatua created DRILL-5270:
-----------------------------------
Summary: Improve loading of profiles listing in the WebUI
Key: DRILL-5270
URL: https://issues.apache.org/jira/browse/DRILL-5270
Project: Apache Drill
Issue Type: Improvement
Components: Web Server
Affects Versions: 1.9.0
Reporter: Kunal Khatua
Assignee: Kunal Khatua
Fix For: 1.10.0
Currently, as the number of profiles increase, we reload the same list of
profiles from the FS.
An ideal improvement would be to detect if there are any new profiles and only
reload from the disk then. Otherwise, a cached list is sufficient.
For a directory of 280K profiles, the load time is close to 6 seconds on a 32
core server. With the caching, we can get it down to as much as a few
milliseconds.
To render the cache as invalid, we inspect the last modified time of the
directory to confirm whether a reload is needed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
