[
https://issues.apache.org/jira/browse/DRILL-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467811#comment-16467811
]
ASF GitHub Bot commented on DRILL-5270:
---------------------------------------
ilooner commented on issue #1250: DRILL-5270: Improve loading of profiles
listing in the WebUI
URL: https://github.com/apache/drill/pull/1250#issuecomment-387502051
@kkhatua I'm still not sure why you want to override the definition of
oldest? Why is the default LRU eviction policy not sufficient?
If you need an ordered list of keys for the cache you can accomplish this
with the Guava cache by adding a key to a TreeSet when the Loader is called,
and removing a key from a TreeSet when the Removal Listener is called.
My main concern is that implementing our own cache creates complexity and
opens up the possibility for bugs. Whereas a pre-existing cache is already
debugged and tested for us.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Improve loading of profiles listing in the WebUI
> ------------------------------------------------
>
> Key: DRILL-5270
> URL: https://issues.apache.org/jira/browse/DRILL-5270
> Project: Apache Drill
> Issue Type: Improvement
> Components: Web Server
> Affects Versions: 1.9.0
> Reporter: Kunal Khatua
> Assignee: Kunal Khatua
> Priority: Major
> Fix For: 1.14.0
>
>
> Currently, as the number of profiles increase, we reload the same list of
> profiles from the FS.
> An ideal improvement would be to detect if there are any new profiles and
> only reload from the disk then. Otherwise, a cached list is sufficient.
> For a directory of 280K profiles, the load time is close to 6 seconds on a 32
> core server. With the caching, we can get it down to as much as a few
> milliseconds.
> To render the cache as invalid, we inspect the last modified time of the
> directory to confirm whether a reload is needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
