[
https://issues.apache.org/jira/browse/DRILL-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16474314#comment-16474314
]
ASF GitHub Bot commented on DRILL-5270:
---------------------------------------
arina-ielchiieva commented on a change in pull request #1250: DRILL-5270:
Improve loading of profiles listing in the WebUI
URL: https://github.com/apache/drill/pull/1250#discussion_r187981529
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/store/ProfileSet.java
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.drill.exec.store.sys.store;
+
+import java.util.Iterator;
+import java.util.TreeSet;
+import java.util.concurrent.atomic.AtomicInteger;
+
+/**
+ * Wrapper around TreeSet to mimic a size-bound set ordered by name
(implicitly the profiles' age)
+ */
+public class ProfileSet implements Iterable<String> {
+ private TreeSet<String> store;
+ private int maxCapacity;
+ //Using a dedicated counter to avoid
+ private AtomicInteger size;
+
+ @SuppressWarnings("unused")
+ @Deprecated
+ private ProfileSet() {}
+
+ public ProfileSet(int capacity) {
+ this.store = new TreeSet<String>();
+ this.maxCapacity = capacity;
+ this.size = new AtomicInteger();
+ }
+
+ public int size() {
+ return size.get();
+ }
+
+ /**
+ * Get max capacity of the profile set
+ * @return max capacity
+ */
+ public int capacity() {
+ return maxCapacity;
+ }
+
+ /**
+ * Add a profile name to the set, while removing the oldest, if exceeding
capacity
+ * @param profile
+ * @return oldest profile
+ */
+ public String add(String profile) {
+ return add(profile, false);
+ }
+
+ /**
+ * Add a profile name to the set, while removing the oldest or youngest,
based on flag
+ * @param profile
+ * @param retainOldest indicate retaining policy as oldest
+ * @return youngest/oldest profile
+ */
+ public String add(String profile, boolean retainOldest) {
+ store.add(profile);
+ if ( size.incrementAndGet() > maxCapacity ) {
Review comment:
Please remove spaces.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Improve loading of profiles listing in the WebUI
> ------------------------------------------------
>
> Key: DRILL-5270
> URL: https://issues.apache.org/jira/browse/DRILL-5270
> Project: Apache Drill
> Issue Type: Improvement
> Components: Web Server
> Affects Versions: 1.9.0
> Reporter: Kunal Khatua
> Assignee: Kunal Khatua
> Priority: Major
> Fix For: 1.14.0
>
>
> Currently, as the number of profiles increase, we reload the same list of
> profiles from the FS.
> An ideal improvement would be to detect if there are any new profiles and
> only reload from the disk then. Otherwise, a cached list is sufficient.
> For a directory of 280K profiles, the load time is close to 6 seconds on a 32
> core server. With the caching, we can get it down to as much as a few
> milliseconds.
> To render the cache as invalid, we inspect the last modified time of the
> directory to confirm whether a reload is needed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
